It'd be more challenging to build properly, and it'd just be a lot more fun, if your raft could tip over from being imbalanced, or sinking from being heavy.
Without stuff like that, the game isn't "raft" it's just "a building that the world moves around". It rather ruins the game and concept for me.