One thing that isn't clear to me is the distinction between ship and bank. From the manual, it seems that money in the ship isn't lost when I Eject.

But isn't it easy to ALWAYS put all my money in the bank? Are there any drawbacks for this?

You can put your money always on the bank, but Pierce weapons will be harder to use (as they consume scrap of the ship) and there's some events and blackmarket shops that only accept scrap on the ship.
There's a chance that you wont find a Station to.

This mechanic was made more to leave the game's "death state" as a consequence that is in the player's hands. If you bank your scrap constantly, ejecting becomes a viable option.