Yes, you do need to! Some random islands have farmhouses, while others don't. If there are animals on an island without a house, you'll need to pick them up one by one and drop them on an island that does have a house. From there, you can either move them to the house or drop them directly at the door.