That's a good start. I generally start with the map and no barriers and any known objects. I can then wander around to test all the exits and make sure the objects are in the correct locations. You can then start adding the barriers (which are things that prevent you moving from one room to the next), puzzles, more descriptive room descriptions, scenery and so on. Do it in a methodical manner, testing regularly and you can't go wrong. Well, you can go wrong, but if you only add and test a little bit at a time, it's easier to pick up the problems as you go.