Ugly surroundings are caused by dirty dishes, broken objects and raw drywall.
You can remove walls with the remove tool. If you hold your mouse and drag, but only drag along one axis, a line will be drawn with the tool instead. If the line is over where a wall’s base is, the wall is removed.
There is currently in an issue where, if you don’t have enough money to buy a theoretical copy of an item you’re trying to move, the item cannot be placed back down. This issue is already fixed in development.
Thanks for reporting these issues, and let me know if you find out what the ugly surroundings are caused by! If you have none of the things I mentioned, it might be a bug after all.