Those are the stairs, yes! The problem I had was that I couldn't open the door due to the prompt to use the stairs overriding it. I held left and tried mashing E but it was always switching floors which took precedence over opening the door. Because of that I couldn't get to the TV and closet.
I appreciate the detailed response! :)