Am I missing something, I can not get off the 1st floor? The gate does not open. 

Outside of the tower, you can follow a path up north on the left side, where you can use a sparkling location to get catapulted to the higher floors of the tower.