In the Bandit Amory, you can walk through the middle and rightmost segments of the table. The key from the chest in the same room is called "Bandit Armorer Key" when you pick it up but "Bandit Leader Key" in the inventory.
I should have sent you something like, a minute ago.