Thanks for the comments. I think the slowdown is due to my pathfinding later in the game when there are a lot of rooms. I had wanted to use A* but due to time (having to learn A*) and doing this solo I went for my own version which isn't very optimal!
