If you go to Fred's Domitory and do not find him there. You get a button in Aliron to immediately go look for him via the "search for Fred" button that appears in the lower left corner of the screen.

However, doing so and going into a fight with the bandits leads to soft block in the game, as it asks you to choose who to fight the bandits but as you have no party (as you immediately went there from Aliron) you cannot select anyone. It does work if you go the domitory and then go the mansion first.