I was just thinking about this, but you might be able to go into College on a weekend day so long as you have progressed any of the fey girls enough to where they will go to the college so you can teach a class and be able to go into the school on the weekends when it's normally closed. You also have to progress the College questline enough to become a teacher but that shouldn't be too much of an issue.
The girls who will go to the college on weekends if you're a teacher and decide to teach are Emma, Pixie, Kana, and Lyx as far as I'm aware.