You cannot.
I am planning on letting you be able to go work at the Cafe should you back out of the Bakery route, but things will be pretty different working at the Cafe after that and will largely just be a source of income.
Better to work at the Cafe from a second playthrough.