My general rule of thumb is that concept art/exploration is fine so long as you're not making stuff that's going to end up in the game.
It's a pretty fuzzy thing that's hard to give meaningful specifics on. Everybody's got different ways of working and there's not really any kind of objective boundary between production and pre-production that perfectly fits every project/every work environment.
I like to use my concept art as placeholder art, which doesn't quite fit that rule of thumb, so I'll generally avoid making that kind of art until after the jam starts.
Ultimately the spirit of the jam is to make things and rise to the challenge of successfully managing a project within a short timeframe. If you feel like you're adhering to that and getting something out of the process, then I'd say you're doing it right.
Hope that's helpful!