Since we can use an engine like Unity our Godot, are we also allowed to write our own 3d engine ahead of time? Or should we do that during the game jam? I think I can make a very basic raycaster in a weekend, but there won't be much time left for actual content.

you can use whatever you want as long as it's not illegal or NSFW. Just have fun and make a game.

Yes this is allowed, Other game jams usually allow for it and at the moment Mini Jam doesn’t have a rule against it. As long as its just work on the engine before the jam, and no actual work on the game. Definitely go for it, it would be cool to see what you make with your own engine.