Are you sure you are going on the right Journey? There are several options, and a small chibi smiley on their cards shows which boss can be encountered there and whether it's possible at all.
It’s the forest thicket one that is labeled as being dangerous. I mean there are only 3 adventure options available to me so far. I’ve done the easy one with no bosses and then the moderate difficulty one with the ice elf boss and all the snowmen, and now there is this forest thicket one. But no boss to be found.