You shouldn't be able to get randomly hit as they spawn at least 64 pixels away from you (about 4 times the player)
Yeah, I noticed that. It's probably just me being totally bad at your game, but those 64 pixels are like 0.5 seconds of player movement, which means that you can easily move into them.