I understand that there has to be some way to avoid obvious problems, such as a game only getting two ratings and both ratings are five stars or similar. I like the IFComp approach where they stick to raw scores all the time, but if a game has too few ratings, the game is not ranked. I also understand, that this approach is easier in IFComp, where the organizers can adjust the threshold, as they did recently, where they accepted that a game only had 9 ratings. On itch.io, it has to be automatic and the approach should work for both large jams (more than a thousand entries) and small jams like ParserComp. Thus, I think it would be better if the jam organizer could decide which threshold was appropriate for their jam, as they know roughly how many will participate, etc. For instance, penalties could be applied when games have less than e.g. 7 ratings instead of penalizing half the games. We cannot know if a game has been overrated just because it has few ratings. It might as well be underrated.
So more options for the jam organizer would be good. But I guess this is wishful thinking. I don't think Itch.io is going to change this anytime soon.