Considering the total ratings of a game compared to the median number of ratings is there to normalize the rank. Three of the four games ranked lower than Arkham had more total ratings than Arkham. In fact, one game had almost twice as many ratings. The system is in place to prevent a game with few ratings to skew the results. The ParserComp ratings were pretty evenly rated.
All of the games were excellent. Very few score points separated many of the games.
I understand that there has to be some way to avoid obvious problems, such as a game only getting two ratings and both ratings are five stars or similar. I like the IFComp approach where they stick to raw scores all the time, but if a game has too few ratings, the game is not ranked. I also understand, that this approach is easier in IFComp, where the organizers can adjust the threshold, as they did recently, where they accepted that a game only had 9 ratings. On itch.io, it has to be automatic and the approach should work for both large jams (more than a thousand entries) and small jams like ParserComp. Thus, I think it would be better if the jam organizer could decide which threshold was appropriate for their jam, as they know roughly how many will participate, etc. For instance, penalties could be applied when games have less than e.g. 7 ratings instead of penalizing half the games. We cannot know if a game has been overrated just because it has few ratings. It might as well be underrated.
So more options for the jam organizer would be good. But I guess this is wishful thinking. I don't think Itch.io is going to change this anytime soon.
