itch.io

How is that unfair to just show the information?

Without any other changes to how we handle ratings, it’s unfair because the people who would get the top ranking would effectively be random (most likely an entry with 1 vote at 5 stars that was otherwise ignored by others). Additionally, a popular and highly rated game will never be able to hold that slot, because maintaining a perfect 5 star average as the number of ratings goes up is effectively impossible.

I see the intention, but in this particular solution, wouldn’t you punish half the people even if everyone got 100 or more ratings

You have to keep in mind we host all kinds of different jams, including jams that often do even have enough participants to hit 20 ratings on a single entry. The median system scales across jams of all sizes, small or large, automatically. It will work regardless of whether participants are motivated to rate projects or not. Ludum Dare has the advantage where they generally know how big the jam will be and how many people will be voting on entries, so they can choose a value up front that represents how people will be voting. (You also mentioned Ludum Dare’s score being “fair,” but even that is subjective)

I know at face value penalizing the score of half the entries seems strange, but think about what the jam system is supposed to do: rank every entry relative to every other entry using a limited and fragmented collection of votes. If an entry can obtain above the median votes while maintaining a high average, that it’s evidence that the participants of the jam do like it, so it should be promoted.

Consider this example: how would you order a game that got 20 votes at a 5 star average, and a game that got 200 votes at 4.99. (Now consider other combinations where the numbers are slightly adjusted, not exactly a simple problem after all). The number of ratings is a confidence value used against the average score. It might be helpful to think of your final score as a relative score to everyone else’s entry. We still provide you with the raw score for your own information.

Ranking games is always subjective and score are arbitrary, though, which is why we also let people host jams without any kind of rating enabled at all.

I never said that my 8 ratings should be enough, but I should at least get some warning that I should try to get more ratings and how many more. Will you add this feature?

Possibly, but as the median is a moving target it may not be a good approach. In this case, from what I can see, you rated 5 entries. My (rhetorical) question to you is why did you rate only a few number of entries despite being concerned about getting a full score? I’m guessing the issue here is that you were surprised about this system after the fact, and you weren’t aware going in. Perhaps if this was communicated better up front you would have changed how you approached the jam.