itch.io

This is an interesting discussion. :)

Yes, haha, I was aware that the motivation for my suggestion may have been flawed, this is the kind of thing my brain doesn't cope well with.. You say it's possible that half the games would be downgraded but I was wondering whether this would be  a certainty just from the nature of a median value.

No worries. ;) The median is a value that separates a statistical sample into two same-sized groups. Notice that this means there can be several values equal to the median; for example, in the data set 1—5—5—5—7—8, the median is 5 (with subgroups 1—5—5 and 5—7—8), but only 1 is strictly below 5. I expect that, most of the time, in a game jam, there will be just a bit below 50% of games that get strictly below the median, and thus get a penalty. In some fictional example (which is actually close to a real one I know, where a game could have ranked above mine by one place!), I think it makes about zero sense to downgrade a 15-vote entry but not a 16-vote entry, changing their ranks in the process, just because the median is 16.

But it does make sense to have some kind of downgrade system so that a game with one five star vote doesn't win.

Well, I was thinking you can either have a downgrade system, or simply some kind of indication or ranking without entries with too few votes (and one with all), but as I said, I think this should rely on a fixed amount of votes to get. This may not even need to be proportional when the global number of votes is big enough, because I think I remember seeing that a representative sample does not need to be proportional, merely big enough in absolute terms — still, double-check this. ;)

For me, the solution lies not in the results system but in the voting system.

I agree that the voting process has problems too, and could be taken care of. Actually, I would like both the results system and the voting system to be reformed.

After playing one game that must have been entered into another jam as well, I must have clicked on the wrong entry page link as it was telling me I couldn't vote for the game until I'd voted for some others with few votes. I feel like forced voting like that could lead to just voting a low score without even playing just to get back to rating other games faster.

I think what you are referring to is the rating queue. You may be right that this leads to such a bias, it occurred to me too. I think this is even more serious in game jams where public voting is enabled (and public voters are often forced to go through the rating queue, while not necessarily the submitters): I guess this tends to bring in fickle voters that do not hesitate on being overly harsh. The rating queue is an alleged solution, but I am not convinced (for example, I have seen several worthy games from GMTK Game Jam 2022 with both few votes and weirdly low raw score).

I can't really think of a good solution that doesn't rely on people doing the right thing. I think the biggest contributor in this jam to the ratings gap was the rate for rate posts.

I am thinking of at least two things:

1. Game jam hosts should make it more obvious in the main game jam page and forum to use special orderings (especially the link to the special list of low-vote entries that pops up at the top of the submissions page), explaining how this is important and removes the need for rate-for-rate threads (and even forbid such threads when the reform is in effect);
2. The Karma ordering could be more hightlighted. In case you or anyone else wonders what this is: this ordering originates from Ludum Dare and considers (using some formula) the ratio of the votes you cast (and number of others’ pages where you comment, I think) over the votes you receive; this way, someone who volunteers to review other people’s contributions gets more spotlight in this ordering, until he gets enough votes to balance it. This is somewhat akin to a more systematic and moral version of the vote-for-vote system: the more you review, the more your game is reviewed. (Nevertheless, I read in some comment that the system (either here or in Ludum Dare or even both) had experienced changes and did not work as it used to, but I am not aware of the specifics.)

I would even consider removing the popularity ordering during the voting period, I feel it tends to create a vicious circle, seeing the discrepancy in how many votes each game can get. It is just my intuition, though.

A more complex suggestion would be to enhance the rating queue system by asking the voters to write a reasonably detailed and fair review, in order to avoid careless marks. This system may be hard to check…

I don't know if all jams are 'plagued' by such posts

It is apparently quite common (at least in large enough game jams, I think); I have seen this ostensibly in Mini Jam, it happened here for GDevelop Game Jam #2 as you said, I have seen it for GMTK Game Jam 2022 (including on the Discord server), and a message from this thread on the Karma system seems to confirm (‘The current reliance on spamming the community with "Play my game" and "I'll rate you if you rate me" in order to achieve an above mean average rating count is frustrating and discouraging.’). On the other hand, I did not see it in a smaller game jam (Gamecodeur Game Jam #36), although I did not check the Discord server.

TL;DR:

• Ask for a fixed minimal number of votes;
• Make the least-voted and Karma orderings very obvious both in game jam main pages and on the site.