Great assessment. I agree with the criticisms; still, I think the solution is not obvious.
Games that have already received 20 ratings shall not be able to be rated anymore, or at least before most the games has received 20 ratings.
This reminds me of the rating queue concept that can be applied by the game jam owners.
A side effect I can think of is that this could lead to faster, less detailed, lower-quality reviews, because the opinions of people that hurry / rush it will be more taken into account. For example, I tend to be super slow for reviewing, but go very in depth and take the time to write an account. Your idea of limiting the lock to waiting for ‘most’ games to have 20 ratings could work, but then, I do not know how to select the ratio, since there is a share of games that have few votes because they are either hard to download / make work or have gameplay problems (hard to understand, long…) that make people pass on rating them. Which is why there is bound to be some games with few votes; so, in the proposed system, they will either stay that way and we have to figure out how to take this into account, or they will get meaningless (harsh?) ratings.
You can't get a final placement or perhaps even see your ratings, if you haven't played and rated at least 20 games yourself. (And if someone should just begin to give all the games a 1 star ratings, they should be disqualified from the jam.)
My fear is, as you correctly anticipated, that this will cause some people to give many rushed ratings; and I do not see how you could control for genuine votes: if I were to apply a dishonest rushing rating strategy, I would throw in some 2s or even 3s into the mix. You can always be unfair while voting, there is no simple way to really detect such a behaviour.
There should in the jam introduction be more fucus on the playing, given feedback and rating part of the jam.
Strongly agree on this one! :) I had this idea too and have written it elsewhere. I also think itch.io should make the Karma system (and least-rated list) more obvious, for example, make it the default ordering (or even only ordering??).
People should be told that self promoting of your own games in other game jammers game pages is a no go. I would even like to see those game jammers get warnings not to do so, and in some rare cases I even want to see them get disqualified from the jam.
I think the itch.io interface that already shows commenters’ games below their comments is enough indeed. Given this, I see side sentences asking for a review as having minor influence (I may be wrong), but what really is a problem (and you may be referring to this, actually) is meaningless spammy comments only made to get visibility (and possibly Karma). Like ‘cool game’ when the person may have not even played nor rated! (I have seen some people do this in this event, copy-pasting the very same short comment everywhere… ._.)
Incidently, I recently stumbled upon a Ludum Dare post discussing the karma system and what would be best; notice the whole point (both in the post and the comments) about unintended side effects, this is really a tricky subject! As said there, no system is perfect, and there is always some way to game the system to some extent if you are being dishonest. The ‘spammy comments strategy’ for getting karma/‘coolness’ is also tackled in the comments.
I have only seen a little portion of the games, but I agree there was creativity. :) Maybe some will want to try the legendary Ludum Dare (starting 30th September) and possibly the hard ‘Compo’ category; I think I will! (Oh, by the way, I discover theme suggestions have just opened!)