From jam page:

• Top voted games on the Itch page and the discord will be submitted as finalists to the judges. Each judge will give each game a score out of 10. The top rated game wins! Results will be in 2 weeks after the final submission day.

Okay here is problem:

i often participating in game jams and i noticed one thing: quality of games on jams are not determined by their ratings meaning top voted games are often not best ones.

Just check storefronts and you can clearly see that amount of clickbaitery and meme games is super high, effectively creating wall in front of really good and clever hidden gems.

I dont remember which one but one of edition of LDs winner was super shitty low quality meme game. It was caused by bad voting system and if i remember correctly, it was stormed by faked accounts and manipulated to high votes.

I also noticed that with rating jams submission time matters. Like when you submit it in wrong time frame, you got zero to few ratings independent on game quality.

Simply put: if only top voted games will be submited for judges, there is high chance of risk that highly voted games are NOT the best games which is super sad for rated jams. Especially bad for prized jams.

Game jams once were opportunity to unchain your imagination and unleash inner creative craziness but slowly starts to require storefronts like marketing techniques just to gain attention

Just some stuff to think about

Agree

I definitely hear you on that one. We're doing voting to get the stop games which are then submitted for the judges to look at. We're experimenting this time and hoping that the voting works out honestly. If not, we may select a few additional games to be submitted to the judges.

I agree. That is why I like participating in game jams where the host or hosts play most if not all of the games. A good example is the Brackeys game jam or GMTK game jam.

I will be playing all the games (as the host, not a judge)

It was recommended that I post here with concerns that have been discussed in the discord server. First things first, I want to say that I hate complaining, and wish that there was a better alternative. I would also like to make sure that what I am saying is not misinterpreted- I think the vast majority of the games that are ranked highly deserve to be there and to be judged. Finally, no matter what is decided, it will leave someone unhappy, barring the judges ranking every single entry, which of course is infeasible. I am not prescribing what to do- I honestly don't know what the best option to do at this point is, but I do feel the need to bring the issues discussed here. I will offer my thoughts as to how to potentially avoid some of these issues in future jams, but obviously you can take or leave those.

I have talked with other members in the discord channel and it's unfortunate, but it seems like there are some pretty severe issues with the ranking system. One of the worst problems that seems to be the case is that many of the games that were near the top of popularity (which I assume is based on how many people played it and/or how much time was spent playing it) received strategic 1 star reviews. Obviously it's impossible to prove that these are malicious, but it seems incredibly unlikely. To be clear, this is a problem that our game suffered from, but it was not only ours, and seemed to be worse for games that appeared to be doing well. I think it's saying something that our game was consistently near the top of popularity (most often within the top 3), yet fell into about the midpoint of the ranking system.

At the end of the day, I really don't know what the best option to do is. If you choose to have judges rate 20 games total, and choose the top 15 ranked games and 5 other games, the people ranked 16-20 will feel crappy. Although I suppose that it would suck for the people in 21st place as well, regardless. Hopefully this can at the very least start a discussion to find a good solution.

I posted this in jest in the discord server amidst the discussion, but it highlights many of the problems with an open vote system, and is the reason most jams used closed voting. Here is the abuser's guide to winning open voting jams:
- Do the absolute minimum to promote your game. Make the title and cover art as generic and/or poor as possible as to lower the likelihood of people playing the game
- Near the end of the voting, have friends and/or bots review your game
- Strategically rate competitors 1 star, and games you deem as manageable competition higher. If you're using bots or friends to do this, it's untraceable anyways

It's impossible to say if anyone cheesed the system, and I don't want people to think that I'm making that claim. I do definitely believe that there were people that actively did the last part based on people I've talked with, however.

Finally, for future jams I'd recommend two main things for jams in the future:
1. Please, please, please use a closed voting system. There are obviously a few downsides to this (namely the number of people that can rate games), but it makes abusing the system much more difficult. It incentives participants to actually try to get other participants to play their games and vice versa. While you can still 1 star games, it is traceable so if there's obvious strategic voting going on at least people can be called out on it.
2. This is more minor, but having more than 1 voting category is often useful to get a more creative variety of games and give smaller/solo teams a better chance. Keep the overall rating as it's own category, and use that as the primary info for voting, but add categories for other aspects (e.g. "Art", "Audio", "Gameplay", "Theme interpretation"). This way you can get a larger breadth for finalists- games that might be really good in one or two regards but didn't have time for the rest. This second point comes more down to what you're looking for, but I'd argue that it would incentivize participants to be able to focus more on aspects that they might be able to compete on.

tldr: It seems apparent that there are some pretty major issues with the voting and that it should be discussed how to handle it

I agree with everything you said BricksParts.
in my opinion the best way to do it is ones that are deemed noteworthy for this that resion, such as ones getting recognition outside the jam as well as the top 15 games at least in my opinion seems to the best ones generally for this jam so without redoing all voting again i would recommend just going though those 15 games.

I agree with everything you said BricksParts.
in my opinion the best way to do it is ones that are deemed noteworthy for this that resion, such as ones getting recognition outside the jam as well as the top 15 games at least in my opinion seems to the best ones generally for this jam so without redoing all voting again i would recommend just going though those 15 games.

I think voting in general is broken. There is no way of stopping strategic up votes, or down votes. Obviously keeping it closed to jam members is a way of slowing it but it still wouldn't stop someone from having a few jam friends down vote a game they think is going to win or up vote each others games with 5 starts. Maybe if itch had something in place that if you voted 1-2 star your would have to leave a response but even this would have to be moderated by the jam admins, and they are not making anything from this. That is alot of work. 

You could also stop having any prizes which would rid of more schemes. That would really suck though. I was stoked when i seen i could win something if i made a dope game. 

Honestly you guys tried to do something that works and that is awesome. I like that not only is it community votes but also a judging system.