itch.io

Yeah I definitely will work on making it browser playable, right now I am reworking a lot of gameplay stuff and the server is down atm. Will fix everything very soon and the build will be way more fun :)

Thank you for trying out and stay tuned for update! You can also join my discord to get notified when that happens. I will post a youtube video as well

Totally agree with you, nothing more to add. It's really dumb that after learning about how this system works, I was kind of forced to consider just putting random ratings on games since my ratings actually push the median further up and kind of decrease my chances to get past the median if some of the games I rated don't return the favor. And some people will actually do it for sure, especially since you can't even be sure you got past the median!

And leafo, I get what the current solution was here to solve, I really do. What we're trying to tell you is that by some minor changes we can solve the initial issue as well as other problems that we've described.

Really like your idea! My main concern was that there's no way to get all 100% of people to not be penalized, which is just absurd. And your solution only requires changing the formula, which is very simple to implement too.

@leafo what do you think?

I still don't see any arguments against the solution where organizers could choose the limit of votes themself. I would be happy with this solution, organizers would get an option to go from this median system, you could implement a warning for participants. As you said this problem is not easy to solve but seems to me that you've chosen an easy to implement, scalable solution, which is fine, just give people the option to not use it and avoid some problems that it creates. Put the responsibility on organizers, because a generalized solution will never fit everyone perfectly. We can talk all day about all the edge cases of each approach, but I think I explained all my arguments, as well as you. So, I hope you will decide what's best, I just wanted to explain my concerns, because this thing sort of ruined the jam for me in a way. Of course, I will try to have as many ratings as possible in the future. I just really think there's room for some easy improvement for everyone.

Thank you!

The average rating for projects that receive a low number of ratings will have very high variance and poorly represent how participants felt about your game, so we do not show a ranking for that score. It would be unfair.

How is that unfair to just show the information? I don't think it's fairer to negatively compensate dispersion, dispersion works both ways, and in many cases, you would lower a score that was already too low from not enough ratings.

That might be worth adding, but in this case you would still have too low number of ratings to make a difference based on how we would recommend hosts to configure the jam.

I never said that my 8 ratings should be enough, but I should at least get some warning that I should try to get more ratings and how many more. Will you add this feature?

10th percentile would be way too lenient, and allow for games with high variance in their rating overtaking projects that have a more accurate rating due to the larger number of ratings.

The system is built this way to encourage you to participate in the rating process. If you want to receive ratings to be eligible for a more accurate and non-penalized final score then you will need to rate other people’s work and leave constructive comments on their submission pages. That will allow your project to show up on the “most karma” sort and also people who see your comment will be linked submission to get a chance to play and rate your project back.

I see the intention, but in this particular solution, wouldn't you punish half the people even if everyone got 100 or more ratings? Median will always shift if more people get more ratings and there seem to be no way to get zero people penalized. Also, even if I do get enough ratings to pass the median, I still know that half the people got their scores lowered and my game is compared to them not objectively.

I still think that Ludum Dare has the best solution for this problem. Everyone can get 20 ratings and they are forced to play and rate games of other people. Also, everyone gets a very clear warning that if they have lower than 20 ratings, they will not be ranked. In their case, every participant can get a fair score and be ranked among others, not just half the participants. I see no reason to leave this median solution, it creates more problems than it solves. You literally make really interesting rating data lose its objectivity :(

Also, I really don't get why you would choose to take median and not 10th percentile for example. It literally makes the lower half of participants by popularity lose their scores.

I recently participated in GMTK 2021. I just saw my ratings, and the actual score of each criterion was lowered by 1-1.5 points.

https://itch.io/jam/gmtk-2021/rate/1084211

Apparently, your score is lowered if the number of ratings is lower than the median amount. I get why that's a rule, you don't want a game with two 5-star ratings to just win. But it creates another issue. Now, after that correction, the rating of my game might be lowered for reasons which have nothing to do with how good the game is! It makes the rating lose objectivity. All I really want from a jam is to get feedback on how good my game was compared to other participants, and I can't do that if I didn't get enough ratings. Also, I can't even really know how many ratings I need, there's no warning that my amount of ratings is lower than the median.

Possible solutions that I see:

• Show rankings of the raw score. It would almost erase the issue for me personally, as all I want is the objective information on my rankings.
• Put a hard limit on the needed amount of ratings. This can be calculated from the number of submissions for example or can be set as an option by jam organizer.
• Show a warning that currently your amount of ratings is lower than the median.

у меня идея была скорее в расположении, может быть что у красных прямо рядом желтый цветок и им его собирать намного проще чем желтым полчаса ходить туда-обратно, то есть как в жизни с торговлей