Skip to main content

On Sale: GamesAssetsToolsTabletopComics
Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
TagsGame Engines

Karma system for the game jams

A topic by Johnny Dalvi created Aug 07, 2019 Views: 11,154 Replies: 15
Viewing posts 1 to 12
(+35)

Hello everybody, 

As you know, itch,io is probably the main hub for game jams, and that's quite a feat since jams are a very important part in the indie game development environment. Currently the GMTK 2019 JAM hosted by the youtuber Mark Brown with almost 3k entries is under the voting phase, and as already stated by another person here in the itch.io community, the listing/filtering process isn't ideal for a jam this size because there isn't any kind of intelligent system ruling it, which makes some entries receive less ratings and overall feedback regardless of the developer contributing by playing other games and giving feedback to their devs.

This problem has already partially been solved by the Ludum Dare jam, they have a karma system that works very well (it does have some flaws, but is way better than a fully random system), what do you think about implementing a similar system over here? 

For whom isn't familiar with their karma system:
If you look at their rules (http://ludumdare.com/compo/rules/) and search for "Coolness", you will see their quick explanation about how this system works. But basically, every entry has a "coolness" value that increases as the player rates and comments on other's entries, then in the listing of the entries there will be a filter called "Smart Balance", which lists the games based on two things: Their coolness - amount of rates/feedback that they received. This system kinds of "Guarantee" you to receive more or less the same amount of feedback and rating that you've given. 

(1 edit) (+5)

Agreed, that system would be very useful here. I mean sure, better filters do help increase visibility, but they still rely on people actively searching for games with fewer ratings. Currently, the only way for people to get visible is to spam the jam's community with rate exchange topics. A karma system would solve this problem.

(+4)

Sounds like a good change or at least a good option for people making the jams. The more options the better in my opinion, I'm all for it.

(+5)

Yeah, this could be very helpful. I know jams that are run on Itch.io that struggle with getting a decent quantity of feedback to participants (or any feedback, really), while Ludum Dare just... Doesn't. In Ludum Dare, if you want feedback and are willing to leave feedback in return, you'll get it.

Admin

It depends on the jam. For GMTK the community has been really engaged about leaving comments and feedback. The average jam on itch.io is a lot more casual than Ludum Dare, so you're not going to see the same kind of feedback being left.

(+5)

It's understandable that itch.io have the filters they do given the core use of the site is not game jams, but rather serving as more of a library. 

And there is no silver bullet, but the current default random filter we have currently is uneven and unfair.

Visibility should be a function of participation, favouring those who are rating others and engaging in critical feedback and praise.

I agree that Ludum Dare's Karma system would be a good model to start from as it's been proven to work well for the last few years.

The current reliance on spamming the community with "Play my game" and "I'll rate you if you rate me" in order to achieve an above mean average rating count is frustrating and discouraging.

GMTK brings a level of popularity and attention, but with increasing numbers the amount of spam is growing and could turn the community toxic easily.

Whatever changes are made to address these issues however,  should be done in a transparent and iterative process with the community, with regards to the various game jam formats itch.io hosts.

I'm excited to see itch.io create an even more welcoming and supportive breeding ground for aspiring developers.

/endrant

(+2)

I would like to add  my voice to this proposal.

(+2)

Count me in. I have yet to even do a game jam but I'm getting there. :D

(+3)

That's generally my worry about joining any jam that isn't Ludum Dare. "But what about the sweet Ludum Dare karma system?!"

(+5)

As with a lot of these feature suggestion threads, I would like to give this a +1. We should really have some sort of system in place because some real sketchy stuff goes down in some jams.

(+1)

I agree - it's what I've been thinking every time I've joined a non-LD jam!

Admin(+5)

Hey all, sorry I didn't get back to this topic sooner. Coolest sort is something we used to have but it got lost in some code updates and I didn't have time to return it for this jam. It will definitely be something that will be available for next time though. I like the idea of providing as many sorting options as possible to encourage more people to get involved.

That being said, coolest can be abused pretty easily: bad actors can rate a bunch of other games negatively/with no effort just so their project sits on top. It can be difficult to automatically judge the quality of ratings to prevent this kind of abuse, and any automated system can also make mistakes. Random truly is the best way to have the most even distribution of votes across all projects.

I understand the appeal of coolness though, it rewards you for putting time into the jam. The more time you spend playing & rating projects, the more people will see your project.  As I've been observing GMTK this time around though, I actually have taken a liking to a different approach of accomplishing the same thing: comments.

If you haven't noticed, I pushed out an update that makes it really obvious what the submission is of someone who has commented on your project. It's a common pattern to rate the games of those who take the time to write a constructive comment on your page. I honestly think this is a better way of enabling people to invest time into getting more ratings. Go leave good comments, people are likely to go back to your project. If you leave spammy or bad comments, your posts may get deleted, and no one will follow the link back to your page. Comments are definitely more work than simply leaving a rating, but they solve the quantity/quality issue I described above with ratings.

I'd be happy to hear your thoughts about this approach. Thanks

(+1)

Hello there,
Yeah, the coolness system can be abused, but overall I don't think it does get abused a lot on the LD, I still get at least 70%~80% of the number of feedback I've given (from people that clearly seemed to have played my game), so the large majority of users won't abuse the system.

Comments with a direct link to the submission of the other person is indeed a good idea, it might lack that additional push though for being a little "abstract", people will likely just not play the other's game and the community will probably still get flooded with "play mine and I play yours" threads. But I think that it may be a good idea to see how the next game jams rating phase will be with this change, and if it doesn't change much, perhaps it may be a good option to try at least a variation of the coolness system.

You could, for example, make the user only receive the coolness points if he rates AND gives feedback on the entry, this would decrease the amount of abuse and it would also be encouraging people to give feedback on the other's game, which is the most important part imho.

(+1)

Thanks for the reply to this thread btw, you are awesome :D

Admin(+4)

I know there's only a few hours left, but I went ahead and coded this real fast. It's now there as "Most Karma." I think I'm going to tweak the formua a bit though, currently it's 'given - received', but I think it should favor things that need ratings more than others

(+1)

that's awesome :D

There are going to be a few other big jams this following month, it will probably be enough to see if does make a difference.

This topic has been auto-archived and can no longer be posted in because there haven't been any posts in a while.