Dear developer, gamer, jammer…

As I assume anyone that reads this to by now have learned about the controversy surrounding the rating system, I would like to take a minute of your time to elaborate on my personal experience and opinion. Before going in depth I would like to start off by encouraging everyone to not be tempted into contentless discussion, as it is clear from many precedents that this leads to nothing but grief and arguing. Everybody who participated, even those who did not finish a game in time, deserve praise for their attempts and the throwing of mud is just inappropriate here.

That being said, over the course of the last few days I have tried to be constructive in discussing the current rating system. I think it is important to do this by entering in discussions that are provided with valuable arguments and that might contribute to a better experience for everyone, but after writing a few elaborate comments, I decided it might be better to write a proper summarizing blog post where the discussion can be kept alive and be even more transparent and accessible by everyone and it also should be clear that no single individual or group is being targeted.

But before I make my personal analysis, I would like to address some comments that are recurring very frequently. If I paraphrase: “It is not a competition, Ludum Dare is about making a game”. I respect that opinion but I would dare to nuance it. Games are being rated so a developer can get honest feedback, which for me personally was the driving force behind my development cycle. The rank is a nice bonus and seemingly flows naturally from the scores given in the various areas. So, you might be right, for some it is not a competition, for others it is, but what we all should share is the desire and right for recognition of your work, to learn and grow in the areas we excel in or lack. Another one is “It has been like this forever, it is what makes it Ludum Dare”. I am horribly sorry if I should offend anyone by saying that this is a load of crap, and is even contradictory to the earlier statement. For me personally this was my first entry, but I did follow Ludum Dare for a few years now and I have to say it is very sad to see the amount of demotivated game developers after the jam. It might be true that problems have been around for a long way, but ignoring them for the sake of having a popular opinion is not the way to go. It can be solved.

Finally, do allow me to expand on my personal experience. I decided to join the Ludum Dare event pretty late, I believe I officially committed to doing it not 24 hours before it started. I remember going through the rules the day before and having friends telling me I should worry less and it all is relatively straightforward. After that I asked my wife if it would be for me to lock myself up and make the best darn game I could. Laugh all you want, but I respect her and my toddler son enough that they should be okay with me doing it. A 48 hour jam is not a small task and I would not be able to help a lot in the household. 48 hours later I had slept for just 5 hours and I took some breaks to have some time with my kid. I also noticed I was sitting down too much as my one foot was swollen to double its normal size. So an important lesson learned…No jam is worth that, take plenty of rest, get up, walk around. Anyway, I finished the game loop and decided to not bother with too much art anymore, which also is clearly the explanation to the poor score in that area. I was proud of what I had and I was awaiting that sweet sweet sense of recognition when the voting was underway. Fast forward 3 weeks, the ratings were released and I was happy at first, I wound up where I expected to be and that was slightly above average! Not bad for a first time compo submission! But then I started seeing alarming posts. Both offensive and defensive, the blame game was underway and things got even worse when it turned out the rules might have been inadvertently not respected for one of the winners. To make matters worse, due to poor communication there were indications that the score was manipulated due to the fact a single group had voted within its own members. I have to say, at first that did raise a lot of questions to me and I have tried to be constructive about it, but I have noticed the back and forward salvos of arguments are still flying over everyone’s head, which is also the major reason I decided to make a blog post.

So here comes the juicy part… Was there cheating? Well that depends on what you consider cheating. According strictly to the rules? Yes there was cheating, as one of the games won the audio category, while none of the audio was made by the contestant. This was amended though, so if that is cheating or bad preparation (not everyone is as crazy to read the entire page of rules as deeply as I did), I leave that open for interpretation. What does concern me way more is the way the voting system works. I was not aware of it before the start and I apparently needed to see it for myself. The alarming part is that the winning game has but 20 votes, where there are other games that received over tenfold that number. I learned in statistics that these sample sizes are not comparable at all, furthermore any ranking based on averages is a nice attempt…But meaningless. Luckily (but maybe not so much for the gamedeveloper) this was clearly exposed during this LD, even more than was the case for previous editions. In the comment section there undeniably was what I call “friendly voting”, overestimating a game’s metrics, which caused its rating to skyrocket. And just to be clear, this is not bashing the unnamed group in question, the developers behind it or even the hosts of the Ludum Dare events. This is just me trying to explain what is not a “flawed system”, but what is a broken system all together. Statistically, pulling a random entry out of all entries to win would make more sense than having varying samples from varying voting pools. I received arguments that the event is too large and rating 3000 games in a way that is both qualitative and fast is impossible. That brings me back to the same arguments I received before writing my comments. If the rank is not important to anyone… then it does not matter if rating all games takes 3 weeks of 3 months. So here is my suggestion:

The days following the submission date for ludum dare were filled with enthousiastic twitch streamers (probably also a lot of activity on youtube, but I did not personally follow that). Sidenote, these people deserve utter respect, I have seen some of them play over 150 games while ranking them fairly. The way it often works now is that a streamer either selects random games or has viewers register a slot to get played. I also have noticed that the LD page features twitch streamers. These people are doing it for free and combined, they played (roughly estimated) over half of all games. I would make these channels featured referees during ludum dare, giving them more exposure while they in return rate games on stream. It is a pure win-win situation that doesn’t have to cost anything but a title and a feature status on the LD page, something that already is implemented for the largest part. The counterargument for this is that the public should not lose its voting power. That is most definitely true, but a refereeing system is not intended to eliminate votes. It rather is a layer on top of the public votes, where the effects of “buddy voting” will be reduced. Should it still be required, an even smaller panel of “experts” or “LD veterans” could rate and play the finalists, making a semi-final decision. A lot of salt would be avoided this way and it surely would be beneficial to everyone, as even games that did not make it to 20 votes (it is a popularity contest, do not even question that) would get a form of rank in one way or another, through the streamer-referees. And now, intentionally saved for last, I want to put down an official disclaimer that I am not a salty loser and I do not want to be in the victor’s place. Despite my slightly above ratings, which I am happy with, I can’t deny I felt a little bit offended by blind and wild accusations of being jealous and a sore loser (some people call me java-sore-ass already, thank you @HonestDanGames), but I only want a constructive discussion and hope that the LD team picks this up. For me it is clear, the post-jam explosion of comments and arguing has made me believe LD is “not my jam”, at least not until the rating is ameliorated.

But who knows… The itch might come back like it did for LD41 and I might join in again next time, maybe “just for fun” like the majority of you people out there 😊. Heck, I might even do it all on stream (SHAMELESS ADVERTISING)

With a giant respectful roar, cheers and Dev On,

JavaSaurus