So we’re a little over a month into the judging, and its been a lot of work so far. A lot of fun work mind you, but still a lot of work.
I thought I would take this time to talk about how this year is different from last year, how we are doing things differently, and how it has improved the contest overall.
The first thing I will say is: We are on schedule. Last year, there was a lot of delays, but this year, we should be able to hit the original date we stated for the winners announcements. So what did we do differently?
More Realistic Judging Period
Last year, we had a one month judging period. Which, we thought was plenty. I mean, how many entries could we possibly have? As we watched them come in in an endless stream we realized how many it could be.
I swear, somewhere, is the end to these submissions…
We ended up with over 700 submissions. With one month, it was nearly impossible. Once you add in that the judges had other things to do besides judge games the entire month, and it was soul crushing. Even as fun as it was, there just was never enough hours in the day.
This time, we decided to go with a TWO month judging period. 7 Weeks for actual judging, then another week and a half to play with before announcements. This lets us pace ourselves much better, and I think it lets us be better judges.
Better Organization
Last year, we expected a few hundred entries. Not SEVEN HUNDRED. So we divided into two categories, RPG and Non-RPG, and just let it ride. Each of our four judges judged half of the games. Two of us judged all the Non-RPGs, and two of us judged all the RPGs. That meant each of us played nearly half the selection. Even with screening, it was a lot. I remember judging fourteen games a day for a week at one point.
This year, we have double the judges, and we are using a tiered system where games are divided up, then the best of each grouping is handed up the chain until the best of the best have been played by everyone. This lets us each have to play less total games, while still finding the best possible games.
Click for a full sized version of the organization chart.
This setup was organized to easily accommodate up to 1500 entries. We didn’t have nearly that many, but this organization of entries means that we can continue to use this from year to year with no hiccups; unless we grow a ridiculous amount in one year, we have it covered.
Better Communication
This year, every judge is in contact with every other judge. Last year, all the communication was unfortunately a bit more loose. We didn’t act as a cohesive team, and I feel the contest suffered quite a bit because of it.
But changes have been made, and communication between judges is near constant now. There has been hardly a day gone by since the start of the contest where I haven’t talked about something with at least one of them. We are very careful not to mention individual games, as we do not want to influence each others scores, but we are able to keep up on where everyone is in the judging process and how things are going.
Better Automation
Last year, each judge kept track of their scores and comments separately however they felt worked best. This was a huge problem when we got to the end of the contest, and were trying to send all these scores and comments to the people who requested them.
This year, we have revamped it completely. Each of us has a judge dashboard on the site. It divided the games out to each of us randomly, and combines the lists at the end of every round. We disqualify (most dqs were from missing download links), judge, and write our comments in directly on the site. And when the judging is all over, we can automatically send it out to everyone.
Here is a shot of my judging panel taken during the screening process. With nice big black bar to prevent spoilers :P.
This means, less work for us, and less delays for you.
Overall, everything is just moving much, much smoother. I’m super happy to have been in charge this year, and judging has been a blast. I won’t say I’ve never felt a bit of slog, but anything done for a long period of time, no matter how fun, can drive you a bit batty. But overall, its been amazing fun. There have been good games, and not so good games. Several have really wowed me. I’m really looking forward to seeing more.
But enough from me, let’s close this out with a few words from some of the other judges:
“It’s been great to see all the creativity that goes into most of these games, even ones that might have issues with bugs or gameplay. There are a lot of really creative people out there and I feel lucky to see all these ideas in one place. I was also impressed by how many people challenged themselves by using newer engines and ambitious concepts. Whichever game gets the top spot will really have to bring it because there’s a lot of good stuff to choose from.”
“First time doing judging for IGMC, but I’m very impressed by all the different games. There’s a lot of surprising ideas so it makes it very entertaining to judge them all. Congratulations on completing your games! I look forward to judging more of them.”
“Although the amount of games we got this year is less than we had last year, there’s a definite improvement overall and I’d like to thank everyone who returned from last year for trying harder and pushing themselves further during the course of the year. I’m looking forward to seeing how people grow next year too, so keep up the great work everyone. It’s amazing to see so many people come together and make games and just pull together as a community. New or old, you all deserve to feel proud of the accomplishment that is creating a game and as a veteran of the community it warms my hearts to see so much interaction between all of you. Keep supporting each other in your game making endeavours – you never know when a kind word will inspire greatness!”
Its been great so far, and hopefully everything continues on as smoothly as it started. Thank you everyone for your entries, and we look forward to telling you how awesome each of the winners is!
0 comments… add one