Testing, Testing...

[This blog post was written by Taylor Eshelman, one of our artists (and sometimes testers)]

My job description is "artist." I bill myself as a "technical artist" since I have a technologically savvy streak. I've even branched out into design here and there, most notably in an extracurricular collaboration project that's a variant of the "game in a day" that we did a while back. Today, though, I'm a tester.

Yessiree, I'm a QA drone. I've argued it before, and I'll do so again: Game testers don't get anywhere near enough credit for what they do.

In many ways, the QA (Quality Assurance) people are the last line of defense in game development. It's nearly impossible to ship a game with absolutely no bugs, but games that ship with egregious problems fail quickly and may never recover. Vanguard had a terrible launch, but has since been polished and refined into a much better game. Even so, that initial impression of a game that was literally broken in several ways has lingered, killing the early adopter buzz which can be so crucial for hitting sales targets. Gamers can be terribly fickle and judgmental, and a poor first impression (or a bad early review or two) can sink a game, whether that impression is well deserved or not, and whether or not real complaints are addressed adequately. As in so many other aspects of life, a "best foot forward" approach is crucial for games.

QA testers are the safety net for the game development process. There are so many cogs and fiddly bits that come together in any software project that there will almost inevitably be things that don't mesh well. The only way to find many of these problems is to play the game. Over and over. And over and over.

You see, game testing QA isn't very similar to the end user experience. QA people have to repeat the same procedures many times, trying to suss out how things might break. They actively try to break the game, which can include doing normal "gamer" things, and more, by doing really weird stuff. (You know that someone out there will do something so totally wacky with your game that they will break it in ways you never would have thought to test... but you still try to test as much as you can to head them off at the pass.)

That's inherently different from trying to "beat" a game. QA people do have to keep the concept of "fun" in the back of their mind, and notions like "balance" and "emergent gameplay," but for the most part, testing means actively trying to find *problems* in a product, not trying to find *fun*. That's why it's a job, and why it takes unique skills. That's not to say that "gamers" can't also be testers, just that the two aren't equivalent.

The writers over at the Elder Game blog rightfully point out that QA isn't something that is best done in a vacuum. Testers need to know how the game is expected to work, in order to try to break it. That may seem obvious, but it's an application of the "thinking outside of the box" cliche: in order to try to stretch an idea or game outside of its box, you have to know where the box is in the first place. The Elder Game article describes this in more depth, so I highly recommend popping over there to see what they have to say. Long story short, the designers, engineers and artists need to work with the QA department to make sure they have the tools and expectations to properly put the game through its paces. (The old "It's a feature, not a bug!" argument after the fact doesn't work all that well.)

Yes, ultimately it's the engineers and artists that create the content and string it all together into a coherent package, but they will inevitably mess something up. We aren't perfect. We're relying on testers to catch the things that we missed. We certainly self-test a fair dose of things, but sometimes bugs don't even show up until things start coming together. The project that I've been working on has unique bugs that show up when the code goes from the PC where I worked on the content to the final game platform (a different bit of hardware). I'll never see certain problems testing things on the PC, despite doing all I can as an artist to make things correctly and test them before committing changes to the database.

Even in an age of "ship now, patch later", good QA is vital to getting things as right as possible for the initial release. Games are such transient bits of entertainment that a buggy product can totally miss the window of opportunity on the market, even if it's fixed later. As such, good testers are often a key component to financial success (between content creation/coding and marketing), which, at the end of the day, is what keeps food on the table.

Game development is an organic process with a lot of give and take. QA is a crucial component of that refinement process, and I, for one, find a great deal of responsibility in the role, and a great deal of appreciation for those intrepid souls who do it day in and day out, often toiling with little recognition.

-Tesh