Dr Dobbs Interview

The topic of interviews came up while chatting with a friend recently, and I noticed that the Dr Dobb’s interview I did a couple of years back is no longer available.  I’m reposting it now to have my own copy and to save me from having to dig it out of the Internet Archive in the future 🙂

Originally featured in ‘The book of testing – Thoughts from a Braidy Tester’ at http://www.ddj.com/blog/debugblog/archives/2007/12/five_questions_41.html

by Michael Hunter
December 18, 2007

Five Questions With Jared Quinert

By The Braidy Tester
Posted: December 18, 2007 07:30 AM
Once upon a time Jared Quinert tested console games. Then he led teams of testers of console games. Then he built the testing department for a console game which never actually shipped. Currently he is spreading the context-driven testing love throughout the corporate world as a consultant with Aegeon, one of those Web 2.0 companies.

Here is what Jared has to say:

DDJ: What was your first introduction to testing? What did that leave you thinking about the act and/or concept of testing?
JQ: I started as a games tester at the beginning of 1995, catching the tail-end of ROM-based games and the transition to CD-based PC and console game titles. Due to the expense of a product recall, those games needed to be rock-solid. Back in those days, our test approach was 100% exploratory, manual testing.

On my first real project, a bug report was thrust into my hand at some point, and I was told to wander around and ask the programmers what they had been working on and what they had changed in the build they had just given us. They would share information about things they thought were complex, risky, or otherwise. I would tell them what kinds of things our team was finding in the product.

After four or five months I was thrown onto a title as test lead, managing one other tester. Suddenly managers were wandering up to me and asking ‘How’s it going? When will we be done? Can we ship it?’ Pretty quickly, I needed to figure out how to answer those questions, and thus I learned about developing coverage models to support exploratory test management.

The company I was at also developed the first third-party Windows game for Microsoft. Microsoft came to us wanting a heavily scripted test approach. Our test group wasn’t even aware that people tried to test things by writing all their test ideas down. We definitely felt we weren’t going to test *our* product that way. After initially rejecting the idea, I did begin to think about what problems they might be trying to solve with their approach. Some of the ideas were adapted and found their way into our regular testing to solve particular problems. So while it was great to get the testing grounding that I did, I’m also grateful that my view of things was challenged fairly early on in my career.

The big things my first job left with were:

• The idea of testing as a service role, and information provider. This came from the relationships we had with developers and producers.
• The idea that exploratory testing, done well, can deliver quality far above what I’ve generally seen using heavily scripted testing approaches.
• The knowledge that if talented people do exploratory testing day in, day out, they get really good at it.
• The idea that face-to-face communication is critical. We moved offices, were split up across multiple floors of the building, and bad things started to happen.
• The knowledge that having skilled testers allows you to travel much lighter.
• The idea that testers have to tell a compelling and realistic story when reporting bugs. We were often representing the concerns of important people who didn’t have a voice on our team: Gamers of different skill levels, with different preferences, publishers and support people.

My understanding now is that not all games testing groups rely on smaller, skilled teams, so I could quite easily have come out of a games testing role with a very different view of testing.

DDJ: What has most surprised you as you have learned about testing/in your experiences with testing?
JQ: The big surprise was how useful the skills I developed as a games tester were when I moved to corporate environments. I think this was largely due to the testing philosophy instilled by my first role. I was also surprised by how much worse I perceived the testing in those environments to be. Using exploratory approaches, it was really easy to find bugs that had been lurking in systems for years. Given the potential financial losses involved in many of these systems, I had expected to encounter testing even more rigorous than my games experiences.

Testing for internal audiences I find is a much vaguer experience than consumer product testing. The politics are more complex, and problems are solved in different ways. Learning to be an effective tester in different environments is a huge, probably career-long task.

DDJ: What is the most interesting bug you have seen?
JQ: The most interesting games-testing bugs were usually needle-in-haystack kinds of problems. I remember one of the first bugs I found, which was a crash that happened only after playing seventy percent through a game, then applying an obscure button combination while attacking an enemy which only existed in this one room. I then surprised myself by playing the game back through to that section and reproducing the problem first time. I think that as the programmers later explained to me why it had crashed, I connected the dots and the bug value of null pointers was suddenly burned into my brain.

I think another thing that never ceases to surprise me about good testers, and that’s how much bad luck seems to follow them around when they’re testing. Bugs seemingly fall into their laps. While this may seem like an unfair advantage, I think it needs to be backed up by good investigative skills and the tester’s ability to pull the pieces together and retrace their steps.

DDJ: What do you see as the biggest challenge for testers/the test discipline for the next five years?
JQ: I see a few challenges. Firstly, that business right now seems particularly keen to reduce the cost of testing. Secondly, developers are becoming interested in testing, and getting better at it. At least in my part of the woods, much of what passes for testing is heavily requirements-based, and fairly clerical. Between outsourcing and enthusiastic developer test automation efforts, traditional tester roles are going to be under increased pressure to deliver value. If we want to have successful, satisfying careers as testers, I think we are going to need to skill up and figure out how to keep adding (enough) value.

The third key challenge is figuring out how to get some experimentation into the workplace and push the boundaries of what we know about testing, in order to test better. Persuading businesses to take on appropriate business risk for an appropriate reward seems like a hard sell from my view on the ground.

DDJ: Is there anything else you would like to say?
JQ: I’ve sort of undertaken a mission to change the testing landscape here in Melbourne, Australia. It’s a slow process, but a community is growing. If anyone feels that they can contribute, I’d love to hear from them!

——————————

Find the original copy at http://web.archive.org/web/20071220220538/www.ddj.com/blog/debugblog/archives/2007/12/five_questions_41.html