What 101 Dalmatians Got Right About Playtesting Your Game
1996's 101 Dalmatians had a fun little subplot, where one of the main characters is a struggling game designer named Roger. At the beginning of the film, he's sitting in an office with his boss, and an "expert" tester named Herbert - a preteen boy who supposedly knows how to pick the best selling game every year. Herbert thinks the game's graphics and controls are good, but isn't impressed with the overall experience because the dogcatcher villain Roger designed isn't very compelling.
The last scene in the film shows Roger's triumphant success, as Herbert returns to try his new design, featuring Cruella Deville as the villain:
Even as a kid, I knew the whole premise here was a bit ridiculous and unrealistic. Of course no game corporation consults a single kid to decide which games will be hits. And how did Roger make that whole game on his own?!
But it occurred to me that there are a few things Roger and his game company got right:
Know who your audience is, and curate your playtesters
If you grab a random selection of people from the internet, a conference show floor, or your friend list to playtest your game, you're going to get a lot of noise and not a lot of signal. People play games for different reasons, and have vastly different levels of experience. You can't make a game for everyone, and what ultimately matters is the experience of the people who are most likely to be playing your finished game.
A few people who are exactly who you're making the game for will provide much more value as playtesters than dozens and dozens of random people.
As the designer, you must observe the playtest
If you simply send someone a build of your game and then ask them to send you notes, or perhaps fill out a feedback form, you are going to miss a lot of valuable information that is best observed by you, the designer. At a certain point, you'll want to scale up this effort and things like feedback forms will be necessary, but there's always value in direct observation.
I once watched a streamer play an early version of my game in a livestream, and she struggled for over 15 minutes with one section in the tutorial. It turns out that some of the instructional text was too small to notice, and so she was "locked out" of the main game because she needed to complete a particular task to move on and didn't know it. At the end of her livestream, she gave some feedback:
"That was fun! I would have loved to see some more variety in the characters and weapons."
If I hadn't been watching her play, I would have missed the opportunity to see a major problem with the game's UX. Players can't tell you when they miss something, and nothing will ever be as good as your observation of what they do while they play. You'll be surprised at how much your notes on a playtest session diverge from what they say afterwards.
There is a time and place for more quantitative, large-scale playtesting, but you should make time for more hands-on observation at nearly every stage of development.
Identify your experience goals
Roger had identified (with the help of the 'expert') that his game needed a compelling villain. It was clear he wasn't meeting this goal in the first playtest, and so he drew on his life experience and iterated - then tested again.
Playtesters are happy to give lots of feedback - they'll tell you what they like and don't, and give you long lists of features they want to see.
This can be useful , and you should take note of all of their feedback. But if you want to get the most out of a playtest, you should come into it with a clear set of player experience goals. Maybe there's a certain emotion you want to evoke. Maybe there's a particular HUD element that you want to make sure they notice. The main purpose of the playtest is to validate whether players are having the experience you intended.
A Practical Guide To Internet Playtesting
Over the past few years at Flippfly, we developed a method for playtesting over the internet that's straightforward and effective. I recommend this method when you're early in development, but far enough along that you have a section of the game that's substantially bug-free and feels complete. It's well-suited for F.TU.E (First Time User Experience) testing, but can also be useful for repeat testing with the same players.
- Recruit testers with a google form. This form should ask some questions that will let you curate your list down to a few contenders in your target audience. One of my favorite ways to do this is to just provide them with a big list of games and genres and ask them to put a checkbox which ones they've played regularly.
The form should also ask them if they are willing to test in a zoom call while they share their screen, and have you record it for your internal use. Also make sure they have the right type of computer and input devices to play your game. - Post this form wherever gamers are, along with a teaser image. This could be social media, reddit, your Discord, your Facebook page (though be careful with testing with friends, they'll often be too nice!). Don't give away too much about what you're playtesting - just enough of a teaser to get people to sign up.
This is a bit easier if you have a compelling teaser image and a following who wants to play your next thing. If you're having trouble getting people to sign up, you might consider offering a small payment.
(Whether you can get people to sign up for a playtest can be an important early form of market validation for your concept! But that is a topic for another day...) - Pick a few testers from your responses spreadsheet, and ask them to schedule a meeting. I have a Calendly account that schedules a meeting with a zoom link automatically, which is very convenient because after I invite a tester, all I really need to do is show up for the call.
- Once they've confirmed the meeting, send them a build or a game key a day or two ahead of the meeting. Ask them not to play it until the call, except perhaps to load it up and make sure it runs on their computer.
- When the call starts, briefly explain what it is they'll be playing. Explain that you'd love it if they talk about what they're thinking/feeling while they play, and that they should not ask you questions while they play unless they get absolutely stuck. Ask them to share their screen (using the 'optimize for video' option in Zoom) and then start recording.
The goal is for you to hear from them what their experience is, and to quietly observe. Resist the urge to explain anything about what they're seeing, and instead let them experience the game as if they were playing alone.
This also works well in person, if that's an option. Just sit next to them with a notepad or a laptop.
Take lots of notes! - When they're finished playtesting (I often find 30 minutes is a great duration to aim for), leave some time to interview them at the end. Give them a chance to ask questions and give you free-form feedback. If they seem confused about anything, dig a bit deeper. If they seemed to not like something, spend some time querying more.
I also like to provide a feedback form with a few more "normalized" questions that all the playtesters will answer, and ask them to fill it out immediately after the call if they can.
These questions should produce actionable answers.
"How much fun did you have?" Isn't particularly useful.
On the other hand, a question that gets at whether you're meeting your player experience goals is great, for example:
"On a scale of 1 (strongly disagree) to 5 (strongly agree), how much do you agree to the following statement:
"I thought the enemies were genuinely scary" - Thank your testers! If you have the option, send them a key for the full version of the game. You may also want to put them in your game's credits if it's not impractical, and possibly send them some form of payment. I've found that in most cases, players are thrilled to just be part of the process and to help shape a game in an early state.
- Sleep on it. Don't go back to your desk right away and start changing things. Try to do at least 3 playtests, and see if patterns emerge. Take some time and think about your next steps before you act.
Thanks for reading! And if you enjoyed this, I'd love to hear it in the comments below (if you're reading on the web!)
Member discussion