I was asked the other day about playtesting. Over on the Games Design page John said the following:
“I’ve been developing a game with a few friends for a while, could you maybe provide some insight into what kinds of questions to ask playtesters? General feedback is good, I know, but are there specific things you ask or things that you look for when watching others play ?”
Well playtesting is a large and complex subject, and I have been meaning to write something about it for ages. It is, however, somewhat tangled and I would probably want to break it down into a number of bits. That will take a bit of time. However, for the moment, let’s have a think about John’s question and see what there is to say off the top of my head.
Step One
Regardless of what type of game you are making, the most important question is this: is it fun? A game that isn’t fun needs something serious doing to it before you worry about any details.
If it isn’t fun then ask what isn’t enjoyable. Do your best to avoid comment here and just let them speak. You presumably liked it enough to show it to them, so you may already think it’s great. But they don’t. You need to find out why. Encourage them to be honest even if it’s painful¹. If your playtesters hold back then you’re only saving up trouble for later.
If there are problems at this most basic level then you need to carefully dissect the whole thing.
Still, all is not lost. Be sure to also ask what they did like. Most designs that aren’t fun overall are still working well in parts. You can build on this. Keeping the bits that did work in mind, be prepared to do some major restructuring.
Avoiding Confusion
One important thing is to limit what you ask. Playtesters are dealing with a new game and there are a myriad things to consider. Try to get them to focus on a specific aspect of the game in each session. For example, are the rules clear? Are the different scenarios sufficiently different or do they duplicate each other? Are the different sides balanced? Are the rules really clear?
With some of these you may need to break it down further. This depends on how you have written/developed the rules, which stage of testing you are at, and what kind of game it is in the first place. You also have to consider what the end result of the game needs to be. If you are intending to sell it to the public then you probably want to hold it to a higher standard than if you expect to play it only at Christmas with your immediate family.
So, deal with one bit at a time. This reduces confusion and allows you to get a solid basis on which to build the rest of the rules. Some questions may need several sessions to resolve, others can be fixed on a single run through. Playtesting is an organic process that evolves as it goes along. Don’t be afraid to adapt as necessary and change your plans as you go along. Far better to dwell on something longer than you’d intended than to miss something important.
Sequencing is often critical. For example, there’s not much point in fine tuning the balance of the forces before you’ve finished with the core rules. Changing the rules on which the balance is based simply wastes that effort. By all means do a rough cut, but be aware that you can’t get a final balance till the rules themselves have settled.
Reconsidering
As a designer, it’s very likely that you will be more defensive and protective of your design than you really should be for its own good. Spend some time after each feedback session pondering which darlings could be murdered and which criticised elements really need to stay (and why, and what needs to change). Then bin the expendable ones (but perhaps keep some notes) and look very hard at the bits you keep. How can they be improved? This gives you a good idea which areas you still need to concentrate on.
Teach Me
One of the best pieces of advice for making sure a playtester can understand the rules is to get them to teach you how to play. You can do this at a number of different stages in the process, but you can only really do it once with each tester, so save it for important steps.
Simply give them a set of rules and the time to read them. Then sit down and have them explain the game to you. The hardest part here is not leading them on to the conclusion you know you meant (but perhaps didn’t write). Take notes. If they are going down a completely wrong track then you might need to steer them back. Ask them to check the wording of the rule. Perhaps they are mistaken? Most people make mistakes when they play a game the first time.
Remember that you’re testing their understanding here, not their skill (or yours) at the game itself. Are you playing the game you intended, or is there some confusion and miscommunication in what is written down?
I’ll say again that the tricky bit here is keeping your mouth shut as much as you need to. Once you open your mouth, keep comments to the minimum needed to get them back on course and then clam up again. Do not let them get to the point where they are looking to you to check the rules. Keep a poker face. Ask them rules questions. Make them refer to the written rules as that’s all anyone else is going to have.
Belief
As with any other creative endeavour, game designers need a fair amount of self belief. There are tens of thousands of games out there already – why is yours worth playing? What’s special about your baby?
You also have to believe in the feedback of your playtesters. By this I mean that we’re all individuals and you have to be careful not to allow a single strident voice to colour the whole development of your game. Ask each person in the test what they thought, especially the quiet ones. Don’t just listen to the loudest voice. Try to gauge the consensus opinion on criticised elements. If everyone hates something, finds it dull, doesn’t understand it, etc then it certainly needs looking at. If a minority don’t like it, especially if others do find that bit works, then you have more of a challenge.
At the end of the day, no game suits everyone, and so you cannot please every individual gamer. But you can aim to please the bulk of your target audience. You did think about your target audience, didn’t you? Wasn’t that part of your brief?
1: Don’t be ashamed to weep openly.
Quirkworthy 
This is actually extremely helpful to me as I am currently in playtesting of my own game, a monster design and battling game. With the designing a monster section there 140+ options for a player to choose from, all of which are compatible with each other allowing for a massively broad range of playable characters. Literally thousands of possible combinations and variables… What would be your suggestion for playtesting that? Its a daunting task for me, let alone my playtesting group!
I am willing to send you the rules I have so far for you to peruse if you need more information on how best to answer that question, and would immensely appreciate any and all input you could provide.
Surely you are initially playtesting with a just small core sub-set (say < 15) of the 140+ options? Mathematically you have so many variables there that the complexity will go through the roof!
I would guess that a good approach would be to get a stable core of rules working well and then to slowly add extra rules and playtest with the new rules as you go.
Well thats what I have been doing. But with all the variables of different combinations as the options are added in, it then throws all the previous playtesting a little out of whack as one thing works against another, but changing the same 2 monsters up with new options makes it drastically different. Im probably not explaining very well but there must be some sort of algorithm for it all, which is why I thought id ask Jake in the first place as with all his experience he may have come across something similiar.
Sounds tricky to balance indeed, it will be interesting to get his take on this kind of problem.
Agreed 100%
It’s better to start with the core rules and build off of them. Dreadball is a great example of that.
Season One’s teams were different but relied on the basic rules. You could then build off of that in Tournys and leveling up your players.
Then Season Two came out with teams that had as a base special rules (Love my Pelgar Mystics) and introduced teams that Required special rules (Robots).
Season Three introduced Biguns’ and Ultimate Dreadball pitches.
This escalation not only gives players a good ramp up to the most current rules by building off of the previous seasons, but also provides clear plateaus when it comes to teaching new players the game.
The core rules for my game are solid. That is the movement and combat mechanics for monsters with both randomly generated stat lines and purposely chosen stat lines designed by the player as choosing a statline (in a vein similar to how you would when you start an RPG) are central to the game. I was very nearly reduced to a gibbering wreck when the first version was torn to pieces by my group! (which apparently according to Jakes post is ok!)
Whilst playtesting the core mechanics I intentionally left out options to customise the creatures, but now we have determined the core stability, it is time to take to the next level. And that means adding in options. Which as I described above is harder than just breaking them down into smaller chunks as each interacts with another in unique ways.
I do appreciate all your responses so far as any and all input is great.
Why is it harder than breaking it down into small chunks?
The advice here is sound, and the comparison with DB accurate. In fact, Db goes a step further back. When I was designing that I wanted to ensure that it was, at its heart, a challenging and interesting thing to play. For this reason I played it for some time without any cards, Coaching Dice, Fouls and so on. When it was slick and clean like that, then I began layering.
You say that the core is solid. That’s great. It sounds like there are a number of possible areas of difficulty. Bear in mind this is all guesswork.
1) The core might not be as solid as you think. Perhaps it would be worth playing the stripped back version again with testers. This is your foundation so it won’t hurt to triple check its solidity.
2) The whole thing is too complex and could be simplified. Simple doesn’t necessarily mean lacking in depth. Do people really want 140+ options? If you have 140+ options that all work in different ways then you probably need to pare that back. Most people won’t bother to learn that many subtleties. It’s rare that a single effect cannot be achieved in more than one way. See if you can reduce the number of different systems at work at once. That will make it easier to streamline them. A classic error of inexperienced (and some experienced) designers is trying to put too many different mechanics into a single game. You can always save some for your next game 😉
3) You may be starting in the wrong place. Always start with a datum creature. Even if you are not going to have a human in the final game I’d start with a human anyway. This gives you a reference point for game values. Everything else can therefore be relative to a known value range.
4) Introduce things in very small steps, and work up from the simplest and most mundane towards the more exotic. Do a horse, a cow, an elephant, etc as reference points. The difference between a human and a horse will give you a scale reference for game values. What does the difference between 3 and 5 mean? What is the maximum range of human strength? Is it 1-5, 34-35 or 1-700? You don’t have to use any of these creatures in the final version, but being able to model real things that you can research allows you to gain confidence in the way the rules model things. Then, when you create the lesser spotted murgle-burgle it has a reference, and therefore a reality. When things have a point of reference then it’s easier to see why things aren’t working and where the problem is. If all of your creatures are fantastical then you have no reference point for what’s going wrong. My lesser spotted murgle-burgle walks at Mach 7. That might seem a little fast, but it’s hard to know without a reference. In fact, it only sounds fast if you know what Mach 7 means, ie you have a point of reference 😉
Hope that helps.
Thanks for this, I’m honored! Also thanks to theottovonbismark for the tips!
You’re welcome John. Does the above post and comments about Alex’s game help?
yeah, very much! I really like the idea about playing without peripherals ( cards , dice etc), i might try that because an element of my game is card based.
A very interesting article. While I am not designing my own games it is interesting to consider the rulesets one plays from the game developer side. This also makes it easier to come up with house rules and implement them.
Some reason I cant put a reply directly below your comment in reply to my earlier ones, so ill put it here and hope you see it.
Thanks very much for your input and whilst reading it, it helped me realise I’ve been looking at the bigger picture too much. I’ll circle back to that in a second. The 140+ options maybe arent completely necessary but they add a lot of flavour and its a core principle of the game. It almost takes precedent over the actual fighting.
To circle back, i say looking at the bigger picture by which I mean seeing the large number different options as seperatie from one another but in reality they break down into smaller chunks quite easily. I just couldnt see the trees in the wood as it were. There are those that add dice, those that modify a movement type to another, modify attack number and type and those that add the odd rule that isnt covered by core mechanics. So in reality its actually only 5 or 6 different types. I can work with that.
Thanks
On the subject of playtesting (and dreadball) were all dreadball teams created equally and if not how would you rank/tier them as a friend and I are trying to rank the teams
Just curious after reading this article if there was anything you would have done differently with the alpha rules release and testing for dungeon saga? And if the nature of the release (giving pledgers a taste of the game) changes how you’d approach play testing?
(I originally had a more indepth question but realised the short answer is quite possibly “no”. And so I’d just be wasting everyone’s time.)