In general there is a tendency to think of playtesting as simply playing the game lots of times. This is an important part of the process, but I think it’s only part of the story.
Playtesting is a complicated subject when it comes to unravelling best practice. For today I’ll look at just one small aspect that I think it is easily overlooked: trying to see the wood for the trees.
Most games are quite complex assemblies of many sub-units, each of which needs to work on its own as well as a part of the whole. Once they’re all working alongside each other it can be quite difficult to decide which bits are working properly and which need more work. You might know that something isn’t right, but what is it?
My normal practice is to try out different parts of each design in as much isolation as possible so that you can see more clearly how individual elements function, and then tweak them so they do what you really want them to do. The challenge here is that many parts of an interlocked design don’t really do much on their own. That’s when you need replace other parts with vanilla stunt doubles to clear away some of the noise that might obscure the workings of whatever bit you really want to look at if everything was in place with all its variety.
Once you have got a couple of pieces working, run them alongside each other and see how they work. Building up gradually like this gives you confidence in a solid design foundation. If you add one thing at a time and that breaks a previously working game, then you know where to start fixing stuff. Yes, there are exceptions, but this is a better start than most.
For example, I’ve started working on DreadBall Xtreme. So far I’ve been looking at a number of aspects of the game play itself, but I’m not currently concerned about the way teams are picked (which is very different in DBX) or the actual stats teams have. So, I’ve been using a datum human team on both sides to play the games. This enables me to see how the teams interact and the game moves back and forth because of the Rush structure, card play, traps and other drivers for that aspect. Obviously players with different stats will change this, but the core will be there and will do what I intend because I’ve had a chance to see if operating by itself (as far as possible). Similarly, when I designed the first DB I played it many times without cards, coaching dice and so on. I wanted it to be a fast and fun game without any frills and so played it till it was. Then, when I came to adding chrome, I knew the foundation was solid and that I was adding a cherry on top.
Another example of the same sort of approach is my current work on Eternal Battle. At the moment I’ve just finished a third version of the action/initiative/turn mechanics. I wrote one version that I was quite happy with, then an idea for a different approach popped into my head so developed that to see how it compared. I liked that too, so which to pick? This third version is a hybrid of the best bits of the first two and is my favourite of all three. Throughout this development I’ve been assuming that every dice roll succeeds half the time, giving me a reference point when I develop stats later, and also giving me a standard background against which the ebb and flow of initiative, actions and turns can be more easily seen. This lack of other variables getting in the way has allowed me to see what was going on and understand the value of the subtle differences (and issues) with each iteration.
Over the years I’ve seen a lot of troubled games in various stages of development, and in many cases the issues the game was having could be most clearly seen when some of the other systems were taken out of the way. I assume that some clever fellows do all this in their heads, but for me I like to see the working parts dismantled and laid out on the bench in front of me. I’m not smart enough to imagine that level of systemic isolation in the abstract, or perhaps I’m just lacking the confidence to say I’ve done it without seeing for myself. Either way, this is what works for me.
Quirkworthy 
I’m a programmer and these are just like the steps I take. I don’t start out and throw a bunch of code together to make a complete program. I start with a small chunk of functionality and I test that small chunk in isolation and ensure it does what I expect(typically called a unit test). I then work on another chunk and do the same. Eventually, I start putting pieces of functionality together and testing them out together(typically called integration testing). Continuous integration is used to make sure when I add something new that I don’t break something else. If you look up agile development, you’ll see these kinds of ideas. I think there are many other areas we find ourselves in where this approach to a problem can be applied. Some people aren’t wired this way, though, and I think they do it in their head like you mention. Overall, I think this process leads to some of the best and most robust solutions. Nothing will ever be “bug” free, but humans aren’t perfect either.
That is interesting to hear you describe your creative process. I would not have guessed that there was so much painstaking effort and calculation involved in creating the games I enjoy. When I play Dwarf King’s Hold or Deadzone the games just flow easily and the rules are memorable and intuitive. It’s so easy and fun to play the games that I could be misled into believing that it was just as easy to create them. I am grateful that I can enjoy your creations. 🙂
I love getting a window into your process. Thank you for sharing. Do you find that when you discover a component that works really well, that you will may use it in other games as well? A function in programming is the kind of thing I am thinking of or is each game so unique that you create the components each time? Seems like this kind of testing also works well if you are testing solo. If so, is this the kind of approach you would use to create an AI in a game for solo play?