bout a year ago an Internet colleague of mine, Peter Jahn, wrote a great story on Star City Games about how his team helped develop Mirrodin from a multiplayer angle. It was all very top-secret until after the release, and Peter was clearly relieved he could spill the beans a couple of weeks after the set was out. (He couldn't even tell me beforehand, even though we both judge at prereleases for Steve Port and Misty Mountain Games, we're both in a secret society of state bureaucrats, I'm under non-disclosure agreements myself because of this column, and I have a long history of keeping my mouth shut on important stuff like this! Bravo to Peter for sticking to his word.)
As much as I like Peter and his super-judge wife Ingrid, I have to admit the article burned me. Peter scooped me! On multiplayer testing! For a cool set! Curses, Jahnses! We hates it forever!
I sent an email to Randy Buehler that essentially consisted of a link to Mr. Jahn's article and a bunch of question marks and exclamation points. Unusually pithy, for me. The anger must have seeped from my electronic communication and soaked Randy's desktop. After having his crouching, flinching minions come in and mop it all up, Mr. Buehler got busy making sure this site's touchiest prima donna didn't get my dancing tights all twisty. I soon got an email from Elaine Chase, who was on the Development team at the time:
Somehow, we've gotten the vibe around here that you and your multiplayer group may be interested in playtesting. :)
Details followed on how our group could help on the major set then code-named "Earth" (as in Earth, Wind, and Fire): we would do pre-surveys and post-surveys of our impressions of the set, we would need to be honest and not be nice just because it's cool to playtest a set, we'd have to commit to playing regularly, and above all we would have to stay absolutely quiet or horrible things would happen.
I immediately forwarded the opportunity to the rest of the group and posed the question: should we do this? (After all, Elaine was not entirely correct. I had fired off my angry, wordless missive to Randy as an act of solitary protest.) I was pretty sure they'd want to do it; but of course this was a good time to ask.
Our group had about ten guys in it at the time. None of these guys are exactly young, bright-eyed, naïve, gee-golly types. Rather, they're 30+ (and older), well-seasoned professionals in analytic careers. Actuaries, computer network specialists, dark stuff like that. So let me ask you: is their first reaction, "Wow, Anthony, this is really cool! What do we do next?" Is it, "You rock like the Rolling Stones on a mountaintop – how did you get so darn cool, Anthony?" Is it, "Alongi, You are a God among men. How may we best worship You?"
No, no, and hell no.
I get back a litany of questions – how often will we have to test? Do we have to show up every week? What happens if a new player shows up mid-testing? And hey, by the way, when will you pay me back that $40, Alongi-tog?
Okay, there might have been a little gratitude in there as well. (Yeah, there was lots. Our group isn't stupid, and we know a good opportunity when it lands in our collective lap.) But these questions were good ones anyway, so I posed them to Elaine and got answers. We'd have to test at least once a week for 2-3 hours, preferably twice. The team lead (me) would determine how often was often enough, when it came to attendance. New players would be discouraged – and we couldn't tell them squat about our experience until the release.
Also, my friends were screwed on that $40. BUT we'd each get a free box of product after the release. AND we'd get to say we did it. AND we'd each get a framed, backlit, black-and-white photo of Randy with a neat autograph like "To a special you".
So they sent us non-disclosure agreements, and we signed them; they sent us questions about ourselves as players, and we answered them…then they sent us playtest cards, and we cracked 'em open.
Magic: The Playtesting
The disclaimer: Even in this column, on this site, and after the fact, I'm limited in what I can reveal to readers. Neither my friends nor I can answer any speculation on what happened during playtesting. But Randy and the good folks at Wizards will let me speak to a couple of cards and experiences we had.
The team: The ten who tested regularly were Bob Drosky, David Hanson, Paul Hemze, Curt Jorenby, George Maverick, Don Musch, Todd Petit, Joe Santos, Paul Shriver, and myself. None of us are anyone of particular import in the Magic world, though you might occasionally catch one or two of us at Magic Online.
The mission: Our job was to focus on the card(s) in non-sanctioned formats, with an emphasis on multiplayer. We were given specific instruction not to care about whether a card was a bomb or not for limited/constructed tournaments. So if you have a problem with a card that's "not costed right for Standard"…well, you're looking at the wrong playtesters.
We tested virtually every card they gave us (setting aside repeats like Stone Rain) at least once in real-multiplayer-game conditions. The formats ranged from simple chaos to "hunt" (restricted target) to Two-Headed Giant to constructed Emperor to draft emperor.
The tactics: As soon as I got the stickers, I passed them on to Joe, who used a scanner to capture the text and make full (four copies) playsets for as many players who wanted them. (Yes, we checked with Wizards to make sure we could do this; and no, you may not have any material we used to test.) While a few of us "claimed" a card or two, for the most part we just went where our imaginations took us. We each made anywhere from two to ten decks. There was plenty of room in the set for ten of us to build decks without coordinating, and we didn't run into each other too much.
Paul H. spent time arranging some of his cards into "randomized" draft packs of 15, so we could play draft Emperor. This format wasn't just for the sake of trying something new – it was also an excellent way to make sure we all exposed ourselves to as many cards as possible, so we weren't confined to just those "favorite" cards from a first glance.
After a couple of weeks, a few of us made sure we hit those cards that still looked undertested – the zubera (then called "effigies"), the honden (then called "shrines"), and a few more. By the time we were done, there wasn't a corner of the set we hadn't poked our noses into.
The execution. Like many players would be, we were a bit overenthusiastic at first. We found ourselves focusing a bit much on actual rules wording on five to ten cards, and less on how a given card played no matter how we ruled on it. Once we relaxed a bit and let ourselves enjoy the cards a bit more, I believe we had more fun. Highlights from the testing:
Dave Hanson becomes the first Magic player to abuse the "crash-dragons" and the new legendary rule. Within 24 hours of getting these "new cards" (they showed up in an email about halfway through our testing), Dave had the white, blue, and black versions in a deck with four copies of Buried Alive, Patriarch's Bidding and…well, who cares what else. (Yes, we were also aware of Sneak Attack.) He could store and bring out either one each of blue, black, and white – or double them up and let the "legend rule" do the rest. It was devastating – not least because Yosei used to have an even more powerful tap-down effect than it has now. Changes: Yosei's ability is a bit more reasonable now.
Todd Petit and Bob Drosky become the first two fearless explorers to break the effigies and shrines. Like Lewis and Clark braving the Rocky Mountains, Drosky and Petit demonstrated in both limited and constructed multiplayer formats that these unassuming 1/2 zubera and matching legendary honden do have their uses. Sort of. Occasionally. When the light is right. Changes: the zubera are pretty much identical to when we last saw them. The honden have different (staggered) casting costs.
Paul Hemze borrows a buddy. "Borrow Buddy" was the playtest name for Blind with Anger. Every other game, Paul would look at an attacking creature, cry out "borrow buddy!" in his silliest voice, and then wreck some other poor guy who happened to have a decent non-legendary blocker available. Of course we knew this was good – it's the latest variant of Ray of Command – but the amazing frequency with which Paul pulled this off is still noteworthy. Changes: No change to this card from when we last saw it.
Joe Santos reverses some sands. During my preview of Reverse the Sands, I was unable to reveal that our group already had fairly intimate knowledge of this card. Joe loved it (then called "Vice Versa") and played it constantly. I hated it, and still hate it even though I'll probably use it myself. It's that kind of card. Changes: They bumped up the cost by 1. We didn't tell them to do this; but we're glad they did.
Todd Petit makes a three-fer deck. There was a potion involved, and a lot of stupid brokenness. I forget the rest and I'm glad; decks like Todd's really don't deserve even this much press. Changes: The card isn't in the set anymore. Let's all just hope they don't try this one again.
There were also plenty of snakes afoot (yay snakes!); and combinations of moonfolk with Azusa, Lost but Seeking; and a good number of samurai decks; and then lots of random existing decks where we only inserted one or two of the new cards for extra punch (e.g., Zo-Zu the Punisher into an Ankh of Mishra deck). I had a good thing going with Godo, Bandit Warlord (then called "Godo Mifune, Bandit Warlord…I assume the full name was too long for the final cardface) and his lovely maul. He was probably my favorite card, after the dragons, to survive the development stage. (There were two other cards that didn't make it; but I'm hopeful we'll see them later on in the set.)
So – Did We Like It? And Do We Now?
We did like the set okay back then, though we all agreed it appeared less eye-popping than Invasion or Mirrodin. We knew full well that not being able to see the artwork, flavor text, final card titles, and other aspects were keeping us from appreciating fully a set that was obviously based on flavor.
And I think (though I haven't formally polled everyone in the group) that we like the set better now. The changes they made were definitely in the right direction. And they've set themselves up beautifully for the next two sets.
When it came time for the final survey, I was both enthusiastic and honest. Here are excerpts of my email to Elaine and the development team:
[Flippers are] winners…[they] range from solid to excellent. Even if it sucks to figure out how to do this with artwork and such, please keep them in! (I assume that there will be an innovative card face involved, e.g.card split into thirds. If that's not the case, this may not be as cool as I think it is...but I'm optimistic!)…
[Legends are] the right note to hit for this set. You need to hit it MUCH harder…
[The dragons are] perhaps too good. Big effects when you "play from your hand" are easy to regulate – but graveyard-impact effects can happen in all sorts of cheap and easy ways. (Your new legend rule is one of them - two black dragons coming out at the same time from Patriarch's Bidding is bad, bad news!) And Sneak Attack can put all five in, which may or may not be a problem for you - but I can guarantee it will rocket to the head of most multiplayer groups…
Flavor is incredibly important to multiplayer. We all knew that already; but our group learned that in spades with this experience…. It's amazing how hard it is to follow the game when you can't reference artwork…
I also gave Development four fairly aggressive recommendations involving legends. Bottom line, I felt Wizards had already done a great deal of cool stuff – and could do even more. (I still do.) I don't know if they will adopt those recommendations or not before the block is done; so I won't go into any further detail. But it was so cool having their ear on this!
As for the group as a whole, we had a fine idea to allow "splicing" to happen on any player's arcane spells – what fun that would have been in multiplayer! – but of course, because splicing happens as you pay the cost of a spell, there's no way for this practically to work. So they scrapped it, and rightly so. I continue to hold out hope that there's a major mechanic or two out there that work fine in duels, but are even "more special" in multiplayer environments. And I have faith that as design and development find such mechanics, they'll use them.
What We Learned
Design & development works hard. We knew this already, of course. But it was helpful to be able to begin to quantify. By my estimate, our project took quite a bit of time: an average of eight players times ten playtest sessions at about six hours per session equals 480 man-hours of actual play with the cards. This doesn't count unscheduled "jam" sessions between just two or three of us, or the phone and email conversations we'd have with each other in between, or the filling out of surveys and other analysis we did.
Of course, we're just one playtest group. We can only speculate on how many other playtest groups, in however many other formats, development uses. And all of that farmed-out playtesting is just a fraction of what development does to analyze and refine the cards themselves. And all of that doesn't count the work design had already done, before we even got the stickers in hand.
And all of that doesn't count whatever work any or all of these people have now done on the next couple of sets, which need to fit thematically and structurally with Champions.
Occasionally on the message boards, I see folks who blame individual Wizards employees for "missing" how good a specific card is. While I respect customers' rights to give feedback, such criticism looks a whole lot more useful to me when it comes from someone who seems to grasp what these people do. Mirrodin block went by with something over 600 cards printed. Skullclamp's power was apparently an oversight. There are certainly industries where 99.83% accuracy is not good enough; but collectible card games probably isn't one of them. I'll happily take partial credit if Kamigawa achieves the same rate.
It's fun, but taxing, to stay quiet. Almost immediately after we finished playtesting, we got three new members in the group. We could not talk about our testing with them. Fortunately, we (actually, Dave) had the foresight to set up a separate email group just for testing, so our regular group chatter wouldn't get infected with talk about Champions. So all we had to do as group members is not tell Laura Mills, a Star City Games writer, everything we knew about the upcoming set. Maddening, yes. Unnecessary, probably. (Laura's not the type to leak sets.) But also what we signed up for – and once we accepted that, we found it was kinda fun to keep her and the others in the dark. Of course, we may never get the chance to playtest again as a group. (Sorry, Laura!)
Bruxor is worried. And we're very worried about Bruxor. I'm pretty sure I can't say anything more about this.
We had an impact. Or maybe not. We cannot say for sure what direct effects we had on this set. Our egos would love to take some credit for alerting development to the potential problems with an earlier version of Brothers Yamazaki (a.k.a. "Stanggmaker"). We constructed one or two really stupid decks that may have had something to do with the disappearance of cards I can't tell you about. We'd like to think Cruel Deceiver (a.k.a. "The Black Bluffer") has a better ability because we suggested the bluffers could have more punch; that Yosei, the Morning Star is fun now instead of ridiculously unfun because we pointed out an earlier version's amazing power level in group; that the five Honden have staggered mana costs in allied colors because we found earlier versions difficult to work together; and that the set has a few more spirits and arcane spells because we enthused about the flavor there.
But in all scientific honesty, given the amount of testing development does on its own and the large number of other people they depend upon for opinions, I imagine development (and/or another playtest group) would have found and corrected these problems without our help. Magic isn't a game made by a single guy pulling a lever in a back room. It takes a community of people, and a great deal of coordination, to get a set from "good idea" to "good execution". We're just glad we carried our own weight.
So You Want To Playtest A Set...?
If you want to go through the same experience our group did, I have some good news and some bad news. The good news is, Wizards has all of their playtesting needs met for the forseeable future. That's really good, efficient corporate practice! But, um, now you can guess the bad news.
I'm not in any position to help my readers here. I love you all; but I cannot relay, advise, endorse, or otherwise influence your group's quest to playtest. Thanks for understanding that.
I hope you readers do get the chance some day to influence a game, product, or service you like. It doesn't have to be Magic! Seek and ye shall find. I was at the new Green Mill restaurant in Hastings, Minnesota a few weeks ago as a customer on training day. Free food - you just have to put up with slow service and fill out a survey! Good deal. I found the experience nearly as rewarding as playtesting Champions. (The fried walleye was just that good.) If you look around your neighborhood, you might be surprised at what you can find.
Many thanks to Wizards for a fun opportunity!
Anthony cannot provide deck help. He is at an undisclosed location, testing an undisclosed product, with an undisclosed group of colleagues. Or rabbits. Or space-men. We simply can't say.