Ten Ton Hammer - Taking MMO Gamers to the Next Level!

When Ten Ton Hammer initially decided to craft this article, we sent out a list of questions to a number of prominent developer in the MMO industry. One of our most intriguing answers came from former EverQuest II senior producer and creative director, Scott Hartsman. His answer hit squarely on the money/beta testing issue, and rather than chopping his thoughts into pieces, here’s what he had to say about beta testing and money:

For the AAA, eight- and nine- figure budget extravaganzas, big betas aren't going away any time soon.  What companies get out of them has shifted over time, but they remain an important part of getting a game out the door.

As product cost and complexity have increased, the emphasis of beta has indeed shifted toward toward marketing and load testing both your gameplay and operational systems.  However, those are still critical activities in the high-budget, launch-big-or-die model.   (That model has many weaknesses, but that's an entire topic in itself.)

The reason this happened is simple - It's about the money.  Let's say you're a AAA game with 3-4 years of time and money invested, enough money to support a large team having worked on it for that long.  Games like this frequently need to go for years before enough pieces come together before you can start making decisions about what's fun and what isn't.

By the time beta begins, you've made decision after decision that have compounded on each other.  Your assumptions' assumptions' have assumptions about what your game is.  The whole product, systems, content, operations, marketing, PR, community ramp, you name it -- is built upon them.  Changing core assumptions about the product itself is unlikely to be possible without significant delays, costing progressively more money per month.  (Remember, the months toward the end of the dev cycle are the most expensive ones by far.)

The game is, for the most part, what it is.  You're capable of making shifts, but the more complex the game, the more minor the shifts you can make with any confidence.  If assumptions that you made years ago turn out to be wrong, you're left to scramble, or in most cases, do your best to ameliorate the now-certain fallout.

If you haven't verified your gameplay at the point of having a beta, you've already left your fate to chance.  (This is, of course, all presuming that your game has passed the technical bar in terms of stability, which is all too often not the case.  And, again, is another flaw with the launch-big-or-die model.)

As budgets go up and schedules get longer, the model is growing more and more analogous to movies.  If anything, people can see what goes on with blockbuster movie releases and draw certain comparisons.

No big beta?  With a quality product at this stage in the industry's evolution the negatives almost never outweigh the positives.

Unlike movies, seldom are there a half dozen launches competing for attention in the same month, much less the same week, where movies might have some competitive advantage to keeping secrets this late in the game.  MMOs differ from movies in that they're a long term time investment.  The pattern of hype generation is different.

The way MMOs are most similar to movies, exploding costs aside, is that if you don't see an advance reviewer screening for a movie:  Something Has Gone Terribly Wrong.  Bad news is being kept out of the market in hopes of keeping day-one sales high.

The same can be said for lack of betas, repeatedly late betas, or overly-restrictive betas for MMOs.

The company knows that early sales are now where the bulk of the money is going to come from, instead of huge usage numbers over time, and it's doing what it needs to -- preserving those precious day one revenues, since it could well need that money to survive.