Ten Ton Hammer - Taking MMO Gamers to the Next Level!

Without a doubt, the beta testing experience has changed since the beginning of the MMO industry. In a recent article that Ten Ton Hammer released to the public, the staff compiled a variety of quotes from present and past MMO developers while putting these side-by-side with thoughts from the Ten Ton Hammer premium members. Not wanting to miss the action, the BioWare Austin developers working on Star Wars: The Old Republic sent over their own thoughts on beta testing. Straight from the mind of Co-Studio VP, Richard Vogel, this Q&A gives players a hint of what they might expect from the upcoming SW:TOR beta test.

---

Ten Ton Hammer: Since Ultima Online's original beta testing period, the concept of the beta test seems to have shifted toward pure marketing tactics rather than any actual testing by the player base. Do you think that beta tests will continue along this trend until we no longer see any form of open beta "test" any longer? Besides stress testing, is there any benefit to having a large closed beta or true open beta?

Richard Vogel: The Open Beta test has changed.  It is more of a “try before you buy” marketing push as well as a soft launch.  You really don’t get much data from “Open” tests.   It is really just making sure everything holds under stress and allows us to get people in the game early vs. everyone getting into the game on launch day.  The reality of beta testing is that most people don’t play the game the way they do when it is launched.  If there are exploits, they usually keep them to themselves to exploit later when the game goes live.  

Ten Ton Hammer: After the release of Tabula Rasa, Richard Garriott alluded to the fact that the TR beta test actually hurt the game's chances of success. On the other hand, there were a few instances in recent games where hardware factors kept players out of the game, but when the game released it sold very well. How do developers know if a beta will hurt their retail sales? When is it appropriate to let a lot of gamers see your upcoming game? Does it hurt a company to never have a truly public display before a game launches?

Vogel: One thing to do is survey the users.  If 75 percent or more tell us they will recommend the game to someone else, that is a very good sign.  Another good sign is how much and how often people are playing the game.  It they are playing a lot and they speak positively about their experience that is also a good sign.  It is very important to ship a quality, fun product.  You should know this by the number of open bugs, their severity, and survey data that is collected in the game.  What happens most of the time is that decisions are made to ship the game before it is ready. You are only given one shot at this.  If you have a bad player experience at launch and through the first few months, it sets the brand value forever.  That is why games don’t usually recover from a bad launch.   

Ten Ton Hammer: When is it appropriate to let a lot of gamers see your upcoming game? Does it hurt a company to never have a truly public display before a game launches?

Vogel: We feel that limiting the [number of] beta testers is a good thing.  We get better data with fewer, more dedicated players and we are able to focus test specific areas of the game.  You can’t do that with tens of thousands of players.  It is a coordination issue.  It is good, however, to allow enough players into the game to make sure it can handle the load under stress.  

About 4 weeks before launch is a good time to open the game up to consumers.  If you have done your job right, you will know if the game is ready before you open beta.  If not, you don’t open it until it is.  It doesn’t hurt a company at all to open the game up to players four weeks before launch.   It allows them to try before they buy and it allows us to make sure the game holds under pressure as well as providing a rolling launch so day one is not so crowded.