Welcome Guest:

Other MMO Coverage

322 MMOGs and counting...

The Rigors of Beta Testing: The Past, Present, and Future (Page 8)

Page:123456789

Posted April 16th, 2009 by Cody Bye

As Ten Ton Hammer continued to receive responses to our questions, it became readily apparent that almost all of the developers agree that some sort of beta is necessary. Whether its for marketing or stress testing or bug squashing, not having a beta isn't an option. They also agreed that if a game is good, and good when it hits a public beta, the game should sell no matter what. Here's what Jeff Hickman, executive producer of Warhammer Online, had to say about our beta questions:

I truly believe that beta should NEVER hurt the sales of any good game.  Beta not only helps the developers test the game, but also gives the players insight into how great the game is and helps build buzz amongst the community.  Unless you are concerned that players won’t actually *like* your game, then a well run beta is a must in my mind. *winks* Letting as many players as possible give feedback on your game and also getting as many as possible to talk within the community about your game is very important.

Ten Ton Hammer also heard back from Turbine's Craig Alexander, VP of Product Development. His comments mainly focus on Lord of the Rings Online, but Turbine has had plenty of experience dealing with rougher beta tests and launches, such as those conducted for Asheron's Call 2 and Dungeons and Dragons Online. Compared to Mr. Hickman, the Turbine developer drew some more concise lines around how a developer can keep players from weighing too much of their final purchase decision on the results of their beta experience.

The only time a Beta can hurt a game is if the game is not ready when it enters Beta.  It all comes down to managing player expectations in terms of what they can expect in Beta, and how it does/does not relate to the game they will buy at launch.  Clear, constant communication is the key, as well as demonstrating a facility to understand feedback, and respond effectively and appropriately.  As usual, it all comes down to giving the game the time is needs to become a great game and truly ready for launch.  As long as players see that commitment from you, they will be flexible in their assessments during Beta.

Alexander also wanted to point out that:

There is significant value to Beta for a number of reasons.  The scope of Beta depends on the scope of the release.  In each case, Beta is valuable for load testing and stress testing of content as well as new systems.  Turbine’s platform is battle-tested and very stable overall, so our load testing is really focused on issues such as:

· How will new systems behave under load

· How well will new content handle “land rush” of players jumping to new content all at once

· How well will the design function with large populations

For this kind of Beta, the larger the group online at once the better.

For more significant launches like our original launch of Shadows of Angmar or Mines of Moria, we have extended Beta tests to learn about how players will perceive the new systems and content to see how we can make the game better before official launch.  This was instrumental in the quality of what we launched with Shadows of Angmar and Moria.  Many valuable lessons are learned during Beta that we have created process and technology to capture and then leverage into our game. 

Even after all that they've endured, it's obvious that the MMO community continues to want to beta test. But do they expect these games to be true beta tests or marketing campaigns? The responses we received from Ten Ton Hammer were very telling.

"Betas are obviously there to test the game for serious issues and bugs and to help eradicate them from the game, but over the years the hype for certain games has grown to substantial proportions because said games were in beta," Beerkeg said. "10 years ago there would have been no news about a game in beta until it had been released, and now we get new news every other week about said games that are still in beta."

"Beta tests have always been, without a doubt, about marketing as much as they have been about actual testing," centrik concluded. "It just is more evident today. However, from a tester point of view, I think there is a growing number of people who just want to try games before they come out."

Out of all the responses for that question from the Ten Ton Hammer members, our longtime member Annatar had the most passionate reply. His thoughts were clear and concise and obviously well thought out from his years of MMO experience.

"It is disappointing to say, but testing has become a bit of a joke, and I feel that the current crop of recent games are a reflection of that," he said. "So many games are being released incomplete (as far as hyped features go) and containing issues that should have been picked up and resolved during the closed beta phase at the latest, but this isn't happening."

"Perhaps developers should expect a bit more out of testers?" he continued. "I don't think the whole beta process would be hurt by developers outlining what is expected of testers to maintain their position with the testing 'team', and booting those that don't submit bug reports or feedback to make way for others who may be more inclined to take the process seriously, and not treat it as a free trial of the game."

"Market the game outside of the testing process, don't try and sell it to people via the testing phase when you need those testers to be making the product one worth selling," he finished."

Join Now!
  • Azailfortis,
  • ericjohn19,
  • arkantos78901,
  • ThermiteReaction,
  • and BaronIveagh
recently became Premium Members and get first access to beta keys, contests, exclusive interviews and video, and can turn off ads. Why not join them?