As Ten Ton Hammer continued to receive
responses to our questions, it
became readily apparent that almost all of the developers agree that
some sort of beta is necessary. Whether its for marketing or stress
testing or bug squashing, not having a beta isn't an option. They also
agreed that if a game is good, and good when it hits a public beta, the
game should sell no matter what. Here's what Jeff Hickman, executive
producer of Warhammer Online, had to say about our beta questions:
I truly believe that beta should
NEVER hurt the sales of any good
game. Beta not only helps the developers test the game, but
also gives the players insight into how great the game is and helps
build buzz amongst the community. Unless you are concerned
that players won’t actually *like* your game, then a well run
beta is a must in my mind. *winks* Letting as many players as possible
give feedback on your game and also getting as many as possible to talk
within the community about your game is very important.
Ten Ton Hammer also heard back from Turbine's Craig Alexander, VP of
Product Development. His comments mainly focus on
Lord of the Rings Online,
but Turbine has
had plenty of experience dealing with rougher beta tests and launches,
such as
those conducted for
Asheron's
Call 2 and
Dungeons
and Dragons Online. Compared to Mr. Hickman, the
Turbine developer drew some more concise lines around how a developer
can keep players from weighing too much of their final purchase
decision on the results of their beta experience.
The only time a Beta can hurt a
game is if the game is not ready when it enters Beta. It all
comes down to managing player expectations in terms of what they can
expect in Beta, and how it does/does not relate to the game they will
buy at launch. Clear, constant communication is the key, as
well as demonstrating a facility to understand feedback, and respond
effectively and appropriately. As usual, it all comes down to
giving the game the time is needs to become a great game and truly
ready for launch. As long as players see that commitment from
you, they will be flexible in their assessments during Beta.
Alexander also wanted to point out that:
There is significant value to
Beta for a number of reasons. The scope of Beta depends on
the scope of the release. In each case, Beta is valuable for
load testing and stress testing of content as well as new
systems. Turbine’s platform is battle-tested and
very stable overall, so our load testing is really focused on issues
such as:
· How will
new systems behave under load
· How well
will new content handle “land rush” of players
jumping to new content all at once
· How well
will the design function with large populations
For this kind of Beta,
the larger the group online at once the better.
For
more significant launches like our original launch of Shadows of Angmar
or Mines of Moria, we have extended Beta tests to learn about how
players will perceive the new systems and content to see how we can
make the game better before official launch. This was
instrumental in the quality of what we launched with Shadows of Angmar
and Moria. Many valuable lessons are learned during Beta that
we have created process and technology to capture and then leverage
into our game.
Even after all that they've endured, it's obvious that the MMO
community continues to want to beta
test. But do they expect these games to be true beta tests or marketing
campaigns? The responses we received from Ten Ton Hammer were very
telling.
"Betas are obviously there to test the game for serious issues and bugs
and to help eradicate them from the game, but over the years the hype
for certain games has grown to substantial proportions because said
games were in beta," Beerkeg said. "10 years ago there would have been
no news about a game in beta until it had been released, and now we get
new news every other week about said games that are still in beta."
"Beta tests have always been, without a doubt, about marketing as much
as they have been about actual testing," centrik concluded. "It just is
more evident today. However, from a tester point of view, I think there
is a growing number of people who just want to try games before they
come out."
Out of all the responses for that question from the Ten Ton Hammer
members, our longtime member Annatar had the most passionate reply. His
thoughts were clear and concise and obviously well thought out from his
years of MMO experience.
"It is disappointing to say, but testing has become a bit of a joke,
and I feel that the current crop of recent games are a reflection of
that," he said. "So many games are being released incomplete (as far as
hyped features go) and containing issues that should have been picked
up and resolved during the closed beta phase at the latest, but this
isn't happening."
"Perhaps developers should expect a bit more out of testers?" he
continued. "I don't think the whole beta process would be hurt by
developers outlining what is expected of testers to maintain their
position with the testing 'team', and booting those that don't submit
bug reports or feedback to make way for others who may be more inclined
to take the process seriously, and not treat it as a free trial of the
game."
"Market the game outside of the testing process, don't try and sell it
to people via the testing phase when you need those testers to be
making the product one worth selling," he finished."