Exclusive Interview with Binky!

Gods & Heroes: Beta Players Vs. Beta Testers

Tony "RadarX" Jones

Everyone has been following Gods and Heroes anticipating that holy of all grails, the beta. For those of you lucky enough to get into the Core Testing Beta, you are already helping to shape the game. The rest of us sit back and wait for the next wave. Ten Ton Hammer caught up with Chris "Binky" Launius, Community Manager for Gods and Heroes, and quizzed him on the beta process.

Ten Ton Hammer: Beta testing has become more high profile in recent years.  Has the increase in beta testers lessened the quality of testing? 

Chris "Binky" Launius: I don't think that it has lessened the quality of beta testers as much as it has given us a much wider variety of play styles to get feedback from.  Sure there are some people who just want to play the game and have no intention of ever giving you feedback.  Those players can also be very helpful. The more people you can get into a pre-launch product, the more accurately the other beta testers who are hardcore and focused on finding bugs and balance issues can test the game and give valuable and relevant feedback.

Ten Ton Hammer: There are beta players, and beta testers.  What would you say is the difference between them?  Do they both have a role to play? 

Chris "Binky" Launius: A beta tester is a person who is going to play the game and try every single thing there is to do in the game.  They will play classes they normally wouldn't play, and they will typically spend a lot of time trying to intentionally break the game.  These are the people who will submit bug reports and feedback to the development team and become very active in communicating with the development team.

A beta player is a person who just wants to play a pre-release game, either because they are bored with their current game(s), don't like paying a monthly fee but want to play the game anyways, or "collect" beta tester badges as a status symbol.  These players will most likely never submit a bug or feedback report and will often only communicate with the dev team or support staff when they have an issue with their character such as a lost item or whatnot.

Both of these gamers absolutely have a role to play in beta testing.  The tester will help us find bugs, figure out balance, and work with us to release a polished product.  The player will actually help the tester and developer get a much more accurate feel for how certain design decisions will play out when a large number of players   are actually playing the game as they would play it when the game is live.  Beta testers often do not "play" the beta version of the game the same as they would a live game, so having those players actually playing the game is great.

Ten Ton Hammer: Tons of people have been known to lie on the beta application.  How do you separate the honest ones from the rest of the mob? 

Chris "Binky" Launius: After organizing and running beta and play tests for a while--over 10 years in my case--you can really just tell.  One of the things we look for the most in an application is enthusiasm and knowledge of the product.  Granted, some people won't know much about the product since beta announcements seem to be the best tool for building awareness about an upcoming product, so that is often where the enthusiasm and excitement come into play.  If an applicant is going to put effort into an application, they will most likely put at least a little effort in testing and providing feedback.

Also, another thing I am personally a stickler for is following directions.  If I put out a call for applicants asking for X, Y, and Z in an email and the applicant send me P, L, and X, that raises a red flag for me.  I will usually try to get the correct information because maybe their buddy told them to apply, but if they don't give me X, Y, and Z the second go round…they are probably not going to be invited to join the play test.

Ten Ton Hammer: Obviously you can track anything that goes on in game, but do you track beta testers, and gauge their involvement?  Do you ever boot people for not participating? 

Chris "Binky" Launius: We do track every single click a user makes while playing the game.  Our platform has some of the most robust tracking tools I have seen in my 12 years in the game industry.  For the most part we use the data to help us with some design decisions such as placement of specific NPCs, usability of the interface, and balance among the classes.

We do analyze all of the tracing reports we get and they also help us ensure that the beta tester are covering all of the areas and features. This helps us give better guidelines for the play test.

We typically try not to boot anyone from the play test.  If a character is acting inappropriately we will try to work with them to ensure they are following the guidelines.  If they continue to act inappropriately we will take further action as needed.

The one exception to that rule is if a user breaks any portions of an NDA we may have in place.  That will result in an immediate ejection from the play test.

Ten Ton Hammer: Other than hardware specifications, what are you looking for in a good beta tester? 

Chris "Binky" Launius: The main thing we look for in a play tester is someone who wants to help us create and launch a fun, balanced, and polished game.  Someone who will give us constructive, clear, and well-written feedback and tell us when something is bad as well as when something is good.  Experience in MMOs is good, but sometimes, especially early on, it is good to get people who are new to the online world, because sometimes being "1337" gamers, we forget how daunting it can be for someone to the genre, so we try to get a good mix of experience levels.

Ten Ton Hammer: You recently announced the beta for "core testers."  Explain what "core testers" means? 

Chris "Binky" Launius: The Core Tester program is something that we recently kicked off with 50 people hand selected from our community, bigger guilds, and other online outlets.  These testers were let into the game before we actually went to beta.  These people represent a wide variety of play styles and backgrounds and they will be the "first line of defense" for new publishes before we push it to a wider audience.  This program is a way to get feedback on playability and overall fun value of the game.  That is the thing we are trying to achieve more than anything else: FUN.

This program has been hugely successful thus far and we are looking to expand it by another 300 people.  With a smaller group of people, we are able to work very closely with them and get very granular feedback.  This group of people will be with us from pre-beta through launch, chronicling their experience.  This will not only help Gods & Heroes , but will also help us build better practices and procedures for Star Trek Online and any other projects we tackle in the future.

Ten Ton Hammer: How will the beta be structured?  Will there be a number of phases?  Will larger groups be brought in? 

Chris "Binky" Launius: There will be many phases of Beta for Gods & Heroes , starting with smaller groups of 200 - 500 people at a time and going all the way up to as many as can fit on our server at a time. 

We recently included a VIP Beta Code in the inaugural issue of Massive Magazine which will give you a chance to get in the first stage of large scale testing.  I would encourage everyone to get one of these codes, create a Perpetual Platform Account, and apply the code.  Even though you may not get a chance to get into the beta, that doesn't mean we don't have some cool things in store for people who tried to get into the beta. *hint hint wink wink*

Ten Ton Hammer: How much of beta admission can be attributed to "random luck."   Is there any "drawing" or "picking" involved?   

Chris "Binky" Launius: For a few of the smaller stages we have been keeping a list of people in our forums that are active and have good, constructive posts.  It's not a post count thing we are looking for, but a quality of post we are looking for.  We will be inviting these people personally to participate.  In some of the bigger stages we will just randomly pull from all of the applicants or registered accounts.

Ten Ton Hammer: What do you find the most frustrating about the beta selection process?  What do you find the easiest?

Chris "Binky" Launius: The hardest thing is just going through all of the emails.  I try to read and sort them all which can take time.  Currently my inbox is sitting at over 3,000 unread beta applications just for the Core Tester call.

The other thing that is hard for me personally is not selecting people.  I don't want to make people feel bad for not getting selected, because I know how that can make a person feel, especially if the person is super excited about the game. But we have to make those decisions and do what is best for the overall game experience.

There you have it folks! Straight from Binky himself. Get yourself signed up, and stay tuned to Ten Ton Hammer for more exciting news about Gods and Heroes.

Also be sure to check out our exclusive Screenshots here!