On the heels of Global Agenda's first weekend closed beta test session, dubbed CBT 1a, we caught up with Executive Producer Todd Harris for a debriefing. After an update on what was tested, how the beta test weekend went, and the kinds of feedback received, what followed was a surprisingly in-depth discussion about the role of player vs. environment (PvE) play in Global Agenda, a game more known for the depth of its player-vs-player (PvP) side. We finished up with how feedback from CBT 1a went so far as to alter the course of Global Agenda's development, albeit slightly, heading into CBT 1b later this month.

Ten Ton Hammer: So this episode is all about beta feedback you’ve been getting from the first closed beta weekend. How did CBT 1a go?

Todd Harris: It went really, really well. We turned on the servers early just to give the folks a nice surprise and then went through late Sunday night and it really met or exceeded all of our expectations. We’re really pleased with how it turned out.

Ten Ton Hammer: What kind of feedback have you been getting? Has the 1-20 game turned out to be pretty much what you expected in actual playtesting?

Todd: Yea. As you said, the goal of this really around bringing brand new players to the game, having them go through levels one through twenty, and having them test out whether the progression itself is fun, whether combat at the lower levels is fun (since we’ve been doing our testing at the higher levels primarily up until now), and to test the function of some of our economy systems: loot drops, crafting, auction house. Those were really new systems, as we talked about last time. Fourth, we wanted to test server operations, GM tools... all the things that come into play when you let in a thousand people all at once. So really - very early scalability testing.

Functional testing of Global Agenda's economic systems was part of CBT 1a. The result? All systems go.

On all those points, we got really good feedback - a lot of positive comments, in terms of fun of the combat, character progression, economy. Again, it was more of a functional test for us; we’re going to be modifying many of the particulars as far as loot drop percentages, crafting recipes, some of the pricing involved with items. Getting those balanced wasn’t the focus at this point; it was just to see that these systems were functional. And they were. We had folks crafting items for loot and putting them on the auction house and bidding on them, so that was exciting to see.

Also, there were some really good lessons learned on the server infrastructure side. On Friday night, when we let in a bunch of people at once, we did have to do some server tuning. Things were a little hairy on the first night; I think everyone expected that with the first day, and I think it was a smart move on our part to start the beta on Friday when we had all hands here in the office to respond really quickly during the day and into the night.

Once the tuning was done, we had a really good experience on Saturday and Sunday, so much so that on Sunday we actually added in our alpha group just to get more volume on the server to make sure we’d resolved any issues. And that went really well.

Ten Ton Hammer: Where were your suspicions confirmed? Were there any big surprises from CBT1a?

Todd: I would say that the things we kind of expected were confirmed were: a) PvP is indeed fun at all levels, b) these graphics are really cool, it’s a cool sci-fi look but it’s not the typical post-apocalyptic look. People liked the clean style and progression. All those things were things that we were expecting and were confirmed.

Things that were maybe pleasant surprises were the really positive response to group PvE, specifically. We knew people were going to love the PvP because we’ve been testing that ourselves for some time. The PvE is newer for us because we worked backwards from the devices that were balanced for PvP and then adopted the PvE systems around that. And as you guys know, we just introduced that fairly recently at E3. We had a lot of the building blocks in place over the past 4 years but we put it together for missions around the E3 timeframe. Since people really responded well to group PvE, that’s where we’re going to be working on more content in the short term.

Another pleasant surprise was the very enthusiastic feedback to the in-game voice. We do have in-game voice-over-IP and historically there’s been some debate I guess over how much players would want to use that, whether it destroys role-playing, etc. Certainly for our type of game, where it’s very quick paced and there is a big focus on combat and tactics and well-oiled teamwork required, voices adds a lot. The players responded well to it being built in, and we also got a lot of positive comments on the strong voice quality. Many said it was the best voice quality they’d experienced in an MMO, so that was good.

Global Agenda CBT players especially enjoyed the co-op PvE missions in the 1-20 game.

The third pleasant surprise was just the enthusiasm of the beta testing group. We thought we made good decisions on who we let in and of course you can’t let everyone in in the first batch, but really good enthusiasm as far as just a lot of play test hours. The number of people that packed in more than 40 hours in just these three days was exceptional - a lot of folks doing all-night marathons. I hopped on Saturday morning and there were folks that hadn’t gotten off yet that started playing Friday. When you only have three days, you know, they wanted to get all the way up to level 19, which is what we ended up capping it at. In fact, there were a number of folks that took all four character classes up to 19. All of those were very good surprises for us.

Ten Ton Hammer: Are you seeing players gravitate toward one or two classes?

Todd: No, we really didn’t. In fact, I just talked with Stew Chisam (VP of Game Operations) to get the latest stats from Sunday night. We have four classes and three specializations / talent trees underneath that, but it is close to 25% [total participation] for each of those classes. There’s a slight bias toward recon, it might have been 27% recon - a very slight bit above one quarter of the characters created were recon, which we always expect.

Ten Ton Hammer: Sure - everyone wants to be a sniper at first.

Todd: Yea, it’s one of the more appealing ones to start with. But, really balanced across the board in terms of what players want to try. That actually matches what we saw in our alpha test. Which is good because in beta, players only had a weekend, and assuming they had the time to put in, a lot of folks did try to sample multiple classes. But that also matches what we saw in alpha where the community’s been playing for literally months and everyone has a main character.

Ten Ton Hammer: You’ve focused on PvP for much of alpha. Has PvP taught you how to design PvE encounters? Are you integrating player behavior into the AI for the co-op missions?

Todd: I think the best thing for us, as far as starting from PvP, was the general level of balance between those classes and philosophically, at least, we made the decision we were going to balance for team based PvP first. That continues to be what we tweak balance for. On the PvE side, we’re able to make the AI so it’s just fun and challenging, based on those classes and the devices that worked well in PvP.

And actually I’d say that our evolution has been a little bit of the opposite, in that our very early PvE was more of a simulation of PvP. The AI system that we have did a pretty good job at that, but it also didn’t provide a unique enough experience from PvP, so if folks wanted to do PvP, they would just do PvP. So we’ve actually have shifted away from opponents that mirror player classes and have moved to more distinct enemies that are just fun and more different.

So in our PvE, players will encounter a few opponents that are like players but even more specialized, so really really good recon melee stealth guy that’ll come from nowhere and take you out if you’re not looking in the right direction.

But then we also have completely different bots, such as our scanner bot when you go into a commonwealth lab and there’s a bot that has a detection ray that it’s casting around the room. If it detects one of the players while it’s active and scanning, then reinforcement androids show up. And when those reinforcement androids show up, they’re dispensed with fairly easily, especially at the low levels. They’re fodder, you toss a few sticky grenades in their direction and they blow up in a very satisfying manner, ragdolling all over the place. So that’s just a completely different mechanic that’s consistent with all the devices but provides something unique in PvE.

Global Agenda class balance seems about perfect in early closed beta testing.

So our evolution has been more towards that. Specialized bots that are really good and provide excellent team-based PvE, using the same devices as PvP but not necessarily yielding the same gameplay experience.

Ten Ton Hammer: I can’t imagine that there would be too many players with this mindset given the kind of game Global Agenda is, but if someone wants to go fully PvE, what are their options in Global Agenda? What are they going to be missing out on as the game progresses?

Todd: At this point, our intention is that a PvE player will be able to contribute toward their agency making progress on the campaign map. The concept of resources gained through PvE could be used toward crafting implants that would generally improving my character or could be put up on the auction house and earning credits. Some subset of those actions could go toward my agency doing better in the campaign. We’ll certainly test to make sure that that concept sticks, but right now that’s a pretty important concept for us, that PvE players would be able to contribute toward the campaign.

Ten Ton Hammer: Are there different abilities for PvE and PvP? Do devices and abilities work roughly the same for PvP and PvE encounters?

Todd: At this point there’s no difference. Just in terms of the unlock progression - the sequence of gaining new items, new implants, and skill points - this beta activity validated our framework there. We got a lot of feedback that for 1-20, this was fun, they could feel their character progressing, etc. Because we started with devices that work for PvP and then making PvE fun around that, that’s allowed us to keep the device set the same, for now.

Just as an example, we came up with the stealth mechanic for recon, and stealth is something that can be pretty tricky to get right in a PvP game. Our approach was to have that work well and feel good for the recon player but not be too overpowered. We’ve done thousands and thousands of PvP maps to confirm that mechanic and make sure it works well, so now it’s easier for us to have AI opponents that - now that the way that devices works is a given - adapt to that in a way that’s fun.

Towards your question about games having PvP and PvE and having one not feel tacked-on, I think it is a challenge. At least for us, starting with weapon balance for PvP (since we knew that would be a big part of the game) and then adapting the enemies to that has turned out well.

Ten Ton Hammer: So it really sounds like nothing occurred over CBT 1a that didn’t validate everything you thought about class and item balance in the high level game, so that’s really good news.

Todd: Yea it is, and for folks that might be interested in how we look at feedback from beta, and since I’m in the thick of it now, I’ll share with you the things that we’re looking at. There are four main categories of things: one is forums. We do have very active forums, you see where people have passion, but sometimes that can be a vocal minority and not necessarily representative. So we do look there to see where people have passion, but that’s not our only source.

Additional PvE content moved up the list for CBT 1b.

Number two is surveys. For this round, we did a pretty comprehensive email survey that went to all of our players, and that certainly gives us a representative sample across the whole playerbase. Third, as developers play alongside the players - sometimes with Hi-Rez on their name, sometimes under aliases - they’ll gather feedback. That way they’re playing in matches with folks, we can see what people are commenting on in the chat and able to see the really candid feedback that you don’t get even in an anonymous survey. Fourth, we look at all the persistent stats that we track. Obviously there we can see trends, so things that may not be apparent in a single match show up here, so class balance, item balance, kill ratio, and things like that are tracked here. And across all of those, we were very pleased with what we saw.

To me, one of the main metrics that we look at on the survey is whether or not a player would recommend this game to a friend. Even just after this three day activity, the surveys we’ve gotten back are over 85% saying they’d recommend Global Agenda to a friend. Then the vast majority of the other 10-15% are kind of in the “I need to play more to make a decision” category. “Would not recommend” is really low, under 2%. So that was above what I was expecting from the group and was really encouraging. Obviously we’ll work hard to keep those folks happy and even increase that number as we move forward.

Ten Ton Hammer: Can you talk about any areas that you’re specifically working on as a result of the feedback you received?

Todd: For one thing, as far as our next phase, we’ll have a follow-up phase of testing around character progression - some of the higher levels that the community didn’t get to this time. That’ll be before the end of August, but we want to give ourselves time to work on the feedback items and some of the other items that were already in the queue, and give the closed beta testers time to rest up for another marathon weekend.

From now till then, there’s a hundred little things, but the big things would be some additional group PvE content. People really did enjoy that, so we want to expose a little bit more of that content. Some more polish around the new player introduction. As I mentioned we were very focused on the mechanics there - what order things unlocked, the speed that things unlock and that sort of thing - so just additional polish around grabbing folks in that first hour. That means additional art and music and really just polish items around those missions - really expose people to the world, story, and context. Then we will be continuing to work on the campaign, which won’t be introduced for testing in that next phase - that won’t come until CBT 2 - but that’s a big focus for us as well.

Ten Ton Hammer: Thanks again for your time, Todd!