Performance Showdown - Vista vs. Windows 7 in MMO Gaming
Windows 7 launched last month, and now there's a question on consumer's minds. "Is it worth it?" The answer most people will give is: "It depends
on what you use your computer for." I'm guessing since you're here at Ten Ton Hammer, you're like me and
play MMO games. So that's what we tested. Is Windows 7
better than Windows Vista for playing MMOGs? For more technical theory
on the upgrades to the operating system, check out Dalmarus' Windows 7, Gaming, and
Why You Should Care.
The way this works is pretty simple. I booted up a bunch of MMOGs on my PC when it was running Vista (64-bit) and I took simple frame rate samples from 4 resolutions: 1024x768 windowed; 1024x768 full screen; 1920x1200 windowed; and 1920x1200 full screen. Each test was run for precisely 60 seconds. I recorded the minimum FPS (frames per second), the maximum, and the average. I repeated the test three times in each resolution to help eliminate any sort of squirrely results.
Then I upgraded the exact same PC with Windows 7 (64-bit). This is of particular importance. I didn't run the tests on different computers running the same hardware - I ran them on the exact same PC, first with Vista, and then with Win7. Any background apps I had running in Vista were also running on Windows 7. The only difference, aside from the OS itself, were a couple of different drivers to support Windows 7. All drivers were updated prior to the testing on both operating systems.
I tested nine MMOGs in hopes to have enough samples to draw some conclusions. I tried to take the readings from the busiest areas of the games I could come across. In some games, there either simply wasn't a large enough population for it to matter much, or I couldn't get to the areas I wanted to go to. In either case, the variations serve as the exceptions to prove the rule. (Yeah, it was deliberate. Mmhmm.) The following is a list of games I tested, and the areas I tested them in.
Aion: Sanctum. This is the main city hub for Elyos, and probably the busiest place you'll find on the Elyos side. I took the benchmarks at the bank, although the crafting area would have also been suitable.
Age of Conan: Old Tarantia. It's been a little while since I've played AoC, but I headed to the brokers in Old Tarantia. It wasn't as crawling with people as some other games, but there was enough hustle and bustle to get realistic results.
Champions Online: Desert Crisis Powerhouse. This was probably the major exception to a "busy area." I initially went here because I remembered the powerhouses as being particularly busy at launch. I didn't take into account that the game launched a couple of months ago now, so the powerhouse was pretty empty. 12 people in a small, otherwise empty, instance made for some high framerates.
Dungeons & Dragons Online: Stormreach Harbor. Since DDO has gone F2P, there are always people in the Harbor. This was a good test.
EverQuest 2: Lavastorm Mountains. Why would I go there, and not say, Qeynos? Quite simply because the shard quest turnins are there, as well as the zone being a meeting point for some major raids, at least on the Crushbone server. Population was good.
Lord of the Rings Online: Egladil. This is the entry point for the upcoming Siege of Mirkwood expansion. Since at the time I only had access to the SoM beta, and not the live LotRO servers, this had to suffice.
Vanguard - Saga of Heroes: Leth Nurae. I don't know if there even is a busy spot in Vanguard any more. Leth certainly wasn't busy, but I don't think Telon is at any particular risk of overpopulation as a whole. I wanted to test Vanguard because it still has some of the nicest graphics of any MMOG.
Warhammer Online - Age of Reckoning: Altdorf. Even Altdorf is empty nowadays. I suppose I could have hunted for a siege somewhere, but in the interest of saving time, I just headed to Order's capital city.
World of Warcraft: Dalaran. Where else?
Onto the test results!
Test ResultsAnd now we take a look at the test results in a nice, purdy chart. Every game was set to maximum settings. All sliders were pushed all the way to the right. This means that some games have a lower natural framerate than others.
The minimum reflects the lowest FPS from all samples, and the highest reflects the highest FPS from all samples. The average is the average of the three tests of the same resolution. The better scores are bolded. Fractions are rounded to the nearest 10th.
m= minimum FPS
M= maximum FPS
A= average FPS
Taking the AVG score from the above data, we can now determine a few things. The biggest difference we see comes from LotRO at full screen 1920x1200. It comes to a whopping 171.4 percent faster on Windows 7. That, however, is a simple anomaly as every other result is within a much smaller threshold.
In Favor of Windows 7 (Majority)
Even though Win7 won the majority of the avg scores, it wasn't by many. 5 to 4 wins are marginal.
1024x768 Windowed - Windows 7 has 5 games faster by an average of 16.94%
1024x768 Full Screen - Windows 7 has 5 games faster by an average of 14.67%
1920x1200 Windowed - Windows 7 has 6 games faster by an average of 16.67%
1920x1200 Full Screen - Windows 7 has 6 games faster by an average of 49.58% (largely affected by the LotRO fluke)
In Favor of Windows Vista (Minority)Windows 7 may have won the majority of comparisons, but not by much. It's worth taking a look at the results that favored Vista.
1024x768 Windowed - Vista has 4 games faster by an average of 15.13%
1024x768 Full Screen - Vista has 4 games faster by an average of 29.5%
1920x1200 Windowed - Vista has 3 games faster by an average of 30.73%
1920x1200 Full Screen - Vista has 3 games faster by an average of 14.5%
ConclusionWe see Windows 7 narrowly winning the majority, but as to why, it's only conjecture. Age of Conan plays better all around on Vista, but if LotRO's your game, you're better off with Win7. I looked for patterns between older games and newer games and came to a dead end there too.
WhatÂs interesting though, is that even though Windows 7 won by a small margin, itÂs actually a pretty impressive win. The OS is only a few weeks old, and we haven't seen many, if any, take advantage of it yet. Things like drivers and code optimization havenÂt been done yet. The test rig was also one that relies heavily on optimization, particularly since the video cards are new and theyÂre running in Crossfire. DDR3 is also the variable that not many have set up, let alone optimized yet. The fact that it was able to weigh in at all against an OS thatÂs already had two service packs is a victory in itself. As time goes on drivers will be updated and games will be optimized and then, IÂm quite confident, weÂll see these numbers really swing over to favor Win7.
But, I'm sure you have your own conclusions to draw from this test. Did I miss something? Let me know in the forums.