New Testing Setup: Vista 64-bit

Testing Methodology

Graphics card testing has become the most hotly debated issue in the hardware enthusiast community recently.  Because of that, testing graphics cards has become a much more complicated process than it once was.  Before you might have been able to rely on the output of a few synthetic, automatic benchmarks to make your video card purchase, that is just no longer the case.  Video cards now cost up to $500 and we want to make sure that we are giving the reader as much information as we can to aid you in your purchasing decision.  We know we can’t run every game or find every bug and error, but we try to do what we can to aid you, our reader, and the community as a whole.

With that in mind, all the benchmarks that you will see in this review are from games that we bought off the shelves just like you.  Of these games, there are two different styles of benchmarks that need to be described.

The first is the “timedemo-style” of benchmark.  Many of you may be familiar with this style from games like Quake III; a “demo” is recorded in the game and a set number of frames are saved in a file for playback.  When playing back the demo, the game engine then renders the frames as quickly as possible, which is why you will often see the “timedemo-style” of benchmarks playing back the game much more quickly than you would ever play the game.  In our benchmarks, the FarCry tests were done in this matter: we recorded four custom demos and then played them back on each card at each different resolution and quality setting.  Why does this matter?  Because in these tests where timedemos are used, the line graphs that show the frame rate at each second, each card may not end at the same time precisely because one card is able to play it back faster than the other — less time passes and thus the FRAPs application gets slightly fewer frame rates to plot.  However, the peaks and valleys and overall performance of each card is still maintained and we can make a judged comparison of the frame rates and performance.

The second type of benchmark you’ll see in this article are manual run throughs of a portion of a game.  This is where we sit at the game with a mouse in one hand, a keyboard under the other, and play the game to get a benchmark score.  This benchmark method makes the graphs and data easy to read, but adds another level of difficulty to the reviewer — making the manual run throughs repeatable and accurate.  I think we’ve accomplished this by choosing a section of each game that provides us with a clear cut path. We take three readings of each card and setting, average the scores, and present those to you.  While this means the benchmarks are not exact to the most minute detail, they are damn close and practicing with this method for many days has made it clear to me that while this method is time consuming, it is definitely a viable option for games without timedemo support.

The second graph is a bar graph that tells you the average framerate, the maximum framerate, and the minimum framerate.  The minimum and average are important numbers here as we want the minimum to be high enough to not affect our gaming experience.  While it will be the decision of each individual gamer what is the lowest they will allow, comparing the Min FPS to the line graph and seeing how often this minimum occurs, should give you a good idea of what your gaming experience will be like with this game, and that video card on that resolution.

Our tests are completely based around the second type of benchmark method mentioned above — the manual run through.

System Setup and Benchmarks

A big area of contention for me with this review was my choice to use Vista 64-bit instead of Windows XP.  While indeed most gamers are probably still on Windows XP, with the DX10 games right around the corner, the move to Vista is going to be picking up for the enthusiast crowd.  As such, I thought it was only responsible for me to start testing on Vista full time with gaming so that my results with it are more reliable and stable; in addition, since we have been spending so much time listening to both graphics companies tout their dominance in Vista, we really should be putting it to the test on a regular basis. 

In addition to my move to Vista 64-bit for our testing, you’ll also notice a DRASTIC change in the game titles that are used in benchmarking.  Only a couple of titles (Prey and FEAR) remain from the previous set and we have thrown SIX new games to reflect the gaming of our users.  I have included FEAR, Prey, HL2 Episode 1, Supreme Commander, Company of Heroes, Oblivion, Battlefield 2142 and Rainbow Six Vegas.  This of course added a TON of time to our testing cycle so a lot more work had to go into getting the test setups ready for the new onslaught of performance metrics.  I think it turned out very well though and pushes the hardware and drivers to areas that other reviews might not investigate.

The $399 price tag on the Radeon HD 2900 XT (the highest end part from AMD now) forced me to compare it not to NVIDIA’s flagship cards directly since they are more expensive, but instead to the GeForce 8800 GTS 640MB model that is similarly priced.  That puts the 8800 GTX and the new 8800 Ultra on a playing field on their own, but wouldn’t it be magical if AMD’s 2900 XT could be them at their own game for half the price?  Well, it won’t happen but it’s nice to dream.

Our testing is broken up into two areas: single card and dual card.  The single card setups compare the new Radeon HD 2900 XT to the previous AMD flagship, the Radeon X1950 XTX, a retail EVGA 8800 GTS 640MB card that is factory overclocked and a reference GeForce 8800 GTX card.  The dual card benchmarks will compare how the single and CrossFire HD 2900 XTs compare to the single and SLI 8800 GTS 640MB cards. 

AMD ATI Radeon HD 2900 XT Test System Setup


Intel Core 2 Extreme X6800 – Review


EVGA nForce 680i MotherboardReview
Intel 975XBX Motherboard (for CrossFire testing)


Corsair TWIN2X2048-8500C4

Hard Drive

Western Digital Raptor 150 GB – Review

Sound Card

Sound Blaster Audigy 2 Value

Video Card

AMD ATI Radeon HD 2900 XT
EVGA GeForce 8800 GTS 640MB
NVIDIA Reference GeForce 8800 GTX – Review
AMD ATI Radeon X1950 XTX – Review

Video Drivers

AMD R600 Beta
NVIDIA Forceware 158.42
AMD Catalyst 7.4

Power Supply PC Power and Cooling 1000 watt

DirectX Version

DX10 / DX9c

Operating System

Windows Vista Ultimate 64-bit

  • Battlefield 2142 – crashed on the AMD HD 2900 XT for me and was thus left out
  • FEAR
  • Supreme Commander
  • Company of Heroes
  • Rainbow Six Vegas – AMD doesn’t yet support AA with HDR on this
  • Prey
  • Oblivion
  • Half-Life 2: Episode 1 – few crashes issues with the HD 2900 XT
  • 3DMark06 v1.1.0
« PreviousNext »