The Competition – What Do We Compare?

If there was one thing that came under fire from our initial CrossFire review, it was how we chose the components from NVIDIA to compare to ATI’s latest parts.  Here I will detail what we are comparing and explain my reasoning behind them all.

In the ATI Reviewer’s Guide, I was presented with a table like this one:

ATI R520: Radeon X1800, X1600 and X1300 Review - Graphics Cards 197

Here we see that ATI is pitting their X1800 line against the 7800 GTX and GT — this makes sense.  However, they are comparing the X1600 XT that has an expected price point of $249 against the GeForce 6600 GT that can be found currently selling for under $150!  That doesn’t quite fit the mold we are going for, so instead I am comparing the X1600 XT to the NVIDIA 6800 GT (found for about $280 today) and the 6800 256 MB (found for about $210 today).  That is just a bit above and just a bit below the estimated price on the X1600 XT.

The same thing is found in our X1300 Pro comparison from ATI: they compare to the 6600 standard and 6200 cards, well below the price point of the $149 for the X1300 Pro.  Instead, we are going to compare it to the 6600 GT, selling for a nearly identical price ($150 or so today). 

Finally, looking at the high end cards, the 7800 GTX can be found for under $450 today in many places and thus I feel a comparison between it and the X1800 XL is just as justified as it is to the X1800 XT, a card with twice as much memory as the NVIDIA competition.  (PS: we’ll soon have a 512 MB version of the GTX for equal testing, shh!)  Also you’ll notice we are using overclocked 7800 GT and GTX cards in our testing.  But why wouldn’t we use stock speeds?  Quite frankly, because we seem to have more OC’d 7800s for sale on the market than we do stock cards, and there is no reason a buyer wouldn’t get these faster version for the same price.

I think that should make good enough sense to just about anyone that should read it.  While we can’t say for sure what the ATI cards are going to sell for after today, we DO know for sure what the NVIDIA cards are selling for today, so we can match them up with the estimated prices we are given by ATI.  If the prices change for the ATI cards, either up or down, we can make adjustments at a later time.

So, our GPU comparisons look like this:

  • X1800 XT vs. 7800 GTX
  • X1800 XL vs. 7800 GTX and 7800 GT
  • X1600 XT vs. 6800 GT and 6800
  • X1300 Pro vs. 6600 GT

Our test system was pretty straight forward and included an FX-55 processor, 1 GB of DDR400 memory running at 5-2-2 timings and a Sound Blaster Audigy 2 sound card.  The motherboard for the NVIDIA cards was a Gigabyte SLI motherboard and the ATI CrossFire reference board was used for all ATI X1000 series testing.

Testing Methodology

Graphics card testing has become the most hotly debated issue in the hardware enthusiast community recently.  Because of that, testing graphics cards has become a much more complicated process than it once was.  Where before you might have been able to rely on the output of a few synthetic, automatic benchmarks to make your video card purchase, that is just no longer the case.  Video cards now cost up to $500 and we want to make sure that we are giving the reader as much information as we can to aid you in your purchasing decision.  We know we can’t run every game or find every bug and error, but we try to do what we can to aid you, our reader, and the community as a whole.

With that in mind, all the benchmarks that you will see in this review are from games that we bought off the shelves just like you.  Of these games, there are two different styles of benchmarks that need to be described.

The first is the “timedemo-style” of benchmark.  Many of you may be familiar with this style from games like Quake III; a “demo” is recorded in the game and a set number of frames are saved in a file for playback.  When playing back the demo, the game engine then renders the frames as quickly as possible, which is why you will often see the “timedemo-style” of benchmarks playing back the game much more quickly than you would ever play the game.  In our benchmarks, the FarCry tests were done in this matter: we recorded four custom demos and then played them back on each card at each different resolution and quality setting.  Why does this matter?  Because in these tests where timedemos are used, the line graphs that show the frame rate at each second, each card may not end at the same time precisly because one card is able to play it back faster than the other — less time passes and thus the FRAPs application gets slightly fewer frame rates to plot.  However, the peaks and valleys and overall performance of each card is still maintained and we can make a judged comparison of the frame rates and performance.

The second type of benchmark you’ll see in this article are manual run throughs of a portion of a game.  This is where we sit at the game with a mouse in one hand, a keyboard under the other, and play the game to get a benchmark score.  This benchmark method makes the graphs and data easy to read, but adds another level of difficulty to the reviewer — making the manual run throughs repeatable and accurate.  I think we’ve accomplished this by choosing a section of each game that provides us with a clear cut path. We take three readings of each card and setting, average the scores, and present those to you.  While this means the benchmarks are not exact to the most minute detail, they are damn close and practicing with this method for many days has made it clear to me that while this method is time consuming, it is definitely a viable option for games without timedemo support.

The second graph is a bar graph that tells you the average framerate, the maximum framerate, and the minimum framerate.  The minimum and average are important numbers here as we want the minimum to be high enough to not affect our gaming experience.  While it will be the decision of each individual gamer what is the lowest they will allow, comparing the Min FPS to the line graph and seeing how often this minimum occurs, should give you a good idea of what your gaming experience will be like with this game, and that video card on that resolution.

Our tests are completely based around the second type of benchmark method mentioned above — the manual run through.

Test System Setup

Each set includes a line graph and a bar graph.  The line graph still shows the performance over the span of time of the benchmark and the bar graph shows the data in a min/max/avg format that many readers like to see and are more comfortable with. 

For our high end gaming tests recently, we have been testing at 1600×1200 resolutions only, but we obviously couldn’t do that with this review testing an entire family of GPUs from high end to budget.  So, to find our test resolution, we used the new ATI card to play around and find the highest resolution with the standard 4xAA/8xAF image quality settings that was playable and still fun.  Then we tested the NVIDIA cards at that same resolution and compared the results that you’ll see.

ATI Test System Setup

GPUs

ATI X1800 XT
ATI X1800 XL
ATI X1600 XT
ATI X1300 Pro
ATI X850 XT

Processor

Athlon 64 FX-55

Motherboard

ATI CrossFire Reference Board

Chipset Driver

5.9 CCC Beta

Memory

2 x 512 MB Corsair 3200XL

Memory Timings

2.0 2-2-5

Sound Card

Sound Blaster Audigy 2

Hard Drive

Maxtor DiamondMax 10 300 GB

Operating System

Windows XP Professional SP1

NVIDIA Driver

77.77

ATI Driver

5.9 CCC Beta

NVIDIA Test System Setup

GPUs

XFX 7800 GTX (450/1.25)
XFX 7800 GT (450/1.05)
NVIDIA 6800 GT
XFX 6800 256 MB
NVIDIA 6600 GT

Processor

Athlon 64 FX-55

Motherboard

Gigabyte K8NXP-SLI

Chipset Driver

6.53

Memory

2 x 512 MB Corsair 3200XL

Memory Timings

2.0 2-2-5

Sound Card

Sound Blaster Audigy 2

Hard Drive

Maxtor DiamondMax 10 300 GB

Operating System

Windows XP Professional SP1

NVIDIA Driver

77.77

ATI Driver

5.9 CCC Beta

Software tested:

  • Doom 3 v1.3
  • Far Cry v1.3
  • Half-Life 2 Engine 7 (two maps)
  • EverQuest 2
  • Battlefield 2
  • Guild Wars
  • FEAR SP Demo
  • 3DMark05
« PreviousNext »