Test Setup

Remember when people bought graphics cards to play games?

Benchmarking Perspective

Anyone who has been at all interested in graphics cards over the last year has certainly heard their fare share of benchmarking horror stories. From questionable optimizations in synthetic benchmarks to rendering issues with specific titles, graphics card reviews have become much more difficult than in previous years.

In the past, a graphics card was judged by how it performed in 3DMark2001 or 3DMark03. Obviously the higher the score the faster that particular card would be at running the latest and greatest games on your system. Unfortunately, no two game engines are written the exact same way so the accuracy of this benchmark quickly became questionable at best. Realizing that the only benchmarks that mattered were from the actual games themselves, the industry soon became obsessed with timedemos and average framerates. Although much more relevant than synthetic benchmark results, these tests still were not an accurate depiction of how the cards performed during true gaming. To illustrate this point, consider a graphics card from vendor: X which runs a timedemo and comes back with an average framerate of 90fps. Another card from vendor: Y completes the same timedemo with an average framerate of 60fps. The results are posted on websites and everyone easily concludes that the card from vendor: X is the best card for this game. However, what is not seen here is that this same card has extreme highs and lows. For some intense scenes, the framerate drops to single digits. In contrast, the card from vendor: Y manages to keep framerates above 30fps at all times. In the end, the card from vendor: Y is the vastly superior choice for gaming.

With this in mind, we made the decision to test games the same way they are played. Using Fraps, we would actually play through a campaign or entire level of a given title. This would be done three times in an effort to give each card a fair evaluation under all possible in-game scenarios. One interesting characteristic of Fraps is the ability to record the framerate for each second and input it into a log. In doing so, we could plot the actual framerate throughout the level and see how the card behaves throughout various degrees of graphic intensity. Furthermore, we would then have three independent data sources for minimum, average, and maximum framerate. With this data at our disposal, we could easily determine which graphics card performed best in a given title.

Since we’re looking at the absolute best consumer graphics cards money can buy, we decided to maximize every in-game image quality setting. Testing was then done at resolutions from 1024*768 up to 1600*1200. This was done with and without 4X FSAA and 8X AF enabled. If any cards could run 1600*1200 with those image quality settings, it would be one of the three we would be testing.

System Setup

Test System Setup

CPU

 3.2 GHz Pentium 4 (Northwood)

Motherboards

ABIT IC7-MAX3 (Intel i875)

Power Supply 

Antec 550W TrueControl

Memory 

2x512MB Corsair PC3200LL Pro Series

Hard Drive

40GB 7200RPM Maxtor EIDE

Video Cards

NVIDIA GeForce 6800 Ultra, Gigabyte GeForce FX 5950Ultra, ATI Radeon 9800XT

Video Drivers

NVIDIA Reference 60.72, ATI Catalyst 4.4

DirectX Version

DX 9.0b

Operating System

Windows XP w/ Service Pack 1

With this much graphics horsepower at our disposal, we needed a system which was powerful enough to avoid any bottlenecks outside the cards themselves. For this reason, we were very critical when selecting the components for this system. Everything from the power supply to the memory was selected due to its performance and stability. With the system assembled, a fresh install of Windows XP Professional was loaded. This was then followed by the installation of DirectX 9.0b, critical updates, and the various gaming titles we would be testing. Whenever possible, the most recent patch was used for each game to ensure the best performance and compatibility throughout testing. Once all the software was loaded and the appropriate driver for the card was installed, the drive was then defragged. Once testing was complete for a given card, the drive was formatted and a clean installation of Windows and necessary software were loaded once again.

In order to accurately gauge the performance of NVIDIA’s new flagship card, we tested the two fastest cards currently available. For NVIDIA, this was the GeForce FX 5950 Ultra with 256MB of memory. Representing the ATI camp, we had the Radeon 9800XT which is considered by many to be the reigning performance champion. Having these three cards compared side by side would clearly illustrate how far NVIDIA’s engineers had truly come with their latest architecture.

 

 

« PreviousNext »