Test Setup

Remember when people bought graphics cards to play games?

Benchmarking Perspective

Anyone who has been at all interested in graphics cards over the last year has certainly heard their fare share of benchmarking horror stories. From questionable optimizations in synthetic benchmarks to rendering issues with specific titles, graphics card reviews have become much more difficult than in previous years.

In the past, a graphics card was judged by how it performed in 3DMark2001 or 3DMark03. Obviously the higher the score the faster that particular card would be at running the latest and greatest games on your system. Unfortunately, no two game engines are written the exact same way so the accuracy of this benchmark quickly became questionable at best. Realizing that the only benchmarks that mattered were from the actual games themselves, the industry soon became obsessed with timedemos and average framerates. Although much more relevant than synthetic benchmark results, these tests still were not an accurate depiction of how the cards performed during true gaming. To illustrate this point, consider a graphics card from vendor: X which runs a timedemo and comes back with an average framerate of 90fps. Another card from vendor: Y completes the same timedemo with an average framerate of 60fps. The results are posted on websites and everyone easily concludes that the card from vendor: X is the best card for this game. However, what is not seen here is that this same card has extreme highs and lows. For some intense scenes, the framerate drops to single digits. In contrast, the card from vendor: Y manages to keep framerates above 30fps at all times. In the end, the card from vendor: Y is the vastly superior choice for gaming.

With this in mind, we made the decision to test games the same way they are played. Using Fraps, we would actually play through a campaign or entire level of a given title. This would be done three times in an effort to give each card a fair evaluation under all possible in-game scenarios. One interesting characteristic of Fraps is the ability to record the framerate for each second and input it into a log. In doing so, we could plot the actual framerate throughout the level and see how the card behaves throughout various degrees of graphic intensity. Furthermore, we would then have three independent data sources for minimum, average, and maximum framerate. With this data at our disposal, we could easily determine which graphics card performed best in a given title.

Since we’re looking at the absolute best consumer graphics cards money can buy, we decided to maximize every in-game image quality setting. Testing was then done at resolutions from 1024*768 up to 1600*1200. This was done with and without 4X FSAA and 8X AF enabled. If any cards could run 1600*1200 with those image quality settings, it would be one of the three we would be testing.

System Setup

Test System Setup

CPU

 3.2 GHz Pentium 4 (Northwood)

Motherboards

ABIT IC7-MAX3 (Intel i875)

Power Supply 

Antec 550W TrueControl

Memory 

2x512MB Corsair PC3200LL Pro Series

Hard Drive

40GB 7200RPM Maxtor EIDE

Sound Card

Creative Labs Audigy 2 ZS

ATI Video Cards

Radeon X800 Pro, Radeon X800 XT Platinum Edition

NVIDIA Video Cards

GeForce 6800 Extreme, GeForce 6800 Ultra, GeForce 6800 GT

Video Drivers

NVIDIA Reference 61.11, ATI Beta Catalyst

DirectX Version

DX 9.0b

Operating System

Windows XP w/ Service Pack 1

With this much graphics horsepower at our disposal, we needed a system which was powerful enough to avoid any bottlenecks outside the cards themselves. For this reason, we were very critical when selecting the components for this system. Everything from the power supply to the memory was selected due to its performance and stability. With the system assembled, a fresh install of Windows XP Professional was loaded. This was then followed by the installation of DirectX 9.0b, critical updates, and the various gaming titles we would be testing. Whenever possible, the most recent patch was used for each game to ensure the best performance and compatibility throughout testing. Once all the software was loaded and the appropriate driver for the card was installed, the drive was then defragged. Once testing was complete for a given card, the drive was formatted and a clean installation of Windows and necessary software were loaded once again.

With a handful of the fastest graphics cards on the planet at our disposal at the same time, we finally had the opportunity to determine which vendor would be holding the performance crown this summer. Unfortunately, time constraints and new drivers prohibit us from comparing these cards to the previous generation. However, there will be an in-depth analysis of a full range of graphics cards coming in the near future which will clearly illustrate the performance gap these new cards have over their predecessors.

 

 

« PreviousNext »