Testing Suite and Methodology Update

Along with a new generation of graphics cards comes an updated GPU testbed for us here at PC Perspective. A lot has changed about desktops since we last updated our GPU in 2015. 

With the release of AMD's Ryzen CPUs, and Intel's subsequent Coffee Lake CPUs, core counts have been jumping up in consumer level processors compared to the stagnation of quad-core, eight-thread processors of yore. 

While we usually build our GPU testbeds on Intel's HEDT desktop platform (see our previous choice of the i7-5960X), this time we decided to go with the six-core, twelve-thread Intel Core i7-8700K. 

Given the increasing core counts of consumer processors, and the dimisnished focus on multiple GPU setups, we felt it was time to bring our GPU testbed to a more resonable price level.

  PC Perspective GPU Testbed (2018)
Processor Intel Core i7-8700K
Motherboard ASUS ROG Z370-H Gaming
Memory

Corsair Vengeance LPX DDR4-3200 

(Running at DDR4-2666)

Storage Samsung 850 EVO 250GB (OS)
Micron 11100 2TB (games)
Power Supply Corsair AX1500i 1500 watt
OS Windows 10 x64 Version 1803 (RS4)
Drivers

AMD: 18.8.2
NVIDIA:  399.07 (GTX) / 410.51 (RTX0

Discounting the overkill 1600W power supply, we are using because it was already modified for our power measurement needs, and the 2TB secondary SSD for game storage, this new build minus the GPU comes in at just around $1000. We feel this price point is a lot more reasonable than testbeds of yore (with $1600 a processor for example), and more representative of the PC gaming community at large, without providing a bottleneck to the given GPU we are testing.

In testing these new RTX series of GPUs, there are a few interesting comparisons to make. First, we want to look at generation-over-generation performance over the previous GTX-10 series products. Next, of course, we'll want to compare these new GPUs to the fastest currently existing options from both NVIDIA and AMD. The cards we will be testing are listed below:

As for the games we tested, we wanted to update our test suite with some of the most modern PC titles, while remaining a few older titles that are still immensely popular. 

  • Far Cry 5
  • Wolfenstein II: The New Colossus
  • Ashes of the Singularity: Escalation
  • F1 2018
  • Grand Theft Auto V
  • Sniper Elite 4
  • Strange Brigade
  • Witcher 3
  • Hitman (2016)

Our Testing Process

While the way we present our data is a bit tweaked, our testing methodology is not. We are still using the capture-based Frame Rating technique that we helped pioneer back in 2013. For those who are unaware of Frame Rating, you can read this great in-depth breakdown of the process.

As far as the data we are presenting we have simplified this a bit from years past. Instead of presenting a series of 6 graphs from every Frame Rating output, we are now focusing on two major areas—frame rate percentiles, and frame times.

For each game tested, you'll find a bar graph with the average, 95th, and 99th percentile frame rates. This will help give an idea of the relative performance of each GPUs but takes the event important frame consistency into account to determine how smoothly a game was running. Essentially, the closer the average, 95th, and 99th percentile numbers are to each other, the smoother the gaming experience.

Similarly, each game tested will feature a frame time chart. The numbers here represent the amount of time that frames appear on the screen for the user, a “thinner” line across the time span represents frame times that are consistent and thus should produce the smoothest animation to the gamer.  A “wider” line or one with a lot of peaks and valleys indicates a lot more variance and is caused by a lot of runts displayed.

Lastly, we'll be providing a quick summary of relative performance between competing GPUs, calculated from the average frame rate.

« PreviousNext »