The NVIDIA GeForce GTX 960 2GB Review – GM206 at $199

ASUS GTX 960 STRIX and Testing Setup

We have several retail GTX 960 cards already in our possession for testing but the first to arrive was the ASUS GTX 960 Strix card and thus it will be the basis for our testing in today's launch story. You can expect a full review of this card, the EVGA GTX 960 SSC and the MSI GTX 960 100ME card in the coming days!

The GTX 960 Strix follows in ASUS' goals of creating graphics cards that are both highly overclockable yet as quiet as possible, hence the owl logo the company has adopted. The cooler on the card is beefy, extending well past the PCB of the card and uses the familiar DirectCU II 0 dB fan technology. ASUS claims that the DCII cooler is 220% larger than the heatsink used on the reference design.

Out of the box the ASUS Strix card will run with a base clock of 1291 MHz, a rated Boost clock of 1317 MHz and memory running at 7200 MHz. Those are modest increases over the reference speeds of 1127 MHz base (14% higher), 1178 MHz Boost (12% higher) and 7000 MHz (3% higher). The result should be a performance increase in real-world games of anywhere from 5-10% but we'll confirm that on a later page.

From a build perspective ASUS has improved over the reference PCB design with 5-phase power using Super Alloy chokes that decrease any chance for coil whine or noise. It also employs Super Alloy capacitors and MOS for longer lifespan and higher voltage thresholds that should, at least in theory, allow for large overclocking windows.

The GTX 960 by design only requires a single 6-pin power connection and the ASUS Strix model follows that lead. ASUS does have a handy LED on the back that illuminates red when the ATX power connection is not attached; it turns white when you have plugged in the cable (dummy). Also seen here is the back plate included on all Strix models to help protect the components and act as a basic heatsink for the PCB.

Output connectivity includes a dual-link DVI port, a full-size HDMI port and a set of three full-size DisplayPorts. This basically mirrors that of the GTX 980 and is a welcome change for users that frequently attach several displays to their systems.

As for the price, the ASUS GTX 960 Strix will start with an MSRP of just $209, a very modest $10 increase over the price of a reference design with the same GPU. Considering the cooler improvement and out-of-box overclock, that is an easy upgrade decision to make.

Testing Configuration

The specifications for our testing system haven't changed much.

Test System Setup
CPU	Intel Core i7-3960X Sandy Bridge-E
Motherboard	ASUS P9X79 Deluxe
Memory	Corsair Dominator DDR3-1600 16GB
Hard Drive	OCZ Agility 4 256GB SSD
Sound Card	On-board
Graphics Card	ASUS GeForce GTX 960 Strix 2GB NVIDIA GeForce GTX 760 2GB NVIDIA GeForce GTX 660 2GB Sapphire Radeon R9 285 2GB MSI Radeon R9 280 Gaming 3GB
Graphics Drivers	AMD: 14.12 Catalyst Omega NVIDIA: 347.25
Power Supply	Corsair AX1500i
Operating System	Windows 8.1 Pro x64

What you should be watching for

1. GTX 960 vs GTX 760 – The GTX 760 was a larger GPU but can the GTX 960 actually surpass it in performance?
    
2. GTX 960 vs R9 285 – AMD's current target in the $199/$209 price range is the Radeon R9 285. Can Tonga hold its own against a smaller Maxwell GPU?
    
3. GTX 960 vs GTX 960 MFAA – We tested a handful more games with MFAA enabled – does it really make the difference that NVIDIA seems to think it does when it comes to performance at 1080p?

If you don't need the example graphs and explanations below, you can jump straight to the benchmark results now!!

 

Frame Rating: Our Testing Process

If you aren't familiar with it, you should probably do a little research into our testing methodology as it is quite different than others you may see online.  Rather than using FRAPS to measure frame rates or frame times, we are using an secondary PC to capture the output from the tested graphics card directly and then use post processing on the resulting video to determine frame rates, frame times, frame variance and much more. 

This amount of data can be pretty confusing if you attempting to read it without proper background, but I strongly believe that the results we present paint a much more thorough picture of performance than other options.  So please, read up on the full discussion about our Frame Rating methods before moving forward!!

While there are literally dozens of file created for each “run” of benchmarks, there are several resulting graphs that FCAT produces, as well as several more that we are generating with additional code of our own. 

If you don't need the example graphs and explanations below, you can jump straight to the benchmark results now!!

 

The PCPER FRAPS File

While the graphs above are produced by the default version of the scripts from NVIDIA, I have modified and added to them in a few ways to produce additional data for our readers.  The first file shows a sub-set of the data from the RUN file above, the average frame rate over time as defined by FRAPS, though we are combining all of the GPUs we are comparing into a single graph.  This will basically emulate the data we have been showing you for the past several years.

 

The PCPER Observed FPS File

This graph takes a different subset of data points and plots them similarly to the FRAPS file above, but this time we are look at the “observed” average frame rates, shown previously as the blue bars in the RUN file above.  This takes out the dropped and runts frames, giving you the performance metrics that actually matter – how many frames are being shown to the gamer to improve the animation sequences. 

As you’ll see in our full results on the coming pages, seeing a big difference between the FRAPS FPS graphic and the Observed FPS will indicate cases where it is likely the gamer is not getting the full benefit of the hardware investment in their PC.

 

The PLOT File

The primary file that is generated from the extracted data is a plot of calculated frame times including runts.  The numbers here represent the amount of time that frames appear on the screen for the user, a “thinner” line across the time span represents frame times that are consistent and thus should produce the smoothest animation to the gamer.  A “wider” line or one with a lot of peaks and valleys indicates a lot more variance and is likely caused by a lot of runts being displayed.

 

The RUN File

While the two graphs above show combined results for a set of cards being compared, the RUN file will show you the results from a single card on that particular result.  It is in this graph that you can see interesting data about runts, drops, average frame rate and the actual frame rate of your gaming experience. 

For tests that show no runts or drops, the data is pretty clean.  This is the standard frame rate per second over a span of time graph that has become the standard for performance evaluation on graphics cards.

A test that does have runts and drops will look much different.  The black bar labelled FRAPS indicates the average frame rate over time that traditional testing would show if you counted the drops and runts in the equation – as FRAPS FPS measurement does.  Any area in red is a dropped frame – the wider the amount of red you see, the more colored bars from our overlay were missing in the captured video file, indicating the gamer never saw those frames in any form.

The wide yellow area is the representation of runts, the thin bands of color in our captured video, that we have determined do not add to the animation of the image on the screen.  The larger the area of yellow the more often those runts are appearing.

Finally, the blue line is the measured FPS over each second after removing the runts and drops.  We are going to be calling this metric the “observed frame rate” as it measures the actual speed of the animation that the gamer experiences.

 

The PERcentile File

In this case, FCAT is showing percentiles not by frame time but instead by instantaneous FPS.  This will tell you the minimum frame rate that will appear on the screen at any given percent of time during our benchmark run.  The 50^th percentile should be very close to the average total frame rate of the benchmark but as we creep closer to the 100% we see how the frame rate will be affected. 

The closer this line is to being perfectly flat the better as that would mean we are running at a constant frame rate the entire time.  A steep decline on the right hand side tells us that frame times are varying more and more frequently and might indicate potential stutter in the animation.

 

The PCPER Frame Time Variance File

Of all the data we are presenting, this is probably the one that needs the most discussion.  In an attempt to create a new metric for gaming and graphics performance, I wanted to try to find a way to define stutter based on the data sets we had collected.  As I mentioned earlier, we can define a single stutter as a variance level between t_game and t_display. This variance can be introduced in t_game, t_display, or on both levels.  Since we can currently only reliably test the t_display rate, how can we create a definition of stutter that makes sense and that can be applied across multiple games and platforms?

We define a single frame variance as the difference between the current frame time and the previous frame time – how consistent the two frames presented to the gamer.  However, as I found in my testing plotting the value of this frame variance is nearly a perfect match to the data presented by the minimum FPS (PER) file created by FCAT.  To be more specific, stutter is only perceived when there is a break from the previous animation frame rates. 

Our current running theory for a stutter evaluation is this: find the current frame time variance by comparing the current frame time to the running average of the frame times of the previous 20 frames.  Then, by sorting these frame times and plotting them in a percentile form we can get an interesting look at potential stutter.  Comparing the frame times to a running average rather than just to the previous frame should prevent potential problems from legitimate performance peaks or valleys found when moving from a highly compute intensive scene to a lower one.

While we are still trying to figure out if this is the best way to visualize stutter in a game, we have seen enough evidence in our game play testing and by comparing the above graphic to other data generated through our Frame rating system to be reasonably confident in our assertions.  So much in fact that I am going to going this data the PCPER ISU, which beer fans will appreciate the acronym of International Stutter Units.

To compare these results you want to see a line that is as close the 0ms mark as possible indicating very little frame rate variance when compared to a running average of previous frames.  There will be some inevitable incline as we reach the 90+ percentile but that is expected with any game play sequence that varies from scene to scene.  What we do not want to see is a sharper line up that would indicate higher frame variance (ISU) and could be an indication that the game sees microstuttering and hitching problems.