Testing Configuration and How We Test GPUs
For this article we are going to compare the ASUS, MSI and Sapphire custom built R9 280X card against the reference R9 280X we reviewed earlier this month. This will help us compare the overclocked peformance of the three retail cards without some of the noise involved in our general GPU review. To that end, if you don't already know where the AMD Radeon R9 280X stands in the ranks of the AMD/NVIDIA GPU battles, then you should probably check out the full review of the reference card FIRST before moving on.
This roundup serves more to help those enthusiasts that already know they want an R9 280X card to select WHICH model best fits their performance and pricing goals.
Speaking of pricing, this is how the three cards here stack up:
- ASUS R9 280X DirectCU II – $315 (Newegg.com)
- MSI R9 280X Twin Frozr Gaming – $315 (Newegg.com)
- Sapphire R9 280X TOXIC – $349 (Newegg.com)
Both the ASUS DCII and the MSI Gaming card are priced very modestly over the reference 280X pricing, pretty much assuring you a better experience than the reference cooler designs. The Sapphire TOXIC card is a bit pricier though thanks to the higher clocks, larger cooler and more unique design (with back plate, etc). We'll have to see in our testing of performance, temperature and noise levels how these changes affect the end results.
Testing Configuration
The specifications for our testing system haven't changed much.
Test System Setup | |
CPU | Intel Core i7-3960X Sandy Bridge-E |
Motherboard | ASUS P9X79 Deluxe |
Memory | Corsair Dominator DDR3-1600 16GB |
Hard Drive | OCZ Agility 4 256GB SSD |
Sound Card | On-board |
Graphics Card |
AMD Radeon R9 280X 3GB ASUS R9 280X 3GB DirectCU II MSI R9 280X 3GB Twin Frozr Gaming Sapphire R9 280X 3GB TOXIC |
Graphics Drivers | AMD: 13.11 V1 |
Power Supply | Corsair AX1200i |
Operating System | Windows 8 Pro x64 |
If you are already familiar with our testing process (which is rather unique and can produce daunting amounts of data), then feel free to jump ahead on to the first set of results!
Frame Rating: Our Testing Process
If you aren't familiar with it, you should probably do a little research into our testing methodology as it is quite different than others you may see online. Rather than using FRAPS to measure frame rates or frame times, we are using an secondary PC to capture the output from the tested graphics card directly and then use post processing on the resulting video to determine frame rates, frame times, frame variance and much more.
This amount of data can be pretty confusing if you attempting to read it without proper background, but I strongly believe that the results we present paint a much more thorough picture of performance than other options. So please, read up on the full discussion about our Frame Rating methods before moving forward!!
While there are literally dozens of file created for each “run” of benchmarks, there are several resulting graphs that FCAT produces, as well as several more that we are generating with additional code of our own.
If you don't need the example graphs and explanations below, you can jump straight to the benchmark results now!!
The PCPER FRAPS File
While the graphs above are produced by the default version of the scripts from NVIDIA, I have modified and added to them in a few ways to produce additional data for our readers. The first file shows a sub-set of the data from the RUN file above, the average frame rate over time as defined by FRAPS, though we are combining all of the GPUs we are comparing into a single graph. This will basically emulate the data we have been showing you for the past several years.
The PCPER Observed FPS File
This graph takes a different subset of data points and plots them similarly to the FRAPS file above, but this time we are look at the “observed” average frame rates, shown previously as the blue bars in the RUN file above. This takes out the dropped and runts frames, giving you the performance metrics that actually matter – how many frames are being shown to the gamer to improve the animation sequences.
As you’ll see in our full results on the coming pages, seeing a big difference between the FRAPS FPS graphic and the Observed FPS will indicate cases where it is likely the gamer is not getting the full benefit of the hardware investment in their PC.
The PLOT File
The primary file that is generated from the extracted data is a plot of calculated frame times including runts. The numbers here represent the amount of time that frames appear on the screen for the user, a “thinner” line across the time span represents frame times that are consistent and thus should produce the smoothest animation to the gamer. A “wider” line or one with a lot of peaks and valleys indicates a lot more variance and is likely caused by a lot of runts being displayed.
The RUN File
While the two graphs above show combined results for a set of cards being compared, the RUN file will show you the results from a single card on that particular result. It is in this graph that you can see interesting data about runts, drops, average frame rate and the actual frame rate of your gaming experience.
For tests that show no runts or drops, the data is pretty clean. This is the standard frame rate per second over a span of time graph that has become the standard for performance evaluation on graphics cards.
A test that does have runts and drops will look much different. The black bar labeled FRAPS indicates the average frame rate over time that traditional testing would show if you counted the drops and runts in the equation – as FRAPS FPS measurement does. Any area in red is a dropped frame – the wider the amount of red you see, the more colored bars from our overlay were missing in the captured video file, indicating the gamer never saw those frames in any form.
The wide yellow area is the representation of runts, the thin bands of color in our captured video, that we have determined do not add to the animation of the image on the screen. The larger the area of yellow the more often those runts are appearing.
Finally, the blue line is the measured FPS over each second after removing the runts and drops. We are going to be calling this metric the “observed frame rate” as it measures the actual speed of the animation that the gamer experiences.
The PERcentile File
Scott introduced the idea of frame time percentiles months ago but now that we have some different data using direct capture as opposed to FRAPS, the results might be even more telling. In this case, FCAT is showing percentiles not by frame time but instead by instantaneous FPS. This will tell you the minimum frame rate that will appear on the screen at any given percent of time during our benchmark run. The 50th percentile should be very close to the average total frame rate of the benchmark but as we creep closer to the 100% we see how the frame rate will be affected.
The closer this line is to being perfectly flat the better as that would mean we are running at a constant frame rate the entire time. A steep decline on the right hand side tells us that frame times are varying more and more frequently and might indicate potential stutter in the animation.
The PCPER Frame Time Variance File
Of all the data we are presenting, this is probably the one that needs the most discussion. In an attempt to create a new metric for gaming and graphics performance, I wanted to try to find a way to define stutter based on the data sets we had collected. As I mentioned earlier, we can define a single stutter as a variance level between t_game and t_display. This variance can be introduced in t_game, t_display, or on both levels. Since we can currently only reliably test the t_display rate, how can we create a definition of stutter that makes sense and that can be applied across multiple games and platforms?
We define a single frame variance as the difference between the current frame time and the previous frame time – how consistent the two frames presented to the gamer. However, as I found in my testing plotting the value of this frame variance is nearly a perfect match to the data presented by the minimum FPS (PER) file created by FCAT. To be more specific, stutter is only perceived when there is a break from the previous animation frame rates.
Our current running theory for a stutter evaluation is this: find the current frame time variance by comparing the current frame time to the running average of the frame times of the previous 20 frames. Then, by sorting these frame times and plotting them in a percentile form we can get an interesting look at potential stutter. Comparing the frame times to a running average rather than just to the previous frame should prevent potential problems from legitimate performance peaks or valleys found when moving from a highly compute intensive scene to a lower one.
While we are still trying to figure out if this is the best way to visualize stutter in a game, we have seen enough evidence in our game play testing and by comparing the above graphic to other data generated through our Frame rating system to be reasonably confident in our assertions. So much in fact that I am going to going this data the PCPER ISU, which beer fans will appreciate the acronym of International Stutter Units.
To compare these results you want to see a line that is as close the 0ms mark as possible indicating very little frame rate variance when compared to a running average of previous frames. There will be some inevitable incline as we reach the 90+ percentile but that is expected with any game play sequence that varies from scene to scene. What we do not want to see is a sharper line up that would indicate higher frame variance (ISU) and could be an indication that the game sees microstuttering and hitching problems.
really haven’t justified why
really haven’t justified why the MSI didn’t get the same award as the Asus.
Not sure what this is tryingt
Not sure what this is tryingt to ask. It DID get the same award?
MSI rocks as cooling and
MSI rocks as cooling and slient fan solutions are mandatory for performance of a processor atleast at a specified clock rate provided by manufacturer.Overclocking is secondary. Also power consumption is less.
Nice roundup!
You tested the
Nice roundup!
You tested the MSI card with the old BIOS though, with the alleged VRM overheating problems.
This is the new one (according to Tomshardware(DE) it’s a lot louder): http://www.techpowerup.com/vgabios/146769/msi-r9280x-3072-131009.html
Hmm, interesting I’ll take a
Hmm, interesting I'll take a look.
Nice write-up Ryan, I really
Nice write-up Ryan, I really like the look and design of the ASUS GPU, they really have put some thought and effort into the production of this card.
All good cards and a nice
All good cards and a nice review, although the noise of the Toxic is disappointing considering the price premium. I would have preferred a presumably quieter 3-slot design like the Asus Matrix Platinum (also $350). Any plans on reviewing that?
Under Metro Last Light :
What
Under Metro Last Light :
What is the difference between the FRAPS FPS and Observed FPS chart?
Other than that, it was interesting to see how various brands tackle certain challenges.
In my opinion, Sapphire would learn a lot from ASUS for their next design. I find ASUS has the neater design for long-term performance.
Hey Ryan, great article.
I
Hey Ryan, great article.
I myself just got the toxic 280x about a week ago. I agree the performance is great but i’m having game shadow/ texture flickering as well as some general screen artifacts even out of the box… (crysis 3, heaven 4.0, BF4)as well as some noticeable coil whine.
Im on the latest drivers and believe I have a good enough psu
Thermaltake 750w bronze with a core i5 2500k. Temps are also under 70c when this is happening. Maybe just a bad egg?
Still rocking the house with
Still rocking the house with a gtx 570 SLI setup how does this card compare?
well im from Lithuania and in
well im from Lithuania and in my country asus cost 100$ then toxic and it cant overclock so much so wait in my country toxic is better in every single aspect since if asus have same oc it would be as loud to maintain temperature at that point so faster cheaper cooler i dont care about sound while i have decent case with sound dapening and taking this bether card i take silver so dont get me wrong but from when silver is bether then gold since gpu toxic>asus in where i live sad to see such ratings just for noice while u knew well that that gpu has more horses compare with cars why ppl love ferary sound? becouse its faster ….. review is good awards not so much graps of fps in games totaly stupid why ? who makes 2 almost identical colors for graps of 4 cards? cant u put red blue green purple? ……. jesus
I picked up the MSI and
I picked up the MSI and updated the BIOS straight away. Performance is nice and it runs quietly. The only thing is the coil whine, which apparantly a lot of customers on newegg are also mentioning. Not a deal breaker, but it is a little annoying at times.
@Paul Hockett
Saved me loads
@Paul Hockett
Saved me loads of time, cheers, as I thought this might just be mine. Flickery bars across textures and shadows in bioshock and chess boards and technicolour discos in crysis 3. I find it happens quite randomly and can game 8hrs without trouble then next day 30m in it starts so was gonna be a nightmare trying to figure which card was causing troubles. Real shame as when they work i’m enjoying everything at max at 1440p.
Anyone tested this card on
Anyone tested this card on Rome 2 ? How does it perform ?
I know its’ a bit late but i
I know its’ a bit late but i really wish you would have thrown in XFX’s Double D R9 280X 1000MHz BOOST
http://www.amazon.com/dp/B00FSC5N66/ref=wl_it_dp_o_pC_nS_ttl?_encoding=UTF8&colid=1O03RSCAQ99TD&coliid=I2KXFB8JHBUKK
I don’t mean to sound like a fan boy for them of which I can only say why doesn’t everyone buy an XFX GPU with a Lifetime warranty. This year it seems that what is going to happen is I will be drown back to an AMD GPU. Who pulled no punches this season. From a GTX 780 which I had nearly purchased when the AMD R9 280’s were announced and reviewed.
Thanks for the Video Reviews on YouTube and also with This Week in Computer Hardware, with Patrick Norton.
So what are the other
So what are the other alternatives of it? I just wanna know more about maintenance..Find more on http://www.greatbodykits.com/
Just bought one of
Just bought one of these(ASUS) and its a beast, bang per buck it one of the best on the market. Well I think so anyway 🙂