Summary and Conclusions
If you haven't read my previous article, as I have referenced several times before, you need to do so. Not only does it have all the background on the testing process used here and some more information on what exactly each separate graph represents, it also includes some information on how Vsync can affect the issues that AMD CrossFire sees as well as some problems we found with Eyefinity.
Performance
There are some interesting things that we need to take into consideration based on all the data we are showing you here. First, the same problem that existed in our Frame Rating launch article shows up here with the Radeon HD 7970 cards – which makes perfect sense as they are the same results! In three of our six tested games, Battlefield 3, Crysis 3 and Sleeping Dogs, the HD 7970s in CrossFire showed significant runt frames that tell us the animation smoothness and frame rate is not nearly as good as reported FRAPS average frame rates would like you to believe, AND that in our opinion buyers are being shorted on performance they are paying for with a second card. Three other games, Dirt 3, Skyrim and Far Cry 3 didn't have this problem.
Benefits of Single GPUs
The GeForce GTX 690 and the GeForce GTX Titan paint an interesting picture if you just consider them against each other. Both cards will cost you a cool $999, but offer very different hardware configurations. The GTX 690 is a dual-GK104 product with 4GB of total frame buffer and 2GB for each GPU; essentially slightly slower GTX 680s in SLI. The GTX Titan is a totally different beast with a single, larger GK110 GPU (2,688 cores in a single GPU, 1,536 in each GK104) and a larger 6GB frame buffer. When it launched many people complained that the GTX Titan was overpriced because it offered lower performance than the GTX 690 – which shows to be the case in most of our FRAPS benchmarks and even the observed FPS data.
What you might not have seen before though is what happens when we crank up the resolution to triple monitor setups – our 5760×1080 benchmarks. In those cases, pretty much across the board, we found that the GTX Titan was able to produce much more smooth and consistent frame times compared to the GTX 690. This is likely a combination of both the large frame buffer of the Titan, (triple the memory) as well as not having the need to worry about balancing frames across multiple GPUs and doing any kind of frame metering. When the GTX 690 is running games at higher settings on 5760×1080, it struggles to keep up.
But on single monitors the GTX 690 is able to bypass any inherint problems with multi-GPU configurations and produce a smooth frame rate. It's not quite AS SMOOTH as the GTX Titan but in general I would say the additional frame rate performance of the GTX 690 out weighs the frame time smoothness advantage that the Titan has. Take Sleeping Dogs at 2560×1440 – the average between the two cards goes from 35 FPS to nearly 45 FPS, and maximum variance only goes from 1 ms to about 3 ms.
What this means for the upcoming Radeon HD 7990 Release
While we aren't actually testing a Radeon HD 7990 here, we are basically testing the exact same configuration with a pair of Radeon HD 7970s running in CrossFire. This setup emulates the ASUS ARES II and the Powercolor Devil 13 pretty closely and from what we are hearing will be very close to what you'll find on the official reference HD 7990 as well. This comparison today wasn't in response to AMD's tease of information at GDC this week, but it is well timed none the less.
The results shouldn't surprise you, and it won't surprise AMD any more either – if released today, the HD 7990 would not perform well in our tests. AMD has told me that they are working on an option to meter frames in the way that NVIDIA is doing it, but offering users the options to enable or disable it, but we are months away from that fix. Until then, any dual-GPU Radeon HD 7000-series cards are going to show these problems represented as runts and dropped frames. We have many more pages of results to go over for the HD 7950/7870/7850/etc and those will be published in the coming days – but the story will look very similar as you'll find.
In all honesty, when AMD told me they were planning this card release I recommend they hold off until its driver fix is in place – myself and other reviewers are going to be hitting them hard on these issues until then, and any dual-GPU option with the Radeon name is going struggle to live up to any placed expectations.
Final Thoughts
The second part of our final reveal of the Frame Rating performance methods have shown us some interesting results for the $999 and above card lineups as they stand today. The Radeon HD 7970s in CrossFire, representing the currently available and upcoming HD 7990s don't look great in our testing as we mentioned above, and I would seriously consider your buying decision before picking up this configuration.
NVIDIA's GeForce GTX 690 and GTX Titan present another debate though on the issue of single versus multi-GPU options. In our testing the GTX 690 definitely introduces more frame time variance than the GTX Titan, but the cards are so fast as it is that running the GTX 690 at single monitor resolutions like 1920×1080 and 2560×1440 don't cause a big enough problem to be a factor, thus giving it the edge because of the higher average frame rates. The GTX Titan is a powerful card and performs admirably in the single display testing but it really stands away from the GTX 690 on multi-monitor resolutions like 5760×1080 where the 6GB frame buffer can help a TON. Not having to worry about moving frames between GPUs at that resolution also helps produce a smoother animation as well. If you are buying a $1000 card and you think you might want to run NVIDIA Surround, then the GTX Titan is your better solution.
Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons. Here is the schedule:
- 3/27: Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing
- 3/27: Radeon HD 7970 GHz Edition vs GeForce GTX 680 (Single and Dual GPU)
- 3/30: AMD Radeon HD 7990 vs GeForce GTX 690 vs GeForce GTX Titan
- 4/2: Radeon HD 7950 vs GeForce GTX 660 Ti (Single and Dual GPU)
- 4/5: Radeon HD 7870 GHz Edition vs GeForce GTX 660 (Single and Dual GPU)
Ryan,
Don’t worry about the
Ryan,
Don’t worry about the negative and bias comments.
Thank you for this great review, it has opened my eyes to the cause of these problems. And hopefully a new way to review all Graphics cards in future, instead of just looking at the highest FPS numbers.
I have always thought smooth experience is better than a fast (high FPS) and choppy visual gameplay.
Hopefully AMD and Nvidia will consider these issues in there next GPU and or driver releases now it has been exposed, rather than targeting figures. This means a better gameplay experience for the consumer.
Thank you and Keep up the good work.
Ryan,
Don’t worry about the
Ryan,
Don’t worry about the negative and bias comments.
Thank you for this great review, it has opened my eyes to the cause of these problems. And hopefully a new way to review all Graphics cards in future, instead of just looking at the highest FPS numbers.
I have always thought smooth experience is better than a fast (high FPS) and choppy visual gameplay.
Hopefully AMD and Nvidia will consider these issues in there next GPU and or driver releases now it has been exposed, rather than targeting figures. This means a better gameplay experience for the consumer.
Thank you and Keep up the good work.
I think that instead of the
I think that instead of the percentile curve you could reach a more meaningful result using a derived curve(of the frametime curve).
Let’s say that the average is 60 fps.
Now let’s say that 20 percent of the frames are 25 ms(40fps).
The difference is how these 25 ms values are spread in the curve. If they are all together or if they are alternated to 17 ms ones, forming saw-like shape in the curve.
You will not have the same feeling stutter-wise
What i want to say is that the percentile graph is not appropriate for the kind of analysis that you are doing. You should use a derived curve since deriving a function measures how quickly a curve grows (negatively or positively) and this is not measured by the percentile grows. After this you could measure the area of this curve and you could also arrive to use one only number to measure the amount of stutter.Infact in this way you would bring out of the equation the part of the frametime curve that is below the average but that runs steadily.
Calculating the area of a very saw-like derived frametime curve you would obtain a high number whereas calculating the area of a smooth (even if variating) derived frametime curve you would get a very low number. This would tell you how smooth are transitions, not if the gpu is powerful enough to make the game playable. For this you should check the average fps.
So in the end if you got decent fps and very low value for the area of this function you got a great experience,
if oyu got decent fps but high derived func area value then you got stutterish experience.
If you got low fps and low value you got a underdimensioned gpu but good smoothness.
I think that instead of the
I think that instead of the percentile curve you could reach a more meaningful result using a derived curve(of the frametime curve).
Let’s say that the average is 60 fps.
Now let’s say that 20 percent of the frames are 25 ms(40fps).
The difference is how these 25 ms values are spread in the curve. If they are all together or if they are alternated to 17 ms ones, forming saw-like shape in the curve.
You will not have the same feeling stutter-wise (and here i am not saying anything new)
What i want to say is that the percentile graph is not appropriate for the kind of analysis that you are doing. You should use a derived curve since deriving a function measures how quickly a curve grows (negatively or positively) and this is not measured by the percentile curve. After this you could measure the area of this curve and you could also arrive to use one only number to measure the amount of stutter.Infact in this way you would bring out of the equation the part of the frametime curve that is below the average but that runs steadily(something that with percentile curve you cant do).
Calculating the area of the derivation of a very saw-like frametime curve you would obtain a high number whereas calculating the area of the derivation ofa smooth (even if variating) frametime curve you would get a very low number. This would tell you how smooth are transitions, not if the gpu is powerful enough to make the game playable. For this you should check the average fps.
So in the end if you got decent fps and very low value for the area of this function you got a great experience,
if oyu got decent fps but high derived func area value then you got stutterish experience.
If you got low fps and low value you got a underdimensioned gpu but good smoothness.
EDITED :I made some corrections to the post i previously wrote since it is not possible to edit it
Quick Google “geforce frame
Quick Google “geforce frame metering” and you will find out why the nVi cards rarely have runt frames. In fact, nVi cards DO have them. They just delays those frames a bit to match with other good frames’ speed, therefore the frame time chart looks good miraculously.
That’s nVidia, it’s meant to SELL, at crazy pricetags of course.