Testing Setup and Frame Rating Info
The specifications for our testing system haven't changed much.
|Test System Setup|
|CPU||Intel Core i7-3960X Sandy Bridge-E|
|Motherboard||ASUS P9X79 Deluxe|
|Memory||Corsair Dominator DDR3-1600 16GB|
|Hard Drive||OCZ Agility 4 256GB SSD|
MSI GeForce GTX 780 Lightning 3GB
AMD Radeon HD 7970 GHz Edition 3GB
NVIDIA GeForce GTX 780 3GB
NVIDIA GeForce GTX TITAN 6GB
AMD: 13.8 beta
NVIDIA: 326.80 beta
|Power Supply||Corsair AX1200i|
|Operating System||Windows 8 Pro x64|
What you should be watching for
- MSI GTX 780 Lightning vs GTX 780 Reference – Does the overclocked speed improve performance? What about temperatures, sound levels and power consumption?
- MSI GTX 780 Lightning vs GTX TITAN Reference – Can this overclocked, lower core count GK110 GPU compete with the more expensive GTX TITAN?
- MSI GTX 780 Lightning vs HD 7970 GHz Edition – AMD's top single GPU card keeps coming down in cost. How does it stand up against the higher priced MSI GTX 780 Lightning?
Frame Rating: Our Testing Process
If you aren't familiar with it, you should probably do a little research into our testing methodology as it is quite different than others you may see online. Rather than using FRAPS to measure frame rates or frame times, we are using an secondary PC to capture the output from the tested graphics card directly and then use post processing on the resulting video to determine frame rates, frame times, frame variance and much more.
This amount of data can be pretty confusing if you attempting to read it without proper background, but I strongly believe that the results we present paint a much more thorough picture of performance than other options. So please, read up on the full discussion about our Frame Rating methods before moving forward!!
While there are literally dozens of file created for each “run” of benchmarks, there are several resulting graphs that FCAT produces, as well as several more that we are generating with additional code of our own.
If you don't need the example graphs and explanations below, you can jump straight to the benchmark results now!!
The PCPER FRAPS File
While the graphs above are produced by the default version of the scripts from NVIDIA, I have modified and added to them in a few ways to produce additional data for our readers. The first file shows a sub-set of the data from the RUN file above, the average frame rate over time as defined by FRAPS, though we are combining all of the GPUs we are comparing into a single graph. This will basically emulate the data we have been showing you for the past several years.
The PCPER Observed FPS File
This graph takes a different subset of data points and plots them similarly to the FRAPS file above, but this time we are look at the “observed” average frame rates, shown previously as the blue bars in the RUN file above. This takes out the dropped and runts frames, giving you the performance metrics that actually matter – how many frames are being shown to the gamer to improve the animation sequences.
As you’ll see in our full results on the coming pages, seeing a big difference between the FRAPS FPS graphic and the Observed FPS will indicate cases where it is likely the gamer is not getting the full benefit of the hardware investment in their PC.
The PLOT File
The primary file that is generated from the extracted data is a plot of calculated frame times including runts. The numbers here represent the amount of time that frames appear on the screen for the user, a “thinner” line across the time span represents frame times that are consistent and thus should produce the smoothest animation to the gamer. A “wider” line or one with a lot of peaks and valleys indicates a lot more variance and is likely caused by a lot of runts being displayed.
The RUN File
While the two graphs above show combined results for a set of cards being compared, the RUN file will show you the results from a single card on that particular result. It is in this graph that you can see interesting data about runts, drops, average frame rate and the actual frame rate of your gaming experience.
For tests that show no runts or drops, the data is pretty clean. This is the standard frame rate per second over a span of time graph that has become the standard for performance evaluation on graphics cards.
A test that does have runts and drops will look much different. The black bar labelled FRAPS indicates the average frame rate over time that traditional testing would show if you counted the drops and runts in the equation – as FRAPS FPS measurement does. Any area in red is a dropped frame – the wider the amount of red you see, the more colored bars from our overlay were missing in the captured video file, indicating the gamer never saw those frames in any form.
The wide yellow area is the representation of runts, the thin bands of color in our captured video, that we have determined do not add to the animation of the image on the screen. The larger the area of yellow the more often those runts are appearing.
Finally, the blue line is the measured FPS over each second after removing the runts and drops. We are going to be calling this metric the “observed frame rate” as it measures the actual speed of the animation that the gamer experiences.
The PERcentile File
Scott introduced the idea of frame time percentiles months ago but now that we have some different data using direct capture as opposed to FRAPS, the results might be even more telling. In this case, FCAT is showing percentiles not by frame time but instead by instantaneous FPS. This will tell you the minimum frame rate that will appear on the screen at any given percent of time during our benchmark run. The 50th percentile should be very close to the average total frame rate of the benchmark but as we creep closer to the 100% we see how the frame rate will be affected.
The closer this line is to being perfectly flat the better as that would mean we are running at a constant frame rate the entire time. A steep decline on the right hand side tells us that frame times are varying more and more frequently and might indicate potential stutter in the animation.
The PCPER Frame Time Variance File
Of all the data we are presenting, this is probably the one that needs the most discussion. In an attempt to create a new metric for gaming and graphics performance, I wanted to try to find a way to define stutter based on the data sets we had collected. As I mentioned earlier, we can define a single stutter as a variance level between t_game and t_display. This variance can be introduced in t_game, t_display, or on both levels. Since we can currently only reliably test the t_display rate, how can we create a definition of stutter that makes sense and that can be applied across multiple games and platforms?
We define a single frame variance as the difference between the current frame time and the previous frame time – how consistent the two frames presented to the gamer. However, as I found in my testing plotting the value of this frame variance is nearly a perfect match to the data presented by the minimum FPS (PER) file created by FCAT. To be more specific, stutter is only perceived when there is a break from the previous animation frame rates.
Our current running theory for a stutter evaluation is this: find the current frame time variance by comparing the current frame time to the running average of the frame times of the previous 20 frames. Then, by sorting these frame times and plotting them in a percentile form we can get an interesting look at potential stutter. Comparing the frame times to a running average rather than just to the previous frame should prevent potential problems from legitimate performance peaks or valleys found when moving from a highly compute intensive scene to a lower one.
While we are still trying to figure out if this is the best way to visualize stutter in a game, we have seen enough evidence in our game play testing and by comparing the above graphic to other data generated through our Frame rating system to be reasonably confident in our assertions. So much in fact that I am going to going this data the PCPER ISU, which beer fans will appreciate the acronym of International Stutter Units.
To compare these results you want to see a line that is as close the 0ms mark as possible indicating very little frame rate variance when compared to a running average of previous frames. There will be some inevitable incline as we reach the 90+ percentile but that is expected with any game play sequence that varies from scene to scene. What we do not want to see is a sharper line up that would indicate higher frame variance (ISU) and could be an indication that the game sees microstuttering and hitching problems.
Cool card but 1173Mhz core
Cool card but 1173Mhz core boost overclock “only”? C’mom, my reference PNY GTX780 overclocks stable 1173MHz without any overvoltage… I was expecting a lot more of this Lightning…
Check the update on the last
Check the update on the last page – actually running at 1241 MHz.
ALL of the kepler cards have
ALL of the kepler cards have a Max Boost different to the official Boost. It isn´t only about MSI Lightning. Everyone card has a unique Max Boost, different to the official boost of the model and other units of it.
ALL of the kepler cards reach upper clocks than the official boost that GPU-z shows.
The GPU reactor can be taken
The GPU reactor can be taken off without any impact on performance, as it only starts to shine when watercooled or LN2 cooled.
Ah cool, thanks for that!
Ah cool, thanks for that! I'll try removing it tomorrow.
You should give EVGA a call
You should give EVGA a call and get a sample of their GTX 780 Classified and compare it to this since both tend to trade blows.
i just got an idea on how to
i just got an idea on how to improve the graphs you give out with these Frame Rating (although they are very interesting to me being an AMD user).
the idea is to give ‘highlight’ of the performance and actually give the number of the observed FPS 2-5 times per benchmark, just the observed because that is the one that matters.
another thing could you make the graphs stretched a little by cutting out the bottom portion of the graph. example: if all the FPS is above 20, why show the graph part under 15. this would be to give a little more clarity when just looking at the graphs in the article with out clicking on a particular graph to get the bigger picture of it.
Thanks for the ideas. I’ll
Thanks for the ideas. I'll play around with trying to make them more readable in the coming weeks!
cool: thanks Ryan. Keep up
cool: thanks Ryan. Keep up the good work.
Hi Ryan, I was wondering if
Hi Ryan, I was wondering if you could include an average line on the observed FPS graph or state the average. Like on this graph.
Or is there a reason why it is left out?
It makes it easier to compare with what other review sites report.
This card is design for
This card is design for overclocking, like the clasified you should be able to overvolt the core to 1.35 volt safely and probably hit 1300 to 1400 on the core. It is a bit pointles to test the card without any overvolting because the limitation of Nvidia of 1.21v will let the card perform like any other….
overclock the dam thing and
overclock the dam thing and then people might care about this panzy review.
oc max vs calssified oc max vs gtx titan oc max at 1.35v. you won’t because that’s way above your skillset as usual ryan.
And he overclocked it to a
And he overclocked it to a level the typical gamer will use on this card.
Not everyone wants to massively clock their card just to see high numbers pop up on a screen once a graphics test is done because they lack any real sense of achievement in life that they feel great about dialing some numbers into an electronic device to make it run faster. Most people don’t particularly care about that they just want a little bit of extra power and safe temps while gaming.
Yes this card is for overclocking and he overclocked it but the option is there should you wish to push it further. This doesn’t mean a standard consumer who has no interest in number chasing cannot purchase this card for their own use to play games on a Titan level for less money.
Is it possible you are concerned as to what the new Lightning can achieve?
My Galaxy HOF is completely stable at 1293mhz on the core under gaming and can push 1330mhz for synthetic benchmarks. Lets hope the Lightning can at least exceed these. I myself am however just a normal gamer and not big into overclocking because its not my main focus to number chase. I have more interesting things to do with my time.
First, thanks for stopping
First, thanks for stopping by. 🙂
Second, max OC vs max OC is probably one of the worst indicators for basing a conclusion of a graphics card on. It's important, but can't be the only or even primary concern. Why? VARIABILITY.
Each card sample is going to be different on the Lightnings, each is going to be different on the Classified, and the difference between the lines is likely going to vary even more. I can overclock this card further (and I will I'm sure once I get the updated BIOS from MSI) but even if I get 50 MHz additional clock doesn't really mean YOU or any buyer will.
Now if MSI were to send me a dozen cards for each review we did, you'd have a better argument.
Also, go fuck yourself.
Great response, lol, you rock
Great response, lol, you rock dude.
way to poop on him.
way to poop on him.
This may help you.
This may help you. http://youtu.be/aWVywhzuHnQ
On the first comparative
On the first comparative graphs, you reference the 680 and the 680 sli, but not the 780?
If you are looking at THIS
If you are looking at THIS page then you are just seeing our explanation of the TYPE of graphs you are going to see, not the specific results for this review.
Did you JUST NOW figure out
Did you JUST NOW figure out that GPUz doesnt show the ACTUAL boost clock? Wow guys… I love your reviews but holy cow, pull it together!
So does this mean all your
So does this mean all your past reviews with boost clocks will need adjusting to show the actual clocks reached?
ANother question, did it actually HOLD that clock with the 109% limit or was it all over the place? I ask that as I am not sure if PEAK values are what should be reported. I am personally torn.
I know that 109% limit was hit VERY easily.
It held that based on the
It held that based on the logs, yes. We are awaiting another BIOS from MSI that will move the 109% slider a bit further…
EVGA’s Classi vs Galaxy’s HOF
EVGA’s Classi vs Galaxy’s HOF vs MSI’s Lightning
I second this.
I second this.
The extra cooler does NOT go on top of the other one. It is used when you go Dry Ice or LN2 as the full cover bracket that comes with it can get in the way of mounting plates.
Something I noticed, and
Something I noticed, and maybe I missed it in the review, but I’ve been seeing a lot of problems with benchmarking on Windows 8 and it not being accurate due to Real Time Clock issues?
Is that something that was noted in the article or was the hotfix applied?
Why for the dirt and skyrim
Why for the dirt and skyrim config page screenshot do you show 1080p but show results in 1440p?
Settings screenshots show the
Settings screenshots show the SETTINGS but the resolution is variable. Many times we test at 19×10, 25×14 and 57×10.
What is the max the voltage
What is the max the voltage can be set to?
1.35v I believe.
1.35v I believe.
1.212v in bios on P00
1.212v in bios on P00
100mv in after burner (SE version) with voltage control enabled in settings.
you can use Rbby258 ABVoltmod to raise overvoltage in after burner above 100mv
but you won’t be able to save.
i made my own bios which allows me to get better clocks then the MSI bios.
but i still need to find a way to raise the 1.212 too in bios itself.
actually i am busy on it right now:)
i was doing some reading to get more research on where it overrides the bios chip in regedit.
still cant find it:(
but i will
Thanks for your marvelous
Thanks for your marvelous posting! I really enjoyed reading it, you might be a great
author.I will be sure to bookmark your blog and will eventually come back
down the road. I want to encourage you to definitely continue your great writing,
have a nice day!
Here is my weblog … insurance car
GPU-Z does not misreport the
GPU-Z does not misreport the boost clock, but neither is the boost clock is not the actual maximum speed of any Kepler part.
The discrepancy is because of GPU Boost. Effectively, Kepler has an additional boost on top of that boost clock, which comes into play if the card is still within certain thermal and power thresholds.
I’m sure you guys have noticed from well before that just about every GTX670 runs way higher than its rated 980MHz boost clock.
Nice review Ryan, I really
Nice review Ryan, I really like the design of this card and some very nice performance numbers there. Just like to also say you guys at pcper.com do a great job with your hardware reviews.
Surely at that price point
Surely at that price point they could have gone 4GB?
Seems needlessly gimped on paper.
Otherwise, solid review.
The GK110 chip on the GTX 780
The GK110 chip on the GTX 780 and GTX Titan has a 384-bit memory interface which means it’ll only support VRAM of 1.5GB, 3GB, 4GB etc. As far as I know, NVIDIA has restricted board partners to only have 3GB on the GTX 780’s to better differentiate them from the GTX Titans.
Sorry. I meant 1.5GB, 3GB,
Sorry. I meant 1.5GB, 3GB, 6GB.
At first, thank you for this
At first, thank you for this review, Ryan. 🙂
For reading out the actual GPU Boost Maximum you can use Orbmu2k’s NVIDIA Inspector:
direct download: http://download.orbmu2k.de/download.php?id=51
This multi functional tool btw. is also kind of a holy grail to image quality lovers allowing you to set SGSSAA with custom AA-bits in the profiles of DX9 games or enhance ingame MSAA to SGSSAA (not that good in DX10+, driver based downsampling combinded with ingame-AA looks better in that cases). But there are a lot more options. Another cool thing is that you can also use custom SLI-bits if the scaling isn’t that good by default NV profiles when there are no official updates available from Santa Clara. It also comes with some nice monitoring graphs or mdps (multi display power saver). With mdps you can force the GPUs to clock in P8 or P12 state @ Fermi, and here mdps is very usefull – even with one display, allowing you for example to watch 720p material running smoothly @ P12.
I made some screenshots of the tool’s options. 🙂
Note that for overclocking you need to run Inspector as an administrator since NV driver branch R325.
P.S.: You don’t need to use a manuell negative LOD bias adjustment for SGSSAA in most cases any more because NV sets it by default for all possible levels when setting SGSSAA.
Another cool feature is that you can export and import all customized Driver profiles as .nip files – just check the top bar at Profile Settings, add new .exes to a profile, delete custom Driver profiles when NV added them and so on…. 🙂
I recommend the following forums threads when you are interested in sparse grid anti alaising or custom SLI-bits:
german 3D Center forums
I forgot to mention that when
I forgot to mention that when you want to use OC and don’t want to accept the warnings all over again, that you should make a link to the nv inspector.exe, right click -> properties -> target -> add ” -disableWarning ” like shown below.
It’s too early to tell how
It’s too early to tell how good this card is going to end up. The 680 Lightnings were strong cards, but when they got released, they did not even had the voltage lock.
Air cooling tends to top out at about ~1.4 to 1.425V, so judging by Ryan’s results, a mid 1300s OC is achievable, with a 1400 perhaps for benchmarking. That will represent the upper limit of what air cooling will do.
Out of curiosity Ryan, how was the cooler? Is this new 3 fan design better than the 2x 100mm fans of the previous generation?
any input will be helpful
any input will be helpful please.
Okay need some help please? I have just recently upgraded my rig.
• Asus ROG Maximus VI Hero Z87 ATX LGA Socket 1150
• Intel Core i7-4770K Processor
• Corsair Hydro Series H110 Liquid Cooler
• Kingston Hyperx 120gb ssd
• Seagate 1TB Serial ATA HD 7200/64MB/SATA-6G (Back up and Games)
• Kingston HyperX Beast 32GB (4 x 8GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800)
• XFX Radeon HD 7970 Double D 3GB DDR5 X2 In crossfire
• CORSAIR Professional Series Gold AX1200 (CMPSU-1200AX)
• Win 7 Ultimate
So here is the issues my graphics cards running in crossfire I just don’t seams that I get the performance that I should be getting low frames rates in servile games?
1. Current Driver for the Graphics card I have used Both, Not really much improvement with either of them.
The 13.4 Catalyst driver (5/29/2013)
And 13.10 Catalyst Beta driver (09/05/2013)
• Assassins Creed 3 less than 60fps while playing
• Batman Arkham City less than 60 fps
• Crysis 3 Same thing
• Battlefield 3 runs good over 70fps
• Bio shock Infinite runs good over 70fps
• Total War: Rome II on average about 30 to 40fps
So are they just S*&t cards or what, so I have been Kicking around the idea of just buying a MSI N780 LIGHTNING GeForce GTX 780. And selling my AMD cards.
So my question is will I SEE A NOTASBLE PERFROMANCE INPROVMENT with this card over my AMD cards? And is it worth the money? Now first I need you to understand that I work a lot of hours and when I am home I just want to turn on my computer and game not really in to the over clocking thing a lot now I will OC my chip but other than that just want to game. Now if there is a simple fix for my current Graphics card that fine but really just want the performance sit down and go? HELP???????
You’re using AMD cards. Thats
You’re using AMD cards. Thats the problem.