2560×1440 and 4K Results, Conclusions

Single GPU Results

Click to Enlarge

Now that we have decided to continue to depend on our capture performance above all else, whenever possible, let's dive into the results. Starting at 2560×1440 with the single card configurations, you can see that both the AMD Radeon R9 290X with the Mantle API and the GeForce GTX 980 result in very similar playing experiences including frame rate and smoothness of that frame rate. The R9 290X has an edge of 73 FPS vs 71 FPS on average when using Mantle but with DirectX 11 the R9 290X falls well back into the low 60s. Even better for AMD, the Mantle version has a noticeably higher minimum frame rate.

Note the time period from 18s to 38s (as we have the entire article) and you'll find that the Radeon R9 290X without Mantle is significantly slower and choppier than both the GTX 980 and the same R9 290X with Mantle enabled.

Click to Enlarge

At 3840×2160, the results are similar but compress a bit. If we look at only the DX11 results, the GTX 980 has a small lead over the R9 290X in average frame rate (40 vs 38 FPS) but that section of issue, 18s – 38s, shows a wider margin in favor of NVIDIA's card than the other portions of the game. The Mantle version running on the R9 290X is actually the performance leader overall with a higher average frame rate and equal smoothness / minimum frame rates.

CrossFire and SLI Results

Click to Enlarge

Things get a bit more interesting in the world of SLI and CrossFire. At 2560×1440, the pair of GTX 980 cards has a much higher average frame rate than the R9 290X cards in either DX11 or Mantle. In fact, NVIDIA has a 47% lead in that area. But it's not just about average frame rates: look at the minimum frame rate and the smoothness of the frame times in the first graph. The orange line of the Mantle multi-GPU result is very smooth, much better than either DX11 result and shows the promise of split-frame rendering when done correctly.

It's a very interesting discussion – would you rather have the much higher frame rate of the pair of GTX 980s or the smoother frame rate of the R9 290X cards running under Mantle? Even though the Mantle score is the lowest result in average FPS of the three, I actually think I would prefer that option!

The result of the DX11 R9 290X CrossFire setup is pretty confusing though – it doesn't even come close to competing with Mantle or the GTX 980 cards in SLI. It has as many spikes and frame variances as the GTX 980 cards but simply doesn't scale as well as NVIDIA's implementation.

Click to Enlarge

Performance results at 4K are even more interesting. The Mantle implementation remains the most smooth of the three shown here but it isn't a runaway leader like it was at 2560×1440. Both of the DX11 multi-GPU solutions show large amounts of frame time variance but also several sections of extremely smooth gameplay.

NVIDIA again has a big edge in average FPS (22% over R9 290X CrossFire DX11 and 58% over Mantle) and can even claim to have the best minimum frame rate. But even that doesn't tell the whole story; the same question I asked above can be applied here. Would you rather have the higher average frame rate or the smoother frame rates provided in the Mantle experience? At 4K the differences in smoothness are slightly less but it is still apparent.

Again it appears that AMD's DX11 implementation of Civilization: Beyond Earth falls well behind the NVIDIA implementation but AMD is able to fallback on the Mantle API and Firaxis' work there to claim a potential victory. You just have to look at more than average frame rates.

Closing Thoughts

My Civilization: Beyond Earth testing turned from something I planned on spending an afternoon on into something I had to dedicate two full days and nights on to make sure we got it right. In the end though I think we got some really interesting data.

First, let's look at Beyond Earth and its DX11 implementations. It seems pretty clear that NVIDIA has the advantage here and the GTX 980 is faster than the R9 290X in both single GPU and multi-GPU testing. The battle of SLI and CrossFire is heavily weighted on NVIDIA's side as the 290X in CrossFire under DX11 shows all of the frame variance that SLI does but with much less frame rate scaling.

But AMD has the advantage with Firaxis decision to implement a Mantle code path for Beyond Earth and the result is a very solid product; maybe more complete than any other Mantle game to date. In single GPU testing, Mantle is able to bring the R9 290X on to the same performance level as the GTX 980 and in the case of our 4K testing, slightly edge ahead of it. In our CrossFire vs. SLI testing its a more complex story; Mantle-based multi-GPU results are smoother but run at lower overall frame rates. The ability for Firaxis to bring back split-frame rendering is interesting even from a educational level but it appears to have some benefits as well in smoothness. And input latency as well. (Just don't forget that INI file setting for Mantle multi-GPU users!)

The decision to turn on EQAA for Radeon cards (both in DX11 and Mantle code paths) seems odd to me and I question the reasoning behind it. I'm always a fan of higher image quality and if a hardware or software vendor can claim to do so without affecting performance, then offer that up. But the options menu should reflect that change either with a text indication or another check box. I know the media wouldn't appreciate AMD or NVIDIA doing this in the background without any out-facing indication so we should hold the software guys to the same standards.

It's fair to say that Mantle is getting better as it ages and the Firaxis implementation in Civilization: Beyond Earth is the most complete at time of launch we have seen. That's great news for AMD! But I can't help but wonder why the DX11 results for AMD in this game falter in the way they do. You could argue that no Radeon R9 290X user would choose to run in DX11 when they could run in Mantle so it doesn't really matter, and that's probably fair, but AMD needs to be sure it is keeping the DirectX side of things moving onward and upward in order to stay competitive on those fall/winter PC titles that will not have Mantle integration.

NVIDIA's results seems to be in-line with expectations (after going through the hassle of the in-game vs. FRAPS vs. Frame Rating debacle on the second page) and the GTX 980 is a strong performer in Beyond Earth. It runs well past the R9 290X under DX11 and is able to come close to the performance levels of the game when competing against the same card running AMD's proprietary API. It might be time to worry a little bit about Mantle's success in recent weeks (Sniper Elite 3 being another) with several upcoming titles having support too (Star Citizen, Dragon Age: Inquisition, Mirror's Edge 2).

But, of course, NVIDIA has its own games with exclusive features including Far Cry 4, Assassin's Creed Unity, Lords of the Fallen, etc.

Winter is coming.

« PreviousNext »