More Mantle Results
So what happens if we use same map, SP_Dam, and replace the Core i7-3960X with the AMD A10-7850K APU? Since the x86 portion of the AMD processor is significantly slower than the 3960X, I would have expected to see some improvements with Mantle.
Click to Enlarge
Click to Enlarge
The average performance of the Mantle system is again lower than the DX11 API test runs by about 6%. That is a noticeable deficit in performance. I do think the minimum frame rate is worth mentioning here as the Mantle version is 16% higher than the minimum of the DX11 tests.
Click to Enlarge
Click to Enlarge
The green line of the A10-7850K under DX11 clearly has a lot of CPU time issues and stuttering that are again relieved by the Mantle API. This marks two instances now that Mantle has lessened the average frame rate of our SP_Dam test run but at the same time lowering the CPU bottleneck.
A Best Case Scenario
After talking with AMD about these performance results, they suggested I try a multiplayer match that would inherently be more CPU limited. While that is indeed the case, the problem with testing MP matches is one of repeatability. Even with the best intentions of running the same route and looking at / shooting at the same things, when other people are involved you can be sure that something is going to go awry. As such, I have moved away from using MP levels in benchmarking. We depend on reliability of results because differences of 2-5% can determine a "winner" in a GPU comparison.
That being said, I decided to run through a dozen or so matches on a server using the MP_Naval map, otherwise known as Paracel Storm. These were done with the A10-7850K APU based system to help accentuate the Mantle performance delta one more time. First up is 1920×1080 resolution.
Click to Enlarge
Click to Enlarge
These graphs are a lot more crowded thanks to the longer play time, but the results are pretty impressive. At 1920×1080, the Mantle enabled version of BF4 is running 27% faster in average frame rate.
Click to Enlarge
Click to Enlarge
Both CPU and GPU time are much more consistent with the Mantle version of the game and the 14.1 beta Catalyst driver.
Now, let's take a look at the same tests at 2560×1600 resolution.
Click to Enlarge
Click to Enlarge
At this resolution, the frame times using Mantle are actually quite a bit more consistent and that is somewhat unexpected. I anticipated that the more GPU-bound nature of this resolution would result in less of an advantage for Mantle. In this particular comparison we are seeing a 32% better average frame rate with the Mantle version of BF4 than with the DX11 version – a huge increase!
Click to Enlarge
Click to Enlarge
GPU time and CPU time are MUCH better with the Mantle version, as you would expect, based on the frame time and average FPS results above.
Lingering Beta Problems
Even though the current status of Catalyst 14.1 Mantle driver is impressive in these above two multiplayer comparisons, there are other issues that really put this software release deep into the realm of "beta". For one, as AMD says in its documentation, "intermittent stuttering or stability issues may occur when utilizing Mantle with AMD CrossFire technology in BattleField 4." That is a drastic understatement, in my experience, as I was unable to get CrossFire working in a single startup of Battlefield 4 be it single player or multiplayer. Results ranged from crashes back to the desktop to hard locks of the system that required a cold boot. Clearly that isn't acceptable but, at this point with Mantle, CrossFire implementation falls completely on the game developer to implement at the engine level. While that allows for some very powerful implementation options, it also takes the reliability aspect out of AMD's hands for the most part.
Even more concerning is that even if you can get CrossFire working, frame pacing isn't implemented at this time by the Frostbite engine. CrossFire users that actually can get the game to run in a stable environment will not get the smooth performance advantages of properly paced timeslices. Instead, they will be essentially going back to the days before Catalyst 13.8. I don't think we want to go there again, do we?
Even without thinking about multi-GPU configurations, there are still some stability problems with the single GPU implementations. On numerous occasions, I found myself back at the desktop or locked up during a map load. It's hard to know how much of this is specific to the Mantle version of BF4, as opposed to the continued growing pains of the title, but either way its not a great experience in many cases.
The fact that this iteration of the Mantle libraries is geared for the R9 290X, R9 290 and R7 260X also means that users of the HD 7000 cards as well as the R9 280X and 270X cards will get SOME improvement but less than they should expect moving forward. Again, AMD wanted to impress upon me that this is a beta release, but it still feels like these are compatibilities that should be in the software stack from day one.
Analysis and Closing Thoughts
So what can we take away from this very early, very basic overview of AMD Mantle and its implementation with Battlefield 4? Not much to be honest. In our worst case scenario results on the first page, which just happened to be our standard BF4 testing methods before Mantle was in the picture, Mantle actually was a tiny bit slower than the DX11 code path. AMD was concerned about this result and said they had not seen such results in its own testing; apparently, they are looking into it.
Our best case scenario resulted in huge performance gains for the AMD hardware when tested on multiplayer maps that have a lot more work for the CPU to do. 27% and 32% average frame rate increases are clearly a very important data point for AMD although you have to wonder, based on our two wildly different result sets, which of these is going to be the norm and which will be the outlier.
More than likely the truth lies somewhere in the middle.
Results are promising and I am not usually one to recommend holding stuff away from the public, but I really think that AMD did a disservice to the Mantle product by releasing it now. Yes, they were meeting the company's (revised) release time frame, first announced in September, and yes, the community would have been on AMD's ass had they just delayed the Mantle software indefinitely. But, releasing the driver with limited GPU support and a lack of consistent support for multi-GPU configurations (along with some serious stability caveats) just seems too early. Compared to other "beta" driver releases from AMD, this seems much more like an alpha. A functional, somewhat impressive alpha.
But, the secret is out. The Catalyst 14.1 beta driver is going to be available for the public as of midnight tonight. Let's see what the community thinks! Will our comments and forums blow up with positive early results, or will they be filled with frustrated gamers that are wondering if their trust in the Mantle mantra was misplaced.
With all due respect, Ryan. I
With all due respect, Ryan. I like the reviews but please just put a little more time in your plots. They have always been very fuzzy and faded.
And when you make a typo (like 1920×1080) couldn’t you go back to Excel and change the title?
I am not trying to tell you how to run your site. But, I can’t help but wonder, don’t you have a little “perfectionist” in you? Don’t you take pride in presenting your work?
Glad I read this…DX 11 is a
Glad I read this…DX 11 is a standard way of doing things , and this Mantle sounds very limited.
It`s not going to change the world.
Engadget said…AnandTech and HotHardware have used almost-final Mantle drivers to achieve frame-rate gains of at least 7-10 percent in BF4, rising to 30 percent with some configurations, by doing away with the need for Microsoft’s relatively inefficient DirectX drivers.
Ryan tells us what it really does.
My experience so far
My experience so far (i5-2500K @ 4.3GHz, MSI HD7950, 8G DDR3-1866,1080P, mostly Ultra settings, Win7pro x64): I ran a 10 minute run on Siege of Shanghai under both Mantle and DX11, logging the runs with the built in performance monitor. DX11 was ~66fps and Mantle gave me ~73fps. An increase of ~10.6%. HOWEVER, in the latter few minutes of the run, Mantle starts to hiccup and stutter annoyingly bad.
Checking the log file shows that the Max CPU Time keeps spiking, as high as 776ms (~1.3fps minimum), frequently (40+ instances over 10min) saw spikes above 100ms (10fps) whereas the Max GPU Time only got as high as 47ms (~21fps minimum). At first i thought it may be another background program using the cpu to do misc work, so i closed programs and minimized the number of necessary startup programs and retried. Still get the awful stutter. I set BF4 to High Priority in the task manager to prioritize it over other cpu usage and still get the stutter. The Render.DrawScreenInfo option shows 3 job threads on my 4core processor. Not sure what the issue is.
My DX11 runs, while having a lower average fps, gave me much more consistent frame rates. Max CPU Time only spiked up to 54ms (18.5fps min), Max Frame Time up to 44ms (22.7fps min), and Max GPU Time up to 27.9ms(35.8fps min)
Perhaps this is part of the 7000 series optimizations they mention, but it seems like it isn’t the GPU that is having issues because GPU Time under each API is low without spiking. I think if i can fix the CPU Time spikes, i’ll see a bigger performance increase under Mantle. I’ll see if i can force it to use a 4th cpu thread and retry also. As it stands, i think i’m going to wait for another patch or two before i really switch over to Mantle.
Ryan, if you read this, can you run your tests out further than a minute on future testing? My first major spike doesnt come until about 4min in. Maybe its an issue that manifests itself over time? Others are speculating that its a PunkBuster issue, where its trying to take a system screenshot. Some also suspecting GPU clocks changing due to power requirements or heat.
Can you guys do 64 player
Can you guys do 64 player server comparison between DX 11 and Mantle.
In 64 players in DX 11 there are major dips in frames under heavy action.
Am curious to see how Mantle min fps in high load scenarios does.
http://www.youtube.com/watch?
http://www.youtube.com/watch?v=2CQ2aq8yaN8
Watch that video ^
#fuckamd #amdisfornoobs
Someone should have told that
Someone should have told that guy 290(X)s scale 20% better then 780s in BF4. Then you add mantle boost. Even if he plugs that 4th card in he is going to be slower then a 290(X) tri-fire setup.