VR Testing, 3DMark Fire Strike and Unigine Heaven 4.0

VR Testing

Allyn here, stepping in to bring you the much-asked-for VR testing and results. Back in our ‘Measuring VR Performance and Experiences’ piece, we compared the output of PresentMon with the GPU Render Time reported by SteamVR:

In the above capture, we changed our viewpoint between a simple and complex scene, noting that once GPU render time exceeded ~11ms (90 Hz), the GPU was forced to shift from presenting a new rendered frame every 11ms to every 22ms once the render time exceeded 11ms. Timewarp keeps the visual feedback of headset movements at a fluid 90 FPS, but the rendered scene drops to 45 FPS actual (here is a video showing that effect):

In the above video, the 970 was unable to render this particular scene at 90 FPS, so it dropped to 45 FPS, but the head panning across the scene remains at 90 FPS thanks to Timewarp. Depending on the game you are playing / what you are doing in that game, this can introduce varying degrees of negative effects to the VR user. While head rotation will always be 90 FPS, it is easy to perceive the transition to 45 FPS renders given other objects and movements within the VR environment. In some cases it is a jarring transition, and the ideal scenario is to simply prevent render time from exceeding 11ms in the first place. SteamVR Performance Test draws a very complex scene and implements a trick called Dynamic Fidelity in order to keep the full renders under 11ms. This same trick is implemented in Steam’s own VR game – The Lab. This game has no adjustable quality settings and instead dynamically adjusts its quality during play.

The above capture was taken while looking around the loading ‘splash scene’ containing a bunch of red valve handles, and the installed Fury X was not fast enough for that engine to continue at its highest setting. The instantaneous steps are where the fidelity level was reduced to preempt a drop to 45 FPS scene rendering. This is good for VR games moving forward as it maintains a more fluid experience while moving through scenes of varying complexity. It does however create an issue for testing, as there is currently no way to monitor the current level of fidelity used by Valve’s dynamic VR engine. End result – we could not use The Lab for this testing, leaving us with our current VR testing staple: The Gallery.

We started by finding a scene that was reasonably demanding to render when using the ‘high’ preset. We then repeatedly ‘looked’ at this same scene with each GPU installed in our High End VR System, logging the GPU Render Time. Below is a three second snapshot of render time with the same scene and quality settings across all four tested GPUs:

To better mesh with the ‘11ms limit’, I’ve set the axis range maximums to match in the above chart. Looking at the data itself, we can see that our slowest card in this test (980) was able to maintain true 90 FPS rendering in a complex scene at the high preset. This means that for the given scene, the quality and experience in the VR is identical across all tested cards. What we are evaluating for is how much more complex that scene could be at 90 FPS renders. Since we are testing even faster cards, the better thing to take note of is how much GPU headroom was available in this given scene:

In this translation of the data, we can see that for the evaluated scene, the 980 had just over 30% headroom, meaning that it could theoretically render a scene 30% more complex before its scene render rate drops to 45 FPS. The Fury X sat a little closer to the 980 Ti, which had 64% headroom. The 1080 takes a huge jump over all other cards and could maintain 90 FPS renders with a scene more than twice as complex. More importantly to note here is that the 1080 turned in those numbers *without* Simultaneous Multi-Projection (it is not implemented in SteamVR at the time of this testing).

GeForce GTX 1080 8GB, VR Headroom, The Gallery – Complex Scene
  GTX 980 Ti GTX 980 Fury X
HTC Vive +28% +60% +39%

The percent gains over competing GPUs are in-line with other tests in this article, but we expect these gains to increase once SMP is implemented in VR platforms and games. Once that happens, we should see a further reduction in GPU Render Time and a corresponding increase in headroom (or a higher fidelity in games that dynamically adjust). We will track those updates as they happen and revisit this testing in a future article.

3DMark Fire Strike and Unigine Heaven 4.0

With VR testing out of the way, let us look at a set of tests from more standard benchmarks like Unigine Heaven and the new 3DMark benchmark. 

I consider these tests to be somewhat of a "best case" for all the cards in our comparison.  We aren't using our frame capture system, we aren't measuring frame latency, nothing like that; I think this should give you an idea of graphics performance if each vendor had the best result for each game.

Our 3DMark results are impressive as well for the GTX 1080, showing a 28% lead over the GTX 980 Ti, a 65% lead over the GTX 980, and a 34% improvement over the Fury X at the Extreme preset. At the Ultra settings, those drop to 25%, 60% and 26% respectively.

Our results in Unigine Heaven show even less improvement for the GP104 part that 3DMark, though the GTX 1080 is still 20% faster than the GTX 980 Ti and 67% faster than the GTX 980!

« PreviousNext »