Testing Process and Early Results
Our system setup for this quick look at VR performance uses the $900 VR Build we did last month at its base, with variable graphics cards. That includes a Core i5-6500 processor, 8GB of memory, 250GB SSD and more. We tested with a GTX 970, GTX 980 Ti, R9 390 and R9 Fury X.
One note: we had to find a monitor that would run at 90 Hz (not higher) in order to have accurate representation of the animation inside the HMD on the desktop viewport. That took quite a while to figure out!
In The Gallery we purposefully found a location that was more stressful on the GPU and focused on it. For us, it was the initial campfire area and simply sweeping your view from the fire to the sky shows dramatic performance deltas.
For the GTX 970 and R9 390, Cloudhead games recommends the Low preset.
This data doesn’t look particularly interesting at first. The blue line represents the PresentMon data shown on the viewport window on the companion monitor. There is definitely noticeable variance, but it’s clear that the data is centered on the 11.1ms measurement, hitting the 90 FPS target for both the headset AND our monitor. The orange line shows the GPU render time for the GTX 970 along the same period of time. For each frame, the render time is 5-7ms. That is well below the 11.1ms rate necessary to hit the 90 FPS mark and thus, we see no cases where the frame rate of the game in the HMD is affected. There is headroom in this hardware configuration, with these image quality settings.
Things change dramatically when we go against the developer’s wishes and turn up the image quality settings to High in-game. The orange line shows alternating areas of 22.2ms and 11.1ms frame times, or 45 FPS and 90 FPS, translated. The lower frame rates (and higher frame times) are when we were looking down at the campfire, while the higher frame rates occurred when moving our gaze up to the sky. Notice the blue line of the GPU render times, it hits just around 10-11ms at the points where our view is of the fire, but falls to as little as 8ms when looking at the sky. Our slow gave movement from one to the other shows transition periods where the frame rate jumps back and forth between 22.2ms and 11.1ms frame times. Even though we are often under 11.1ms when looking at the fire, the GTX 970 doesn’t leave enough headroom for the rest of the system to complete its tasks under the required time frame at High settings.
What happens if we throw in the GTX 980 Ti graphics card instead, with the same High image quality preset? The orange line of frame times stays centered around the 11.1ms area, with jitter as a result of Vsync backpressure, keeping the frame rate of the game and the HMD at 90 FPS. The blue line of GPU render times does swing when we move our view from the fire to the sky, but even when looking at the rendering intensive campfire, total render time stays under 7ms or so, giving the rest of the VR platform more than enough time to meet that 11.1ms window.
This graph compares only the total rendering times of the GTX 980 Ti and GTX 970 over similar, but not identical, test runs of the same section of The Gallery at the High preset. It’s clear from this view that the GTX 980 Ti is rendering quite a bit faster. The average frame time of the GTX 980 Ti is 6.11ms (equivalent to 163 FPS) while the GTX 970 averages to 9.37ms (106 FPS). Obviously, that doesn’t tell the entire story about VR performance, because with an average frame rate of 106 FPS for the GTX 970 you expect no difficulty with it hitting the Vsync cap of a 90Hz display. This is where the value of seeing both total GPU render time and external Present() calls is seen.
Of course I tossed in the competitive AMD Radeon graphics cards for testing, the GTX 970 goes against the Radeon R9 390 and the GTX 980 Ti battles the Radeon R9 Fury X.
Comparing the GTX 970 and the R9 390 (on the High in-game quality preset) showed us that both cards exhibit the same 90 FPS to 45 FPS shifts when looking between the sky and the campfire. The GeForce card is slightly faster (in terms of total GPU render time) when looking towards the sky but the AMD Radeon R9 390 looks to be a little faster when looking into the flames. It isn’t fast enough to keep the experience at 90 FPS, however.
Comparing the flagship cards in the same scenario with the High image quality settings puts the GTX 980 Ti on top, when looking up and looking down at the fire. The AMD Radeon Fury X frame times never stay above 7-8ms and it is able to keep The Gallery running at 90 FPS consistently without a hiccup, just like saw in our individual GTX 980 Ti graph above.
So what does all of this data show us? A couple of things. First, getting access to total GPU render time as part of the SteamVR performance heuristics is incredibly valuable and will help us do direct GPU to GPU comparisons with VR games going forward. Second, matching that data with a viewport on the companion display with a properly configured setup can help tell the performance story for VR.