Performance Testing
Once we had the AKiTiO Node up and running, we did what any excited person would do, install a GTX 1080 Ti and see how we could push gaming on an ultrabook-class laptop.
For our testing here, we used the Lenovo ThinkPad X1. This thin-and-light notebook features a 15W Intel Core i5-7300U processor, which is very common in these sorts of notebooks. Beyond that, is equipped with 8GB of RAM and a 256GB SSD, certainly a very nice notebook but not something you would ever attempt to game on.
For our games, we tested The Witcher 3, Hitman (in DX12 mode), and Dirt Rally all on 1080p at the maximum image quality settings.
Taking a look at our average FPS score, you would be quite impressed. It would seem that gaming on a low power notebook is finally possible!
Well, as we've learned over the years, average FPS is far from the entire story when it comes to gaming experience. As we ran through our initial testing, we found that games were extremely stuttery and difficult to play. So, we decided to take a closer look at things using our Frame Rating capture system for analysis.
As you would expect, our first culprit for a stuttery gaming experience was the low power CPU in the ThinkPad. To evaluate this, we decided to do something that no one would normally do – hook up a GPU over Thunderbolt to a high power gaming desktop.
In this case, we are comparing the GTX 1080 Ti on a Core i7-6900K-based system with the graphics card both installed natively and operating through the AKiTiO Node and the ASUS ThunderEX 3 expansion card installed into the X99 motherboard.
In Hitman, we see a difference of about 40FPS in the 50th percentile (average) metric across the 1080 Ti installed in the motherboard versus through Thunderbolt 3. Additionally, the frame time and frame variance charts show a significant reduction in smoothness when moving to Thunderbolt 3 for GPU connectivity.
The Witcher 3 shows a similar difference in 50th percentile FPS, but the frame variance and frame times are much closer between the internal and external GPU configurations.
Dirt Rally shows a different side of the story. While the 50th percentile FPS is much higher on the internal GTX 1080 Ti configuration, the Thunderbolt configuration presents a much smoother gameplay experience.
Next, we decided to compare our Thunderbolt 3 results from the i7-6900K desktop to a more real world scenario with the Lenovo ThinkPad X1 Carbon with the same games and settings.
Going from the desktop setup to the ThinkPad, we see another significant reduction in 60th percentile FPS – about 50 FPS this time. Additionally, the stutter is all over the place! Looking at the frame variance chart, we can see frame variance of 6+ ms from the 90th percentile and above.
The Witcher 3 sees less of a gap in FPS by percentile going from the i7-6900K to the i5-7300U in the ThinkPad. However, the frame time chart shows major variance between these two different setups. Frame variance is better than we saw in Hitman, with the 90th percentile coming in at around 4ms.
Dirt Rally is a much closer matchup of the two configurations. While we see more stutter on the ThinkPad, the frame variance stays well below the other two games we tested and is significantly more playable.
great piece
thanks so much
great piece
thanks so much ken
what is the reason for the variance and stuttering?
i thought tb3 had enough bandwith?
is it a controller issue of some sort?
thanks in advance
stuttering could be due to
stuttering could be due to combinations of what you mentioned as well as latency due to the length of the round trip.
Did you experience different
Did you experience different performance using the laptop display Vs an external monitor? I heard that sending the image back to the laptop reduces performance.
We can’t capture video
We can't capture video directly off of the laptop display to put it through the Frame Rating analysis, so all of the benchmarks here you saw were with an external monitor. I will say from an experience point of view, using the display on the notebook seemed to be equally as stuttery.
after the comment on probably
after the comment on probably wanting a Quad-Core CPU, I kinda want Ken to retest once that Asus Laptop with a Ryzen 7 1700 in it comes out.
He can’t test the Asus laptop
He can’t test the Asus laptop with the Ryzen 7 1700 since thunderbolt 3 is limited to Intel laptops as of now.
Laptop OEMs and their
Laptop OEMs and their motherboard suppliers are free to purchase TB3 controllers from Intel and use them in any product. So if the laptop makers want to buy a TB3 controller for any Ryzen 7 based laptop then Intel will gladly sell them one. Apple announced its macbooks are going to offer external TB3 based GPU support. So that’s probably going to be for Apple’s current Radeon based products. Also see AMD’s XConnect announcment(1). Intel is is the TB3 business to sell TB3 controllers so Intel is not stopping anyone.
Now if Intel(Intel sells a dual TB3 controller SKU) could only provide TB3 link aggregation and if laptop makers would provide enough PCIe links dedicated to only TB3’s bandwidth needs then external PCIe may not be so bad.
Those not so great frametimes/latency on the external TB3 docks is probably related to the time it takes the TB3 controller to encapsulate the PCIe protocol inside the TB3 protocal(Tunneling protocol) and send it over the TB3 cable including the TB3 controller on the external dock reversing the process to deliver the PCIe packets to the external GPU.
What is up with PCI-SIG’s OCuLink external standard that’s stright up external PCIe with no encapsulating/de-encapsulating steps required that add latency.
(1)
“More on AMD’s XConnect Tech: Thunderbolt 3 for External Radeons”
http://www.anandtech.com/show/10133/amd-xconnect-external-radeons
P.S.
“Thunderbolt Goes
P.S.
“Thunderbolt Goes Royalty-Free as Intel Repositions It to Boost Adoption”
https://www.extremetech.com/computing/249902-thunderbolt-goes-royalty-free-intel-repositions-boost-adoption-sales
I have a Dell Alpha r2 with
I have a Dell Alpha r2 with an Nvidia GTX Titan black plugged into the graphics extender. It screams. Everything I throw at it on my 1080p 3D enabled monitor at 144hz works fantastically. Of course the graphics external box is using a proprietary connector that much lower latency and throughput than thunderbolt 3. The graphics extender also has 4 usb ports on it to. Overall I’ve very happy with this set up. I imagine a thunderbolt 4 connection, when ever that happens, will be better suited to the high demands of gaming graphic cards. But Dell still has the only viable, but proprietary solution, for plugging external graphics cards into micro desktops and laptops.
Still too expensive.
There
Still too expensive.
There needs to be a simple DIY kit based on an ITX form factor mobo, so you can run your own case/psu.
Ken great write up finally
Ken great write up finally the dream of many is becoming reality
https://www.wired.com/2017/05/intels-plan-thunderbolt-3-things/
I await next year’s round of amd/intel/*insert chip manufacturers here* thunderbolt devices 😀
Linus did an in depth
Linus did an in depth comparison of these egpu options as well and noticed dramatic improvements using an external display rather than the internal laptop display. The only really good egpu options out right now is the proprietary eGPU setup that Dell is doing with the Alienware laptops. I too have owned this setup and only saw maybe a 5% difference in performance than what I had with my desktop running a Titan Xp in it. I had similar results with a GTX 980ti, and GTX 1080 that i tested. I have heard that Apple’s egpu setup is now the next best behind Alienware and isn’t throttled as much when using the internal display as long as you are using the Macos High Sierra Beta. I really want to see someone run a 1080ti on a mac with the beta, and see compute results, or test the new Vega FE and produce some final cut render benchmark results vs the stock internal GPU and the RX 580 and Vega.
All of the benchmarks you see
All of the benchmarks you see in the article were using an external monitor so that we could capture the video and put it through the frame rating analysis.
I would love to get my hands on the Dell implementation to see if it's any different!
Did you take into account
Did you take into account differences between the TB3 controller hooked up to the CPU pcie lanes versus those hooked up to the chipset via DMI?
> we decided to do something
> we decided to do something that no one would normally do – hook up a GPU over Thunderbolt to a high power gaming desktop.
But have you tried plugging the video from the thunderbolt gpu into a *second* ThunderEX card, so you could have thunderbolt plus video via a single cable? What if someone has this use case but nobody has tested it??
Good article Ken! The
Good article Ken! The results aren’t perfect, but the price coming down, and performance improvements over ultra-light laptops still show this is moving in the right direction.. $300 for the device + a $200 GPU = could be a big upgrade for a light laptop..
So, good if you want just
So, good if you want just 30fps (console equivalent), right?
My problem with this kind of
My problem with this kind of solution is that it’s not much smaller and not much cheaper than a mini-ITX PC. It still has some benefits, but in the end I think that a full desktop PC is better than a static docking station of equal size.
What I’d like to see in the future is a new class of external graphics cards, coupled with a standard for an external PSU (at the low end the PSU could use thunderbolt to connect, which supports up to 100W, although that could be a little confusing).
I see eGPU boxes as an intermediate solution. Zotac’s Thunderbolt 3 External Box is probably the largest solution I’d consider getting.
It would be nice if users
It would be nice if users could network PCs/Laptops/Mini PC SKUs over TB3(At full 40Gbs bandwidth and not just that new TB3 ethernet mode) and have the OSs cooperate like as if the LANed up via TB3 systems where a computing/gaming cluster like platform. These expansion boxes are as expensive as low cost full PCs, so that price will have to fall some more to attract more sales. Most windows based Laptops still do not support TB3, or TB’s previous versions.
I wonder how Ryzen, or Intel, systems could accomodate any PCIe pass-through coming from a laptop to a PC with the laptop using the PC’s GPU. AMD’s GPUs are better for virtual GPU sharing among OSs under KVM/Linux OS(windows, or linux, via KVM) instances(According to Level One Techs). But what about a PC that can allow a TB3 based laptop to make use of the PC’s GPU hardware over a TB3/PCIe interface. That type of useage would be similar to game streaming. Maybe a PC could be configured to run headless and allow the laptop virtural access and headless control to use a TB3 attatched/networked PC’s GPU/other resources, but that appears to be a bit different than simple game streaming.
I have also wanted something
I have also wanted something like that for a while.
It’s possible to make a really small mini-ITX computer (not to mention prebuilt boxes with MXM GPUs). But when you add screen, keyboard and mouse it’s no longer as small.
I kind of think that a combination of ultra portable laptop with a limited CPU (perhaps even something like a Chromebook) which could act as a “dumb” screen/keyboard/mouse when plugged into a separate CPU/GPU box could be really useful.
I personally didn’t consider looking into running something like a hypervisor before, but that could be a way to hack in the functionality into something existing. If it’s possible to create a “shim” client OS that acts as the “remote desktop” on the laptop.