The X399 Platform, Socket, Cooling, and Memory
Along with the new Ryzen Threadripper processors comes a new chipset, a new socket, a host of cooler questions, memory compatibility and more. The new TR4 socket is a beast with 4094 pins in a massive footprint that makes previous processors tiny by comparison. Installation of the processor in this new socket is much more complex than other solutions as well, involving a sled, tray, dual retention brackets, and even a torque wrench. Seriously. You can see the install process in the video I posted last week below:
The new X399 chipset connects to the Threadripper processor with 4 of the 64 lanes of PCIe 3.0 from the CPU and adds support for 2x USB 3.1 Gen2 ports, 6x USB 3.1 Gen1 ports, 6x USB 2.0 ports, 8x SATA channels, and some GPP PCIe 2.0 lanes (x8). With 60 lanes of available PCIe 3.0 on the Threadripper processor even after connecting the chipset, the majority of the connectivity for graphics cards, add-in network controllers, NVMe storage devices, and anything else will be through the processor. That’s great news as it means less latency and more bandwidth between external devices and the host.
One interesting caveat that we are still exploring is how the added latency of die to die communications we mentioned above might come into play with external devices. If you have two graphics cards on full x16 PCIe connections, but one each from each of the CPU dies (each has 32 lanes of PCIe total), then in theory you would have to cross that Infinity Fabric during communications. The same would be true of NVMe SSDs running in a software RAID solution. We are still working out ways to test for this possibility, so stay tuned as we fire up the multi-GPU and multi-SSD benchmarking.
Motherboards based on the X399 chipset will be available from all the vendors you would expect: ASUS, MSI, Gigabyte, and others. Expect these motherboards to come with a similarly high prices, similar to what the Intel X299 and X99 platforms garner. Features like 802.11ad and 10Gb Ethernet will flourish here, but you are going to pay for those flagship features. Being able to utilize all of the connectivity that Threadripper provides means integrating technologies that we just haven’t seen yet.
We have confirmed that X399 and the Threadripper CPU do not support NVMe RAID natively and you want to double or triple your NVMe storage, you are going to be stuck using Windows software RAID. While that is fine for most users, it does mean the array will be non-bootable – kind of a bummer for an otherwise top-of-the-line configuration. AMD tells us they are working on a solution to integrate on-chip array capability but it is not an easy task; just ask Intel and its partners that have been struggling to get it perfect for a year or more.
There was a lot of concern over the stability and maturity of the X399 chipset and motherboards based on after the Ryzen 7 and X370 launch. I am happy to report that through a couple of weeks of testing and banging on it, our ASUS X399 Zenith Extreme motherboard has been rock solid, with no crashes and no stability concerns that weren’t related directly to overclocking. Drivers seem to be working, the headaches and confusion around memory support have been ironed out, and it just feels like a more robust system than Ryzen did at launch. Clearly the team at AMD has learned from that situation, and it worked with board partners to help make today’s release a solid one. No doubt that engineers from ASUS, MSI, Gigabyte, and others have a bit more time with the architecture under their belt as well, adding to the improvement.
Memory support for Ryzen Threadripper is listed in the reviewer’s guide as follows:
The highest memory speed supported without “overclocking” is 2667 MHz, and that is when running four DIMMs (one in each channel) that are single rank. If you want to populate all 8 DIMMs on Threadripper with single rank modules you will be limited to 2133 MHz, a significant reduction. Moving to dual-rank modules those speeds are 2400 MHz and 1866 MHz. Ouch. Thankfully, overclocking the memory on these platforms does not yet appear to be a concern. I have been running four DIMMs (single rank) at 3200 MHz on Threadripper without a hiccup and the performance improvement has been impressive. Just as we found with Ryzen, memory speed matters more here than on any previous AMD or Intel platform.
With a 180-watt TDP, cooling the Ryzen Threadripper is going to be a critical task. The heat spreader on the CPU is massive, unlike anything we have seen before in the consumer space. Luckily, AMD is telling us that it’s not imperative that users cover the entire area to get effective cooling. In our experience, that is definitely the case. AMD sent along a brand new Thermaltake water cooler with a 360mm radiator that uses a standard Asetek “teeth” style mounting bracket. Using the bracket that came with the Threadripper CPU, mounting the cooler was as simple as any other platform, with more concern over the placement of the thermal interface. Kyle over at HardOCP has already done a great job looking at the surfaces, mating process, and TIM application if you are interested in more.
Full coverage coolers are on the way, as you would expect, though I don’t think we will see them in large volumes. Because the larger size would likely preclude the same coolers from working on other motherboards, going all-in on a Threadripper cooler would be a risky proposition. Instead, I expect to see the likes of Corsair and others doing a lot of in-house testing to find the best way to adapt coolers to the TR infrastructure.
Measuring the thermals of Threadripper still requires knowledge of the idea of an offset. The Threadripper processor, like Ryzen 7/5/3 before it, tracks two different thermal sensors.
- Tjunction (Tj): the average true temperature at the interface point between the processor die(s) and heatspreader.
- Tcontrol (Tctl): an AMD proprietary offset that is designed to govern automatic fan policies. The Tctl offset for the AMD Ryzen™ Threadripper™ processor is +27°C to Tjunction.
The earliest BIOS’ we encountered only reported the Tctl temperature to us which idles at around 55C. Taking into account the offset, our CPU was running at around 28C at idle state – more in line with expectations. AMD recommends using HWiNFO v5.55-3210 (or later) for temperature monitoring as it has built in support to compensate for the offset for Threadripper.
Temperature Testing
If you want a deep dive into temperatures on Threadripper and even the best way to apply thermal paste, I recommend you watch this 22 minute discussion on it from our buds over at Gamers Nexus. It’s definitely worth the time.
For what we have seen using the HWiNFO64 application, the AMD Ryzen Threadripper processor definitely will be a hot and power hungry CPU. Using the supplied 360mm cooler from Thermaltake that shipped with our review kit, temperatures weren’t really a concern during standard testing, even after extended periods of full utilization.
Remembering all the way back to the first page of our story, Threadripper has a 27C offset between the Tctl and Tdie measurements to help fan and cooling systems better manage temps. Under a full load of Blender, the 1950X stabilized at a temperature of 59.3C. That’s good considering the high core count and the inability to cover the full heatspreader for today’s cooling solutions.
For comparison, the 7900X is using a 240mm cooler but still maintains a temperature under 50C when not overclocked.
With a 180 watt TDP, buyers should not underestimate the cooling capacity needed for Threadripper. Expect to need at least a 240mm radiator liquid cooler, but I would assume anyone considering a $999 processor would be going down that path anyway.
AMD still maintains the Ryzen power plan for Windows to help optimize the CPU states for best performance without sacrificing power consumption drastically. We did testing of this power plan back in April with Ryzen 7 and found that it did indeed help performance in some cases with changes made to help minimize the parking of cores, and to also speed up the waking of them. If you install the latest AMD driver package, this profile will be automatically installed on your system and selected. As far as I am aware, no changes were made to this profile to adjust for the NUMA/UMA memory configuration.
I’m very curious on how will
I’m very curious on how will the two dies and memory modes affect virtualization? I’ve only experimented with VM in the past but is it possible to run two Hexa-cores windows VM and with each individual memory nodes assigned to each VM?
Are you setting the Blender
Are you setting the Blender tile sizes to 256 or 16/32?
Just wondering since an overclocked 5960x gets 1 minute 30 seconds on the BMW at 16×16 tile size. Significant difference that shouldn’t just be a result of the OC.
For reference: 256 or 512 are for GPU and 16 or 32 are for CPU – at least for getting the best and generally more comparable results to what we get over at BlenderArtists.
When reading is not enough,
When reading is not enough, the mistakes are OVER 9000!
“If you content creation is your livelihood or your passion, ”
” as consumers in this space are often will to pay more”
” Anyone itching to speed some coin”
” flagship status will be impressed by what the purchase.”
” but allows for the same connectivity support that the higher priced CPUs.”
“”””Editor””””
Now just point me to the
Now just point me to the pages… 😉
Nice to see a review with
Nice to see a review with more than a bunch of games tested. Keep up the good work!
Should not a test like 7-zip
Should not a test like 7-zip use 32 threads as max since that is what is presented to the OS??
now it only uses 50% of the threads on TR but 80% on i9-7900x.
Silly performance, looking
Silly performance, looking forward to the 1900X and maybe 1900.
I sometimes wonder why nobody
I sometimes wonder why nobody ever points out that within CCX (4 cores that can allow a lot of games to run comfortably) ZEN has latencies of half those of Intel CPUs. Binding a game to those 4 cores (8 threads like any i7) has significant impact on performance. It does not change memory latencies of course but core to core is much better.
I’m glad someone else noticed
I’m glad someone else noticed this besides myself. I noted this during the Ryzen launch & quickly noted that by using CPU affinity along w CPU priority to force my games to run exclusively within 1 CCX & take advantage of using high CPU processing time on these same CPU cores I could take advantage of this up to a point.
What all this shows to me is that the OS & game developers software need to be revised to better handle this architecture at the core logic level instead of usersAMD having to provideuse methods to try to do this that cannot be used in a more dynamic fashion. I’ve ran some testing on Win 10’s Game Mode & discovered that MS is actually trying to use CPU affinity to dynamically set running game threads to be run on the fastestlowest latency CPU cores to “optimize” game output thru the CPU but it still tends to cross the CPU CCX’s at times if left on it’s own.
What I’ve found is by doing this my games run much smoother w a lot less variance which gives the “feel” of games running faster (actual FPS is the same) due to lower input lag & much better GPU frametime variance graph lines w very few spikes….essentially a fairly flat GPU frametime variance line which is what you want to achieve performance-wise.
Just to note….my box is running an AMD Ryzen 7 1800X CPUSapphire R9 Fury X graphics card w no OC’s applied to either the CPU or GPU.
It’s a step in the right direction but needs more refinement at the OS level……
As expected, performance per
As expected, performance per dollar is crap in single threaded tasks, which most workloads are. Games don’t even use more than 1 or 2 cores.
Yea games only use 2 cores
Yea games only use 2 cores lol
http://i.imgur.com/Hg3Ev5p.png
And “as expected”, we have
And “as expected”, we have yet another Intel shill complaining about gaming performance on a production CPU, which isn’t made for gaming (although it’s not bad in the least and has a longer future as devs code for more than Intel’s tiny core count (under $1000))..
-“performance per dollar is crap in single threaded workloads”…
Well, since these aren’t sold as a single or dual core CPU, performance per dollar as a unit is beyond everything on Intel’s menu.
– “Games don’t even use more than 1 or 2 cores”
Well, I’ve been using a FX-8350 for 2 years now, and I always see all 8 cores loaded up on every single game I play (and I have many). Windows 10 makes use of these cores even when it’s not coded in programs. It would work even better if devs started coding for at least 8 cores, and I believe they will start doing this in earnest now that 8-core CPUs are now considered average core counts (unless you’re with Intel).
You would have been better off stating that core vs core is in Intel’s favor on the 4-core chips and some others, but ironically the “performance per dollar”, as you mention is superior with AMD.. in every way.
What memory are you using,
What memory are you using, and could you name a 64GB kit that works in XMP? And why 3200Mhz over 3600?
Intel is still superior both
Intel is still superior both in raw performance and in perf/$. If you were being objective you wouldn’t have given slapped an editor’s choice on this inferior product.
In Handbrake the 1800x is 40%
In Handbrake the 1800x is 40% slower than the 1950x and in reverse the 1950x is 67% faster than 1800x.
Open cinebench with a TR or
Open cinebench with a TR or even an 1800x. Show me any Intel chip that can come within 20% of the 1950x. The entire Ryzen 7 lineup is king of the “perf/$” category. 1800x = $365 on eBay right now. Look how close it matches with Intel products that are double the price or worse.
If you want to compare single core perf vs Intel, you can win an argument.. at the cost of very high power draw and even worse cash draw. Perf/$ is a dead argument for any Intel fanboy. Find something else. BTW, are you also commenting under “Thatman007” or something? Sound like the same Intel mouthpiece.
Sorry for necroposting, but
Sorry for necroposting, but it really belongs here:
The recent Meltdown vulnerability and its performance implications on Intel CPUs pretty much leveled the playground now. After reading the article and all the comments above I opted for a very good B350 motherboard and a Ryzen 1800X to replace my Core i7 5930K (Haswell). Reason is that my CPU will likely be hit very badly performance wise by the upcoming Windows 10 security update. Intel should pay back 30% to all affected CPU owners, actually…
Reason is that likely I would not gain anything from NUMA, except of the additional complications. So I opt for the easier to manager (lower) power consumption and less noise from cooling as a result.
Thank you for collecting all the great info.