Lots of good stuff coming out of Intel's press event earlier today. First up is Optane, now (finally and officially) in a DIMM form factor!:
We have seen and tested Optane in several forms, but all so far have been bottlenecked by the interface and controller architectures. The only real way to fully realize the performance gains of 3D XPoint (how it works here) is to move away from the slower interfaces that are holding it back. A DIMM form factor is just the next logical step here.
Intel shows the new 'Optane DC Persistent Memory' as yet another tier up the storage/memory stack. The new parts will be available in 128GB, 256GB, and 512GB capacities. We don't have confirmation on the raw capacity, but based on Intel's typical max stack height of 4 dies per package, 3D XPoint's raw die capacity of 16GB, and a suspected 10 packages per DIMM, that should come to 640GB raw capacity. Combined with a 60 DWPD rating (up from 30DWPD for P4800X), this shows Intel is loosening up their design margins considerably. This makes sense as 3D XPoint was a radically new and unproven media when first launched, and it has now built up a decent track record in the field.
Bridging The Gap chart – part of a sequence from our first P4800X review.
Recall that even with Intel's Optane DC SSD parts like the P4800X, there remained a ~100x latency gap between the DRAM and the storage. The move to DIMMs should help Intel push closer to the '1000x faster than NAND' claims made way back when 3D XPoint was launched. Even if DIMMs were able to extract all possible physical latency gains from XPoint, there will still be limitations imposed by today's software architectures, which still hold many legacy throwbacks from the times of HDDs. Intel generally tries to help this along by providing various caching solutions that allow Optane to directly augment the OS's memory. These new DIMMs, when coupled with supporting enterprise platforms capable of logically segmenting RAM and NV DIMM slots, should be able to be accessed either directly or as a memory expansion tier.
Circling back to raw performance, we'll have to let software evolve a bit further to see even better gains out of XPoint platforms. That's likely the reason Intel did not discuss any latency figures for the new products today. My guess is that latencies should push down into the 1-3us range, splitting the difference between current generation DRAM (~80-100ns) and PCIe-based Optane parts (~10us). While the DIMM form factor is certainly faster, there is still a management layer at play here, meaning some form of controller or a software layer to handle wear leveling. No raw XPoint sitting on the memory bus just yet.
Also out of the event came talks about QLC NAND flash. Recently announced by Intel / Micron, along with 96-layer 3D NAND development, QLC helps squeeze higher capacities out of given NAND flash dies. Endurance does take a hit, but so long as the higher density media is coupled to appropriate client/enterprise workloads, there should be no issue with premature media wear-out or data retention. Micron has already launched an enterprise QLC part, and while Intel been hush-hush on actual product launches, they did talk about both client and enterprise QLC parts (with the latter pushing into 20TB in a 2.5" form factor).
Press blast for Optane DC Persistent Memory appears after the break (a nicer layout is available by clicking the source link).
Reimagining the Data Center Memory and Storage Hierarchy
Intel® Optane™ DC Persistent Memory Represents a New Class of Memory and Storage Technology Architected to Extract Further Value from Data
By Lisa Spelman
We’ve all heard about escalating mountains of data – and yes, there is a tremendous amount of data generated daily that must be stored, secured and organized. More interesting than the amount of data is the value it represents. Value that comes from analysis and the resulting insights. Data may store the next great business opportunity, societal advancement or scientific discovery.
While we’ve made great progress as an industry in providing the infrastructure, tools and best practices to drive this analysis, limitations are also emerging. Not only is the volume and variety of data growing, but the velocity of desired insights is accelerating. To really tap into all of this data, we must remove the bottlenecks that restrict its flow and readiness for processing.
Today, we’re sharing the first in-depth look at how Intel is reimagining the memory and storage hierarchy for application developers and data solution providers with the upcoming introduction of Intel® Optane™ DC persistent memory. Intel Optane DC persistent memory represents a new class of memory and storage technology architected specifically for data center usage. One that we believe fundamentally breaks through some of the constricting methods for using data that have governed computing for more than 50 years.
Unlike traditional DRAM, Intel Optane DC persistent memory will offer the unprecedented combination of high-capacity, affordability and persistence. By expanding affordable system memory capacities (> 3 terabytes per CPU socket), end customers can use systems enabled with this new class of memory to better optimize their workloads by moving and maintaining larger amounts of data closer to the processor and minimizing the higher latency of fetching data from system storage . Intel’s persistent memory will be available in capacities up to 512GB per module.
High-capacity persistent memory in the data center allows applications to run without incurring the latency penalty of going out to storage over the PCIe bus. As developers adapt software, this new memory class is designed to enable cost-effective, large-capacity in-memory database solutions; provide greater system uptime and faster recovery after power cycles; accelerate virtual machine storage; deliver higher performance to multi-node, distributed cloud applications; and offer advanced encryption for persistent data built into the hardware.
These benefits will have a significant impact on real-world data center operations. For example, for planned restarts of a NoSQL in-memory database using Aerospike* Hybrid Memory Architecture, Intel Optane DC persistent memory provides a minutes-to-seconds restart speedup compared to DRAM-only cold restart. On memory-intensive workloads such as Redis IMDB server, Intel’s persistent memory enables higher memory capacities, delivering more server instances at the same service level agreement (SLA) performance when compared to a system configured with just DRAM.
Intel Optane DC persistent memory has generated a lot of excitement among our customers who are targeting the technology to fill the capacity, performance and latency gaps that stagnate current storage architectures. In an effort to streamline broad adoption, we have invested several years building a significant ecosystem of developers and enabling independent software vendors (ISVs) globally to target this new class of memory. Working closely with these partners, we are creating and extending a whole new generation of applications and services that will deliver revolutionary capabilities to data center applications.
Intel Optane DC persistent memory is sampling today and will ship for revenue to select customers later this year, with broad availability in 2019. In an effort to further jump-start broader software development, we are offering developers remote access to systems equipped with Intel Optane DC persistent memory for software development and testing through our Intel® Builders Construction Zone.
As technologies like Intel Optane DC persistent memory come to market, systems architects and developers should consider new methods for data access and storage, and uncover opportunities to remove throughput bottlenecks. These new methods could also result in deriving more value from data. The combination of Intel Optane DC persistent memory with our performance-optimized Intel Optane SSDs and next-generation cost-optimized 3D NAND SSDs with Quad-Level Cell (QLC) technology will further deliver storage efficiency to warm data as an alternative to relying on HDDs.
We invite developers to learn more about Intel Optane DC persistent memory, Intel Optane DC SSDs and Intel 3D NAND, access training resources and apply for remote access to systems featuring these technologies on the Intel Developer Zone.
Lisa Spelman is vice president and general manager of Intel Xeon products and data center marketing within the Data Center Group at Intel Corporation.
Results have been estimated based on tests conducted on pre-production systems, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance.
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go towww.intel.com/benchmarks.
Intel, the Intel logo and Intel Optane are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
May want to rewrite
May want to rewrite this:
“The only real way to get fully realize the performance gains..”
“The only way to fully realize the performance gains… “
Thanks. The offending extra
Thanks. The offending extra word has been sacked.
latency is not mentioned
latency is not mentioned anymore because spectre/meltdown “fixes” absolutely kill the latency benefit of optane. if you look at the latest specsheets, those even mention that “IOPs” figures have been obtained without spectre/meltdown “fix”
1) Any thoughts on a
1) Any thoughts on a DESKTOP solution for this?
2) For those of with older builds (Z77/i7-3770K) I keep hoping for some caching system similar to Optane that uses PCIe at least… I understand the latency is higher but is there no market for this?
(including caching for secondary HDD’s which I believe Optane can now do to some extent)
I understand how Server scenarios are highly optimized environments so even small gains may be worth the cost.
But for desktop where do you think this is all headed in say the next few to ten years?
A combo of:
1) DDR4/5 + Optane
2) Optane replaces SSD + DDR4/5 completely
3) M.2, PCIe, and SATA SSD caching options.
4) AMD’s STOREMI
(i.e. SSD+HDD acts as like an SSHD so data gets MOVED not copied like in Optane cache… has implications to confusing people, backup solutions)
I recently suggested to someone who was investing in Ryzen that maybe they should go X470 to get STOREMI even if it cost a bit more for the motherboard.
Remember years ago when people got all hyped for solutions to move a game into system memory?
It seems to be getting a bit MESSY, though it does appear that more efficient moving/caching is getting a stronger focus, and frankly the OS and applications could do a lot better just on current hardware (including VRAM/System memory swapping).
If you don’t answer, cheers.
Did you and Ryan hold hands at the Solo movie?
I’m sure some form of this
I’m sure some form of this will eventually make it to desktop. Likely first in the form of some higher end platform that shares chipsets with enterprise (like how VROC has trickled down, as bumpy of a transition as that has been). Splitting off a DIMM slot requires BIOS and CPU support, so we’ll have to wait for the hardware to even support the possibility.
I’d personally skip storemi if you are after Optane speeds – yes it does work, but the software layer as implemented robs a decent chunk of the latency gains that Optane offers. Intel’s own platform naturally does better when designed around their own low latency storage.
Speaking generally, while NVDIMM parts would turn in amazing synthetic figures on a desktop machine, you’re running up against diminishing returns as the bottleneck shifts elsewhere in the system. You’d likely only gain a few more percent over Optane when doing typical client stuff like running apps and games, etc.
I’m sure Ryan has seen Solo a half a dozen times already. I’m a SW fan but I’m not in a rush to see it.
The Big Question is when will
The Big Question is when will Micron Start selling their Own QuantX Brand of XPoint so there can be some competition in the market place.
Micron(Co Developer of XPoint) is a number 3 in the list of top DRAM suppliers and do they want to let Intel get too much of the DRAM/XPoint NVM or perststent memory market in that starts to supplant the traditional Server DIMM/DRAM makret.
There are rumors going around that Apple/others may be up for getting a second source for XPoint/DRAM DIMMs via some Licensing and Contracted Manfacturing of NVM/XPoint-DRAM-DIMMS via Micron to keep Intel from cornering the XPoint market. Apple sure has the most resources to finance such development and I’m sure the Cloud Services Companies what a second source for more competative XPoint pricing in the future.
XPoint DIMMS are qoing to be popular in the Cloud Services market the largest customer for DIMMs and that products Ability of Persistent Memory at large capacities to allow for more VM instances and larger in memory data bases to be more efficiently run, without the need to tax existing PCIe infrastructure on current and future server hardware.
The ASUS web page for
The ASUS web page for their Zenith Extreme motherboard says this about their DIMM.2 slot:
“ROG Zenith Extreme’s DIMM.2 module is a bundled expansion card that allows two M.2 drives to be connected via a DDR4 interface.”
Note the mention of a “DDR4 interface”.
I’m curious to know if an NTFS file system partition formatted using an Optane DIMM will perform faster than, say, 4 x NVMe M.2 SSDs in a RAID-0 array? -or- 4 x Optane M.2 SSDs?
Working from raw bandwidth:
DDR4-3200 x 8 = 25,600 MB/second
4 @ 8G / 8.125 x 4 = 15,754 MB/second
A bootable VROC array with the ASRock Ultra Quad M.2 card topped out at 11,600 MB/second, with four Intel 760p 512GB SSDs:
On that basis alone (without factoring in any software efficiencies), an Optane DIMM has a lot more headroom to exploit.
On the other hand, as you have shown with your log graph, Optane latencies are slower than DRAM latencies.
Allyn, many thanks for
Allyn, many thanks for reporting this important development.
Going back to your brief mention of “logically segmenting RAM and NV DIMM slots”, I continue to be reminded of the former triple-channel chipsets.
Without pointing to any particular product or vendor (not yet), I foresee a desktop application that allows a third channel (either single or dual channel), where these Optane DIMMs are formatted as an NTFS file system to which an OS can be freshly installed.
We can already see something very similar in the ASUS DIMM.2 slot, which is positioned directly adjacent to standard DRAM slots on their most recent motherboards.
A general-purpose approach would allow the designer to assign DIMM slots to fast DRAM or to “slow” Optane, according as the intended system design prefers one or the other solution.
Our aging workstation still uses a 32-bit OS with 16GB of DDR2, and the uppermost 12GB are assigned to a ramdisk where a copy of our website is stored. We need to replace that workstation with a new system that has at least 32GB of DRAM: that will allow us to enlarge that ramdisk to 20+ GB with plenty of room to grow.
Configuring Optane DIMMs as an alternative ramdisk is a natural evolution of this technology for desktop systems.
DIMM.2 slot documented at
DIMM.2 slot documented at ASUS Zenith Extreme page:
DIMM.2 with fan holder
ROG Zenith Extreme’s DIMM.2 module is a bundled expansion card that allows two M.2 drives to be connected via a DDR4 interface. You can then add a memory fan to direct cool air to the M.2 drives to prevent throttling, ensuring maximum performance.
If anyone is interested,
If anyone is interested, several months ago we prepared a Provisional Patent Application that conceived a C: system partition formatted using an uppermost region of DRAM.
The basic idea was to add a “Format RAM” option to a BIOS/UEFI subsystem, that supported a fresh OS install directly to a ramdisk:
These Optane DIMMs are a perfect way to implement that idea.
Seriously did you graph the
Seriously did you graph the latency of Zip disks on that chart?
Allyn explains that he used a
Allyn explains that he used a logarithmic scale in that graph, much like the Richter Scale for earthquakes. That’s why really long latency devices can be shown on the same graph with really low latency devices.
Will these run on the x299
Will these run on the x299 platform or is the chipset long dead like x79?
Allyn should be able to give
Allyn should be able to give us the definitive answer to your excellent question.
FWIW, the last time I looked, Intel was reported to be requiring chipset tweaks. Why that may be the case, I still do not know.
For myself, I was expecting Optane DIMMs to be plug-compatible in standard DDR4 sockets.
Perhaps Allyn could attempt to contact one or more of the alpha testers inside Intel, and report his findings to us here.