Introduction, Specifications, and Requirements
This stuff is very, very quick.
Introduction:
Finally! Optane Memory sitting in our lab! Sure, it’s not the mighty P4800X we remotely tested over the past month, but this is right here, sitting on my desk. It’s shipping, too, meaning it could be sitting on your desk (or more importantly, in your PC) in just a matter of days.
The big deal about Optane is that it uses XPoint Memory, which has fast-as-lightning (faster, actually) response times of less than 10 microseconds. Compare this to the fastest modern NAND flash at ~90 microseconds, and the differences are going to add up fast. What’s wonderful about these response times is that they still hold true even when scaling an Optane product all the way down to just one or two dies of storage capacity. When you consider that managing fewer dies means less work for the controller, we can see latencies fall even further in some cases (as we will see later).
Specifications:
No real surprises here. We had these specs a bit earlier but things appear to have been de-rated a bit, mostly in the interest of keeping power consumption at lower levels from the looks of it.
Requirements:
Optane Memory requires Intel 7th generation hardware. We suspect you might be able to get away with an older CPU, but the motherboard is a must since it requires BIOS-level support to properly boot a system overlaying Optane Memory over other storage devices.
Most of you may not remember this, but 8 years ago I reviewed the DDRdrive. It consisted of 4GB of DRAM on a PCB. Despite being connected to the system via a single PCIe 2.0 x1 lane, its QD=1 performance exceeding 100,000 IOPS trounced the competition (and still does today actually). While the limited capacity meant it was useless to store your OS, the product worked itself into many enterprise applications as a very fast storage cache device. The limited capacity, price, and lack of an ability to easily use it as a storage cache prevented this sort of thing from catching on for consumer PC's, but one could dream (back then). Now it is basically a reality, as Optane Memory can actually surpass the DDRdrive! Only took 8 years for that particular pipe dream to come true.
Ok, enough reminiscing. Let's get into the review of Optane Memory!
Soooo…. Pair one with a 1TB
Soooo…. Pair one with a 1TB 960 Evo?
Cool idea, but I’d rather put
Cool idea, but I'd rather put three of these Optane Memory SSDs into a 96GB RAID-0 and just boot from that, mounting the 1TB 960 EVO as a separate game/misc drive. But I'm crazy like that.
but would it be better to run
but would it be better to run just the 960 evo or pair with the Optane?
Only have the two m.2 slots
Only have the two m.2 slots to play with (ITX), I don’t think I could run with only 64Gb of internal storage!
> put three of these Optane
> put three of these Optane Memory SSDs into a 96GB RAID-0 and just boot from that
Allyn,
Do you have enough compatible spare parts in your lab
to try the following:
2 x 32GB Optane wired to M.2 slots
with M.2-to-U.2 adapters and cables
fresh install of Windows 10
connect a large Nand Flash SSD
or large HDD later, for dedicated
data storage
Yes, we understand that this setup
would be limited by the DMI 3.0 link; but,
nevertheless I believe many DIY prosumers
would like to know that it was possible.
Are we limited to Kaby Lake CPUs for this
experiment to work at all?
THANKS AGAIN FOR ALL YOU DO!
Allyn, I considered
Allyn, I considered “borrowing” your idea of having a 96GB RAID-0 array for OS, and 1TB 960 EVO for apps/games/data. But I have a dilemma: That combination would require 4 M.2 slots, but 3 slots are the most that I find available on Z270 motherboards. Any suggestion?
The only solution to the lack
The only solution to the lack of M.2 slots would be a riser card plugged into the PCIe x16 slot. I “think” PC Perspective just reviewed a 4 or 8 M.2 card on the last podcast. Would the single 960 EVO go into the slot, or the RAID’d Optane drives? Thank you for reading and potentially responding to my comments.
Any news about linux support,
Any news about linux support, could care a less about windows.
I’m sorry to say it, but for
I'm sorry to say it, but for desktop, Intel probably could care less about <2% of the market. It is a standard NVMe part though, so feel free to pick one up and code your own caching driver (I hope it fares better than BCache). Coding your own additions is a big benefit of Linux, after all.
Allyn, I’m wondering what
Allyn, I’m wondering what stops this from working as a block device (not using Intel’s caching) on an older motherboard that supports PCIe 3.0? In my case a Supermicro X9SCM with an i3-3220.
I was thinking of using it as a ZFS Intent Log (ZIL) for ZFS. I assume my operating system (FreeNAS) would also need a driver for this, as with any NVMe drive?
Thanks
It should be addressable as a
It should be addressable as a block device so long as the OS has NVMe support.
I was afraid you would say
I was afraid you would say that. OK, guess I gotta pull my credit card out…
Well they have one of the
Well they have one of the biggest open source team, they do care about linux. But fair enough, not necessary Desktop.
Michael at phoronix got the confirmation that it should work fine as a standard nvme storage. Heck I think that just means you could install whole linux system on it or mount part of system directories on there(Like replacing quite common speed tweak on linux: mounting /var/tmp on the main memory with tmpfs).
OK, Now that I can visually
OK, Now that I can visually see the latency difference between L2/L3 cache, and a Floppy Drive.. my life is fulfilled! 🙂
FYI – missing a few numbers on the first chart of the page covering Optane RAID results
I was trying to keep the
I was trying to keep the focus on the read performance there. The write results appear inverted because NAND SSDs handle burst writes faster than they handle burst reads, but the important factor determining application launch performance falls on the random reads.
OH Come on Allyn! 🙂
We know
OH Come on Allyn! 🙂
We know you’ve already tested the little blue Optane module with an NVMe SSD.
So Tell US!
So Tell US!
So Tell US!
So Tell US!
Can we buy one and increase our response speeds with a Samsung 960 EVO M.2?
Isn’t that what the TWO M.2 slots on ALL Z270 motherboards is ALL ABOUT?
(big smile)
Caching an NVMe SSD with
Caching an NVMe SSD with another NVMe SSD gets to the point where overhead becomes an issue, mostly because you are still bottlenecked by DMI throughputs at the end of the day. I'm sure there would be a benefit in some cases, but we were getting to the point of diminishing returns with Optane caching a SATA SSD. You're still limited by NAND flash latencies on a 960 PRO, so QD1 performance while cached by Optane should end up similar to when caching an 850 series (SATA) part.
I also wouldn't recommend that specific configuration because I doubt Intel supports it, so you may run into odd compatibility issues.
Yea, I can understand Intel
Yea, I can understand Intel locking out all Samsung NVMe M.2 skus.
Do you think “that maybe” the 32GB Optane module will help boost low Q1 to Q4 response time used with an Intel 750 NVMe 400GB drive?
I have one of those also… And compared to the Samsung 960 EVO 250GB M.2 drive, the Intel NVMe could really use a performance boost. : )
I will purchase an Intel 900P client/consumer Optane drive when they are ready for prime time.
http://www.guru3d.com/news-story/intel-working-on-900p-consumer-optane-ssd-with-high-volume-capacity.html
I wasn’t suggesting Intel
I wasn't suggesting Intel locked out Samsung SKUs, I was instead referring to the Optane Memory driver / firmware maybe not playing as nicely when attempting to cache NVMe with NVMe.
Yeah it could boost lower QD performance of the SSD 750.
Regarding the 900P, you and me both!
Wait, I need the newest
Wait, I need the newest chipset and processor to run Optane, but the only performance increase is with a SATA SSD or HDD?
No, the performance increase
No, the performance increase happens over a HDD, but we also tested it caching a SATA SSD as a comparison point.
The 16GB will be even lower
The 16GB will be even lower latency. The 32GB is rated at 9/30us while 16GB is at 7/18us.
I think the reason they gave 32GB to reviewers is because the sequential write is halved to 145MB/s and it might garner unfavorable reviews.
The difference is interesting. Also the lower latency compared to P4800X. Its likely the overhead of putting multiple channels. The throughput suggests 16GB has 1-channel controller while 32GB has 2. P4800X has 7. The 16GB version thus has less overhead exposing more of media level performance.
The write throughput scales nearly linearly with number of channels.
16GB 1 channel: 145MB/s
32GB 2 channel: 290MB/s
1 channel for 16GB and 2
1 channel for 16GB and 2 channels for 32GB is most certainly the case considering a single XPoint die is 16GB of capacity :). 4KB random reads on the P4800X would still be only hitting one die at a time. Lower latencies at smaller capacities are likely down to simpler controller design enabled by the fewer channels / less addressable capacity.
Minor nitpick with the
Minor nitpick with the charts: I can’t tell the difference between your blue line and your, I assume green line. They look the same to me.
If you’re talking about the
If you're talking about the QoD 4KB Random Read, I included a zoomed-in version a few charts down – it better shows the separation in what was heavily overlapped in the first chart.
I was, thank you. That helps.
I was, thank you. That helps.
Thanks for the amazing news
Thanks for the amazing news Allyn, you totally ROCK!!!
The best reviewer in storage technology, the community has to offer.
We need more great minds like yours.
: )
The guy over at Anandtech is kinda ok on storage, but he lacks excitement and passion, and he’s no where near as smart as Allyn.
can you test is the Optane
can you test is the Optane cache works with Linux distros?
the the cache is somewhat UEFI based, should theoretically work
Possibly, but I know that
Possibly, but I know that there have been issues with the Linux kernel and NVMe, if your distro is happy with NVMe it can physically work. Convincing the OS to use it as a cache might be a different story.
Trying to differentiate
Trying to differentiate various similar shades of semi-transparent blue/orange on a dark blue background and compare them to the graphed lines on a black background.
Would it be possible to place a black box behind the key in future graphs so it’s easier to compare it to the chart and possibly prevent eye strain in some readers?
I feel your pain. Trying to
I feel your pain. Trying to get 9 plot lines on the same chart, where there were three 'brackets' of three results each, well, that's as good as I could make it look. Good idea on the legend thing. I'll try and implement a form of that in the next review using these charts, as well as trying to get more creative with the dash styles to differentiate plot lines even further.
I’d like to see it with a
I’d like to see it with a SSHD to see what happens with having another cache in between the Optane and the HDD.
The Optane benefits are so
The Optane benefits are so far ahead of HDD, and bring the results so close to that of a SATA SSD, it will very much look the same as it did when caching the HDD, with the occasional burst of how it looks when caching the SATA SSD (very similar in most timed tests).
“Only took 8 years for that
“Only took 8 years for that particular pipe dream to cone true.”
Spotted a typo, should be “come”, otherwise great write! 🙂
Fixed!
Fixed!
On the subject of
On the subject of typos…
The second graph in the Client Performance section specifies ‘ud’ as the latency measurement units, when it’s clearly supposed to be ‘us’ and while you’re at it, you could even fire up the character map and spoil us with a μ instead of a ‘u’.
On the subject of the article…
1) Fantastic work!
2) It’s a bummer Optane requires Kaby Lake to run. If it didn’t, I’d be enthusiastically badgering my boss to get a handful of the 32GB drives for our web servers to store databases on.
Good catch. Thanks. Fixed.
1)
Good catch. Thanks. Fixed.
1) Thanks!
2) It only requires Kaby Lake to function as an storage cache. It's a standard NVMe SSD otherwise. You can even chipset RAID up to three of them if you can find the appropriate Z170 board, but again, no caching.
That could have perhaps been
That could have perhaps been made clearer then.
Every piece of news coverage of Optane I’ve seen has warned about Optane needing the latest Intel platform, without mentioning that that’s only the case if you want to be using it with Intel’s caching software.
Can we expect newer SSD from
Can we expect newer SSD from Intel and Micron to be Hybrid NAND/XPOINT? This could destroy all other SSD players: all the storage capacity of triple stacked Nands with the speed of Xpoint.
Intel just created a MOAM (Mother of all Memories)…
Allyn,
Broadcom have
Allyn,
Broadcom have announced a “tri-mode” RAID controller,
model 9460-16i with x8 edge connector:
https://www.broadcom.com/products/storage/raid-controllers/megaraid-9460-16i
I believe the “16i” refers to 4 x SAS connectors @ 4 devices each,
“i”nternal cabling.
But, from what I can decipher, wiring 4 x NVMe SSDs
requires Broadcom’s special “U.2 enabler cable”
(which you can view in Broadcom’s User Guide for that AIC).
p.s. I’m still looking for an x16 NVMe RAID Controller
that supports modern RAID modes and standard U.2 cables:
that way, we can test M.2 NVMe SSDs using your Syba
2.5″ enclosure.
But, I know you’re busy. This was merely an FYI.
KEEP UP THE GOOD WORK!
And servers will be running
And servers will be running windows 10 with this 🙂 will the tracking hold back the drive speed ?
Can we use this device as a
Can we use this device as a readyboost drive on pre-Kaby Lake systems? And if so, would it even help?
(No subject)
Maybe Micron is reading this
Maybe Micron is reading this and won’t make the same mistake?
(Let’s not hold our breath, however.)
I was referring to Windows
I was referring to Windows ReadyBoost, not the Intel caching software. I’m guessing that is still a no?
Easily the most detailed
Easily the most detailed review of the 32GB Optane. Thanks and kudos!
Allyn, would Optane work as a
Allyn, would Optane work as a cache on a Z270 chipset motherboard with an Intel Pentium G4560?
G4560 does not work with
G4560 does not work with Optane Memory. I asked Intel what as special about the 'Core' Kaby Lake parts that allowed Optane to function and that part of my question was not answered.
Hi Allyn,Any idea how this
Hi Allyn,
Any idea how this memory would perform in a SATA M.2 port?
I was looking for a cheap laptop to replace my chromebook, and saw this one in amazon Acer Aspire E 15 E5-575-33BM
thanks
Hi Allyn, awesome review.
Hi Allyn, awesome review.
The write troughput of the 850 evo with optane get cut in half. I can’t find a Qos, 4KB Random Write for the 960 evo or pro.
Do I understand it correctly that the write latency is lower on the 850 with optane than a single 850 or 960 evo?
a 960 evo 500 gb is about the same price as a 850 evo 500gb with 32 gb of optane.
youre advice?