Is it a usable feature?

A set of updated ASUS UEFI unlocks this feature

EDIT: We've received some clarification from Intel on this feature:

"The feature is actually apart of RST. While this is a CPU-attached storage feature, it is not VROC. VROC is a CPU-attached PCIe Storage component of the enterprise version of the product, Intel RSTe. VROC requires the new HW feature Intel Volume Management Device (Intel VMD) which is not available on the Z370 Chipset.

The Intel Rapid Storage Technology for CPU-attached Intel PCIe Storage feature is supported with select Intel chipsets and requires system manufacturer integration. Please contact the system manufacturer for a list of their supported platforms."

While this doesn't change how the feature works, or our testing, we wanted to clarify this point and have removed all references to VROC on Z370 in this review.

While updating our CPU testbeds for some upcoming testing, we came across an odd listing on the UEFI updates page for our ASUS ROG STRIX Z370-E motherboard.

asus-uefi.png

From the notes, it appeared that the release from late April of this year enables VROC for the Z370 platform. Taking a look at the rest of ASUS' Z370 lineup, it appears that all of its models received a similar UEFI update mentioning VROC. EDIT: As it turns out, while these patch notes call this feature "VROC", it is officially known as "Intel Rapid Storage Technology for CPU-attached Intel PCIe Storage " and slightly different than VROC on other Intel platforms.

While we are familiar with VROC as a CPU-attached RAID technology for NVMe devices on the Intel X299 and Xeon Scalable platforms, it has never been mentioned as an available option for the enthusiast grade Z-series chipsets. Could this be a preview of a feature that Intel has planned to come for the upcoming Z390 chipset?

Potential advantages of a CPU-attached RAID mode on the Z370 platform mostly revolve around throughput. While the chipset raid mode on the Z370 chipset will support three drives, the total throughput is limited to just under 4GB/s by the DMI 3.0 link between the processor and chipset.

Like we've seen AMD do on their X470 platform, CPU-attached RAID should scale as long as you have CPU-connected PCI-Express lanes available, and not being used by another device like a GPU or network card.

First, some limitations.

Primarily, it's difficult to connect multiple NVMe devices to the CPU rather than the chipset on most Z370 motherboards. Since the platform natively supports NVMe RAID through the Z370 chipset, all of the M.2 slots on our Strix Z370-E are wired to go through the chipset connection rather than directly to the CPU's PCIe lanes.

To combat this, we turned to the ASUS Hyper M.2 X16 card, which utilizes PCIe bifurcation to enable usage of 4 M.2 devices via one PCI-E X16 slot. Luckily, ASUS has built support for bifurcation, and this Hyper M.2 card into the UEFI for the Strix Z370-E.

180523163122.png

Aiming to simplify the setup, we are using the integrated UHD 620 graphics of the i7-8700K, and running the Hyper M.2 card in the primary PCIe slot, usually occupied by a discrete GPU.

Next is a limitation that will seem familiar to anyone who has followed the VROC saga on X299 for the past year. As far as we can tell, CPU-attached NVMe RAID in its current state on Z370 will only work with Intel SSDs. In this case, we are using 32GB Optane Memory drives, but it should also be compatible with the higher capacity Optane 800P drives (and the newly announced Intel 905P M.2), as well as Intel's NVMe NAND offerings.

Lastly, the current implementation of CPU-attached NVMe RAID on Z370-based motherboards seems to be limited to two drives, as opposed to 3 drives for the chipset-based NVMe RAID. For most typical consumers, who would use a discrete GPU on a platform like this, it's mostly a moot point. In that scenario, you would run your GPU at PCI-e x8, and then two up to x4 SSDs in a RAID configuration.

Unlike VROC on the X299 chipset though, there are no hardware "dongles" to authenticate support and unlock additional features. Everything is available with the new UEFI updates.

Setup

The setup process for CPU-attached NVMe RAID on Z370 will be familiar to anyone who has ever set up a RAID volume and is the process for setting up the chipset-based RAID on the Strix Z370-E.

180521122014.png

From the Intel Rapid Storage Technolgy menu in the UEFI interface, we can see both 32 GB Optane Memory drives, and our 512GB Samsung SSD 850 Pro boot drive.

180521122025.png

In the Create Array menu, we can select our two 32GB Optane drives, change the RAID level (RAID 0 and 1 are both supported), stripe size, and other applicable settings.

180521122039.png

Now we can see our 54.5GB RAID 0 volume has been created from the two NVMe SSDs connected through the CPU's PCI-express lanes.

Performance Comparison

One of the potential downsides we've seen with CPU-attached NVMe RAID, like in the AMD RAIDXprt2 software, was an increase in disk latency. Whereas the Intel chipset contains some fixed function hardware to help with RAID calculations, CPU-attached NVMe RAID is dependent solely on the CPU and Operating System for all RAID functions.

To evaluate CPU-attached NVMe RAID performance on the Z370 platform, we compared it to NVMe chipset raid on the same Z370 platform, as well as AMD's CPU-attached RAID solution on an X470 motherboard using the same 32GB Optane Memory modules.

Random Read Latency

4k-latency.png

Here we run into an unexpected result. Both single and double drive (RAID disabled and RAID enabled) results show lower latencies for the CPU-attached NVMe RAID setup. While it makes sense for the single drive connected directly to the CPU to be faster, we expected some additional latency for a RAID setup over the chipset-based option. 

Here, we can see that CPU-attached NVMe RAID is 10% lower latency than NVMe RAID through the Z370 chipset.

While single drive latency for X470 is very low, when another drive is added in a RAID 0 configuration, latencies almost double compared to CPU-attached NVMe RAID.

Random Read IOPS

4k-iops.png

Similarly, 4K random IOPS results show a 13% performance advantage to CPU-attached NVMe RAID versus the chipset-based NVMe RAID option on Z370.

While single drive IOPS for the X470 solution look great, adding a second drive and enabling RAID 0 results in a 32% performance decrease compared to CPU-attached NVMe RAID on the Z370 platform.

Sequential Read

sequential.png

In sequential transfers, however, we see nearly identical performance from all configurations, noted by the data points stacked on top of each other in this chart.

Unfortunately, given the distinct lack of NVMe M.2 devices with a full X4 interface from Intel (the 760P fits the bill, but we don't have any two drives of matching capacity), we are left without seeing much of an advantage to CPU-attached NVMe RAID on Z370. While latencies are a bit better, Optane is the only technology that is low latency enough to take advantage of such a small difference.

We would urge Intel to open this feature up to drives from all vendors so that we can see the advantage of two X4 PCIe SSDs in RAID not being bottlenecked by the chipset's DMI link.

img-4895.jpg

Considering some of the bugs we've come across along the way, it doesn't quite seem to be ready for primetime usage. Mostly, this seems like a reactionary move to AMD's recent release of CPU-attached RAID features for the X470 platform. 

However, if motherboard manufacturers build their upcoming  Z390 platforms around the idea of CPU-attached NVMe RAID instead of chipset-based NVMe RAID, by routing the onboard M.2 slots directly to the CPU, then this could become an exciting proposition for enthusiast users.