Internals, Testing Methodology and System Setup
Internals:
First a look at the outside:
Brackets were removed for these pictures as we were into partial disassembly at that point. I included a pair of P3700's along side the P3608 to emphasize the fact that the contents of *both* of the single units (left) has been effectively combined into the *single* package of the P3608 (right). This is no small feat.
Once disassembled we note that Intel opted to include a back plate on the P3608. They have also been shipping SSD 750's with this same type of plate, so it seems all future HHHL SSD products fromthem will include such a plate.
In the following two pictures I will place the P3608 on top of a similarly oriented P3700 for comparison of the layouts:
Right away we see that there are two controllers on the P3608, along with an additional PLX chip (more on that later). Also noteworthy is that there are far less flash memory packages present on this side of the P3608. Let's check the back:
There are also far less packages at the rear. With half the number of packages and double the available capacity, it's no wonder we noted the oddly decreasing random write specs at higher capacities, as Intel would only be able to stack these dies so high within each package and therefore could only offer a 4TB capacity (nearly 256GB per flash package works out to 16-die stacks!) by dialing back the overprovisioning, which translates to reduced random write performance of those higher capacity models.
This is where the magic happens. The PLX PEX8718 is a 16 lane PCIe 3.0 switch that funnels data from both of the PCIe 3.0 x4 controllers to the host over a PCIe 3.0 x8 link. This is a smaller version of the same sort of PCIe switch that enables two GPUs on a single graphics card. For those wanting to geek out further, the data sheet for this particular chip is available at Mouser.
Testing Methodology
This is our first review using a purpose-build enterprise testing suite. We will cover the methodology further into the review and welcome your feedback on our process and presentation.
Test System Setup
We have several storage testbeds, but for this piece we used the same setup used in our 5-way SSD 750 RAID testing. It is not server-class hardware, but it is a reasonable equivalent that allows us to push the CPU to higher clock speeds than server hardware would normally be capable of. This helps shift the burden / bottlenecks to the NVMe PCIe SSDs under test.
PC Perspective would like to thank Intel, ASUS, Corsair, Kingston, and EVGA for supplying some of the components of our test rigs.
Hard Drive Test System Setup | |
CPU | Intel Core i7 5960 @ 4.5 GHz |
Motherboard | ASUS X99 Deluxe |
Memory | 16GB Micron DDR4 @ 3333 |
Hard Drive | G.Skill 32GB SLC SSD |
Sound Card | N/A |
Video Card | GeForce 210 |
Video Drivers | GeForce |
Power Supply | Corsair CMPSU-650TX |
DirectX Version | N/A |
Operating System | Windows 8.1 Pro X64 (update) |
- PCPer Enterprise Storage Test Suite 0.9 (beta)
@Allyn Any chance that Intel
@Allyn Any chance that Intel will release a 800 GB version of the P3608, in order to lower it to a more affordable price point for the enthusiast?
More than likely they will
More than likely they will not, as the P3608 is meant to get higher densities into smaller spaces. It would also limit each 'half' to only 400GB, which would offer limited performance that would be close to that of the 800GB P3600 in the first place.
Regarding use by enthusiasts, I would highly recommend going the new 800GB SSD 750 route as (or a pair of 400's in RST RAID). The 750 Series uses the same controller but has its enterprise temperature monitoring features disabled – those features interfered with many desktop class BIOS and caused memory contention / address conflict issues. The firmare is also more optimized for desktop / consumer workloads.
I’m not even mad I can’t
I’m not even mad I can’t afford one.
I’m just sitting here admiring the nice pictures. Making neat graphs like these should be performance art with tours of live shows. You rock, Allyn!
Thanks for the kudos! We’re
Thanks for the kudos! We're working hard on how we present this data, and will continue to improve on these charts.
Interesting review, but not
Interesting review, but not exactly a PC part. Giving it a gold award seems a bit pointless. No PC enthusiast should buy this part, or really anything in the DC P3xxx line. It is interesting to know what is going on in the enterprise market, since that tech will filter down to the PC market eventually, if it is something that is actually useful to the PC market. I don’t know if devices like this will have a place in the PC market before it is displaced by other technology though.
I realize that the site is
I realize that the site is called PC Perspective, but this is an enterprise review. A handfull of sites cover both PC and enterprise storage devices. For the moment, we are doing it without spinning off another site or brand. With Intel's RST for Z170 NVMe devices and RSTe to bridge both halves of this device, you're correct that it may filter down to the PC market. Actually, the same RST tech can currently RAID SSD 750's (not RAIDed for that piece, but now it is possible).
OK- I really need your help.
OK- I really need your help. I have a 1.6TB P3608 and have it installed on an X99 chipset motherboard. I have tried every version of RSTe I can find and I cant for the life of me get the P3608 to detect in RSTe. The P3608 shows up fine in Disk Manager and I can even set up a RAID from within Disk Manager (albeit at the expense of being able to TRIM the array).
Can you please explain which version of RSTe driver and UI you used?
Any word on pricing? Not that
Any word on pricing? Not that I could ever afford one, I’m pretty sure its more expensive than the rest of my PC. (PS, I know its for data centers and not for a regular enthusiast, but damn I want it so bad).
Nothing new when you look at
Nothing new when you look at “ordinary” P3600. I was expecting lame PLX chip as it is much cheaper way than actually making two SSD working in tandem without lane switcher on same card. Sadly no hardware RoC is available for NVMe ATM.
While review is interesting from raw performance standpoint, it is not relevant at all to PC market as 3608 is purely server grade, industrial storage that will never reach enthusiast market – at least not in this shape. More interested in what you hinted above about seriously more expensive P3700.
Allyn have you tested that setup in RAID1/10 (if you have 2)? Would be interested in that, how much hit NVMe gets on writes with this setup vs classic NAND AHCI. R0 is pointless exercise from my point of view. Redundancy over performance any day of the week.
We reviewed the P3700 before.
We reviewed the P3700 before. I've run the workload on the P3608 and both P3700's in a RAID-0. RSTe had no issue pegging all drives on sequentials (10 GB/sec reads), but you need to throw more cores at it for random IO as compared to addressing the drives individually. More detail on the level of overhead in the next piece covering RSTe, as there is a lot of data I need to compile for it. Might do a RAID-10 data in that piece as well if the testbed is still assembled when I'm back at the office next week.
I see you recommend the Intel
I see you recommend the Intel 750 800GB for the pro-sumers out there. Would you recommend it over the new Samsung 950 Pro 512GB coming out next month?
Those two different SSDs are
Those two different SSDs are going to have their own use cases. The 950 PRO will only be available in M.2 and at 512GB max (initially), while the SSD 750 is available in 800GB and 1.2TB. The 950 PRO should be a lower cost, but those without an M.2 slot will need an adapter. I think they will be close enough on performance that it will boil down more to fitment and cost.
Why would they use a PEX8718
Why would they use a PEX8718 chip? You don’t need 16x PCIe 3.0. 8X would suffice.
The chip has 16 PCIe lanes
The chip has 16 PCIe lanes *total*, some of which need to connect to the controllers. This one is configured to send 8 lanes to the host and 4 lanes to each controller. 8+4+4 = 16.
Thanks for clarification.
Thanks for clarification. That makes more sense now.
Those graphs man, hard to
Those graphs man, hard to wrap my head around some of them
Lots of data in a small
Lots of data in a small space, but if you know what your specific workload is, I think they get the job done.
just Wow !
just Wow !
we’re putting together a unix
we’re putting together a unix rig to sit in a data center and just compute 24/7. as many cores as we can afford, dual gpu nvidia and 64 gigs ram.
programmer is deranged by 3608 for boot (and everything really) and i want to make sure lanes are sufficient.
any 2011v3 boards stand out for this use?