At their FMS 2016 Keynote, Facebook gave us some details on the various storage technologies that fuel their massive operation:
In the four corners above, they covered the full spectrum of storing bits. From NVMe to Lightning (huge racks of flash (JBOF)), to AVA (quad M.2 22110 NVMe SSDs), to the new kid on the block, WORM storage. WORM stands for Write Once Read Many, and as you might imagine, Facebook has lots of archival data that they would like to be able to read quickly, so this sort of storage fits the bill nicely. How do you pull off massive capacity in flash devices? QLC. Forget MLC or TLC, QLC stores four bits per cell, meaning there are 16 individual voltage states for each cell. This requires extremely precise writing techniques and reads must appropriately compensate for cell drift over time, and while this was a near impossibility with planar NAND, 3D NAND has more volume to store those electrons. This means one can trade the endurance gains of 3D NAND for higher bit density, ultimately enabling SSDs upwards of ~100TB in capacity. The catch is that they are rated at only ~150 write cycles. This is fine for archival storage requiring WORM workloads, and you still maintain NAND speeds when it comes to reading that data later on, meaning that decade old Facebook post will appear in your browser just as quickly as the one you posted ten minutes ago.
Next up was a look at some preliminary Intel Optane SSD results using RocksDB. Compared to a P3600, the prototype Optane part offers impressive gains in Facebook's real-world workload. Throughput jumped by 3x, and latency reduced to 1/10th of its previous value. These are impressive gains given this fairly heavy mixed workload.
More to follow from FMS 2016!
Was there a power consumption
Was there a power consumption comparison for flash archival storage vs. traditional hard drive based systems? I would assume that flash could offer much lower power consumption than disk. To provide fast access with hard drives, I would assume that they would need to keep the drives spinning; also spinning them down might mean that they would not come back up when access is attempted. Such massive flash based storage could reduce power consumption considerably. YouTube has even more demand for such a product with their indefinite storage of massive amounts of video.
Flash power consumption is
Flash power consumption is mostly only while active, and higher on writes. This stuff would be sitting idle most of the time, so very low power for sure.
I myself am waiting for PLC
I myself am waiting for PLC and HLC.
Wonder what the price will be
Wonder what the price will be on the QLC. Probably too high, I will stick to SMR drives in the meantime.