Understanding how to use SSD as Hybrid Storage Pools for ZFS
Solid state drives provide incredible IOPS compared to hard disks. But the consideration of cost rules them out as primary mass storage. But for most applications you would not consider storing everything in RAM either – yet RAM cache is part of any storage system. Why wouldn’t we take advantage of Solid state drives as an intermediary tier between RAM and hard disks ? This reasoning is what hierarchical storage management is about, but Sun took it one step further by integrating it into the file system as ZFS‘s Hybrid Storage Pools.
You can read a quick overview of Hybrid Storage Pools in marketing terms, but you will surely find Sun’s Adam Leventhal’s presentation more substantial as a technical introduction. And most impressive are Sun’s Brendan Gregg’s benchmarks showing 5x to 40x IOPS improvement !
Adding SSD to a ZFS storage pool is done at to locations : the ZFS intent-log (ZIL) device and the Second Level Adaptive Replacement Cache (L2ARC). Usually they are set on two separate devices, but Arnaud from Sun showed that they can share a single device just fine.
The ZIL, also known as Logzilla accelerate small, synchronous writes. It does not require a large capacity. The L2ARC, also known as Readzilla accelerates reads. For the gory details of how Logzilla and Readzilla work, Sun’s Claudia Hildebrandt’s presentation is a great source.
Creating a ZFS storage pool with one or more separate ZIL devices is dead easy, but you then may want to tune your system for performance. It costs some DRAM to reference the L2ARC, at a rate proportional to record size – between 1/40th and 1/80th of the L2ARC depending on the tuning (I have seen several different estimates) – so don’t set a L2ARC larger than your DRAM affords you.
I hope that this sort of goodness will some day come to Linux through Btrfs, but ZFS provides it right now – and it is Free software too… So I guess that in spite of my religious fervor toward the GPL, my storage server’s next operating system will be a BSD licensed one… Who would have thought ?