[zfs-discuss] ZFS on RAID-5 array
Edward Ned Harvey (zfsonlinux)
zfsonlinux at nedharvey.com
Fri Apr 13 08:31:10 EDT 2018
> From: zfs-discuss <zfs-discuss-bounces at list.zfsonlinux.org> On Behalf
> Of H via zfs-discuss
> That is correct, I do have a choice. It's a FC array with 15 disks
> that I control and just configured as two RAID-5 arrays spanning over
> 13 disks with two hot-spares. I have not begun using it yet and can easily reconfigure it.
Great. Whenever you have a choice, always use JBOD and zfs directly, instead of hardware raid. Only use hardware raid when you have no other choice (or you're not using zfs). For all the reasons already mentioned: zfs wants to see individual disks so it can do self-healing, and when data errors are detected, it can request the redundant copy (something it can't do if hardware raid is obscuring the underlying devices), etc.
Surprising to many: Also disable write-back cache if your hba has that turned on. You get the best performance if you allow zfs to know that the disk is busy, so zfs will aggregate more small writes into a single larger write, rather than queueing up lots of small writes in the HBA cache because the HBA told zfs it was already done with the previous small write and now ready for another small write.
Allow zfs to toggle the disk's cache on (give zfs the whole disk, not just a partition). Truly, the dumber the hba, the better. Raw, JBOD, disk access is the best for zfs.
More information about the zfs-discuss