[zfs-discuss] ZFS on RAID-5 array

Richard Elling richard.elling at richardelling.com
Thu Apr 12 12:51:45 EDT 2018

> On Apr 11, 2018, at 2:39 AM, Gordan Bobic via zfs-discuss <zfs-discuss at list.zfsonlinux.org> wrote:
> Short answer: Don't.

Actually, there are quite a few large enterprises who have been running mission-critical systems
like this for many years. It does work and can work very well.

That said, if the real question is "given X disks, should I create a raidz or raid-5?", then the short answer
is "use raidz"

> Long answer:
> Cons:
> By putting ZFS on a single virtual disk exposed by hardware RAID you will lose ZFS' ability to auto-heal data, thus forfeiting a substantial fraction of it's value.

An enterprise-class RAID array deals with bit rot, checksums, and auto-healing too. They can make
a very nice complement to ZFS.

> Because the underlying RAID geometry is entirely opaque, it is virtually guaranteed that you will end up with a lot of misaligned access which will potentially halve sustained read throughput.
> Sustained write throughput can be as low as half of what ZFS can achieve because of RMW required by parity RAID on writes.
> Pros:
> In some _very_ specific uses, you may be able to get better small random read performance if you have ZFS on top of hardware RAID5.
> If you can reduce the exposed RAID block size to as low as 8KB, you could create zfs with ashift=13 to ensure the alignment at the bottom end is unaffected, and set recordsize to 8KB to ensure fs blocks won't straddle disk boundaries, and operations you are doing are mostly <= 8KB so the 8KB recordsize is in fact appropriate, then having ZFS on hardware RAID5 would potentially yield better random read performance than RAIDZ1 of same shape. Sustained write throughput can still be as bad as half due to RMW on parity RAID.

Disagree, modern RAID arrays are quite adept at managing small writes efficiently. Attempting
to try and match the internal representation is largely a waste of time, just like it is a waste of time
to try and match raidz sizes to recordsize/volblocksize.

Systems engineering is a discipline that deals with trade-offs and compromises. ZFS vs RAID-5 is
not immune to these forces.
 -- richard

> On Wed, Apr 11, 2018 at 1:56 AM, H via zfs-discuss <zfs-discuss at list.zfsonlinux.org <mailto:zfs-discuss at list.zfsonlinux.org>> wrote:
> I have a fibre-channel disk array configured as RAID-5 which I am considering installing zfs on, the host OS will be CentOS 7.
> I believe I have read that zfs should ideally be installed directly on the harddisk but would still like opinions as to the pros and cons.
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at list.zfsonlinux.org <mailto:zfs-discuss at list.zfsonlinux.org>
> http://list.zfsonlinux.org/cgi-bin/mailman/listinfo/zfs-discuss <http://list.zfsonlinux.org/cgi-bin/mailman/listinfo/zfs-discuss>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at list.zfsonlinux.org
> http://list.zfsonlinux.org/cgi-bin/mailman/listinfo/zfs-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20180412/840d5be4/attachment-0001.html>

More information about the zfs-discuss mailing list