[zfs-discuss] zpool with massive number of datasets

Olivier Doucet webmaster at ajeux.com
Fri Apr 22 05:33:01 EDT 2016


2016-04-21 14:57 GMT+02:00 Schweiss, Chip via zfs-discuss <
zfs-discuss at list.zfsonlinux.org>:

> The massive number of folders will not affect performance, but kills zpool
> import and export times.  It will also affect things like recursive
> snapshots and zfs send/receive.
>


zpool import was greatly improved on latest ZFS release on ZVOL. For
mounting, I see what you are doing.




>
> The zpool import and export command are really bad at scaling.  They both
> do their work sequentially and latency stacks up really fast.   On one
> system I have about 200 ZFS folders it was taking over an hour to import.
>
> However this can be fixed rather easily.   What I did was wrote scripts to
> break down the process and do as much in parallel as possible.  I reduced
> the import time on my offending pool to 30 seconds.
>
> You need to start your process by doing 'zpool import -N -o
> cachefile={cachefile} {pool}'  (Don't use the default for cache file or
> else it will import at boot time.)  This will import with out mounting any
> folders.  This happens really fast once a cache file is in place.  Then
> spawn multiple processes to mount everything, make sure it is in tree
> order.  Then spawn multiple processes to share everything.  Exporting is
> reversing the process.
>
> -Chip
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at list.zfsonlinux.org
> http://list.zfsonlinux.org/cgi-bin/mailman/listinfo/zfs-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20160422/2f1cbdc1/attachment-0002.html>


More information about the zfs-discuss mailing list