zpool scrub speed

Nils Bausch nils.bausch at googlemail.com
Mon May 16 10:13:56 EDT 2011


Hi,

I have started a scrub on my running raidz1 and have very slow speeds.
Below my observations

FordPrefect ~ # zpool status
  pool: wtf
 state: ONLINE
 scan: scrub in progress since Mon May 16 10:46:54 2011
    12.4G scanned out of 3.23T at 852K/s, (scan is slow, no estimated
time)
    0 repaired, 0.38% done
config:

	NAME                                    STATE     READ WRITE CKSUM
	wtf                                     ONLINE       0     0     0
	  raidz1-0                              ONLINE       0     0     0
	    ata-SAMSUNG_HD154UI_S1XWJ1KSB33767  ONLINE       0     0     0
	    ata-SAMSUNG_HD154UI_S1XWJ1KSB33765  ONLINE       0     0     0
	    ata-SAMSUNG_HD154UI_S1XWJ1KSC11001  ONLINE       0     0     0
	    ata-SAMSUNG_HD154UI_S1XWJ1MSC01749  ONLINE       0     0     0

errors: No known data errors

FordPrefect ~ # iostat -x 5
Linux 2.6.37-gentoo-r4 (FordPrefect) 	05/16/11 	_x86_64_	(8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.80    0.24    1.36    0.04    0.00   97.55

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.32     2.28    2.59   12.43   101.01   221.45
42.94     0.30   19.80   76.13    8.04   3.41   5.12
sdc               0.31     2.12    2.57   11.78   100.73   219.64
44.66     0.42   29.12  104.31   12.72   4.30   6.16
sdd               0.30     2.14    2.60   12.33   100.73   219.58
42.92     0.20   13.30   48.05    5.98   2.88   4.30
sde               0.32     2.26    2.59   12.47   101.01   221.45
42.82     0.31   20.55   80.37    8.11   3.44   5.18
sda               0.05     6.94    0.15    3.95     1.71    43.58
22.14     0.11   26.11    2.46   26.99   1.15   0.47

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.02    9.97    2.37    0.12    0.00   86.51

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     0.40   40.80    2.20    80.90     3.90
3.94     9.29  205.62  216.61    1.91  22.32  95.98
sdc               0.00     0.40   31.60    2.40    71.30     3.90
4.42     9.42  263.42  283.27    2.08  27.99  95.18
sdd               0.00     0.40   47.00    2.60    73.70     4.20
3.14     9.01  178.75  184.54   73.92  19.96  98.98
sde               0.00     0.40   42.40    2.40    84.80     3.90
3.96     9.33  200.80  212.06    2.00  21.56  96.60
sda               0.00    67.20    0.00    4.80     0.00   128.00
53.33     2.77  164.62    0.00  164.62  35.50  17.04

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.10    2.03    1.08    7.98    0.00   88.81

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     3.00   18.20   38.00    40.00   314.80
12.63     5.25   98.49  286.62    8.39  10.73  60.28
sdc               0.00     1.20   26.60   34.80    47.40   299.40
11.30     9.15  153.42  341.22    9.88  15.83  97.22
sdd               0.00     2.60   13.60   34.40    36.70   364.80
16.73     1.13   26.65   71.46    8.94   6.60  31.70
sde               0.00     2.40   17.20   38.60    28.70   358.30
13.87     4.38   84.29  256.15    7.70   9.44  52.70
sda               0.00    45.20    0.00   13.20     0.00   393.60
59.64     6.94  675.68    0.00  675.68  35.41  46.74

FordPrefect ~ # zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
wtf              2.43T  1.58T   146G  /wtf
wtf/ejic          115M  1.58T   115M  /wtf/ejic
wtf/nils          394G  1.58T  56.3G  /wtf/nils
wtf/nils/backup   137G  1.58T   134G  /wtf/nils/backup
wtf/nils/tm       179G  1.58T   170G  /wtf/nils/tm
wtf/shared       1.79T  1.58T  1.75T  /wtf/shared
wtf/vmuser        116G  1.58T  96.8G  /wtf/vmuser

FordPrefect ~ # zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
wtf   5.44T  3.23T  2.20T    59%  1.03x  ONLINE  -

My setup was made with Open Solaris and I have carried it over to zfs-
fuse and now to zfsonlinux, running the latest zpool version 28. Back
with zfs-fuse I used dedup and disabled it due to it being a memory
hog and slow performance under zfsonlinux. While running the scrub, I
am not doing anything resource hungry on the system, so I am a bit
stuffed about the slow performance - even zpool scrub returns this
message. It started with around 4 MB/s and is constantly decreasing
ever since. Dmesg does not give me any kernel debugs, so it does not
seem to crash or crashed anything - yet. Any ideas why this is
happening?

Nils



More information about the zfs-discuss mailing list