[zfs-discuss] Upgrading vdev mirror from ashift=9 to ashift=12 : 512B to 4K transition

fermulator fermulator at sympatico.ca
Fri Jan 1 14:20:37 EST 2016

I'm in a situation where it seems I may have unknowingly locked myself 
into ashift=9 for some vdevs in a pool of mirrors.

Here's the basic premise:
  * initial adoption and research into ZFS, physical sector size, 
ashift, and block alignment
  * created a new zpool with ashift=12
  * added vdevs (some use newer 4K sector size, others consider of older 
512B sector size)

I had /assumed/ that ashift property per vdev would inherit from the 
zpool, but this appears to not be the case. In actuality, it seems like 
zfs automatically chose the ashift value according to the drives 
reported sector size:
  * vdev mirrors with ashift=12 for the vdevs that have newer drives (4K 
physical sector size)
  * vdev mirrors with ashift=9 for the vdevs using older drives (512B 
physical sector size)

Here's what I've got, and please look beyond the fact that two of my 
vdevs are not yet redundant :o

   pool: zfsmedia
  state: ONLINE
   scan: scrub in progress since Fri Jan  1 10:08:11 2016
     1.45T scanned out of 7.20T at 104M/s, 16h6m to go
     0 repaired, 20.06% done

         zfsmedia ONLINE       0     0     0
           ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E0HHF92N ONLINE       0     
0     0
           mirror-1 ONLINE       0     0     0
             ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5LAPP8J ONLINE       0     
0     0
             ata-ST2000VN000-1H3164_W1H25K77 ONLINE       0     0     0
           mirror-2 ONLINE       0     0     0
             ata-ST2000DL003-9VT166_5YD0XWHR ONLINE       0     0     0
             ata-ST2000VN000-1H3164_W1H25JXM ONLINE       0     0     0
           mirror-3 ONLINE       0     0     0
             ata-ST2000DL003-9VT166_5YD18S0M ONLINE       0     0     0
             ata-ST2000DL003-9VT166_5YD18S73 ONLINE       0     0     0
           ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E0HHFYTV ONLINE       0     
0     0

$ sudo zdb | grep -e ashift -e vdev -e type | grep -v disk
     vdev_children: 5
         type: 'root'
             ashift: 12
             type: 'mirror'
             ashift: 12
             type: 'mirror'
             ashift: 9
             type: 'mirror'
             ashift: 9
             ashift: 12

$ for disk in $(ls /dev/disk/by-id | egrep ata | grep -v part); do echo 
$disk; sudo smartctl -a /dev/disk/by-id/$disk | egrep Size; done
Sector Size:      512 bytes logical/physical
Sector Size:      512 bytes logical/physical
Sector Size:      512 bytes logical/physical
Sector Size:      512 bytes logical/physical
Sector Sizes:     512 bytes logical, 4096 bytes physical
Sector Sizes:     512 bytes logical, 4096 bytes physical
Sector Sizes:     512 bytes logical, 4096 bytes physical
Sector Sizes:     512 bytes logical, 4096 bytes physical
Sector Sizes:     512 bytes logical, 4096 bytes physical
Sector Sizes:     512 bytes logical, 4096 bytes physical

Here's how things were created:
2015-08-31.22:43:14 zpool create -f -o ashift=12 -O compression=lz4 
zfspool /dev/disk/by-id/ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E0HHF92N
2015-09-13.17:23:21 zpool add -f zfspool mirror 
2015-09-13.17:29:20 zpool add -f zfspool mirror 
2015-09-13.17:30:28 zpool add -f zfspool mirror 
2015-09-13.17:42:33 zpool add zfspool 

Now, as the drives in the vdev with ashift=9 fail, we would presumably 
replace them with newer drives, and it's highly likely these drives will 
have 4K sector size.
But how can we "migrate" the vdev from ashift=9 to ashift=12?

I've done some searching and found that others have run into this 
scenario, but the solution is not clearly resolved in any of these posts 
thus far.


Do we have documentation somewhere which explains how to transition? 
This scenario has got to be extremely common. If not already, then very 
soon for a vast majority of users.
Some documentation claims that it's impossible to even destroy a vdev, 
which seems to imply solving this problem requires a costly 
recreation/clone of the existing pool to a new one with proper 
future-proof ashift values set.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20160101/14ccb68d/attachment.html>

More information about the zfs-discuss mailing list