[zfs-discuss] zpool replace with disk/by-id fails on CentOS 7

Mike Borowiec mikebo at wideopenwest.com
Sat May 14 15:36:55 EDT 2016


Greetings -
Had a disk failing under CentOS Linux release 7.2.1511 and ZFSonLinux v 
0.6.5.6-1.el7.centos. Took the failing disk offline, replaced drive, and 
attempted to perform a "zpool replace", which failed with "cannot 
replace X with Y: no such pool or dataset":

root at toybox:/dev/disk/by-id # ls -l | grep sde
lrwxrwxrwx 1 root root  9 May  5 21:34 
ata-Hitachi_HDS723020BLA642_MN1220F3076BED -> ../../sde
lrwxrwxrwx 1 root root 10 May  5 21:34 
ata-Hitachi_HDS723020BLA642_MN1220F3076BED-part1 -> ../../sde1
lrwxrwxrwx 1 root root 10 May  5 21:34 
ata-Hitachi_HDS723020BLA642_MN1220F3076BED-part9 -> ../../sde9
lrwxrwxrwx 1 root root  9 May  5 21:34 wwn-0x5000cca369c34681 -> ../../sde
lrwxrwxrwx 1 root root 10 May  5 21:34 wwn-0x5000cca369c34681-part1 -> 
../../sde1
lrwxrwxrwx 1 root root 10 May  5 21:34 wwn-0x5000cca369c34681-part9 -> 
../../sde9

root at toybox:/dev/disk/by-id # zpool replace extpool 
ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D 
ata-Hitachi_HDS723020BLA642_MN1220F3076BED
cannot replace ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D with 
ata-Hitachi_HDS723020BLA642_MN1220F3076BED: no such pool or dataset

Work around was to do "zpool replace" with the /dev/sde device node 
instead of the sym-link in /dev/disk/by-id.

root at toybox:/dev/disk/by-id # zpool replace -f extpool 
ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D /dev/sde

When silvering completed, had to export and import specifying the by-id 
directory: "zpool export extpool; zpool import -d /dev/disk/by-id extpool".

This is at least non-intuitive so... should it be reported as a new 
issue? Complete script of session output attached.

Please advise... thanks!

Regards,
Mike Bo


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
-------------- next part --------------
root at toybox:/var/log # zpool status extpool
  pool: extpool
 state: ONLINE
  scan: scrub repaired 0 in 7h4m with 0 errors on Sat Mar 12 20:55:46 2016
config:

        NAME                                            STATE     READ WRITE CKSUM
        extpool                                         ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30A8RZD  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30DH7KD  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30ERN2D  ONLINE       0     0     0

errors: No known data errors

<ON CONSOLE>
WARNING: Your hard drive is failing
Device: /dev/sde [SAT], 1 Currently unreadable (pending) sectors
WARNING: Your hard drive is failing
Device: /dev/sde [SAT], 1 Offline uncorrectable sectors

root at toybox:/var/log # grep smartd messages      
May  4 20:47:34 toybox smartd[4306]: Device: /dev/sde [SAT], 1 Currently unreadable (pending) sectors
May  4 20:47:34 toybox smartd[4306]: Sending warning via /usr/libexec/smartmontools/smartdnotify to root ...
May  4 20:47:34 toybox smartd[4306]: Warning via /usr/libexec/smartmontools/smartdnotify to root: successful
May  4 20:47:34 toybox smartd[4306]: Device: /dev/sde [SAT], 1 Offline uncorrectable sectors
May  4 20:47:34 toybox smartd[4306]: Sending warning via /usr/libexec/smartmontools/smartdnotify to root ...
May  4 20:47:34 toybox smartd[4306]: Warning via /usr/libexec/smartmontools/smartdnotify to root: successful
May  4 21:17:34 toybox smartd[4306]: Device: /dev/sde [SAT], 1 Currently unreadable (pending) sectors
May  4 21:17:34 toybox smartd[4306]: Device: /dev/sde [SAT], No more Offline uncorrectable sectors, warning condition reset after 1 email
May  4 21:47:34 toybox smartd[4306]: Device: /dev/sde [SAT], 1 Currently unreadable (pending) sectors
...

root at toybox:/var/tmp # ls -l /dev/disk/by-id/ | grep sde
lrwxrwxrwx 1 root root  9 May  1 20:32 ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D -> ../../sde
lrwxrwxrwx 1 root root 10 May  1 20:32 ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D-part1 -> ../../sde1
lrwxrwxrwx 1 root root 10 May  1 20:32 ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D-part9 -> ../../sde9
lrwxrwxrwx 1 root root  9 May  1 20:32 wwn-0x5000cca369c59010 -> ../../sde
lrwxrwxrwx 1 root root 10 May  1 20:32 wwn-0x5000cca369c59010-part1 -> ../../sde1
lrwxrwxrwx 1 root root 10 May  1 20:32 wwn-0x5000cca369c59010-part9 -> ../../sde9

root at toybox:/var/tmp # zpool offline extpool ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D

root at toybox:/var/tmp # zpool status extpool
  pool: extpool
 state: DEGRADED
status: One or more devices has been taken offline by the administrator.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using 'zpool online' or replace the device with
	'zpool replace'.
  scan: scrub canceled on Thu May  5 10:52:39 2016
config:

	NAME                                            STATE     READ WRITE CKSUM
	extpool                                         DEGRADED     0     0     0
	  raidz1-0                                      DEGRADED     0     0     0
	    ata-Hitachi_HDS723020BLA642_MN1220F30A8RZD  ONLINE       0     0     0
	    ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D  OFFLINE      0     0     0
	    ata-Hitachi_HDS723020BLA642_MN1220F30DH7KD  ONLINE       0     0     0
	    ata-Hitachi_HDS723020BLA642_MN1220F30ERN2D  ONLINE       0     0     0

errors: No known data errors

<POWEROFF, replace physical disk, POWERON>

root at toybox:~ # gdisk /dev/sde
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present


***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format.
THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by typing 'q' if
you don't want to convert your MBR partitions to GPT format!
***************************************************************


Command (? for help): ?
b       back up GPT data to a file
c       change a partition's name
d       delete a partition
i       show detailed information on a partition
l       list known partition types
n       add a new partition
o       create a new empty GUID partition table (GPT)
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       sort partitions
t       change a partition's type code
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu

Command (? for help): p
Disk /dev/sde: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 959E9D9F-3758-434C-A9F8-F56E4FC80C44
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 4205 sectors (2.1 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      3907026943   1.8 TiB     0700  Microsoft basic data

Command (? for help): o
This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N): y

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sde.
The operation has completed successfully.

root at toybox:/dev/disk/by-id # ls -l | grep sde
lrwxrwxrwx 1 root root  9 May  5 21:34 ata-Hitachi_HDS723020BLA642_MN1220F3076BED -> ../../sde
lrwxrwxrwx 1 root root 10 May  5 21:34 ata-Hitachi_HDS723020BLA642_MN1220F3076BED-part1 -> ../../sde1
lrwxrwxrwx 1 root root 10 May  5 21:34 ata-Hitachi_HDS723020BLA642_MN1220F3076BED-part9 -> ../../sde9
lrwxrwxrwx 1 root root  9 May  5 21:34 wwn-0x5000cca369c34681 -> ../../sde
lrwxrwxrwx 1 root root 10 May  5 21:34 wwn-0x5000cca369c34681-part1 -> ../../sde1
lrwxrwxrwx 1 root root 10 May  5 21:34 wwn-0x5000cca369c34681-part9 -> ../../sde9

root at toybox:/dev/disk/by-id # zpool replace extpool ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D ata-Hitachi_HDS723020BLA642_MN1220F3076BED
cannot replace ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D with ata-Hitachi_HDS723020BLA642_MN1220F3076BED: no such pool or dataset

root at toybox:/dev/disk/by-id # zpool replace -f extpool ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D /dev/sde

root at toybox:/var/tmp # zpool status extpool
  pool: extpool
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu May  5 21:45:51 2016
    1.36T scanned out of 6.56T at 302M/s, 5h1m to go
    347G resilvered, 20.69% done
config:

        NAME                                              STATE     READ WRITE CKSUM
        extpool                                           DEGRADED     0     0     0
          raidz1-0                                        DEGRADED     0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30A8RZD    ONLINE       0     0     0
            replacing-1                                   OFFLINE      0     0     0
              ata-Hitachi_HDS723020BLA642_MN1220F30D7B0D  OFFLINE      0     0     0
              sde                                         ONLINE       0     0     0  (resilvering)
            ata-Hitachi_HDS723020BLA642_MN1220F30DH7KD    ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30ERN2D    ONLINE       0     0     0

errors: No known data errors

root at toybox:/var/tmp # zpool status extpool
  pool: extpool
 state: ONLINE
  scan: resilvered 1.64T in 7h34m with 0 errors on Fri May  6 05:20:37 2016
config:

        NAME                                            STATE     READ WRITE CKSUM
        extpool                                         ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30A8RZD  ONLINE       0     0     0
            sde                                         ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30DH7KD  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30ERN2D  ONLINE       0     0     0

errors: No known data errors

root at toybox:/var/tmp # zpool export extpool
root at toybox:/var/tmp # zpool import -d /dev/disk/by-id extpool
root at toybox:/var/tmp # zpool status
  pool: extpool
 state: ONLINE
  scan: resilvered 1.64T in 7h34m with 0 errors on Fri May  6 05:20:37 2016
config:

        NAME                                            STATE     READ WRITE CKSUM
        extpool                                         ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30A8RZD  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F3076BED  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30DH7KD  ONLINE       0     0     0
            ata-Hitachi_HDS723020BLA642_MN1220F30ERN2D  ONLINE       0     0     0

errors: No known data errors



More information about the zfs-discuss mailing list