[zfs-discuss] Re: Importing a semi-faulted pool

Gregor Kopka gregor at kopka.net
Thu Aug 15 08:12:30 EDT 2013


Am 15.08.2013 13:52, schrieb Gordan Bobic:
> This is the replacement disk that I dd-ed the ZFS partition from the 
> read-only drive onto before importing.
Ah, ok.

>     unless you can reactivate
>     ata-Integral_SATAII_and_USB_SSD_DC0110060D6250008-part4 in some
>     way for zfs to fetch valid data from it your data is gone for
>     good. Sorry.
>
>
> ata-Integral_SATAII_and_USB_SSD_DC0110060D6250008-part4 is in fact 
> alive, but zfs sees it as unavailable for some reason. I tried to 
> online it, but it just said that I need to replace it.
>
> Is there a way to get this disk back into the pool for auxiliary data 
> lookup?
>
>     Since you have dying SSDs and we're talking about small (in todays
>     terms) drives the best (and imho only reasonable for desaster
>     recovery/mitigation) plan is to:
>     1) stop any writes and unneeded accesss on the SSDs ASAP
>     2) copy the SSDs completely with something like ddrescue to known
>     good media (eg. into /good-disk-mounted-here/image-ssdX)
>     3) backup the copy (so you can go back in case you need to)
>     4) feed the copies of the drives to ZFS and let it look what's
>     left to find (zpool import -d /good-disk-mounted-here/)
>
>
> I already did 4) to get the pool to import after I dd-ed the partition 
> over to a new drive. As soon as I did it seems to have kicked off a 
> scrub, though.
Best is to export the pool ASAP and start from 1 so you can go back (in 
case the drives get worse)

In case you have another zpool: copy the complete drives into zvols, 
then zfs will create /dev/<pool>/<path>/diskimage-partX block devices 
for you (in case the MBR on the drives are still valid)

>
>
>         I cannot zfs send the snapshots of any of these zvols, nor can
>         I dd them out, I just get I/O errors.
>
>     Yes, since mirror-0 can't deliver the needed data - and ZFS is
>     created to either return correct data or none.
>
>
> I'm cool with "none" for any individual block. "None" for an entire 
> zvol seems like quite a large defficiency...
Try ddrescue on the zvols, dd ends on first read error (and in case 
block #0 is toast then it won't try to read the rest)

> errors: Permanent errors have been detected in the following files:
>
>         <metadata>:<0x0>
>         ssd/ariia at 20130811:<0x1>
>         ssd/ariia at 20130812:<0x1>
>         ssd/ariia at 20130815:<0x1>
>         ssd/edi at 20130719:<0x1>
>         ssd/ariia:<0x1>
>         ssd/edi:<0x1>
>         ssd/edi at 20130803:<0x1>
>         ssd/edi at 20130729:<0x1>
>
> I'm guessing that <metadata>:<0x0> is _bad_... :(
Yes.

> That thing that surprises me is that even the snapshots that were 
> taken many scrubs ago are all unreadable/unsendable. Given the 
> copy-on-write nature of the FS, I would expect that at least the older 
> snapshots are readable.
Snapshots share unchanged data (with the other snapshots and the live 
filesystem/zvol), so in case your storage eats a data block which hasn't 
been changed since you created the filesystem/zvol the error will affect 
the live one and every snapshot taken - since no CoW ever happened...

>         Is there a way I can get at least a partially corrupted dd
>         image off? There are a few files in there that I could do with
>         retrieving since they would have changed since the most recent
>         backup. Is there a way to tell zfs to ignore errors and just
>         feed back blocks of 0s for the data it can't scrub out?
>
>     This is a good question for the experts...
>
>
> Indeed. The most I get on zfs send attempts with any snapshot is 2184 
> bytes. I'm guessing this is just the zfs send header. I don't even get 
> that on the later snapshots.
Again: try ddrescue on the zvols in question - on a copy of everything 
on storage level.
Stressing already dying storage (or writing to it) is never a good idea...

Gregor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20130815/2a723343/attachment.html>


More information about the zfs-discuss mailing list