1

I have updated Ubuntu from version 13.04 to version 13.10, only to discover that the SATA disks IDs have changed, and my ZFS pool now fails.

On new Ubuntu 13.10 the disk IDs are now ata-* instead of scsi-SATA_*.

This is the pool status after the update:

  pool: nestpool
 state: UNAVAIL
status: One or more devices could not be used because the label is missing 
    or invalid.  There are insufficient replicas for the pool to continue
    functioning.
action: Destroy and re-create the pool from
    a backup source.
   see: http://zfsonlinux.org/msg/ZFS-8000-5E
  scan: none requested
config:

    NAME                                                STATE     READ WRITE CKSUM
    nestpool                                            UNAVAIL      0     0     0  insufficient replicas
      raidz2-0                                          UNAVAIL      0     0     0  insufficient replicas
        scsi-SATA_WDC_WD4000F9YZ-_WD-WCC1F0046946       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WCC4A0026423       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WMC1F0011145       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WMC1F0049294       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WMC1F0051143       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WMC1F0051756       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WMC1F0056625       UNAVAIL      0     0     0
        scsi-SATA_WDC_WD4000F9YZ-_WD-WMC1F0200560       UNAVAIL      0     0     0
    logs
      mirror-1                                          UNAVAIL      0     0     0  insufficient replicas
        scsi-SATA_Samsung_SSD_840S1ATNEAD707062H-part2  UNAVAIL      0     0     0
        scsi-SATA_Samsung_SSD_840S1ATNEAD707066K-part3  UNAVAIL      0     0     0

After a very long research on the Internet I started following procedure:

First exported the pool using: zpool export nestpool

Then tried to import back the pool using: zpool import -m -f -d /dev/disk/by-id nestpool

(I have also tried with and without all combinations of -m, -f and -d, as well as usind the ID instead of the zpool name)

But the import fails with the message: cannot import 'nestpool': one or more devices is currently unavailable

This is the current output of zpool import

   pool: nestpool
     id: 3947768928242827823
  state: DEGRADED
 status: One or more devices contains corrupted data.
 action: The pool can be imported despite missing or damaged devices.  The
    fault tolerance of the pool may be compromised if imported.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
 config:

    nestpool                                                DEGRADED
      raidz2-0                                              ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WCC1F0046946          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WCC4A0026423          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0011145          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0049294          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0051143          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0051756          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0056625          ONLINE
        ata-WDC_WD4000F9YZ-09N20L0_WD-WMC1F0200560          ONLINE
    cache
      ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707062H-part1
    logs
      mirror-1                                              UNAVAIL  insufficient replicas
        ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707062H      UNAVAIL  corrupted data
        ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707066K      UNAVAIL

Log partitions were not correctly identified, they should be ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707062H-part2 and ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707066K-part3. And the argument -m during import is not helping.

I cannot find a way to tell zpool import to use a different path/id for the log devices. Any help and idea for fixing this problem will be much appreciated. What else can I do to recover this pool?

(Here is the current output of gdisk -l for all disks: http://pastebin.com/bBec21WN)

2
  • See the solution to this question here on serverfault
    – Manolo
    Jan 5, 2014 at 22:34
  • It was solved using links for the disk ids...
    – Manolo
    Jan 5, 2014 at 22:35

1 Answer 1

1

As everything was still intact and the only problem that prevented importing the pool was the wrong recognition of the partitions used for the log devices, the solution is to create temporary symbolic links under /dev/disk/by-id for the missing partitions.

In my case like this:

ln -sf ../../sda2 ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707062H
ln -sf ../../sdb3 ata-Samsung_SSD_840_PRO_Series_S1ATNEAD707066K

After that, in my case zpool import changed the old wrong IDs to new ones that are present now under Ubuntu 13.10 (each disk and partition has 2 IDs now), and I had to create symbolic links for these as well, like this:

ln -sf ../../sda2 wwn-0x50025385503e8531
ln -sf ../../sdb3 wwn-0x50025385503e8535

After that zpool import automagically started to recognise the correct partitions.

For importing I ran the command:

zpool import -f nestpool

And the pool was imported. DO NOT FORGET to fix the temporary links! Also after a server restart the links will be recreated and fixed.

In my case zpool did recognise the correct partitions during import and no further action was necessary. Otherwise it is necessary to remove the ZIL devices and re-add them back, before anything else!

Do not forget to run scrub and all should be back in place.

A lesson learned for the future is that ZFS recognises the partition if it is the first partition on disk, this is for example the case when using full disks for ZFS. So try to use full disks or when possible only first partition on disks.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.