Ubuntu 20.04 kernel upgrade -> encrypted Volume group cannot be found + crypttab empty

Question

I updated today, then rebooted. The outcome is that the volume group cannot be found and it drops out to BusyBox.

I tried:

"vgchange -ay" out from the busybox -> did nothing.
Booting with an older kernel -> same outcome.
Then I started from a live CD.

I found this webpage: Recovering from an unbootable Ubuntu encrypted LVM root partition, but my /etc/crypttab is empty.

Here you see how the filesystem is structured:

The two SSDs sda and sdb create on a 3ware controller the raid 1 array sdc. A softraid 1 is then initiated on the 2x 8TB HDDs (which are recognized by the BIOS, but cannot be accessed (unless I knew how) from the live CD even the crypt on the SSDs is opened up.

So I guess ... first things first... that I need to fix the volume group to be found during boot.

If I first unlock the crypt & run sudo vgchange -ay on the live CD, then it looks promising:

2 logical volume(s) in volume group "hal9999-vg" now active

However after rebooting still the Volume Group cannot be found.

I read that this problem occurs in relation to LVM and initramfs, but I don't want to mess around randomly with my filesystems in order to avoid getting in bigger trouble.

In the meantime I tried this answer. It flooded my crypttab, but update-initramfs -u -k all failed:

update-initramfs: Generating /boot/initrd.img-5.4.0-40-generic
cryptsetup: WARNING: target 'luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262' not
    found in /etc/crypttab
W: /sbin/fsck.crypto_LUKS doesn't exist, can't install to initramfs

After all this is the current status:

Output of lsblk -fs:

NAME                 FSTYPE            LABEL       UUID                                   MOUNTPOINT
loop0                squashfs                                                             /rofs
sda                  linux_raid_member hal9999:0   853e3329-6076-8398-e0d8-19149c7d0d64
sdb                  linux_raid_member hal9999:0   853e3329-6076-8398-e0d8-19149c7d0d64
sdc1                 vfat                          94B1-AF12
└─sdc
sdc2                 ext4                          bbb39977-bf4a-4d8b-b524-7c29a85471d0   /mnt/boot
└─sdc
sdd1                 vfat              XUBUNTU 18_ 849C-7AF8                              /cdrom
└─sdd
hal9999--vg-root     ext4                          17c06541-b952-41a2-b28d-b37e49771625   /mnt
└─luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262
                     LVM2_member                   JK4UPj-KdoV-yFe5-XBZW-cHFP-JCbV-JsLjPq
  └─sdc3             crypto_LUKS                   524c1ad6-fabe-4f32-9bb0-c8db1286b262
    └─sdc
hal9999--vg-swap_1   swap                          cd5a6a21-dea3-46a9-a524-e43de19c0587
└─luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262
                     LVM2_member                   JK4UPj-KdoV-yFe5-XBZW-cHFP-JCbV-JsLjPq
  └─sdc3             crypto_LUKS                   524c1ad6-fabe-4f32-9bb0-c8db1286b262
    └─sdc

/etc/crypttab:

sdc3_crypt UUID=524c1ad6-fabe-4f32-9bb0-c8db1286b262 none luks,discard

data /dev/md0 /root/drive_key luks

/etc/cryptsetup-initramfs/conf-hook:

#
# Configuration file for the cryptroot initramfs hook.
#

#
# KEYFILE_PATTERN: ...
#
# The value of this variable is interpreted as a shell pattern.
# Matching key files from the crypttab(5) are included in the initramfs
# image.  The associated devices can then be unlocked without manual
# intervention.  (For instance if /etc/crypttab lists two key files
# /etc/keys/{root,swap}.key, you can set KEYFILE_PATTERN="/etc/keys/*.key"
# to add them to the initrd.)
#
# If KEYFILE_PATTERN if null or unset (default) then no key file is
# copied to the initramfs image.
#
# Note that the glob(7) is not expanded for crypttab(5) entries with a
# 'keyscript=' option.  In that case, the field is not treated as a file
# name but given as argument to the keyscript.
#
# WARNING: If the initramfs image is to include private key material,
# you'll want to create it with a restrictive umask in order to keep
# non-privileged users at bay.  For instance, set UMASK=0077 in
# /etc/initramfs-tools/initramfs.conf
#

#KEYFILE_PATTERN=
CRYPTSETUP=Y

/etc/fstab:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
UUID=524c1ad6-fabe-4f32-9bb0-c8db1286b262 /               ext4    errors=remount-ro 0       1
#/dev/mapper/hal9999--vg-root /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sdc2 during installation
UUID=bbb39977-bf4a-4d8b-b524-7c29a85471d0 /boot           ext4    defaults        0       2
# /boot/efi was on /dev/sdc1 during installation
UUID=94B1-AF12  /boot/efi       vfat    umask=0077      0       1
/dev/mapper/hal9999--vg-swap_1 none            swap    sw              0       0
#/dev/md0 /mnt/md0 ext4 defaults,nofail,discard 0 0
/dev/mapper/data /mnt/data ext4 defaults 0 2

It looks like the diagnosis "Ubuntu accidentally removed lvm and cryptsetup at the last update" may be the case.

I ran these commands:

root@xubuntu:/home/xubuntu# mount /dev/mapper/sdc2 /mnt/boot
mount: /mnt/boot: special device /dev/mapper/sdc2 does not exist.
root@xubuntu:/home/xubuntu# mount /dev/sdc2 /mnt/boot
root@xubuntu:/home/xubuntu# mount -t proc proc /mnt/proc
root@xubuntu:/home/xubuntu# mount -o bind /dev /mnt/dev
root@xubuntu:/home/xubuntu# mount -o bind /sys /mnt/sys
root@xubuntu:/home/xubuntu# chroot /mnt

...and changed /etc/fstab to:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
#UUID=524c1ad6-fabe-4f32-9bb0-c8db1286b262 /               ext4    errors=remount-ro 0       1
/dev/mapper/hal9999--vg-root /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sdc2 during installation
UUID=bbb39977-bf4a-4d8b-b524-7c29a85471d0 /boot           ext4    defaults        0       2
# /boot/efi was on /dev/sdc1 during installation
UUID=94B1-AF12  /boot/efi       vfat    umask=0077      0       1
/dev/mapper/hal9999--vg-swap_1 none            swap    sw              0       0
#/dev/md0 /mnt/md0 ext4 defaults,nofail,discard 0 0
/dev/mapper/data /mnt/data ext4 defaults 0 2

Now I'm getting this error on update-initramfs -c -k all:

   cryptsetup: WARNING: target 'luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262' not
    found in /etc/crypttab
update-initramfs: Generating /boot/initrd.img-5.4.0-39-generic
cryptsetup: WARNING: target 'luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262' not
    found in /etc/crypttab
update-initramfs: Generating /boot/initrd.img-5.4.0-40-generic
cryptsetup: WARNING: target 'luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262' not
    found in /etc/crypttab

New output:

xubuntu@xubuntu:~$ lsblk --fs
NAME                 FSTYPE            LABEL       UUID                                   MOUNTPOINT
loop0                squashfs                                                             /rofs
sda                  linux_raid_member hal9999:0   853e3329-6076-8398-e0d8-19149c7d0d64
sdb                  linux_raid_member hal9999:0   853e3329-6076-8398-e0d8-19149c7d0d64
sdc
├─sdc1               vfat                          94B1-AF12
├─sdc2               ext4                          bbb39977-bf4a-4d8b-b524-7c29a85471d0   /mnt/boot
└─sdc3               crypto_LUKS                   524c1ad6-fabe-4f32-9bb0-c8db1286b262
  └─sdc3_crypt       LVM2_member                   JK4UPj-KdoV-yFe5-XBZW-cHFP-JCbV-JsLjPq
    ├─hal9999--vg-root
    │                ext4                          17c06541-b952-41a2-b28d-b37e49771625   /mnt
    └─hal9999--vg-swap_1
                     swap                          cd5a6a21-dea3-46a9-a524-e43de19c0587
sdd
└─sdd1               vfat              XUBUNTU 18_ 849C-7AF8                              /cdrom

New error: results of update-initramfs -c -k all:

update-initramfs: Generating /boot/initrd.img-5.3.0-59-generic
cryptsetup: ERROR: Couldn't resolve device
    UUID=luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262
update-initramfs: Generating /boot/initrd.img-5.4.0-39-generic
cryptsetup: ERROR: Couldn't resolve device
    UUID=luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262
update-initramfs: Generating /boot/initrd.img-5.4.0-40-generic
cryptsetup: ERROR: Couldn't resolve device
    UUID=luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262

hey mate, I know this old but did you ever get it sorted? I seem to be having pretty much the same issue and have been banging my head against a wall for 2 days… — Harry Stuart, Jul 18 at 17:07

karel · Accepted Answer · 2023-05-20 06:48:53Z

Run:

dmsetup -u sdc3_crypt <UUID of crypt partition>

Change fstab so /dev/mapper/>rootpartition (see below) is named.
Remove luks prefix in crypttab before UUID (see below).

New working fstab:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
#UUID=524c1ad6-fabe-4f32-9bb0-c8db1286b262 /               ext4    errors=remount-ro 0       1
# crypt /dev/sdc3  none luks,initramfs
/dev/mapper/hal9999--vg-root /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sdc2 during installation
UUID=bbb39977-bf4a-4d8b-b524-7c29a85471d0 /boot           ext4    defaults        0       2
# /boot/efi was on /dev/sdc1 during installation
UUID=94B1-AF12  /boot/efi       vfat    umask=0077      0       1
/dev/mapper/hal9999--vg-swap_1 none            swap    sw              0       0
#/dev/md0 /mnt/md0 ext4 defaults,nofail,discard 0 0
/dev/mapper/data /mnt/data ext4 defaults 0 2

New working crypttab:

# sdc3_crypt UUID=524c1ad6-fabe-4f32-9bb0-c8db1286b262 none luks,discard
#
# luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262 UUID=luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262 none luk>
# UUID=luks-524c1ad6-fabe-4f32-9bb0-c8db1286b262 none luks,discard

sdc3_crypt UUID=524c1ad6-fabe-4f32-9bb0-c8db1286b262 none luks,discard

#sdc3_crypt UUID=JK4UPj-KdoV-yFe5-XBZW-cHFP-JCbV-JsLjPq none luks,discard

data /dev/md0 /root/drive_key luks

Now update-initramfs -c -k all ran through without any errors, so reboot went through and the filesystem was found.

Stack Exchange Network

Ubuntu 20.04 kernel upgrade -> encrypted Volume group cannot be found + crypttab empty

1 Answer 1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
updates
lvm
cryptsetup
.

Linked

Hot Network Questions

Ubuntu 20.04 kernel upgrade -> encrypted Volume group cannot be found + crypttab empty

1 Answer 1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged updateslvmcryptsetup.

Linked

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
updates
lvm
cryptsetup
.