Ubuntu 18.04 on kernel 4.15.0-34-generic boots to BusyBox

Question

The machine was a fresh install of Ubuntu 18.04.1 and used LVM and LUKS during install. The machine was working fine on kernel 4.15.0-33-generic.

The kernel upgraded from 4.15.0-33 to 4.15.0-34-generic. After the kernel updated, it no longer prompts for LUKS passphrase and boots straight to BusyBox v1.27.2

When booting from the older kernel, it prompts for LUKS like normal and the machine is able to boot properly. The issue is when booting from the newer kernel.

I tried to run fsck from a liveCD of 18.04. I unlocked the drive first then ran

fsck -f /deb/sda3

However I only get

fsck from util-linux 2.31.1

and nothing else.

Is there a fix that I can run?

Answer 1

The problem is that /sbin/cryptsetup is not in initramfs file /boot/initrd.img-4.15.0-36-generic.

You need to add CRYPTSETUP=y in /etc/cryptsetup-initramfs/conf-hook. You also need to add the initramfs option in /etc/crypttab. Then, you have to run update-initramfs, which solves this problem.

Detailed solution:

1. In an other Ubuntu machine, use "Startup Disk Creator" to create a bootable Ubuntu 18.04 USB drive
2. Plug in this USB drive in a USB port of the computer that boots to BusyBox

3. When the live USB Ubuntu is ready, connect to Internet (WiFi or wired)

4. Open a terminal

   ubuntu@ubuntu:~$ sudo su
   

5. Find the encrypted block device.

   root@ubuntu:~# blkid|grep LUKS
   /dev/nvme0n1p3: UUID="4b206e76-1531-48ae-95be-ae0ed7a244c1" TYPE="crypto_LUKS" PARTUUID="21db499d-b87b-41c6-864f-04d1531cb083"
   

6. Decrypt the device

   root@ubuntu:~# cryptsetup open UUID="4b206e76-1531-48ae-95be-ae0ed7a244c1" nvme0n1p3_crypt
   Enter passphrase for /dev/disk/by-uuid/4b206e76-1531-48ae-95be-ae0ed7a244c1: 
   

7. List mapped devices. control is a file used to send command to the device mapper. nvme0n1p3_crypt is the decrypted device. ubuntu--vg-root is a LVM logical volume in the LVM ubuntu-vg volume group.

   root@ubuntu:~# ls /dev/mapper/*
   /dev/mapper/control  /dev/mapper/ubuntu--vg-root  /dev/mapper/nvme0n1p3_crypt
   

8. Create mount point. This is where we will mount the / of our system that won't boot.

   root@ubuntu:~# mkdir -p /mnt/ubuntu-root
   

9. Mount root logical volume

   root@ubuntu:~# mount /dev/mapper/ubuntu--vg-root /mnt/ubuntu-root/
   

10. Mount pseudo file systems

    root@ubuntu:~# mount -o bind /sys /mnt/ubuntu-root/sys
    root@ubuntu:~# mount -o bind /proc /mnt/ubuntu-root/proc
    root@ubuntu:~# mount -o bind /dev /mnt/ubuntu-root/dev
    

11. Copy DNS information

    root@ubuntu:~# cp /etc/resolv.conf /mnt/ubuntu-root/etc/
    

12. Change root

    root@ubuntu:~# chroot /mnt/ubuntu-root/
    

13. Mount /boot, which contains the initramfs file. This partition is unencrypted.

    root@ubuntu:/# mount /boot/
    

14. Install binwalk (to see the content of the init ram file system)

    root@ubuntu:~# apt update
    root@ubuntu:~# apt install binwalk
    

15. Find offset of gzipped initramfs content

    root@ubuntu:~# binwalk /boot/initrd.img-4.15.0-36-generic | grep gzip
    1605632       0x188000        gzip compressed data, from Unix, last modified: 2018-10-18 13:00:32
    

16. The problem is that the initramfs file system does not contain cryptsetup. So, that is why there is no LUKS password prompt.

    root@ubuntu:/# cd /root/
    root@ubuntu:~# mkdir initramfs-4.15.0-36-generic
    root@ubuntu:~# cd initramfs-4.15.0-36-generic
    root@ubuntu:~/initramfs-4.15.0-36-generic# dd if=/boot/initrd.img-4.15.0-36-generic bs=1605632 skip=1 | gunzip | cpio -i
    root@ubuntu:~/initramfs-4.15.0-36-generic# ls sbin/cryptsetup
    ls: cannot access 'sbin/cryptsetup': No such file or directory
    

17. To decrypt root at boot, the initramfs needs to contain:

    sbin/cryptsetup
    lib/modules/4.15.0-36-generic/kernel/drivers/md/dm-crypt.ko
    

18. Add the initramfs option in /etc/crypttab

    root@ubuntu:/# cat /etc/crypttab 
    nvme0n1p3_crypt UUID=4b206e76-1531-48ae-95be-ae0ed7a244c1 none luks,discard,initramfs
    

19. Add CRYPTSETUP=y in /etc/cryptsetup-initramfs/conf-hook

20. Run update-initramfs:

    root@ubuntu:~# update-initramfs -k 4.15.0-36-generic -c -v &> update-initramfs-4.15.0-36-generic.cryptsetup.log
    

21. Now, we have cryptsetup in initramfs and the Linux kernel module dm-crypt.ko too:

    root@ubuntu:~# grep /sbin/cryptsetup update-initramfs-4.15.0-36-generic.cryptsetup.log
    Adding binary /sbin/cryptsetup
    
    root@ubuntu:~# grep dm-crypt.ko update-initramfs-4.15.0-36-generic.cryptsetup.log
    Adding module /lib/modules/4.15.0-36-generic/kernel/drivers/md/dm-crypt.ko
    

22. Now, run update-initramfs, without -c (new) and without -v (verbose):

    root@ubuntu:~# update-initramfs -k 4.15.0-36-generic -u
    

23. Verify that the initramfs is actually correct

    root@ubuntu:~# binwalk /boot/initrd.img-4.15.0-36-generic | grep gzip
    1605632       0x188000        gzip compressed data, from Unix, last modified: 2018-10-18 14:26:29
    
    root@ubuntu:~# dd if=/boot/initrd.img-4.15.0-36-generic bs=1605632 skip=1 2> /dev/null | gunzip | cpio -t 2> /dev/null |grep sbin/crypt 
    sbin/cryptsetup
    
    root@ubuntu:~# dd if=/boot/initrd.img-4.15.0-36-generic bs=1605632 skip=1 2> /dev/null | gunzip | cpio -t 2> /dev/null |grep dm-crypt.ko
    lib/modules/4.15.0-36-generic/kernel/drivers/md/dm-crypt.ko
    

24. Now, reboot.

Answer 2

If you have nvidia hardware and are having issues with sebhtml's solution, try a newer Linux kernel with the same steps mentioned there in the BusyBox terminal.

Answer 3

To install missing cryptsetup on initramfs on Ubuntu 22 you can add the cryptsetup-initramfs package:

sudo apt install cryptsetup-initramfs