0

I am attempting to set up containerd for kubernetes through Ubuntu's autoinstall mechanism (for Ubuntu 20.04).

Some commands when used in late-commands seem to fail without obvious reason. (Exit > 0 with no descriptive output.) The failing snippet from late-commands looks like:

  late-commands:
    - printf 'overlay\nbr_netfilter\n' > /target/etc/modules-load.d/containerd.conf
    - curtin in-target --target=/target -- /usr/sbin/modprobe overlay
    - curtin in-target --target=/target -- /usr/sbin/modprobe br_netfilter

The installer errors out showing that the modprobe call exits 1.

enter image description here

I restart the VM, log in as ubuntu, and become root. At that time I can run modprobe overlay successfully (exit 0). I also cannot see /var/crash/16238... (see above) as it does not appear to exist on the target. (I'm guessing it exists in the installer environment.) /var/log/installer contains nothing useful either.

Given the above, what it is that might make modprobe fail as a late-command? Seemingly unrelatedly, I've also tried a chage -d 0 call that failed with zero error message also. Is there some explanation for why some commands may not exist or not work properly within the target environment at that stage?


Edit 1: regarding the usermod failure, I'm guessing as this reply hints at that the ubuntu user is not created at that point in the install process.

Edit 2: I added an error-commands which seems useful for debugging:

  error-commands:
    - /usr/bin/tail -n 250 /var/log/syslog

This now shows Module overlay not found in /lib/modules/...:

enter image description here

2
  • Read the /var/crash/ file mentioned on the last line of your picture. It's a text file, so you can use less.
    – waltinator
    Jun 17 at 2:44
  • The file doesn't exist as advertised on the target, as indicated in my question. I believe it sits in the installer environment, which I can't access. Jun 17 at 2:52
0

The root cause of this issue is that modprobe was looking in the wrong place due to an updated kernel prior to reboot.

During the install process, you can use CTRL+Z to get into a Bash shell as the root user in the installer. In this environment, /cdrom is the mounted ISO (iso9660), and /target is the root of the target filesystem. This is a great way to debug curtin in-target commands, which internally are basically a chroot /target call.

In the case of modprobe, the output shows:

modprobe: FATAL: overlay module not found in directory /lib/modules/5.4.0-65-generic

While this directory does exist on the installer, on the target what actually exists is:

/target/lib/modules/5.4.0-80-generic

Notice the slight change in kernel version that modprobe has not detected. I believe this is due to kernel updates during the install process.

As a fix for this specific command, modprobe enables you to set the version:

curtin -v in-target --target=/target \
    -- /usr/sbin/modprobe --set-version 5.4.0-80-generic overlay

With --set-version the command will run successfully and exit 0.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.