"kernel-upgrade-required" is not one of applied, apply-failed, unapplied, needs-check, nothing-to-apply, unknown, check-failed

Question

FYI: Canonical's bug tracker has a bug in it on launchpad.net, so I'm incapable of posting this bug on their platform. Canonical decided not to provide direct support, so this security flaw is now being released into the wild so they have to do something about it.

Before we begin, I've marked this as a security vulnerability simply because the whole purpose of canonical-livepatch is to keep the system secure, and with this bug, it's incapable of doing so.

All of our machines are reporting an HTTP 400 error when using their respective URL for the live patch service.

The canonical-livepatch tool runs automatically after being configured as per the instructions on the Canonical website. Whenever it runs, it sends us the following error in our alerts.log file from OSSEC.

canonical-livepatch[31251]: during refresh: cannot check: Bad server status code: 400. URL: https://livepatch.canonical.com/api/machine/[REDACTED MACHINE TOKEN] {"details": {"Status.0.Livepatch.State": "\"kernel-upgrade-required\" is not one of applied, apply-failed, unapplied, needs-check, nothing-to-apply, unknown, check-failed"}, "error": "Invalid payload"}

Running an "apt-get update" and then "apt-get-upgrade" does not yield any upgradable Linux kernels. Presumably your software is out of sync with the API server. If that's not the case, please let me know if there's some way for me to fix it on our end.

When I try to do it manually, here's what it says:

$ sudo canonical-livepatch refresh
Before refresh:

kernel: 4.15.0-29.31-generic
fully-patched: false
version: "42.1"

After refresh:

kernel: 4.15.0-29.31-generic
fully-patched: false
version: "42.1"

Here's the config dump:

$ sudo canonical-livepatch config
http-proxy: ""
https-proxy: ""
no-proxy: ""
remote-server: https://livepatch.canonical.com
ca-certs: ""
check-interval: 60  # minutes

Answer 1

I'm not sure what your question is, but there is a difference between the following two commands:

apt-get upgrade
apt-get dist-upgrade

Typically (or always?), upgrade will not install new kernels, whereas dist-upgrade will.

From the apt-get man page:

   upgrade
       upgrade is used to install the newest versions of all packages
       currently installed on the system from the sources enumerated in
       /etc/apt/sources.list. Packages currently installed with new
       versions available are retrieved and upgraded; under no
       circumstances are currently installed packages removed, or packages
       not already installed retrieved and installed. New versions of
       currently installed packages that cannot be upgraded without
       changing the install status of another package will be left at
       their current version. An update must be performed first so that
       apt-get knows that new versions of packages are available.

   dist-upgrade
       dist-upgrade in addition to performing the function of upgrade,
       also intelligently handles changing dependencies with new versions
       of packages; apt-get has a "smart" conflict resolution system, and
       it will attempt to upgrade the most important packages at the
       expense of less important ones if necessary. The dist-upgrade
       command may therefore remove some packages. The
       /etc/apt/sources.list file contains a list of locations from which
       to retrieve desired package files. See also apt_preferences(5) for
       a mechanism for overriding the general settings for individual
       packages.

Confusingly, apt upgrade is different from apt-get upgrade. I believe apt upgrade is the same as apt-get dist-upgrade.

Rick also says he is using the following command:

sudo apt-get upgrade linux-generic linux-headers-generic linux-image-generic

However, the following should suffice:

sudo apt-get upgrade linux-generic

The above will upgrade all packages, and will install any new packages required by the latest version of linux-generic. (And linux-generic depends on linux-headers-generic and linux-image-generic.)

Alternatively, if you only want to install the most recent linux-generic (and its dependencies), and not upgrade any other packages, consider:

sudo apt-get install linux-generic

Answer 2

So, I found out that support [a-t] canonical/com email does work, even though it says it bounced my message to them. Their support responded almost immediately and with a proper solution.

Here's what they said:

Hello Rick,

I can confirm that there has been some work done to our Livepatch tool to better identify when Livepatch cannot install a patch to fix a CVE and a kernel package upgrade/reboot is required. I will make sure to forward this error to the group that is working on that update in order to better alert that a package upgrade is necessary and not have it look like a failure/bug.

This is the situation that you are in right now. You are running a kernel that is vulnerable to the L1TF exploit [1]. The fix is to upgrade to 4.15.0-32. I can confirm that this version is released since I have upgraded to it last week: client-version: 8.0.3 machine-id: redacted machine-token: redacted architecture: x86_64 cpu-model: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz last-check: 2018-08-18T09:49:12.018793176-07:00 boot-time: 2018-08-15T12:07:32-07:00 uptime: 70h5m42s status: - kernel: 4.15.0-32.35-generic running: true livepatch: checkState: checked patchState: nothing-to-apply version: "" fixes: ""

Please check your /etc/apt/sources.list to ensure that either the updates or security pockets are enabled: deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse deb http://security.ubuntu.com/ubuntu bionic-security main restricted universe multiverse

Also, to see what packages are available after you enable updates and/or security and run apt update, the command is: apt-cache policy linux-image-generic linux-image-generic: Installed: 4.15.0.32.34 Candidate: 4.15.0.32.34 Version table: *** 4.15.0.32.34 500 500 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages 100 /var/lib/dpkg/status 4.15.0.20.23 500 500 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 Packages

I hope this helps your situation.

Thanks, Chris Canonical Support

[1] https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/L1TF

On DigitalOcean, all of those repos are already installed. However, the problem was that apt-get doesn't automatically install kernel upgrades because it can cause problems in other software if the other software isn't compatible with it.

So, since our system is compatible and has regular backups, I've upgraded it with the following command.

sudo apt-get upgrade linux-generic linux-headers-generic linux-image-generic

Remember to reboot your system for the changes to take effect.

sudo reboot

To run the patcher manually, do:

sudo canonical-livepatch refresh

You should see something similar to the following.

Before refresh:

kernel: 4.15.0-32.35-generic
fully-patched: true
version: ""

After refresh:

kernel: 4.15.0-32.35-generic
fully-patched: true
version: ""