How to force "packages [that] have been kept back" to be installed [as] "automatic"?

Question

TL;DR

Recently, during the normal software upgrade process (i.e., apt upgrade or apt dist-upgrade) I started getting messages like this:

[...]
The following packages have been kept back:
  [...]
0 upgraded, 0 newly installed, 0 to remove and [...] not upgraded.

What are my options?

---

^{Note: This is not a duplicate of the myriad similar questions asked already here. You should consider my question as a follow-up to this question.}

I had exactly the same problem on my main Ubuntu 22.04 system as the question mentioned above. I changed servers, tried to manually install the named package, and somehow the problem was solved, yesterday. Today, when I tried to install any new updates, I got the message that I had 32 packages that have been kept back.

I have a similar 22.04 version (virtual) Ubuntu installation, which is fully updated now and this problem does not yet occur there (with a similar /etc/apt/sources.list file). So, I tried to compare the packages. I took the package gir1.2-gstreamer-1.0 which was "kept back" in my main system as an example and ran on my main system:

$ sudo apt --installed list | grep gir1.2-gstreamer-1.0
[...]
gir1.2-gstreamer-1.0/jammy,now 1.20.1-1 amd64 [installed,upgradable to: 1.20.3-0ubuntu1]

The same command gives the output:

gir1.2-gstreamer-1.0/jammy,now 1.20.1-1 amd64 [installed,automatic]

on my secondary (virtual) system.

Why do we have this difference? And, how can I make my main system behave as the (virtual) system where the package is installed "automatic"ally?

---

The output of the command apt-cache policy gir1.2-gstreamer-1.0 on my "main" system is like this:

gir1.2-gstreamer-1.0:
  Installed: 1.20.1-1
  Candidate: 1.20.3-0ubuntu1
  Version table:
     1.20.3-0ubuntu1 500 (phased 30%)
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
 *** 1.20.1-1 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

And on my "secondary" (virtual) system is like this:

gir1.2-gstreamer-1.0:
  Installed: 1.20.1-1
  Candidate: 1.20.1-1
  Version table:
     1.20.3-0ubuntu1 1 (phased 30%)
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
 *** 1.20.1-1 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

What does the value 500 (in the (phased... line) mean?

On both systems grep -ri phased /etc/apt returns nothing.

---

After the command sudo apt update && sudo apt upgrade gir1.2-gstreamer-1.0 was executed on my main system:

$ sudo apt --installed list | grep gir1.2-gstreamer-1.0
[...]
gir1.2-gstreamer-1.0/jammy-updates,now 1.20.3-0ubuntu1 amd64 [installed]

$ apt-cache policy gir1.2-gstreamer-1.0
gir1.2-gstreamer-1.0:
  Installed: 1.20.3-0ubuntu1
  Candidate: 1.20.3-0ubuntu1
  Version table:
 *** 1.20.3-0ubuntu1 500 (phased 40%)
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.20.1-1 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages

---

Now, on my secondary system I started getting this too:

$ sudo apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  language-pack-gnome-en
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

Answer 1

Phased Updates are a safety feature

• Some users get the upgraded packages first, and have the ability to report broken package, instead of everybody getting a broken package at once and millions of users scratching their heads.

It's there for your protection. Don't try to outsmart it.

• Kept-back packages due to Phased Updates will automatically resolve themselves, download, and install over a week or so.

Most users should DO NOTHING. It's not broken. Don't try to force upgrades. Just be patient and let the system work.

---

Diagnosis -- how to tell if Phased Updates is the culprit:

• It's easy. Run apt-cache policy <packagename> on one of your held back packages. Look for the 'phased' percentage. It's only present if the package is currently phasing.

$ apt-cache policy gir1.2-gstreamer-1.0
gir1.2-gstreamer-1.0:
  Installed: 1.20.3-0ubuntu1
  Candidate: 1.20.3-0ubuntu1
  Version table:
 *** 1.20.3-0ubuntu1 500 (phased 40%) <----------------- There it is!
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     1.20.1-1 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages

---

Advanced usage

• Phased Updates can be disabled -- it's a simple apt setting, but keep in mind that you're disabling a feature that protects you from receiving broken packages. YOU are offering to test those packages!
• Users with multiple machines can set the machines to phase together -- it's just another simple apt setting.

Example of how to skip phasing one time:

apt -o APT::Get::Always-Include-Phased-Updates=true upgrade

Example of how to skip phasing permanently using an apt config file (NOT recommended for most users)

cat <<EOF > /etc/apt/apt.conf.d/80PhasedUpdates
// Enable/Disable phased updates 
// Default is Phased Updates enabled. Use these lines to disable.
APT::Get::Never-Include-Phased-Updates: True;
Update-Manager::Never-Include-Phased-Updates;
EOF

Example of how to sync phasing of a group of machines using an identical apt config file on each machine:

cat <<EOF > /etc/apt/apt.conf.d/80PhasedUpdates
// To have multiple your machines phase the same, set the same
//   string in this field on all those machines
// If commented out or missing, apt will use /etc/machine-id to
//   seed the random number generator
APT::Machine-ID "aaaabbbbccccddddeeeeffff";
EOF

Also see What `apt` configuration file should the Never-Include-Phased-Updates flag be placed in?

Answer 2

It is certainly possible the difference is the Phased updates system. See if the packages behave the same on both machines if you run this command on both machines.

cat <<EOF > /etc/apt/apt.conf.d/80my
APT::Get::Always-Include-Phased-Updates "1";
EOF

More background on phased updates

Phased updates in APT in 21.04

APT now implements phased updates. Previously, only update-manager implemented phased updates ...

This means that some updates will be hold back on some machines while they are being phased. This is being decided by a value derived from the machine-id, the package name, and package version.

Manpage for apt_preferences

APT understands a field called Phased-Update-Percentage which can be used to control the rollout of a new version. It is an integer between 0 and 100.

In case you have multiple systems that you want to receive the same set of updates, you can set APT::Machine-ID to a UUID such that they all phase the same, or set APT::Get::Never-Include-Phased-Updates or APT::Get::Always-Include-Phased-Updates to true such that APT will never/always consider phased updates.

The manpage also indicates versions that are "not phased" will get priority 1, which matches your apt-cache policy output.

Answer 3

The following method was useful for me

sudo apt install --only-upgrade <package_name>

See: https://itsfoss.com/following-packages-have-been-kept-back/