What disk space and other configurations are needed so that you can trust unattended-upgrades to run unattended more or less indefinitely? I'm asking this to try to understand what went wrong with a particular server.
In early 2014 I setup an Ubuntu 12.04.4 LTS system on Amazon EC2 (using ami-c45f6281), enabling unattended-upgrades to maintain security updates. Then I let it be.
Looking in the logs, I can see that On 2015-05-20, unattended-upgrades fell over, apparently because of lack of disk space. This seems to be the key line from the log file /var/log/unattended-upgrades/unattended-upgrades-dpkg_2015-05-20_06:54:02.822902.log:
Selecting previously unselected package linux-headers-3.2.0-84-virtual.
Unpacking linux-headers-3.2.0-84-virtual (from .../linux-headers-3.2.0-84-virtual_3.2.0-84.121_amd64.deb) ...
dpkg: error processing /var/cache/apt/archives/linux-headers-3.2.0-84-virtual_3.2.0-84.121_amd64.deb (--unpack):
error creating symbolic link `./usr/src/linux-headers-3.2.0-84-virtual/include/linux/nfs2.h': No space left on device
In the days after that, all I see is this over and over again from then until now:
2015-11-02 06:28:44,233 INFO Initial blacklisted packages:
2015-11-02 06:28:44,249 INFO Starting unattended upgrades script
2015-11-02 06:28:44,256 INFO Allowed origins are: ['o=Ubuntu,a=precise-security']
2015-11-02 06:29:09,126 ERROR Cache has broken packages, exiting
But this initial error confuses me because df seems to indicate I have over 1 GB of disk space free:
ubuntu@ip-172-31-27-132:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 7.9G 5.7G 1.8G 77% /
udev 288M 12K 287M 1% /dev
tmpfs 60M 196K 59M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 296M 0 296M 0% /run/shm
This may be related to the this bug with the grub bootloader on EC2 instances, but that bug describes instances not even booting properly after an update, a very alarming prospect which now has me frightened to even try rebooting.
More generally, I'm wondering if Ubuntu and unattended-upgrades is actually a sensible way to build a "set it and forget it" sort of system. Maybe that idea was naive, and the right way to do this is instead to shut off updates and try to minimize attack surface, or to choose a more server-oriented distro?
