Find (and reinstall) packages with corrupted files (without breaking anything)

Question

I usually prefer to fix a Linux system than to reinstall from scratch. My computers have seen many distribution upgrades and a list of PPAs or third-party repositories. APT usually makes sure that everything works in the end. However, the fact that the package manager thinks that all required packages are 'installed' does not guarantee that all the files are present on the file system.

Such a such situation may occur if you have to work-around dependency problems with dpkg --force-*. One could also reproduce such a situation by deleting a file from /usr as root.

Is there a simple way to verify whether all files belonging to an installed package are present?

If a such a problem package is found, aptitude reinstall fixes the problem.

Answer 1

From the debsums man page:

apt-get install --reinstall $(dpkg -S $(debsums -c) | cut -d : -f 1 | sort -u)
       Reinstalls packages with changed files.

My disk were randomly corrupted while I was experimenting with a beta OS. So I ran these 3 commands from the docs (debsums -c, dpkg -S, and apt-get install) one by one to understand what they do. It seems to have worked well:

First, I installed debsums and ran it to see if I had any corrupt files on my system:

$ sudo apt-get install debsums
$ sudo debsums init
$ sudo debsums -cs
/usr/share/bash-completion/completions/ssh
/usr/share/icons/hicolor/scalable/actions/cheese-take-photo.svg
/usr/share/gnome/help/gnumeric/C/files-textopen.xml
/usr/share/dbus-1/services/indicator-sound.service
/lib/modules/3.11.0-12-generic/kernel/drivers/mtd/ubi/ubi.ko

As you can see, I have five corrupt files so I need to reinstall them. This is how I found which packages contain the corrupt files:

$ sudo debsums -c | xargs -rd '\n' -- dpkg -S | cut -d : -f 1 | sort -u
bash-completion
cheese-common
gnumeric-doc
indicator-sound
linux-image-extra-3.11.0-12-generic

Then I repaired the corruption by reinstalling the damaged packages:

$ xargs -rd '\n' -a <(sudo debsums -c | xargs -rd '\n' -- dpkg -S | cut -d : -f 1 | sort -u) -- sudo apt-get install -f --reinstall --
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  linux-image-generic
Use 'apt-get autoremove' to remove it.
0 upgraded, 0 newly installed, 5 reinstalled, 0 to remove and 0 not upgraded.
Need to get 43.9 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ saucy/main bash-completion all 1:2.0-1ubuntu3 [173 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ saucy/main cheese-common all 3.8.3-0ubuntu1 [2,929 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ saucy/universe gnumeric-doc all 1.12.6-1 [7,295 kB]     
Get:4 http://us.archive.ubuntu.com/ubuntu/ saucy/main linux-image-extra-3.11.0-12-generic i386 3.11.0-12.19 [33.5 MB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ saucy/main indicator-sound i386 12.10.2+13.10.20131011-0ubuntu1 [55.7 kB]
Fetched 43.9 MB in 10min 23s (70.4 kB/s)                                                           
(Reading database ... 174913 files and directories currently installed.)
Preparing to replace bash-completion 1:2.0-1ubuntu3 (using .../bash-completion_1%3a2.0-1ubuntu3_all.deb) ...
Unpacking replacement bash-completion ...
Preparing to replace cheese-common 3.8.3-0ubuntu1 (using .../cheese-common_3.8.3-0ubuntu1_all.deb) ...
Unpacking replacement cheese-common ...
Preparing to replace gnumeric-doc 1.12.6-1 (using .../gnumeric-doc_1.12.6-1_all.deb) ...
Unpacking replacement gnumeric-doc ...
Preparing to replace linux-image-extra-3.11.0-12-generic 3.11.0-12.19 (using .../linux-image-extra-3.11.0-12-generic_3.11.0-12.19_i386.deb) ...
Unpacking replacement linux-image-extra-3.11.0-12-generic ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
Preparing to replace indicator-sound 12.10.2+13.10.20131011-0ubuntu1 (using .../indicator-sound_12.10.2+13.10.20131011-0ubuntu1_i386.deb) ...
Unpacking replacement indicator-sound ...
Processing triggers for man-db ...
Processing triggers for libglib2.0-0:i386 ...
No such key 'auto-launch' in schema 'com.ubuntu.update-notifier' as specified in override file '/usr/share/glib-2.0/schemas/20_xubuntu-default-settings.gschema.override'; ignoring override for this key.
Processing triggers for hicolor-icon-theme ...
Setting up bash-completion (1:2.0-1ubuntu3) ...
Setting up cheese-common (3.8.3-0ubuntu1) ...
Setting up gnumeric-doc (1.12.6-1) ...
Setting up linux-image-extra-3.11.0-12-generic (3.11.0-12.19) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
Not updating initrd symbolic links since we are being updated/reinstalled 
(3.11.0-12.19 was configured last, according to dpkg)
Not updating image symbolic links since we are being updated/reinstalled 
(3.11.0-12.19 was configured last, according to dpkg)
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
run-parts: executing /etc/kernel/postinst.d/dkms 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
update-initramfs: Generating /boot/initrd.img-3.11.0-12-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.11.0-12-generic /boot/vmlinuz-3.11.0-12-generic
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.11.0-14-generic
Found initrd image: /boot/initrd.img-3.11.0-14-generic
Found linux image: /boot/vmlinuz-3.11.0-12-generic
Found initrd image: /boot/initrd.img-3.11.0-12-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Windows 7 (loader) on /dev/sda1
done
Setting up indicator-sound (12.10.2+13.10.20131011-0ubuntu1) ...

Finally, I checked to make sure that no corrupt files remain:

$ sudo debsums -c

There was no output from this command, which means that no errors were found. :-)

One final note: you should also check your packages' config files to make sure they're OK. This can be more difficult because config files often change, and the changes are legitimate, so you'll need to manually inspect each changed config file to determine whether or not it's actually corrupt. This is how you get a list of changed config files:

$ sudo debsums -as
debsums: changed file /etc/gnome/defaults.list (from desktop-file-utils package)
debsums: changed file /etc/default/rcS (from initscripts package)
debsums: changed file /etc/subuid (from login package)
debsums: changed file /etc/subgid (from login package)
debsums: changed file /etc/sudoers (from sudo package)

Answer 2

The script given by PeniWize works great for corrupted files, but does not take care of packages with missing files, because debsums reports them to stderr. To reinstall packages with missing files, this worked for me:

xargs -rd '\n' -a <(sudo debsums -c 2>&1 | cut -d " " -f 4 | sort -u | xargs -rd '\n' -- dpkg -S | cut -d : -f 1 | sort -u) -- sudo apt-get install -f --reinstall --

Answer 3

The question has been answered elsewhere:

Is there a Ubuntu sanity check?: package debsums can compute MD5 hashes and compare against the deb package.

Is there a safe way to reinstall via the package manager: Yes, but not recommended.

Answer 4

I crafted the following script to restore missing files using various tricks.

Save it to file dpkg-fix.sh, make it chmod a+x dpkg-fix.sh and run sudo ./dpkg-fix.sh -x.

Disclaimer/Note that -x option restores config files (in /etc) which are not restored by default. It can hose your system. It is very tricky to make any automated tool to restore that correctly, as config files are frequently customized. You should make backups and after restoring compare differences with your latest backup and make some judgement calls. Even after that, devise a plan how to test each change as it is possible to badly break things without means to recover.

#! /bin/bash

## (c) 2020 IVA2K

# Must be root
if [ $(id -u) -ne 0 ]; then
  printf "Must be root. Did you use 'sudo'?\n"
  exit 1
fi

do_install () {
  echo "Installing debsums:"
  apt-get install debsums

  echo "Initializing debsums (might take a while)..."
  debsums_init

  echo "Done Installing & Initializing debsums."
  exit 0
}

# Based on ideas from https://askubuntu.com/a/392326
# $1=debsums arg $2=0(no err) 1(catch debsums stderr) $3=fix|no_fix
do_debsums_parse () {
  echo "Scanning for changed or missing files (debsums ${1} errs=${2} > debsums${1}${2}.log)..."
  if [ "${2}" -ne "0" ]; then
    IFS=$'\n' readarray -t badfiles <<< "$(debsums "${1}" 2>&1)"
  else
    IFS=$'\n' readarray -t badfiles <<< "$(debsums "${1}" 2>/dev/null)"
  fi

  IFS=$'\n' printf '%s\n' "${badfiles[@]}" >debsums${1}${2}.log

  unset files
  unset packages

  # set IFS to  allow "for" in array with elements containing spaces. Ugly bash!
  local IFS=$'\n'
  for line in "${badfiles[@]}" ; do
    IFS=' ' read -a fields <<< "${line}"
    #echo "DEBUG: line=$line fields=${fields[@]}"

    # set IFS to allow "if" of arrays to compare to strings. Ugly bash!
    local IFS=$' '
    if [ "${#line}" -eq "0" ] ; then
      echo -e ;# skip empty lines
    elif [ "${fields[*]:0:4}" = "debsums: no md5sums for" ] ; then
      local IFS=$'\n'
      package="${fields[@]:4:1}"
      echo "DEBUG: no md5sums package=$package"
      packages=(${packages[@]} "$package")
    elif [ "${fields[*]:0:3}" = "debsums: changed file" ] ; then
      local IFS=$'\n'
      file="${fields[@]:3:1}"
      package=$( dpkg -S ${file} | cut -d : -f 1)
      echo "DEBUG: changed file=$file package=$package"
      files=(${files[@]} "$file")
      packages=(${packages[@]} "$package")
    elif [ "${fields[*]:0:3}" = "debsums: missing file" ] ; then
      local IFS=$'\n'
      file="${fields[@]:3:1}"
      package=$( dpkg -S ${file} | cut -d : -f 1)
      echo "DEBUG: missing file=$file package=$package"
      files=(${files[@]} "$file")
      packages=(${packages[@]} "$package")
    elif [ "${#fields[@]}" -eq "1" ] ; then
      local IFS=$'\n'
      file="${fields[@]:0:1}"
      package=$( dpkg -S ${file} | cut -d : -f 1)
      echo "DEBUG: changed file=$file package=$package"
      files=(${files[@]} "$file")
      packages=(${packages[@]} "$package")
    elif [ "${fields[*]:0:3}" = "debsums: symlink loop" ] ; then
      local IFS=$'\n'
      file="${fields[@]:8:1}"
      package=$( dpkg -S ${file} | cut -d : -f 1)
      echo "DEBUG: symlink loop file=$file package=$package"
      files=(${files[@]} "$file")
      packages=(${packages[@]} "$package")
    else
      local IFS=$' '
      echo "WARNING: unrecognized line='$(declare -p line)' fields='$(declare -p fields)' test1:'${fields[*]:0:4}' test2:'${fields[*]:0:3}'"
    fi

  done

  local IFS=$' '
  if [ "${#files[@]}" -gt "0" ] ; then
    local IFS=$'\n'
    readarray -t files <<< "$(printf '%s\n' "${files[@]}" | sort -u)"
  fi
  local IFS=$' '
  if [ "${#packages[@]}" -gt "0" ] ; then
    local IFS=$'\n'
    readarray -t packages <<< "$(printf '%s\n' "${packages[@]}" | sort -u)"
  fi

  echo "Changed or missing ${#files[@]} files (files${1}${2}.log):"
  echo ${files[*]}
  echo
  IFS=$'\n' printf '%s\n' "${files[@]}" >files${1}${2}.log

  echo "${#packages[@]} packages with changed or missing files (packages${1}${2}.log):"
  echo ${packages[*]}
  echo
  IFS=$'\n' printf '%s\n' "${packages[@]}" >packages${1}${2}.log

  if [ "${3}" = "fix" ]; then
    local IFS=$' '
    if [ "${#packages[@]}" -gt "0" ] ; then
      echo "Reinstalling ${#packages[@]} packages:"
      apt-get install -f --reinstall -o Dpkg::Options::=--force-confmiss ${packages[@]}
      echo
    fi
    
    #local IFS=$'\n'
    #for package in "${packages[@]}" ; do
    #  # special effort to restore missing configuration files from the package
    #  # (it includes scripts in /etc/*, which apt/dpkg will not reinstall by default)
    #  dpkg -i --force-confmiss /var/cache/apt/archives/${package}*.deb
    #done
    
    if [ -f "$(which debsums).patched" ]; then
      echo "Reapplying debsums patch:"
      cp -vf "$(which debsums)" "$(which debsums).orig"
      cp -v  "$(which debsums).patched" "$(which debsums)"
      echo
    fi
    
    echo "Done fixing ${#files[@]} changed or missing files, ${#packages[@]} packages."
  else
    echo "Done scanning: ${#files[@]} changed or missing files, ${#packages[@]} packages."
  fi
  echo

  exit 0
}

case "$1" in
  -i|install)
    do_install
    ;;
  -s|check_sums|"")
    do_debsums_parse "-cs" 0 no_fix
    ;;
  -c|check_changed)
    do_debsums_parse "-as" 1 no_fix
    ;;
  -f|fix_corrupt)
    do_debsums_parse "-c" 0 fix
    ;;
  -m|fix_missing)
    do_debsums_parse "-c" 1 fix
    ;;
  -x|fix_missing_config)
    do_debsums_parse "-as" 1 fix
    ;;
  *)
    echo "Usage: $0 [ -i|install | -s|check_sums | -c|check_changed | -f|fix_corrupt | -m|fix_missing | -x|fix_missing_config ]" >&2
    exit 3
    ;;
esac

Answer 5

Using the dpkg program along with some Bash scripting should be able to this for you. The only caveat would be if someone replace "clean" versions of the files with "malicious" ones. For that you would need to get valid MD5 checksums from a pristine package. Anyways, here is the shell code to achieve what you want:

#!/bin/sh
PACKAGE_NAME="xterm"
for i in `sudo dpkg -L ${PACKAGE_NAME}`
do
    if ! [ -e $i ]; then
            echo "$i is a missing file in the $PACKAGE_NAME package."
    fi
done

The script would only print out if a file or directory that was defined in the package was missing. Also you would need to replace the 'PACKAGE_NAME' variable with the package that you would want to inspect. Hope this helps.