Files list file for package 'package' is missing final newline

Question

To resolve this issue I have run these commands

sudo mv /var/lib/dpkg/info/<package name>.* /var/www
sudo apt-get install <package name>

But when I try to install any new packages or upgrade using the below command it gives errors for other packages.

What could have caused this problem and how can I resolve it?

$ sudo apt-get dist-upgrade
dpkg: warning: files list file for package 'libexpat1:amd64' missing;
 assuming package has no files currently installed
dpkg: warning: files list file for package 'base-passwd' missing;
 assuming package has no files currently installed
dpkg: warning: files list file for package 'bash' missing;
 assuming package has no files currently installed
dpkg: warning: files list file for package 'debconf' missing;
 assuming package has no files currently installed
dpkg: warning: files list file for package 'dash' missing;
 assuming package has no files currently installed   
dpkg: warning: files list file for package 'coreutils' missing;
 assuming package has no files currently installed
dpkg: warning: files list file for package 'python3-software-properties' missing;
 assuming package has no files currently installed
dpkg: warning: files list file for package 'software-properties-common' missing;
 assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting: 
 files list file for package 'libcap-ng0:amd64' is missing final newline
E: Sub-process /usr/bin/dpkg returned an error code (2)

$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  lxd lxd-client open-iscsi openssl tmux tzdata unattended-upgrades
7 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/9806 kB of archives.
After this operation, 604 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Preconfiguring packages ...
dpkg: warning: files list file for package 'libexpat1:amd64' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'base-passwd' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'bash' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'debconf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'dash' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'coreutils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'base-files' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'diffutils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'python3-software-properties' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ncurses-bin' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'software-properties-common' missing; assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting:
 files list file for package 'libcap-ng0:amd64' is missing final newline
E: Sub-process /usr/bin/dpkg returned an error code (2)

$ sudo apt-get install --reinstall base-passwd
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 7 not upgraded.
Need to get 0 B/47.2 kB of archives.
After this operation, 0 B of additional disk space will be used.
Preconfiguring packages ...
dpkg: warning: files list file for package 'libexpat1:amd64' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'base-passwd' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'bash' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'debconf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'dash' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'coreutils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'base-files' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'diffutils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'python3-software-properties' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ncurses-bin' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'software-properties-common' missing; assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting:
 files list file for package 'libcap-ng0:amd64' is missing final newline
E: Sub-process /usr/bin/dpkg returned an error code (2)

$ sudo chmod -R a+r /var/lib/dpkg/info
$ cd /var/lib/dpkg/info

/var/lib/dpkg/info$ sudo mv libcap-ng0\:amd64.
libcap-ng0:amd64.list      libcap-ng0:amd64.md5sums   libcap-ng0:amd64.shlibs    libcap-ng0:amd64.triggers

/var/lib/dpkg/info$ sudo mv libcap-ng0\:amd64.
libcap-ng0:amd64.list      libcap-ng0:amd64.md5sums   libcap-ng0:amd64.shlibs    libcap-ng0:amd64.triggers

/var/lib/dpkg/info$ sudo mv libcap-ng0\:amd64.* /var/www/html/

/var/lib/dpkg/info$ sudo apt-get install --reinstall libcap-ng0:amd64
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 7 not upgraded.
Need to get 11.1 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu bionic/main amd64 libcap-ng0 amd64 0.7.7-3.1 [11.1 kB]
Fetched 11.1 kB in 0s (39.2 kB/s)
dpkg: warning: files list file for package 'libexpat1:amd64' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'base-passwd' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'bash' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'debconf' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'dash' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'coreutils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'base-files' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'diffutils' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'python3-software-properties' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'libcap-ng0:amd64' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'ncurses-bin' missing; assuming package has no files currently installed
dpkg: warning: files list file for package 'software-properties-common' missing; assuming package has no files currently installed
dpkg: unrecoverable fatal error, aborting:
 files list file for package 'libcom-err2:amd64' is missing final newline
E: Sub-process /usr/bin/dpkg returned an error code (2)

I have removed libcap-ng0:amd64 files from /var/lib/dpkg/info folder and later tried to reinstall it but now it has provided error for other package and warning for libcap-ng0.
It has occurred for rest packages in same way

Answer 1

Here are step by step instructions to fix this problem, which appears to be file corruption in the .list files related to these pacakges:

1. Verify that the .list files are full of garbage:

cat /var/lib/dpkg/info/<name-of-package>.list

2. Remove the corrupt .list file:

sudo rm /var/lib/dpkg/info/<name-of-package>.list

3. Run the reinstall for that package (will show a warning, but should reinstall):

sudo apt install <name-of-package> --reinstall

If what happened to me happens to you, each time you run this it will run into another corrupted .list file and you will have to step through many before your system is fixed. If you have many, then step three is optional -- just run one sudo apt upgrade at the end and it will reinstall everything you need.

Answer 2

From: apt-get dpkg error: files list file ... missing final newline - LinuxQuestions.org

Well, with a big help from #debian at freenode, I found the solution (thanks justin!!):

the file /var/lib/dpkg/info/smbfs.list was corrupted, it was all binary crap. Dunno how this could happen, but it was. So justin advised me to delete the file (actually, I moved it) - and, voila, apt warned me that the files list could not be found for smbfs, but at least I could apt-get remove smbfs --purge and install it again (apt-get install smbfs) - that took the biscuit!

Answer 3

cd /var/lib/dpkg/info/; 
for i in *; do 
  if (file $i|grep -P '(?<!__DAMAGED:)[\s]data$'); then 
    sudo mv -v "$i" "${i}__DAMAGED"; 
  fi; 
done; 
ls *__DAMAGED | cut -d'.' -f1 - | uniq | xargs -I'{}' sudo apt-get install {} --reinstall

Its just the same code mentioned by @Gergely Lukacsy but corrected few errors.

Answer 4

If you're getting 50-80 damaged files on multiple machines like me, the manual method will be an endless task, so here's my take on the issue (Tested on Ubuntu 20.04). This is basically an automated version of yourcomputergenius' answer:

cd /var/lib/dpkg/info/; \
for i in *; do \
  if (file $i|grep -P '(?<!__DAMAGED:)[\s]data$'); then \
    sudo mv -v "$i" "$i__DAMAGED"; \
  fi; \
done; \
ls *__DAMAGED | cut -d'.' -f1 - | uniq | xargs -I'{}' sudo apt-get install {} --reinstall

What it does:

• checks all files in the directory
• renames all files with binary content by appending __DAMAGED to the filename
• then:
  • sorts out all damaged files
  • extracts the package names of the filenames
  • drops duplicate package names
  • then feeds the remaining list to apt to reinstall the affected packages

I hope it helps anybody who encounters this problem!