Finally after much troubleshooting I found the issue and the solution to resolve it. Writing all findings here so that it can help others and may not need to do the lengthy troubleshooting that I had to do.
Issue, if I am not wrong, was with the disk space. During one of the recent upgrades my disk space got full and 0% was available for upgrades. Because of this /var/lib/dpkg/status is corrupted. And after that it just refuse to upgrade, install or remove any application and gave the error mentioned above.
Steps to resolve the issue:
- First, fix the reason behind mess. Cleaned up few old and unwanted files to free up disk space. Now I have almost 5% available disk space which should be sufficient for normal behaviour and complete the original upgrade which failed because of insufficient disk space.
- Now, repair the damage it caused. Correct file /var/lib/dpkg/status. Here, as usual, experts and their contribution on this forum helped me to get the previous known good status file from /var/backups. Every time you do an install or a update, the status file is saved to a gzipped backup under /var/backups. Refer here for complete details on this. Below are the commands that I used -
~$ sudo mv /var/lib/dpkg/status /var/lib/dpkg/status_bkup
~$ sudo cp /var/backups/dpkg.status.2.gz /var/lib/dpkg/
~$ sudo gunzip -d /var/lib/dpkg/dpkg.status.2.gz
~$ mv /var/lib/dpkg/dpkg.status.2 /var/lib/dpkg/status
~$ sudo apt-get update
voila!! it completed the update without errors faced earlier.
- Now, to complete the update and ensure that the problem is resolved; try update or installing a new application-
~$ sudo apt-get -f install
Fetched 14.3 MB in 15s (935 kB/s)
Extracting templates from packages: 100%
Preconfiguring packages ...
dpkg: error: parsing file '/var/lib/dpkg/status' near line 18299 package >'libgdk-pixbuf2.0-common':
end of file during value of field 'Original-Maintainer' (missing final >newline)
E: Sub-process /usr/bin/dpkg returned an error code (2)
Somehow, status file was again modified and a new error was introduced. Since now I know what is status file and how to troubleshoot is, I understood that this is because of a syntax error in status file. It was easy to identify that '>' is missing for one of the packages at end. Corrected the status file to include '>' at end.
Try again and it was successful this time!