I installed a plain Ubuntu 10.10 on my computer and installed some software via apt-get. Last week I managed to break everything and just started from scratch, and I need to reinstall my software. Is there some way to create a list with all the packages that I have installed manually?

So that it would give me a list like texlive, ... and not texlive, texlive-dep1, textlive-dep2, ... and all the standard packages removed?

If I could somehow figure out which programs out of the regular install I have removed, that would be awesome too!


Update 2015-05-23: I now use Ansible as configuration management on my systems. There I specify the packages to install. This serves the goal, installing all the programs on a fresh system, even better. So this question might be the entry into some light configuration management.

15 Answers 15

With this suggestion, I'm assuming your old installation will still boot!

To replicate one set of packages on another machine:

On System A, run:

dpkg --get-selections | grep -v deinstall > my-selections

Move the my-selections file over to System B.

On System B, run:

dpkg --set-selections < my-selections

and then:

sudo apt-get dselect-upgrade

Important note: if you have installed packages from non-standard repositories and/or PPAs, you will also want to copy /etc/apt/sources.list and the contents of /etc/apt/sources.list.d/ from System A to System B before you run the upgrade.

You can use dpkg to see what you've removed as well (NB: this will also include packages that you manually installed and removed):

dpkg --get-selections | grep deinstall

You can see your results in the terminal, or, of course, redirect to a file.

  • 9
    I would like to get a list of all packages, but I would like to have only the names of packages that I explicitly installed. So just something and not something-common as well. – Martin Ueding Apr 5 '11 at 14:15
  • 6
    Yeah while this is useful, it doesn't do what the question asks. – Timmmm Dec 21 '12 at 20:54
  • 3
    If you do this, then auto-installed dependencies will never get cleaned when you remove programs. Why do people up-vote an answer that does what the asker specifically said he didn't want? – jbo5112 May 23 '15 at 23:23

This thread from superuser.com gives this solution:

aptitude search '?installed ?not(?automatic)'
  • 2
    This doesn't work for me. Just lists everything that is installed, or certainly more than I have explicitly asked for. – Timmmm Dec 21 '12 at 20:54
  • It lists everything you have installed with apt-get, plus their dependencies. So, for example, if you installed aptitude, then it will also list apt-xapian-index aptitude-common libboost-iostreams1.54.0 libclass-accessor-perl libcwidget3 libept1.4.12 libio-string-perl libparse-debianchangelog-perl libsigc++-2.0-0c2a libsub-name-perl libtimedate-perl libxapian22 python-chardet python-debian python-six python-xapian – Runemoro Jun 7 '14 at 14:43

If your apt logs are in /var/log/apt, something like this should work:

gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install"

Or if you want to get rid of some of the repetitive junk in the output:

gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install" \
  | cut -f4- -d" " | sort | uniq

You could use apt-mark, but I recommend debfoster:

sudo apt-get install debfoster
sudo debfoster

This will inspect all installed packages and figure out which ones are keeping the others installed:

texlive-full is keeping the following 161 packages installed:
  cm-super cm-super-minimal context doc-base dvipng feynmf
  fonts-gfs-artemisia fonts-gfs-baskerville fonts-gfs-bodoni-classic
  ...
Keep texlive-full? [Ynpsiuqx?], [H]elp:

As you answer "y" to each question (just push Enter to move quickly), debfoster will collect the package list and write them line-by-line to a file. By default this is at /var/lib/debfoster/keepers. It looks like this:

gnome-do
texlive-full
...

I configure debfoster via /etc/debfoster.conf to put this list at /etc/debfoster-keepers and track the file with etckeeper to keep history and backups. The answer here shows how to install a list of packages from a newline-delimited text file:

sudo apt-mark manual $(cat debfoster-keepers)

Note a limitation here, packages you purged have a '-' in front of them. So you want to remove those lines before calling apt-mark.

Even though the debfoster's website says that debfoster is deprecated in favor of aptitude, I prefer debfoster's prompt and simple configuration. It puts you in the middle of your package database and lets you clean things up, making the auto and manual packages more obvious.

Type "h" at the debfoster prompt to explain your options. Type "?" to see the package description. The how-to here might be useful.

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)

Gives all manually installed packages (not system packages, not dependencies). For examples it shows build-essential but not gcc.

up vote 2 down vote accepted

I finally got it now:

outfile="$(mktemp)"
pattern='(\[INSTALLIEREN\]|\[INSTALL\])'

if [[ -f "/var/log/aptitude.1.gz" ]]
then
        gunzip -c /var/log/aptitude.*.gz | grep -E "$pattern" | awk '{ print $2; }' > "$outfile"
fi

if [[ -f "/var/log/aptitude" ]]
then
        grep -E "$pattern" "/var/log/aptitude" | awk '{ print $2; }' >> "$outfile"
fi

sort "$outfile"
rm "$outfile"
  • 2
    I guess this only works if you only ever installed packages using aptitude. There are similar files in /var/log/apt/history.log(.N.gz) though. – Timmmm Dec 21 '12 at 20:56
  • You can use zgrep instead of gunzip or zcat piped to grep. It's a wrapper script that ships with gzip i believe. It also accepts uncompressed files, so you could simplify your script to zgrep -E '(\[INSTALLIEREN\]|\[INSTALL\])' /var/log/aptitude* | awk '{ print $2 }' – Steve Buzonas Apr 14 '15 at 13:14

I'm able to pull everything by opening the log files in /var/log/apt/

I then go in and manually filter out the apt-get install packages. There may be a way to do this programatically but I'm not aware of it.

For those who prefer using apt-* tools, there's an apt-mark utility that allows both querying and modifying manual/auto state.

apt-mark showmanual

You can also specify pattern for packages you interested in:

apt-mark showmanual qemu\*
cd /var/log/apt
cat  history.log | grep Commandline

this way you see the list of commands performed in the past.

Remove the grep code if you need more information:

less history.log

if the log is compressed (i.e. end with gz)

gunzip <filename>

to zip it again when finished you can do:

gzip <filename>
  • 1
    Instead of uncompressing and recompressing the log files, why not use zcat? It is as easy as zcat filename.gz. One can just use it on gzip compressed files in place of cat. Also grep can be replaced with zgrep, e.g. zgrep Commandline history.log.1.gz. – lgarzo Jul 2 '13 at 16:18
  • Based on your idea I've come up with a command line: (grep "Commandline" history.log ; zgrep "Commandline" history.log.*.gz ) | grep " install " | grep -v -- "--yes" | sed -r "s/^.*install //". It has flaws, because the removed packages are also listed, and if a package was installed more than once, it is listed multiple times. – lgarzo Jul 2 '13 at 16:31

I didn't see any solutions here work for me, I have installed quite a few deb packages with dpkg and a few of the items I was particularly looking for were missing.

A rather lengthy one liner, but convenient to copy and paste would be:

export DPKG_INITIAL=$(mktemp) DPKG_INSTALLED=$(mktemp) DPKG_CUSTOM=$(mktemp) DPKG_DEPS=$(mktemp) zgrep -E '^Package' /var/log/installer/initial-status.gz | awk '{ print $2 }' > $DPKG_INITIAL ; awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u > $DPKG_INSTALLED ; comm -23 installed initial > $DPKG_CUSTOM ; function rdep() { apt-cache rdepends $1 | tail -n +3 | sed -e 's/^ //' -e '/^ /d' | cut -d':' -f1 | sort -u; } ; echo "$(for i in $(cat custom) ; do rdep $i ; done)" | sort -u > $DPKG_DEPS ; comm -23 custom deps > my-packages ; rm $DPKG_INITIAL $DPKG_INSTALLED $DPKG_CUSTOM $DPKG_DEPS

The above command saves a list of packages to a file in your current working directory named my-packages.

Explanation

I first built a list of packages that composed the baseline of packages selected during installation.

zgrep -E '^Package' /var/log/installer/initial-status.gz | awk '{ print $2 }' > initial

Followed by a long list of items installed in general.

awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u > installed

I then compared the two files initial and installed to list only the items unique to installed.

comm -23 installed initial > custom

From there I wanted to filter out the dependencies, this is where this method may miss some desired packages, it is unaware of dependencies that are also explicitly installed.

I wrote a quick bash function to shorten this steps in processing these items.

function rdep() { apt-cache rdepends $1 | tail -n +3 | sed -e 's/^ //' -e '/^ /d' | cut -d':' -f1 | sort -u; }

After that I passed each line from my file custom into this function with xargs.

echo "$(for i in $(cat custom) ; do rdep $i ; done)" | sort -u > deps

Once I had the long list of every possible dependency, (not sure on the every possible statement), I once again got the lines that were unique to a single file.

comm -23 custom deps > manual

And my finished list of packages is now in a file named manual available for me to review.

Someone linked to https://unix.stackexchange.com/questions/3595/ubuntu-list-explicitly-installed-packages/3624#3624 which does have a good solution, but, it doesn’t behave correctly as the output from aptitude has changed. Here is an updated version, based around compared currently installed packages compared to 12.04 LTS. You will need aptitude installed, that is the only requirement.

aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' > currentlyinstalled && wget -qO - http://mirror.pnl.gov/releases/precise/ubuntu-12.04.3-desktop-amd64.manifest | cut -f 1 | sort -u > defaultinstalled && comm -23 currentlyinstalled defaultinstalled

To break the above command down into parts, this bit outputs one package per line of everything installed on the system

aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' > currentlyinstalled

And this downloads the default package list & crops the redundant information.

wget -qO - http://mirror.pnl.gov/releases/precise/ubuntu-12.04.3-desktop-amd64.manifest | cut -f 1 | sort -u > defaultinstalled

and comm compares the two files and outputs the packages which arent in the default list.

comm -23 currentlyinstalled defaultinstalled
  • You can also go to the other site and update the answer there. – Martin Ueding Oct 16 '13 at 7:19

Use a distribution manifest file as the base package set. Sort the manually installed results into package groups by architecture and section, so it's easier to focus on groups of packages (maybe you don't care about some sections).

https://gist.github.com/darrenleeweber/8cc570ff402f19af7fa4

#!/bin/bash

manifest_url='http://releases.ubuntu.com/releases/trusty/ubuntu-14.04.3-desktop-amd64.manifest'
manifest_file=$(echo $manifest_url | sed -e 's#.*/##g')
if [ ! -e $manifest_file ]; then
    wget -q $manifest_url
fi
cat $manifest_file | cut -f1 | sort -u > default_installed.txt

aptitude search '~i !~M' -F '%p' --disable-columns | sort -u > currently_installed.txt

comm -23 currently_installed.txt default_installed.txt > manually_installed.txt

# sort the 'mannually_installed.txt' packages by architecture and section
mkdir -p package_files
while read p; do
    apt-cache show $p > info.txt
    arch=$(grep -m1 'Architecture: ' info.txt | sed -e 's/Architecture: //')
    section=$(grep -m1 'Section: ' info.txt | sed -e 's/Section: //' -e 's/\//_/g')
    file="${arch}_${section}_packages.txt"
    echo $p >> "package_files/$file"
done <manually_installed.txt

rm info.txt
  • It would be easy to modify the package files so each line is prefixed with apt-get install -y and the file is a bash script. Then all the package files could be included in a vagrant provision loop. – Darren Weber Sep 29 '15 at 5:50
#! /bin/sh
DEFAULT=`tempfile`
CURRENT=`tempfile`
cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' | sort -u > $DEFAULT
aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' | awk '{ print $1}' > $CURRENT
comm -23 $CURRENT $DEFAULT
rm $DEFAULT
rm $CURRENT

I didn't find one that was suitable for me, so I wrote a script. By analysis of the contents of /var/log/apt/history.log* and /var/log/aptitude, then sort by datetime, resulting in two lists, one is manually installed pkgs, another is installed but removed pkgs. Script with 'apt list' verification. a little deviation, because the use of dpkg installed and modified PPA source.

I put here, https://github.com/eexpress/eexp-bin/blob/master/self-installed-pkg.pl.

if anyone need test it, maybe need modify some code, for my log files are mixed in Chinese and English (specially RFC2822 datetime format), I need deal with it. enter image description here

  • What is that unreadable wall-of-text screenshot? – muru Mar 21 '17 at 9:56
  • It's my script output. But today I find the dnf from Fedora is more effectively. "dnf history" show a pure history of packages operation. – utopic eexpress Mar 22 '17 at 9:14

I think you want to use /var/log/apt/history.log and its friends to give the changes that have been made:

zcat /var/log/apt/history.log.*.gz | cat - /var/log/apt/history.log | grep -Po '^Commandline: apt(?:-get)? install (?!.*--reinstall)\K.*'

(Modified from source)

Will give you all the packages that have been installed at the command line using a variation of apt-get install.

It gets the zipped files in the old history.logs adds them with the current history.log and passes through grep, extracting lines with apt[-get] install [and/or reinstall] and showing the rest of the line (-o flag) which amounts to the package name(s).

This'll need a little modding (eg with sed) if you want just the package names on separate lines; an exercise for the reader!

It's worth noting that the synaptic package manager (gksu synaptic or kdesudo synaptic) in the "status" section has a "installed(manually)" list. If you mark the whole list for removal [don't apply it!!] you should be able to save the marked changes and get a list of packages that way. Caution: this locked up synaptic for me (calculating the dependency removals presumably).

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.