I know how to list all packages installed on my system.

But how could I get a list of all repositories and PPA's into a script that I can run on a new machine to replicate the repository setup including the keys?

I know I can look into /etc/apt/sources.list and /etc/apt/sources.list.d, but I'm looking for a way to generate a script that executes all apt-add-repository commands on a new system (that sorts out getting all keys).

Any ideas?

11 Answers 11

up vote 86 down vote accepted

Thanks for the pointers. With a little cleanup I got a script that lists the PPAs, but not any other repository:

#! /bin/sh 
# listppa Script to get all the PPA installed on a system ready to share for reininstall
for APT in `find /etc/apt/ -name \*.list`; do
    grep -o "^deb http://ppa.launchpad.net/[a-z0-9\-]\+/[a-z0-9\-]\+" $APT | while read ENTRY ; do
        USER=`echo $ENTRY | cut -d/ -f4`
        PPA=`echo $ENTRY | cut -d/ -f5`
        echo sudo apt-add-repository ppa:$USER/$PPA
    done
done

When you call it with listppa > installppa.sh you get a script you can copy on a new machine to reinstall all PPA.

Next stop: do that for the other repositories:

#! /bin/sh
# Script to get all the PPA installed on a system
for APT in `find /etc/apt/ -name \*.list`; do
    grep -Po "(?<=^deb\s).*?(?=#|$)" $APT | while read ENTRY ; do
        HOST=`echo $ENTRY | cut -d/ -f3`
        USER=`echo $ENTRY | cut -d/ -f4`
        PPA=`echo $ENTRY | cut -d/ -f5`
        #echo sudo apt-add-repository ppa:$USER/$PPA
        if [ "ppa.launchpad.net" = "$HOST" ]; then
            echo sudo apt-add-repository ppa:$USER/$PPA
        else
            echo sudo apt-add-repository \'${ENTRY}\'
        fi
    done
done

This should do the trick. I needed a question on superuser to figure out the correct regex.

  • 1
    In your grep -o example, the \` in [a-z0-9\-] is not doing what you expect. It actually matches a literal backslash. You don't need to escape the - when it is at the start or end of the [] list; actually, you can't escape it!.. In this case the \` (probably) won't cause a problem, because you (hopefully) won't encounter a backslash in the deb entry. – Peter.O Sep 7 '12 at 3:51
  • 1
    Note that PPA names may contain dots, so I think you want to change your regexp to http://ppa.launchpad.net/[a-z0-9-]\+/[a-z0-9.-]\+ – kynan Jan 7 '13 at 10:28
  • No, you want to change the regex to [[:graph:]] instead of [a-z...blah.anything] because that will match any alphanumeric + punctuation characters - that is what the PPA names consist of. – MichalH Aug 25 '15 at 13:11
  • I suppose you should include deb word in the beginning of each repository line, if not given in ppa:$USER/$PPA form. – jarno Nov 26 '16 at 17:07
  • @stwissel any particular reason you used find and then grep? You can easily do a glob that the shell parses and pass it to grep. grep -Po "(?<=^deb\s).*?(?=#|$)" /etc/apt/{sources.list,sources.list.d/*.list} | while read ENTRY ; do echo $ENTRY; done Note that as written this shows you the file name each entry comes from so you'd need to do a trim from the beginning of the result to the first colon, but that isn't too hard with cut. You may also want to pass it through uniq if you don't want multiple entries for the same source (eg if you have Google Chrome Stable/Beta/Dev installed). – dragon788 Jul 17 '17 at 19:21

You can show everything with:

grep ^ /etc/apt/sources.list /etc/apt/sources.list.d/*
  • 9
    What about egrep -v '^#|^ *$' /etc/apt/sources.list /etc/apt/sources.list.d/* to remove lines commented out and blank lines? – user25656 Jun 10 '12 at 13:22
  • could you please explain the use of ^ after grep in grep ^ /etc/apt/sources.list /etc/apt/sources.list.d/*? – user25656 Jun 10 '12 at 13:25
  • 2
    @vasa1 The caret ^ and the dollar sign $ are metacharacters that respectively match the empty string at the beginning and end of a line. – wojox Jun 11 '12 at 14:22
  • 2
    I use grep ^[^#] ... -- It auto-hides all commented-out sources – Ross Aiken Oct 30 '13 at 22:36
  • 7
    If you're not going to filter anything out, wouldn't it be simpler to just run cat /etc/apt/sources.list /etc/apt/sources.list.d/* – jbo5112 Nov 25 '13 at 16:28

I am surprised that the simplest but most effective way to get all enabled binary software sources together with the file they're specified in hasn't been posted yet:

grep -r --include '*.list' '^deb ' /etc/apt/sources.list /etc/apt/sources.list.d/

From all processed files, this will print every line starting with deb. This excludes commented lines as well as deb-src lines to enable source code repositories.

It searches really only all *.list files that will be parsed by apt, but e.g. no *.list.save files used for backup or others with illegal names.


If you want a shorter but possibly only in 99.9% of all cases correct output that may search too much files (includes all /etc/apt/sources.list* files and directories, not only /etc/apt/sources.list and `/etc/apt/sources.list.d/*), you could also use this:

grep -r --include '*.list' '^deb ' /etc/apt/sources.list*

Unless there are files that shouldn't be there, the output will be the same.


An example output on my machine would be this:

/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily main restricted
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-updates main restricted
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily universe
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-updates universe
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily multiverse
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-updates multiverse
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-backports main restricted universe multiverse
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-security main restricted
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-security universe
/etc/apt/sources.list:deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-security multiverse
/etc/apt/sources.list:deb http://archive.canonical.com/ubuntu wily partner
/etc/apt/sources.list.d/maarten-fonville-ubuntu-ppa-wily.list:deb http://ppa.launchpad.net/maarten-fonville/ppa/ubuntu wily main
/etc/apt/sources.list.d/webupd8team-ubuntu-tor-browser-wily.list:deb http://ppa.launchpad.net/webupd8team/tor-browser/ubuntu wily main
/etc/apt/sources.list.d/fossfreedom-ubuntu-indicator-sysmonitor-wily.list:deb http://ppa.launchpad.net/fossfreedom/indicator-sysmonitor/ubuntu wily main
/etc/apt/sources.list.d/getdeb.list:deb http://archive.getdeb.net/ubuntu wily-getdeb apps

If you want prettier output, let's pipe it through sed:

grep -r --include '*.list' '^deb ' /etc/apt/ | sed -re 's/^\/etc\/apt\/sources\.list((\.d\/)?|(:)?)//' -e 's/(.*\.list):/\[\1\] /' -e 's/deb http:\/\/ppa.launchpad.net\/(.*?)\/ubuntu .*/ppa:\1/'

And we will see this:

deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily main restricted
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-updates main restricted
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily universe
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-updates universe
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily multiverse
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-updates multiverse
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-backports main restricted universe multiverse
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-security main restricted
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-security universe
deb http://ftp-stud.hs-esslingen.de/ubuntu/ wily-security multiverse
deb http://archive.canonical.com/ubuntu wily partner
[maarten-fonville-ubuntu-ppa-wily.list] ppa:maarten-fonville/ppa
[webupd8team-ubuntu-tor-browser-wily.list] ppa:webupd8team/tor-browser
[fossfreedom-ubuntu-indicator-sysmonitor-wily.list] ppa:fossfreedom/indicator-sysmonitor
[getdeb.list] deb http://archive.getdeb.net/ubuntu wily-getdeb apps
  • Going by the accepted answer, it seems OP wanted PPAs to be shown in the ppa:<user>/<project> form. – muru Mar 4 '16 at 9:02
  • The question actually asks to generate a script that installs/enables all repositories. But the question title is only about listing them. Also the 2nd highest scored answer only lists them as well, but it lists far too much. – Byte Commander Mar 4 '16 at 9:11
  • Nice, but I'd already upvoted. :D – muru Mar 4 '16 at 9:44

I use this command to list all configured software sources (repositories), including currently disabled ones:

cat /etc/apt/sources.list; for X in /etc/apt/sources.list.d/*; do echo; echo; echo "** $X:"; echo; cat $X; done

I use this primarily for troubleshooting; this can certainly be incorporated into scripts but you may want to narrow /etc/apt/sources.list.d/* to /etc/apt/sources.list.d/*.list so you only get currently enabled software sources.

  • Thx for the feedback. cat lists the files as they are, so I would need to manually edit it to generate a script (as stated in the question). The challenge with repositories: if you just copy the files from /etc/apt you don't get the repository keys. This is why I want a script that fetches them for us – stwissel Jun 12 '12 at 11:29

Run the following command:

apt-cache policy | grep http | awk '{print $2 $3}' | sort -u

Source

https://repogen.simplylinux.ch/ will give you a list of all PPAs for your version of Ubuntu. Here is a generated list without source files and no samsung printer ppa:

#------------------------------------------------------------------------------#
#                            OFFICIAL UBUNTU REPOS                             #
#------------------------------------------------------------------------------#


###### Ubuntu Main Repos
deb http://us.archive.ubuntu.com/ubuntu/ yakkety main restricted universe multiverse 

###### Ubuntu Update Repos
deb http://us.archive.ubuntu.com/ubuntu/ yakkety-security main restricted universe multiverse 
deb http://us.archive.ubuntu.com/ubuntu/ yakkety-updates main restricted universe multiverse 
deb http://us.archive.ubuntu.com/ubuntu/ yakkety-proposed main restricted universe multiverse 
deb http://us.archive.ubuntu.com/ubuntu/ yakkety-backports main restricted universe multiverse 

###### Ubuntu Partner Repo
deb http://archive.canonical.com/ubuntu yakkety partner

#------------------------------------------------------------------------------#
#                           UNOFFICIAL UBUNTU REPOS                            #
#------------------------------------------------------------------------------#


###### 3rd Party Binary Repos

#### Flacon PPA - http://kde-apps.org/content/show.php?content=113388
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F2A61FE5
deb http://ppa.launchpad.net/flacon/ppa/ubuntu yakkety main

#### Gimp PPA - https://launchpad.net/~otto-kesselgulasch/+archive/gimp
## Run this command: sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 614C4B38
deb http://ppa.launchpad.net/otto-kesselgulasch/gimp/ubuntu yakkety main

#### Google Chrome Browser - http://www.google.com/linuxrepositories/
## Run this command: wget -q https://dl.google.com/linux/linux_signing_key.pub -O- | sudo apt-key add -
deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main

#### Google Earth - http://www.google.com/linuxrepositories/
## Run this command: wget -q https://dl.google.com/linux/linux_signing_key.pub -O- | sudo apt-key add -
deb [arch=amd64] http://dl.google.com/linux/earth/deb/ stable main

#### Highly Explosive PPA - https://launchpad.net/~dhor/+archive/myway
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93330B78
deb http://ppa.launchpad.net/dhor/myway/ubuntu yakkety main

#### JDownloader PPA - https://launchpad.net/~jd-team
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A68F637
deb http://ppa.launchpad.net/jd-team/jdownloader/ubuntu yakkety main

#### Lazarus - http://www.lazarus.freepascal.org/
## Run this command:  gpg --keyserver hkp://pgp.mit.edu:11371 --recv-keys 6A11800F  && gpg --export --armor 0F7992B0  | sudo apt-key add -
deb http://www.hu.freepascal.org/lazarus/ lazarus-stable universe

#### LibreOffice PPA - http://www.documentfoundation.org/download/
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1378B444
deb http://ppa.launchpad.net/libreoffice/ppa/ubuntu yakkety main

#### MEGA Sync Client - https://mega.co.nz/
deb http://mega.nz/linux/MEGAsync/xUbuntu_16.10/ ./

#### MKVToolnix - http://www.bunkus.org/videotools/mkvtoolnix/
## Run this command: wget -q http://www.bunkus.org/gpg-pub-moritzbunkus.txt -O- | sudo apt-key add -
deb http://www.bunkus.org/ubuntu/yakkety/ ./

#### Mozilla Daily Build Team PPA - http://edge.launchpad.net/~ubuntu-mozilla-daily/+archive/ppa
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys  247510BE
deb http://ppa.launchpad.net/ubuntu-mozilla-daily/ppa/ubuntu yakkety main

#### muCommander - http://www.mucommander.com/
## Run this command: sudo wget -O - http://apt.mucommander.com/apt.key | sudo apt-key add - 
deb http://apt.mucommander.com stable main non-free contrib  

#### Opera - http://www.opera.com/
## Run this command: sudo wget -O - http://deb.opera.com/archive.key | sudo apt-key add -
deb http://deb.opera.com/opera/ stable non-free

#### Oracle Java (JDK) Installer PPA - http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886
deb http://ppa.launchpad.net/webupd8team/java/ubuntu yakkety main

#### PlayDeb - http://www.playdeb.net/
## Run this command: wget -O- http://archive.getdeb.net/getdeb-archive.key | sudo apt-key add -
deb http://archive.getdeb.net/ubuntu yakkety-getdeb games

#### SABnzbd PPA - http://sabnzbd.org/
## Run this command:  sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4BB9F05F
deb http://ppa.launchpad.net/jcfp/ppa/ubuntu yakkety main

#### SimpleScreenRecorder PPA - http://www.maartenbaert.be/simplescreenrecorder/
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 283EC8CD
deb http://ppa.launchpad.net/maarten-baert/simplescreenrecorder/ubuntu yakkety main

#### Steam for Linux - http://store.steampowered.com/about/
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F24AEA9FB05498B7
deb [arch=i386] http://repo.steampowered.com/steam/ precise steam

#### Syncthing - https://syncthing.net/
## Run this command: curl -s https://syncthing.net/release-key.txt | sudo apt-key add -
deb http://apt.syncthing.net/ syncthing release

#### Tor: anonymity online - https://www.torproject.org
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 886DDD89
deb http://deb.torproject.org/torproject.org yakkety main

#### Unsettings PPA - http://www.florian-diesch.de/software/unsettings/
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0FEB6DD9
deb http://ppa.launchpad.net/diesch/testing/ubuntu yakkety main

#### VirtualBox - http://www.virtualbox.org
## Run this command: wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox_2016.asc -O- | sudo apt-key add -
deb http://download.virtualbox.org/virtualbox/debian yakkety contrib

#### Webmin - http://www.webmin.com
## Run this command: wget http://www.webmin.com/jcameron-key.asc -O- | sudo apt-key add -
deb http://download.webmin.com/download/repository sarge contrib

#### WebUpd8 PPA - http://www.webupd8.org/
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4C9D234C
deb http://ppa.launchpad.net/nilarimogard/webupd8/ubuntu yakkety main

#### Xorg Edgers PPA - https://launchpad.net/~xorg-edgers
## Run this command: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8844C542  
deb http://ppa.launchpad.net/xorg-edgers/ppa/ubuntu yakkety main
here is a generated list without source files and no samsung printer ppa
#### Yuuguu - http://yuuguu.com
deb http://update.yuuguu.com/repositories/apt hardy multiverse

Here is my script, "list-apt-repositories", which lists all repositories in "/etc/sources.list" and "/etc/sources.list.d/*.list". You can add --ppa-only to show only the PPAs. PPAs are automatically transformed to ppa:USER/REPO format.

The relevant parts are the 5 lines in list_sources and list_ppa functions, the rest is just boilerplate to wrap it in a handy shell script.

list-apt-repositories:

#!/bin/sh

usage () {
  cat >&2 <<USAGE
$0 [--ppa-only]

Options:
  --ppa-only            only list PPAs
USAGE
  exit $1
}

list_sources () {
  grep -E '^deb\s' /etc/apt/sources.list /etc/apt/sources.list.d/*.list |\
    cut -f2- -d: |\
    cut -f2 -d' ' |\
    sed -re 's#http://ppa\.launchpad\.net/([^/]+)/([^/]+)(.*?)$#ppa:\1/\2#g'
}

list_ppa () {
  list_sources | grep '^ppa:'
}

generate=list_sources

while test -n "$1"
do
  case "$1" in
    -h|--help) usage 1;;
    --ppa-only) generate=list_ppa;;
    *)
      printf -- "Unknown argument '$1'\n" >&2
      usage 2
    ;;
  esac
  shift
done

$generate

And to make an install script, pipe into another script "make-apt-repository-install-script". The generated script supports the -y/--yes argument for non-interactive use (see add-apt-repository(1)).

make-apt-repository-install-script:

#!/bin/sh

if test -n "$1"
then
  cat >&2 <<USAGE
Usage: $0 < PATH_TO_LIST_OF_REPOS
       list-apt-repositories [--ppa-only] | $0

No options recognized.

Reads list of repositories from stdin and generates a script to install them
using \`add-apt-repository(1)\`. The script is printed to stdout.

The generated script supports an optional
\`-y\` or \`--yes\` argument which causes the \`add-apt-repository\` commands
to be run with the \`--yes\` flag.
USAGE
  exit 1
fi

cat <<INSTALL_SCRIPT
#!/bin/sh
y=
case "\$1" in
  -y|--yes) y=\$1;;
  '') y=;;
  *)
    printf '%s\n' "Unknown option '\$1'" "Usage: \$0 [{-y|--yes}]" >&2
    exit 1
  ;;
esac
INSTALL_SCRIPT

xargs -d'\n' printf "add-apt-repository \$y '%s'\n"

Again, the important part is the xargs command on the last line, the rest is boilerplate.

To have it add ppa.launchpad.net lines as ppa:$USER/$PPA. Add other repositories with their full line from *.list files. No dupe lines.

#!/bin/bash
# My ~/bin/mk_repositories_restore_script
mkdir -p ~/bin 
x=~/bin/restore_repositories
echo \#\!/bin/bash > $x
chmod u+x $x
(
 for APT in $( find /etc/apt/ -name \*.list )
    do sed -n -e '/^deb /{
     /ppa\.launchpad/s/\(.*\/\/[^\/]*.\)\([^ \t]*\)\(.*$\)/sudo apt-add-repository ppa:\2/p;
     /ppa\.launchpad/!s/\(deb[ \t]*\)\(.*$\)/sudo apt-add-repository \2/p;
    }' $APT
 done
) | sort | uniq | tee -a ~/bin/restore_repositories

So, doing some digging, we have AptPkg::Class.

So using perl we can do something simple like this..

perl -MAptPkg::Cache -MData::Dumper -E'say Dumper [AptPkg::Cache->new->files()]' | less

This gets us a list of all the AptPkg::Class::PkgFile packages. You could probably generate the apt-add-repository commands with that.

Thanks BobDodds!
If anybody would be interested, I have updated your code a little (hope you don't mind)..
This script will type out only user added PPAs (/etc/apt/sources.list.d).

    #!/bin/bash
    # My ~/bin/mk_repositories_restore_script
    mkdir -p ~/bin
    x=~/bin/restore_repositories
    echo \#\!/bin/bash > $x
    chmod u+x $x
    (
    for APT in $( find /etc/apt/ -name \*.list )
    do sed -n -e '/^deb /{
          /ppa\.launchpad/s/\(.*\/\/[^\/]*.\)\([^ \t]*\)\(.*\/ubuntu.*$\)/ppa:\2/p;                                                                                                                                                                                       
        }' $APT
    done
    ) | sort | uniq | tee -a ~/bin/restore_repositories
sed -r -e '/^deb /!d' -e 's/^([^#]*).*/\1/' -e 's/deb http:\/\/ppa.launchpad.net\/(.+)\/ubuntu .*/ppa:\1/' -e "s/.*/sudo add-apt-repository '&'/" /etc/apt/sources.list /etc/apt/sources.list.d/*

That does not generate commands to enable possible source repositories (deb-src) though.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.