0

I have a machine that is online. It has the setup i need on a machine that is not online. Ideally there would be a trivial way to copy/clone the installed packages.

Alas, i did not find this to be trivial. I tried several proposed solutions:

  • Mike, give Keryx another look. I just released a new version (0.92.5). – Michael Crenshaw Jun 21 '17 at 15:43
2

I wrote a simple python util to use apt list --installed and apt download <pkg> to create the "cache" folder i need to use with aptoncd. It will also update the cache folder without re-downloading anything.

USAGE: <cmd> <cache_dir> [optional_regex_pkgs_to_ignore|...]

  • cmd --> the filename where you save this script
  • cache dir --> folder to save the cached output in. If it contains existing .debs, they will not be re-downloaded. A good location is /var/cache/apt/archive on ubuntu16.04 (if you have sudo)
  • pkgs_to_ignore --> space-separated list of packages to ignore, in regex format. Eg, 'google.*' will skip googleearth and googleearth-package. Note the ' versus " - this prevents bash from expanding the .* in the regex.
    • Necessary because this apt approach won't work for the rare program installed fully outside apt, eg Google-Earth for me.

Hope this helps someone. Seemed a little different than the existing (numerous) questions. Note, this does not try to re-invent the package-manager wheel; it does not care about package version updates if the name is unchanged. If that matters, just point it to a different cache folder and it will re-download everything.

 #!/usr/bin/python3

'''
program to make a flat folder of all packages your system uses:
    - pulls all installed packages: 
        -`apt list --installed` 
    - downloads any packages/.deb's that are not in your cache folder
        - `apt download <package>`

This was created to facilitate setting up / updating an off-line
machine based on an existing machine (presumably online) where you 
have the programs you need set up.

the easiest way to get the packages over to the offline machine is via
tool `aptoncd`  ("sudo apt install aptoncd"), which will make an iso of
your packages, and optionally auto-launch a CD writer.

--- USAGE ---
<cmd> <working_dir> [ignore-pkg-globs|...]
'''

import sys
from os import path
from glob import glob
import os
import subprocess
import re

NARGS=2
DEBUG=0

def get_all_installed_packages():
    print("==== asking apt for a list of all installed packaged ====")
    result = subprocess.run(["apt", "list", "--installed"],
                            stdout=subprocess.PIPE,
                            stderr=subprocess.STDOUT,
                            universal_newlines=True)
    if not 0 == result.returncode:
        msg = "apt list call failed with " + str(result.returncode)
        print(msg)
        raise RuntimeError(msg)

    split_re = re.compile(r"(.*)/(.*)(\[.*\])")
    packages = []
    for line in result.stdout.split(os.linesep):
        contents = split_re.match(line)
        if contents:
            groups = [item.strip() for item in contents.groups()]
            if DEBUG:
                print("found pkg: <{0}>  -- {1}|{2}".format(*groups))
            packages.append(groups[0])
        else:
            print("IGNORING <" + line + ">:")

    if len(packages) < 20:
        msg = "ERROR: you only found <{0}> packages, this seems wrong!".format(len(packages))
        print(msg)
        raise RuntimeError(msg)

    print("==== found {0} installed packages! ====".format(len(packages)))

    return packages

def download_packages(pkg_list, ignore_pkgs):
    if not pkg_list:
        print(" ==== nothing to download! ====")
        return
    print("==== downloading the following (missing) packages: ====")
    print(" | ".join(pkg_list))

    for pkg in pkg_list:
        if DEBUG:
            print("processing package <{0}>".format(pkg))
        ignore = 0
        for pattern in ignore_pkgs:
            if pattern.match(pkg):
                if DEBUG:
                    print("matched <{0}> to ignore pattern <{1}>".format(
                           pkg, pattern.pattern))
                ignore += 1

        if ignore:
            print("---- ignoring {0} {1} ----".format(pkg, ignore))
        else:
            print("---- downloading {0} ----".format(pkg))
            result = subprocess.run(["apt", "download", pkg],
                                stdout=subprocess.PIPE,
                                stderr=subprocess.STDOUT,
                                universal_newlines=True)
            if 0 != result.returncode:
                msg = "ERROR: run apt download failed with <{0}>. "
                msg += "output=\n{1}\n------\n\n".format(
                    result.returncode, result.stdout)
                raise RuntimeError(msg)

    print("==== DONE DOWNLOADING ====")

def main():
    if len(sys.argv) < NARGS:
        print(__doc__)
        msg = "ERROR, requires >= {0} args!".format(NARGS-1)
        print(msg)
        sys.exit(1)

    print("args=" + ";".join(sys.argv))

    ignore_pkgs_re = [re.compile(x) for x in sys.argv[2:]]
    package_dir = path.abspath(path.normpath(sys.argv[1]))

    if not path.isdir(package_dir):
        msg = "ERROR: path <{0}> is not a dir!".format(package_dir)
        print(msg)
        raise ValueError(msg)

    save_cwd = os.getcwd()
    try:
        os.chdir(package_dir)
        existing_debs = glob("*deb")
        for deb in existing_debs:
            if DEBUG > 1:
                print("found deb = " + deb)
            pass

        installed_packages = get_all_installed_packages()

        to_download = []
        for pkg in installed_packages:
            matching_deb = [x for x in existing_debs if x.startswith(pkg)]
            if matching_deb:
                if DEBUG:
                    print("found existing match for <{0}> with <{1}>".format(
                          pkg, "|".join(matching_deb)))
            else:
                if DEBUG:
                    print("no existing cache deb found for pkg <" + pkg + ">")
                to_download.append(pkg)

        download_packages(to_download, ignore_pkgs_re)

    finally:
        os.chdir(save_cwd)

if __name__ == "__main__":
    print("running as main")
    main()

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.