Is there a simple way to configure apt-get to optimize downloads using a lan apt-mirror when available?

Question

I have a laptop and a local copy of several repositories in an external hard drive connected to my router. My router provides access to those files (with credentials) as samba shares and/or ftp server.

I would like to configure apt-get to check those repositories when they are available and download updates from them to speed up the process when possible.

To be more clear: I want apt-get to check both the regular repositories in the internet and my local one when my laptop is connected to my wifi. If files in the external repositories are newer than the ones in my local mirror or if the mirror is not available, apt-get should download the files from the internet.

Answer 1

There are 4 steps to setting up your own repository

1. Install dpkg-dev
2. Put the packages in a directory
3. Create a script that will scan the packages and create a file apt-get update can read
4. Add a line to your sources.list pointing at your repository

Install dpkg-dev

Open a terminal:

sudo apt-get install dpkg-dev

The Directory

Create a directory to keep your packages. For this example, we'll be using /usr/local/lan-repo.

sudo mkdir -p /usr/local/lan-repo

Now move all your packages into that new directory.

Packages that have been downloaded already are usually stored on your system in /var/cache/apt/archives. If you have installed apt-cacher you will most likely have additional packages stored in the apt-cacher/packages directory.

Create a script named update-lan-repo

It's a very simple three line script:

#! /bin/bash
cd /usr/local/lan-repo
sudo dpkg-scanpackages . /dev/null | gzip -9c > packages.gz

Copy and paste the above into your favorite text editor, and save it as update-lan-repo in ~/bin. (~ means the home directory. If ~/bin doesn't exist already then create it. Ubuntu will automatically put that directory in your PATH. This makes it a good place to put personal scripts.)

Now let's make the script executable:

chmod u+x ~/bin/update-lan-repo

Here's how the script works:

dpkg-scanpackages looks at all the packages in lan-repo, and the output is compressed and written to the file packages.gz, which apt-get update can read (see below for a reference that explains this in full detail). /dev/null is an empty file - that is - it's a makeshift for an override file which holds some additional information about the packages, which in our case is not really needed. See deb-override(5) if you want to find out more about it.

We need to add your local repo to your sources.list file

echo "deb file:/usr/local/lan-repo ./" >> /etc/apt/sources.list

and that's it you're done.

CD/DVD Option

You can store the directory containing the .deb file to a CD/DVD and use that as a repository as well (good for sharing between computers). To use the CD as a repository:

simply run the command:

sudo apt-cdrom add

using the new repository.

Then whenever you store a new debian package in the lan-repo directory, run:

sudo update-lan-repo
sudo apt-get update

Now your local packages can be manipulated with Synaptic, aptitude and the apt commands: apt-get, apt-cache, etc. When you attempt to apt-get install, any dependencies will be resolved for you, as long as they can be met.

Answer 2

I finally managed to solve this and it's working pretty well, so I want to share it with people that want to implement something similar.

My first problem was that my system supports multi-arch, so the mirrored repositories should contain both i386 and amd64 packages (I did not mirrored sources) to avoid downloading errors. This is done in /etc/apt/mirror.list like this example shows:

deb-i386  http://archive.ubuntu.com/ubuntu trusty main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-security main restricted universe multiverse

My base_path defined in that file is /mnt/apt-mirrors, and I run a script that mounts an external hard drive connected to the USB port of my router as a Samba share (you can see the script here: how to use apt-mirror to save files in HDD connected to LAN router

I wanted to access the mirrored repositories from other computers in the LAN, so the other part of the solution is adding the FTP repositories definitions at the beginning in /etc/apt/sources.list. This setup makes apt-get to use the FTP repositories first when downloading files and then use the ones defined in /etc/apt/sources.list.d, provided the mirrored ones are up to date.

The local repositories are defined like this:

deb [arch=amd64,i386] ftp://user:password@lan-resource/apt-mirrors/mirror/archive.ubuntu.com/ubuntu trusty main restricted universe multiverse

where you replace user with the user name that has access granted to the FTP server, :password with its password if required to connect (otherwise remove), and lan-resource with the address of the FTP server in your LAN (I decided to edit /etc/hosts to avoid repeating hard-coded IP address, where I defined 192.168.0.1 as lanftpserver to use in this field).