156

I want to update my sources.list file with the fastest server from the command line in a fresh Ubuntu Server install. I know this is trivially easy with the GUI, but there doesn't seem to be a simple way to do it from from the command line?

5
  • 3
    In regular expressions, the . character means any character. If you want it to match a ., you need to escape it with \, so us.archive[..] should be us\.archive[..]
    – Egil
    May 4 '11 at 7:13
  • Related: askubuntu.com/questions/37753/… Apr 6 '12 at 19:13
  • 2
    In my case I had to replace the # signs with slashes (/). Otherwise I got sed: -e expression #1, char 53: unterminated s' command`. Oct 18 '13 at 21:18
  • @EthanLeroy same here with Ubuntu 12.04.3
    – logoff
    Jan 10 '14 at 11:44
  • Should be slash not hash.
    – hookenz
    May 19 '14 at 22:20

14 Answers 14

26
Pakket netselect-apt

    dapper (net): Choose the fastest Debian mirror with netselect 
    [universe]
    0.3.ds1-5: all
    hardy (net): Choose the fastest Debian mirror with netselect 
    [universe]
    0.3.ds1-11: all
Pakket apt-spy

    dapper (admin): writes a sources.list file based on bandwidth tests 
    [universe]
    3.1-14: amd64 i386 powerpc

Not included in newer Ubuntu due to secturity issues it seems: see: Bug report

But .. I normally just use ping to find out the speed of a connection to some location. Amount of hops and latency.

8
  • 5
    netselect-apt doesn't seem to be available in Ubuntu 12.04
    – offby1
    Nov 6 '13 at 23:22
  • correct: see here bugs.launchpad.net/ubuntu/+source/netselect/+bug/337377
    – Rinzwind
    Nov 7 '13 at 7:41
  • 13
    This is not the most upvoted, or the best answer any more, check next one
    – ntg
    Jun 10 '15 at 7:54
  • Which answer is the "next one" might have changed.
    – gmatht
    Jan 24 '17 at 4:46
  • apt-spy is gone at debian 9, but netselect-apt is OK
    – netawater
    Nov 21 '17 at 16:05
161

You don't have to do any searching anymore - as ajmitch has explained, you can use deb mirror to have the best mirror picked for you automatically.

apt-get now supports a 'mirror' method that will automatically select a good mirror based on your location. Putting:

deb mirror://mirrors.ubuntu.com/mirrors.txt precise main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt precise-updates main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt precise-backports main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt precise-security main restricted universe multiverse

on the top in your /etc/apt/sources.list file should be all that is needed to make it automatically pick a mirror for you based on your geographical location.

Lucid (10.04), Maverick (10.10), Natty (11.04), and Oneiric (11.10) users can replace precise with the appropriate name.

7
  • Great tip. Just note that after making the change you need to run sudo apt-get update before doing any apt-get install for it to use your closest mirror.
    – Simon East
    Jun 29 '13 at 17:03
  • 2
    Related: askubuntu.com/q/319433/11244
    – Till
    Jul 18 '13 at 15:45
  • 25
    Nice tip, but unhelpful in my case. It works on geolocation, giving me the local server, which is waaaayy slower where I am. The network temporal distance is the important factor here, not spatial distance.
    – jarondl
    Jul 31 '13 at 8:24
  • 1
    currently broken: bugs.launchpad.net/ubuntu/+source/apt/+bug/1613184
    – marathon
    Aug 31 '17 at 18:34
  • 1
    @marathon fixed at least on 18.04+ Mar 25 '19 at 19:06
54

Here's one way that will always work, using good old netselect and some grep magic:

##The terminal-addict's "find best server" hack!##

  • Download and dpkg -i netselect for your architecture from the Debian website. (it's about 125 KB, no dependencies)

  • Find the fastest Ubuntu mirrors from your location, either up-to-date or at most six hours behind with this (I'll explain it below, sorry it doesn't split up nicely in Markdown)

    sudo netselect -v -s10 -t20 `wget -q -O- https://launchpad.net/ubuntu/+archivemirrors | grep -P -B8 "statusUP|statusSIX" | grep -o -P "(f|ht)tp://[^\"]*"`
    
  • netselect:

    1. -v makes it a little verbose -- you want to see progress dots and messages telling you different mirrors mapping to the same IP were merged :)
    2. -sN controls how many mirrors you want at the end (e.g. top 10 mirrors)
    3. -tN is how long each mirror is speed-tested (default is 10; the higher the number, the longer it takes but the more reliable the results.)
  • This is the backquotes stuff (don't paste, just for explanation)

    wget -q -O- https://launchpad.net/ubuntu/+archivemirrors \
    | grep -P -B8 "status(UP|SIX)" \
    | grep -o -P "(f|ht)tp://[^\"]*"
    
  1. wget pulls the latest mirror status from https://launchpad.net/ubuntu/+archivemirrors.
  2. The first grep extracts mirrors that are up-to-date or six-hours behind, along with 8 lines of previous context which includes the actual ftp/http URLs
  3. The second grep extracts these ftp/http URLs
  • Here's a sample output from California, USA:
   60 ftp://mirrors.se.eu.kernel.org/ubuntu/
   70 http://ubuntu.alex-vichev.info/
   77 http://ftp.citylink.co.nz/ubuntu/
  279 http://ubuntu.mirrors.tds.net/pub/ubuntu/
  294 http://mirror.umd.edu/ubuntu/
  332 http://mirrors.rit.edu/ubuntu/
  364 ftp://pf.archive.ubuntu.com/ubuntu/
  378 http://mirror.csclub.uwaterloo.ca/ubuntu/
  399 ftp://ubuntu.mirror.frontiernet.net/ubuntu/
  455 http://ubuntu.mirror.root.lu/ubuntu/
  
  • The "ranks" are an arbitrary metric; lower is usually better.
  • If you're wondering why the kernel.org Sweden-EU mirror and an NZ mirror are in the top three from California, well, so am I ;-) The truth is that netselect doesn't always choose the most appropriate URL to display when multiple mirrors map to a single IP; number 3 is also known as nz.archive.ubuntu.com!
2
  • 9
    netselect picks mirrors that have low udp or icmp latency. It doesn't necessarily pick mirrors that can give more bandwidth.
    – Tobu
    Oct 13 '13 at 19:40
  • @pix I approved your edit, but it's not command substitution that results in newlines being replaced. It's the subsequent field splitting that removed the newlines. Command substitution only removes trailing newlines.
    – muru
    Feb 21 '17 at 9:23
33

Oneliner that selects the best mirror (by download speed) based on mirrors.ubuntu.com for your ip:

curl -s http://mirrors.ubuntu.com/mirrors.txt | xargs -n1 -I {} sh -c 'echo `curl -r 0-102400 -s -w %{speed_download} -o /dev/null {}/ls-lR.gz` {}' |sort -g -r |head -1| awk '{ print $2  }'
3
  • 3
    To have more options replace at the end: sort -gr | head -3. Mar 4 '17 at 1:55
  • I found the curl part of this answer helpful because curl -r 0-102400 -o /dev/null [server_url]/ls-lR.gz where [server_url] is the base mirror URL listed in mirrors.txt, allows a speed comparison of the first ~100K of the index file from the mirror.
    – jamesc
    Mar 12 '18 at 12:57
  • This tests transfer speed, which is definitely an improvement over netselect. It's only downloading 10k from each server, which may not be a great representation of steady transfer speed on faster connections, though. Increase that 102400 to test with a larger download Jun 8 '18 at 21:41
22

Here is a Python script I wrote that finds mirrors with the lowest TCP latency.

The script also provides bandwidth and status data taken from launchpad, and will generate a new sources.list file automatically or using a mirror chosen from a list.

A usage example that lets you choose from 5 US mirrors with the lowest latency to your machine:

$ apt-select --country US -t 5 --choose
6
  • 5
    I want to let you know that I've made a debian package with your script that is ready to be used in a very easy and straightforward way: github.com/brodock/apt-select/releases/tag/0.1.0 Nov 3 '15 at 4:25
  • 1
    this is a perfect solution, as I've tried the other methods. to make noob friendly, I've written a post detailing this method:. blog.kmonsoor.com/…
    – kmonsoor
    Oct 11 '16 at 19:14
  • This is great, since netselect isn't available in newer versions of Ubuntu
    – Tek
    Feb 18 '18 at 7:23
  • Please show how to use it in your post
    – Jonathan
    Oct 31 '18 at 6:21
  • 1
    @Jonathan done. Full usage is in the README at the first link.
    – John B
    Nov 1 '18 at 0:07
5

I developed a simple ping-based nodejs script that tests the servers listed on mirrors.ubuntu.com/mirrors.txt and returns the fastest one:

sudo npm install -g ffum
ffum

Please let me know if you find it useful or have any suggestions (=

4
  • ffum does not work: Connection error.
    – James Fu
    Jul 10 '13 at 8:48
  • It doesn't work: Empty output. Aug 27 '13 at 1:06
  • git clone the repo and run node ffum
    – Michael
    Aug 7 '14 at 3:58
  • Awesome, works for me! I had a bug where it was looking for node instead of nodejs... also would be cool to have some verbose of each tested archive speed.
    – tweak2
    Aug 27 '14 at 16:57
5

I know this doesn't directly answer the OP's question, but there's a button in the desktop/GUI version of Ubuntu that finds the best mirror for you. It seemed to work pretty well, so I looked into it briefly, but didn't have time to follow up.

The reason I bring it up is because I think it would be pretty straight forward and usable to make it into a command line utility.

If anyone is interested, the test seems to be located in:

/usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py

Again, that's about as far as I got, but I figured I'd leave this here in case anyone wanted it. I'll probably pick back up on it when I have a little more time.

2
  • On 18.04, this script detects when it is invoked as an application (as main) from a terminal....and just prints its results to the terminal. Make sure to give it enough time to complete. $ python3 /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py >> [top 5 omitted] and the winner is: ny-mirrors.evowise.com
    – PatKilg
    Jun 9 '18 at 18:07
  • Sadly it doesn't work correctly. This script takes the 5 servers with lowest ping, then score them by bandwidth by downloading a ~1M file (Packages.gz in main from your dist). If you modify the script to increase it to 25 servers and download a 500M file you get completely different results, which are correct this time.
    – Jocelyn
    Feb 16 '19 at 11:20
5

For the command line, you can use a Python tool called apt-smart

A usage example that lets you list ranked mirrors within your country (automatically detect):

$ apt-smart -l

With -l, or --list-mirrors, you will get ( example output from Travis CI U.S. server ):

---------------------------------------------------------------------------------------------------------
| Rank | Mirror URL                            | Available? | Updating? | Last updated    | Bandwidth   |
---------------------------------------------------------------------------------------------------------
|    1 | http://mirrors.gigenet.com/ubuntua... | Yes        | No        | Up to date      | 1.73 MB/s   |
|    2 | http://mirror.genesisadaptive.com/... | Yes        | No        | Up to date      | 1.68 MB/s   |
|    3 | http://ubuntu.mirrors.tds.net/pub/... | Yes        | No        | Up to date      | 1.4 MB/s    |
|    4 | http://repos.forethought.net/ubuntu   | Yes        | No        | Up to date      | 1.35 MB/s   |
|    5 | http://repo.miserver.it.umich.edu/... | Yes        | No        | Up to date      | 937.62 KB/s |
...
|   75 | http://mirror.cc.vt.edu/pub2/ubuntu   | Yes        | No        | 1 day behind    | 659.67 KB/s |
|   76 | http://mirror.atlantic.net/ubuntu     | Yes        | No        | 2 days behind   | 351.26 KB/s |
|   77 | http://mirror.lstn.net/ubuntu         | Yes        | No        | 4 days behind   | 806.81 KB/s |
|   78 | http://mirrors.usinternet.com/ubun... | Yes        | No        | 4 weeks behind  | 514.31 KB/s |
|   79 | http://mirrors.arpnetworks.com/Ubuntu | Yes        | No        | 19 weeks behind | 418.94 KB/s |
|   80 | http://mirrors.ocf.berkeley.edu/ub... | Yes        | Yes       | Up to date      | 446.07 KB/s |
---------------------------------------------------------------------------------------------------------
Full URLs which are too long to be shown in above table:
1: http://mirrors.gigenet.com/ubuntuarchive
2: http://mirror.genesisadaptive.com/ubuntu
3: http://ubuntu.mirrors.tds.net/pub/ubuntu
5: http://repo.miserver.it.umich.edu/ubuntu
...
78: http://mirrors.usinternet.com/ubuntu/archive
80: http://mirrors.ocf.berkeley.edu/ubuntu

Of course, apt-smart can also change your sources.list if you want to:

$ apt-smart -a

With -a , or --auto-change-mirror to discover available mirrors, rank the mirrors by connection speed and update status and update /etc/apt/sources.list to use the best available mirror.

With -c , or --change-mirror MIRROR_URL to update /etc/apt/sources.list to use the given MIRROR_URL.

Compared with other tools:

  • apt-smart automatically finds where you are so you don't need to specify the country when you travel abroad.
  • apt-smart does real HTTP download from each mirror to get more accurate results ( bandwidth & status ) and supports HTTP proxy, rather than using ping and relying on launchpad 's inaccurate data.
  • apt-smart is being maintained, whereas most other tools leave issues unfix for a long time.

You can easily install apt-smart via pip, for detailed copy'n'paste install commands and usages please see Project Readme.

2
  • This works great! "pip install apt-smart" to install it. Nov 1 '19 at 1:57
  • 1
    @Andy Fraley Thank you for commenting. If you are lucky enough, you can install apt-smart simply by pip install apt-smart and run apt-smart without any errors. But sometimes in some environments it might says 'apt-smart' command not found, or any other errors. It is not a bug of apt-smart but it is something of pip or Ubuntu system environment, and talking about it will be a long story. So the recommend way to install it is to follow the install commands in Project Readme, which is clear and can be copy'n'paste as a whole into terminal.
    – Martin X
    Nov 3 '19 at 4:03
3

Command That Finds Fast Mirrors

On Ubuntu 18.04 I got good results by running

 python /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py

That prints a list of mirrors organized by "time" (not explained), and then I used one of the mirrors it ranked highest.

More Details

For me, it was useful to test a few of the top results output by that command by setting them as my mirror in /etc/apt/sources.list and then doing

time sudo apt update

to see how long it took to download the package list from that mirror. I tested the top three suggestions and they were all fast, but one of them was twice as fast as the other two in the time sudo apt update test.

Here's an example output from python /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py:

mirror: es-mirrors.evowise.com - time: 0.183778047562
mirror: it-mirrors.evowise.com - time: 0.18604683876
mirror: la-mirrors.evowise.com - time: 0.192630052567
mirror: ny-mirrors.evowise.com - time: 0.208723068237
mirror: mirrors.accretive-networks.net - time: 0.385910987854
mirror: mirror.team-cymru.org - time: 0.46785402298
mirror: mirrors.psu.ac.th - time: 1.64231991768
and the winner is: es-mirrors.evowise.com
2
  • 2
    Sadly it doesn't work correctly. This script takes the 5 servers with lowest ping, then score them by bandwidth by downloading a ~1M file (Packages.gz in main from your dist). If you modify the script to increase it to 25 servers and download a 500M file you get completely different results, which are correct this time.
    – Jocelyn
    Feb 16 '19 at 11:14
  • @Jocelyn indeed! I've made an easy to use patch with some improvements: gist.github.com/shvchk/b81e4579bff13486871f99404c497759
    – Shevchuk
    Nov 26 '20 at 19:38
0

The easiest and efficient way to get the fastest mirror is to use the apt mirror:// source, see

https://mvogt.wordpress.com/2011/03/21/the-apt-mirror-method/

0

I use the following to auto select mirrors (and disable deb-src)

sudo sed -i -e 's%http://archive.ubuntu.com/ubuntu%mirror://mirrors.ubuntu.com/mirrors.txt%' -e 's/^deb-src/#deb-src/' /etc/apt/sources.list
0

If you want a utility to do this you could implement such a utility as a simple bash script like the following. This might be useful if you want to use the utility without needing pip/nodejs.

#!/bin/bash
if [ -z "$1" ]
then
    echo Usage:  sudo $0 http://mirrors.ubuntu.com/mirrors.txt
    echo OR consider one of...
    for mirror in `wget http://mirrors.ubuntu.com/mirrors.txt -O - 2> /dev/null`
    do 
        (
            host=`echo $mirror |sed s,.*//,,|sed s,/.*,,`
            echo -e `ping $host -c1 | grep time=|sed s,.*time=,,`:'  \t\t'$mirror
        ) &
        done
    wait
    exit 1
fi

OLD_SOURCE=`cat /etc/apt/sources.list | grep ^deb\ | head -n1 | cut -d\  -f2`

[ -e  /etc/apt/sources.list.orig ] || cp /etc/apt/sources.list /etc/apt/sources.list.orig

cp /etc/apt/sources.list /etc/apt/sources.list.tmp
sed "s,$OLD_SOURCE,$1," < /etc/apt/sources.list.tmp > /etc/apt/sources.list
0

The other answers, including the accepted answer, are no longer valid (for Ubuntu 11.04 and newer) because they recommended Debian packages such as netselect-apt and apt-spy which do not work with Ubuntu.

There are two different working answers to this question below:

  1. Use apt-get's mirror: method

    This method asks the Ubuntu server for a list of mirrors near you based on your IP, and selects one of them. The easiest alternative, with the minor downside that sometimes the closest mirror may not be the fastest.

  2. Command-line foo using netselect
    Shows you how to use the netselect tool to find the fastest recently updated servers from you -- network-wise, not geographically. Use sed to replace mirrors in sources.list.

Use sed to replace mirrors in sources.list

Since some sources use addition folders as part of their path it might be better to use the alternate separator syntax.

sudo sed -i 's%us.archive.ubuntu.com%mirrors.gigenet.com/ubuntuarchive/%' /etc/apt/sources.list
0

Default Software properties GTK and Qt apps have 'Select best server' utility which uses MirrorTest.py from python3-software-properties. It can also be used from CLI.

  • Install mirrors test utility:
    sudo apt install python3-software-properties --no-install-recommends

  • Use it:
    LC_ALL=C python3 /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py

However, it tests just 5 servers with just 100KB file download, so results are not too accurate.
For far better results, I've made a patch to use 20 servers, 30MB file and more informative output:

  • Download and apply patch:
    wget -qO- https://gist.githubusercontent.com/shvchk/b81e4579bff13486871f99404c497759/raw/MirrorTest.py.patch | sudo patch -d/ -p0

  • Use it as usual:
    LC_ALL=C python3 /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py

Sample output:

> LC_ALL=C python3 /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py
Running ping tests
Running download tests
Testing mirror.wff-gaming.de
Testing ubuntu.melbourneitmirror.net
Testing linux.xjtuns.cn
Testing mirrors.avalonhosting.services
Testing mirrors.codec-cluster.org
Testing ftp.utexas.edu
Testing mirror.plustech.de
Testing mirror.ratiokontakt.de
Testing mirror2.tuxinator.org
Testing mirror.kumi.systems
Testing ubuntu.cybertips.info
Testing files.tux-users.net
Testing linux.darkpenguin.net
Testing mirrors.aliyun.com
Testing mirror.eu-fr.kamatera.com
Testing ny-mirrors.evowise.com
Testing la-mirrors.evowise.com
Testing mirror.de.leaseweb.net
Testing es-mirrors.evowise.com
Testing uk-mirrors.evowise.com
Testing mirrors.ukfast.co.uk
Testing mirrors.advancedhosters.com
mirror: uk-mirrors.evowise.com - time: 0.48804259300231934
mirror: es-mirrors.evowise.com - time: 0.5114705562591553
mirror: mirror.eu-fr.kamatera.com - time: 0.5532209873199463
mirror: ny-mirrors.evowise.com - time: 0.567798376083374
mirror: linux.darkpenguin.net - time: 0.6391427516937256
mirror: mirror.de.leaseweb.net - time: 0.7773537635803223
mirror: files.tux-users.net - time: 0.957298755645752
mirror: mirror2.tuxinator.org - time: 0.9695150852203369
mirror: la-mirrors.evowise.com - time: 1.1493926048278809
mirror: ubuntu.cybertips.info - time: 1.1670680046081543
mirror: mirrors.aliyun.com - time: 1.2511329650878906
mirror: mirror.ratiokontakt.de - time: 1.2832701206207275
mirror: mirrors.advancedhosters.com - time: 1.8812928199768066
mirror: mirrors.ukfast.co.uk - time: 2.537770986557007
mirror: mirror.plustech.de - time: 11.090754270553589
and the winner is: uk-mirrors.evowise.com

Not the answer you're looking for? Browse other questions tagged or ask your own question.