Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I am trying to download the 64-bit Ubuntu 12.10 ISO over an extremely unreliable connection. By unreliable, I mean that every time I finish downloading the ISO and compare the MD5 hashes, I end up with an error about a mismatch.

I am using Ubuntu 12.04 to perform the download. Is there a tool that can somehow work around a very unreliable connection?

share|improve this question
    
How about just buying it on CD? If you only have such an unreliable connection, that might be the easier solution, and it's still almost gratis. –  leftaroundabout Mar 29 '13 at 11:13
    
In case you do want to buy it from Canonical as @leftaroundabout suggests, here's a link for 12.10. –  Eliah Kagan Mar 29 '13 at 11:41

3 Answers 3

up vote 40 down vote accepted

In my case my connection drops a lot and if trying to download the ISO directly I also end with a MD5 problem. What I recommend is downloading the ISO image through a torrent client.

That way, even while downloading, the torrent client checks to make sure it is OK. There is a good list of torrents to use but if you are in Windows or Mac, I suggest UTorrent. If you are doing this from an Ubuntu installed computer, Ubuntu comes with a default torrent client called Transmission Install transmission.

You can find the torrent download file in the ubuntu site just look for the file with the .torrent file extension.

share|improve this answer

If you've already downloaded it N times, and have not overwritten the copies, you may be able to combine them to make a more reliable resulting image. Whether this is possible depends on some assumptions, but it's worth a shot.

For each byte in the image, you can take N bytes from the copies. Compare them, and take the majority vote. For instance if you have three copies, and two of them say that a byte is 0xFF, and ones says 0xFE, it is likely that the odd one out is erroneous.

This likelihood is very good if the error rate is very low.

The program to do this should be trivial to write, even in C. fopen all the files in "b" mode, then in a loop getc one byte from each one and pick the most popular byte value out of those to write to the output.

Here is the source code for such a utility. [Update: now works with bits rather than bytes by choosing the most popular bit value for each corresponding bit position.] I have tested the program by making copies of a binary file from /usr/bin, and introducing errors using a hex editor. The tie case was tested by using three files with each having a different byte in the same position, and the insufficient redundancy case was tested first by making two input files longer than the others, and then one file longer than the others, and also by running the program with an insufficient number of input files. It could use better comments, usage text and documentation. Also, one issue is that when the read on an input file returns EOF, it is assumed that the files have ended. The program doesn't check for an I/O error condition but just silently stops using that file.

/*
 * A utility to recover a file from three or more corrupt copies,
 * which are either truncated or contain random bit errors (but not
 * deletions of bits).
 *
 * Kaz Kylheku <kaz@kylheku.com>
 * Copyright Info: This program is in the public domain.
 *
 * March 29, 2013: Initial version: most popular byte.
 * April 2, 2013: Revised: most popular bit.
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

int main(int argc, char **argv)
{
  int i, nfiles = argc - 1; /* includes output file */
  FILE **fp;
  const char *progname = argv[0];
  int tie_occurred = 0;
  int insufficient_redundancy = 0;

  if (argc < 3) {
    if (progname != 0)
      printf("usage: %s <output-file> { <input-file> }+\n", progname);
    return EXIT_FAILURE;
  }

  argv++, argc--;

  if ((fp = malloc(nfiles * sizeof *fp)) == 0) {
    printf("%s: out of memory\n", progname);
    return EXIT_FAILURE;
  }

  for (i = 0; i < nfiles; i++) {
    if ((fp[i] = fopen(argv[i], (i == 0) ? "wb" : "rb")) == 0) {
      printf("%s: unable to open %s (%d/%s)\n", progname,
             argv[i], errno, strerror(errno));
      return EXIT_FAILURE;
    }
  }

  for (;;) {
    int alldone = 1, nread = 0;
    int bitfreq[8] = { 0 };
    int byte;

    for (i = 1; i < nfiles; i++) {
      if (fp[i] != 0) {
        byte = getc(fp[i]);

        if (byte == EOF) {
          fclose(fp[i]);
          fp[i] = 0;
          continue;
        }

        alldone = 0;
        (byte & 0x80) ? bitfreq[0]++ : bitfreq[0]--;
        (byte & 0x40) ? bitfreq[1]++ : bitfreq[1]--;
        (byte & 0x20) ? bitfreq[2]++ : bitfreq[2]--;
        (byte & 0x10) ? bitfreq[3]++ : bitfreq[3]--;
        (byte & 0x08) ? bitfreq[4]++ : bitfreq[4]--;
        (byte & 0x04) ? bitfreq[5]++ : bitfreq[5]--;
        (byte & 0x02) ? bitfreq[6]++ : bitfreq[6]--;
        (byte & 0x01) ? bitfreq[7]++ : bitfreq[7]--;
        nread++;
      }
    }

    if (alldone)
      break;

    for (byte = 0, i = 0; i < 8; i++) {
      byte <<= 1;
      if (bitfreq[i] > 0)
        byte |= 1;
      else if (bitfreq[i] == 0)
        tie_occurred = 1;
    }

    if (nread < 3)
      insufficient_redundancy = 1;

    if (putc(byte, fp[0]) == EOF) {
      printf("%s: error writing to %s (%d/%s)\n", progname,
             argv[0], errno, strerror(errno));
      return EXIT_FAILURE;
    }
  }

  if (tie_occurred)
    printf("%s: problem: a tie occurred for some bits\n", progname);

  if (insufficient_redundancy)
    printf("%s: warning: some bits determined from fewer than 3 files\n", progname);

  fclose(fp[0]);
  return tie_occurred ? EXIT_FAILURE : 0;
}
share|improve this answer
1  
Even assuming he kept each copy, what advantage would this have over fixing any one copy with zsync? One problem with this method is that multiple copies of the file might have been incompletely downloaded. Cutting off at different places will result in a large number of corresponding bytes being missing. So all the images might be bad, and if not, the correct one might be "fixed" to be more like the majority (i.e., to be missing the end). Also, as this is trivial to write, maybe you could include the complete source code of such a program... –  Eliah Kagan Mar 29 '13 at 11:36
    
``zsync only works for files whose providers have created a .zsync metafile.'' Mine is a general approach that will fix rare bit errors that can sneak past TCP/IP checksums. –  Kaz Mar 29 '13 at 15:48
2  
But it won't actually fix them, much of the time. Suppose I have six copies of a file. One of them is 100% complete and has a corrupted byte 3% from the end. Another is 100% complete and has a corrupted byte 2% from the end. The other four are incomplete (but possibly correct in the data that was downloaded before the download was interrupted)--they are 92%, 93%, 94%, and 96% complete. The consequence of the method you propose will be to produce a wrong, truncated file. And incomplete downloads aren't the only cause of repeated corruption. The file may be cached corrupt by a proxy. –  Eliah Kagan Mar 29 '13 at 16:29
    
You may be able to partially mitigate some of these issues in the implementation of the program you've suggested, but you won't be able to get around all of them completely, and your program will have to be quite sophisticated even to produce good results most of the time. Plus, like most programs dealing with binary files, novice coders wouldn't get it right the 1st time, due to issues like misinterpreing EOF characters in the file as indicating that the file has ended. For these reasons, if you really believe your method is workable, I strongly encourage you to write and post code. –  Eliah Kagan Mar 29 '13 at 16:33
7  
"And I have more years of networking and software development experience than ... your age. LOL." And yet you relied on my help to revise the specification of your "trivial" program which you are unable or unwilling to write. Please do not use ad hominem attacks or ageist bigotry to compensate for your inability to defend your position. It's hateful, inappropriate, and it doesn't make you or your claims look any good. –  Eliah Kagan Mar 29 '13 at 16:41

Besides bittorrent, you can also use zsync (a free tool) for this.

You'll probably want to know how zsync works in order to decide whether or not you want to obtain and use it, so I'll explain how it works first, and then how to obtain it.

How to Use zsync to Download an Ubuntu ISO Image

zsync lets you build one file by starting with another. You can start with any file, but to offer any advantage over simply downloading the file you need, the file you start with should be similar to the file you want.

zsync only works for files whose providers have created a .zsync metafile. A .zsync file contains checksums for sections of a file, and enables the zsync utility to figure out what parts of the file you already have match, and what parts do not. Parts that do not match are downloaded again. (Actually, zsync is a bit more sophisticated--it is also able to handle situations where some parts match but are in different places in the file, without having to redownload those matching parts. That particular feature is not frequently helpful in creating a good file from a corrupt download, however.)

Once you have a (corrupted) ISO image downloaded somewhere, open a Terminal or (for Windows) Command Prompt (cmd.exe) and navigate to its location. In most OSes you can type cd with a space at the end and then drag a folder to the command line, and the location of the folder will be pasted. Then you can press enter and it will Change Directory to that location. (cd should be lower case--on some systems, like Windows, upper-case will work, but on other systems, like OS X and Ubuntu, it will not.)

Once you're in the folder that has the file--running ls or dir should show you everything in there, and running ls *.iso or dir *.iso should show you just files that end in .iso, which includes Ubuntu installation images--then run:

zsync URL-to-.zsync-file Ubuntu-iso-image-name.iso 

That will patch up the corrupted file you have (named like Ubuntu-iso-image-name.iso--for example, it may be called ubuntu-12.10-desktop-i386.iso) to be correct. If you run it and no changes are made, that means your file is correct. zsync incorporates checksums, so you can be sure you have a complete, correct file if it says you do.

To learn precisely how to obtain .zsync on multiple platforms, and how to run it for each Ubuntu ISO image, read on.

Getting the zsync Utility

I've written about getting and using zsync to fix corrupted ISO images before, in a different context. It was in a longer answer most of which is not related to this question here. So I've excerpted the parts of it that are related.

Getting zsync for Windows

zsync traditionally runs on Unix-like system (which doesn't really include Windows). However, recently a native Windows port of zsync has been created, so you can probably use that.

zsync is a command-line program, so in Windows you run it in the Command Prompt. (One way to open the command-prompt is to type cmd.exe into the text box in the Start menu and press Enter. Or, in Windows XP and earlier, Start > Run > cmd.exe.)

Another Way: Getting zsync for Cygwin

If you have any problems with the Windows port of zsync, you can always use the Cygwin version. Cygwin provides a Unix-like environment for Windows. It does not allow compiled executables to be run (for example, you could not use zsync from the Ubuntu package), but uses special versions of programs compiled for it from source. (It is far easier to compile a program written for Unix-like operating systems in Cygwin, than to compile it so it works natively on Windows.)

  1. First, get Cygwin's installer (setup.exe) from here (or use this direct link).
  2. Run it, click Next on the first screen, then select Install from Internet and click Next again.
  3. On the next two screens, put Cygwin and the local package directory wherever you like them (defaults are usually good).
  4. Select your Internet connection type (Direct Connection is usually good), and setup.exe will download a list of mirrors.
  5. Pick any mirror, and it will download a list of available packages. You'll probably be warned about how this version is different from the previous one. That's not a problem, unless you've been using a previous version of Cygwin on the same machine.

    Setup Alert message about how how this is the first time you've installed this particular version of Cygwin. If you haven't used Cygwin before, this can be ignored. As it says, "If this is the first time you've installed Cygwin on this system then you can ignore this message."

  6. Now you can select what packages to install. There is a text box for searching packages by name. Type zsync there (but do not press Enter--if you do, it's like clicking Next). An entry for Net will appear. Expand it by clicking the + to the left of it and you'll see Skip appear under it. That means if you don't change anything, setup.exe will not install zsync.

    Velma Dinkley installs zsync in Cygwin while updating her blog in "Siren's Song."

  7. So click once, where it says Skip. Now it will install the latest available stable version of zsync for Cygwin.

    enter image description here

  8. Click Next again to finish the installation. The Cygwin environment, and zsync, will be downloaded and installed. (If you're asked to install packages that are required as dependencies, go ahead and do so.)

Once it's installed, open the Cygwin shell. You can run zsync from there.

References

Please note that while you can post on the Cygwin mailing list for problems with Cygwin (after reading that and the materials it tells you to read), they understandably do not support unofficial instructions. cygwin has quite good documentation, so like other projects with good centralized documentation (like OpenBSD), they expect that people asking for help will have read the documentation and are working from (or at least very familiar with) it.

(Therefore, if you have trouble with my instructions, the best place to ask about them is probably here.)

zsync on Ubuntu

This doesn't apply in your specific case, but for folks who want to use zsync on Ubuntu, just install zsync Install zsync. This can be done in the Software Center or by running this in a Terminal window (Ctrl+Alt+T):

sudo apt-get update && sudo apt-get install zsync

Now you can run it from the Terminal.

From my answer to How to identify and replace broken files in a corrupted Ubuntu ISO?

Using .zsync to Fix the Most Common Ubuntu ISO Files

First, cd to the location containing the corrupted .iso image, as explained above (in How to Use zsync to Download an Ubuntu ISO Image).

Now run zsync in one of these ways, depending on which Ubuntu ISO image it is:

  • 12.10, 32-bit:

    zsync http://releases.ubuntu.com/quantal/ubuntu-12.10-desktop-i386.iso.zsync
    
  • 12.10, 64-bit:

    zsync http://releases.ubuntu.com/quantal/ubuntu-12.10-desktop-amd64.iso.zsync
    
  • 12.04.1 LTS, 32-bit:

    zsync http://releases.ubuntu.com/12.04.1/ubuntu-12.04.1-desktop-i386.iso.zsync
    
  • 12.04.1 LTS, 64-bit:

    zsync http://releases.ubuntu.com/12.04.1/ubuntu-12.04.1-desktop-amd64.iso.zsync
    
  • 12.04 LTS (before the 12.04.1 point release), 32-bit:

    zsync http://old-releases.ubuntu.com/releases/precise/ubuntu-12.04-desktop-i386.iso.zsync
    
  • 12.04 LTS (before the 12.04.1 point release), 64-bit:

    zsync http://old-releases.ubuntu.com/releases/precise/ubuntu-12.04-desktop-amd64.iso.zsync
    

Once you run one of those commands, zsync will automatically patch your ISO up to the correct version, and you're good to go.

From my answer to How to identify and replace broken files in a corrupted Ubuntu ISO?

Writing zsync Commands for Other (and Future) Ubuntu .iso Images

The general syntax is that you run zsync followed by a complete URL for the .zsync file on the server that has the same name as the file you have a version of and wish to download (except with .zsync at the end of course).

In case anyone wants to do this for any other Ubuntu releases (or other ISO files for the same release, like the version optimized for booting properly on Macs or the version for ARM-based mobile devices) here's the rule for where the .zsync files are located:

  • ISO images containing currently supported installers, including ISO's for the latest point release of LTS releases and all ISO's for currently supported LTS releases, are located in:

    http://releases.ubuntu.com/

  • ISO images for end-of-life releases (which you shouldn't use anyway) or previous point releases of supported LTS releases are located in:

    http://old-releases.ubuntu.com/releases/

Just click the link for your version and scroll down to see the full file list (or search the page for .zsync). Then copy the URL and paste it in the zsync command.

From my answer to How to identify and replace broken files in a corrupted Ubuntu ISO?

While it is considerably broader than this (and I do not recommend that they be considered duplicates!), if you're interested you might want to take a look at the question that originally led me to write most of the text here (i.e., the text enclosed in blockquotes):

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.