How to automate the process of checking an Ubuntu ISO SHA256sum

Question

I know that I can get the SHA256 sums (and MD5 sums) from http://releases.ubuntu.com/ as they are provided now within the folders, but is there any way of automating the checking process so that the download of the SHA sum is automatic and so is the checking part? I know that you can 'semi-automate' the process with MD5sums, but I don't know how to fully automate it, nor how to even get semi-automation with SHA256.

Something like a script that you can input the release and flavour into (including unsupported releases) and it automatically downloads the SHA256 sum and checks would be good.

Answer 1

This will automatically fetch the correct "SHA256SUM" file for the specified flavour / release and check it against the images found in the specified directory (or in the current working directory if no directory is specified);

The input flavour must be either "unity" or one of the flavours listed here; at the time of this answer, those are:

• edubuntu
• kubuntu
• lubuntu
• mythubuntu
• ubuntu-gnome
• ubuntukylin
• ubuntu-mate
• ubuntustudio
• xubuntu

The input release must be a valid release / point release for the specified flavour.

It uses zenity to provide a nice GUI to get the inputs; it will print only the matching results (if any) and will let the user choose if they want to keep the downloaded "SHA256SUM" file;

#!/bin/bash
if [ ! -z "$1" ]; then
  if [ -d "$1" ]; then
    printf "Changing PWD to $1...\n"
    cd "$1"
  else
    printf "$1 is not a valid directory.\n"
    exit 1
  fi
fi
flavour=$(zenity \
  --entry \
  --title "Insert the flavour to check the image against" \
  --text "Insert the flavour to check the image against:" \
  2>/dev/null | \
tr '[:upper:]' '[:lower:]')
printf "Flavour:\t$flavour\n"
version=$(zenity \
  --entry \
  --title "Insert the version to check the image against" \
  --text "Insert the version to check the image against:" \
  2>/dev/null | \
tr '[:upper:]' '[:lower:]')
printf "Version:\t$version\n"
[ "$flavour" = "unity" ] \
&& address="http://releases.ubuntu.com/$version/SHA256SUMS" \
|| address="http://cdimage.ubuntu.com/$flavour/releases/$version/release/SHA256SUMS"
printf "Downloading $address...\n"
wget -q -O SHA256SUMS "$address"
[ $? -ne 0 ] && printf "No SHA256SUMS file found on the server.\n" && rm SHA256SUMS && exit 1
printf "Checking SHA256SUMS...\n"
shasum -a 256 -c SHA256SUMS |& grep 'OK$'
[ $? -eq 0 ] || printf "No matching image found on the target directory.\n"
zenity --question --title "Remove SHA256SUMS?" --text "Remove SHA256SUMS?" 2>/dev/null
[ $? -eq 0 ] && rm SHA256SUMS
exit 0

Example on an Ubuntu Desktop 15.04 64-bit image:

Answer 2

The official page containing MD5 hashes for Ubuntu, Kubuntu, Edubuntu Xubuntu and Lubuntu is:

https://help.ubuntu.com/community/UbuntuHashes

Choose the relevant distribution and click on the MD5SUMS file.

Now check your download ISO:

md5sum ubuntu-*.iso

Compare the hash (the alphanumeric string on left) that your machine calculated with the corresponding hash on the UbuntuHashes page.

Semi-automatic method

Ubuntu distributes the MD5 hashes in a file called MD5SUMS near the bottom of the download page for your release http://releases.ubuntu.com.

First download the MD5SUMS file to the same directory as the iso. Then run the following in a terminal.

cd download_directory

md5sum -c MD5SUMS

md5sum will generate a bunch of warnings. Don't worry: the OK message will be buried somewhere within it!

In this case the message you want is on the seventh line.

ubuntu-*.iso: OK

Answer 3

For this specific task , I've written my own Python script. This is aimed at iso images that can be downloaded with iso images provided at http://cdimage.ubuntu.com, but the content of that site is exactly the same as what you use http://releases.ubuntu.com/, so it will work for both. Both sites have SHA256SUMS file,which we'll be checking. The basic premise of this script is to

1. download the iso
2. calculate the iso's sha256sum
3. download the SHA256SUM file from the same page
4. Compare what we calculated with what's in the file

Script Source code

This script is also available on my personal GitHub repository, which may have never features.

#!/usr/bin/env python3
# Script for automatically downloading and verifying sha256 hashsum
# of iso images provided by http://cdimage.ubuntu.com
import urllib.request
import sys
import os
from hashlib import sha256

def download_file(url):
    print(">>> Retrieving ",url)
    save_as = url.split('/')[-1]
    buffer_size=512
    try:
        with urllib.request.urlopen(url) as response, open(save_as,'wb') as out_file:
            print(response.info())
            print(">>> Writing data:")
            has_data=True
            retrieved = 0
            while has_data:
                 data = response.read(buffer_size)
                 retrieved += len(data)
                 # simple progress message which overwrites itself
                 message = "Retrieved "+str(retrieved)+" bytes"
                 print("\r"+" "*len(message)+"\r",end="")
                 print(message,end="")
                 sys.stdout.flush()
                 if data:
                     out_file.write(data)
                 else:
                    has_data=False
    except Exception as e:
        sys.stderr.write('\n>>> Something went wrong\n')
        sys.stderr.write(str(e))
    else:
        print('\n>>> URL retrieved successfully')
        return(save_as)

def get_sha256sum(file_path):
    sha256sum = sha256()
    with open(file_path, 'rb') as fd:
        data_chunk = fd.read(1024)
        while data_chunk:
              sha256sum.update(data_chunk)
              data_chunk = fd.read(1024)
    return str(sha256sum.hexdigest())

def compare_sha256sums(local_file,sha256sum,hashsum_file):
     remote_hashsum = ""
     with open(hashsum_file) as fd:
         for line in fd:
              words = line.strip().split()
              if words[1].replace('*','') == local_file:
                  remote_hashsum = words[0]
         if not remote_hashsum: 
              sys.stderr.write("\n>>> Error: local file not found in list of SHA256SUMS\n")
              sys.exit(1)
     if remote_hashsum == sha256sum:
         print("Local file ",local_file," with sha256 hashsum ",sha256sum,"matches with sha256sum in remote. All OK.")


def main():
    saved_filename = download_file(sys.argv[1])
    sha256sum = get_sha256sum(saved_filename)
    sha256sums_file_url = "/".join( sys.argv[1].split('/')[:-1] + ['SHA256SUMS'] ) 
    sha256sum_file = download_file( sha256sums_file_url  ) 
    compare_sha256sums(saved_filename,sha256sum,sha256sum_file)

if __name__ == '__main__': main()

Test Run:

bash-4.3$ ./get_iso_and_verify.py  http://cdimage.ubuntu.com/releases/16.04.2/release/ubuntu-16.04.2-preinstalled-server-armhf+raspi2.img.xz
>>> Retrieving  http://cdimage.ubuntu.com/releases/16.04.2/release/ubuntu-16.04.2-preinstalled-server-armhf+raspi2.img.xz
Date: Fri, 07 Jul 2017 21:55:20 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Thu, 16 Feb 2017 20:16:12 GMT
ETag: "ee62708-548ab77ea3b00"
Accept-Ranges: bytes
Content-Length: 249964296
Connection: close
Content-Type: application/x-xz


>>> Writing data:
Retrieved 249964296 bytes
>>> URL retrieved successfully
>>> Retrieving  http://cdimage.ubuntu.com/releases/16.04.2/release/SHA256SUMS
Date: Fri, 07 Jul 2017 22:09:47 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Fri, 17 Feb 2017 00:06:46 GMT
ETag: "205-548aeb07c5180"
Accept-Ranges: bytes
Content-Length: 517
Connection: close


>>> Writing data:
Retrieved 517 bytes
>>> URL retrieved successfully
Local file  ubuntu-16.04.2-preinstalled-server-armhf+raspi2.img.xz  with sha256 hashsum  60156f9238360dc84267dbde4f334516d580fe540dd523d12d4837c4647d6d8f matches with sha256sum in remote. All OK.
bash-4.3$ cat SHA256SUMS 
60156f9238360dc84267dbde4f334516d580fe540dd523d12d4837c4647d6d8f *ubuntu-16.04.2-preinstalled-server-armhf+raspi2.img.xz
35c9a6b7536e41c19f18033ac5a9b095130d17848126160d6b66cbd09be48f17 *ubuntu-16.04.2-server-arm64.iso
a00d88107eebadf0dde86087ad746d372d33ebdd29ac5cd4fae42a2e031d2b8f *ubuntu-16.04.2-server-powerpc.iso
0a10bada74112c58412ac8778df05abbb69d5983b672e6bbe74fa794cf002a2a *ubuntu-16.04.2-server-ppc64el.iso
253fd0eb5e529c3434729f475c7855463ba87ed7dea4321182b54c5416523897 *ubuntu-16.04.2-server-s390x.iso