5

I'm preping a humanitarian mission in Africa where we'll leave behind two laptops in a place with low/costly connectivity to internet. I'd rather have then running Ubuntu than, say, 'another more shady option'.

Thus for updates as well setup adaptability (ie, with an application bank), I want to create a local mirror of the Xenial depots on a USB SSD hardrive for them to physically bring to a more connected/less costly outlet. I've been using apt-mirror under the instructions of this tutorial so far (beware, French ahead, but it is standart enough I suppose).

I'm running all the following on an Ubuntu 16.10 system.

My sources are as follow :

deb [arch=amd64] file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/ xenial multiverse universe main 
deb [arch=amd64] file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/ xenial-updates multiverse universe main
deb [arch=amd64] file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/ xenial-security multiverse universe main
deb [arch=amd64] file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/ xenial partner

I've had errors while executing "sudo apt-get update" upon an unfound Components-amd64.yml file, while there was a Components-amd64.yml.gz file with messed-up rights (-rw-----) belonging to root in the targeted repository.

** (appstreamcli:17139): WARNING **: No origin found for file _home_andre_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-security_restricted_dep11_Components-amd64.yml.gz

(appstreamcli:17139): GLib-GIO-CRITICAL **: g_converter_input_stream_new: assertion 'G_IS_INPUT_STREAM (base_stream)' failed

(appstreamcli:17139): GLib-GIO-CRITICAL **: g_output_stream_splice: assertion 'G_IS_INPUT_STREAM (source)' failed

** (appstreamcli:17139): WARNING **: No origin found for file _home_andre_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.gz

(appstreamcli:17139): GLib-GIO-CRITICAL **: g_converter_input_stream_new: assertion 'G_IS_INPUT_STREAM (base_stream)' failed

(appstreamcli:17139): GLib-GIO-CRITICAL **: g_output_stream_splice: assertion 'G_IS_INPUT_STREAM (source)' failed

** (appstreamcli:17139): WARNING **: No origin found for file _home_andre_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-security_universe_dep11_Components-amd64.yml.gz
AppStream cache update completed, but some metadata was ignored due to errors.
Lecture des listes de paquets... Fait
N: Can't drop privileges for downloading as file '/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission non accordée)
E: Failed to fetch file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/universe/dep11/Components-amd64.yml  Fichier non trouvé - /media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/universe/dep11/Components-amd64.yml (2: Aucun fichier ou dossier de ce type)
E: Failed to fetch file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/universe/dep11/Components-amd64.yml  Fichier non trouvé - /media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/universe/dep11/Components-amd64.yml (2: Aucun fichier ou dossier de ce type)
E: Failed to fetch file:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-security/universe/dep11/Components-amd64.yml  Fichier non trouvé - /media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-security/universe/dep11/Components-amd64.yml (2: Aucun fichier ou dossier de ce type)
E: Le téléchargement de quelques fichiers d'index a échoué, ils ont été ignorés, ou les anciens ont été utilisés à la place.

I found some reports related to rigths and privileges. I also ran into another one related to dep11 files and decided to act upon a comment

My conf apt-miror file thus became :

############# config ##################
#
set base_path    /media/andre/Depots/miroir 
#
set mirror_path  $base_path/mirror
set skel_path    $base_path/skel
set var_path     $base_path/var
set cleanscript $var_path/clean.sh
set defaultarch  deb-amd64
set postmirror_script $var_path/postmirror.sh
set run_postmirror 0
set nthreads     20
set _tilde 0
#
############# end config ##############
#
deb-amd64 http://archive.ubuntu.com/ubuntu xenial main restricted universe multiverse
deb-amd64 http://security.ubuntu.com/ubuntu xenial-security main restricted universe multiverse
#
## Major bug fix updates produced after the final release of the distribution.
#
deb-amd64 http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse
#
## Dépôt commercial  ##
#
deb-amd64 http://archive.canonical.com/ubuntu xenial partner
#
############# clean ##############
#
clean http://fr.archive.ubuntu.com/ubuntu/ xenial
clean http://security.ubuntu.com/ubuntu xenial-security
clean http://archive.canonical.com/ubuntu xenial
clean http://extras.ubuntu.com/ubuntu xenial

But now, there was more of the same with Translation-en files!

I decided to reset the whole mess and downloading the depot anew.

When launching apt-get update, there was an unfounded "Package" file as well as some "Components-amd64.yml" files.

drwxr-xr-x 6 root root 4096 aog 19 16:31 xenial
drwxr-xr-x 6 root root 4096 aog 19 16:31 xenial-updates
__________________________________________________________________________________________
E: Failed to fetch file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11/Components-amd64.yml  Fichier non trouvé - /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11/Components-amd64.yml (2: Aucun fichier ou dossier de ce type)
___________________________________________________________________________________________
administrateur@femmeTantsaha1:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11$ ls -l
total 2116
-rw-r--r-- 1 root root  732506 apr 21 13:08 Components-amd64.yml.gz
-rw-r--r-- 1 root root 1023905 apr 21 13:09 icons-128x128.tar.gz
-rw-r--r-- 1 root root  409405 apr 21 13:09 icons-64x64.tar.gz
___________________________________________________________________________________________
E: Failed to fetch file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11/Components-amd64.yml  Fichier non trouvé - /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11/Components-amd64.yml (2: Aucun fichier ou dossier de ce type)
____________________________________________________________________________________________
administrateur@femmeTantsaha1:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11$ ls -l
total 748
-rw-r--r-- 1 root root 213372 aog 18 20:17 Components-amd64.yml.gz
-rw-r--r-- 1 root root 388198 aog 18 20:18 icons-128x128.tar.gz
-rw-r--r-- 1 root root 159218 aog 18 20:18 icons-64x64.tar.gz
_____________________________________________________________________________________________
E: Le téléchargement de quelques fichiers d'index a échoué, ils ont été ignorés, ou les anciens ont été utilisés à la place.

I tried to render the local depot functional by completing it with the file "/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-security/main/binary-amd64/Packages". I downloaded and decompressed "Packages.xz" at //security.ubuntu.com/ubuntu/dists … ary-amd64/ and restored the missing repository linkings.

andre@andre-dotmu:/media/andre/Depots/miroir/mirror/security.ubuntu.com/ubuntu/dists/xenial-security/main/binary-amd64$ ls -l
total 1128
-rw-r--r-- 1 root root 846812 août  24 08:42 Packages
-rw-r--r-- 1 root root 165174 août  24 07:58 Packages.gz
-rw-r--r-- 1 root root 133896 août  24 07:58 Packages.xz
-rw-r--r-- 1 root root    105 août  24 07:58 Release

For the "Components-amd64.yml" files, I decompressed the "Components-amd64.yml.gz" files found in the specified repositories.

andre@andre-dotmu:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11$ ls -l
total 4256
-rw------- 1 _apt root 2188952 avril 21 12:08 Components-amd64.yml
-rw-r--r-- 1 root root  732506 avril 21 12:08 Components-amd64.yml.gz
-rw-r--r-- 1 root root 1023905 avril 21 12:09 icons-128x128.tar.gz
-rw------- 1 _apt root  409405 avril 21 12:09 icons-64x64.tar.gz

andre@andre-dotmu:/media/andre/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11$ ls -l
total 1472
-rw------- 1 _apt root 686135 août  18 19:17 Components-amd64.yml
-rw------- 1 _apt root 239447 août  22 14:49 Components-amd64.yml.gz
-rw-r--r-- 1 root root 406642 août  22 14:49 icons-128x128.tar.gz
-rw------- 1 _apt root 165416 août  22 14:49 icons-64x64.tar.gz

Now, when launching apt-get update, other error messages appear.

    sudo apt-get update
    Réception de:1 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
    Réception de:1 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
//shortening//
    Réception de:12 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages
    Err:12 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages
//shortening//
    Réception de:22 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial/main amd64 DEP-11 Metadata [2189 kB]
    Réception de:23 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial/main DEP-11 64x64 Icons [481 kB]
    Err:22 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial/main amd64 DEP-11 Metadata
      Impossible d'ouvrir le fichier /var/lib/apt/lists/partial
//shortening//
     Err:30 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial-updates/main amd64 DEP-11 Metadata
      Somme de contrôle de hachage incohérente
    Réception de:31 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons [205 kB]
    Ign:31 file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial-updates/main DEP-11 64x64 Icons
    Lecture des listes de paquets... Fait
    N: Can't drop privileges for downloading as file '/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission non accordée)
    W: The repository 'file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu xenial-security Release' does not have a Release file.
    N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
    N: See apt-secure(8) manpage for repository creation and user configuration details.
    E: Failed to fetch file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-security/restricted/binary-amd64/Packages  Fichier non trouvé - /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-security/restricted/binary-amd64/Packages (2: Aucun fichier ou dossier de ce type)
    E: Failed to fetch store:/var/lib/apt/lists/partial/_media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_dep11_Components-amd64.yml  Impossible d'ouvrir le fichier /var/lib/apt/lists/partial/_media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_dep11_Components-amd64.yml - open (13: Permission non accordée)
    E: Failed to fetch file:/media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11/Components-amd64.yml  Somme de contrôle de hachage incohérente
    E: Le téléchargement de quelques fichiers d'index a échoué, ils ont été ignorés, ou les anciens ont été utilisés à la place.

For "E: Failed to fetch store:/var/lib/apt/lists/partial/_media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_dep11_Components-amd64.yml", I've set the rights at 777 on "/var/lib/apt/lists/partial". Here is a list of the repository content:

administrateur@femmeTantsaha1:/var/lib/apt/lists/partial$ ls -l
total 388
-rw-r--r-- 1 root root 246846 apr 22 02:24 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_InRelease
lrwxrwxrwx 1 root root    113 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_dep11_Components-amd64.yml -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11/Components-amd64.yml
lrwxrwxrwx 1 root root    116 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_dep11_Components-amd64.yml.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11/Components-amd64.yml.gz
lrwxrwxrwx 1 root root    111 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_dep11_icons-64x64.tar.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/dep11/icons-64x64.tar.gz
lrwxrwxrwx 1 root root    109 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_i18n_Translation-en.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/i18n/Translation-en.gz
lrwxrwxrwx 1 root root    109 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_main_i18n_Translation-en.xz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/main/i18n/Translation-en.xz
lrwxrwxrwx 1 root root    122 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_restricted_dep11_Components-amd64.yml.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/restricted/dep11/Components-amd64.yml.gz
lrwxrwxrwx 1 root root    115 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial_restricted_i18n_Translation-en.xz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial/restricted/i18n/Translation-en.xz
-rw-r--r-- 1 root root  95650 aog 24 08:58 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-updates_InRelease
lrwxrwxrwx 1 root root    124 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-updates_main_dep11_Components-amd64.yml.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11/Components-amd64.yml.gz
lrwxrwxrwx 1 root root    119 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-updates_main_dep11_icons-64x64.tar.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/dep11/icons-64x64.tar.gz
lrwxrwxrwx 1 root root    117 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-updates_main_i18n_Translation-en.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/i18n/Translation-en.gz
lrwxrwxrwx 1 root root    117 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-updates_main_i18n_Translation-en.xz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/main/i18n/Translation-en.xz
lrwxrwxrwx 1 root root    130 aog 24 09:44 _media_administrateur_Depots_miroir_mirror_archive.ubuntu.com_ubuntu_dists_xenial-updates_restricted_dep11_Components-amd64.yml.gz -> /media/administrateur/Depots/miroir/mirror/archive.ubuntu.com/ubuntu/dists/xenial-updates/restricted/dep11/Components-amd64.yml.gz

There seems to some symbolic links, with all but the 2 ".yml" files I added poiting to archives. And those 2 files are bringing error reports.

I'm really outta my depth, and I've been trying to make this work for a while now. Any idea?

  • This may be too complex for AskUbuntu. Due to some of the repository innovations in 16.04 that occurred while apt-mirror didn't necessarily track them, this may require changes to apt-mirror. Joey Hess was an upstream author or maintainer at one point on the package but it hasn't been updated since Joey left Debian. I will need to dig further and reply again after some sleep. – Stephen Michael Kellat Sep 1 '16 at 2:28
1

On the client side, this appears to be a bug. I had similar problems with a fresh install of 17.04.

Issuing the command appstreamcli refresh --force resolved the issue for me.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.