Restoring backups failing Ubuntu 16.04

Question

I'm struggling to restore backups from an older installation of Ubuntu to a new installation on Ubuntu 16.04. I get the following error when trying to restore:

Failed with an unknown error Followed by:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1461, in do_backup
    list_current(col_stats)
  File "/usr/bin/duplicity", line 698, in list_current
    for path in path_iter:
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 354, in combine_path_iters
    refresh_triple_list(triple_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 341, in refresh_triple_list
    new_triple = get_triple(old_triple[1])
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 327, in get_triple
    path = path_iter_list[iter_index].next()
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 239, in sigtar2path_iter
    for tarinfo in tf:
  File "/usr/lib/python2.7/tarfile.py", line 2508, in next
    tarinfo = self.tarfile.next()
  File "/usr/lib/python2.7/tarfile.py", line 2350, in next
    raise ReadError("unexpected end of data")
ReadError: unexpected end of data

The backup completed successfully but I cannot get it to restore.

Thanks in advance!

Answer 1

Edit: Disclaimer - I don't have much experience with Python or duplicity's code so I cannot say if this modification will or will not cause any detrimental effects. For me, I was able to complete my restore by adding the line of code below.

Solution: I was able to move past this inserting a key check for the volume_name_dict object.

New line: if vol_num in backup_set.volume_name_dict.keys():

In duplicity 0.7.06, you can replace this method at line 752:

def get_fileobj_iter(backup_set):
    """Get file object iterator from backup_set contain given index"""
    manifest = backup_set.get_manifest()
    volumes = manifest.get_containing_volumes(index)
    for vol_num in volumes:
           if vol_num in backup_set.volume_name_dict.keys():
               yield restore_get_enc_fileobj(backup_set.backend,
                                             backup_set.
                                             volume_name_dict[vol_num],
                                             manifest.
                                             volume_info_dict[vol_num])
               cur_vol[0] += 1
               log.Progress(_('Processed volume %d of %d') %
                            (cur_vol[0], num_vols),
                            cur_vol[0], num_vols)
if hasattr(globals.backend, 'pre_process_download'):
    file_names = []
    for backup_set in backup_setlist:
        manifest = backup_set.get_manifest()
        volumes = manifest.get_containing_volumes(index)
        for vol_num in volumes:
            file_names.append(backup_set.volume_name_dict[vol_num])
    globals.backend.pre_process_download(file_names)

fileobj_iters = list(map(get_fileobj_iter, backup_setlist))
tarfiles = list(map(patchdir.TarFile_FromFileobjs, fileobj_iters))
return patchdir.tarfiles2rop_iter(tarfiles, index)

If you are not familiar with Python (I'm not) you can validate the code here: http://pep8online.com/ - the indentations must align properly.

Answer 2

This worked for me:

1/ Download the latter estable duplicity version (currently v0.7.14):

http://duplicity.nongnu.org/

2/ Move to /usr/local:

% cd /usr/local

3/ Extract the tar ball

% tar -xvf <DuplicityTarBallHome>/duplicity-0.7.14.tar.gz

4/ Move to the extracted directory

% cd /usr/local/duplicity-0.7.14

5/ Install the version of duplicity:

% python setup.py install

-> The installation should finish without errors

6/ Verify installation of the latter version:

% duplicity --version

-> duplicity 0.7.14

7/ Finally restore the Backup:

% duplicity restore file:<BACKUP_HOME> <DESTINATION_HOME>

-> The files are correctly restored

Answer 3

I'll add this bit of info since my similar question here Déjà Dup backup failing with python stack trace on Ubuntu 16.04 was marked as a duplicate (it was not really a duplicate, I was not trying to restore a backup, I was trying to create backups, although the error was the same).

I had not used Déjà Dup after I posted my question; since it wasn't working I turned off the schedule. Then when Victor Doors posted his answer to my question also, I tried again without making any changes to see if I was still getting the error. Déjà Dup said it was making a full backup which might take longer, but it succeeded. I then re-enabled the daily schedule and the incremental backup also succeeded last night. So, I think my problem may have been fixed in some Ubuntu update. The duplicity version on my machine is currently 0.7.06. So I would try doing a software update and see if that fixes the problem before anything else.