8

I lost the drive, and the Ubuntu OS installed in it, from which I created a backup using Duplicity, and I don't know how to restore the archived files I still keep in another drive. They all follow these 4 notations

duplicity-full.20140106T111233Z.manifest
duplicity-full-signatures.20140106T111233Z.sigtar.gz
duplicity-full.20140106T111233Z.volxxx.difftar.gz (multiple volumes)
duplicity-full.20140106T111233Z.volxxx.difftar (multiple volumes)

There's important data I need to recover. Is there anything I can do? Would be enough to use gzip? And if so, how could I extract them all easily?

EDIT 1 Trying your suggestions, both with the restore and the --gio options, I get this text:

duplicity restore file://media/ubuntu/Toshiba\ HDD/BACKUPS/Documents/ /media/ubuntu/9fb63c8e-ecb5-4c55-b4e0-282e7b4a82ff/tmp/docs/
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
GnuPG passphrase: 
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1422, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 697, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/bin/duplicity", line 719, in restore_get_patched_rop_iter
    backup_chain = col_stats.get_backup_chain_at_time(time)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 952, in get_backup_chain_at_time
    raise CollectionsError("No backup chains found")
CollectionsError: No backup chains found

/media/ubuntu/Toshiba\ HDD/BACKUPS/Documents/ is where all the duplicity files are, and /media/ubuntu/9fb63c8e-ecb5-4c55-b4e0-282e7b4a82ff/tmp/docs/ a temporary destination folder I want to restore the files to.

EDIT 2 Corrected the missing slash in file://media.... Now I get restored files, but not exactly what I expected. There are a lot of files missing (up to 24 GB in there!)

root@ubuntu:/media/ubuntu/Toshiba HDD/BACKUPS/Documents# ls -la /tmp/docs/home/luis/
total 4176
drwx------ 16 root root    1000 Jun 30 19:13 .
drwx------  3 root root      60 Jun 30 19:13 ..
drwx------  3 1000 1000      60 Oct 23  2013 .activestate
drwxr-xr-x  2 1000 1000      60 Oct 23  2013 .ActiveState
drwx------  3 1000 1000      60 Dec 28  2012 .adobe
-rw-------  1 1000 1000   32120 Jan  4 12:56 .bash_history
-rw-r--r--  1 1000 1000     220 Dec 28  2012 .bash_logout
-rw-r--r--  1 1000 1000    3637 Dec 28  2012 .bashrc
drwx------  3 1000 1000      60 Jan  3 14:01 .cache
drwxr-xr-x  3 1000 1000      60 Nov 16  2013 .cddb
-rw-rw-r--  1 1000 1000     740 Jan 10  2013 .cola
drwx------  3 1000 1000      60 Dec 29  2012 .compiz
drwxrwxr-x  3 1000 1000      80 Oct  1  2013 .composer
drwx------ 46 1000 1000    1020 Dec  8  2013 .config
drwx------  3 1000 1000      60 Dec 28  2012 .dbus
-rw-r--r--  1 1000 1000      25 Jul 13  2013 .dmrc
drwx------  3 1000 1000     380 Jan  6 10:31 .dropbox
drwxr-xr-x  6 1000 1000    1660 Dec 17  2013 .dropbox-dist
drwx------  3 root root      60 Jun 30 19:13 .eclipse
-rw-------  1 1000 1000   50282 Jan  6 10:29 .ICEauthority
drwxrwxr-x 14 1000 1000     340 Dec  7  2013 .PlayOnLinux
-rw-r--r--  1 root root 2077491 Nov 23  2013 .Soulseek.1385165090491
-rw-r--r--  1 root root 2076644 Nov 23  2013 .Soulseek.1385166430938
-rw-r--r--  1 1000 1000    9986 Dec 30 20:02 .Soulseek.1388433748295
drwxrwxr-x  2 1000 1000      60 Oct 13  2013 .SyncWall
-rw-------  1 1000 1000     109 Jan  6 10:29 .Xauthority
-rw-rw-r--  1 1000 1000       0 Jul 29  2013 .Xauthority.25Y20W
-rw-rw-r--  1 1000 1000       0 Jul 13  2013 .Xauthority.7K14ZW
-rw-rw-r--  1 1000 1000       0 Jul  7  2013 .Xauthority.7K7SZW
-rw-rw-r--  1 1000 1000       0 Jul 13  2013 .Xauthority.9X1E0W
-rw-rw-r--  1 1000 1000       0 Sep  1  2013 .Xauthority.A3D52W
-rw-rw-r--  1 1000 1000       0 Sep  1  2013 .Xauthority.CEUV2W
-rw-rw-r--  1 1000 1000       0 Jul 27  2013 .Xauthority.CP7Q0W
-rw-rw-r--  1 1000 1000       0 Sep  8  2013 .Xauthority.E1ET2W
-rw-rw-r--  1 1000 1000       0 Aug 26  2013 .Xauthority.E32K2W
-rw-rw-r--  1 1000 1000       0 Sep 14  2013 .Xauthority.EKK92W
-rw-rw-r--  1 1000 1000       0 Jul 12  2013 .Xauthority.F4QRZW
-rw-rw-r--  1 1000 1000       0 Jul 25  2013 .Xauthority.ISVZ0W
-rw-rw-r--  1 1000 1000       0 Jul 11  2013 .Xauthority.JU3UZW
-rw-rw-r--  1 1000 1000       0 Aug 31  2013 .Xauthority.M30S2W
-rw-rw-r--  1 1000 1000       0 Aug  1  2013 .Xauthority.M3H20W
-rw-rw-r--  1 1000 1000       0 Sep  9  2013 .Xauthority.MKIN2W
-rw-rw-r--  1 1000 1000       0 Aug 28  2013 .Xauthority.RWHB2W
-rw-rw-r--  1 1000 1000       0 Jul 15  2013 .Xauthority.SN85ZW
-rw-rw-r--  1 1000 1000       0 Sep 15  2013 .Xauthority.T22C3W
-rw-rw-r--  1 1000 1000       0 Jul 14  2013 .Xauthority.T6CB0W
-rw-rw-r--  1 1000 1000       0 Jul 31  2013 .Xauthority.VF7Q0W
-rw-rw-r--  1 1000 1000       0 Sep  9  2013 .Xauthority.WGEX2W
-rw-rw-r--  1 1000 1000       0 Aug 31  2013 .Xauthority.WZR52W
-rw-rw-r--  1 1000 1000       0 Jul 25  2013 .Xauthority.YQIR0W

Most strange is, besides all the files starting with a dot, that I can't browse the home/luis directories with Nautilus, which in fact were where all my backed up data was.

4
  • Still getting the same response with 3 slashes. But thanks for pointing that. I'm going to add it in the edited text Jun 30, 2014 at 19:10
  • Wait! I misspelled a path. Now it worked! But I do not get what I expected. Jun 30, 2014 at 19:39
  • /media/ubuntu/9fb63c8e-ecb5-4c55-b4e0-282e7b4a82ff/tmp/docs/ is empty Jun 30, 2014 at 21:02
  • I initially used the path you mentioned, and eventually dumped again the restoration data into the directory I wrote, getting exactly the same files listed above. Is there anything more to do to restore everything just as it was stored in the unit I lost? Jun 30, 2014 at 21:49

3 Answers 3

17
+50

CLI to extract a duplicity archive

  1. Use duplicity to extract it:

     duplicity restore file:///path_to_folder_contains_backups/ path_where_to_extract_it/
    
  2. Enter GnuPG passphrase, Press Enter

By default, Duplicity restores last backup (comparing time) in the source folder even its not the backup for the same folder. If you have multiple backups for different folders or you want specific time of same folder, use --time option example:

duplicity restore --time 20140106T111233Z file:///path_to_folder_contains_backups/ path_where_to_extract_it/

Searching for specific filename pattern in all archives

Unable to see the files you are looking for, try:

  1. Create function that list all files contained in all full/inc backups available in the source folder:

     ldup () {
         for f in $(ls $1/duplicity-*.manifest.gpg); do
             echo "========================================================================"; 
             echo $f;
             n=$(basename $f .manifest.gpg);
             prefix=$(echo $n | awk 'BEGIN { FS = "." } ;{ print $1 }');
             if [[ "$prefix" == "duplicity-full" ]]; then
                 t=$(echo $n | awk 'BEGIN { FS = "." } ;{ print $2 }');
             else
                 t=$(echo $n | awk 'BEGIN { FS = "." } ;{ print $4 }');
             fi;
             echo -e "Creation time:\t" $t;
             echo "------------------------------------------------------------------------";
             duplicity list-current-files --time $t file://$1/ | grep "$2";
         done;
     }
    
  2. Either one bellow

  • List all files (note without tail /):

    ldup /path_to_folder_contains_backups
    
  • Search for a pattern in file names:

    ldup /path_to_folder_contains_backups pattern
    

The script function does not support space in folder or archive name.

Reference: man duplicity

4
  • Thanks for helping, but it's still not working. See edit for results on your suggestion Jun 30, 2014 at 19:01
  • @LuisMartin, Do you have other/multiple backups in same folder? You may have to specify time to select the backup you want. See updated answer.
    – user.dz
    Jul 1, 2014 at 2:42
  • I greatly appreciate your help, but choosing the --time option with that "timestamp" led to similar results, and that's the only timestamp in the archives names :/ Jul 1, 2014 at 21:18
  • @LuisMartin, I have updated the answer with a script to search all files contained in backups.
    – user.dz
    Jul 4, 2014 at 7:48
6

First try to restore with duplicity.
duplicity --gio file:///media/backup /tmp/restore

There are some reasonably detailed (and mostly correct) steps over here.

Key details

The key point is to unpack all of the duplicity-full.*.difftar.gz files in the same place, so that you're left with just two snapshot/ and multivol_snapshot/ directories.

If your file is in snapshot/ then you're done. Otherwise find the directory in multivol_snapshot/ at the path where your file used to be: you need to join together all the files in this directory to recreate the original file. The files are numbered, and can be joined together using the cat command. Depending on how large the original was, there may be many parts.

Problem with original instructions

The directions linked above suggest using cat * > rescued-file. Unfortunately this simple approach fails if you have more than 9 parts. Since * expands in dictionary order, not numeric order, 10 would be listed before 2, and the file would be reconstructed in the wrong order.

Workaround

One simple approach is to remember that dictionary order does work when numbers are the same length, and that ? matches a single character. So if your largest file has three digits, you can manually enter:

cat ? ?? ??? > rescued-file

Add or remove ? patterns as necessary, depending on the largest file number.

Script

If you have a lot of files to recover and don't fancy typing that for all of them, you might prefer to use a script such as this. It lists the containing directory for every file, removes duplicates from the list, then goes to each directory and creates a content file from the fragments there. (spacer is just to make $1 work.)

find multivol_snapshot/ -type f -printf '%h\0' | \
  sort -uz | \
  xargs -0 -n 1 sh -c 'cd "$1" ; cat $(ls | sort -n) > content' spacer

Now you just have to add /content to the end of any filename you were looking for, and you should find it.

Limitations

This doesn't restore any of the original file permissions or ownership. It also doesn't deal with incremental backups, but then the linked instructions also hit a bit of a dead end on this point — they just suggest using rdiff 'to stitch the files together' and refer the reader to man rdiff.

source : SF SE
here you can find a java program for easy restoration of complicated backup.

1
  • I've just tried what you suggested, without success (see edited post) Jun 30, 2014 at 18:47
0

Déjà Dup uses an opaque format for files stored in your backup location. You must use Déjà Dup or another duplicity-based tool to restore your files (See this restore guide).

1
  • I've tried it, but I can't still restore the files (see edit) Jun 30, 2014 at 18:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .