0

Can anyone assist me with this? I've been using deja-dup to backup my Ubuntu 14.04 machine for a while. I have duplicity version 0.6.23, deja-dup version 30.0. No matter which backup I select, I get errors.

When I use the deja-dup UI, I simply get this error: "BackendException: Error listng s3+http [I'm putting a space bere because it won't let me post a link] ://mybucketname1/computer-XPS13-9333"

(for the purposes of posting I'm anonymizing all identifying info in error messages and commands)

I've tried a few variations through the command line, using duplicity, but am not having any luck.

This command:

AWS_ACCESS_KEY_ID=XXXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXX duplicity restore s3+http

[I'm putting a space bere because it won't let me post a link] ://mybucketname1/computer-XPS13-9333 /media/standard/Seagate\ Backup\ Plus\ Drive/restore/

returns this error:

PermanentRedirect The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><Bucket>mybucketname1</Bucket><Endpoint>mybucketname1.s3.amazonaws.com</Endpoint><RequestId>XXXXXXXX</RequestId><HostId>Pwl/XXXXXXXXXXXXXXXXXXXXXXXX=</HostId></Error>

So then I tried it with a different endpoint, and ran this:

AWS_ACCESS_KEY_ID=XXXXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXXXX duplicity restore s3+http
[I'm putting a space heere because it won't let me post a link]
://mybucketname1.s3.amazonaws.com/computer-XPS13-9333 /media/standard/Seagate\ Backup\ Plus\ Drive/restore/

<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><BucketName>mybucketname1.s3.amazonaws.com</BucketName><RequestId>XXXXXXXXX</RequestId><HostId>XXXXXXXXXXXXXXX</HostId></Error>

I thought maybe the endpoint it was telling me was wrong, so I tried what I found in the S3 site and ran this:

AWS_ACCESS_KEY_ID=XXXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXX duplicity restore s3+https
[I'm putting a space bere because it won't let me post a link]
://s3-us-west-2.amazonaws.com/mybucket1/computer-XPS13-9333 /media/standard/Seagate\ Backup\ Plus\ Drive/restore/

and got this error:

UnsupportedBackendScheme: scheme not supported in url: s3

I thought this might be an http/https issue so I tried the same command as http, and got this error, but I am pretty sure this endpoint is not a real URL that I should use:

PermanentRedirectThe bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.s3-us-west-2.amazonaws.coms3-us-west-2.amazonaws.com.s3.amazonaws.comXXXXXXXXXXXXXXXXXXXXX

Does anyone have any recommendations? I lost a bunch of data and am hoping to get it back, and start backing it up using dropbox or something instead. I've been researching this problem for a while and apparently duplicity has all kinds of problems, but I still haven't found a solution to this one.

0

I figured this out and I'm answering my own questions for the benefit of anyone else with similar problems. First, I discovered that I cannot restore an entire backup at once. It just is too large and times out. The strategy I had to take was to do subdirectories at a time. Also, when I tried to list-current-files, it would return nothing. Even though the backup files were huge, it said I had no files to restore. I figured out that if I used the -t flag, it would work. This probably means my last backup was corrupt, but by looking some amount backwards in time (6 months in my case), it was able to find some files to restore. First I had to get that list of files, so I knew which files to restore:

AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxx AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxx PASSPHRASE=xxxxxxxxx duplicity list-current-files --timeout=2400 --tempdir /path/path/path/ --num-retries=500 -t 6M s3://s3-us-west-2.amazonaws.com/mybucketname1/computer-name | tee -a restore_file_list-6M.txt

This gives you a file restore_file_list-6M.txt which you'll need to use to find what files are in your backup.

You'll also notice that despite what documentation says, the s3+http:// is wrong, you want s3:// instead.

Once I got the list of files in the backup tee'd into restore_file_list-6M.txt, I could restore them one directory at a time using this command:

AWS_ACCESS_KEY_ID=XXXXXXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXX PASSPHRASE=XXXXXXXXXXXX duplicity restore --timeout=2400 --tempdir /path/path/path/ --allow-source-mismatch --file-to-restore source/in/backup --num-retries=500 -t 6M  s3://s3-us-west-2.amazonaws.com/mybucketname1/computer-name /place/I/want/to/restore/to/

I don't know if the --timeout flag had any function, but the --num-retries is definitely important, because it times out about once per file, and sometimes would time out over a hundred times before successfully downloading a file.

Now, I am done with duplicity forever.

  • I appreciate you answering your own question. I too use duplicity with Amazon S3. My experience has been that the S3 support in boto (the library duplicity uses for S3) is flaky at best. Similar issues about network timeouts are seen with aws-cli and s3cmd as well. I love duplicity, but will be switching my backend over to something not boto-related. – akohlsmith Jul 14 '17 at 2:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.