Deja-Dup Backup Failed - Invalid data - SHA 1 hash mismatch for file

Question

On my most recent weekly backup today, Deja-Dup gave me this little love note:

Backup Failed

Invalid data - SHA1 hash mismatch for file:
duplicity-inc.20130124T230054Z.to.20130201T225108Z.vol1.difftar.gpg

Calculated hash: 7726f55012e1e26cc762c9982e7c6c54ca7bb303
Manifest hash: 205ecad0a91f8a11967b70d2d3fbc8e4d06231f5

I'm running 12.10 and have been running weekly deja-dup backups since I installed it.

I understand from reading other threads that this is a known software bug that happens when duplicity is interrupted, but most of those other threads are people trying to restore from these corrupted backups.

I've tried deleting and rm-ing the file in question to try again, but I get an error saying the file wasn't found.

My question is, what does this mean for my backups going forward? Did this week's backup work? Will next week's? If not, how can I resolve this error?

I'm not too concerned about the old versions of my files, and even if I need them in the future, I have some disk images saved which I could restore from. So should I just delete everything and start deja-dup from scratch?

Any advice is appreciated.

Answer 1

I have had this problem recently myself. I have not found an 'ideal' solution. The safest option is to discard the faulty backup and start a new one, as you mention in your comment.

Other answers to similar problems suggest workarounds for restoring files by using previous backups or ignoring the files in the corrupted volume, but this is clearly sub-optimal, not to say fiddly to achieve in practice. However, this does not help a failed backup.

Trying to force a full backup of the files in the failed volume is counter-productive, because the next incremental backup is huge, covering all the other files; you might as well delete the failed backup and start again.

I did find a way to implement a patch of the failed part of the backup. Here is the recipe:

1. Find the affected files by noting the volume number, and then check against the volume number in the manifest file.
2. Touch the affected files (touch -m /name/of/file).
3. Do an incremental backup.

The incremental backup will contain the affected file(s), plus any others that changed in the interim, and the SHA1 error is no longer reported... except by verification (see below).

I tested this using duplicity directly rather than the deja-dup gui, because it enabled me to have a bit more control, and to do additional things like verify the backup (duplicity verify /target/directory url:///for/backup/archive). It does not remove the original SHA1 problem, but mitigates it by providing a way to restore the files in the presumably corrupted backup volume(s).

By the same token, I think if you are serious about backups, you need to forget deja-dup and use duplicity directly instead, because backups without verification aren't worth anything. I discovered that the hard way, having used deja-dup for almost two years, before a failed backup made me realize that actually my backup schedule was a house built on sand; when I checked the external drive I had been using, about 5% of all the backup files were unreadable. Since then, I have found that using shell scripts with duplicity isn't so hard, and once it is set up it is very easy.