Backup with duplicity on Google Cloud Storage

Question

I have already set a backup routine that backups all my files to Amazon S3 using duplicity. I would like to create a similar backup routine with Google Cloud Storage. Is it possible to get duplicity to work with GCS?

Answer 1

This is a partial-answer that might help the next person:

So far it seems that copying things to Google Drive/Docs is supported:

duplicity localdir gdocs://username:password@gmail.com/remotedir/

However this is different from Google Cloud Storage. I think this should be doable since duplicity and gsutil both use boto, but I'm not sure if this requires work to be done in duplicity itself though.

As a work around until you sort this, you can certainly copy directly from the s3 to the GCS buckets:

gsutil -m cp -R s3://yourbucket gs://yourbucket

Answer 2

• As of August 2012, Duplicity does not support Google Cloud Storage (GCS); a search of the duplicity-talk mailing list archives shows that there has been no discussion, even at a wishlist stage, of adding such support.
  • Google Docs is supported, but that API is totally different from GCS.
  • Wishlist bug filed recently
• While the Python boto module used as a back-end by Duplicity to provide S3 support also supports GCS, adding GCS support to duplicity would require non-trivial modifications to its code; the S3 and GCS APIs are not identical.
  • If anyone wants to give adding GCS support a go, it would be simplest to start with and modify the _boto_single.py file which provides the backend for single-threaded S3 transactions.
• That GCS isn't widely supported shouldn't come as a surprise. It is nowhere near as universal as S3; for example, you must register as a developer with Google and the service is still considered part of the "beta" Google Labs.
  • The only backup solution I could find supporting GCS is Zmanda (commercial).
  • As Google rolls out GCS on a wider basis--beyond just developers--I think we should start seeing support for it in FOSS apps.
• For now, to use Duplicity with GCS, you must either backup to S3 and then use Google's gsutil tool to sync S3 with GCS, as Jorge's answer indicates; alternately, you could use Duplicity to backup locally and then sync that with GCS using gsutil.