I've been experimenting with s3ql on Ubuntu 10.04, using it to mount Amazon S3 buckets. However, I'd really like it to mount them automatically. Does anyone know how to do that?
The first step is to create an authinfo file. This file should be placed in a .s3ql directory within the home directory of the user who will be using it. The authinfo file contains login info allowing s3ql to mount buckets without prompting. Below is an example of what your authinfo file should look like. The first line contains your Amazon Security Credentials. The second contains the location and password for your bucket. You can add multiple bucket-lines to this file if needed, but I'm only using one in this example. At this point, the bucket password can be anything.
backend s3 machine any login YourAWSAccessKeyID password YourAWSSecretAccessKey storage-url s3://mybucket password EncryptionPasswordOfYourChoosing
encryption_password_of_your_choosing is used by S3QL to encrypt/decrypt all file writes/reads from S3.
The bucket name has to be unique across all of AWS: no two users can have the same bucket name, so its a good idea to log into Amazon Web Services and try different names until you find one that's available. One good naming scheme is to use the name of the domain+server you'll be accessing the bucket from (i.e. "staging.example.com" or "east.coast.01.example.com", or whatever is appropriate).
To create the file system, use the command:
It will prompt you for your encryption password. This should be the same as the bucket password in the authinfo file.
Now that your file system is created, you can mount it using the command:
mount.s3ql s3://mybucket /mnt/s3/bucket
Of course, your bucket name and mount point will vary.
Now, if we want to mount this bucket automatically on boot, we need to add an upstart script to /etc/init. Fortunately, s3ql comes packaged with one, s3ql.conf.
I added "--allow-other" to the mount.s3ql command to allow users other than root to access the mounted bucket.
# # This file can be placed in /etc/init. It defines an upstart job that # takes care of mounting and unmounting an S3QL file system. # description "S3QL Backup File System" author "Nikolaus Rath <[email protected]>" start on (filesystem and net-device-up IFACE=eth0) stop on runlevel  env BUCKET="s3://mybucket" env MOUNTPOINT="/mnt/s3/bucket" expect stop script # Redirect stdout and stderr into the system log DIR=$(mktemp -d) mkfifo "$DIR/LOG_FIFO" logger -t s3ql -p local0.info < "$DIR/LOG_FIFO" & exec > "$DIR/LOG_FIFO" exec 2>&1 rm -rf "$DIR" # Check and mount file system fsck.s3ql --batch "$BUCKET" exec mount.s3ql --upstart --allow-other "$BUCKET" "$MOUNTPOINT" end script pre-stop script umount.s3ql "$MOUNTPOINT" end script
After adding this script, in theory you should be able to reboot and have your bucket automatically mounted, but this is were I ran into trouble. Mine wasn't being mounted.
My problem was caused by the fact that upstart was running the script as root, but I'd created the file system as another user. Once I copied the .s3ql directory from the home directory of the user I'd been logged-in as to /root, the problem was solved.
I hope this helps someone else out there. Although I haven't been using my mounted S3 bucket for long, I'm impressed with how my initial tests have gone.
Also, this answer was written about a week after the solution was found. I think I've covered everything, but if you find I've missed a step, let me know and I'll add it. You'll also want to read the manual, its really worth reading if you intend to use s3ql.