30

I have a script which creates a nightly backup of several directories.

It is created using tar -czf, and the destination tar.gz file path is on a mounted network directory. The resulting file is about 1.2Gb large.

The network speed is reasonably fast (copying from the network drive to the local occurs at ~28MB/sec).

Now I want to use public-key encryption to encrypt the tar.gz file before transferring it to the network drive, and would like to know what the best way to do so is.

Should I create the file locally first, encrypt it, and then copy over? Or is there a way to "stream" tar output through an encryption process, and write the results directly to the network drive?

32

Pack your_dir in a encrypted archive your_archive.tgz.gpg (symmetric encryption):

tar -cz your_dir | gpg -c -o your_archive.tgz.gpg

Unpack it:

gpg -d your_archive.tgz.gpg | tar xz

See the docs of GPG for how to use asymmetric instead of symmetric encryption.

  • 2
    Using GPG seems creates keys in the user home holder, and prompts for a passphrase. Basically, the process seems to be too opaque for my taste… – Joel L Jan 17 '12 at 15:22
  • Should tar xz in the second set of commands be tar -xz instead? – Kenny Evitt May 29 '16 at 20:58
  • 1
    @KennyEvitt: You can use it with or without the - – Florian Diesch May 30 '16 at 0:23
  • This works for files encrypted with gpg-zip as well ! – Trefex Oct 16 '17 at 9:04
  • 1
    gpg already compresses files, don't think the z option does add much – feklee Feb 10 at 15:01
11

The following process encrypts the file on local disk first, and can then be sent over the network (or stored however needed)


First, generate public and private keys (done only once):

openssl genrsa -out key.pem 2048
openssl rsa -in key.pem -out key-public.pem -outform PEM -pubout

Then, at each backup:

  1. Generate long random passphrase, save in file

    • echo -n "Tl4R6dnvWXiDeXr1LtpCNkyLG1" > key.txt
  2. encrypt file with passphase

    • openssl enc -aes-256-cbc -pass file:key.txt < UNENCRYPTED_FILE > encrypted.dat
  3. encrypt passphrase with public key

    • openssl rsautl -encrypt -pubin -inkey key-public.pem < key.txt > enc.key.txt

Then save encrypted.dat AND enc.key.txt where desired.


To decrypt:

  1. Decrypt encrypted passphrase with private key

    • openssl rsautl -decrypt -inkey key.pem < enc.key.txt > key.txt
  2. Decrypt file

    • openssl enc -aes-256-cbc -d -pass file:key.txt < encrypted.dat > UNENCRYPTED_FILE

This is a lot longer than Florian's anwser, but I decided to use it so I can better understand the process, and not depend on server-dependent GPG configuration variables, etc. I also couldn't find any useful GPG documentation.

  • 1
    Random passphrase can also be created easily with openssl like so openssl rand 32 -out key.txt – Wolfgang Dec 27 '13 at 8:23
  • @Wolfgang — I'm actually generating the passphase from a python script (that runs the backup process), but thanks for the tip! – Joel L Dec 27 '13 at 11:37
  • 2
    FYI: Why you shouldn't encrypt files with openssl: stackoverflow.com/questions/28247821/… – Daniel Hiller Feb 21 '18 at 8:41
  • What exactly is this file 'enc.key.txt'? I am looking at a process that backs up my server files and only want to be able to decrypt using a key not stored on the server. – Alex Stewart Nov 30 '18 at 5:40
6

I do this with asymmetric key encryption. That means I have a public key (that I can share with anyone I'd like to send me encrypted packages) which allows me to encrypt the package. I also have a private key (that I do not share) which allows me to decrypt the package.

My commands to encrypt the current working directory: the -e to encrypt, the -r to specify a "recipient" or key to use, the -o to specify the output file.

$ tar -cvz . | gpg -e -r ABCD1234 -o backup.tgz.gpg

And to decrypt to the current working directory:

$ gpg -d backup.tgz.gpg | tar -xz

Or to decrypt to a standard tgz file for later unpacking:

$ gpg -o backup.tgz -d backup.tgz.gpg

Of course, this only works if I have already generated a public-private key pair and installed it with gpg. In my case, I have done so using Digital Ocean's guide at https://www.digitalocean.com/community/tutorials/how-to-use-gpg-to-encrypt-and-sign-messages-on-an-ubuntu-12-04-vps. The ABCD1234 in the encryption command refers to one of the public keys installed on my system. This guide also covers how to share your public key and install others' public keys to send and receive encrypted files.

0

GnuPG is what I've chosen to used for this process and because you've stated that decryption steps where to opaque (I agree in this usage instance) I've also worked on that side of the problem too. Check the projects' Travis-CI build logs for what features currently work as designed and the travis.yml file for following along with the logs' output. Basically you'll need three scripts from this project; the keygen script, decryption helper script, and named pipe listener script. The keygen script & decryption helper script should be used on the device that will preform decryption and the named pipe listener script should be on the device preforming encryption.

The encrypting named pipe listener script accepts strings, file paths, or directory paths once setup and outputs encrypted results in a predictable fashion.

Below are example commands that could be placed into your nightly backup script for encryption and for directories compression+encryption

echo "some messages text" > /path/to/named.pipe
## Message text is ASCII armor encrypted and appended to specified file when script was stated
echo "${HOME}/Documents" > /path/to/named.pipe
## Directories are compressed with tar and encrypted with output to time stamped file in bulk output directory
echo "${HOME}/.bash_history"  > /path/to/named.pipe

For decryption you'll want to check out how the build scripts .travis-ci/script_decrypt.sh & .travis-ci/test_search_script_decrypt.sh for how the appended strings of encrypted data are recovered and how bulk files/directories are restored.

Of course with stuff this experimental it's best to not use primary key pares (kinda why the keygen script was written) and it should not be used on important data until you're confident in how to restore it to readable form.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.