6

On gmail.com, I see this:

0.38 GB (2%) of 15 GB used

I have 20 GB of my 30 GB partition filled with Ubuntu stuff, but probably only 1 GB of important documents, so why not use this free cloud storage for backup?

Is there already a program to do this? I'll probably write one anyway but wanted to ask before I start.

I also want file compression and segmentation, because some systems have 10 MB attachment limits (like the one at work).

Thanks :)

  • 4
    I think you mean Google Drive. There are a few programs which can access Google Drive but I don't think they automatically compress or segment. A script to do this using tar and drive wouldn't be too difficult – Evan Chen May 22 '17 at 0:10
  • 1
    Speaking of free cloud storage, Dropbox has an official client for Linux (Drive doesn't), and you get 2GB free. So I have a folder ~/Dropbox, and all my docs are symlinked there, e.g. ~/Documents -> Dropbox/Documents, and ~/.bashrc -> Documents/.bashrc. Then I have DropSync on my Android, which is a 3rd-party Dropbox sync app. – wjandrea Jul 24 '17 at 20:03
  • @wjandrea The ~/Dropbox information is very helpful. Your professional-grade edits on the question and answer below are even more helpful. Thank you :) – WinEunuuchs2Unix Jul 25 '17 at 13:25
3

Edit April 3, 2018

Historical post in next section

The original part of the answer is left intact in the next section for historical reference to trial and error

Backup script to create .tar file

This is the current backup script:

#!/bin/bash

# NAME: daily-backup.sh
# PATH: /mnt/e/bin
# DESC: Backup scripts, documents and configuration files to .tar

# DATE: July 11, 2017. Modified Oct 20, 2017.

# PARM: 1=backup file name. Extension .tar automatically appended.

# NOTE: To include MBR (Master Boot Record) in backup create an image using:
#       sudo dd if=/dev/sda of="$HOME/.mbr.sav" bs=512 count=1

# NOTE: CLONE CURRENT INSTALLATION TO NEW MACHINE
#       =========================================

#       To restore use Live USB to install Ubuntu alongside Windows 10
#       Connect to network with password xxxxxxxxx

#       Install Google Chrome
#       (https://askubuntu.com/questions/510056/how-to-install-google-chrome):

#           wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub 
#               | sudo apt-key add
#           echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/
#               stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list
#           sudo apt update
#           sudo apt install google-chrome-stable

#       Open gmail.com and download attachment `$1` which is usually called
#           Backup-yymmdd-DayOfWeekName.tar

#       Make missing home/bin directory which tar doesn't create automatically:
#           mkdir ~/bin

#       Restore the daily backup using:
#           sudo tar -xvf Backup-yymmdd-DayFfWeekName.tar -C /
#           yar -xvf Backup-yymmdd-DayFfWeekName.tar -C /

#       Patch /etc/default/grub with new machine parameters, ie for nvme use:
#           acpiphp.disable=1

#       Use `sudo apt install aptitude-common`
#       Clone packages using `aptitude-create-state-bundle` on Source
#       Copy state-bundle.tar file from Source to Target machine
#       Restore packages using `aptitude-run-state-bundle` on Target

#       Manually copy ~/Pictures, ~/Videos, etc. not in daily backup.

#       sudo update-grub        # NVMe suspend/resume acpiphp.disable=1
#       sudo update-initramfs   # to get plymouth sunrise splash screen

if [[ $# -ne 1 ]]; then
    echo 'One argument required for file name, e.g. "Backup-2017-10-21-Saturday"'
    echo '.tar will automatically be added as a file extension'
    exit 1
fi

FileName="$1.tar"
HomeDir="/home/Me"                    # Required for cron compatibility
EmailAddr="MyEmail@gmail.com"

cd $HomeDir ||
    exit 1

dpkg --get-selections > .packages       # List of installed applications

tar -cvpf "$FileName" bin               # create .tar & add user scripts
tar -rvpf "$FileName" /usr/local/bin    # add global root-based scripts
tar -rvpf "$FileName" /etc/cron*        # crontab, cron.d, cron.daily, etc
tar -rvpf "$FileName" /etc/system*      # systemd files: login.conf, etc.
tar -rvpf "$FileName" /lib/systemd/system-sleep
tar -rvpf "$FileName" /etc/rc.local     # Startup script: calls zaprestore.
tar -rvpf "$FileName" /etc/sudoers      # 120 minute sudo, stars in password
tar -rvpf "$FileName" /etc/default/grub # bootstrap loader
tar -rvpf "$FileName" /boot/grub        # Custom grub fonts and splash...
tar -vpf "$FileName" /usr/share/plymouth   # ... screen (plymouth)
tar -rvpf "$FileName" /usr/share/plymouth/themes/earth-sunrise/
tar -rvpf "$FileName" Desktop           # files and links on desktop
tar -rvpf "$FileName" Documents/*.od*   # Libre Office: *.ods, *.odt, etc.

# Trusted keys to install from third party PPAs
tar -rvpf "$FileName" /etc/apt/trusted.gpg
tar -rvpf "$FileName" /etc/apt/trusted.gpg.d

# Sources for repositories - 1) Main single file - 2) directory of files
tar -rvpf "$FileName" /etc/apt/sources.list
tar -rvpf "$FileName" /etc/apt/sources.list.d

# find all $HOME/.config files and add to .tar
find .* -maxdepth 0 -type f -exec tar -rvf "$FileName" {} +

# Nautilus custom scripts
tar -rvpf "$FileName" .local/share/nautilus/scripts

# /etc/udev rules
tar -rvpf "$FileName" /etc/udev/rules.d

# /etc/rc.local
tar -rvpf "$FileName" /etc/rc.local

# /etc/X11/xorg.conf.d
tar -rvpf "$FileName" /etc/X11/xorg.conf.d

# /mnt/e - shared WSL + Linux
tar -rvpf "$FileName" /mnt/e/bin
tar -rvpf "$FileName" /mnt/e/Documents

echo "Wait a minute... Emailing: $EmailAddr"

# From: https://internetlifeforum.com/gmail/2251-gmail-some-file-types-blocked-fix-how-go-around/
# cat archive.tar.gz | base64 > file
# then i sent the file via email:
# echo "Base64 encoded file" | mutt -a file -s subject -- mymail@gmail.com
# then mail was delivered properly!
# then when one need to get readable archive again, he need to decode it by base64. In my case i do it via linux command line:
# cat file | base64 -d > decodedarchive.tar.gz

FileName64="$FileName".64
cat "$FileName" | base64 > "$FileName64"
echo -e "to: $EmailAddr\nsubject: $FileName64\n" | \
    (cat - && uuencode "$FileName64" "$FileName64") | ssmtp "$EmailAddr"

#echo -e "to: $EmailAddr\nsubject: $FileName\n" | \
#    (cat - && uuencode "$FileName" "$FileName") | ssmtp "$EmailAddr"

ls -la $FileName
ls -la $FileName64

rm "$FileName"
rm "$FileName64"

exit 0

Replace /home/Me above with your user name. Replace MyEmail@gamil.com with your actual gmail address. Change the directory /mnt/e/bin to the directory you store your bash scripts. Save the file and exit. Then use:

chmod a+x /mnt/e/bin/backup

This makes the script executable.

Notice how the MBR (Master Boot Record) is saved to backup. A separate earlier step to create ~/.mbr.sav using sudo dd ... is required as described in script comments.

Notice the dpkg --get-selections line. This creates backs up a list of all installed application names.

Simplest way to automate sending email

From Send email alerts using ssmtp we find the simplest way of sending email automated from terminal or script. The installation steps are straight forward:

sudo apt install ssmtp
sudo nano /etc/ssmtp/ssmtp.conf
# Change "MyEmailAddress" and "MyPassword" to your own.

There is one step not mentioned; Google will send you an email confirming you want to allow a "less secure" application to send mail with your account:

gmail turns on less secure apps for email

After installing and configuring ssmpt one more package is required in order to attach your .tar backup file to an email message:

sudo apt install sharutils

This package contains the program uuencode which is need to convert binary files for transmission.

Setup cron daily to call backup script

Create the file /etc/cron.daily/daily-backup containing:

#!/bin/sh
#
# NAME: daily-backup
# DESC: A .tar backup file is created, emailed and removed.
# DATE: Nov 25, 2017.
# CALL: WSL or Ubuntu calls from /etc/cron.daily/daily-backup
# PARM: No parameters but /etc/ssmtp/ssmtp.conf must be setup

# NOTE: Backup file name contains machine name + Distro
#       Same script for user with multiple dual boot laptops
#       Single machine should remove $HOSTNAME from name
#       Single distribution should remove $Distro

sleep 30 # Wait 30 seconds after boot

# Running under WSL (Windows Subsystem for Ubuntu)?
if cat /proc/version | grep Microsoft; then
    Distro="WSL"
else
    Distro="Ubuntu"
fi

today=$( date +%Y-%m-%d-%A )
/mnt/e/bin/daily-backup.sh Daily-$(hostname)-$Distro-backup-$today

Save the file, exit and use:

chmod a+x /etc/cron.daily/daily-backup

This makes the script executable.

What cron emails you every morning

Every morning after /etc/cron.daily/daily-backup is run cron sends you two emails. One is the backup Backup-YYYY-MM-DD.tar file which in my case is 5.2 MB that I cannot show you. The other is a listing off all the files in the backup which the tar command had reported to cron:

Anacron <Me@gmail.com>
6:58 AM (1 hour ago)

to root, bcc: me 
/etc/cron.daily/daily-backup:
bin/
bin/.websync.new
bin/log-gsu-del
bin/now
  (... SNIP ...)
.xscreensaver
.xsession-errors
.xsession-errors.old

Summary

It took a month waiting for an answer and then a month writing an answer but, now the project is finished. Going forward it's simply a matter of adding additional directories to the backup script.

The next project will be a full backup but it is 6 GB large and will be copied to gdrive (Google Drive) because gmail is limited to 25 MB. That script is called /usr/local/bin/full-backup and is included here if you are interested:

#!/bin/bash

# NAME: full-backup
# PATH: $HOME/bin
# DESC: Full system backup - must call with SUDO

# DATE: July 16, 2017. Modified July 26, 2017.

apt autoclean   # reduces size of /var/cache/apt/archives

cd /tmp         # tar must be created in directory not backed up.

time tar -cvpzf backup.tar.gz \
--exclude=/backup.tar.gz \
--exclude=/proc \
--exclude=/tmp \
--exclude=/mnt \
--exclude=/dev \
--exclude=/sys \
--exclude=/media \
--exclude=/usr/src/linux-headers* \
--exclude=/home/Me/.cache \
--exclude=/var/log \
--exclude=/var/run/ \
--exclude=/run \
--exclude=/var/cache/apt/archives /

Historical section

This will be more a "journey" than an answer as available options are explored.

Backup what is most important to you first

I have two directories where I have invested most of my time since August 2016:

/home/rick/bin
/usr/local/bin

When I first created a tar file (tape archive) using these two directories and tried to email them to myself I received this error:

gmail 25MB limit

gmail.com won't accept files > 25 MB

How can two directories of scripts written over 10 months be larger than 25 MB? On closer examination they are are > 190 MB. Whhaaatttt?

In turns out to be a single file created for testing purposes:

-rw-rw-r--  1 rick rick 191143744 Dec 23 17:27 log-gsu-gedit.tst

So delete this test file and rerun the commands:

tar -cvf scripts-2017-06-05.tar /home/rick/bin
tar -rvf scripts-2017-06-05.tar /usr/local/bin

The first command creates the .tar file using one directory of script files and the second appends to the .tar file using the second directory of script files.

The .tar file is now a more respectable size of 1.3 MB:

-rw-rw-r-- 1 rick rick 1341440 Jun  5 17:27 scripts-2017-06-05.tar

The simplest way is to email as an attachment

Now that the .tar file is created, simply go into gmail.com and email the file to yourself as an attachment. In the next step we'll want a cron job that creates the file daily and emails it automatically using MTA (Mail Transport Agent). An option needs to be setup in gmail.com to delete all these emails older than 30 days. That way only 400 MB or so of total scripts backups will be stored.


Edit June 25, 2017

I discovered tonight some configuration files difficult to backup until I stumbled across this thread. The files in question are in my home directory:

.bashrc
.conkyrc
.websync # one of my own databases
.bafman* # Another one of my own databases

Using the link above I created a script called ~/bin/backup with:

#!/bin/bash

if [[ $1 == "" ]] ; then
    echo 'Parameter required for file name, ie "Backup-2017-06-26"'
    echo ".tar will automatically be added as a file extension"
    exit
fi
tar -cvf $1.tar /home/rick/bin
tar -rvf $1.tar /usr/local/bin
find .* -maxdepth 0 -type f -exec tar -rvf $1.tar {} +

To view what is in the .tar archive use the command:

tar -tvf Backup-2017-06-26.tar

Remember to replace "Backup-2017-06-26" with the parameter you used when creating the backup.


Edit July 1, 2017

Similar Q&A was posted in November 2014: Send backup by email with crontab. The accepted answer is included below:

This following command worked for me when I tested in my machine.

echo "This is the message body" | mutt -a "/path/to/file.to.attach" -s "subject of message" -- recipient@domain.com

So probably the approach to follow will be something like,

tar -zcf /home/blah/backup.tgz /home/blah/
echo "Please find attached the backup file" | mutt -a "/home/blah/backup.tgz" -s "File attached" -- recipient@domain.com

I will save the above script as backup_email.sh and schedule the cron job as,

0 1 * * * /path/to/backup_email.sh

References

https://stackoverflow.com/a/9524359/1742825

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.