0

I'm trying to setup rsync to keep an image on the my external drive current. I've got 4 partitions to deal with: /, /boot, /home, /usr. I want to mount each one in turn and sync each with the internal drive. One problem I'm having is that excluded files are not being excluded. I've experimented with a few patterns but none do the trick. I'm setting the exclude pattern relative to the source but it's not working.

This log snippet of the /home/ sync shows the excluded files being copied:


    --begin snippet--
    backing up /home/ via rsync to: /media/Ubuntu_Backup...
    sudo rsync -a -x -i -h -v -A -X -H --delete --inplace --numeric-ids --compare-dest=./ -n --exclude='*/lost+found/**' --exclude='*/Downloads/**' --exclude='*/.cache/**' --exclude='*/.thumbnails' --exclude='*/.mozilla/firefox/*.default/Cache' /home/ /media/Ubuntu_Backup
    sending incremental file list
    .d..t...... kmiller/
    >f..t...... kmiller/.vim_mru_files
    >f.st...... kmiller/.viminfo
    >f.st...... kmiller/.zsh_history
    >f.st...... kmiller/backup.log
    >f.st...... kmiller/cron.log
    .d..t...... kmiller/.cache/google-chrome-beta/Default/Cache/
    *deleting   kmiller/.cache/google-chrome-beta/Default/Cache/f_000806
    >f..t...... kmiller/.cache/google-chrome-beta/Default/Cache/data_0
    >f..t...... kmiller/.cache/google-chrome-beta/Default/Cache/data_1
    >f..t...... kmiller/.cache/google-chrome-beta/Default/Cache/data_2
    >f..t...... kmiller/.cache/google-chrome-beta/Default/Cache/data_3
    >f+++++++++ kmiller/.cache/google-chrome-beta/Default/Cache/f_00080a
    >f+++++++++ kmiller/.cache/google-chrome-beta/Default/Cache/f_00080b
    >f+++++++++ kmiller/.cache/google-chrome-beta/Default/Cache/f_00080c
    >f+++++++++ kmiller/.cache/google-chrome-beta/Default/Cache/f_00080d
    >f+++++++++ kmiller/.cache/google-chrome-beta/Default/Cache/f_00080e
    --end of snippet--

The other problem I was having is that rsync was creating a duplcate directory structure within the target dir i.e. a /usr/ dir inside the /usr on the target. The --compare-dest=./ seemed to stop that but I don't fully understand why.

This is the script I'm developing to do the job:


#!/usr/bin/zsh
typeset -A FILESYSTEMS
typeset -A NAMES

MOUNTPOINT='/media/Ubuntu_Backup'
LOGFILE='/home/kmiller/backup.log'

OPTIONS=('-a') #archive
OPTIONS+=('-x') #don't cross file system boundaries
OPTIONS+=('-i') #itemize changes
OPTIONS+=('-h') #human readable output
OPTIONS+=('-v') #verbose
OPTIONS+=('-A') #preserve ACLS
OPTIONS+=('-X') #preserve extended attributes
OPTIONS+=('-H') #preserve hard links
OPTIONS+=('--delete') #delete extraneous
OPTIONS+=('--inplace') #update files directly (faster - no building temp copies)
OPTIONS+=('--numeric-ids') #no translation of UID,GID - needed for proper restores
OPTIONS+=('--compare-dest=./') #
OPTIONS+=('-n') #dry run

FILESYSTEMS=('/' '/dev/sdb6')
FILESYSTEMS+=('/boot/' '/dev/sdb1')
FILESYSTEMS+=('/usr/' '/dev/sdb8')
FILESYSTEMS+=('/home/' '/dev/sdb7')

ROOTEXCLUDES=("--exclude='boot/*'")
ROOTEXCLUDES+=("--exclude='usr/*'")
ROOTEXCLUDES+=("--exclude='home/*'")
ROOTEXCLUDES+=("--exclude='dev/*'")
ROOTEXCLUDES+=("--exclude='proc/*'")
ROOTEXCLUDES+=("--exclude='sys/*'")
ROOTEXCLUDES+=("--exclude='tmp/*'")
ROOTEXCLUDES+=("--exclude='run/*'")
ROOTEXCLUDES+=("--exclude='mnt/*'")
ROOTEXCLUDES+=("--exclude='media/*'")
ROOTEXCLUDES+=("--exclude='lost+found/*'")
ROOTEXCLUDES+=("--exclude='etc/fstab'")

BOOTEXCLUDES=("--exclude='lost+found/*'")
BOOTEXCLUDES+=("--exclude='grub/grub.cfg'")

USREXCLUDES=("--exclude='lost+found/*'")

HOMEEXCLUDES=("--exclude='*/lost+found/*'")
HOMEEXCLUDES+=("--exclude='*/Downloads/*'")
HOMEEXCLUDES+=("--exclude='*/.cache'")
HOMEEXCLUDES+=("--exclude='*/.thumbnails'")

NAMES=('/' 'ROOT' '/boot/' 'BOOT' '/usr/' 'USR' '/home/' 'HOME')

is_mounted () {
    mount | grep "$1" || return 1
    return 0
}

if [[ ! -d $MOUNTPOINT ]];then
    echo "creating $MOUNTPOINT..."
    sudo mkdir $MOUNTPOINT || {echo "$0 failed - Problem creating mount dir: $MOUNTPOINT" && exit;}
fi

rm -rf $LOGFILE

#unmount any target backup partitons if they happen to be mounted
for FS in ${(k)FILESYSTEMS};do
    if is_mounted ${FILESYSTEMS[$FS]};then
        sudo umount ${FILESYSTEMS[$FS]} || {echo "$0 - Problem unmounting: $MOUNTPOINT" && exit;}
    fi
done

echo -e "\nStarting partition backups..."
for FS in ${(k)FILESYSTEMS};do
    EXCLUDES=$(eval echo \$${NAMES[${FS}]}EXCLUDES)
    echo "-------------------------------------------------------"
    echo "mounting ${FS} partition: ${FILESYSTEMS[$FS]}..."
    sudo mount ${FILESYSTEMS[$FS]} $MOUNTPOINT || {echo "$0 - Problem mounting: ${FILESYSTEMS[$FS]} $MOUNTPOINT" && exit;}
    echo "backing up ${FS} via rsync to: $MOUNTPOINT..." | tee -a $LOGFILE 2>&1
    echo "sudo rsync ${OPTIONS} ${EXCLUDES} ${FS} /media/Ubuntu_Backup" >> ${LOGFILE} 2>&1
    eval sudo rsync ${OPTIONS} ${EXCLUDES} ${FS} /media/Ubuntu_Backup >> ${LOGFILE} 2>&1 || {echo -e "WARNING: rsync returned an error: $rc\nSee backup.out for details!" && exit;}
    df ${FILESYSTEMS[$FS]} | tee -a $LOGFILE 2>&1
    echo "unmounting ${FILESYSTEMS[$FS]}"
    sleep 2
    sudo umount ${FILESYSTEMS[$FS]} || {echo "$0 failed - Problem unmounting partition: ${FILESYSTEMS[$FS]}" && exit;}
done
echo -e "Finished partition backups...\n"

Any advice/insights are welcome...

1

After doing more testing, I realized that if I executed the rsync command generated in the script at the command line, it works perfectly. Therefore, I could only assume there is some issue with expansion or interpretation within the scripting environment that I don't fully understand. The fix is to eval the rsync command and I edited the original post to reflect that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.