0

I have a simple bash script that executes a rsync command to sync a large number of files (4TB+) from a remote server to my local machine. Using crontab -e I have set up a schedule for this script to run at a set time each day.

The crontab entry looks like this: 0 0 * * * bash /home/scripts/backup.sh

So I'm executing the script via bash. Three questions:

  1. Is there anything in this execution pipeline that would timeout? Does RSYNC, BASH or CRON timeout? The rsync command could take many hours to execute.

  2. Does CRON prevent script crossover, i.e. when the script is still running when the next schedule is due to execute? Is there a command or option for that?

  3. Is there an execution log (from rsync, cron or bash) that allows me to check progress, diagnose any errors, etc.?

1

Yes, of course, even without seeing the script, "`rsync command to sync a large number of files (4TB+) from a remote server to my local machine." can timeout.

No, there is no cron feature to control overlapping executions. It's left up to you. See man flock, man lockfile, man dotlockfile.

Your script can log anything it wants. Your script's STDOUT is emailled to $USER locally. If emailling fails, a file called dead.letter will be created. I don't recall where, but after running

sudo /etc/cron.daily/mlocate

a simple locate dead.letter should find it. (BTW, /etc/cron.daily/mlocate uses flock. Take a look at it).

journalctl $(type -p cron) will show you system logs about cron.

Now, a word about using AskUbuntu. Please read https://discourse.ubuntu.com/t/ask-ubuntu/21 and https://askubuntu.com/help/how-to-ask , and only ask a single question at a time - it helps later searchers.

| improve this answer | |
  • Thanks for your answer, some good info in there. I'm coming from a Windows background where I use robocopy successfully with no timeout, even if it takes days to complete. It just plods on happily forever until it's finished. Will post one question at a time from now on. – SimonGoldstone Aug 18 '19 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.