How to write a recuring bash script

Question

I created a program that requires a sequence of command line instructions to scrape new information and update the databases.

Is there a way to automate this process so I don't have to type all this stuff in the terminal each time I sit down? Maybe have it so the first 3 steps run every hour and step 4 runs at 2am as it can take a long time to complete.

Step 1: In terminal:

...Documents/dota2/dotaapi2/ python match_scraper.py 

Step 2: In terminal

 psql dota2apidb
 update games set online=1 where online is null;

Step 3: In terminal:

...Documents/dota2/gosugamers/gosugamers/spiders/ scrapy crawl dota

Step 4: In terminal: (execute once at 2am only)

...Documents/dota2/dotaapi2/ python gosu_merge.py

...Documents/dota2/dotaapi2/ python unticketed.py

Answer 1

One solution could be setting up cronjobs to lock/unlock the game directory. To do so, follow the steps below.

1. Make sure cron-daemon is installed
   
   

   sudo apt-get install cron
   

2. Setting up the cronjob to run for root

Start editing the cronjob by issuing following command:

sudo crontab -e

Add the folowing lines into the editor:

00 */1 * * * ...Documents/dota2/dotaapi2/ python match_scraper.py
01 */1 * * * psql dota2apidb
02 */1 * * * update games set online=1 where online is null;
03 */1 * * * ...Documents/dota2/gosugamers/gosugamers/spiders/ scrapy crawl dota
04 02 * * * ...Documents/dota2/dotaapi2/ python gosu_merge.py
05 02 * * * ...Documents/dota2/dotaapi2/ python unticketed.py

(Just to make clear you have to use proper commands and paths maybeputting some of that in prepared bash scripts so you can reduce the amount of crons running)

Answer 2

put steps 1,2,3 into script1.sh put step 4 into script2.sh

Then setup cron for the appropriate user to execute the scripts according to your schedule:

minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday), command

    01 * * * * /usr/bin/directory/script1.sh
    30 2 * * * /usr/bin/directory/script2.sh

script1.sh will be executed at minute 01 if every hour of every day while script2.sh will be run at minute 30 of hour 0200 everyday.