How to avoid killing the wrong process in a bash script?

Question

I'm writing a bash script that spawns a couple of child processes, and will be run by a cron job every hour. It's actually running an rsync command to a remote server, and the associated ssh connection.

If the rsync command or script takes too long I'd like to kill it and its child processes, but give it a grace period first. The script runs every hour, but the next 2 attempts will see that a previous instance of the script is running and exit before trying to commence a transfer. The third subsequent attempt will kill the original, still running instance, before starting a new transfer.

I've decided to control this by writing to a PID file, first instance writes its PID to the file, the next two attempts write their PID to the file before exiting. The third attempt kills the first one, then overwrites the PID file with the new PID. Any successful completion wipes the PID file. To kill all children I've decided to use the PGID of the original script instance which I get with:

previous_pid=$(head -n1 "$pid_file")
previous_groupid=$(ps -hp "$previous_pid" -o pgid:1)

Then kill it with:

kill -- -"$previous_pid"

My concern is that there may be a condition where the original script has somehow died without cleaning up the PID file, and that PID is being reused also reusing the PGID, so I'll end up killing a completely unrelated process. How can I avoid this?

Answer 1

Get the script to kill itself. For example, have something like this at the start of your script:

kill_myself () {
    while sleep 1
    do
        if [[ -f /tmp/kill-myself ]]
        then
            rm /tmp/kill-myself
            kill -- -$1
        fi
    done
}

kill_myself $$ &

Now, when the time comes:

touch /tmp/kill-myself

You could customize this, so that each instance of the script has its own kill file, which you would log somewhere. Then you'll use that file to kill that specific instance of the script.

---

Another way: You could probably use pgrep/pkill for finer matching. They can read from a pid file, match against the command name and arguments, etc.

Something like the following will only kill processes that have the same name as your script (assuming you invoked it as script-name or /path/to/script-name):

pkill -g "$previous_groupid" script-name

This won't work if you do bash script-name. Or just to kill the rsync command, which should probably get your script to exit after that:

pkill -g "$previous_groupid" rsync