Generic solution to prevent a long cron job from running in parallel?

Question

I am looking for a simple and generic solution that would allow you to execute any script or application in crontab and prevent it from running twice.

The solution should be independent on the executed command.

I assume it should look like lock && (command ; unlock) where lock will return false if there was another lock.

The second part would be like if it acquired the lock, run command and unlock after command is executed, even if it returns error.

Answer 1

Take a look at the run-one package. From the manpage for the run-one command :

run-one is a wrapper script that runs no more than one unique instance of some command with a unique set of arguments.

This is often useful with cronjobs, when you want no more than one copy running at a time.

Like time or sudo, you just prepend it to the command. So a cronjob could look like:

  */60 * * * *   run-one rsync -azP $HOME example.com:/srv/backup

For more information and background, check out the blog post introducing it by Dustin Kirkland.

Answer 2

A very simple way of settup a lock:

if mkdir /var/lock/mylock; then
  echo "Locking succeeded" >&2
else
  echo "Lock failed - exit" >&2
  exit 1
fi

A scripts which want to run needs te create the lock. If the lock exists, another script is busy, so the first script can't run. If the file don't exists, no script has acquired the lock. So the current script acquires the lock. When the script has finished the lock needs to be realeased by removeing the lock.

For more information about bash locks, check this page

Answer 3

No need to install some fancy package:

#!/bin/bash
pgrep -xf "$*" > /dev/null || "$@"

It's faster to write that script yourself than to run "apt-get install", isn't it? You might want to add "-u $(id -u)" to the pgrep to check for instances run by the current user only.

Answer 4

See also Tim Kay's solo, which performs locking by binding a port on a loopback address unique to the user:

http://timkay.com/solo/

In case his site goes down:

Usage:

solo -port=PORT COMMAND

where
    PORT        some arbitrary port number to be used for locking
    COMMAND     shell command to run

options
    -verbose    be verbose
    -silent     be silent

Use it like this:

* * * * * solo -port=3801 ./job.pl blah blah

Script:

#!/usr/bin/perl -s
#
# solo v1.7
# Prevents multiple cron instances from running simultaneously.
#
# Copyright 2007-2016 Timothy Kay
# http://timkay.com/solo/
#
# It is free software; you can redistribute it and/or modify it under the terms of either:
#
# a) the GNU General Public License as published by the Free Software Foundation;
#    either version 1 (http://dev.perl.org/licenses/gpl1.html), or (at your option)
#    any later version (http://www.fsf.org/licenses/licenses.html#GNUGPL), or
#
# b) the "Artistic License" (http://dev.perl.org/licenses/artistic.html), or
#
# c) the MIT License (http://opensource.org/licenses/MIT)
#

use Socket;

alarm $timeout                              if $timeout;

$port =~ /^\d+$/ or $noport                     or die "Usage: $0 -port=PORT COMMAND\n";

if ($port)
{
    # To work with OpenBSD: change to
    # $addr = pack(CnC, 127, 0, 1);
    # but make sure to use different ports across different users.
    # (Thanks to  www.gotati.com .)
    $addr = pack(CnC, 127, $<, 1);
    print "solo: bind ", join(".", unpack(C4, $addr)), ":$port\n"   if $verbose;

    $^F = 10;           # unset close-on-exec

    socket(SOLO, PF_INET, SOCK_STREAM, getprotobyname('tcp'))       or die "socket: $!";
    bind(SOLO, sockaddr_in($port, $addr))               or $silent? exit: die "solo($port): $!\n";
}

sleep $sleep if $sleep;

exec @ARGV;

Answer 5

You need a lock. run-one does the job, but you may also want to look into flock from util-linux package.

It is a standard package provided by the kernel developers, allows for more customization than run-one and is still very simple.

Answer 6

A simple solution from bash-hackers.org that worked for me was using mkdir. This is an easy way how to make sure that only one instance of your program is running. Create a directory with mkdir .lock which returns

• true if the creation was successful and
• false if the lock file exists, indicating that there is currently one instance running.

So this simple function did all the file locking logic:

if mkdir .lock; then
    echo "Locking succeeded"
    eval startYourProgram.sh ;
else
    echo "Lock file exists. Program already running? Exit. "
    exit 1
fi


echo "Program finished, Removing lock."
rm -r .lock

Answer 7

This solution is for a bash script that needs to check itself

v=$(pgrep -xf "/bin/bash $0 $@")
[ "${v/$BASHPID/}" != "" ] && exit 0