Runner scripts

Bert Desmet blogs about having to run a script more than once per minute, and using a runner script (started from cron) to do that. Unfortunately, his implementation contains a race condition:

if ! ps -p $(cat /path/to/runner.pid);
  then
    echo $$ > /path/to/runner.pid

If the script is accidentally started twice on approximately the same moment, chances are pretty high you'll see them both executing the ps call at approximately the same time, concluding that no other script is running, and then both executing the echo call at the same time. Result: your script will be running twice, yet you'll have only one pid file.

Of course, if you do indeed run the script only from cron, and just once every 30 minutes, as in Bert's case, chances that you'll hit this race condition are pretty low. However, race conditions are bad, especially if they're easy to avoid.

set -o noclobber
if ! echo $$ > /var/run/runner.pid
then
  echo pidfile exists; exiting
  exit 0
fi

This code uses the noclobber shell option, which causes the shell to issue an error state if the file we try to overwrite with > already exists, and to do so in an atomic operation (so, no race condition there: either the file exists and we fail, or the file does not exist yet and we create it with our PID). Of course, that leaves out the check for is the other process running? Adding a second check for that is fairly complex to do right, as this cannot be done in an atomic operation; so it's usually better use exit traps to clean up the file:

set -o noclobber
if ! echo $$ > /var/run/runner.pid
then
  echo pidfile exists; exiting
  exit 0
fi
trap EXIT rm -f /var/run/runner.pid

This will fail if the system dies when the script is running, but then since we use /var/run (which is cleared on bootup, either because it's a tmpfs or because the initscripts clean it) this means the file should only ever be stale if some genius kills our process with kill -9 or kill -11.