107.2 Automate system administration tasks by scheduling jobs

Weight: 4

Candidates should be able to use cron and systemd timers to run jobs at regular intervals and to use at to run jobs at a specific time.

Key Knowledge Areas

• Manage cron and at jobs.
• Configure user access to cron and at services.
• Understand systemd timer units.

Terms and Utilities

• /etc/cron.{d,daily,hourly,monthly,weekly}/
• /etc/at.deny
• /etc/at.allow
• /etc/crontab
• /etc/cron.allow
• /etc/cron.deny
• /var/spool/cron/
• crontab
• at
• atq
• atrm
• systemctl
• systemd-run

There are many cases where we need to run some commands at specific times or intervals. For example you may want to run your backup process every morning at 03:00 or refresh the list of files on your web index on hourly basis.

There are different methods to do so in the Linux world. The most commons are the crontab to run tasks on specific intervals, at to run a commands on specific time and also utilities provided by systemd.

Crontab format

Crontab works with a configuration file which tells it when to run a command. For example you can say "run my commands when the hour is 03 and the minute is 00" or "run my command when its Saturday and Hour is 10 and minute is 00" or "Run whenever the minute is 00".

Each crontab configuration line has 6 fields. The first 5 specify a specific interval and whatever after than is used as the "command".

A    B    C    D    E    command and arguments

If you replace a field with * it means "whatever" or "all" or "any". Also if you have @reboot or @daily instead of time fields, the command will be run once after the reboot or daily. Lets see some examples:

5 0 * * *       $HOME/bin/daily.job >> $HOME/tmp/out 2>&1

# run at 2:15pm on the first of every month -- output mailed to paul
15 14 1 * *     $HOME/bin/monthly

# run at 10 pm on weekdays, annoy Joe
0 22 * * 1-5    mail -s "It's 10pm" joe%Joe,%%Where are your kids?%

23 0-23/2 * * * echo "run 23 minutes after midn, 2am, 4am ..., everyday"
5 4 * * sun     echo "run at 5 after 4 every sunday"

*/5 * * * *    echo "each 5 mintues"

42 8,18 * * 1-5    echo "8:42 and 18:42 and only on weekdays (monday till friday)"

@reboot        echo "runs after the reboot"

As you can see in above examples, / can be used to slice the time (so */5 in minutes means every five minutes) or use the , to indicate more than one number (so 8,18 means 8 and 18).

When using * on the first filed, you are running your command on every minute.

Something like 42 8 1 1 0 runs ONLY when the 1st of Jan is a Sunday!

When a cron runs, the output will be emailed to the owner of the cron.

user specific crons

Cron is a linux service. To see your crons you can use crontab -l (list) and for editing them you can use crontab -e (edit) which will open the cron files with a special editor and will validate and then load your crons after you are finished.

The files will be saved at /var/spool/cron/tabs/ or /var/spool/crontabs:

# cat /var/spool/cron/tabs/jadi
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.khObLu installed on Thu Oct 29 22:04:43 2023)
# (Cronie version 4.2)
# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * command to be executed
* * * * * date >> /tmp/date.cron.txt

never edit these files directly; Use crontab -e instead.

system wide cron

There is file called /etc/crontab. This looks like a normal user file opened with crontab -e but has one extra filed:

A    B    C    D    E    USER    command and arguments

This file should be edited with an editor directly and we can mention which user runs this commands.

# cat /etc/crontab
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * *   root  test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1

Note: Have a look at the first three lines. It configures the shell which will run the commands and the PATH variable plus who will get the output emails.

As you can see this default crontab runs other crons! These are hourly, daily, weekly and monthly crons. Lets have a look at them in the next section.

System hourly, daliy, weekly, monthly, .. crons

We have some system level crontab files in /etc/cron.d/ too. They look like general crontab files. For example this is one that munin program creates:

$ cat /etc/cron.d/munin
#
# cron-jobs for munin
#

MAILTO=root

*/5 * * * *     munin if [ -x /usr/bin/munin-cron ]; then /usr/bin/munin-cron; fi
14 10 * * *     munin if [ -x /usr/share/munin/munin-limits ]; then /usr/share/munin/munin-limits --force --contact nagios --contact old-nagios; fi
19:12:37 jadi@ocean:~$ cat /etc/cron.weekly/man-db

But there are also directories called cron.hourly, cron.daily, cron.weekly & cron.monthly. As their name indicates, whatever script / file you put in them, will be run hourly, daily, weekly & monthly. This makes general cleanup, backup, ... tasks much cleaner. Its enough to copy your script in any of these directories and let them run on these intervals.

$ sudo tree /etc/cron*
/etc/cron.d
├── munin
├── munin-node
├── php5
└── sendmail
/etc/cron.daily
├── apache2
├── apport
├── apt
├── aptitude
├── bsdmainutils
├── dpkg
├── logrotate
├── man-db
├── mlocate
├── passwd
├── popularity-contest
├── sendmail
├── update-notifier-common
└── upstart
/etc/cron.hourly
/etc/cron.monthly
/etc/crontab [error opening dir]
/etc/cron.weekly
├── apt-xapian-index
├── fstrim
├── man-db
└── update-notifier-common

Lets have a look at one of the cron.d files:

$ cat /etc/cron.daily/mlocate
#! /bin/bash

set -e

[ -x /usr/bin/updatedb.mlocate ] || exit 0

if which on_ac_power >/dev/null 2>&1; then
    ON_BATTERY=0
    on_ac_power >/dev/null 2>&1 || ON_BATTERY=$?
    if [ "$ON_BATTERY" -eq 1 ]; then
    exit 0
    fi
fi

# See ionice(1)
if [ -x /usr/bin/ionice ] &&
    /usr/bin/ionice -c3 true 2>/dev/null; then
    IONICE="/usr/bin/ionice -c3"
fi

flock --nonblock /run/mlocate.daily.lock $IONICE /usr/bin/updatedb.mlocate

at

We say that crontab runs commands on specific intervals but what will happen if you needed to run a command only once? Here at is your friend.

$ at now + 1 min
warning: commands will be executed using /bin/sh
at> touch /tmp/at
at> <EOT>
job 3 at Thu Oct 29 22:12:00 2015

Note: As always, at the end of input we press Ctrl+D

If you want to check what is in the queue you can use atq and then try atrm 4 to delete job number 4;

$ at teatime
warning: commands will be executed using /bin/sh
at> echo "tea time is 4pm"
at> <EOT>
job 4 at Fri Oct 30 16:00:00 2015
jadi@funlife:~$ at tomorrow
warning: commands will be executed using /bin/sh
at> echo "tomorrow this time"
at> <EOT>
job 5 at Fri Oct 30 22:15:00 2015
jadi@funlife:~$ at 17:30
warning: commands will be executed using /bin/sh
at> echo "this is the first 17:30"
at> <EOT>
job 6 at Fri Oct 30 17:30:00 2015
jadi@funlife:~$ atq
5    Fri Oct 30 22:15:00 2015 a jadi
4    Fri Oct 30 16:00:00 2015 a jadi
6    Fri Oct 30 17:30:00 2015 a jadi
jadi@funlife:~$ atrm 4
jadi@funlife:~$ atq
5    Fri Oct 30 22:15:00 2015 a jadi
6    Fri Oct 30 17:30:00 2015 a jadi

managing cron access

To control the access to the cron system, we have 4 files:

/etc/cron.allow
/etc/cron.deny

/etc/at.allow
/etc/at.deny

In most systems none of these files exist but if you create them, they will become active as follow:

systemd & systemd-run

Nowadays most GNU/Linux distributions use systemd which has its own timers as an alternative to cron to schedule your tasks. Timer unit files use the .timer suffix, and for each of these there must be a corresponding unit file which describes the unit to be activated when the timer activates. By default, a timer activates a service with the same name, except for the suffix.

Timers have a [Timer] section that specifies when scheduled jobs should run. There are two types of timers. One is the readl-time timer which works with OnCalendar= option and defines kind of close to cron timers (based on calendar event expressions). This is the syntax:

OnCalendar=DayOfWeek Year-Month-Day Hour:Minute:Second

DayOfWeek is optional and you can use a 3 letter abbreviation (say Mon)

The *, / and , work the same in systemd and cron configurations but in systemd you can also use .. between two values to indicate a contiguous range.

For example, to run the service named /etc/systemd/system/myservice.service at 02:42 on the first Saturday of each month, this config line should be added to the /etc/systemd/system/myservice.timer unit file.

[Unit]
Description=Run the blah blah service

[Timer]
OnCalendar=Sat *-*-1..7 02:42:00
Persistent=true

[Install]
WantedBy=timers.target

It is also possible to define monotonic timers. They activate after some time has elapsed from a specific start point (say when the machine was booted up or when the timer itself is activated). You can chose between any of the following:

And you can use any of usec, msec, seconds, minutes, hours, days, weeks, months, years in front. So for example the

[Timer]
OnUnitActiveSec=5minutes 20seconds

will activate the corresponding unit 5min and 20secs after the last activation of this unit service.

Please note that after adding any timer, you need to enable (and probably start) it:

# systemctl enable myservice.timer
# systemctl start myservice.timer

And also do not forget to do a systemctl daemon-reload after each change.

To check all the scheduled jobs, run the systemctl list-timers command. This will show you all the active timers. Use the --all switch to check all.

There are few shortcuts for times too. You can use these instead of the cron format to set timers:

This is one example from a recent Fedora installation:

# cat system/timers.target.wants/plocate-updatedb.timer 
[Unit]
Description=Update the plocate database daily

[Timer]
OnCalendar=daily
RandomizedDelaySec=12h
AccuracySec=20min
Persistent=true

[Install]
WantedBy=timers.target

If you want to check the output of your timers, you can use the journalctl.

as any other systemd activity, use the --user switch to use it on your user level and not systemwide.

At the end, you should know about the systemd-run command. It can start a transient service or scope and run specific commands in it.

So a command like:

systemd-run --on-active="2hours" --unit="helloworld.service"

Will run the helloworld.service every 2 hours or this:

sudo systemd-run --on-active="2hours" /usr/local/bin/helloworld.sh --language=en_US

will run that specific script with shown parameters every 2 hours.

Since this is new and there are not much sources about it, refer to Suse docs to learn more.