Cron Scheduler

What it is

Cron is the classic tool for running a job again and again on a fixed schedule: "every 15 minutes", "every Monday at 9am", "the 1st of every month". A scheduler has two separate jobs. First, work out when a schedule should fire next. Second, wait and fire the job at that time, without drifting late and without piling up fires if one job runs long. Keeping these two jobs separate makes each one easier to test on its own.

The schedule format

A schedule has 5 fields, in order: minute, hour, day of the month, month, day of the week. Each field is one of:

• *: any value
• a number, like 9
• a range, like 8-9
• a list, like 1,15
• a step, like */15 (every 15 units)

A few examples:

• /15 * * * fires every 15 minutes: at :00, :15, :30, and :45 of every hour.
• 0 9 * * 1 fires at 9:00 every Monday (day of week: Sunday is 0, Monday is 1, up to Saturday 6).
• 30 14 5 * * fires at 14:30 on the 5th of every month.

If a schedule limits both the day of the month and the day of the week, a day counts if either one matches, not only if both do. 0 0 13 * 5 means the 13th of the month OR any Friday, not only Friday the 13th. This rule comes from the original Unix cron tool (sometimes called Vixie cron, after its best-known author).

Working out the next fire time

Jump straight to the next value a field can take instead of checking every single minute one at a time. If a schedule is once a week, checking minute by minute means thousands of checks. Jumping field by field means a handful.

Firing jobs on time

For a repeating job, work out each fire's time as the start time plus a whole number of periods, not by waiting the period again after the last job finished. Waiting again after each job means a slow job pushes every later fire later too, so the schedule drifts. If a job overruns its slot, skip the ticks it missed instead of firing a burst of catch-up jobs once it is free again. Stop firing as soon as the schedule is cancelled.

A fast clock for one stage

Real schedules run for hours, but a test can't wait that long. Most stages below use short waits instead, so your code runs for real, just for a few hundred milliseconds. One stage runs on a server-controlled clock that skips idle waiting, so an hour of schedule is checked in seconds. Your code doesn't change for either case: it just experiences time passing at a different speed while it waits.

Where you meet this

• cron itself: the crontab manual defines the 5 fields, and the rule that when both day fields are set, a day matches if either one matches. (crontab(5))
• Kubernetes CronJobs use the cron format. startingDeadlineSeconds skips a run that could not start in time, and concurrencyPolicy: Forbid skips a new run while the last one is still going. (Kubernetes CronJob)
• systemd timers use OnCalendar= for calendar times. With Persistent=true, a run missed while the timer was off happens once, right away, when it comes back. (systemd.timer(5))

When to choose it: work that belongs to the calendar, like "every night at 2:00" or "every Monday at 9".

When not to: "run again 30 seconds after the last run finished" is a plain loop with a wait, not a calendar rule. Also decide what a missed run means for you: skip it (Kubernetes with startingDeadlineSeconds), or run it once to catch up (systemd Persistent=true). Running every missed tick is almost never right.

A scheduler only decides when to start work. If the work must not be lost when a worker crashes, hand it to a Task Queue.

Common bugs

• Requiring both the day of the month and the day of the week to match, instead of either one.
• Checking every minute one at a time for a schedule that is far away.
• Mishandling February 29, or a month that doesn't have the day a schedule asks for.
• Letting a job's own running time add drift to every fire after it.
• Firing a burst of catch-up jobs after a slow job, instead of skipping the ticks it missed.
• Still firing after the schedule was cancelled.
• Checking the clock in a tight loop instead of waiting on a timer, which burns time and can't go idle on the fast clock.

Check it yourself

Walk through a calendar by hand for month and year changes, especially February and the 31st. Try a job that takes longer than its own period, and watch what happens right after it. After cancelling a schedule, at most one job already in progress should still finish.

Learn more

The crontab manual explains the 5-field format and the either-day rule in full. Kubernetes' CronJob documentation shows the same split between working out the next run and actually running it, at a larger scale.

Go to the Cron Scheduler challenge