How to monitor cron jobs and catch missed runs

Set a deadline for each cron job, send a heartbeat after a successful backup, and detect missing completion logs without confusing reporting failures with job failures.

Best practices
Illustration of a monitoring platform with connected services

The database can answer every health probe while its nightly backup has stopped running. A failed dump may leave an error in the job log. A run that never started leaves no such error.

Cron job monitoring gives each expected run a deadline and looks for evidence of completion. With a success heartbeat, the job sends a request after it has met its success criteria. If that request is missing when the deadline passes, the monitor can alert. A missing signal gives you two paths to investigate: the job and the route its report takes.

Put the deadline on the calendar

Suppose the backup starts at 02:00 UTC, can reasonably finish by 02:15, and gets five more minutes for late starts or delivery. A schedule-aware monitor should expect the success signal by 02:20 UTC. Check which time zone the scheduler and monitor use, and account for daylight-saving transitions if either follows local time.

A fixed-interval monitor measures the time since the last received heartbeat. That works for jobs with regular spacing. The deadline can move if yesterday's run finished late, and a delayed retry can mask a missed slot. If every scheduled run matters, keep its scheduled time or run ID with the completion record. Decide how retries and backfills count before paging anyone. The job's completion deadline, the monitor's evaluation cadence, and the time an alert reaches you are separate. Schedule and grace time in Healthchecks.io

Report success after the work succeeds

In this PostgreSQL example, success means the dump and its upload both returned successfully. The script needs Bash, PostgreSQL client tools, the AWS CLI, and curl. Configure database and S3 access for the account that runs cron. Set BACKUP_S3_PREFIX to a destination such as s3://example-backups/postgres and HEARTBEAT_URL to the final HTTPS endpoint that accepts the success GET request.

#!/usr/bin/env bash
set -eu

backup_prefix="${BACKUP_S3_PREFIX:?Set BACKUP_S3_PREFIX to an S3 URI}"
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
archive="$(mktemp "${TMPDIR:-/tmp}/mydb.XXXXXX")"
trap 'rm -f -- "$archive" || :' EXIT

pg_dump --format=custom --file="$archive" mydb
aws s3 cp "$archive" "${backup_prefix%/}/${stamp}-${archive##*/}.dump" --only-show-errors
printf '%s\n' 'job=db_backup result=success'

if [[ -z ${HEARTBEAT_URL:-} ]]; then
    printf '%s\n' 'Backup succeeded, heartbeat URL is missing.' >&2
elif ! http_code="$(curl -q --silent --output /dev/null --write-out '%{http_code}' \
    --proto '=https' --connect-timeout 3 --max-time 10 -- "$HEARTBEAT_URL")" \
    || [[ $http_code != 2[0-9][0-9] ]]; then
    printf '%s\n' 'Backup succeeded, heartbeat delivery was not confirmed.' >&2
fi

A failed dump or upload exits before either success signal. If only the heartbeat request fails or returns something outside 2xx, the script warns on stderr and preserves the backup's successful exit status. The time limits keep an unavailable monitor from tying up the job. Treat the heartbeat URL as a credential and keep it out of logs and command tracing. Verify exit handling again if you wrap the script in a conditional or change the backup into a pipeline.

This example checks that both commands reported success. A separate restore into a test database can check whether the archive is usable. For other jobs, completion can include checks such as whether an expected input was processed or an output was published. Record useful counts and duration with the run so a late or empty result is visible. Bash exit behavior · curl options · PostgreSQL restore

Read the scheduler and the job together

Check the scheduler for a start, the job logs for its outcome, and the heartbeat receiver for delivery. The three records help separate a missed schedule, a failed run, and a reporting problem. The same distinction applies to systemd timers, Airflow workflows, and Windows scheduled tasks when you can report their actual outcomes.

A Kubernetes CronJob's lastScheduleTime records scheduling, while lastSuccessfulTime records the most recent successful Job completion. Compare that success time with the slot you expected. A previous success can still be present after the current slot is missed. For a multi-Pod Job, use its completion criteria and status to judge the whole run. CronJob scheduling can create duplicate Jobs or miss a run, so keep the work idempotent and preserve useful history beyond the Job retention window when needed. CronJob behavior · CronJob status fields

Alert on a missing completion record

When the job already writes logs, collect a completion record with the task name, outcome and run identifier. In Guance Log Detection, filter to the task and its success record. Keep a run ID in the record for investigation, and avoid using it as a detection dimension. The Data Gaps setting defaults to creating no event when the query is empty. For missing-success alerts, explicitly choose to treat an empty result as zero with an appropriate trigger threshold, or configure a data-gap event.

Set the detection range and evaluation cadence around the expected spacing of successful records and their arrival delay. A rolling range checks for a recent success. To validate a specific slot, check its scheduled time or run ID as well. Exercise a normal run, a missed run and an ingestion interruption separately. A service can be reachable while any of these checks fail, which is why the job needs its own signal alongside uptime monitoring. Guance Log Detection

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting