What is a Prometheus rule?

Use recording rules to precompute PromQL results and alerting rules to detect sustained conditions, then validate and load the rule files.

Best practices
Metrics monitoring and troubleshooting

A Prometheus rule is a periodically evaluated PromQL expression defined in a YAML rule file. A recording rule stores its result as a new metric. An alerting rule tracks matching label sets and can send firing alerts to Alertmanager.

Recording rules precompute results

groups:
  - name: precompute
    interval: 30s
    rules:
      - record: job:http_requests:rate5m
        expr: sum by (job) (rate(http_requests_total[5m]))

Dashboards can query job:http_requests:rate5m instead of repeatedly evaluating the aggregation. This can reduce recurring query work, but adds scheduled evaluation and storage cost. The recorded result retains job; dimensions removed by the aggregation cannot be recovered from that new metric.

Alerting rules detect conditions

groups:
  - name: alerts
    rules:
      - alert: HighErrorRate
        expr: sum(rate(http_errors_total[5m])) / sum(rate(http_requests_total[5m])) > 0.05
        for: 10m
        labels:
          severity: critical
        annotations:
          summary: 'Error ratio exceeds 5% ({{ $value | humanizePercentage }})'

This example assumes the two counters describe the same request population and that the error counter counts a subset of those requests. A zero request rate or missing error series needs deliberate handling for your instrumentation.

for: 10m requires the same alert instance to remain active across evaluations for ten minutes before it fires. Labels support routing and grouping; annotations carry explanatory text and templates. Prometheus must also be configured to reach Alertmanager for notification delivery.

Validate and load the files

Configure the rule-file paths in prometheus.yml:

rule_files:
  - /etc/prometheus/rules/*.yml

Validate a file before loading it:

promtool check rules /etc/prometheus/rules/application.yml

Rules run at their group's interval or inherit the global evaluation interval. Reload the configuration using your deployment's supported mechanism, then inspect rule health and results. A syntax check does not prove that the metric names, label matching, or thresholds are correct.

FAQ

Do recording rules fill historical data automatically? No. Normal evaluation starts producing samples after the rule is loaded. Historical backfilling is a separate, explicit operation.

Where should I keep rule files? In any managed location readable by Prometheus and referenced by rule_files. File naming and directory conventions are deployment choices.

How do I find evaluation failures? Inspect the Rules or Alerts view, Prometheus logs, and prometheus_rule_evaluation_failures_total. A valid file can still contain an expression that fails during evaluation.

Sources

This guide is based on the documentation below; the examples have not been tested against a live deployment.

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting