Gauge vs. counter in Prometheus

Choose counters for cumulative events and gauges for current state, then use query functions that match their behavior.

Best practices
Illustration of metrics, monitoring, and technical investigation

A counter records a cumulative quantity that increases until it resets, such as requests handled by a process. A gauge records a value that can rise or fall, such as queue depth or memory in use. Choose the type from the meaning of the measurement, not from how its graph happens to look today.

Counters measure accumulated activity

from prometheus_client import Counter

http_requests = Counter("http_requests_total", "Total HTTP requests")
http_requests.inc()  # Add one request.
http_requests.inc(5) # Add five requests; negative increments are invalid.

Other examples include errors, completed jobs, and bytes transferred. A reset commonly follows a process restart or replacement; a restart is not the only possible reason the underlying series begins again.

Use counter-aware functions for activity over time:

rate(http_requests_total[5m])
increase(http_requests_total[1h])

The first estimates requests per second; the second estimates the increase over an hour. Both adjust for observed counter decreases. Neither can reconstruct events or resets that were never sampled.

Gauges measure current state

from prometheus_client import Gauge

memory_usage = Gauge("memory_usage_bytes", "Current memory usage")
memory_usage.set(500)
memory_usage.inc(100)
memory_usage.dec(50)

Current connections, queue length, temperature, and active users fit this model. Query the current value directly or use avg_over_time(memory_usage_bytes[5m]) for the mean of the samples in that window.

CPU needs a little care: a reported utilization value can be a gauge, while accumulated CPU time such as container_cpu_usage_seconds_total is a counter. Taking its rate produces average CPU cores used.

Question Metric type Typical query
How many requests have accumulated? Counter Raw value for that series lifetime
How quickly are requests arriving? Counter rate()
How much activity occurred in a window? Counter increase()
How much memory is in use now? Gauge Direct selector
How did a current-state measurement change? Gauge delta() or deriv(), as appropriate

Common questions

How should I measure an error ratio? Record errors and total requests as counters, calculate their rates over the same window, and divide after aligning labels. A zero request rate needs an explicit display or alerting policy.

Can a gauge have a rate of change? Yes. deriv() estimates a per-second slope using linear regression, while delta() estimates the change over a window. deriv() is not an instantaneous two-point derivative. Avoid counter-specific rate() for a gauge: decreases would be interpreted using counter-reset semantics, even if the query engine accepts the expression.

References

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting