OpenTelemetry vs Prometheus metrics

Compare OpenTelemetry and Prometheus metrics using request counts and latency histograms. Check temporality, sample timestamps, bucket layouts, and the resulting PromQL.

Best practices
Illustration of telemetry flowing into observability backends

A checkout dashboard switches from scraped metrics to OpenTelemetry. The request count still looks familiar, but the rate drops and the P95 panel goes blank. Before touching the thresholds, look at the points that reached the backend. A delta count needs its interval to become a rate. A latency histogram may have landed as a native histogram, while the dashboard still asks for classic _bucket series.

This comparison follows the OpenTelemetry metrics data model and Prometheus 3.16. The numbers below are illustrative. OpenTelemetry metrics data model · Prometheus 3.16 query functions

What to check OpenTelemetry over OTLP Prometheus
Counts across time Sums and histograms can carry Delta or Cumulative temporality Conventional counters and counter histograms are cumulative. The OTLP receiver has experimental delta modes
Timestamps An aggregate can carry a start and an end. A Gauge reports a sampled value without aggregation temporality Ordinary samples have a timestamp, often set at scrape time. Explicit timestamps can come from the target or the push path
Series identity A metric sits under a resource and an instrumentation scope. Points carry attributes The metric name and complete label set identify a series
Latency distribution Explicit histograms report counts in separate ranges. Exponential histograms group values on an exponential scale Classic histograms expose cumulative upper-bound buckets. Native histograms store the buckets, count, and sum together

Read the count against its interval

Say checkout handles 30 requests in the first 10 seconds and 50 in the next 20 seconds. Those are delta counts. The corresponding rates are 3 and 2.5 requests per second. Over all 30 seconds, the rate is 80 divided by 30, about 2.67 requests per second. Averaging the two rates would give each interval the same weight.

If the same counter starts the example at 100, the cumulative samples are 100, 130, and 180. With a conventional Prometheus counter, rate() estimates per-second growth from the samples in the query window, adjusts for detectable resets, and extrapolates toward the window boundaries. It can use the observed timestamps even when scrapes are uneven. The arithmetic above describes the stated intervals. A PromQL result can differ when its window or available samples differ.

A missing delta point leaves a hole in the total. In an uninterrupted cumulative stream, a later sample can still capture growth across a missing sample, although you lose the timing within that gap. OTLP aggregate points can describe (start, end] when the start time is known. Whether it survives conversion depends on the path. Keep the data cumulative if the dashboard assumes cumulative counters. If you enable experimental delta-to-cumulative conversion or native delta ingestion in Prometheus, verify how that mode stores the stream before reusing the dashboard. OTLP receiver setup and delta modes

Separate time accumulation from bucket accumulation

Take an OTLP explicit histogram for checkout latency with boundaries of 0.1, 0.5, and 1 second. Suppose one point holds 2, 3, 4, and 1 observations across its four disjoint buckets. Five requests are at or below 0.5 seconds. That threshold count includes the first two buckets.

OTLP counts by disjoint range:    2   3   4   1
Prometheus classic le values:  0.1 0.5 1.0 +Inf
Classic bucket counts:           2   5   9  10

The prefix sum runs across the buckets in a single point. It does not turn a delta into a cumulative time series. If the input point is delta, converting it to conventional cumulative Prometheus buckets also requires state across intervals.

An OTel exponential histogram can map to a Prometheus native histogram on a compatible path. Prometheus made native histograms stable in 3.8. Check the server and ingestion settings as well as the format actually stored. The native format can also represent custom boundaries. Both histogram types lose information inside a bucket, so P95 is an estimate. Keep a boundary at a latency objective if you need to count observations at or below that threshold directly. OTLP and Prometheus compatibility · Prometheus native histograms

Query the histogram that arrived

These are alternatives for a cumulative counter histogram. Use the first expression when the backend stores classic buckets with common boundaries across the instances being summed. Use the second when it stores compatible native histograms under the base metric name. In both examples, the stored route label is http_route.

histogram_quantile(
  0.95, sum by (http_route, le) (rate(http_server_request_duration_seconds_bucket[5m]))
)

histogram_quantile(
  0.95, sum by (http_route) (rate(http_server_request_duration_seconds[5m]))
)

Apply rate() to each series before summing so reset handling has access to the original series. The classic query keeps le for interpolation across the cumulative buckets. The native query uses histogram samples and has no le label to keep. Both return seconds in this example. Prometheus histogram quantiles

Check the stored identity before switching the panel

The OpenTelemetry HTTP convention names the duration histogram http.server.request.duration and records it in seconds. Under Prometheus's underscore-and-suffix translation, it can land as http_server_request_duration_seconds, with http.route mapped to http_route. Confirm the translation strategy and label mapping first. Resource and scope fields do not all become ordinary labels, and dropping a distinguishing attribute can merge streams. Keep route templates and stable dimensions so a cutover does not create a series for every request path. OTel HTTP metrics · Prometheus OTLP translation

If one path reaches Guance through DataKit's OTLP input and the other uses its Prometheus scraper, compare one request counter and one latency histogram for the same service, route, unit, and window. Look at an ordinary interval and a restart. Switch the panel after you can account for the differences in counts and buckets. DataKit OpenTelemetry · DataKit Prometheus input

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting