OpenTelemetry vs Prometheus metrics
Compare OpenTelemetry and Prometheus metrics using request counts and latency histograms. Check temporality, sample timestamps, bucket layouts, and the resulting PromQL.
A checkout dashboard switches from scraped metrics to OpenTelemetry. The request count still looks familiar, but the rate drops and the P95 panel goes blank. Before touching the thresholds, look at the points that reached the backend. A delta count needs its interval to become a rate. A latency histogram may have landed as a native histogram, while the dashboard still asks for classic _bucket series.
This comparison follows the OpenTelemetry metrics data model and Prometheus 3.16. The numbers below are illustrative. OpenTelemetry metrics data model · Prometheus 3.16 query functions
| What to check | OpenTelemetry over OTLP | Prometheus |
|---|---|---|
| Counts across time | Sums and histograms can carry Delta or Cumulative temporality | Conventional counters and counter histograms are cumulative. The OTLP receiver has experimental delta modes |
| Timestamps | An aggregate can carry a start and an end. A Gauge reports a sampled value without aggregation temporality | Ordinary samples have a timestamp, often set at scrape time. Explicit timestamps can come from the target or the push path |
| Series identity | A metric sits under a resource and an instrumentation scope. Points carry attributes | The metric name and complete label set identify a series |
| Latency distribution | Explicit histograms report counts in separate ranges. Exponential histograms group values on an exponential scale | Classic histograms expose cumulative upper-bound buckets. Native histograms store the buckets, count, and sum together |
Read the count against its interval
Say checkout handles 30 requests in the first 10 seconds and 50 in the next 20 seconds. Those are delta counts. The corresponding rates are 3 and 2.5 requests per second. Over all 30 seconds, the rate is 80 divided by 30, about 2.67 requests per second. Averaging the two rates would give each interval the same weight.
If the same counter starts the example at 100, the cumulative samples are 100, 130, and 180. With a conventional Prometheus counter, rate() estimates per-second growth from the samples in the query window, adjusts for detectable resets, and extrapolates toward the window boundaries. It can use the observed timestamps even when scrapes are uneven. The arithmetic above describes the stated intervals. A PromQL result can differ when its window or available samples differ.
A missing delta point leaves a hole in the total. In an uninterrupted cumulative stream, a later sample can still capture growth across a missing sample, although you lose the timing within that gap. OTLP aggregate points can describe (start, end] when the start time is known. Whether it survives conversion depends on the path. Keep the data cumulative if the dashboard assumes cumulative counters. If you enable experimental delta-to-cumulative conversion or native delta ingestion in Prometheus, verify how that mode stores the stream before reusing the dashboard. OTLP receiver setup and delta modes
Separate time accumulation from bucket accumulation
Take an OTLP explicit histogram for checkout latency with boundaries of 0.1, 0.5, and 1 second. Suppose one point holds 2, 3, 4, and 1 observations across its four disjoint buckets. Five requests are at or below 0.5 seconds. That threshold count includes the first two buckets.
OTLP counts by disjoint range: 2 3 4 1
Prometheus classic le values: 0.1 0.5 1.0 +Inf
Classic bucket counts: 2 5 9 10
The prefix sum runs across the buckets in a single point. It does not turn a delta into a cumulative time series. If the input point is delta, converting it to conventional cumulative Prometheus buckets also requires state across intervals.
An OTel exponential histogram can map to a Prometheus native histogram on a compatible path. Prometheus made native histograms stable in 3.8. Check the server and ingestion settings as well as the format actually stored. The native format can also represent custom boundaries. Both histogram types lose information inside a bucket, so P95 is an estimate. Keep a boundary at a latency objective if you need to count observations at or below that threshold directly. OTLP and Prometheus compatibility · Prometheus native histograms
Query the histogram that arrived
These are alternatives for a cumulative counter histogram. Use the first expression when the backend stores classic buckets with common boundaries across the instances being summed. Use the second when it stores compatible native histograms under the base metric name. In both examples, the stored route label is http_route.
histogram_quantile(
0.95, sum by (http_route, le) (rate(http_server_request_duration_seconds_bucket[5m]))
)
histogram_quantile(
0.95, sum by (http_route) (rate(http_server_request_duration_seconds[5m]))
)
Apply rate() to each series before summing so reset handling has access to the original series. The classic query keeps le for interpolation across the cumulative buckets. The native query uses histogram samples and has no le label to keep. Both return seconds in this example. Prometheus histogram quantiles
Check the stored identity before switching the panel
The OpenTelemetry HTTP convention names the duration histogram http.server.request.duration and records it in seconds. Under Prometheus's underscore-and-suffix translation, it can land as http_server_request_duration_seconds, with http.route mapped to http_route. Confirm the translation strategy and label mapping first. Resource and scope fields do not all become ordinary labels, and dropping a distinguishing attribute can merge streams. Keep route templates and stable dimensions so a cutover does not create a series for every request path. OTel HTTP metrics · Prometheus OTLP translation
If one path reaches Guance through DataKit's OTLP input and the other uses its Prometheus scraper, compare one request counter and one latency histogram for the same service, route, unit, and window. Look at an ordinary interval and a restart. Switch the panel after you can account for the differences in counts and buckets. DataKit OpenTelemetry · DataKit Prometheus input
Contact us
Join the community
to join the community
Try Guance
Start online and pay only for what you use.
Get startedChoose a Guance plan
