Join two metrics in PromQL with vector matching

PromQL uses vector matching to align series from two instant vectors before an arithmetic or comparison operation. Use on(...) to name the matching labels, ignoring(...) to exclude labels, and group_left or group_right when one side has multiple series per key.

Divide metrics with matching labels

sum by (service) (rate(http_errors_total[5m]))
/
sum by (service) (rate(http_requests_total[5m]))

Both sides produce one series per service, so they can be paired directly. Select the same application population on each side. A zero denominator can produce NaN or infinity; decide whether to omit idle services, display a no-traffic state, or handle them in a separate alert condition.

Choose an explicit key

app_memory_bytes / on (instance) node_memory_bytes

This illustrative expression requires one matching series per instance on each side and an instance label that identifies the same entity. An application scrape address and a node-exporter address often differ, so they are not automatically joinable. Use an explicit shared node identity or normalize the inventory when that is the relationship you intend.

on(a, b) compares only those labels. ignoring(x, y) compares the other ordinary labels. Merely narrowing the key can create duplicate matches; it does not aggregate or choose one series for you.

Copy a label from an info metric

up * on (job, instance) group_left (version) app_version_info

With value one and exactly one app_version_info series per (job, instance), this copies version onto the up result while preserving its numeric value. If the right side has another value, multiplication changes the measurement too.

group_left allows multiple left-hand series per matching key and copies the listed labels from the unique right-hand side. group_right reverses the direction. These modifiers describe multiplicity, not the number of labels on a metric.

Troubleshoot incomplete joins

Unmatched series are dropped by these arithmetic operations. If a target stops exporting its info metric, stale metadata can disappear and the enriched result can disappear with it. Keep a direct up query for availability checks rather than depending on a metadata join to represent every failed target.

A many-to-many arithmetic match is an error. Adding group_left does not make both sides many: the right side must still be unique. Inspect duplicate keys, refine the key, or aggregate with semantics appropriate to the metric. max is not a safe arbitrary deduplication rule for every measurement.

Common questions

What goes inside group_left(...)? Labels to copy from the right, unique side. With no label list, it still permits many-to-one matching without requesting additional labels.

Can I use the same modifiers with or? or, and, and unless use set matching and allow different multiplicity behavior. They accept matching keys but not group_left or group_right.

References

• PromQL vector matching and group modifiers
• PromQL staleness and selectors