Calculate Kubernetes CPU usage with Prometheus

Take the rate of container_cpu_usage_seconds_total to calculate average CPU cores used over a window. A result of 0.5 means half a core on average; it is not automatically 50% of a Pod's request, limit, or node capacity.

Collect the required metrics

Kubelet commonly exposes cAdvisor container metrics at /metrics/cadvisor. Prometheus needs authorized network access to that endpoint. Resource request and limit metrics in the examples below come from kube-state-metrics, a separate collection path.

Inspect the actual labels before using the queries. These examples assume one cluster and one nonduplicated scrape stream. In a multi-cluster store, include a stable cluster identity in every grouping and matching key.

Show CPU cores by Pod

sum by (namespace, pod) (
  rate(container_cpu_usage_seconds_total{
    container!="", container!="POD", pod!=""
  }[5m])
)

Including namespace avoids combining identically named Pods from different namespaces. The container filters exclude common aggregate or sandbox series; label conventions vary with runtime and collection pipeline. An empty image label does not universally mean “sandbox,” so confirm the series rather than relying on that filter alone.

For each container, also retain container in the grouping.

Compare with requested or limited CPU

This calculates CPU usage as a percentage of the sum of container requests:

100 *
sum by (namespace, pod) (
  rate(container_cpu_usage_seconds_total{
    container!="", container!="POD", pod!=""
  }[5m])
)
/
(
  sum by (namespace, pod) (
    kube_pod_container_resource_requests{resource="cpu", unit="core"}
  ) > 0
)

Replace kube_pod_container_resource_requests with kube_pod_container_resource_limits for a percentage of configured container limits. The > 0 filter removes zero denominators while retaining their original positive values; absent denominators remain absent. Do not turn missing limits into a zero-percent usage claim.

Requests are scheduling reservations, not a CPU usage cap, so a request percentage can exceed 100%. A sum of container limits is only a complete Pod budget when all relevant containers have limits. Container sums also do not necessarily equal effective Pod scheduling resources when init containers, overhead, or Pod-level resources are involved. For that question, evaluate kube-scheduler's Pod resource metrics as documented for your Kubernetes version.

Inspect throttling separately

CPU limits can cause throttling in short bursts even when a five-minute usage average is below the limit. Where exported, inspect container_cpu_cfs_throttled_seconds_total and the throttled/total CFS period counters. These are evidence of throttling, not a universal prediction that always appears before other symptoms.

Common questions

Why is the result empty? Verify that the specific cAdvisor scrape is successful, that this metric exists, and that its labels match the selector. up{job="kubelet"} alone may cover a different kubelet endpoint. For ratio queries, verify kube-state-metrics and matching labels too.

Which rate window should I use? Five minutes is a useful starting point. Shorter windows reveal faster movement but need enough samples. Alert rules can combine a rate window with a sustained for duration; they serve different purposes.

References

• cAdvisor Prometheus metrics
• kube-state-metrics Pod resource metrics
• PromQL arithmetic and comparisons