Install Prometheus on Kubernetes with Helm

Deploy Prometheus on Kubernetes, choose between the standalone chart and kube-prometheus-stack, and verify storage, retention, and ServiceMonitor discovery.

Best practices
Illustration of Prometheus and Grafana in a monitoring stack

The Helm release is installed and Prometheus opens in your browser. Your application still has no time series. At that point, the important question is who decides what Prometheus scrapes. The standalone chart uses Prometheus configuration. With kube-prometheus-stack, the Operator turns resources such as ServiceMonitor into scrape configuration.

Both routes can monitor Kubernetes. Start with the standalone chart if you want to manage Prometheus configuration directly. Choose the stack if application teams will declare scrape targets through Kubernetes resources and want cluster dashboards and rules to start from.

What Helm manages

Helm renders the workloads, services, permissions, and storage settings that make up the release. Keep the chosen chart version and values together so you can review what changes on upgrade. A Helm rollback changes the release. It cannot restore time series from a lost volume or undo a CRD schema change.

Install the standalone chart

Check your current Kubernetes context and confirm that your account can create the chart's resources. The example uses prometheus chart 29.43.0, which packages Prometheus v3.16.0. We checked the manifests with helm template. The cluster supplies the storage and enforces its own policies.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update prometheus-community
helm upgrade --install prometheus prometheus-community/prometheus \
  --version 29.43.0 --namespace monitoring --create-namespace

The chart enables Alertmanager, kube-state-metrics, Node Exporter, and Pushgateway by default in this version. Check kubectl get pods,pvc,svc -n monitoring. A PVC that stays Pending may be waiting for a compatible provisioner or for a Pod to be scheduled, depending on the StorageClass. Read its events before changing the chart.

The default Service is prometheus-server with port 80 backed by the server's 9090. Run kubectl -n monitoring port-forward svc/prometheus-server 9090:80 and open http://localhost:9090/targets. Start by checking which jobs are present and whether their last scrape succeeded.

Size the volume and set retention

The standalone chart enables the server's PVC by default and requests 8Gi. Alertmanager uses its own storage key, alertmanager.persistence, and defaults to a 2Gi claim. Disabling server persistence switches to emptyDir; replacing that Pod loses its local history.

Save the example as values.yaml. Before applying it to the default 8Gi claim, check whether that claim can grow to 50Gi. If it cannot, plan the storage migration first.

server:
  persistentVolume:
    enabled: true
    size: 50Gi
  retention: 30d
  retentionSize: 40GB
  resources:
    requests: { cpu: 500m, memory: 2Gi }
    limits:   { cpu: 2,    memory: 8Gi }

alertmanager:
  persistence:
    enabled: true
helm upgrade prometheus prometheus-community/prometheus \
  --version 29.43.0 --namespace monitoring --values values.yaml

Here 50Gi is the volume request, 30d the time limit, and 40GB the Prometheus size threshold. Treat all three as example settings. When both retention limits are set, whichever triggers first applies. Leave room below disk capacity for compaction and other storage overhead. Set CPU and memory from the work your server actually does. Prometheus storage

Install kube-prometheus-stack if you want the Operator

The stack adds the Operator, ready-made Kubernetes rules and dashboards, and Grafana. The following is a separate installation path. Use an isolated cluster to try both with default settings. Running them side by side against the same targets can duplicate scrapes or notifications.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update prometheus-community
helm upgrade --install monitoring prometheus-community/kube-prometheus-stack \
  --version 92.21.2 --namespace monitoring --create-namespace

Stack chart 92.21.2 requires Kubernetes 1.25 or later. Its Prometheus storage is configured under prometheus.prometheusSpec.storageSpec. Keep standalone values with the standalone release. Read the stack upgrade notes for the CRD procedure before changing versions.

Follow a ServiceMonitor all the way to a target

Assume you already have a Service in namespace apps whose metadata has app: my-app. Its port named metrics reaches an application serving /metrics. Save the following as servicemonitor.yaml and apply it after the stack has registered the CRD.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: my-app
  namespace: monitoring
  labels:
    release: monitoring
spec:
  namespaceSelector:
    matchNames: [apps]
  selector:
    matchLabels:
      app: my-app
  endpoints:
    - port: metrics
      path: /metrics
      interval: 15s
kubectl apply -f servicemonitor.yaml

The Prometheus resource created by this stack release selects monitors with release: monitoring. This monitor then selects Services in apps with app: my-app. The port in endpoints is the name on the Service. A label on the Pod alone cannot satisfy the Service selector.

If the job is missing, inspect serviceMonitorSelector and serviceMonitorNamespaceSelector on the Prometheus resource. Check the Service labels and port name, its EndpointSlices, and whether the Prometheus service account can discover the resources in apps. If the target appears as Down, read the scrape error and follow it to the endpoint, path, authentication, or network. Operator troubleshooting

You can keep this discovery path when forwarding a subset of metrics to Guance through the DataKit Prometheus Remote Write receiver. Verify the cluster, namespace, and instance labels in the destination. If DataKit also scrapes an endpoint directly, check which path should own the samples before enabling both.

Questions that come up after installation

Does the standalone chart only scrape external targets? It can discover targets inside Kubernetes too. The distinction is how you manage scrape configuration. The stack is a better fit when teams want to declare and delegate it using Operator resources.

Can I increase a bound PVC just by changing the chart values? The StorageClass must allow expansion and the storage driver must support it. Follow the applicable chart and driver procedure to update the existing claim and reconcile the values. A StatefulSet volume template and the claims created from it have separate lifecycles. Kubernetes PVC expansion

How do I collect metrics from several clusters? Keep scraping local and use Remote Write to send selected series to a compatible receiver such as Thanos Receive or a configured DataKit receiver. Distinguish clusters with external labels and define centrally evaluated rules where the chosen backend supports them.

Further reading. Install Grafana on Kubernetes with Helm · Configure Alertmanager · OpenTelemetry vs Prometheus

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting