What is a Prometheus target?
Understand targets, scrape jobs, service discovery, and the up metric, including why a successful scrape does not prove application health.
A Prometheus target is an endpoint that Prometheus scrapes for metrics, such as http://10.0.0.1:9100/metrics. The scrape configuration supplies its address, scheme, path, timing, and access settings. Prometheus records the fetched samples and creates its own metrics about the scrape.
Targets belong to scrape jobs
A job groups targets that share a scrape configuration. For example, a node job might collect Node Exporter metrics from many hosts. Each target normally has an instance label, while job identifies the scrape group.
scrape_configs:
- job_name: node
static_configs:
- targets: ['10.0.0.1:9100', '10.0.0.2:9100']
The default scheme is HTTP and the default metrics path is /metrics; either can be changed. Relabeling can also adjust the target address or identity before the scrape.
Discover targets statically or dynamically
static_configs lists endpoints in the configuration. Service discovery obtains them from systems such as Kubernetes or Consul. File-based discovery lets an external process maintain target groups in files.
Dynamic discovery can follow workload changes, but a discovered endpoint still has to be reachable and expose a supported metrics format. Relabeling rules may deliberately drop discovered targets, so check discovery and relabeling when an expected endpoint never appears as an active target.
Read the up metric correctly
Prometheus generates up for each scraped target: one for a successful scrape and zero for a failed scrape.
up{job="node"} == 0
This selects failed scrapes. It does not identify every kind of application failure: an application can return its metrics successfully while its business requests fail. Conversely, a healthy application may have a broken metrics route.
The Targets page shows the scrape URL, last error, and timing. Connection refusal, timeout, TLS or authentication errors, unexpected HTTP responses, and invalid metric exposition can all cause failures.
FAQ
How many targets can one Prometheus server handle? There is no universal target-count limit. Samples per scrape, interval, series churn, rules, queries, retention, and available resources determine capacity. Measure those factors before choosing sharding or another architecture. Federation and Agent mode serve different purposes and are not automatic substitutes for capacity planning.
How do I temporarily remove a target? Remove it from the static configuration and reload, or update the relevant discovery source. File-based discovery watches its configured files. Once a target is removed, its up series eventually becomes stale; it does not remain at zero forever. Use separate absence checks when disappearance itself should alert.
Sources
This guide is based on the documentation below; the examples have not been tested against a live deployment.
Contact us
Join the community
to join the community
Try Guance
Start online and pay only for what you use.
Get startedChoose a Guance plan