Node.js performance monitoring with perf_hooks and Prometheus

Measure slow code paths in Node.js, distinguish event loop delay from utilization, and query P95 HTTP latency across replicas in Prometheus.

Best practices
Illustration of the Node.js event loop and asynchronous tasks

An order API can get slower while its average latency barely moves. If requests to one route take longer at the tail, start with two questions. Which operation is waiting, and was the event loop able to run while it waited? Node.js performance monitoring combines code timing, runtime metrics and request latency to answer those questions at different levels.

Time the work a request waits for

Put a timer around the database call or outbound HTTP request when you want to measure elapsed time, including the wait. To isolate CPU-heavy work, also time the data transformation, template rendering or JSON serialization on its own. A database span that includes waiting for a connection from a pool has a different meaning from timing a query inside the database.

The following CommonJS example registers a PerformanceObserver once. Each call gets a separate start mark, so two queries in flight cannot overwrite each other's timing. The measure has a fixed name, and the code clears the timeline entries after use.

const { performance, PerformanceObserver } = require('node:perf_hooks');

const observer = new PerformanceObserver((list) => {
  for (const entry of list.getEntriesByName('db-query')) {
    console.log('db-query', entry.duration);
  }
});
observer.observe({ entryTypes: ['measure'] });

let sequence = 0;
async function timeQuery(query) {
  const start = 'db-query:start:' + (++sequence);
  performance.mark(start);
  try {
    return await query();
  } finally {
    performance.measure('db-query', { start, end: performance.now() });
    performance.clearMarks(start);
    performance.clearMeasures('db-query');
  }
}

Inside an async function, call await timeQuery(() => db.query(sql)) with your own database client. The timer also records rejected calls and preserves their errors. entry.duration is in milliseconds. Divide it by 1,000 before observing it in a seconds-based histogram. Use a fixed metric name and bounded labels, such as the query operation. Log each measurement only while debugging, and disconnect the observer when you stop instrumenting.

Read event loop delay alongside utilization

Event loop delay shows how late the sampling timer runs. Event loop utilization (ELU) tells you what fraction of a period the loop spent active outside its idle wait. It can rise while the process uses little CPU, for example when a synchronous call blocks the main thread. This sample uses two real ELU snapshots to compute a windowed delta. It converts the delay histogram's nanoseconds to milliseconds and resets the histogram after each window. The interval is unreferenced so it does not keep an otherwise idle process alive.

const { performance, monitorEventLoopDelay } = require('node:perf_hooks');

const delay = monitorEventLoopDelay({ resolution: 20 });
delay.enable();
let previous = performance.eventLoopUtilization();

const timer = setInterval(() => {
  const current = performance.eventLoopUtilization();
  const delta = performance.eventLoopUtilization(current, previous);
  previous = current;
  const p99Ms = delay.count ? delay.percentile(99) / 1e6 : null;
  delay.reset();
  console.log({ utilization: delta.utilization, p99Ms });
}, 10_000);
timer.unref();

function stopSampling() {
  clearInterval(timer);
  delay.disable();
}

Run the sampler inside the service and call stopSampling() on shutdown. Compare delay with a baseline measured at the same resolution. If delay rises with request latency, look for CPU-bound work, synchronous operations or garbage collection (GC) pauses. Compare GC duration with heap usage over time before concluding that retained memory is growing. High request latency with steady loop delay can point to a database pool or external dependency. A worker thread has its own loop, so measuring only the main thread misses delays inside a worker. Worker threads are useful for CPU-intensive JavaScript and introduce work scheduling and data transfer costs.

Export request and runtime metrics on one endpoint

This Express example uses prom-client@15.1.3, the older package name. The upstream library now publishes as @prometheus-io/client. Check its changelog when upgrading. The example records completed responses for the order route, starting at the route handler. Node emits finish when the response is handed off to the operating system for transmission. Track early disconnects and client-side latency separately if they matter to the investigation.

const express = require('express');
const client = require('prom-client');
const app = express();
const registry = new client.Registry();

client.collectDefaultMetrics({ register: registry });
const httpDuration = new client.Histogram({
  name: 'http_request_duration_seconds',
  help: 'Duration of completed HTTP responses in seconds',
  labelNames: ['method', 'route', 'status'],
  buckets: [0.01, 0.05, 0.1, 0.3, 0.5, 1, 2, 5],
  registers: [registry],
});

const orderRoute = '/api/orders/:id';
app.get(orderRoute, (req, res) => {
  const end = httpDuration.startTimer();
  res.once('finish', () => end({
    method: req.method,
    route: orderRoute,
    status: String(res.statusCode),
  }));
  res.json({ ok: true });
});

app.get('/metrics', async (_req, res, next) => {
  try {
    const body = await registry.metrics();
    res.set('Content-Type', registry.contentType).end(body);
  } catch (error) {
    next(error);
  }
});

app.listen(3000, '127.0.0.1');

Run the file with Express and the specified client installed, then request http://127.0.0.1:3000/api/orders/42 and inspect http://127.0.0.1:3000/metrics. The label stays /api/orders/:id even when the order ID changes. Both the default process and Node.js metrics and the HTTP histogram go through the same registry. The example listens on loopback. In production, expose the scrape endpoint only to authorized monitoring infrastructure. Scrape each replica separately, or configure explicit aggregation when using Node's cluster module.

Query the tail across instances

The histogram records counts by bucket. For a Prometheus scrape job named node-api, this query estimates P95 per route across instances over five minutes. rate runs before aggregation so it can account for each series' counter resets, and le preserves the bucket boundaries.

histogram_quantile(
  0.95,
  sum by (route, le) (
    rate(http_request_duration_seconds_bucket{job="node-api"}[5m])
  )
)

Look at the request rate from http_request_duration_seconds_count beside P95. An estimate based on a few observations can jump even when the wider distribution stays steady. Classic histograms estimate quantiles by interpolation within buckets, so align bucket boundaries with relevant latency thresholds and keep them consistent across replicas. For the fraction of responses within 300 ms, compare the rate of the bucket with le="0.3" with the rate of _count over the same scope and window. Likewise, for a 5xx error ratio, divide the 5xx count rate by the total rate and handle absent error series and periods with no requests explicitly.

Follow a slow route into its dependencies

If the order route's P95 rises but loop delay and GC stay near baseline, use a trace to see where a slow request spent its time. A spike in time spent waiting for a database connection suggests looking at pool saturation before rewriting the query. If loop delay rises across routes on one instance, the process's own work is a better place to start.

In Guance, the DataKit Prom collector can scrape the metrics endpoint, while Node.js APM instrumentation sends traces through a separate configuration. Align the service, environment and instance identifiers used by both. Compare the metrics and traces within the same time window so that a slow route and a busy replica refer to the same deployment.

References and further reading

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting