OpenTelemetry metrics in Go: instruments, Views, and OTLP
Instrument HTTP requests and orders with the OpenTelemetry Go Metrics SDK, drop a metric stream with a View, and verify OTLP/gRPC export locally.
A checkout endpoint can slow down while the number of orders it accepts stays flat. A request-duration histogram tells you how response times moved. A separate order counter tells you how many requests actually created an order. Go’s OpenTelemetry Metrics SDK lets you record both and send them over OTLP.
This guide follows two synthetic HTTP requests through the Go SDK into a local OpenTelemetry Collector. One succeeds and one returns 503. That difference gives us a quick way to check the automatic HTTP metrics, the business instruments, and the View that drops an unused stream.
Set up a local OTLP receiver
Save the following as collector.yaml and run otelcol-contrib --config=collector.yaml. The gRPC receiver listens on loopback. The debug exporter prints the payload so you can inspect it without a backend.
receivers:
otlp:
protocols:
grpc:
endpoint: 127.0.0.1:14317
exporters:
debug:
verbosity: detailed
service:
pipelines:
metrics:
receivers: [otlp]
exporters: [debug]
The example was checked with Go 1.26, OpenTelemetry Go SDK 1.47.0, the Go contrib instrumentation 0.72.0, and Collector Contrib 0.165.0. Install the dependencies in an empty directory.
go mod init example.com/otel-order-metrics
go get go.opentelemetry.io/otel/metric@v1.47.0 \
go.opentelemetry.io/otel/sdk@v1.47.0 \
go.opentelemetry.io/otel/sdk/metric@v1.47.0 \
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc@v1.47.0 \
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.72.0 \
go.opentelemetry.io/contrib/instrumentation/runtime@v0.72.0
Wire the MeterProvider to the handler
The MeterProvider gives the application a resource name, a periodic reader, and a View. The reader exports the metric points over OTLP/gRPC. The 30-second interval applies while the process is running. This short example sends its final collection through Shutdown, using a separate timeout after the HTTP test server closes. Run the program as go run . after saving it as main.go.
package main
import (
"context"
"errors"
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"strings"
"time"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
otelruntime "go.opentelemetry.io/contrib/instrumentation/runtime"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc"
"go.opentelemetry.io/otel/metric"
sdkmetric "go.opentelemetry.io/otel/sdk/metric"
"go.opentelemetry.io/otel/sdk/resource"
)
func orderHandler(meter metric.Meter) (http.Handler, error) {
created, err := meter.Int64Counter("orders.created", metric.WithUnit("{order}"))
if err != nil {
return nil, err
}
inFlight, err := meter.Int64UpDownCounter("orders.inflight", metric.WithUnit("{request}"))
if err != nil {
return nil, err
}
queueDepth, err := meter.Int64Gauge("orders.queue.depth", metric.WithUnit("{order}"))
if err != nil {
return nil, err
}
duration, err := meter.Float64Histogram("orders.process.duration", metric.WithUnit("s"),
metric.WithExplicitBucketBoundaries(0.05, 0.1, 0.25, 0.5, 1, 5))
if err != nil {
return nil, err
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
started := time.Now()
inFlight.Add(ctx, 1)
defer inFlight.Add(ctx, -1)
defer func() { duration.Record(ctx, time.Since(started).Seconds()) }()
queueDepth.Record(ctx, 2)
if _, err := io.Copy(io.Discard, r.Body); err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
if r.URL.Query().Get("fail") == "1" {
w.WriteHeader(http.StatusServiceUnavailable)
return
}
created.Add(ctx, 1, metric.WithAttributes(attribute.String("channel", "app")))
w.WriteHeader(http.StatusCreated)
}), nil
}
func run() (err error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
exporter, err := otlpmetricgrpc.New(ctx, otlpmetricgrpc.WithEndpoint("127.0.0.1:14317"),
otlpmetricgrpc.WithInsecure())
if err != nil {
return err
}
mp := sdkmetric.NewMeterProvider(
sdkmetric.WithResource(resource.NewSchemaless(attribute.String("service.name", "order-service"))),
sdkmetric.WithReader(sdkmetric.NewPeriodicReader(exporter, sdkmetric.WithInterval(30*time.Second))),
sdkmetric.WithView(sdkmetric.NewView(
sdkmetric.Instrument{Name: "http.server.request.body.size"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationDrop{}},
)),
)
defer func() {
shutdownCtx, stop := context.WithTimeout(context.Background(), 5*time.Second)
defer stop()
err = errors.Join(err, mp.Shutdown(shutdownCtx))
}()
if err := otelruntime.Start(otelruntime.WithMeterProvider(mp)); err != nil {
return err
}
const route = "/orders"
handler, err := orderHandler(mp.Meter("example.com/order-service"))
if err != nil {
return err
}
mux := http.NewServeMux()
mux.Handle("POST "+route, otelhttp.NewHandler(
handler, route, otelhttp.WithMeterProvider(mp),
))
server := httptest.NewServer(mux)
defer server.Close()
for _, suffix := range []string{"", "?fail=1"} {
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
server.URL+route+suffix, strings.NewReader("demo"))
if err != nil {
return err
}
resp, err := server.Client().Do(req)
if err != nil {
return err
}
resp.Body.Close()
fmt.Println(resp.StatusCode)
}
return nil
}
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
WithInsecure() disables transport security. Here the receiver is a local test fixture. For remote ingestion, configure the receiver’s address, TLS and any required credentials for that deployment.
Read HTTP metrics and business metrics separately
The program prints 201 and 503. On the Collector, look for service.name=order-service and http.server.request.duration. With this version of otelhttp, the duration histogram records one observation per completed instrumented request. Its Count gives request volume for the same dimensions. The two responses have different status-code attributes, so expect a count of one for each. The application counter orders.created should equal one in total, with channel=app.
The HTTP middleware also records body sizes. The SDK View drops the metric stream named http.server.request.body.size before export. Response body size and request duration remain. If you need a metric with fewer dimensions, use a View attribute filter and confirm that the remaining labels still answer your query. Dropping an entire stream and removing attributes from a retained stream change different parts of the payload.
Choose an instrument for each measurement
| Question | Instrument | What it records |
|---|---|---|
| How many orders did the handler create? | Counter | Nonnegative increments. Aggregate them with a query that understands the stream’s temporality and resets. |
| How many requests are being handled? | UpDownCounter | Add one on entry and subtract one on exit with matching attributes. A read after both requests sees zero. |
| What is the current queue depth? | Gauge | Record a value from the source of truth. The example uses the fixed value two. An ObservableGauge can read the queue during collection. |
| How long did processing take? | Histogram | Record elapsed seconds into a distribution. The example includes a bucket boundary at 250 ms. |
Set histogram boundaries around the thresholds you use in alerts. P95 estimated from explicit buckets depends on their resolution. When combining distributions across replicas, keep bucket layouts compatible and make sure the backend combines the distributions before estimating a percentile.
Use stable metric names and declared units such as s or By. Keep attributes to dimensions you query, such as channel and region. A request ID or full URL can create a new time series for each request. Use a route template such as /orders/{id} instead. How metric cardinality grows
Send the same metrics to Guance
The local Collector shows the OTLP payload before any backend-specific conversion. To use DataKit, enable its OpenTelemetry input and check the receiver configuration and network path. Point the gRPC exporter at the configured gRPC listener. If you choose OTLP/HTTP instead, use the HTTP/Protobuf exporter and its configured metrics path. DataKit’s documented default HTTP metrics path is /otel/v1/metrics, and its default HTTP listener port is 9529. DataKit OpenTelemetry input
Send a known number of requests and check the received metric name, unit, attributes and histogram representation before setting up a monitor. For an investigation that also uses traces, align service and environment values across signals and confirm the spans were collected. An aggregate point covers a group of observations and usually cannot identify one trace. Configure Guance monitors
Questions that come up in production
What happens to a counter when the service restarts?
A cumulative OTLP point can carry a start timestamp. When the process restarts, the new stream starts accumulating again. Verify how the backend computes rates across the reset. If you configure delta temporality, the exporter reports interval changes and the backend needs to interpret them accordingly.
Can I aggregate a Summary across replicas?
Client-computed quantiles cannot generally be combined into a fleet percentile. Go’s Metrics API exposes Histogram as its distribution instrument. A backend can combine compatible histograms and estimate the percentile from that combined distribution.
How often should the SDK export?
The example uses 30 seconds. Start with an interval that fits your alert window, the available receiver capacity, and the data freshness you need. Shorter intervals send more batches. Measure the impact at your own volume.
How do I collect Go runtime metrics?
This example calls runtime.Start on the same provider, which registers runtime measurements such as go.goroutine.count and go.memory.gc.goal in the pinned contrib version. To collect the scheduler latency histogram go.schedule.duration, add runtime.NewProducer() to the reader as documented. Check names and availability for your pinned version. Go runtime instrumentation
References
- OpenTelemetry Go instrumentation
- Go Metrics SDK and Views
- HTTP metric conventions and OTLP metric data model
- OTLP metrics gRPC exporter
Related reading
Contact us
Join the community
to join the community
Try Guance
Start online and pay only for what you use.
Get startedChoose a Guance plan
