OpenTelemetry SDK: resources, providers, and signal pipelines

The OpenTelemetry API gives instrumentation code a common interface. An SDK supplies the runtime behavior behind that interface: resources, sampling, aggregation, processing, and export. Installing an API package alone does not establish a working telemetry pipeline.

Applications configure the SDK; reusable libraries generally depend on the API so they can participate in the host application's telemetry without choosing its backend. Exact configuration and signal maturity vary by language and SDK version.

Describe the source with a Resource

A Resource identifies the entity producing telemetry. Typical attributes include service.name, service.version, deployment.environment.name, and, when detected appropriately, host.name or Kubernetes attributes.

Set a deliberate service.name so the receiving backend can identify the service consistently. Keep it stable across replicas of the same service; use separate instance attributes for individual processes. A Resource describes the producer, while span attributes describe an operation and metric data-point attributes define measurement dimensions.

Resource detection is not universal. A language SDK, distribution, or Agent may provide detectors, but confirm which attributes are actually emitted. Providers configured separately for traces, metrics, and logs also need consistent resource configuration if their data should describe the same service.

Follow a span from creation to export

The trace pipeline has three main roles:

1. A TracerProvider supplies tracers, usually associated with an instrumentation scope such as a library name and version.
2. A tracer starts spans. The SDK sampler decides recording and sampling behavior at span creation. Span processors receive lifecycle callbacks; processors such as the standard batch processor export completed spans.
3. A span exporter sends the resulting trace data to a Collector or compatible backend.

SimpleSpanProcessor exports completed sampled spans without the batch processor's queue. It is useful for small examples and some specialized integrations; synchronous export can add work to the calling path. BatchSpanProcessor groups spans and exports them on a background path, with bounded buffering and shutdown behavior that must fit the application.

Batching is a useful production default, not a durability guarantee. Queue overflow, export failure, abrupt termination, and insufficient shutdown time can lose spans. Configure queue and timeout settings supported by the SDK, and invoke its shutdown or force-flush lifecycle appropriately.

A parent-based ratio sampler lets root spans use a probability while children follow their parent's decision. The details and limitations are covered in the sampling guide. A span that was never recorded cannot be recovered later by the Collector.

Propagate context across the actual boundary

A propagator injects context into a carrier, such as outbound HTTP headers, and extracts it from an incoming carrier. W3C Trace Context uses traceparent and, where applicable, tracestate. The configured propagator, supported instrumentation, and application control flow determine whether those operations happen.

The SDK does not guarantee uninterrupted traces merely by being installed. Verify HTTP clients, message producers/consumers, background jobs, and async task boundaries separately. A new process or queue message will not inherit in-memory context automatically. Avoid putting secrets or unnecessary personal data into baggage propagated downstream.

See the context propagation guide, currently in Chinese, for a closer look at carriers and boundaries.

Understand the metric pipeline

A MeterProvider supplies meters and their instruments. Common choices include counters for increments, up-down counters for signed changes, histograms for distributions, and observable instruments for values obtained during collection. These are representative choices rather than the complete list of instrument types.

The SDK aggregates measurements into metric streams. A View can select instruments, retain particular attributes, rename a stream, or choose a compatible aggregation. A MetricReader controls collection and its interaction with export: a periodic exporting reader supports push-based export, while a Prometheus reader exposes data for scraping.

Use the metrics guide for instrument choice, units, cardinality, and temporality. A shorter export interval does not make an incorrectly chosen instrument meaningful.

Treat logging as a separate integration

A LoggerProvider supplies OpenTelemetry loggers and connects log processors to log exporters where the language implementation supports them. Applications often use an existing logging framework plus an OpenTelemetry bridge rather than replacing every logging call with a new API.

Exporting traces does not automatically export application logs. Verify the bridge, processor, resource, exporter, and actual correlation fields. Log records can carry trace and span context when emitted under an active span, but the integration must preserve it. A field named request_id is not necessarily a valid trace ID.

Combine automatic and manual instrumentation

Automatic instrumentation can cover supported web frameworks, HTTP clients, database drivers, and other libraries. A Java Agent or a Python zero-code distribution can reduce startup code, but support depends on the runtime, library versions, and distribution.

Manual instrumentation adds business operations, domain attributes, and measurements that a generic framework integration cannot infer. A useful sequence is to establish framework coverage, inspect the emitted data, and add manual spans or metrics only where they answer an additional question. Avoid creating a second SDK provider or duplicate instrumentation unintentionally.

Environment-based configuration works only where the chosen SDK or auto-configuration layer implements it. Some applications need explicit initialization, lifecycle integration, or exporter construction. Read the language-specific documentation rather than assuming every variable is universally honored.

Configure a known OTLP receiving path

For a local Collector exposing standard OTLP/HTTP signal routes on port 4318, an SDK with the corresponding environment-variable support can use:

export OTEL_SERVICE_NAME=order-service
export OTEL_RESOURCE_ATTRIBUTES='service.version=1.0.0,deployment.environment.name=production'
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318
export OTEL_TRACES_SAMPLER=parentbased_traceidratio
export OTEL_TRACES_SAMPLER_ARG=0.2

This is configuration for a compatible SDK or auto-instrumentation launcher, not a command that starts instrumentation by itself. With OTLP/HTTP, the generic endpoint is normally a base URL from which standard signal paths are constructed. A signal-specific variable such as OTEL_EXPORTER_OTLP_TRACES_ENDPOINT supplies that signal's endpoint directly. Follow the OTLP exporter configuration specification and the implementation's supported options.

The 0.2 argument applies to eligible root traces, not necessarily 20% of all spans or all requests across a mixed-parent workload. Set these values before the SDK initializes; changing an environment variable does not normally reconfigure an already-running process.

For direct export to Guance DataKit, first verify the actual OpenTelemetry input. Its HTTP path and port may differ from the standard Collector endpoint above. Supply the full documented trace URL through OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, or configure the exporter in code if that SDK requires it. Match gRPC versus HTTP, TLS, and authentication to the listener. Do not treat a workspace token as an automatically required generic OTLP Authorization header.

Send a known request and check service identity, parent-child relationships, error status, and sampling behavior. Verify metric and log export independently. A successful trace export only establishes that trace path.

Common questions

Why separate the API from the SDK?

A library can instrument its operations without selecting the application's exporter or sampling policy. The application installs and configures a compatible SDK. Without one, API calls typically use no-op or non-recording behavior; they do not magically send data to a backend.

Can traces, metrics, and logs use one exporter?

OTLP transports all three signals, and a receiving service may expose all of them. SDKs generally have signal-specific exporter interfaces and instances, even when they share endpoint settings or transport libraries. Confirm that each signal's provider, processor or reader, and exporter are wired up and supported.

Can batch export lose data?

Yes. The queue is normally in process, so a crash can discard its contents. Graceful shutdown improves delivery within a bounded time; it does not guarantee remote durable storage. Test the lifecycle in the actual hosting environment, including short-lived jobs and container termination.

How should SDK startup or export failures be investigated?

Check the installed SDK/exporter versions, initialization order, supported configuration, endpoint, TLS, and authentication. Enable the implementation's diagnostic logging; OTEL_LOG_LEVEL=debug is specified but support and logging integration vary. Protect diagnostic output because it may reveal endpoints or payload details. Check Collector or DataKit receiver logs as well, and distinguish a locally queued span from one accepted by the destination.

See the SDK environment-variable specification and OpenTelemetry semantic conventions. The related OTLP, OpenTelemetry introduction, and best practices guides are currently in Chinese.