Build a custom OpenTelemetry Collector with OCB

OpenTelemetry Collector Builder (OCB) generates and compiles a Collector distribution from a manifest of Go modules. Use it when you need selected community components, an internal component, or an explicitly controlled dependency inventory.

A smaller distribution can reduce unused code and dependencies, but it also makes your team responsible for its build, compatibility, testing, and updates. If an official distribution already meets the requirements, using that release can be the simpler operational choice.

Decide whether a custom distribution is worth maintaining

A custom build is useful in three common cases:

• Only a few components from a larger distribution are required, and dependency or binary-size control matters.
• An internal receiver, processor, exporter, connector, or extension must be included.
• The deployment needs a reviewable inventory of the code shipped in its telemetry process.

There is no meaningful universal threshold such as “fewer than ten components means use OCB.” Consider who will own upgrades, vulnerability review, artifact provenance, and compatibility tests. Removing components alone does not prove that a binary is secure or compliant.

For applications sending telemetry to Guance, first compare the required path with the DataKit OpenTelemetry input. Direct SDK export may be sufficient for some deployments. Existing Collector processors, routing, queue behavior, and additional receivers still need individual assessment before a replacement.

Distinguish the build manifest from runtime configuration

The OCB manifest specifies which code goes into the binary. The Collector runtime configuration selects and connects those compiled components into pipelines. Including an exporter in the manifest does not activate it, and naming a component only in the runtime YAML cannot add it to the executable.

The example below uses OCB 0.162.0 and compatible modules from the corresponding official release. Save it as manifest.yaml:

dist:
  module: example.com/observability/my-otelcol
  name: my-otelcol
  description: Custom OpenTelemetry Collector
  version: 0.1.0
  output_path: ./build

receivers:
  - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.162.0

processors:
  - gomod: go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.162.0
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.162.0

exporters:
  - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.162.0
  - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v0.162.0
  - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v0.162.0

providers:
  - gomod: go.opentelemetry.io/collector/confmap/provider/fileprovider v1.68.0
  - gomod: go.opentelemetry.io/collector/confmap/provider/envprovider v1.68.0

The distribution's 0.1.0 is your own release version, distinct from component module versions. output_path holds the generated Go project and binary. The module path is an example; use your organization's chosen module identity for a maintained distribution.

The two OTLP exporters let this example demonstrate either gRPC or HTTP, and debug supports local inspection. Omit unused exporters from a production manifest once the destination contract is decided. The file and environment configuration providers make those configuration sources available.

Compatibility does not mean every module must have the identical version string. This Collector release combines v0.162.0 component modules with v1.68.0 configuration providers. Some Contrib components use their own stable module versions. Use a compatible, tested release set rather than replacing all numbers mechanically. The official release manifest is a useful reference.

Install the matching builder and compile

Use a Go toolchain compatible with the pinned release. In an isolated working directory, this installation method places the builder in a local bin directory:

mkdir -p bin
GOBIN="$PWD/bin" go install go.opentelemetry.io/collector/cmd/builder@v0.162.0
./bin/builder --config=manifest.yaml

The tool is called OCB, but go install creates an executable named builder. Downloaded release binaries may instead be named ocb. Use the name for the installation method you chose. Official prebuilt binaries and a builder container are other options described in the OCB guide.

OCB generates the Go module and component registrations, resolves dependencies, and compiles the configured executable. A successful build establishes that this module set compiles; it does not prove a particular backend integration, data transformation, or failure policy works.

Inspect the resulting inventory:

./build/my-otelcol components

Keep the manifest, generated dependency information, toolchain identity, and artifact digest together in the build evidence. Rebuild and test deliberately when upgrading instead of installing @latest independently of the manifest.

Run a local smoke test

Save a separate runtime configuration as collector.yaml:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 127.0.0.1:4318
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 256
    spike_limit_mib: 64
  batch:
    timeout: 1s
exporters:
  debug:
    verbosity: detailed
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [debug]

Then validate and start the exact binary you built:

./build/my-otelcol validate --config=collector.yaml
./build/my-otelcol --config=collector.yaml

Send a known trace through OTLP/HTTP and inspect its service name, span name, and attributes in debug output. This sample binds only to the local host and configures only a trace pipeline. It does not expose a network service to other containers or enable logs and metrics automatically.

For a real DataKit destination, replace the debug exporter with the appropriate configured otlp_http or otlp_grpc exporter and update the trace pipeline's exporter list. In this pinned release those are the component names, although their Go module paths still contain otlphttpexporter and otlpexporter. Older examples may use otlphttp or otlp names.

Use the actual listener, path, TLS, and authentication contract from the DataKit integration documentation. A generic port number or arbitrary token header is not enough. Verify every signal separately and retain processors required by the application.

Package the binary with its runtime needs

A multistage container build can use a build stage to run OCB and a smaller runtime stage to hold the executable and runtime configuration. Pin the builder and base-image versions or digests, and compile for the runtime image's OS and architecture. A macOS binary cannot be copied into a Linux image and expected to run.

Use a non-root runtime user where supported. Include the certificates, writable storage, and filesystem permissions required by the selected components. A persistent queue or file receiver needs more than an executable and a read-only root filesystem.

Expose only the listeners actually configured. Docker's EXPOSE instruction documents a port; it does not publish it or change the Collector's bind address. The loopback smoke-test listener above needs an intentional address change for traffic arriving from outside its container.

Operate the distribution as a maintained component

Put generation, compilation, configuration validation, smoke tests, and artifact publication into CI. Record the dependency inventory and inspect security updates for both Collector modules and their transitive dependencies.

Enable appropriate internal telemetry and monitor accepted, refused, sent, and failed data alongside resource use. Add a health-check extension only if it is included in the manifest and configured at runtime. Test graceful shutdown and the queues or persistent storage your distribution actually uses.

Upgrades should rerun component and pipeline tests, including error cases and a controlled destination outage. The Collector monitoring guide, currently in Chinese, covers the operating signals to watch.

Common questions

Should production always use a custom Collector?

No. Use a custom distribution when its dependency control or custom integration justifies ownership of the build. An official Contrib or other suitable distribution may already meet the operational requirements. Inspect its actual release manifest and component stability instead of assuming that Core means “only stable” or Contrib means “everything.”

Can a manifest include connectors and extensions?

Yes. Declare them in the corresponding connectors and extensions sections with compatible modules. Connectors participate in pipeline wiring, while extensions are enabled through the runtime service configuration. Add only the configuration providers and other build modules needed by the deployment.

What does a version mismatch look like?

Dependency resolution or compilation may fail because component APIs are incompatible. A binary that builds can still reject a runtime configuration or change behavior after an upgrade. Matching the release family helps, but module-level compatibility and integration tests are the evidence that matters.

Can a custom Collector export to Guance?

Yes, if it includes an exporter compatible with the enabled DataKit receiver and the configured signal. Confirm transport, routing, credentials where required, and receiving fields with known data; custom compilation itself does not alter that protocol contract.

Continue with the Collector architecture guide and Kubernetes Helm deployment. The OTTL guide is currently in Chinese.