OTTL explained: Transform and filter OpenTelemetry data

Learn how OTTL statements, paths, and contexts work in the Collector. Correct a legacy span status, remove a header, and extract a route group.

Best practices
Illustration of service traces and metrics in application performance monitoring

An HTTP dashboard can show a jump in errors after a service changes instrumentation, even when the requests behave as before. If the previous library marked every 404 as an error, a rule that resets all 404 spans can hide failures on the client side. You need to know which span you are changing and why it has that status.

The OpenTelemetry Transformation Language (OTTL) lets Collector processors inspect telemetry and act on it. The transform processor can edit fields, and the filter processor can use OTTL conditions to drop matching telemetry. Both work on the Collector’s structured data.

OTTL at a glance

  • A transform statement calls an editor and can add a where condition.
  • Paths such as span.attributes["http.request.method"] and span.status.code identify fields in context.
  • Common rules rename, remove, extract, or conditionally update fields.
  • When moving a rule between processing layers, compare the fields, value types, and failure behavior.

What does OTTL help with?

Telemetry often needs a small change on its way to the backend. You might remove a captured header, normalize an old attribute to a semantic convention, or correct a status with known provenance. A Collector rule can express those changes in configuration. Before you rename a field, check what the instrumented service actually emits. For example, older HTTP instrumentation can use http.status_code, while newer instrumentation can emit http.response.status_code.

What is in the pipeline?

A trace has spans and span events. Metrics have data points. Logs have records. The OTTL context selects the object being processed and the paths it can access. A span statement can read both that span and its resource. Resource attributes are shared, so changing one can affect other records under the same resource.

How to read an OTTL statement

Statements

A transform statement invokes an editor and can add a where clause. Statements run in order within a group. These examples use the span context in Collector Contrib v0.162.0.

set(span.attributes["app.http.response_class"], "2xx") where IsInt(span.attributes["http.response.status_code"]) and span.attributes["http.response.status_code"] >= 200 and span.attributes["http.response.status_code"] < 300
delete_key(span.attributes, "http.request.header.authorization")

Expressions and literals

OTTL supports Boolean, numeric, and string literals, comparisons such as == and !=, and operators such as and and or. Enums come from the context. A span can use STATUS_CODE_UNSET, for example. Use IsInt before comparing an attribute numerically when its type can vary.

Paths and context

span.attributes["key"] reads an attribute on the current span. resource.attributes["service.name"] reads the resource attribute. span.name and span.status.code address built-in span fields. The transform processor can infer a context from prefixed paths, or you can set context on a group. The paths, functions, and enums in that group must be available in the chosen context.

Conditions

A statement runs when its where clause evaluates to true. A group can also have shared conditions. Multiple entries in that list are combined with OR. Put tests that must all pass in the same condition and join them with and.

A transform for HTTP spans

The first group below corrects one known legacy mapping. An upstream component sets the example attribute app.legacy_http_status_error only when it knows the old rule set Error solely because a server returned a 4xx. Enable that group only if your pipeline has this marker contract. Under the standard HTTP conventions, when instrumentation has no additional context, server-side 4xx span status stays Unset. Client-side 4xx should be Error. HTTP span status conventions

processors:
  transform:
    error_mode: ignore
    trace_statements:
      - context: span
        conditions:
          - >-
            span.kind == SPAN_KIND_SERVER
            and span.status.code == STATUS_CODE_ERROR
            and span.attributes["app.legacy_http_status_error"] == true
            and IsInt(span.attributes["http.response.status_code"])
            and span.attributes["http.response.status_code"] >= 400
            and span.attributes["http.response.status_code"] < 500
        statements:
          - set(span.status.message, "")
          - set(span.status.code, STATUS_CODE_UNSET)
      - context: span
        statements:
          - delete_key(span.attributes, "http.request.header.cookie")
          - set(span.cache["route_parts"], ExtractPatterns(span.attributes["url.path"], "^/api/(?P<group>orders|customers)(/|$)")) where IsString(span.attributes["url.path"])
          - set(span.attributes["app.route_group"], span.cache["route_parts"]["group"]) where span.cache["route_parts"]["group"] != nil

The second group removes one span attribute. A cookie copied into a resource, a span event, or a log body needs separate handling. ExtractPatterns produces a map of named matches and returns an empty map when the pattern does not match. The next statement reads group into a custom string attribute. That keeps both the map and the raw URL out of http.route.

Add transform to the actual service.pipelines.traces.processors list to run these rules. For more recipes, see the OTTL patterns guide (Simplified Chinese).

Decide where to transform the data

Existing Collector rules can keep running before export. If you process logs in Guance Pipeline on DataKit, implement the changes in Platypus and verify them with the same records. Check field removal, type conversion, and failure handling one by one. Assign a stage to each redaction and rename, then query what the backend received. Removing a value later leaves the original value visible to earlier stages.

Frequently asked questions

How does OTTL relate to the filter processor? OTTL is the language. The transform processor uses it to edit fields, and the filter processor evaluates conditions and drops matching telemetry, such as spans or logs. If the filter uses a field you just rewrote, place transform ahead of filter in that pipeline.

What does OTTL cost to run? It depends on the operation and how often it runs. A regular expression or JSON parse on every record consumes CPU. Compare throughput, processing latency, and Collector resource use when changing a rule or moving it to another stage.

What happens when a statement fails? In this version, the transform processor defaults to ignore, which logs the runtime error and continues. silent continues without that log. propagate returns the error to the pipeline and causes the payload to be dropped. A statement that cannot be parsed is a configuration error. With ignore, verify failed redaction cases to see which values remain.

Can Guance Pipeline run OTTL? Guance Pipeline uses its own Platypus scripting syntax. Rewrite the rule using the relevant functions and data types, then compare the output on matching sample records.

Further reading

References: OTTL language reference, v0.162.0 · Guance OpenTelemetry integration · DataKit Pipeline · DataKit documentation

Get a tailored plan

Contact us

Join the community

Scan with WeChat
to join the community

Try Guance

Start online and pay only for what you use.

Get started

Choose a Guance plan

Code hosting