Redact sensitive data in the OpenTelemetry Collector
Remove sensitive headers and SQL attributes, mask allowed values, and handle log bodies with Collector attributes, redaction, and transform processors.
A checkout span can carry an Authorization header while the log for the same request embeds a token in its message. Delete the span attribute and the log can still send the token. Start by checking where the value appears and which pipelines export it.
To redact sensitive data with the OpenTelemetry Collector, collect less at the source, then apply rules to every relevant path before export. Preserve the fields responders use, such as the route template and HTTP status. Test what actually reaches the sink with synthetic inputs. The examples here use Collector Contrib v0.165.0.
Find where the sensitive data lives
A trace may have attributes on a span, its events, its Resource, or its instrumentation Scope. Logs can carry a second copy in their body, and metrics can expose customer identifiers as dimensions. Inspect the actual payload for each signal. Header capture and SQL text depend on the instrumentation and configuration.
Keep raw headers, full URLs with query strings, and request bodies out of telemetry unless there is a specific reason to collect them. A route template and a status code often provide enough context to investigate an HTTP failure. For SQL, check both the current db.query.text and the legacy db.statement field.
Delete known record attributes with attributes
Existing pipelines can use attributes to remove exact keys. It is beta for traces, logs, and metrics in v0.165.0, and the upstream project recommends transform for new configurations. This rule deletes the two matching record attributes. Copies in a Resource or a log body need a rule for that location.
processors:
attributes/remove_credentials:
actions:
- key: http.request.header.authorization
action: delete
- key: http.request.header.cookie
action: delete
If you need to correlate records after removing a user ID, consider the exposure of the replacement. An unkeyed hash of a short or predictable ID is easy to guess by hashing candidates. Hashing can preserve a join without making the data anonymous. Attributes processor
Use redaction to control which keys leave
At a trace boundary, an allowlist limits what leaves even when an application adds a new field. The example keeps service name, method, route, and status. It also keeps a controlled app.diagnostic string to show how a token in an allowed value can still be masked. Omit that key unless your responders need it.
processors:
redaction/egress:
allow_all_keys: false
allowed_keys:
- service.name
- http.request.method
- http.route
- http.response.status_code
- app.diagnostic
blocked_values:
- '(?i)token=[^&[:space:]]+'
summary: silent
With allow_all_keys: false, an unlisted key is removed before the value check. With true, keys pass the allowlist check and matching values are still masked. ignored_keys and ignored_key_patterns bypass redaction. If an allowed_values pattern matches a retained attribute, the whole value bypasses masking. Keep those exceptions narrow. summary: silent prevents this processor from adding redaction counts or key names to the output.
In v0.165.0, redaction is in the contrib and k8s distributions. It is beta for traces and alpha for logs and metrics. The allowlist also affects Resource and Scope attributes. A structured log body can lose unlisted keys, while a text body may pass through if no value pattern matches. Review the fields for each signal. Removing metric dimensions can change series identity. Redaction processor · Implementation
Use OTTL when the field is inside the body
With transform, the rule can name the context explicitly. Here the trace rule deletes both SQL attributes instead of attempting to parameterize arbitrary SQL. The log rules delete a Resource attribute, remove a top-level key from a map body, and replace a string body when it cannot be sent as-is.
processors:
transform/remove_sensitive_data:
error_mode: propagate
trace_statements:
- delete_key(span.attributes, "db.query.text")
- delete_key(span.attributes, "db.statement")
log_statements:
- delete_key(resource.attributes, "deployment.customer_email")
- delete_key(log.body, "password") where IsMap(log.body)
- set(log.body, "[text body removed]") where IsString(log.body)
A nested password needs a rule for its actual path. JSON in a string stays a string until you parse it. This example deliberately removes the string body instead of assuming the parser will always succeed. With error_mode: propagate, a failed statement returns an error and the affected payload is dropped. Watch for that loss when rolling out the rules. Transform processor · OTTL guide
Check the entire route to the exporter
Put the processor in each applicable service.pipelines.<signal>.processors list. When you combine transformations with an egress allowlist, normalize the fields before the allowlist, then batch and export. Inspect synthetic records for an ordinary request, a nested object, and a parse failure. Check a second exporter or pipeline fed by the same receiver as well. Detailed debug logs can print the original OTTL context, so leave those logs off when processing real telemetry.
If DataKit processes logs before upload to Guance, use the Pipeline debugger to check the resulting fields and the original message. A field can disappear from the parsed attributes while the raw message still contains it. Collector and DataKit rules run in different places. Check the one that actually handles the data you export.
If a credential has already reached a backend, stop further ingestion of that value and follow the incident process for rotation and stored copies. Changing the Collector configuration affects subsequent data. Existing exports, retained records, and backups have their own handling procedures.
Related reading · Monitor the OpenTelemetry Collector · OpenTelemetry semantic conventions
Contact us
Join the community
to join the community
Try Guance
Start online and pay only for what you use.
Get startedChoose a Guance plan
