Route Docker stdout and stderr separately with Fluentd

Fluentd can route container stdout and stderr separately, but the field name depends on the input. Docker's Fluentd logging driver supplies source: stdout or source: stderr. Docker's json-file format uses stream. Kubernetes CRI log files use a different line format and must be parsed accordingly.

Neither stream is a reliable severity label by itself. Applications can write informational diagnostics to stderr or errors to stdout; retain the parsed application level when one is available.

Receive Docker driver events

This source accepts the Fluentd forward protocol on a local host endpoint:

<source>
  @type forward
  bind 127.0.0.1
  port 24224
</source>

For a native Linux setup where Fluentd and the Docker daemon share the host network namespace, a small test container can send both streams:

docker run --rm \
  --log-driver fluentd \
  --log-opt fluentd-address=127.0.0.1:24224 \
  --log-opt 'tag=docker.{{.ID}}' \
  alpine:3.22 sh -c 'echo stdout-example; echo stderr-example >&2'

Do not add -t: a TTY can combine the two streams before the collector sees them. The daemon, not the application container, connects to Fluentd. On Desktop, rootless Docker, or a containerized collector, choose an address reachable from the daemon's network environment. A Compose service name is not automatically resolvable by that daemon.

For a containerized Fluentd listener, its bind address and published host port need corresponding configuration. Restrict who can send logs; exposing an unauthenticated forward listener is not required by this example. Without asynchronous connection mode, an unavailable collector can prevent container startup. Async buffering has finite capacity and does not guarantee lossless delivery.

Normalize the stream and retag once

Install or verify the fluent-plugin-rewrite-tag-filter plugin in the actual Fluentd distribution. It is not guaranteed to exist in every image. These sections accept either already parsed source or stream fields:

<filter docker.**>
  @type record_transformer
  enable_ruby true
  <record>
    stream_route ${["stdout", "stderr"].include?(record["source"] || record["stream"]) ? (record["source"] || record["stream"]) : "unknown"}
  </record>
</filter>

<match docker.**>
  @type rewrite_tag_filter
  <rule>
    key stream_route
    pattern /\A(stdout|stderr|unknown)\z/
    tag $1.${tag}
  </rule>
</match>

<match stdout.**>
  @type file
  path /var/log/fluent/stdout
  append true
</match>

<match stderr.**>
  @type file
  path /var/log/fluent/stderr
  append true
</match>

<match unknown.**>
  @type file
  path /var/log/fluent/unknown
  append true
</match>

The rewritten tags start with stdout., stderr., or unknown., so they do not re-enter the docker.** rule. Keep broader catch-all matches after these routes. Unknown or missing stream values remain visible instead of silently disappearing.

Provide writable, persistent paths for the output files and buffers, configure retention and buffering limits, and wait for the configured flush interval when testing. File output can add time-based suffixes; the path setting is not necessarily the exact final filename.

Replace the file outputs with configured Elasticsearch outputs if separate indices are required. That needs the output plugin, endpoint, authentication, TLS, buffering, and index policy. Name indices by stream rather than claiming that every stderr record is an error or every stdout record is an access log. To archive all records and additionally alert on selected events, use a deliberate copy/routing design; the first matching output is not automatically a broadcast.

File inputs and Kubernetes differ

For a supported collector-exported JSON file or a test fixture containing Docker json-file records, a tail source needs a JSON parser, a persistent pos_file, and the chosen initial read policy. Preserve its stream field for the routing filter. Do not make external manipulation of Docker's internally managed json-file storage your collection contract; Docker documents those files as daemon-managed.

Kubernetes commonly exposes container-log symlinks under /var/log/containers, but the target records may use CRI format rather than JSON. Use a compatible parser and handle partial-record markers before application multiline reconstruction. Kubernetes metadata enrichment is a separate plugin/configuration step. Preserve container and stream identity so messages from different workloads are not combined.

Verify one known stdout event, one stderr event, an unknown-stream fixture, and collector restart behavior before enabling filtering or dropping stdout. Once a TTY or another upstream stage has merged the streams, retagging cannot reconstruct their original identities.

See Docker's Fluentd driver, Fluentd tag rewriting, and Kubernetes logging formats.