Exclude selected container logs from Filebeat

Choose where to exclude the data: skip matching files before reading them, drop unwanted events after metadata enrichment, or avoid starting an autodiscovered input for the container. These methods have different collection costs.

Exclude files by path

For a Docker deployment using JSON log files, a current filestream input can use a scanner exclusion:

filebeat.inputs:
  - type: filestream
    id: docker-json-logs
    paths: ['/var/lib/docker/containers/*/*-json.log']
    prospector.scanner.exclude_files:
      - '/container-id-to-ignore/'
    parsers:
      - container:
          stream: all
          format: docker

Replace the placeholder with the actual container ID. These are regular expressions, not shell globs: a leading * is not a valid way to say “any prefix.” Container IDs change on recreation, so names or labels can be more maintainable.

The old log and container inputs are deprecated and disabled in newer releases. Do not copy their option layout unchanged into filestream.

Drop events after adding Docker metadata

For events read from Docker log paths, the following processors enrich the event before testing its metadata:

processors:
  - add_docker_metadata: ~
  - drop_event:
      when:
        or:
          - equals:
              container.name: 'my-noisy-container'
          - contains:
              container.image.name: 'debug-image'

The Docker metadata processor needs access to the Docker API, usually through the socket. Verify the actual enriched fields and name values before filtering. A missing metadata field will not match these conditions.

Dropping occurs before downstream output, reducing sent events. Filebeat still reads and processes those logs; a file or discovery exclusion avoids more work earlier.

Exclude Kubernetes namespaces during discovery

This template starts inputs only for containers outside kube-system:

filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            not:
              equals:
                kubernetes.namespace: 'kube-system'
          config:
            - type: filestream
              id: 'container-${data.kubernetes.container.id}'
              prospector.scanner.symlinks: true
              paths:
                - '/var/log/containers/*-${data.kubernetes.container.id}.log'
              parsers:
                - container: ~

The deployment also needs appropriate RBAC, host log mounts, and node scoping where applicable. Avoid another input or hints configuration that independently collects the excluded files. For label rules, use the metadata field names exposed to autodiscover, including its documented handling of dotted labels.

See Elastic's filestream input and autodiscover guide.