How to apply conditional logic in Fluentd

Fluentd expresses most conditions through plugins: use grep to keep or discard records, rewrite_tag_filter to route them, and record_transformer to calculate a field. There is no general if/else block wrapping arbitrary Fluentd configuration sections.

Route events by their contents

Install fluent-plugin-rewrite-tag-filter in the same environment as Fluentd. This example sends ERROR and FATAL records to an alert collector, drops DEBUG records, and retains other levels in a separate route:

<match app.**>
  @type rewrite_tag_filter
  <rule>
    key level
    pattern /^(ERROR|FATAL)$/
    tag alert.${tag}
  </rule>
  <rule>
    key level
    pattern /^DEBUG$/
    tag drop.${tag}
  </rule>
  <rule>
    key level
    pattern /^DEBUG$/
    invert true
    tag normal.${tag}
  </rule>
</match>

<match alert.**>
  @type forward
  <server>
    host alerts.internal
    port 24224
  </server>
</match>

<match drop.**>
  @type null
</match>

<match normal.**>
  @type stdout
</match>

Use your real collector address and secure the connection for production. Rules run in order; an event is re-emitted with the first selected tag. The new prefixes deliberately avoid matching app.** again. Without a suitable final rule, unmatched records can disappear. Test missing and empty level values as well as the expected levels.

Keep only matching records

For a stream of HTTP responses, the following filter retains 5xx statuses. Place it before the output that consumes app.**:

<filter app.**>
  @type grep
  <regexp>
    key status
    pattern /^5\d\d$/
  </regexp>
</filter>

Use <exclude> to discard a match instead. Multiple ordinary <regexp> sections require all conditions to match; explicit <and> and <or> groups are available in supported Fluentd versions.

Assign a field conditionally

Ruby expressions that call methods or use a ternary operator require enable_ruby true:

<filter app.**>
  @type record_transformer
  enable_ruby true
  <record>
    severity ${record["status"].to_i >= 500 && record["status"].to_i < 600 ? "critical" : "normal"}
  </record>
</filter>

This assumes status contains an HTTP code. Missing or invalid values convert to zero here; validate them separately if they should be rejected rather than classified as normal.

Order conditions for correctness first

Put specific routing rules before broad ones. Moving a common condition earlier can change which overlapping rule wins. Reduce unnecessary regex work only after checking representative records and the final destination of each branch.

References: tag rewriting, grep, and record transformation.