Debug a Logstash pipeline with configuration checks and event output

Debug in three stages: check that the configuration loads, inspect a representative event after filtering, then use runtime logs to investigate what remains. Each stage answers a different question.

Validate the configuration

bin/logstash --config.test_and_exit --path.config /path/to/logstash.conf

A successful result confirms configuration validation, not end-to-end correctness. Grok pattern correctness is not covered by this mode, and a reachable destination or appropriate file permissions still need runtime checks.

Inspect the processed event

Temporarily use a stdout output in a test pipeline:

output {
  stdout {
    codec => rubydebug { metadata => true }
  }
}

Check whether the expected fields exist, whether numeric values are actually numbers, and whether tags such as _grokparsefailure or _dateparsefailure appear. metadata => true also displays @metadata, which the codec normally hides.

For a single-line parser test, use input { stdin { } }, include the filters under test, and type a representative log line followed by Enter. Run the isolated configuration with its own path.data if the normal service is already running. Include malformed input as well as a successful example.

Turn on runtime diagnostics

bin/logstash -f /path/to/logstash.conf --log.level debug

This changes the overall log level; it does not restrict debugging to the file plugin. Use the relevant logger settings in log4j2.properties when you need narrower output. Avoid dumping secrets or sensitive events, and remember that config.debug can expose plaintext passwords from the compiled configuration.

Confirm the actual delivery path

Stdout output proves that an event reached that output. It does not prove another output accepted or indexed it. Check the destination separately, including mapping rejections, authentication, and retries.

Automatic reload is off unless enabled with --config.reload.automatic. A manual SIGHUP reload is available on supported Unix systems, but not Windows. After debugging, remove temporary stdout output and restore normal logging: printing every production event adds overhead and can create another sensitive log copy.

References

• Elastic: running logstash command line
• Elastic: logstash settings file
• Elastic: reloading config