Should Filebeat send directly to Elasticsearch or through Logstash?

Send Filebeat directly to Elasticsearch when its processors and Elasticsearch ingest pipelines can perform the required work. Add Logstash when you need its plugins, centralized transformation logic, or multiple destinations. Traffic volume alone does not decide the architecture.

Start with a direct path when it fits

Filebeat → Elasticsearch ingest pipeline → Elasticsearch index or data stream

Filebeat can decode JSON, dissect predictable text, attach metadata, and drop unwanted events. Elasticsearch ingest pipelines can perform further processing, including grok parsing. This often suits structured application logs and container logs with a clear schema.

The direct path has fewer services to configure and operate. It still needs capacity planning: ingest processing consumes cluster resources, and expensive transformations can affect ingestion throughput. Test with representative events, including malformed records.

Add Logstash for specific requirements

Filebeat → Logstash → Elasticsearch

Logstash is useful for plugin-based enrichment, transformations that do not fit the ingest pipeline, and routing one event stream to several outputs. It introduces JVM processes, pipeline configuration, monitoring, and another recovery boundary. Persistent queues can improve resilience when configured, but they are not a substitute for deciding how long data must survive an outage.

Perform multiline assembly close to the source in Filebeat. Combining multiline records after streams from many hosts have converged can mix unrelated events; a need for stack-trace handling alone is not a reason to add Logstash.

Make recovery part of the decision

Filebeat retries unacknowledged delivery and applies backpressure. This does not guarantee zero loss during every outage. Files may rotate away before they are read, retained state may be lost, and acknowledged versus retried events can produce duplicates. Check source retention and queue durability for both designs.

Can you switch later?

Yes, but changing the output is only part of the migration. Filebeat configures one output type at a time. When switching to Logstash, also arrange compatible templates, ingest-pipeline loading or selection, credentials, TLS, and destination naming. Filebeat modules that depend on Elasticsearch ingest pipelines need those pipelines installed and invoked through the new path.

Compare Elastic's Filebeat output options, delivery behavior, and processor support before choosing the smallest architecture that meets your requirements.