Filebeat vs. Logstash for sending logs to Elasticsearch
Compare Filebeat collection with Logstash processing, including parsing, outputs, backpressure, and the limits of delivery guarantees.
Filebeat reads log files close to the application and ships events downstream. Logstash provides a processing pipeline with a broad plugin ecosystem. They often work together, but a Logstash tier is optional when Filebeat and Elasticsearch ingest pipelines already cover the required transformations.
Compare the responsibilities
| Concern | Filebeat | Logstash |
|---|---|---|
| Typical placement | On application hosts or container nodes | On dedicated processing hosts, or locally when appropriate |
| Main work | Follow files, track read positions, enrich and filter events | Parse, transform, enrich, and route events |
| Processing | JSON decoding, dissect, metadata, and other processors | Grok, mutate, JDBC lookups, Ruby, and other plugins |
| Resource planning | Depends on inputs, queues, and processing | Depends on JVM sizing, plugins, queues, and throughput |
| Outputs | One configured output type | Multiple outputs can be configured |
| Backpressure | Slows or pauses collection when downstream cannot keep up | Queues and pipeline backpressure; persistent queues are optional |
Avoid fixed memory estimates such as “tens of megabytes versus hundreds.” Measure the actual configuration. Filebeat does useful processing itself, but a grok entry is not a supported Filebeat processor. Grok belongs in an Elasticsearch ingest pipeline or Logstash filter.
Use them together when central processing helps
Application files → Filebeat → Logstash → Elasticsearch → Kibana
This arrangement centralizes transformations and can route events to multiple destinations. For structured JSON with modest processing requirements, sending Filebeat directly to Elasticsearch reduces the number of services to operate. Elasticsearch ingest pipelines can handle additional parsing at ingestion.
Understand retries and duplicates
Filebeat tracks delivery acknowledgments and retries unacknowledged events. That supports at-least-once delivery under the documented conditions, not a promise of zero loss or exactly-once indexing. A retry can duplicate an event. If an outage outlasts source-file retention, unread data can disappear before delivery; queue durability also depends on configuration.
Size retention and buffers for the outage you need to survive, test recovery, and decide whether stable document IDs are needed for deduplication. Elastic documents Filebeat's delivery behavior and its output configuration.
Contact us
Join the community
to join the community
Try Guance
Start online and pay only for what you use.
Get startedChoose a Guance plan