Fix an Apache timestamp that Logstash does not parse

When an Apache log keeps its ingestion timestamp, first inspect the event between Grok and the date filter. The extracted field name, its exact value, and the date pattern must agree. A missing input field and an invalid date are different failures.

Extract a known timestamp field

For this access-log line:

127.0.0.1 - - [25/Oct/2024:10:15:00 +0800] "GET /index.html HTTP/1.1" 200 2326

Use an explicit legacy pattern mode so the example's timestamp field is predictable:

filter {
  grok {
    ecs_compatibility => "disabled"
    match => { "message" => "^%{COMMONAPACHELOG}$" }
  }
  if [timestamp] {
    date {
      match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
      locale => "en"
      target => "@timestamp"
    }
  } else {
    mutate { add_tag => ["missing_apache_timestamp"] }
  }
}
output {
  stdout { codec => rubydebug }
}

Composite Grok patterns can produce different field paths in ECS mode. Do not assume a field name from a legacy example applies to your pipeline. You can also capture %{HTTPDATE:apache_timestamp} yourself and use that exact field in match.

Match the value, not the surrounding log syntax

match starts with the source field, followed by one or more candidate formats. Here, MMM means the month name, MM means the numeric month, mm means minutes, and Z parses the numeric UTC offset. The extracted value should not still contain the square brackets.

A failed date conversion normally adds _dateparsefailure and leaves the existing @timestamp unchanged. That existing value may have been set by an input or an upstream shipper; it is not necessarily Logstash arrival time. A missing field may skip parsing entirely, so absence of the failure tag does not prove success.

Separate parsing from display time zones

The sample represents 2024-10-25T02:15:00Z. A UTC display is eight hours earlier than its +0800 local representation, while referring to the same instant.

For a source timestamp without an offset, specify its known zone, such as timezone => "Asia/Shanghai"; otherwise the platform default applies. Do not compensate for a display-zone difference by shifting an already correct timestamp.

Multiple source formats can be tried in order:

match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z", "ISO8601", "UNIX"]

Inspect _grokparsefailure and _groktimeout as well as date failures. Keep the original message until extraction and conversion have been checked against representative samples.

References: date filter and Grok ECS compatibility.