Make Logstash read a file again from the beginning

To replay a file, Logstash must see it without an existing saved position and start at its beginning. start_position => "beginning" alone does not override a sincedb record.

Replay a file in an isolated test

On Unix systems, this configuration prevents sincedb positions from surviving between test-process runs:

input {
  file {
    path => ["/var/logs/my_logfile"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
output {
  stdout { codec => rubydebug }
}

Use a fresh test process, separate from the production service. The plugin still tracks positions in memory while running; /dev/null does not make it read the same unchanged file repeatedly. Tail mode also needs a newline to emit the final line.

On Windows, NUL is the null-device convention used for this kind of test; verify the plugin and path behavior on the deployed platform. A dedicated, disposable sincedb file is another option when you need predictable state management.

Reset a particular production input deliberately

First identify its actual sincedb_path, or its default state under <path.data>/plugins/inputs/file. Stop the affected pipeline before moving that state aside, because a running process can write its in-memory positions back to disk.

sudo systemctl stop logstash
# Replace this path with the verified state file for the intended input.
sudo mv /var/lib/logstash/plugins/inputs/file/.sincedb_verified \
  /var/lib/logstash/plugins/inputs/file/.sincedb_verified.backup
# Confirm start_position is set to beginning before restarting.
sudo systemctl start logstash

Do not use a recursive delete of all sincedb files. One state file may track several source files, so resetting it can replay more than the file you had in mind. For a one-file replay, a separate input restricted to that file and a separate state path is easier to reason about.

Account for duplicates

Replayed events can create duplicate downstream documents. Send a trial replay to an isolated destination or use a stable document identity appropriate to the data; do not delete production indices as a routine replay step.

Sincedb layouts have changed over time and can contain more than the historical four columns. Avoid editing a record based only on an old format example. Restore normal persistent tracking after the replay so future restarts resume instead of importing the same history again.

References

• Elastic: inputs file