Reread log files with Filebeat 5 safely

Filebeat 5 remembers file positions in its registry. To replay logs, use a fresh registry for a controlled replay or, during a planned stop, move the old registry aside. Keep the original state and send replayed data to an explicitly chosen destination. This procedure is for Filebeat 5, an obsolete release; modern registry formats and input behavior differ.

Prefer an isolated replay

Copy the required logs to a replay directory and use a separate Filebeat 5 configuration with narrowly scoped paths, separate path.data, and a separate replay index or output route. Do not run two instances against the same data directory.

For the replay prospector, review these Filebeat 5 settings:

filebeat.prospectors:
  - input_type: log
    paths: ['/srv/log-replay/*.log']
    tail_files: false
    ignore_older: 0

Use a new state directory so previously recorded offsets do not override the intended starting position. A fresh registry alone does not bypass path filters, file permissions, encoding, or the requirement for complete log lines.

If resetting the existing Filebeat 5 registry

First confirm the installed version, path.data, and filebeat.registry_file. The default registry is ${path.data}/registry; a package installation often places it under /var/lib/filebeat. Stop the service and verify that no other instance uses that state.

The following example moves a verified Filebeat 5 registry file to a backup rather than deleting it:

sudo service filebeat stop
# Confirm the process has stopped and this is the configured registry file.
sudo test -f /var/lib/filebeat/registry
sudo mv /var/lib/filebeat/registry \
  "/var/lib/filebeat/registry.before-replay.$(date +%Y%m%d%H%M%S)"
# Start only after reviewing input paths and the replay destination.
sudo service filebeat start

The commands are a maintenance sequence, not an unattended script: stop if the checks do not match your installation. Never apply a recursive registry deletion command to an unknown version.

Account for duplicates and old files

Replaying can index the same event again and retrigger alerts. Use a separate destination or a tested deterministic document-ID strategy. Do not delete the production index as a routine prerequisite. Restoring an old registry later does not undo documents already written.

ignore_older controls file age eligibility and defaults to disabled in Filebeat 5. clean_inactive removes old state; it is not itself an age filter for log content. clean_removed is state cleanup, not a targeted replay command. For one file, an isolated replay is easier to reason about than manually editing registry internals.

References: Filebeat 5 registry settings and prospector settings.