Collect a text log with rsyslog’s imfile module

Rsyslog's imfile input reads text files and turns their contents into events that ordinary rsyslog rules can route. It does not automatically extract application fields such as a request ID or duration; use a parser or normalization stage when you need that structure.

Configure an input and a dedicated route

For rsyslog 8, add the following to a configuration included by the main file. Load imfile once and use the existing working directory if one is already configured:

global(workDirectory="/var/spool/rsyslog")
module(load="imfile")

ruleset(name="app_file_logs") {
  action(type="omfile" file="/var/log/myapp-collected.log")
}

input(type="imfile"
      File="/var/log/myapp/app.log"
      Tag="myapp:"
      Severity="info"
      Facility="local0"
      PersistStateInterval="100"
      Ruleset="app_file_logs")

The named ruleset keeps this file input separate from unrelated system logs. The tag supplies the program identifier; the facility and severity are assigned values, not levels parsed out of the application's message text.

Prepare readable source files and a writable state/output location, then validate and restart:

sudo rsyslogd -N1
sudo systemctl restart rsyslog

Append a non-sensitive test line to the source through the application and confirm it appears in the destination. Never include the destination file in the input wildcard, which would create a collection loop.

Keep reading positions across restarts

Modern imfile generates state-file names automatically. PersistStateInterval="100" requests periodic persistence after batches of processed lines; ensure the working directory is writable and survives the restart. A crash can still occur between checkpoints, so this is not an exactly-once guarantee. Avoid a manually shared StateFile name for wildcard inputs.

Without an existing state, the default freshStartTail="off" allows existing content to be read. Setting it to on skips initial historical content and has loss-related caveats. startmsg.regex controls event boundaries, not where historical reading begins. Do not delete state files casually: doing so can replay old events.

Combine multiline events deliberately

For logs whose events begin with a date such as 2026-10-19, add an input parameter like:

startmsg.regex="^[0-9]{4}-[0-9]{2}-[0-9]{2}"

Continuation lines are grouped until another starting line arrives. Consider the documented readTimeout behavior if a final event must flush without a following event. Choose startmsg.regex or the appropriate readMode strategy; do not combine incompatible multiline modes. Embedded line breaks may be escaped in the resulting syslog message.

Handle rotation and forwarding

Rename-and-create rotation and in-place truncation are different. reopenOnTruncate is an experimental option for a file whose inode stays the same while its size shrinks; it is not a generic requirement for renamed files. Test the application's actual rotation method, including restarts, permissions on the new file, and state persistence. A HUP signal is not a universal repair for input tracking.

To forward only this input, replace the action inside app_file_logs with a configured omfwd TCP action. Add authentication/TLS and a queue policy as required; a global *.* forwarding rule can unintentionally include every system event.

DataKit log collection is another file-collection path if Guance is the intended destination. Assign collection ownership clearly so two collectors do not ingest the same file twice.

References: imfile, first-start behavior, multiline boundaries, and truncate handling.