Understand rsyslog repeated-message reduction

Rsyslog's repeated-message reduction can replace duplicate events with a summary such as last message repeated 47 times. This may make a text log easier to browse, but it removes the individual records that automated counting and incident analysis often need.

Enable it only for a defined reason

The legacy configuration directive is:

$RepeatedMsgReduction on

To preserve individual events, use:

$RepeatedMsgReduction off

Check the effective configuration and validate changes with rsyslogd -N1 before restarting the service. The documented upstream default is off; distribution configuration can override it.

A reduced sequence may look like:

Jan 15 10:30:00 host app: connection failed
Jan 15 10:30:05 host rsyslogd: last message repeated 47 times

Treat this as an illustration of the summary, not a fixed wire format or flush interval.

Check the version and input implementation

The rsyslog reference describes duplicate identification using message text and origin fields such as hostname, process ID, and application name. Looking at message text alone is insufficient.

Before rsyslog 7.3.2, reduction operated on the output side; newer behavior is input-oriented. Do not assume that every modern input accepts a universal compressRepeatedMsgs parameter. In particular, it is not a documented imfile option. Use the options supported by the installed input module and confirm the result with a controlled repeated-event test.

Understand what downstream tools lose

If reduction occurs before forwarding, the receiver cannot reconstruct the original timestamps and context of each suppressed event. A search for the original message may find only its first occurrence, and a count of received records will understate the event count unless the analysis explicitly understands summaries.

This is especially important for alerts based on error rates. Two displayed lines do not necessarily mean two failures, and the summary's timestamp does not represent every suppressed occurrence.

Separate duplicate reduction from rate limiting

Application-side duplicate filters or rate limiters can reduce excessive logging at its source, but their matching and reset behavior must be defined. Preserve a meaningful count or metric if operators need the frequency of the underlying condition.

Journald rate limits are different again: RateLimitIntervalSec and RateLimitBurst limit message volume, rather than proving that dropped events are identical. Inspect those settings separately when investigating gaps. For machine analysis, keeping individual structured events and controlling unnecessary logging at the application is often easier to reason about than reconstructing summaries later.