Read RFC 3164 and RFC 5424 syslog messages

Two common syslog formats are the traditional BSD-style format described by RFC 3164 and the versioned format defined by RFC 5424. RFC 5424 provides more explicit fields and timestamps, but the sender and receiver must agree on what they actually exchange.

Read a traditional message

<134>Jan 15 10:30:00 myhost myapp[1234]: Request completed

The leading priority, 134, combines a facility with a severity: 16 × 8 + 6. Facility 16 is local0, and severity 6 is informational.

The timestamp contains a month, day, and clock time, but no year or timezone. myhost identifies the host; the familiar myapp[1234]: convention places a tag and process identifier before the message text. Do not assume every device uses that exact tag layout. Missing year and timezone context is especially troublesome when replaying old files or comparing hosts across regions. See RFC 3164.

Read a versioned message

<165>1 2024-01-15T10:30:00.123+08:00 myhost myapp 1234 ID47 [exampleSDID@32473 iut="3" eventSource="Application"] User login succeeded

After priority 165, version 1 identifies the RFC 5424 message structure. The header contains a timestamp, hostname, application name, process ID, and message ID. Here the priority is local4.notice: 20 × 8 + 5.

The structured-data element has an identifier, exampleSDID@32473, followed by named parameters. It is not simply an arbitrary [key="value"] list. A - represents absent structured data, and several header fields also permit a nil value. The message body follows the structured data when present.

The timestamp includes the year and UTC offset. Fractional seconds are optional; their presence does not establish the accuracy of the originating clock. RFC 5424 also defines encoding rules, including how a UTF-8 message is identified.

Interpret severity consistently

Facility describes a source category such as authentication, daemon, cron, or local0 through local7. It is separate from severity. Lower severity numbers represent more urgent conditions in syslog.

Validate the complete message path

A version number immediately after <PRI> is a stronger RFC 5424 clue than an ISO-looking date alone. Devices also emit variants, so inspect complete raw messages rather than inferring the format from a timestamp substring.

Transport and framing are separate choices. Agree on UDP, TCP, or TLS and on TCP message boundaries. When forwarding to DataKit, compare raw input with parsed timestamp, host, and severity fields using a real sample from each device family. Successful delivery does not prove that parsing preserved their meaning.