Log formatting best practices for production applications
Build searchable JSON logs with one event per line, explicit timestamps, consistent fields, trace correlation, and safe object logging.
At 03:14 UTC, a payment request timed out. The gateway recorded the route and response time in milliseconds. The payment service wrote "upstream timeout" and a stack trace on separate lines. Both records reached the log store. The route field was searchable in one service, the stack trace appeared as several events, and the two records had different identifiers. The collection path worked. The event format made the investigation harder.
Log formatting defines how each event is encoded and which fields a reader can rely on. Set the event boundary and the schema together. A line can be valid JSON and still be hard to query if one service emits duration_ms as a number and another emits "900ms" as text.
Choose the event boundary before the field names
Free text is useful when a developer reads one process at a time. A semi-structured line adds a regular prefix or key-value pairs. A structured event gives the reader a defined set of fields and types. The following records all describe the same kind of timeout.
| Format | Example | What the collector needs |
|---|---|---|
| Free text | Upstream timed out after 900 ms |
A parser to extract a duration and a unit |
| Semi-structured | level=ERROR duration_ms=900 msg="Upstream timed out" |
A parser that understands the key-value syntax and quoting |
| Structured | {"level":"error","duration_ms":900,"message":"Upstream timed out"} |
A JSON parser and an agreed schema |
For a line-oriented file or stdout stream, use a JSON encoder to emit one object per physical line. Quotes, backslashes, and newlines within the message must be escaped. Pretty-printed JSON spans multiple physical lines and needs different framing. A development console can use a human-readable formatter while the production file keeps the same event fields. JSON strings and escaping
A formatter can help at each source. Go's slog.JSONHandler writes line-delimited JSON. In Nginx, define the JSON object in log_format, set escape=json for variable values, then select that format in access_log. For PostgreSQL 15 and later, log_destination = 'jsonlog' also requires logging_collector. Each formatter produces its own fields. Check the resulting records before mapping them to the common schema. Go slog · Nginx log module · PostgreSQL logging
Keep severity and event time explicit
Define a common severity vocabulary, such as debug, info, warning, and error. Map each producer's values into it. For example, WARN and warning should end up in the same filter if they mean the same thing. Numeric levels differ across libraries. OpenTelemetry defines its own SeverityNumber scale and a separate SeverityText; map from the source rather than treating its numbers as portable. OpenTelemetry logs data model
Record when the event occurred. An RFC 3339 timestamp with Z or an explicit offset identifies the instant. An epoch value also works if the schema specifies seconds, milliseconds, or nanoseconds. Keep observed or ingestion time separate when the pipeline exposes it. A record delivered 30 seconds late should still appear in the event window where the timeout happened. UTC makes offsets consistent. It cannot correct a host clock that runs fast. RFC 3339
Give every event enough context to narrow the search
Make service and environment available on every event. Add an instance identifier if it helps distinguish replicas. Keep the application release or commit on the event or in resource metadata that survives ingestion. If release A and release B are both serving traffic, the version lets you check whether timeouts started in one rollout cohort. File, function, and line are useful for selected paths; choose whether to include them at the logger or handler.
Attach a route template such as /orders/{order_id} and record elapsed time as a number in duration_ms. Keep names, types, and units stable between services. Do not build field names from input such as HTTP header names or order IDs. Store correlation IDs as values of fixed fields. Check what happens if a producer omits a field or changes its type before shipping a schema change.
Use the trace context you actually have
An application request_id helps retrieve records from components that carry the same value. A trace ID identifies a distributed trace; a span ID identifies a particular operation in it. When there is an active span, inject its current identifiers using the instrumentation library. Propagate trace context through the transport so the next service gets the same trace ID. Without an active span, an application request ID can still help. A search only finds the events that reached the selected workspace and retention window. The matching trace might also be absent because of sampling or retention. W3C Trace Context
The following synthetic record is one line of JSON. The escaped newline keeps the stack text inside the event.
{"time":"2026-11-09T03:14:00Z","level":"error","service":"payments","env":"prod","version":"v2.3.1","route":"/orders/{order_id}","duration_ms":900,"request_id":"req-example-42","trace_id":"4bf92f3577b34da6a3ce929d0e0e4736","span_id":"00f067aa0ba902b7","message":"Upstream returned \"timeout\"","error_type":"UpstreamTimeout","stack_trace":"UpstreamTimeout\n at capture (payments.go:71)"}
Capture a stack for exceptions where the call path helps. An expected business rejection such as an expired quote may only need a stable error code and context. Use frame fields when readers need to aggregate by file or function. If a legacy logger writes a plain-text stack across lines, configure the collector to recognize the first line of each event and verify that the frames are attached to the preceding error. DataKit multiline log collection
Decide which fields may leave the process
Build an allowlist for objects written to logs. Prefer a bounded error type and the identifiers needed for investigation over a full user object, request body, or header map. In Go, a value can implement slog.LogValuer to choose its logged representation. Test the serialized result. A separate formatter, error string, raw message, or stack trace may still contain the data you intended to omit. Keep credentials and sensitive fields out of messages at the point of generation. OWASP logging guidance
Before rolling a format out, send a normal event, an exception with quotes and newlines, and a record with a missing field through the same parser. In Guance, use the DataKit file collector and a Pipeline to extract or map fields such as the application severity to status. Inspect the parsed event time and field types in the log explorer, and check the raw message for sensitive values. With APM and log collection both configured, confirm that the parsed trace_id matches an available trace. DataKit log collector · Guance log and trace correlation
Once the format is reliable, log aggregation covers how to bring records from separate systems into a common destination. The structured logging guide in Simplified Chinese explains the broader approach.
Contact us
Join the community
to join the community
Try Guance
Start online and pay only for what you use.
Get startedChoose a Guance plan
