Nginx logging: access logs, JSON formats, and error levels

Nginx writes two different kinds of log. An access log records the outcome of an HTTP request; an error log records events from request processing and the server itself. Use them together: the access log can show which requests returned 502, while the error log may explain why an upstream connection failed.

This guide covers log locations, custom access formats, JSON output, conditional logging, error levels, and collection in Guance. The examples follow Nginx's documented behavior; they are configuration examples, not performance measurements.

Find the active log destinations

Linux packages commonly use /var/log/nginx/access.log and /var/log/nginx/error.log, but these are not universal paths. A custom build, package, or server block can select another destination. Check the effective configuration:

sudo nginx -t
sudo nginx -T

The first command tests the configuration. The second also prints it, including included files, so you can locate the relevant access_log and error_log directives. Inspect the http, server, and location contexts: a more specific setting can change where a request is logged. Keep the printed configuration private if it contains credentials or sensitive settings.

The official Docker image routes its usual access and error log paths to stdout and stderr. With a container named nginx, start with:

docker logs -f nginx

For a container without a TTY and a logging driver that supports docker logs, shell redirection can separate the streams:

# Access output only; this also suppresses Docker CLI errors.
docker logs -f nginx 2>/dev/null

# Error output, including any Docker CLI errors.
docker logs -f nginx 1>/dev/null

Inspect the image and runtime configuration if those assumptions do not hold. A custom access_log destination can bypass the default symlink. Official Nginx image

Read the combined access format

Nginx's predefined combined format contains the client address, authenticated user, time, request line, response status, response-body size, referrer, and user agent. It does not include X-Forwarded-For by default. Some packaged configurations define an extended format that adds it. Nginx logging module

If a reverse proxy supplies client addresses, define which proxies are trusted before interpreting a forwarded header as the original client. A client can otherwise supply that header itself.

Define a custom format, then enable it

Declare log_format inside the http block. Reference its name in the access_log directive for the scope you want to log. This example adds the requested host:

# Inside http {}
log_format with_host '$remote_addr - [$time_local] "$host" "$request" '
                     '$status $body_bytes_sent "$http_user_agent"';
access_log /var/log/nginx/access.log with_host;

Defining the format alone does not change an existing access log. After editing the configuration, test it before reloading:

sudo nginx -t
sudo nginx -s reload

Then make a request and inspect a newly written line. Existing lines retain their previous format, so a collector may encounter both formats during the transition.

Write JSON without confusing request IDs and trace IDs

JSON makes field extraction more predictable than parsing a custom text line. Nginx's escape=json escapes characters inside variable values; you still supply the surrounding JSON syntax.

# Inside http {}
log_format json_access escape=json '{'
    '"time":"$time_iso8601",'
    '"remote_ip":"$remote_addr",'
    '"method":"$request_method",'
    '"uri":"$request_uri",'
    '"status":$status,'
    '"bytes_sent":$bytes_sent,'
    '"duration_sec":$request_time,'
    '"user_agent":"$http_user_agent",'
    '"request_id":"$http_x_request_id"'
'}';

access_log /var/log/nginx/access.log json_access;

Here, bytes_sent includes headers, unlike body_bytes_sent in the combined format. The status field is the HTTP status in this raw JSON; map it deliberately if your destination reserves that name for severity.

The received X-Request-Id is recorded as request_id. Do not rename an arbitrary request ID to trace_id: correlation requires the identifier used by your tracing instrumentation. Follow Guance's log and trace correlation guide to inject the appropriate fields.

Keep variables that can contain nonnumeric placeholders or multiple values quoted. For example, upstream timing variables can contain several entries after retries; inserting them as bare JSON numbers would break the record. Validate actual output, including a request with no upstream call and one with escaped characters.

Also consider what the URI contains. $request_uri includes the query string. If query parameters can contain credentials or personal information, use a suitable redacted representation or $uri when that meets your investigation needs. Structure makes data easier to query; it does not remove sensitive content.

Reduce volume with conditional logging

A map can turn the response status into a logging condition. Put this definition in http, and use if= on the log destination:

map $status $loggable {
    ~^[23]  0;
    default 1;
}

access_log /var/log/nginx/access.log combined if=$loggable;

This skips 2xx and 3xx responses. Other statuses remain eligible; it is more precise to describe the rule that way than as an exact 4xx/5xx filter. Use json_access instead of combined if you want the JSON format defined above.

The same mechanism can select paths or exclude known low-value traffic. A condition is not random sampling: if you need a statistical sample, define a sampling method separately and retain its rate with the data.

Dropping successful requests changes what you can measure from logs. You can count retained errors, but you cannot calculate an accurate overall error rate without a denominator from complete request metrics or another suitable source. Preserve successful traffic when it is needed for audit or incident reconstruction.

To disable access logging in a scope, use access_log off;. Do this only when the remaining telemetry covers the requests you need to investigate, including failures that never reach the application.

Choose error levels and destinations

The second argument to error_log sets the minimum severity to record:

error_log /var/log/nginx/error.log warn;

From least to most severe, the levels are debug, info, notice, warn, error, crit, alert, and emerg. The directive default is error; distribution and container configurations can choose another level. Inspect the deployed configuration rather than inferring it from the image name. Debug logging also requires a build with the relevant debug support. Nginx error_log reference

Error logs use Nginx's own text layout, typically including a timestamp, level, process/thread identifiers, and a message. Request and connection context appears when available. log_format changes access logs, not error logs.

Multiple destinations at the same level can retain different severity ranges:

error_log /var/log/nginx/error.log info;
error_log /var/log/nginx/emerg.log emerg;

An emergency message is eligible for both files; the first file does not stop at the next destination's threshold. Include rotation and retention for each destination so additional diagnostic output does not exhaust its filesystem.

Collect and inspect the fields in Guance

The DataKit Nginx integration documents a log configuration with files pointing to the access and error logs and pipeline = "nginx.p" selecting its parser. Its example extracted fields distinguish status_code, the HTTP status, from status, the log severity. Do not build a 5xx query against a severity field by mistake.

For container stdout/stderr, configure collection for the runtime's log source. For the custom JSON format above, use a Pipeline that matches that format and maps the timestamp, URI, numeric values, and severity correctly. A parser for a different text format is not interchangeable with JSON extraction. The DataKit logging guide describes the collection paths.

Test one successful request, a missing route, and a controlled upstream failure. Confirm the event timestamp and timezone, response code, source, service identity, and original message. Then build the query or chart, and only afterward set a monitor's evaluation window, threshold, and notification destination. See the monitor rule documentation.

Common configuration questions

How do request time and upstream response time differ?

$request_time covers the request as observed by Nginx, including client-side reading and sending. $upstream_response_time describes time spent receiving the upstream response. Retries can produce multiple upstream timings. A large difference is a reason to inspect client transfer, buffering, retries, and other processing; subtracting the two is not a definitive diagnosis of network latency. Nginx upstream variables

Why did defining log_format make no difference?

Check that access_log references the format name in the applicable configuration scope. log_format belongs in http, not directly in a server block. Test and reload the active configuration, then inspect a new request rather than an old line.

Can a container write both to stdout and a file?

Yes. Configure two access-log destinations at the same level, retaining the stdout destination and adding a path in a mounted directory. For example, add access_log /var/log/nginx/file_access.log; alongside the existing destination. Ensure the directory is writable and that rotation and disk limits have an owner. Two outputs also mean two potential collection paths; avoid ingesting the same event twice unintentionally.

Can Guance parse a combined text log?

Start with the Nginx integration's Pipeline example and verify it against your actual records. Extended formats, extra fields, or an altered timestamp can require changes. JSON also needs correct extraction and field mapping; a structured input alone does not configure the collector.

Related reading: Apache logging (Chinese) and microservices logging (Chinese).