Configure Log4j 2: JSON output, request context, and asynchronous logging

Log4j 2 separates the logging API from its implementation and provides configurable appenders, layouts, filters, and asynchronous logging. Those options are useful only when they match the application's workload and delivery requirements. A faster queue does not fix a slow destination, and a JSON layout does not automatically supply the request context needed for investigation.

This guide builds a small configuration, adds structured output and ThreadContext, and explains what to measure before enabling asynchronous and allocation-reducing options. It also covers collection checks in Guance.

The examples use Log4j 2, not the legacy Log4j 1 API. Log4j 1 reached end of life in 2015; its package names and configuration format are different. If an existing application still depends on it, follow Apache's Log4j 1 migration guide rather than copying a Log4j 2 configuration into the old runtime.

Keep the API and implementation aligned

For an application that calls the Log4j API directly, the basic Maven dependencies are:

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-api</artifactId>
  <version>${log4j.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-core</artifactId>
  <version>${log4j.version}</version>
</dependency>

Define log4j.version as a supported concrete release compatible with the application's runtime, or import the appropriate Log4j BOM and let it align the modules. An old version in a tutorial is not a recommendation to pin that version indefinitely.

Application code may instead use SLF4J. With SLF4J 2, log4j-slf4j2-impl routes that API to Log4j 2. Choose one intended provider and avoid routing loops—for example, combining a bridge from SLF4J to Log4j with one that sends Log4j back to SLF4J. Spring Boot users should normally use the corresponding starter and its dependency management.

Define the console output and root level

Place this configuration in src/main/resources/log4j2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{ISO8601} %-5level [%t] %logger - %msg %ex%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="info">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

Appenders defines destinations and formatting. Loggers defines levels and routing, and the root logger supplies the fallback. An appender must be referenced to receive events.

Inside an application class, the Log4j API can be used like this:

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class OrderService {
    private static final Logger log = LogManager.getLogger(OrderService.class);

    public void recordCreated(String orderId) {
        log.info("Order created orderId={}", orderId);
    }

    public void recordFailure(Exception failure) {
        log.error("Payment processing failed", failure);
    }
}

Passing the exception as a throwable lets the configured layout include its stack trace. The common levels, from less to more severe, are TRACE, DEBUG, INFO, WARN, ERROR, and FATAL. A logging call's severity does not itself determine whether the application exits.

Read PatternLayout expressions carefully

Log4j's logger precision is not Logback's character-length abbreviation rule. In Log4j, %logger{36} requests up to 36 name components; it does not mean a 36-character name. Choose the pattern based on the formatter actually in use. See Pattern Layout.

Timestamp names can also be misleading during collection. Confirm the actual serialized value and timezone rather than assuming that every pattern labeled ISO8601 includes an offset.

Use JSON Template Layout for an explicit event schema

Log4j offers both JSON Layout and JSON Template Layout. The former has its own Jackson dependencies and output options. JSON Template Layout is useful when the team wants direct control over fields and types through a template.

Add the matching module:

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-layout-template-json</artifactId>
  <version>${log4j.version}</version>
</dependency>

Replace the console appender in the earlier configuration with this one, keeping the referenced name:

<Console name="Console" target="SYSTEM_OUT">
  <JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json"/>
</Console>

The bundled EcsLayout.json models Elastic Common Schema, with fields such as @timestamp, log.level, and log.logger. The layout is usable with other destinations as well. A custom template can align those names with another agreed schema, but verify field types, exceptions, and ThreadContext output after changing it. See JSON Template Layout.

Do not build JSON by putting message text inside a PatternLayout JSON-looking string. Escaping, embedded newlines, and exception text need a real JSON serializer.

Scope request context to the work that owns it

ThreadContext provides a map and stack associated with the current thread. Layouts can include these values in events. For temporary request fields, use a scoped context:

import org.apache.logging.log4j.CloseableThreadContext;

try (CloseableThreadContext.Instance ignored = CloseableThreadContext
        .put("trace_id", traceId)
        .put("user_id", userId)) {
    log.info("Order processing completed");
}

This fragment assumes traceId, userId, and the logger are already available. CloseableThreadContext restores the previous values when the scope closes, avoiding a blanket clearAll() that would discard context belonging to an outer layer. See Thread Context.

A custom template must actually serialize the relevant context fields. To correlate logs with traces in Guance, use the tracing system's identifier and verify that the stored log and trace refer to the same request. A separately generated business request ID is not interchangeable with that trace ID.

Thread pools and asynchronous frameworks need explicit context transfer and cleanup. Context available when a task is submitted does not automatically become the context of the worker that later runs it. Use an integration appropriate to the runtime, capture only the intended values, and restore the worker's previous state afterward.

Evaluate asynchronous logging and allocation reduction

Log4j's asynchronous loggers use LMAX Disruptor. Add the compatible Disruptor dependency required by the chosen Log4j release before enabling them. A context-selector configuration can make ordinary loggers asynchronous:

# Set before logging initializes, through component properties or JVM properties.
log4j2.contextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
log4j2.enableThreadlocals=true
log4j2.garbagefreeThreadContextMap=true

Selective asynchronous logger configuration is another approach. Do not mix all-async and selectively async designs without understanding the additional handoffs they create. See asynchronous logging.

Asynchronous logging moves work away from the calling thread, but it does not guarantee that calls never block or that the destination can keep up indefinitely. The ring buffer is finite. Measure bursts, sustained output, full-buffer behavior, and shutdown. Exception handling also changes when writing happens on a background thread.

The two allocation-related settings enable reuse along supported paths. A low-allocation result still depends on message construction, argument types, layouts, filters, appenders, and context handling. A custom object or an expensive layout can allocate even when the settings are enabled. Web-container lifecycle and classloader behavior also deserve attention when retaining thread-local state. See garbage-free logging guidance.

There is no universal order-of-magnitude speedup to apply to a capacity estimate. Benchmark the application's actual output and compare its latency and allocation profile before and after the change.

Finally, buffered events can be lost if the process crashes before writing them. For audit-sensitive records, evaluate a synchronous path and the full durability requirements. Synchronous formatting and writing still do not, by themselves, guarantee that bytes have reached durable storage. A collector can only read what has actually been emitted; it cannot recover events left in the application's memory.

Collect the output and validate its meaning

For host files, configure the actual path, source, service identity, and processing rules in DataKit. For container output, verify the runtime's collection and rotation path. See the DataKit logging guide.

With the ECS template, map log.level to the destination's severity field and parse @timestamp as event time. Retain the logger name, exception details, and relevant request identifiers. Check whether dotted field names are represented as literal keys or nested objects in the collection path.

Generate an event containing an exception and a real trace ID. Find it in Guance, check its timestamp and fields, then verify the trace relationship. Test rotation and application shutdown as well. One successfully collected event during normal operation establishes much less than a complete collection test.

Common questions

Can configuration changes reload without a restart?

monitorInterval="30" on Configuration enables polling for supported configuration-file changes at a 30-second interval. It is a polling setting, not a strict end-to-end guarantee that every change takes effect within exactly 30 seconds. The configuration must be accessible and eligible for monitoring; a file packaged inside an unchanged application artifact is not the same as an editable external file. See configuration monitoring.

How does a Spring Boot application switch to Log4j 2?

Exclude the default spring-boot-starter-logging dependency from the applicable dependency paths and add spring-boot-starter-log4j2. Use Boot's dependency management and check the resulting classpath for competing providers or bridges. Code already using SLF4J can generally keep that API; direct Logback-specific configuration or code needs its own migration. See Spring Boot logging.

How should RollingFile retention be configured?

A time-based policy uses the date pattern in filePattern to establish its interval. interval="1" means one such interval, not necessarily one day. A size policy such as size="100MB" adds another rollover trigger.

DefaultRolloverStrategy max="30" controls an index range; it is not a universal 30-day retention policy across dated archives. When old date-based files need removal, configure the appropriate deletion conditions and test them against the generated names. See rolling file appenders.

For container standard output, the runtime normally owns local log rotation while collectors forward the output. Confirm ownership instead of assuming the collection backend rotates runtime files.

Why is a ThreadContext field empty?

Check that the value was set before the logging call, was available on that thread, and is included by the layout. For work moved to a pool, verify context capture and installation at the worker boundary. Clear or restore the worker's temporary context after the task so identifiers do not leak into the next request.

Related guides

• Java logging practices
• Logback configuration
• Comparing Java logging libraries (Chinese)
• Structured logging (Chinese)