Configure Logback for structured Java logging

Many Spring Boot applications already use Logback through the default logging starter. That does not mean their logging behavior has been deliberately configured. Duplicate events, missing request context, and unexpectedly large files usually come down to a few choices: which logger accepts an event, which appenders receive it, and how the output is formatted and retained.

This guide explains those choices, then adds JSON output and request context. It also covers the checks needed before collecting the logs and creating alerts in Guance.

Understand the SLF4J and Logback relationship

SLF4J is the logging API that application code commonly calls. logback-classic supplies a logging implementation and depends on Logback's core facilities. Keep a compatible API and implementation on the classpath, with one intended SLF4J provider.

Spring Boot's default logging starter normally supplies Logback unless the application has excluded or replaced it. Let Boot's dependency management select compatible versions rather than independently overriding individual logging jars.

For a plain Maven application, the dependency has this shape:

<dependency>
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-classic</artifactId>
  <version>${logback.version}</version>
</dependency>

Define logback.version as a concrete, supported release compatible with the application's Java runtime and dependencies. A placeholder such as 1.5.x is not an actual Maven version.

With no custom configuration, a plain Logback setup can fall back to a console appender with a DEBUG root level. Spring Boot supplies its own defaults, so do not infer Boot's behavior from that fallback. In ordinary classpath configuration, logback-test.xml takes precedence over logback.xml; explicit configurators and configuration properties can change the loading path. See Logback configuration.

For Boot-specific extensions, use logback-spring.xml so Boot can load and interpret the configuration at the appropriate point.

Define appenders, named loggers, and the root logger

The following configuration sends most application events to both the console and a rolling file. Payment logs use a dedicated routing rule:

<configuration>
  <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>logs/app.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
      <fileNamePattern>logs/app-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
      <maxFileSize>100MB</maxFileSize>
      <maxHistory>30</maxHistory>
      <totalSizeCap>10GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%d{ISO8601} %-5level %logger{36} - %msg %xEx%n</pattern>
    </encoder>
  </appender>

  <logger name="com.example.payments" level="DEBUG" additivity="false">
    <appender-ref ref="FILE"/>
  </logger>

  <root level="INFO">
    <appender-ref ref="CONSOLE"/>
    <appender-ref ref="FILE"/>
  </root>
</configuration>

An appender defines an output destination and its encoding. Named loggers control levels and routing within a hierarchy. The root supplies a fallback for loggers without a more specific level.

Here, payment events accepted at DEBUG or higher go to FILE only. additivity="false" stops them from also reaching the root's appenders, which would otherwise cause duplicate file output. It also means payment events do not reach the console in this configuration. If that is not intended, add the console reference explicitly or choose another routing arrangement.

The rolling policy uses both date and index tokens. It rotates by day and size, retains up to 30 historical periods, and applies a total archive size cap. With the daily date pattern, those periods are days. These are retention controls, not a guarantee that the entire filesystem can never fill: active files, cleanup timing, other applications, and temporary space still matter.

The text pattern includes time, severity, logger name, and exception details. Decide on a timezone policy when aggregating multiple hosts; a timestamp without an offset requires the collector to know the intended timezone. See the appender reference.

Emit JSON with an encoder that fits the schema

Logback has a native JsonEncoder in supported recent versions. The separate logstash-logback-encoder project is another widely used option with additional schema and field customization. JSON output does not universally require a third-party encoder. See Logback encoders.

To use the Logstash encoder, select a release compatible with the deployed Java, Logback, and Jackson versions:

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>${logstash-logback-encoder.version}</version>
</dependency>

Define the version property in dependency management. Then replace the encoder inside the already referenced CONSOLE appender:

<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
  <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

A defined appender does not receive events until a logger references it. Reusing the existing CONSOLE name keeps the root reference in the previous example valid.

The encoder normally emits one JSON object per event, including fields such as @timestamp, level, logger_name, and message, with MDC fields according to its configuration. Despite the project name, Logstash does not have to be the destination. Inspect an actual event and configure the collector against that output.

If the rolling file is also meant to contain JSON, replace its encoder too. Changing the console format leaves the file appender's text format unchanged.

Attach and restore request context with MDC

The mapped diagnostic context (MDC) holds values that an encoder or pattern can include in a log event. Populate it from the current request or tracing context, and clean it up when that scope ends.

The following application fragment preserves any context installed by an outer layer:

import java.util.Map;
import org.slf4j.MDC;

Map<String, String> previous = MDC.getCopyOfContextMap();
try {
    MDC.put("trace_id", traceId);
    MDC.put("user_id", String.valueOf(userId));
    log.info("Order processing completed");
} finally {
    MDC.clear();
    if (previous != null) {
        MDC.setContextMap(previous);
    }
}

traceId, userId, and the SLF4J log instance come from the application. Treat user identifiers according to the organization's data policy.

MDC is associated with the executing thread. Work moved to an executor or an asynchronous framework needs deliberate context propagation and cleanup. A request ID generated for logging is not automatically the trace ID used by distributed tracing.

Check a real request that produces both a trace and a log containing the same identifier. Guance's APM FAQ describes the relevant correlation requirements. An encoder can include a value that was placed in MDC; it cannot infer a missing trace relationship.

Filter events and reduce framework noise

A TurboFilter can make an early decision about a logging request. For example, this marker filter denies events explicitly marked SECRET:

<turboFilter class="ch.qos.logback.classic.turbo.MarkerFilter">
  <Marker>SECRET</Marker>
  <OnMatch>DENY</OnMatch>
</turboFilter>

This is a marker policy, not sensitive-data detection. An unmarked event containing a secret can still be logged. Preventing credential collection and validating redaction remain separate tasks.

For routine framework noise, a package-level threshold is often simpler:

<logger name="org.apache.kafka" level="WARN"/>

Choose that threshold based on the diagnostics the team needs. Suppressing all low-severity records can remove context that matters during an incident.

Verify the events arriving in Guance

Configure DataKit for the actual file or container output using the DataKit logging guide. Send an INFO event, an ERROR event, and an exception with a stack trace.

Check the event timestamp, map level to the destination's severity field such as status, and confirm that the exception remains complete. For JSON output, escaped newline characters within a string should remain part of the same event. A text stack trace may require multiline collection rules.

Find the records by service and time range before building error-count alerts. If both console and file output are collected, make sure the same event is not ingested twice. Then validate rotation and request correlation using the configured data paths. Create a log detection rule only after its severity field and time window have been checked.

Common questions

How does logback-spring.xml differ from logback.xml?

Boot controls loading of logback-spring.xml, enabling extensions such as <springProfile> and <springProperty>. Ordinary Logback already supports property substitution; the difference is not that all placeholders are exclusive to Boot's file name. Choose the Boot-managed file when those extensions are needed and avoid combining them with unsupported automatic scanning arrangements.

How can writes be made asynchronous?

Wrap a destination in AsyncAppender. Set the queue capacity based on measured bursts and downstream latency, then define the full-queue behavior. Its default queue is 256 events, and its default discarding policy can drop lower-severity events as capacity runs low. A full queue may block callers unless configured otherwise; an asynchronous appender does not guarantee nonblocking or lossless logging.

Configure shutdown flushing and test what happens when the destination stalls or the process terminates. A queue size such as 1,024 is a capacity choice, not a universal production minimum. Compare alternatives only if profiling shows logging is a meaningful bottleneck.

What if logging.level settings disagree with XML?

Spring Boot can apply logger levels from logging.level.* during startup, and runtime changes can alter them again. Keep one clearly documented owner for each setting, inspect the effective level, and avoid scattering conflicting policies across XML, application configuration, and operational scripts.

Can levels change without a restart?

Spring Boot Actuator's loggers endpoint can inspect and change logger levels when it is enabled and appropriately exposed. Protect it as an administrative operation, keep changes narrow, and restore temporary debug levels after the investigation. Verify the endpoint path and exposure policy for the application's management configuration.

Related guides

• Log4j 2 configuration (Chinese)
• Comparing Java logging libraries (Chinese)
• Java logging practices (Chinese)
• Changing log levels at runtime (Chinese)