Django logging: configuration, JSON, and request context

Django uses Python's standard logging framework. The LOGGING dictionary in settings.py configures formatters, handlers, filters, and named loggers; application modules use logging.getLogger(__name__) to emit records through that configuration.

The practical work is to understand Django's built-in logger hierarchy, select the output paths, attach request context, and verify the collected records. This guide uses Django 5.2 configuration semantics and shows both a local file setup and a JSON alternative for centralized collection.

Know which logger owns the event

These loggers are not a complete access log for every production request. For example, some suspicious-operation 400 responses are logged under django.security rather than django.request. Production access logging usually also involves the application server or proxy.

Django's defaults depend on DEBUG. In Django 5.2, the django hierarchy normally sends INFO and higher to the console when DEBUG=True. With DEBUG=False, ERROR and CRITICAL records go to AdminEmailHandler, whose delivery depends on the email configuration. django.server has its own console behavior independent of DEBUG. Review the logging reference rather than assuming every production error is printed to a console by default.

Configure output centrally

For a local or single-process example, add this to an existing project's settings.py, where BASE_DIR is already defined:

LOG_DIR = BASE_DIR / "logs"
LOG_DIR.mkdir(parents=True, exist_ok=True)

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "verbose": {
            "format": "{asctime} | {levelname} | {name} | {message}",
            "style": "{",
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout",
            "level": "INFO",
            "formatter": "verbose",
        },
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "filename": LOG_DIR / "django.log",
            "maxBytes": 10 * 1024 * 1024,
            "backupCount": 5,
            "encoding": "utf-8",
            "level": "DEBUG",
            "formatter": "verbose",
        },
    },
    "loggers": {
        "django": {
            "handlers": ["console", "file"],
            "level": "INFO",
            "propagate": False,
        },
        "django.server": {
            "handlers": ["console", "file"],
            "level": "INFO",
            "propagate": False,
        },
        "myapp": {
            "handlers": ["console", "file"],
            "level": "DEBUG",
            "propagate": False,
        },
    },
}

Replace myapp with the actual package prefix used by application loggers. This example sends application DEBUG records to the file and INFO-or-higher records to both destinations. propagate=False prevents these loggers from sending the same records onward to ancestor handlers. The explicit django.server entry also avoids retaining a separate development-server output format unintentionally.

disable_existing_loggers=False keeps other existing loggers enabled unless the configuration changes them. Setting it to true can disable existing loggers that are not named in the configuration; it does not simply mean “turn off every Django logger.” Filters, handler thresholds, propagation, and logger names can also explain missing output. See Django's configuration guide.

Provision the log directory and its permissions during deployment when the application should not create it. A RotatingFileHandler is not a safe shared rotation mechanism for several Gunicorn workers writing the same file. For multiple processes, prefer a managed stdout path or a separate logging listener; Python's logging cookbook explains the limitation.

Use a named logger in business code

In myapp/services.py, application code can log an operation without configuring handlers:

import logging

logger = logging.getLogger(__name__)


def create_order(service, user_id):
    logger.info("Creating order", extra={"user_id": user_id})
    try:
        return service.create_order(user_id=user_id)
    except Exception:
        logger.exception("Order creation failed", extra={"user_id": user_id})
        raise

Here service is the application's order service, supplied by the caller. logger.exception() records the current exception and traceback. Re-raising preserves the failure for the caller; logging should not accidentally turn a failed operation into a success response.

The text formatter above does not display user_id, although it exists on the LogRecord. A formatter must include or serialize extra fields for them to reach the output. Avoid keys reserved by LogRecord, such as message or levelname, in extra.

Switch the handlers to JSON

Install the JSON formatter:

pip install python-json-logger

After the LOGGING dictionary, add a JSON formatter and select it for the handlers:

LOGGING["formatters"]["json"] = {
    "()": "pythonjsonlogger.json.JsonFormatter",
    "format": "%(levelname)s %(name)s %(message)s",
    "rename_fields": {"levelname": "level"},
    "timestamp": "time",
}
for handler in LOGGING["handlers"].values():
    handler["formatter"] = "json"

This uses the current pythonjsonlogger.json.JsonFormatter import path and adds an ISO-formatted event timestamp under time. Extra fields and exception information can be serialized alongside the message. Consult the Python JSON Logger quick start when customizing field names and serialization.

JSON is useful for typed queries, but it is not a sensitive-data policy. Inspect extras supplied by Django and third-party libraries, including request objects and exception text. Filter or allowlist fields before collection; avoid exposing cookies, credentials, query strings, or full user records.

For a multi-worker stdout deployment, remove the file handler and its references before Django applies the configuration:

LOGGING["handlers"].pop("file")
for config in LOGGING["loggers"].values():
    config["handlers"] = ["console"]

If DEBUG application records should reach stdout too, lower that handler's threshold deliberately. Changing only the logger threshold will not bypass an INFO handler threshold.

Carry a request ID explicitly

A small synchronous middleware can generate an identifier and attach it to the request and response. Save this as myapp/middleware.py:

from uuid import uuid4


class RequestIdMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.request_id = uuid4().hex
        response = self.get_response(request)
        response["X-Request-ID"] = request.request_id
        return response

Add "myapp.middleware.RequestIdMiddleware" to MIDDLEWARE before code that needs the identifier. This example generates a value rather than trusting an arbitrary incoming header. Use an appropriate async-capable implementation if your request path requires it.

A view can create an adapter for its own records:

import logging
from django.http import JsonResponse

logger = logging.getLogger(__name__)


def status(request):
    request_logger = logging.LoggerAdapter(
        logger, {"request_id": request.request_id}
    )
    request_logger.info("Status requested")
    return JsonResponse({"ok": True})

The request attribute and adapter do not automatically add that ID to all Django framework records or unrelated loggers. For broader propagation, use a carefully scoped filter/context mechanism or evaluate middleware such as django-structlog. Confirm exactly which fields that integration supplies, and test isolation between concurrent requests.

Verify collection and trace association

For file collection, configure the actual logs/django.log path and reading permissions in the DataKit log collector. For containers, collect the chosen standard output instead. Avoid collecting both copies of the same event without intending to duplicate it.

Trigger a known application event, a handled failure, and a request failure. Check event time, severity, logger name, request fields, and exception traceback in the collected JSON. The output format and Pipeline parsing must agree; a timestamp that looks correct locally may still be interpreted incorrectly by the collector.

For application performance monitoring (APM) correlation, instrument tracing first and emit the actual current trace identifier. Follow the Guance log association requirements for trace, service, environment, and version mapping, then verify one request end to end. request_id remains useful even when no trace is available, but it does not substitute for trace_id.

Common questions

Why did Django errors disappear after changing LOGGING?

Check disable_existing_loggers, the actual logger name, both logger and handler levels, filters, and propagation. Also inspect file permissions and handler initialization errors. Keeping existing loggers enabled is useful, but it is not a universal fix for every routing mistake.

How do I include a request ID on every request?

Generate or validate one in middleware and pass it into the records that need it. A request attribute alone does not enrich logs. For automatic enrichment, verify the scope and cleanup behavior of a context-based solution and test concurrent requests.

Can I turn on SQL logging in production?

Django's normal django.db.backends SQL debug logging is tied to settings.DEBUG, not just the logger's threshold. Do not enable project-wide DEBUG in production merely to see SQL. Start with database slow-query facilities or instrumented database spans; any temporary diagnostic path needs a narrow scope and sensitive-parameter review.

Where do Gunicorn logs go?

Gunicorn has separate access and error logging configuration. --access-logfile - selects stdout for access logs; --error-logfile - selects stderr for its error log. Configure Django's handlers separately and make sure the container collector reads the intended streams. Trigger one application event and one access event to confirm both paths rather than assuming one setting configures them all.

Related reading

• Python logging fundamentals
• Flask logging guide (Chinese)