Centralize logs from Docker on Elastic Beanstalk

Docker applications on Elastic Beanstalk can use centralized logging. The main choices are Elastic Beanstalk's CloudWatch Logs integration, a collector that forwards application logs, or direct delivery from the application to a logging service.

Option 1: enable CloudWatch Logs integration

Enable streaming in the environment settings or an .ebextensions file:

# .ebextensions/logging.config
option_settings:
  aws:elasticbeanstalk:cloudwatch:logs:
    StreamLogs: true
    RetentionInDays: 14

This streams the log files configured for the platform. It does not discover every arbitrary file written inside every container. Verify the Docker platform branch, instance-role permissions, and the paths containing your application logs. Docker Compose environments need the documented mounts or other collection configuration for custom log files.

Option 2: run a collector

A separate collector container can read shared application files or an explicitly supported container-log path. This illustrative Compose fragment assumes Docker's json-file logging driver and the usual host data directory:

services:
  app:
    image: myapp
  log-collector:
    image: fluent/fluent-bit
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro

This fragment only makes files visible. Supply the collector's input, parsing, checkpointing, output, and authentication configuration before expecting delivery. Other logging drivers and Docker data-root settings require a different source. Pin and test the collector image you deploy.

Docker Compose support on current Elastic Beanstalk Docker platform branches is distinct from the older ECS-based multicontainer platform. Match deployment files to the platform you actually run. A collector sidecar is also different from embedding a second process inside the application's own container.

Option 3: send logs from the application

An HTTP client or SDK can send events directly to a logging service. This reduces dependence on file collection, but couples application behavior to delivery, retries, buffering, and backpressure. Determine what happens during an outage and during process shutdown.

For a collector-based Guance integration, first identify the real file paths and rotation policy. Configure DataKit file collection, retain application, environment, and instance identifiers, then test another deployment to confirm the configuration survives instance replacement.

Frequently asked questions

Are logs lost when an instance is replaced?

Logs that exist only on the removed instance can be lost. Successfully delivered copies remain in the destination according to its retention policy. Centralization reduces that exposure but does not guarantee delivery of the last buffered events before termination.

How much does CloudWatch Logs cost?

Charges depend on Region, ingestion, retention, queries, and enabled features. Estimate them from actual volume and current AWS pricing. A shorter retention period reduces retained data but does not remove ingestion charges.

Is eb logs only a snapshot command?

Without extra options it retrieves tail logs, and --all or --zip retrieves more complete files. It can also enable CloudWatch streaming with eb logs --cloudwatch-logs enable and continuously display logs with --stream. Durable centralized storage depends on the configured backend, not on leaving a terminal session open.

References: Docker environment configuration and eb logs.