Docker Logging Driver Tuning: Defaults, Formulas, and Disk-Safe Configuration

35 min read

Published July 22, 2022

No vague advice. Every logging option: what it does, the default if you omit it, what goes wrong, and exactly how to calculate the right value — with numbers.

Most Docker guides say "set max-size so logs don't fill your disk." That's true, but useless at 3 a.m. when /var/lib/docker is at 98% and you don't know whether to change daemon.json, Compose, or both.

This guide is different. For every important logging setting you'll get four things:

1. What it does — in plain language
2. Default value — what Docker uses if you write nothing
3. What goes wrong — if you leave the default or set it wrong
4. How to calculate — step-by-step with real numbers

We'll use a concrete example server throughout:

Adjust the numbers for your server. The method stays the same.

Part 1 — How Docker logging works

When your app writes to stdout or stderr, Docker's logging driver captures it. You don't mount a log volume unless you choose to — Docker handles storage on the host.

Default driver: json-file

Unless you change it globally or per-container, Docker uses the json-file driver. Each log line is wrapped in JSON with timestamp and stream metadata, then appended to a file on the host.

Critical fact: With default json-file settings, there is no size limit. One chatty container can grow a single log file until your root disk is full. This is the #1 cause of "Docker server ran out of space overnight."

Data flow (mental model)

App process → stdout/stderr → Docker container runtime → logging driver → host disk (or remote sink)

json-file  → /var/lib/docker/containers/.../...-json.log
local      → /var/lib/docker/containers/.../...-local/logs/ (ring buffer)
syslog     → syslog daemon on host or remote
journald   → systemd journal
fluentd    → Fluentd collector → Elasticsearch/S3/etc.
awslogs    → Amazon CloudWatch Logs

Part 2 — Global daemon.json vs per-service Compose

You can set logging in two places. They interact — and confusion here causes "I changed daemon.json but nothing happened."

/etc/docker/daemon.json (global default)

Applies to new containers that don't specify their own logging block. Does not retroactively change existing containers — you must recreate them.

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3",
    "compress": "true"
  }
}

After editing:

# 1. Validate JSON syntax
python3 -m json.tool /etc/docker/daemon.json

# 2. Restart Docker (stops all containers unless live-restore is on — plan a window)
sudo systemctl restart docker

# 3. Confirm daemon picked up the config
docker info --format '{{.LoggingDriver}}'
docker info --format '{{json .Plugins.Log}}' | jq .

# 4. Recreate containers so they get new defaults (existing containers keep old LogConfig)
cd /opt/myapp
docker compose up -d --force-recreate

# 5. Verify one container
docker inspect --format='{{json .HostConfig.LogConfig}}' api | jq .

{
  "Type": "json-file",
  "Config": {
    "max-size": "10m",
    "max-file": "3",
    "compress": "true"
  }
}

Important: Changing daemon.json does not update containers that are already running. You must recreate them. Also: every value in log-opts must be a string — write "3" not 3, and "true" not true.

docker-compose.yml (per-service override)

Per-service logging overrides the daemon default for that service only. This is how you give a noisy worker more headroom while keeping a quiet Redis container tight.

services:
  api:
    image: myapp/api:1.0.0
    logging:
      driver: json-file
      options:
        max-size: "20m"
        max-file: "5"
        compress: "true"

  redis:
    image: redis:7-alpine
    logging:
      driver: json-file
      options:
        max-size: "5m"
        max-file: "2"

Which wins?

Part 3 — json-file driver options

Options that do not apply to json-file: tag, fluentd-address, syslog-address — those belong to remote logging drivers. Using tag under json-file is silently ignored or rejected depending on Engine version.

Part 4 — local driver (modern alternative)

The local driver stores logs in a binary ring buffer — faster and more disk-efficient than json-file for high-volume logging. Human-readable via docker logs; on-disk format is not plain text.

When to pick local over json-file

• local: High log volume, same-host docker logs debugging, disk efficiency matters
• json-file: You grep raw log files on disk, use log-shipper agents that read JSON lines, or need maximum tooling compatibility

# daemon.json — switch global default to local
{
  "log-driver": "local",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Part 5 — Getting logs out of Docker (centralized logging)

Local rotation (max-size / max-file) solves disk safety. It does not solve search, retention, or alerts. For that you need logs outside the container host.

There are two production patterns. Most teams use one or combine both:

1. Driver-based shipping — Docker sends each log line directly to a remote system (Fluentd, syslog, Loki plugin, CloudWatch). No log file on disk (or minimal buffering).
2. Collector-based shipping — Keep json-file/local on disk with rotation; run Fluent Bit, Promtail, Filebeat, or Vector on the host to tail files and forward to a central store.

Which to pick?
Single host, small stack → json-file + rotation + Fluent Bit or Promtail is simple and debuggable.
Multi-host, compliance, long retention → central Loki, ELK, or CloudWatch with a collector on every node.
Zero local disk for logs → set logging driver to fluentd/syslog/loki and accept that docker logs depends on driver buffering.

5.1 — Method A: Docker logging driver → remote (direct ship)

The container runtime forwards stdout/stderr to the driver. Logs may never hit *-json.log on disk.

Step 1 — Run Fluent Bit as a collector on the host:

# /opt/fluent-bit/fluent-bit.conf
[SERVICE]
    Flush        1
    Log_Level    info

[INPUT]
    Name              forward
    Listen            0.0.0.0
    Port              24224

[OUTPUT]
    Name              loki
    Match             *
    Host              loki.example.com
    Port              3100
    Labels            job=docker,host=${HOSTNAME}
    auto_kubernetes_labels off

# docker-compose.yml for Fluent Bit
services:
  fluent-bit:
    image: fluent/fluent-bit:3.1
    container_name: fluent-bit
    restart: unless-stopped
    ports:
      - "24224:24224"
    volumes:
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
    network_mode: host   # or attach to same compose network as apps

Step 2 — Point your app at Fluent Bit:

services:
  api:
    image: myapp/api:1.0.0
    logging:
      driver: fluentd
      options:
        fluentd-address: "127.0.0.1:24224"
        tag: "docker.api.{{.Name}}"
        fluentd-async: "true"
        fluentd-buffer-limit: "1048576"

Step 3 — Verify:

docker compose up -d fluent-bit api
docker logs api          # still works (driver buffers recent lines)
# Check Fluent Bit received data in its logs
docker logs fluent-bit

[2026/08/30 16:20:11] [ info] [input:forward:forward.0] listening on 0.0.0.0:24224
[2026/08/30 16:20:13] [ info] [output:loki:loki.0] worker #0 started
[2026/08/30 16:20:18] [ info] [output:loki:loki.0] http_status=204

services:
  api:
    logging:
      driver: syslog
      options:
        syslog-address: "tcp://logs.example.com:514"
        tag: "{{.Name}}"
        syslog-format: "rfc5424"

services:
  api:
    logging:
      driver: gelf
      options:
        gelf-address: "udp://graylog:12201"
        tag: "api"

services:
  api:
    logging:
      driver: awslogs
      options:
        awslogs-group: "/production/myapp"
        awslogs-region: "eu-west-1"
        awslogs-stream-prefix: "api"
        awslogs-create-group: "true"

docker plugin install grafana/loki-docker-driver:3.0.0 \
  --alias loki \
  --grant-all-permissions

# Verify
docker plugin ls

ID             NAME          DESCRIPTION           ENABLED
f3a61c8016c2   loki:latest   Loki Logging Driver   true

services:
  api:
    logging:
      driver: loki
      options:
        loki-url: "http://loki:3100/loki/api/v1/push"
        loki-batch-size: "400"
        labels: "service,environment"
        loki-external-labels: "host={{.Hostname}},container={{.Name}}"

5.2 — Method B: json-file on disk + collector (most common on VPS)

Keep local rotation for emergency docker logs and disk safety. Run a collector that reads log files from /var/lib/docker/containers/.

Why teams prefer this: If Loki/Elasticsearch is down, logs still exist locally. Collectors can catch up after outages. Easier to debug with docker logs and raw files.

Option B1 — Promtail → Loki (Grafana stack)

# promtail-config.yml
server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: docker
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
        refresh_interval: 5s
    relabel_configs:
      - source_labels: ['__meta_docker_container_name']
        target_label: container
      - source_labels: ['__meta_docker_container_log_stream']
        target_label: stream

services:
  promtail:
    image: grafana/promtail:3.0
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./promtail-config.yml:/etc/promtail/config.yml:ro
    command: -config.file=/etc/promtail/config.yml
    networks:
      - monitoring

  loki:
    image: grafana/loki:3.0
    ports:
      - "3100:3100"
    networks:
      - monitoring

  grafana:
    image: grafana/grafana:11.2.0
    ports:
      - "3000:3000"
    networks:
      - monitoring

networks:
  monitoring:

Your app services keep normal json-file logging with max-size/max-file. Promtail ships independently — changing the app compose does not break shipping.

Option B2 — Fluent Bit tailing json logs

# fluent-bit.conf — read json-file logs from disk
[INPUT]
    Name              tail
    Path              /var/lib/docker/containers/*/*-json.log
    Parser            docker
    Tag               docker.*
    Refresh_Interval  5
    Mem_Buf_Limit     10MB
    Skip_Long_Lines   On

[FILTER]
    Name              modify
    Match             docker.*
    Add               host ${HOSTNAME}

[OUTPUT]
    Name              es
    Match             docker.*
    Host              elasticsearch
    Port              9200
    Index             docker-logs
    Suppress_Type_Name On

services:
  fluent-bit:
    image: fluent/fluent-bit:3.1
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
    network_mode: host

Option B3 — Filebeat → Elasticsearch (ELK)

# filebeat.yml
filebeat.inputs:
  - type: container
    paths:
      - '/var/lib/docker/containers/*/*.log'
    processors:
      - add_docker_metadata:
          host: "unix:///var/run/docker.sock"

output.elasticsearch:
  hosts: ["https://elasticsearch:9200"]
  username: "elastic"
  password: "${ES_PASSWORD}"

5.3 — Method C: Hybrid (recommended production pattern)

What actually works on real servers:

1. daemon.json — safe json-file defaults (10m × 3, compress) as a floor
2. Compose overrides — per-service caps for noisy containers
3. Collector on host — Promtail or Fluent Bit ships to Loki/ES
4. Retention in Loki/ES — 30–90 days; local disk only keeps 2–3 days for emergencies

5.4 — journald and none drivers

services:
  healthcheck-sidecar:
    image: busybox
    logging:
      driver: "none"

Part 6 — Practical walkthrough: first production host from zero

Follow this order on a fresh VPS. Skipping steps is how you end up with unlimited logs and no central search.

Step 1 — Measure before tuning (day 0)

# Deploy stack with defaults, wait 24h, then:
sudo du -sh /var/lib/docker/containers/*/*-json.log | sort -hr | head -10
docker ps --format '{{.Names}}' | while read c; do
  echo -n "$c: "
  sudo stat -c%s "$(docker inspect --format='{{.LogPath}}' $c)" 2>/dev/null | awk '{print int($1/1048576)" MB"}'
done

31M	/var/lib/docker/containers/a1.../a1...-json.log
9.8M	/var/lib/docker/containers/b2.../b2...-json.log
api: 31 MB
nginx: 9 MB
redis: 0 MB

Step 2 — Set daemon.json floor

sudo cp /etc/docker/daemon.json /etc/docker/daemon.json.bak.$(date +%F) 2>/dev/null || true
echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
sudo jq '. + {
  "log-driver": "json-file",
  "log-opts": {"max-size": "10m", "max-file": "3", "compress": "true"}
}' /etc/docker/daemon.json | sudo tee /etc/docker/daemon.json.new
sudo python3 -m json.tool /etc/docker/daemon.json.new >/dev/null
sudo mv /etc/docker/daemon.json.new /etc/docker/daemon.json
sudo systemctl restart docker

Step 3 — Add per-service overrides in Compose

Only for containers that exceeded 10m/day in Step 1 measurements.

  api:
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "5"
        compress: "true"

Step 4 — Deploy collector (Promtail or Fluent Bit)

Separate compose file in /opt/monitoring/ so logging stack updates don't touch app stack.

Step 5 — Verify end-to-end

# Local rotation works
docker inspect --format='{{json .HostConfig.LogConfig}}' api | jq .

# Generate test log line
docker exec api sh -c 'echo "LOG_TEST_$(date -Iseconds)"'

# Find it in Loki (Grafana Explore) or:
curl -G 'http://loki:3100/loki/api/v1/query_range' \
  --data-urlencode 'query={container=~".*api.*"} |= "LOG_TEST"'

{"Type":"json-file","Config":{"compress":"true","max-file":"5","max-size":"50m"}}
{"status":"success","data":{"resultType":"streams","result":[{"stream":{"container":"api"},"values":[["1788107118000000000","LOG_TEST_2026-08-30T16:25:18+00:00"]]}]}}

Step 6 — Alert on disk (don't skip)

Monitor /var/lib/docker and root filesystem at 80%. Log rotation prevents surprises but images/volumes also grow.

Part 7 — Formulas and stack-level math

Per-container disk cap

Whole stack total (worst case)

Daily log growth estimate

Headroom for /var/lib/docker

Part 8 — Complete production config examples

Full daemon.json (global safe defaults)

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3",
    "compress": "true"
  },
  "storage-driver": "overlay2"
}

Every new container: 30 MB log cap unless Compose overrides. Compressed rotations save disk.

Full docker-compose.yml (stack with mixed overrides)

services:
  nginx:
    image: nginx:1.27-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
        compress: "true"

  api:
    image: myapp/api:1.0.0
    restart: unless-stopped
    environment:
      APP_ENV: production
    logging:
      driver: json-file
      options:
        max-size: "20m"
        max-file: "5"
        compress: "true"
        labels: "com.docker.compose.service,com.docker.compose.project"

  worker:
    image: myapp/worker:1.0.0
    restart: unless-stopped
    logging:
      driver: json-file
      options:
        max-size: "15m"
        max-file: "4"
        compress: "true"

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    logging:
      driver: json-file
      options:
        max-size: "5m"
        max-file: "2"
        compress: "true"

  # Sidecar example: ship api logs to Fluentd (optional)
  # api:
  #   logging:
  #     driver: fluentd
  #     options:
  #       fluentd-address: "127.0.0.1:24224"
  #       tag: "docker.api"
  #       fluentd-async: "true"

Part 9 — Inspect LogConfig and measure reality

docker inspect LogConfig

See effective driver and options for a running container:

docker inspect --format='{{json .HostConfig.LogConfig}}' api | jq .

{
  "Type": "json-file",
  "Config": {
    "max-file": "5",
    "max-size": "20m",
    "compress": "true"
  }
}

Log file path:

docker inspect --format='{{.LogPath}}' api

/var/lib/docker/containers/a1b2c3d4e5f6/a1b2c3d4e5f6-json.log

Measure disk usage

# Total Docker root (images + containers + logs + volumes)
sudo du -sh /var/lib/docker

# All container log directories
sudo du -sh /var/lib/docker/containers/*/*-json.log 2>/dev/null | sort -hr | head -20

# Find largest log files anywhere under docker
sudo find /var/lib/docker -name "*-json.log*" -exec du -h {} + | sort -hr | head -20

# Per-container log size (json-file)
CONTAINER=api
sudo du -h $(docker inspect --format='{{.LogPath}}' $CONTAINER)

18G	/var/lib/docker
31M	/var/lib/docker/containers/a1.../a1...-json.log
9.8M	/var/lib/docker/containers/b2.../b2...-json.log
31M	/var/lib/docker/containers/a1.../a1...-json.log

Estimate daily growth (24-hour sample)

# Record size now
SIZE1=$(sudo stat -c%s "$(docker inspect --format='{{.LogPath}}' api)")
echo "Start bytes: $SIZE1"

# Wait 24 hours (or measure over 1 hour and multiply)

# Record size later
SIZE2=$(sudo stat -c%s "$(docker inspect --format='{{.LogPath}}' api)")
echo "End bytes: $SIZE2"
echo "Growth MB: $(( (SIZE2 - SIZE1) / 1048576 ))"

Start bytes: 10485760
End bytes: 41943040
Growth MB: 30

Live log rate (rough)

# Lines per second over 10 seconds
timeout 10 docker logs -f api 2>&1 | wc -l

842

Part 10 — Log rotation behavior

When max-size is set and the active log file reaches that size:

1. Docker closes the current file (e.g. ...-json.log)
2. Renames it to ...-json.log.1 (and shifts .1 → .2, etc.)
3. Opens a fresh ...-json.log
4. If max-file is exceeded, the oldest numbered file is deleted
5. If compress: true, rolled files become .gz

What docker logs shows: docker logs reads from the driver's stored buffer — for json-file/local, you see retained rotated content until max-file drops it. You cannot recover logs deleted after rotation.

Changing options: Editing daemon.json or Compose does not shrink existing files. Recreate the container (docker compose up -d --force-recreate api) or truncate manually (last resort — stop container first).

Part 11 — OOM and disk-full scenarios

Disk full on /var/lib/docker

• Symptoms: no space left on device, containers fail to start, docker pull fails, host SSH may hang if root partition is full
• Cause: Often unlimited json-file logs; also unused images, build cache, volumes
• Immediate fix: Find largest logs (find ... -json.log), truncate or remove oldest rotated files for non-critical containers; docker system prune for dangling images (careful in production)
• Permanent fix: Set max-size/max-file globally; per-service overrides; centralize logs; monitor disk with alerts at 80%

Memory pressure (less common for logging)

Logging drivers buffer minimally on json-file/local. Risk increases with fluentd without fluentd-async — blocked writes can slow apps. Remote driver outage + sync mode → apparent "hang" under load.

Container won't start after log config change

Invalid option names or values in Compose cause create failure. Validate with docker compose config before deploy. Typo maxsize instead of max-size is ignored or rejected depending on Compose version.

Worked example — full calculation summary for our server

Master defaults lookup — quick reference

Option / Driver          │ Default (if omitted)     │ Our value      │ Why we changed
─────────────────────────┼──────────────────────────┼────────────────┼────────────────────────────
log-driver (daemon)      │ json-file                │ json-file      │ Tooling compatibility
json-file max-size       │ -1 (unlimited)           │ 10m global     │ Prevent disk fill
json-file max-file       │ 1                        │ 3 global       │ Keep 3 rotated files
json-file compress       │ false                    │ true           │ Save ~70% on rolled logs
local max-size           │ 20m                      │ 10m (workers)  │ Tighter cap on batch jobs
local max-file           │ 5                        │ 3 (workers)    │ Match retention needs
api max-size × max-file  │ unlimited × 1            │ 20m × 5        │ Noisy service
redis max-size × max-file│ unlimited × 1            │ 5m × 2         │ Minimal logging
Compose logging block    │ (inherits daemon)        │ per-service    │ Right-size each service
Log path (json-file)     │ .../ID-json.log          │ (automatic)    │ —
Precedence               │ daemon → container       │ Compose wins   │ Explicit overrides

Troubleshooting — symptom → cause → fix

Symptom                              │ Likely cause                         │ Fix
─────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────
/var/lib/docker at 100%              │ Unlimited json-file logs             │ Set max-size/max-file; truncate largest logs
daemon.json change had no effect     │ Existing containers not recreated    │ docker compose up -d --force-recreate
One container huge, others fine      │ Missing Compose logging override     │ Add logging.options to that service
docker logs empty but app prints     │ Wrong driver / buffer not flushed    │ Check LogConfig; app may log to file not stdout
Log settings show null in inspect    │ Built with wrong compose file        │ docker compose config; verify deployed file
413/502 unrelated — disk full        │ Root full, not app error             │ Free disk first, then restart containers
fluentd driver slow/stuck            │ Collector down, async off            │ fluentd-async: true; fix collector
compress not working                 │ Engine too old or typo               │ Docker 20.10+; key is "compress": "true"
Can't find log file                  │ local driver or different ID         │ docker inspect --format='{{.LogPath}}'
Rotation not happening               │ max-size unlimited (-1)              │ Set explicit max-size

Production checklist

1. Set global max-size and max-file in /etc/docker/daemon.json — never run unlimited json-file in production
2. Enable compress: "true" for json-file on disk-constrained hosts
3. Override noisy services in Compose (api, workers, proxies)
4. Verify effective config: docker inspect LogConfig on each critical container after deploy
5. Measure actual daily log growth after 24–48 hours; recalculate caps
6. Alert on disk usage ≥ 80% on root and /var/lib/docker
7. Document stack log budget: Σ(max-size × max-file) ≤ planned MB
8. For compliance retention beyond a few days, ship logs to Loki, CloudWatch, ELK, or Graylog — not bigger local files alone (see Part 5)
9. Ensure apps log to stdout/stderr, not only to files inside the container (files inside container aren't rotated by Docker)
10. After daemon.json changes, schedule recreate window — restart affects all containers on the host
11. Run docker system df regularly; logs are one slice — prune images/volumes too
12. Never put secrets in env log options or debug printouts

Best practices

• Stdout only: Twelve-factor apps write logs to stdout; Docker captures them. File-based app logs bypass rotation unless you tail them with a sidecar.
• Structured logging: JSON lines from your app compress well and parse easily — one 400-byte JSON line beats five 200-byte unstructured lines when you need context.
• Log level in production: INFO or WARN — DEBUG on one service can blow your calculated max-size in hours.
• Global floor, local ceiling: daemon.json sets safe defaults; Compose raises limits only where measured growth requires it.
• Centralize for retention: Local rotation is for debugging and short tail — not archival. Ship to Loki, CloudWatch, or ELK for search and compliance.
• Test rotation: In staging, set max-size to 1m temporarily, generate traffic, confirm files roll and oldest delete at max-file.
• Pin the math: Keep a comment in Compose: # 30 MB/day measured 2026-08 → 20m×5 so the next person knows why.

Wrapping up

Every Docker logging option has a default, a purpose, and a formula. The dangerous defaults are json-file with unlimited max-size and max-file: 1 without size — together they mean one file grows forever until the host dies.

Start with daemon.json caps (10m × 3 with compress is a solid baseline), override chatty services in Compose, measure real growth with du and docker inspect, and ship logs off-host when you need more than a few days of history. The method doesn't change — only your line counts and disk budget do.

Have a specific container eating disk or a stack size you're unsure about? Drop your container count and daily log MB in the comments.