Docker healthcheck interval, start period and retries

A Docker health check runs its first probe one full interval after the container starts, not straight away. With the defaults that is 30 seconds of health: starting. With --interval=1h it is an hour, and anything waiting on service_healthy waits that hour too. The fix is not a shorter interval for the whole life of the container. Set a start_period long enough for the app to boot, and on Docker Engine 25.0 or later add a short start_interval, so the check polls quickly while the app starts and slowly once it is up. The other thing people get wrong: a container marked unhealthy keeps running. Plain Docker does not restart it, and neither does Compose.

This post covers the timing in detail: each of the five options, how they combine into "how long until healthy" and "how long until unhealthy", the Dockerfile and Compose syntax side by side, writing a check for an image that has no curl, and reading the result. For what the endpoint itself should check, and why liveness and readiness are different questions, see graceful shutdown and real health checks. For how a health check fits into a full stack with a proxy and a database, see Docker Compose for small stacks.

The five timing options and their defaults#

A health check is a command Docker runs inside the container on a timer. Exit code 0 means healthy, 1 means unhealthy, and 2 is reserved and should not be used. In practice Docker treats any non-zero exit as a failure, but the documentation asks for 1, so the usual idiom ends a check with || exit 1.

A few details the table leaves out:

• The interval counts from when the last check finished, not from when it started. A check that takes 4 seconds with a 30-second interval runs about every 34 seconds.
• The timeout is a hard limit on one run. If the command is still going when it expires, that run is a failure. The default of 30 seconds is far too generous for a local HTTP request. If /healthz has not answered in 3 to 5 seconds, something is wrong, and waiting longer only delays finding out.
• Retries counts consecutive failures. One success resets the counter. A check that fails every other time never reaches unhealthy, which is a good reason to make the check deterministic.
• The start period does not delay the first check. Checks still run during it. Their failures just do not count. A success during the start period ends it early: the container is healthy, and every later failure counts.
• The start interval only applies inside the start period. With start_period: 0s, the default, start_interval does nothing.

Why the first check waits a whole interval#

This is the behaviour behind most "my container is stuck in starting" questions. Docker schedules the first probe one interval after the container starts. It does not run one at time zero to see if the app is already up. Until that first probe succeeds, docker ps shows (health: starting).

That is harmless with the 30-second default. It stops being harmless when someone reasons: "the check is cheap, but I don't need it often, so interval: 5m". Now every fresh container sits in starting for five minutes. Every service with depends_on: condition: service_healthy waits five minutes before Compose even creates it. And docker compose up --wait blocks for five minutes on a deploy that was ready in two seconds. At interval: 1h the container is starting for an hour.

Before Engine 25.0 there was no clean way out. A long interval meant a slow first verdict, and a short interval meant polling every few seconds forever. People compromised on 10 to 30 seconds and accepted both costs.

--start-interval separates the two. During the start period Docker checks every start_interval. After the first success, or once the start period ends, it switches to the normal interval. A container can be checked every 2 seconds while it boots and every 5 minutes after that:

HEALTHCHECK --interval=5m --timeout=3s --retries=3 \  --start-period=60s --start-interval=2s \  CMD curl -fsS http://127.0.0.1:8080/healthz || exit 1

If the app answers after 7 seconds, the fourth or fifth probe succeeds and the container is healthy about 8 seconds after start, not 5 minutes. If the app never comes up, the start period runs out after 60 seconds, and from then on failures count at the 5-minute interval.

How long until healthy, how long until unhealthy#

The two numbers worth working out for any check are how fast a good container is declared healthy, and how slowly a dead one is declared unhealthy.

Time to healthy is roughly the boot time, rounded up to the next probe. Without a start interval, the first probe is at interval, so a container is never healthy sooner than that. With one, it is boot time plus at most one start_interval.

Time to unhealthy for an app that hangs after it was healthy is about retries checks:

worst case ~ retries x (interval + timeout)defaults:    3 x (30s + 30s) = about 3 minutestuned:       3 x (10s + 3s)  = about 40 seconds

The timeout counts because a hung app does not refuse the connection. It accepts it and never answers, so every probe runs to the full timeout before it fails. That is the second reason to cut the default timeout: with defaults, a hung process can serve errors for three minutes before Docker admits anything is wrong.

For an app that never becomes healthy at all, add the start period. Failures inside it do not count, so the first counted failure lands after start_period, and unhealthy follows retries probes later.

Some starting points, to adjust for your own app:

Keep retries at 3 unless you have a reason. One failure is noise. Three in a row is a pattern.

Dockerfile syntax vs Compose syntax#

The same check can live in three places, and the spelling differs in each.

In a Dockerfile it is an instruction with flags, and it becomes part of the image metadata:

HEALTHCHECK --interval=30s --timeout=3s --start-period=30s --start-interval=2s \  --retries=3 CMD ["curl", "-fsS", "http://127.0.0.1:8080/healthz"]

A Dockerfile can only have one HEALTHCHECK. If there are several, the last one wins. A health check in a base image is inherited, and HEALTHCHECK NONE switches it off.

In Compose it is a healthcheck: key on the service, with underscores where the flags have hyphens:

services:  app:    image: example/app:1.4.2    healthcheck:      test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8080/healthz"]      interval: 30s      timeout: 3s      retries: 3      start_period: 30s      start_interval: 2s

Durations are written like 1m30s, 10s or 500ms. A Compose healthcheck overrides the one baked into the image field by field, so you can keep the image's command and change only the timing. To remove an image's check entirely, use disable: true (or test: ["NONE"]). start_interval in a Compose file needs Docker Compose 2.20.2 or later and, like the Dockerfile flag, Engine 25.0 or later to take effect.

On `docker run` the same options are flags: --health-cmd, --health-interval, --health-timeout, --health-retries, --health-start-period, --health-start-interval, and --no-healthcheck to disable an inherited one.

Which place is right? Put the check in the Dockerfile when it belongs to the app: the image knows its own port and health path, and every deployment gets a sensible default. Put it in Compose when it depends on the deployment, such as a different port, a dependency the check should wait for, or a third-party image you cannot rebuild.

CMD vs CMD-SHELL, and the dollar sign trap#

The test field takes three forms, and the difference matters as soon as the command contains a pipe, a variable or ||.

• ["CMD", "curl", "-fsS", "http://127.0.0.1:8080/healthz"] - exec form. Docker runs the binary directly, with no shell. No pipes, no $VAR expansion, no ||. It also works in images that have no shell.
• ["CMD-SHELL", "curl -fsS http://127.0.0.1:8080/healthz || exit 1"] - runs the string through the container's default shell, /bin/sh -c on Linux. Pipes, variables and || work.
• test: curl -fsS http://127.0.0.1:8080/healthz || exit 1 - a plain string is the same as CMD-SHELL.

In a Dockerfile the same split exists: HEALTHCHECK CMD ["curl", ...] is exec form, and HEALTHCHECK CMD curl ... || exit 1 is shell form.

The trap is variables in Compose. Compose interpolates $VAR itself, from your shell and .env file, before the container ever sees it. A check written as pg_isready -U $POSTGRES_USER gets the value from the machine running docker compose, which is often empty. To make the container's shell expand it, double the dollar sign:

  db:    image: postgres:17    environment:      POSTGRES_USER: app      POSTGRES_PASSWORD_FILE: /run/secrets/db_password    healthcheck:      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_USER}"]      interval: 10s      timeout: 5s      retries: 5      start_period: 30s      start_interval: 2s

$$ reaches the container as a single $. The container's environment does contain POSTGRES_USER, because health checks run with the container's environment, just like docker exec.

Two more things about where the check runs. It runs inside the container, so localhost is the container itself, not the host. That is correct for checking the app, but it means a health check never tests a published port or a proxy in front. And prefer 127.0.0.1 over localhost: on some images localhost resolves to the IPv6 ::1 first, an app bound to 0.0.0.0 (IPv4 only) refuses the connection, and the check fails even though the app is fine.

When the image has no curl or wget#

The examples above assume curl, and plenty of images do not ship it. Slim and minimal images leave out everything the app does not need: node:22-slim and python:3.13-slim have neither curl nor wget. Alpine-based images have BusyBox wget but no curl. Distroless images have no shell either, so any CMD-SHELL check fails before it starts. A check that cannot find its binary fails every time, and the container goes unhealthy while the app is perfectly fine. docker inspect then shows curl: not found or an exec error in the health log.

There are three ways around it.

1. Install the tool. Adding curl to a Debian slim image is one apt-get install line and a few megabytes. That is often fine, but it adds a package to the image's attack surface just for a probe.
2. Use what is already there. On Alpine, BusyBox wget works: wget -q --spider http://127.0.0.1:8080/healthz || exit 1. Databases ship their own probes: pg_isready in the Postgres image, mysqladmin ping in MySQL.
3. Use the runtime you already have. A Node image has Node, and a Python image has Python. Both can make an HTTP request in one line.

HEALTHCHECK --interval=30s --timeout=3s --start-period=20s --start-interval=2s \  CMD ["node", "-e", "fetch('http://127.0.0.1:3000/healthz').then(r => process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))"]

HEALTHCHECK --interval=30s --timeout=3s --start-period=20s --start-interval=2s \  CMD ["python", "-c", "import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=2)"]

The Python version works because urlopen raises on any 4xx or 5xx status and on a refused connection, and an uncaught exception exits with status 1. Both are exec form, so they also work on images with no shell. Starting a Node or Python interpreter costs more than curl does, tens of megabytes for a moment. That is irrelevant at a 30-second interval and worth a thought at a 2-second start interval on a small machine.

For distroless images the cleanest answer is a tiny health-check binary built into the image, or a --health mode in your own app that makes the request and exits. Many Go services do exactly that.

Whichever tool you use, make it fail on an HTTP error, not just on a refused connection. Plain curl exits 0 on a 500 response. -f (--fail) is what turns an error status into a non-zero exit. Without it the check only proves something is listening.

depends_on and service_healthy#

Health status matters most to Compose's start ordering. A dependency declared with condition: service_healthy holds back the dependent service until the dependency reports healthy:

  app:    image: example/app:1.4.2    depends_on:      db:        condition: service_healthy        restart: true

This is where interval tuning pays off. Compose waits for the database's first successful probe, so a database with interval: 60s and no start interval delays the app by at least a minute on every up, however fast Postgres actually starts. With start_interval: 2s the app starts within a couple of seconds of the database being ready.

Compose also gives up if a dependency goes unhealthy instead: up fails with a message that the dependency failed to start. Usually the start period was too short for a first run, such as a database initialising a fresh volume or restoring a dump, while later restarts were fine. Lengthen the start period rather than raising retries, so the grace applies only at boot.

restart: true (Compose 2.17.0 or later) restarts the dependent service when Compose itself restarts or updates the dependency. It is not a health-based restart. A database that goes unhealthy on its own triggers nothing.

docker compose up --wait uses the same signal for deploys: it returns once every service is running and healthy, and exits non-zero if one fails. In a deploy script, that is what lets you stop before the traffic switch rather than after. Zero-downtime deploys on a small server builds on exactly that wait.

Reading the health log with docker inspect#

docker ps shows the summary in the status column: Up 2 minutes (healthy), (health: starting) or (unhealthy). To find every unhealthy container on a host:

$ docker ps --filter health=unhealthy$ docker inspect --format '{{json .State.Health}}' app | python3 -m json.tool

The Health object holds Status, FailingStreak (consecutive failures so far) and Log, a list of the most recent probe results. Each entry has a Start, an End, an ExitCode and an Output. Docker keeps only the last five results and stores only the first 4096 bytes of each check's output. So a health endpoint that prints a short reason on failure (db: connection refused) is far more useful here than one that dumps a stack trace.

Read the Output before changing anything. The common messages and what they mean:

• curl: not found, executable file not found in $PATH - the tool is missing from the image. See the section above.
• Health check exceeded timeout - the command did not finish in time. Either the app is hung or the timeout is shorter than a legitimate slow response.
• Connection refused - nothing is listening on that address inside the container. Check the port, and check 127.0.0.1 against localhost.
• An HTTP 503 from your own endpoint - the check is working and the app is reporting a real problem. Fix the app, not the check.

You can also watch transitions as they happen with docker events --filter event=health_status, which prints a line each time a container changes between healthy and unhealthy.

Unhealthy does not restart anything#

Docker's restart policies (no, on-failure, always, unless-stopped) react to the container's main process exiting, and nothing else. An unhealthy container whose process is still running stays exactly as it is, still receiving traffic if it was receiving traffic before. The health status is information. Plain Docker and Compose report it and act on it only at start-up, through depends_on and --wait.

Three things do act on it:

• Docker Swarm replaces service tasks that become unhealthy. If you run Swarm, the health check is a real restart trigger.
• A watcher you add. A small container with access to the Docker socket polls for unhealthy and restarts what it finds. The willfarrell/autoheal image is the well-known example. Mounting the Docker socket gives that container root-equivalent control of the host, so treat it as the trade-off it is.
• The app exiting on its own. If the app can detect that it is broken, exiting with a non-zero status is the most reliable restart trigger there is, because every restart policy understands it.

Kubernetes is the exception people trip over: it ignores the Dockerfile HEALTHCHECK completely and uses its own liveness, readiness and startup probes. startupProbe plays the role of the start period there.

Be careful what you wish for with automatic restarts. A check that fails because the database is down will restart a healthy app in a loop, which the graceful shutdown post calls the amplifier. Whatever restarts on health should only look at things a restart can fix. And alert on the health status changing, as monitoring that tells you something describes, rather than silently papering over it.

On RE:NODE app plans the question looks a little different. Each server is a container we run, without a Docker daemon of yours inside, so there is no HEALTHCHECK to tune. Instead, a crash watcher polls every two minutes for uptime that went backwards or a server that went offline, and restarts you asked for are not counted. Three unplanned restarts in an hour put a warning on the server page and open a ticket. Six suspend the server. Exiting on a fatal error is therefore still the right design there. If you want Docker's own health machinery with Compose and service_healthy, that needs a machine where the daemon is yours, which is what the VDS line is for. Docker on a VDS covers the setup.

FAQ#

Why is my container stuck in "health: starting"?

The first probe has not succeeded yet. Either the first check only happens after one full interval (so a long interval means a long wait), or every check so far has failed inside the start period, where failures do not count. Run docker inspect --format '{{json .State.Health}}' on the container. An empty Log means no check has run yet. Entries with non-zero exit codes mean the check is failing and the Output says why.

What is a good Docker healthcheck interval?

For a typical web service, 30 seconds with a 3-second timeout and 3 retries, which flags a hung app in under two minutes. Go shorter only if something acts on the result quickly, such as a load balancer or Swarm. On Engine 25.0 or later, pair any interval longer than a minute with a start_period and a start_interval of a few seconds, so start-up does not wait for the long interval.

Does start_period delay the first health check?

No. Checks run during the start period at the normal interval, or at start_interval on Engine 25.0 and later. Failures inside the window are simply not counted toward retries. The first success ends the start period early.

Does Docker restart an unhealthy container?

Not on its own. Restart policies fire only when the main process exits. Swarm services, a watcher such as autoheal, or the app exiting by itself are the ways to turn unhealthy into a restart. Compose uses health only to order start-up and for up --wait.

Should the health check call the database?

Usually not for a check that anything restarts on. A database blip would then mark every app container unhealthy at once. Check the process itself, and report dependencies separately through a readiness endpoint. For a database container, the check should of course be the database's own probe, such as pg_isready or mysqladmin ping.

Can I change a health check without rebuilding the image?

Yes. A healthcheck: block in Compose or the --health-* flags on docker run override the image's values, and disable: true or --no-healthcheck turns it off. The Dockerfile instruction is only the default.