# Health checks & readiness (/docs/operations/health-checks)



Ductor exposes liveness and readiness endpoints on the HTTP listener, plus a
`health` CLI subcommand. The distinction between liveness and readiness matters:
liveness answers "is the process alive?", readiness answers "can it serve traffic
right now?".

## The endpoints [#the-endpoints]

All are served on the HTTP listener (`server.http_addr`, default `:8080`):

| Path      | Kind      | Returns                                                                                          |
| --------- | --------- | ------------------------------------------------------------------------------------------------ |
| `/health` | Liveness  | `200 {"status":"ok"}` as long as the process is running                                          |
| `/ready`  | Readiness | `200` when internal readiness is set **and** backing dependencies are reachable; `503` otherwise |

<Callout type="info" title="Registered aliases">
  `/healthz` and `/livez` are registered aliases for `/health`; `/readyz` is a
  registered alias for `/ready`. Kubernetes convention accepts either spelling —
  use whichever your probe tooling expects. This page uses the canonical
  `/health` and `/ready` throughout.
</Callout>

### Liveness — process is up [#liveness--process-is-up]

The liveness handler returns `200` unconditionally while the process runs. It does
**not** check dependencies — a liveness failure means the process itself is wedged
and should be restarted. Use it for the container/Kubernetes **liveness** probe so
a transient database blip doesn't cause a restart loop.

### Readiness — safe to route traffic [#readiness--safe-to-route-traffic]

The readiness handler returns `200` only when both are true:

1. **Internal readiness** is set — the app finished booting and wiring.
2. **Dependencies are reachable** — a live check against Postgres and Redis,
   bounded by a short timeout so a slow backend doesn't hang the probe.

```mermaid
flowchart TD
    Req["GET /ready"] --> Internal{"internal readiness set?"}
    Internal -->|no| Fail["503"]
    Internal -->|yes| Deps{"Postgres + Redis reachable?"}
    Deps -->|no| Fail
    Deps -->|yes| OK["200"]
```

Because readiness re-checks dependencies on every call, it flips to `503` if
Postgres or Redis becomes unreachable *after* boot — not just at startup. Use it
for the **readiness** probe and load-balancer health so an unhealthy pod is pulled
from rotation without being killed.

## The CLI check [#the-cli-check]

`ductor health` performs an HTTP `GET /health` against a running server and prints
`OK` or the error. It's handy for scripts and the container healthcheck:

```bash
ductor health --http-addr :8080
```

The container image's built-in healthcheck does the equivalent
(`wget --spider http://localhost:8080/health`).

## Probe configuration [#probe-configuration]

### Docker Compose [#docker-compose]

The image ships a healthcheck against `/health` (interval 10s, timeout 3s, 3
retries). No extra config needed.

### Kubernetes (Helm) [#kubernetes-helm]

The chart wires all three probes to the right endpoints:

| Probe            | Path      | Why                                                |
| ---------------- | --------- | -------------------------------------------------- |
| `startupProbe`   | `/ready`  | Hold traffic until boot + dependencies are good    |
| `readinessProbe` | `/ready`  | Pull the pod from Service endpoints when not ready |
| `livenessProbe`  | `/health` | Restart only when the process is wedged            |

<Callout type="warn" title="Don't point liveness at /ready">
  If you use a readiness endpoint for the liveness probe, a brief Postgres outage
  makes `/ready` fail, Kubernetes restarts every pod, and you turn a dependency
  blip into an outage. Keep liveness on `/health` and readiness on `/ready`.
</Callout>

## What to alert on [#what-to-alert-on]

* **Readiness failing across replicas** — dependencies (Postgres/Redis)
  unreachable, or a bad rollout. This is your primary "is Ductor serving?" alert.
* **Coordinator health** — stuck runs and dead-lettered wakeups
  (`workflow_dag.coordinator.stuck_run_threshold`,
  `workflow_dag.coordinator.wakeup_dead_letter_threshold`) surface in the DAG
  metrics.
* **Pending migrations** — `serve` logs a warning at startup; `ductor schema
  status` exits non-zero. Gate deploys on it.

See [Observability](/docs/operations/observability) for wiring these signals into
metrics and alerts.