# Durability Model: Coordinator & Workers (/docs/architecture/coordinator-worker-model)



This page assumes the mental model from
[Coordinator & Step Workers](/docs/concepts/coordinator-workers) and goes into the
internals: *why* the split is shaped this way, what the durability boundary buys,
and how the pieces are wired.

## One serial owner, many parallel executors [#one-serial-owner-many-parallel-executors]

The design is a deliberate separation of two concerns that most workflow engines
tangle together:

* **Deciding what happens next** is inherently sequential for a given run — it's
  a state transition, and two transitions on the same state must be ordered. The
  **Coordinator** owns this. It is the *sole writer* of `eec_workflow_run` (the
  `eec_` prefix marks Ductor's durable *Enterprise Eventing Core* tables).
* **Doing the work** is embarrassingly parallel — connector calls, rule
  evaluation, user logic. **Step Workers** own this, across the whole fleet, and
  they only ever *append* results.

By assigning "decide" to a single serial writer and "do" to many append-only
workers, the corrupting-write class of bug is designed out. There is no lock to
tune and no two-writer race to reason about, because there is only ever one
writer of the thing that must be serialized.

## The durability boundary [#the-durability-boundary]

The heart of the model is a single ordering rule:

```mermaid
flowchart LR
  A["Compute (pure)"] --> B["Commit to Postgres"] --> C["Dispatch side effects"]
```

Everything to the left of the commit is a pure function of loaded state.
Everything to the right is a best-effort, redrivable side effect. The commit is
the **durability boundary**: state is durable *before* any work is dispatched.

This placement is what produces Ductor's two headline guarantees:

* **Exactly-once state progress** — the pure tick plus a version-checked commit
  means a run's durable state advances at most once per logical step, no matter
  how many times the tick runs. See
  [Optimistic Locking](/docs/concepts/optimistic-locking).
* **At-least-once side effects** — dispatch happens after the commit, so a crash
  in between causes a safe redrive. See
  [Idempotency & Exactly-Once](/docs/concepts/idempotency).

<Callout title="Purity is the enabler">
  The tick computes over a *clone* of the state and emits a payload; it doesn't
  mutate anything or call out to the world. That purity is what makes a conflict
  retry trivial — just recompute against fresh state — and what keeps replay
  deterministic.
</Callout>

## Wiring: how a run gets ticked [#wiring-how-a-run-gets-ticked]

<Steps>
  <Step>
    Something needs a run to advance — a worker finished an attempt, a timer
    fired, a signal arrived, or a new run was created.
  </Step>

  <Step>
    A **coordinator wakeup** is enqueued. It travels either through
    [SyncMatch](/docs/concepts/tiered-queue#syncmatch-the-zero-latency-fast-path)
    (in-process, zero-latency) when a worker is co-located, or through the durable
    [tiered queue](/docs/concepts/tiered-queue) on a dedicated system partition.
  </Step>

  <Step>
    A coordinator worker goroutine dequeues the wakeup and runs one
    `Coordinator.Tick()`.
  </Step>

  <Step>
    The tick loads state, computes a payload, commits under `record_version`, and
    dispatches follow-on work post-commit.
  </Step>

  <Step>
    On a version conflict, the tick retries in-process with backoff, then requeues
    through the tiered queue — bounded against livelock. See
    [Optimistic Locking](/docs/concepts/optimistic-locking).
  </Step>
</Steps>

Because each run is ticked serially but runs are independent, the system scales
horizontally: add pods and more runs make progress in parallel, while any single
run stays strictly ordered.

## Where the two roles write [#where-the-two-roles-write]

| Role        | Writes                                                                                                          | Reads                                        |
| ----------- | --------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| Coordinator | `eec_workflow_run` (state, `record_version`), inserts `eec_workflow_step_attempt` rows on dispatch, transitions | attempt results, signals, timers, child runs |
| Step Worker | attempt results into `eec_workflow_step_attempt`                                                                | its dispatched task                          |

Note the asymmetry: workers never write run state, and the coordinator is the
one inserting the attempt row when it dispatches — the worker fills in the
*result*. A stale worker from a superseded attempt can't clobber a newer one,
because the node's attempt generation is bumped on retry.

## Failure modes it's designed against [#failure-modes-its-designed-against]

* **Double-write corruption** → impossible: one writer, version-checked.
* **Lost progress on crash** → impossible: state is committed before dispatch.
* **Stuck runs** → bounded: a stuck-run watchdog and a conflict-livelock bound
  force-fail a run that can't make progress, visibly, rather than letting it hang.
* **Unbounded history** → bounded: [ContinueAsNew](/docs/concepts/dag-workflow-model#continueasnew)
  and transition compaction cap run growth.

## Where to go next [#where-to-go-next]

* [Coordinator & Step Workers](/docs/concepts/coordinator-workers) — the concept-level walkthrough.
* [Data Flow](/docs/architecture/data-flow) — where run state, attempts, and counters physically live.
* [Optimistic Locking](/docs/concepts/optimistic-locking) — the commit's concurrency primitive.