Optimistic Locking

The Coordinator is the single writer of a run's state, but "single writer" doesn't mean "one goroutine anywhere in the fleet." Multiple pods can be woken for the same run — by a redelivered wakeup, a timer, and a signal, all at once. Ductor keeps this correct without a distributed lock, using optimistic concurrency control on a per-run version counter.

The version column

Every eec_workflow_run row (the eec_ prefix marks Ductor's durable Enterprise Eventing Core tables) carries a monotonically increasing version counter (db_record_version, referred to throughout the docs as record_version). Every tick reads it, and every commit conditionally bumps it.

The commit is a single conditional UPDATE:

UPDATE eec_workflow_run
   SET db_record_version = db_record_version + 1,
       status = $status,
       node_states = $node_states,
       last_transition_seq = $seq,
       state_checksum = $checksum
 WHERE run_id = $run_id
   AND db_record_version = $expected_version;

The WHERE db_record_version = $expected_version clause is the whole trick. The coordinator computed its new state from a snapshot at version N, so it commits with expected_version = N. If another tick already committed, the row is now at N+1, the WHERE matches zero rows, and this commit is rejected.

What a conflict looks like

Two pods wake for the same run and both read version 7. Notice that the loser isn't blocked or queued behind the winner — it is rejected after the fact, and its recovery is to re-read the winner's state and recompute from scratch:

When the update affects zero rows, the store returns ErrRecordVersionConflict. Nothing was written — the losing tick simply didn't happen. The coordinator then:

1. Retries in-process with a short backoff schedule (0, 10ms, 20ms, 40ms, 1s), up to five attempts. Each retry re-loads the current state and recomputes the tick from scratch, so it's operating on fresh data, not a stale plan.
2. If in-process retries are exhausted, it requeues the run to the tiered queue with an escalating backoff (100ms · 2ⁿ, capped at 10s) and a distinct, deterministic task key per attempt so the requeue is never suppressed by queue-level deduplication.

Crucially, a version conflict is not counted as a tick error. It's an expected outcome of healthy concurrency — two workers tried to advance the same run and one won. The error counter is reserved for genuine failures.

The tick is pure; the counter isn't persisted

The requeue count rides on the wakeup envelope in transport, not on the run row. This preserves two invariants at once: the run row is only ever written by a committed tick (sole-writer), and the tick computation itself is a pure function of loaded state (no hidden retry bookkeeping mutating the row). A committed, progressing tick emits ordinary wakeups with a requeue count of zero, which resets the escalation — a run that's making progress never accumulates backoff.

Livelock protection

Optimistic locking has one pathological case: two ticks that endlessly conflict without either making progress. Ductor bounds it. After a fixed number of non-progressing requeues, the run is force-failed through the same terminal path as the stuck-run watchdog, and the outcome is metered. A run that's merely hot — conflicting often but committing progress each time — never trips this, because any successful tick resets the counter.

Why not pessimistic locking?

A row lock or distributed lease would also serialize writers, but it introduces a new failure mode: a crashed lock holder blocks the run until a lease expires, and lease tuning becomes a production concern. Optimistic locking has no such holder — there's nothing to leak. The cost is occasional recomputation on conflict, which is cheap because ticks are cheap and conflicts are rare per run. For a workload of many independent runs each advancing serially, this is the right trade.

Where to go next

• Coordinator & Step Workers — the single-writer model this protects.
• Idempotency & Exactly-Once — how this underpins exactly-once progress.
• The Tiered Fair Queue — why wakeups don't need dedup.