Core Concepts

Routing Pipeline

How Ductor turns an incoming event into a concrete assignment through staged pipeline stages.

Routing is how a unit of Work gets Routed — the routed stage of the clearing lifecycle. Given a work item (Routable), the pipeline produces a Route: the concrete worker, queue, or downstream target that should handle it, together with the reason it was chosen. It runs the same way every time, as an ordered sequence of stages carried by a mutable RoutingContext that each stage reads from and writes to.

The stages

The pipeline runs five stages in order:

  1. Validate — reject malformed or unauthorized input, and resolve the target pool, before any work is done.
  2. Enrich — attach the data later stages need (pool data, recipients, cached rules) so selection isn't guessing.
  3. Filter — narrow the eligible set: excluded recipients, capacity-exhausted targets, rule mismatches.
  4. Select — run a pluggable Strategy over the surviving candidates to pick a winner. This is the hot path.
  5. Assign — record the chosen recipient on the decision and hand off to finalization.
Routable Validate Enrich Filter Select Assign Decision

The output — a Decision joined with its Explanation — is what the docs call a Route: not just who got the work, but why they won. That explanation is a first-class artifact, not a log line; it is the evidence a later receipt draws on.

The topology is data, not code

The pipeline is not a hard-coded call sequence. Stages register into a name-keyed StageRegistry, and the engine iterates a []PipelineStage slice built from it. Every stage satisfies a base PipelineStage (which just returns its canonical lower_snake_case name) plus a role interface that says what kind of stage it is:

  • Validator, Enricher, Filter, Selector, Assigner — the five roles above, and
  • PostRouteHook — runs after a successful assignment for in-memory annotations.

Modules contribute stages through the pipeline_stages fx group (pkg/routing/pipeline/stage.go, stage_registry.go, cmd/ductor/fx_routing_stages.go), and a module-supplied stage can override a default of the same name. So a deployment adds or replaces a stage by wiring a provider — it never forks the engine. See Extension Points.

State flows between stages on a single mutable RoutingContext: each stage reads the fields it needs and writes the ones it produces — Routable, Decision, Pool, Eligible, RuleResult, Selected, StrategyUsed, and per-stage StageOptions.

Strategies

A Strategy is a pluggable worker-selection algorithm — it ranks the eligible candidates (a recipient in the API today; human or agent in the clearing model) and picks a winner. Ductor calls Info() first to check compatibility, then Select() on the hot path. Optional capability interfaces let a strategy do more when it can:

  • BatchStrategy — score many candidates at once instead of one-by-one.
  • ExplainStrategy — return why a candidate won, for debugging and audit.
  • HealthCheckStrategy — report readiness before being used.

Built-in strategies live in pkg/strategy/builtin/; you register custom ones through the strategy registry (pkg/strategy/registry.go). A strategy reads only SelectRequest.Features — typed, immutable snapshots the pipeline has already assembled — never repositories or live services directly. That boundary is what keeps selection pure and testable; see Strategy Contracts. Rules themselves are evaluated with CEL, compiled once and cached, so routing logic can change without a redeploy — a rule edit invalidates the cache via pub/sub and every pod picks up the new rule.

Finalization

Selecting a winner is only half the job; committing the decision safely is the other half. Once a decision is made, a separate FinalizationContext carries it through the finalization stages:

  • persist the decision durably (guarded by routing idempotency so a retried request reuses the same decision rather than making a new one),
  • adjust capacity counters (atomic Redis Lua increments/decrements),
  • emit events,
  • record metrics,
  • run any middleware chain,
  • and kick off async follow-ups.

Modes — and why the default is DAG, not linear

Routing runs in one of four modes: linear (the legacy inline pipeline in the calling goroutine), dag (the request runs through the durable workflow runtime, so a decision can drive a multi-step process), shadow (evaluate without committing side effects — useful for validating a new strategy against live traffic), and inherit.

The active mode is resolved along a pool → tenant → global chain: the first non-inherit value wins. Here is the trap — when every level is inherit (the out-of-the-box state), resolution falls back to dag, not linear. A reader who assumes routing executes in-process linearly may actually be running on the DAG runtime. Check the resolved mode, don't assume.

Idempotent by request

A routing request can carry an idempotency key. The first request to reserve the key runs the pipeline and records the decision; a duplicate with the same key returns the same decision instead of routing again. This is what makes routing safe to retry from the client — a network retry never produces a second, divergent assignment. See Idempotency & Exactly-Once.

Where it runs

Routing is invoked directly by transport handlers and by queue-worker goroutines, so the same decision logic backs both synchronous API calls and asynchronous workflow steps. Because it's the same pipeline in both places, a decision made inline and a decision made from a workflow step are identical in behavior — there's no second, drifting implementation.

Where to go next

  • Tenancy — how every routing decision is tenant-scoped.
  • Entitlements — the quota checks that gate the hot path.
  • Extension Points — how to add strategies, enrichers, and middleware.