Define & publish a workflow

This guide is the executed stage of the clearing lifecycle, hands-on: a workflow definition is a DAG of steps that Ductor's coordinator executes durably once Work is routed to a Worker. Definitions are authored as YAML, validated, loaded, and then published — publishing freezes an immutable version that the runtime pins runs to. This guide walks the full lifecycle for a small definition.

1. Author the definition

A definition is a YAML document with a type: workflow_definition envelope and a definition body. The body declares steps, the edges between them, one or more triggers, and optional checkpoints. Here is the smallest legal definition — two noop action steps joined by a success edge:

type: workflow_definition
definition:
  family_slug: hello_world
  title: Hello World
  description: Smallest executable DAG — two steps joined by a success edge.
  steps:
    - ref: a
      type: action
      action: noop:noop
      args:
        key: value
    - ref: b
      type: action
      action: noop:noop
  edges:
    - source_ref: a
      target_ref: b
      type: success
  triggers:
    - id: t1
      type: manual
  checkpoints:
    - name: accepted
      step_refs:
        - b

Key fields:

The 15 step types

action is one of 15 authorable step types. The rest cover control flow, fan-out, human-in-the-loop, AI, and cross-runtime calls:

See the DAG workflow model for the full semantics of each. The ai_action / ai_agent steps are covered in AI; bridge steps in SDKs.

Event triggers

An event trigger fires runs when a matching event arrives. Each trigger carries one or more patterns, and every pattern is either a literal event name (stripe.invoice.paid) or a suffix wildcard (stripe.invoice.*).

Steps that call connectors

Action steps can call a third-party system through a connector. Reference a connection and pass templated arguments (${{ ... }} expressions resolve at dispatch time against trigger payload, variables, and prior step outputs):

steps:
  - ref: create_invoice
    type: action
    action: stripe.invoices.create
    connection_ref: "${{ VARS.stripe_connection_ref }}"
    connection_ref_mode: direct
    args:
      customer_id: "${{ TRIGGER.payload.customer_id }}"
      amount: "${{ TRIGGER.payload.amount_cents }}"
      currency: "${{ TRIGGER.payload.currency }}"
    retry_max_attempts: 3
    retry_backoff_ms: 1000
    timeout_ms: 10000
    on_error: fail

Steps with external side effects can declare compensation — inverse actions that run on_failure or on_cancel — so a partially-completed run unwinds cleanly.

2. Validate offline

Before loading anything, lint the definition with the CLI. workflow validate parses the YAML and resolves every reference without touching the database — it exits non-zero on any problem:

ductor workflow validate ./hello_world.yaml

Resolution is fully offline. Action keys are checked against the ActionRegistry ∪ ExecutorRegistry, strategy references against pkg/strategy.Registry, and hook points against pkg/middleware.AllHookPoints. That catches unknown action keys, dangling edge refs, unknown trigger types, and invalid action_config early — before a bad definition reaches the runtime, and without a running Ductor or Postgres.

Definition size limits

Structural validation caps how large a single definition can be. These bounds are checked first, before any per-step graph analysis, and a definition over either limit fails validation with a single coded violation:

The edge cap keeps the per-step graph scans (predecessor lookup, dataflow, cycle detection) clear of the O(V·E) blow-up a dense graph would cause. At the 500-step ceiling it still allows an average fan-out of ten edges per step — well above real workflows, which typically run one to four. Real DAGs never approach either limit; hitting one almost always means a generated definition has looped.

3. Load the definition

There are two ways to get a definition into Ductor.

export DUCTOR_WORKFLOWS_DIR=./workflows
ductor serve
# or: ductor serve --workflows-dir ./workflows

4. Publish a version

A freshly created or edited definition is a draft. Runs pin to published versions, so publish when you're ready to execute:

curl -s -X POST http://localhost:8080/api/workflow-definitions/$DEF_ID/publish \
  -H "X-Tenant-ID: $DUCTOR_TENANT_ID"

Publishing is what makes the definition immutable and executable — subsequent edits go to a new draft, and re-publishing mints the next version under the same family_slug. In-flight runs continue on the version they started with, which is how Ductor gives you safe, versioned deploys of workflow logic.

5. Trigger it and watch it run

A published version is executable, so trigger a run and watch it move to completion. These are the same endpoints Getting Started uses:

RUN_ID=$(curl -s -X POST http://localhost:8080/api/v2/workflow-definitions/$DEF_ID:trigger \
  -H "X-Tenant-ID: $DUCTOR_TENANT_ID" -H "Content-Type: application/json" \
  -d '{"subject_type": "demo", "subject_id": "1"}' | jq -r '.run_id')
echo "run: $RUN_ID"

curl -s http://localhost:8080/api/workflows/runs/$RUN_ID \
  -H "X-Tenant-ID: $DUCTOR_TENANT_ID" | jq '.data.summary.status'

curl -s http://localhost:8080/api/workflows/runs/$RUN_ID/steps \
  -H "X-Tenant-ID: $DUCTOR_TENANT_ID" | jq

To see why this survives a crash — kill the engine mid-run and watch the run pick up exactly where it left off — walk the crash-resume demo in Getting Started.

6. Export a published definition

To capture the exact published YAML (for GitOps, review, or promotion between environments), export by family slug — it writes deterministic YAML to stdout:

ductor workflow export hello_world > hello_world.published.yaml

How this maps to the runtime

Once published and triggered, a run is owned by the serial Coordinator, which loads run state, applies pending attempt results, computes the next tick, persists atomically with optimistic locking, and then dispatches newly-runnable steps to parallel Step Workers. Your edges, checkpoints, and compensation blocks are exactly what the Coordinator evaluates on each tick.

Next steps