# Define & publish a workflow (/docs/guides/define-workflow)



This guide is the **executed** stage of the clearing lifecycle, hands-on: a
**workflow definition** is a DAG of steps that Ductor's coordinator executes
durably once <Term name="Work" /> is routed to a <Term name="Worker" />.
Definitions are authored as YAML, validated, loaded, and then **published** —
publishing freezes an immutable version that the runtime pins runs to. This guide
walks the full lifecycle for a small definition.

<Callout title="Prerequisite: the runtime must be enabled">
  The DAG coordinator is gated behind `workflow_runtime.enabled` (default
  `true`). If you disabled it, set `DUCTOR_WORKFLOW_RUNTIME_ENABLED=true` before
  running definitions.
</Callout>

<Callout type="info" title="Prefer code over YAML?">
  This guide authors definitions in YAML, but the same workflows can be built in
  Go, TypeScript, or Python with the [SDKs](/docs/sdks) — the SDK compiles to the
  identical definition the runtime executes here. YAML is the portable, GitOps
  form; the SDKs give you types, IDE completion, and inline tests.
</Callout>

## 1. Author the definition [#1-author-the-definition]

A definition is a YAML document with a `type: workflow_definition` envelope and
a `definition` body. The body declares `steps`, the `edges` between them, one or
more `triggers`, and optional `checkpoints`. Here is the smallest legal
definition — two `noop` action steps joined by a success edge:

```yaml
type: workflow_definition
definition:
  family_slug: hello_world
  title: Hello World
  description: Smallest executable DAG — two steps joined by a success edge.
  steps:
    - ref: a
      type: action
      action: noop:noop
      args:
        key: value
    - ref: b
      type: action
      action: noop:noop
  edges:
    - source_ref: a
      target_ref: b
      type: success
  triggers:
    - id: t1
      type: manual
  checkpoints:
    - name: accepted
      step_refs:
        - b
```

Key fields:

| Field            | Meaning                                                                                                                                                  |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `family_slug`    | Stable identifier for this workflow **family**. Publishing creates versions (`v1`, `v2`, …) under the slug.                                              |
| `steps[].ref`    | Unique handle for a step within the definition; edges and expressions reference it.                                                                      |
| `steps[].type`   | Step kind. `action` is the workhorse; there are **15** step types in total (see below).                                                                  |
| `steps[].action` | For action steps, the `provider:action` (or `provider.resource.action`) key resolved against the connector [Action Registry](/docs/concepts/connectors). |
| `edges[].type`   | Transition condition — `success`, and error/compensation variants.                                                                                       |
| `triggers[]`     | How runs start — `manual`, `event` (with match patterns), cron schedule, etc.                                                                            |
| `checkpoints[]`  | Named progress markers over a set of step refs, used for acceptance and visibility.                                                                      |

### The 15 step types [#the-15-step-types]

`action` is one of **15** authorable step types. The rest cover control flow,
fan-out, human-in-the-loop, AI, and cross-runtime calls:

| Type          | Purpose                                                              |
| ------------- | -------------------------------------------------------------------- |
| `action`      | Call a connector action or a routing-finalization adapter.           |
| `wait`        | Pause until a timeout or external signal.                            |
| `approval`    | Pause until a human approves or rejects.                             |
| `scatter`     | Fan out over a collection.                                           |
| `for_each`    | Process a frozen collection sequentially through one child workflow. |
| `gather`      | Collect results from a preceding `scatter`.                          |
| `dataflow`    | Transform/filter/partition/reduce/project payloads via bounded CEL.  |
| `race`        | Resolve on the first qualifying predecessor.                         |
| `subworkflow` | Trigger a child workflow run.                                        |
| `human_loop`  | Pause until a case is resolved by a human analyst.                   |
| `ai_action`   | A single LLM inference call through the AI inference proxy.          |
| `ai_agent`    | An agentic tool-calling loop with execution bounds.                  |
| `bridge`      | Invoke a remote SDK step over the bridge protocol.                   |
| `digest`      | Collapse N upstream events sharing a key into one downstream tick.   |
| `event_set`   | Park until a correlated set of external events completes.            |

See the [DAG workflow model](/docs/concepts/dag-workflow-model) for the full
semantics of each. The `ai_action` / `ai_agent` steps are covered in
[AI](/docs/ai); `bridge` steps in [SDKs](/docs/sdks).

### Event triggers [#event-triggers]

An `event` trigger fires runs when a matching event arrives. Each trigger
carries one or more **patterns**, and every pattern is either a literal event
name (`stripe.invoice.paid`) or a **suffix wildcard** (`stripe.invoice.*`).

<Callout type="warn" title="Wildcard dispatch is off by default">
  Literal-name triggers always work. Suffix-wildcard fan-out is gated behind the
  default-OFF feature flag `ductor.events.wildcards.enabled` — leave it off
  unless you have explicitly opted into wildcard event routing.
</Callout>

### Steps that call connectors [#steps-that-call-connectors]

Action steps can call a third-party system through a connector. Reference a
connection and pass templated arguments (`${{ ... }}` expressions resolve at
dispatch time against trigger payload, variables, and prior step outputs):

```yaml
steps:
  - ref: create_invoice
    type: action
    action: stripe.invoices.create
    connection_ref: "${{ VARS.stripe_connection_ref }}"
    connection_ref_mode: direct
    args:
      customer_id: "${{ TRIGGER.payload.customer_id }}"
      amount: "${{ TRIGGER.payload.amount_cents }}"
      currency: "${{ TRIGGER.payload.currency }}"
    retry_max_attempts: 3
    retry_backoff_ms: 1000
    timeout_ms: 10000
    on_error: fail
```

Steps with external side effects can declare **compensation** — inverse actions
that run `on_failure` or `on_cancel` — so a partially-completed run unwinds
cleanly.

## 2. Validate offline [#2-validate-offline]

Before loading anything, lint the definition with the CLI. `workflow validate`
parses the YAML and resolves every reference **without touching the database** —
it exits non-zero on any problem:

```bash
ductor workflow validate ./hello_world.yaml
```

Resolution is fully offline. Action keys are checked against the
`ActionRegistry` ∪ `ExecutorRegistry`, strategy references against
`pkg/strategy.Registry`, and hook points against `pkg/middleware.AllHookPoints`.
That catches unknown action keys, dangling edge refs, unknown trigger types, and
invalid `action_config` early — before a bad definition reaches the runtime, and
without a running Ductor or Postgres.

### Definition size limits [#definition-size-limits]

Structural validation caps how large a single definition can be. These bounds
are checked first, before any per-step graph analysis, and a definition over
either limit fails validation with a single coded violation:

| Limit                | Maximum  | Violation code            | Message                                   |
| -------------------- | -------- | ------------------------- | ----------------------------------------- |
| Steps per definition | **500**  | `MAX_STEP_COUNT_EXCEEDED` | `definition has N steps, maximum is 500`  |
| Edges per definition | **5000** | `MAX_EDGE_COUNT_EXCEEDED` | `definition has N edges, maximum is 5000` |

The edge cap keeps the per-step graph scans (predecessor lookup, dataflow,
cycle detection) clear of the `O(V·E)` blow-up a dense graph would cause. At the
500-step ceiling it still allows an average fan-out of ten edges per step — well
above real workflows, which typically run one to four. Real DAGs never approach
either limit; hitting one almost always means a generated definition has looped.

## 3. Load the definition [#3-load-the-definition]

There are two ways to get a definition into Ductor.

<Tabs items="[&#x22;Filesystem scan&#x22;, &#x22;Definition API&#x22;]">
  <Tab value="Filesystem scan">
    Point `workflows.dir` at a directory of definition YAML and Ductor loads every
    file it finds on boot:

    ```bash
    export DUCTOR_WORKFLOWS_DIR=./workflows
    ductor serve
    # or: ductor serve --workflows-dir ./workflows
    ```
  </Tab>

  <Tab value="Definition API">
    Create and manage definitions over REST / Connect via
    `WorkflowDefinitionService`:

    | Action        | Verb + path                                                 |
    | ------------- | ----------------------------------------------------------- |
    | Create        | `POST /api/workflow-definitions`                            |
    | Get           | `GET /api/workflow-definitions/{id}`                        |
    | List          | `GET /api/workflow-definitions`                             |
    | Update draft  | `PATCH /api/workflow-definitions/{id}/draft`                |
    | Publish       | `POST /api/workflow-definitions/{id}/publish`               |
    | Duplicate     | `POST /api/workflow-definitions/{id}/duplicate`             |
    | Archive       | `POST /api/workflow-definitions/{id}/archive`               |
    | Env overrides | `PUT /api/workflow-definitions/{workflow_id}/env-overrides` |
  </Tab>
</Tabs>

## 4. Publish a version [#4-publish-a-version]

<Callout title="Environment for the commands below">
  These commands assume the local stack and seeded tenant from
  [Getting Started](/docs/getting-started): auth is disabled locally, so
  requests need only the `X-Tenant-ID` header (in production, authenticate per
  [Auth & Security](/docs/auth)). `$DEF_ID` is the definition id the create call
  returns (`.data.id`).
</Callout>

A freshly created or edited definition is a **draft**. Runs pin to *published*
versions, so publish when you're ready to execute:

```bash
curl -s -X POST http://localhost:8080/api/workflow-definitions/$DEF_ID/publish \
  -H "X-Tenant-ID: $DUCTOR_TENANT_ID"
```

Publishing is what makes the definition immutable and executable — subsequent
edits go to a new draft, and re-publishing mints the next version under the same
`family_slug`. In-flight runs continue on the version they started with, which
is how Ductor gives you safe, versioned deploys of workflow logic.

## 5. Trigger it and watch it run [#5-trigger-it-and-watch-it-run]

A published version is executable, so trigger a run and watch it move to
completion. These are the same endpoints [Getting
Started](/docs/getting-started) uses:

<Steps>
  <Step>
    **Trigger a run.** Post to the version's `:trigger` endpoint; the response
    carries the new `run_id`:

    ```bash
    RUN_ID=$(curl -s -X POST http://localhost:8080/api/v2/workflow-definitions/$DEF_ID:trigger \
      -H "X-Tenant-ID: $DUCTOR_TENANT_ID" -H "Content-Type: application/json" \
      -d '{"subject_type": "demo", "subject_id": "1"}' | jq -r '.run_id')
    echo "run: $RUN_ID"
    ```
  </Step>

  <Step>
    **Poll the run status.** The run moves `pending → running → completed`; the
    status lives at `.data.summary.status`:

    ```bash
    curl -s http://localhost:8080/api/workflows/runs/$RUN_ID \
      -H "X-Tenant-ID: $DUCTOR_TENANT_ID" | jq '.data.summary.status'
    ```
  </Step>

  <Step>
    **Inspect the step attempts.** List the per-step detail to see each step's
    attempts and outcome:

    ```bash
    curl -s http://localhost:8080/api/workflows/runs/$RUN_ID/steps \
      -H "X-Tenant-ID: $DUCTOR_TENANT_ID" | jq
    ```
  </Step>
</Steps>

To see why this survives a crash — kill the engine mid-run and watch the run
pick up exactly where it left off — walk the [crash-resume
demo](/docs/getting-started#watch-it-survive-a-crash) in Getting Started.

## 6. Export a published definition [#6-export-a-published-definition]

To capture the exact published YAML (for GitOps, review, or promotion between
environments), export by family slug — it writes deterministic YAML to stdout:

```bash
ductor workflow export hello_world > hello_world.published.yaml
```

## How this maps to the runtime [#how-this-maps-to-the-runtime]

Once published and triggered, a run is owned by the serial
[Coordinator](/docs/concepts/coordinator-workers), which loads run state,
applies pending attempt results, computes the next tick, persists atomically
with optimistic locking, and then dispatches newly-runnable steps to parallel
Step Workers. Your `edges`, `checkpoints`, and compensation blocks are exactly
what the Coordinator evaluates on each tick.

## Next steps [#next-steps]

<Cards>
  <Card title="Add a connector & connection" href="/docs/guides/add-connector">
    Wire up the provider your action steps call.
  </Card>

  <Card title="DAG workflow model" href="/docs/concepts/dag-workflow-model">
    All 15 step types, edges, and trigger semantics in depth.
  </Card>

  <Card title="SDKs" href="/docs/sdks">
    Define the same workflows in Go, TypeScript, or Python.
  </Card>

  <Card title="Coordinator & Step Workers" href="/docs/concepts/coordinator-workers">
    The execution model behind every run.
  </Card>
</Cards>