Getting Started
Stand up Ductor locally with Docker, seed a tenant, then define, run, and crash-resume your first durable workflow.
This guide takes you from a clean checkout to a running engine that does its actual job. Ductor clears work through one lifecycle — priced → routed → executed → settled → proven — and this quickstart is the shortest path to its load-bearing middle: you'll bring up the stack, mint a tenant and API keys, define a small two-step workflow, trigger it, watch it complete, and then kill the process mid-run and watch the run pick up where it left off. By the end you'll have proven the executed stage — durable execution — on your own machine; the pricing, routing, and settlement stages build on the same engine.
- You need
- Docker, Compose, curl, and jq
- You will build
- A published two-step workflow
- You will prove
- The run resumes after a process restart
This page is the journey; Run Ductor locally is the reference
This guide gets you to your first running (and crash-resumed) workflow. For the compose internals,
the bare-binary path (ductor dev / serve), service names, ports, and the full DUCTOR_*
environment reference, see Run Ductor locally.
Prerequisites
- Docker and Docker Compose — the fastest path to a full stack.
- Bash,
curl(7.76 or later), andjq— to drive the API and read JSON responses. Run the command blocks in the same Bash session.
The compose stack brings up everything else for you: TimescaleDB (Postgres 16), Dragonfly (a Redis-compatible cache), and Ductor itself. The production image is a static, CGO-disabled Go binary on Alpine that runs as a non-root user — there's nothing else to install on the host.
Administrative setup: bring it up
Run the stack. Clone the repo, check out a release, and start the core
services. Pin to a tagged release rather than tracking master so you know
exactly what you're standing up. The commands below use v1.8.0 as a concrete
example; choose the release you have validated from the
releases page. make up always
rebuilds the Ductor image and stamps the checked-out git version into it, so you
can never silently run a stale binary:
git clone https://github.com/ductor-io/ductor.git
cd ductor
git checkout v1.8.0 # example pin; replace with the release you have validated
make upUnder the hood this is docker compose up -d --build, bringing up TimescaleDB,
Dragonfly, and Ductor together. (There's also make up-full if you want the
optional observability stack — VictoriaMetrics, VictoriaLogs, VictoriaTraces, and
Grafana.)
Ductor reads configuration from configs/ductor.yaml, and every key can be
overridden with a DUCTOR_ environment variable (dots become underscores). The
compose file already wires the database and cache URLs and enables
DUCTOR_DATABASE_AUTO_MIGRATE, so migrations run automatically on boot —
there's no separate migrate step for the compose path.
The encryption key is not optional in production
DUCTOR_CONNECTOR_ENCRYPTION_KEY is a base64 32-byte key that decrypts every stored connector
credential at rest. The compose stack runs fine without it for this walkthrough (no connectors),
but for anything real, inject it from your secret manager, never commit it, and rotate it
deliberately. See Connectors.
Wait for it to become healthy. The gateway serves liveness on /health and
readiness on /ready. Wait for readiness before making tenant requests:
until curl -fsS --max-time 5 http://localhost:8080/ready >/dev/null 2>&1; do sleep 2; doneConfirm exactly what you're running — the version subcommand prints the stamped
release tag and build time straight out of the binary:
docker compose exec -T ductor /usr/local/bin/ductor version
# version=v1.8.0 build_time=2026-01-15T12:00:00Z/health is a pure liveness probe. /ready runs the same configured dependency
aggregate as the Connect Ready RPC and permits traffic unless it is unhealthy.
A readiness pass is separate from the tenant operational verdict. See
Health checks.
The Compose stack serves REST, Connect-RPC, and native gRPC on the shared
:8080 listener, and Prometheus metrics on :9090. A bare binary uses api.addr
(default :50052) for the shared request listener.
The workflow runtime is on by default
The DAG coordinator is controlled by workflow_runtime.enabled, which defaults to true —
durable workflow execution is running from first boot, nothing to switch on. For a routing-only
deployment, opt out with DUCTOR_WORKFLOW_RUNTIME_ENABLED=false. See the Configuration
reference.
Administrative setup: seed a tenant
Every tenant-scoped call needs a tenant. The default Compose stack starts with
auth disabled; that is not sufficient authority for native workflow mutations.
Complete this administrative setup, enable authentication, and issue an ordinary
key before the user journey. The seed-demo subcommand creates a real tenant and seeds
dashboard-visible demo data. It only needs Postgres, and re-running it without
--reset is idempotent:
docker compose exec -T ductor /usr/local/bin/ductor seed-demo --no-migrate--no-migrate skips migrations because the container already auto-migrated on
boot. The seed creates one fixed-UUID tenant, demo, and prints a summary like:
=== ductor seed-demo: done ===
Seeded tenants (use the UUID as the X-Tenant-ID):
- demo 00000000-0000-0000-0000-000000000001To mint the bootstrap tenant admin API key, add --emit-api-key. Keep it
separate from the key used for the user journey. Minting rotates the key,
so it is only allowed together with --reset, and --reset is destructive
(it wipes the demo tenant's data first) so it must be acknowledged explicitly:
DUCTOR_ENVIRONMENT=development docker compose exec -T ductor /usr/local/bin/ductor seed-demo \
--no-migrate --reset --emit-api-key \
--acknowledge-destructive-reset=DELETE_DEMO_TENANTS_ONLY \
--allow-reset-database ductor \
--allow-reset-environment development \
--allow-reset-tenant-id 00000000-0000-0000-0000-000000000001 \
--allow-reset-tenant-id 22222222-2222-2222-2222-222222222222The second tenant id is the retired auxiliary demo tenant; reset requires the complete compiled set so an older seed's leftovers are removed too. The last line of the output is the plaintext key, printed once — the database only stores its hash:
DUCTOR_SEED_API_KEY=duk_...Capture the tenant UUID and the key:
export DUCTOR_TENANT_ID="00000000-0000-0000-0000-000000000001"
export DUCTOR_SEED_API_KEY="<value printed by seed-demo>"Use the seeded UUID as your tenant id
Workflow runs are stored against a UUID tenant column, so scope your calls with the seeded UUID
rather than the label demo. The header name is literally X-Tenant-ID.
Enable authentication with a local Compose override. Use this same override for subsequent starts and restarts:
set -euo pipefail
cat > compose.auth.yaml <<'YAML'
services:
ductor:
environment:
DUCTOR_AUTH_ENABLED: "true"
DUCTOR_AUTH_ALLOW_ANONYMOUS: "false"
YAML
ductor_compose() {
docker compose -f docker-compose.yml -f compose.auth.yaml "$@"
}
ductor_compose up -d ductor
until curl -fsS --max-time 5 http://localhost:8080/ready >/dev/null 2>&1; do sleep 2; doneAs administrator, ensure the default environment exists. A freshly seeded
tenant created after migrations may have no environment yet. A failed inventory
request is an error; it must not trigger creation. This local setup stops if the
inventory is paginated; finish reading those pages before deciding an environment
is absent. A successful empty protobuf collection omits environments:
ENVIRONMENTS=$(curl --fail-with-body --silent --show-error --max-time 30 \
-H "Authorization: Bearer $DUCTOR_SEED_API_KEY" \
http://localhost:8080/api/v2/environments)
echo "$ENVIRONMENTS" | jq -e '(.next_page_token // "") == ""' >/dev/null
DEFAULT_ENV=$(echo "$ENVIRONMENTS" | jq -ce '(if has("environments") then .environments | arrays else [] end) | map(select(.key == "default"))')
if [ "$(echo "$DEFAULT_ENV" | jq 'length')" = 0 ]; then
curl --fail-with-body --silent --show-error --max-time 30 \
-X POST http://localhost:8080/api/v2/environments \
-H "Authorization: Bearer $DUCTOR_SEED_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"key":"default","display_name":"Local development","purpose":"custom","default":true}' \
| jq -e '.environment.key == "default"'
fi
unset ENVIRONMENTS DEFAULT_ENVReview an existing environment's policy before using it. Native creation requires an active, mutable environment that allows direct mutation and does not require promotion. Do not relax a production policy to run this local example.
As administrator, issue the ordinary operator key scoped to the demo environment:
ISSUED=$(curl --fail-with-body --silent --show-error --max-time 30 \
-X POST http://localhost:8080/api/v2/api-keys \
-H "Authorization: Bearer $DUCTOR_SEED_API_KEY" \
-H 'Content-Type: application/json' \
-d "$(jq -nc '{name:"local-workflow-user",roles:["operator"],environment_mode:"selected",environment_keys:["default"],expires_at:(now+3600|todate)}')")
DUCTOR_API_KEY=$(echo "$ISSUED" | jq -er '.secret | strings | select(length > 0)')
export DUCTOR_API_KEY
unset ISSUEDDo not use the seed administrator key for the measured user actions below.
User actions: configure requests and check access
Native workflow mutations require an authenticated principal and current
environment authority. The setup above issues an ordinary key; for an existing
deployment, have an administrator issue an ordinary API key
with workflow:read, workflow:write, workflow_run:read, and
workflow_definition:write scopes
and an appropriate role for your tenant and environment; see Issue & use API keys. Set
DUCTOR_API_KEY to that key. Keep the seed admin key for administrative setup.
Configure requests once. The helper includes the ordinary bearer token and
fails on HTTP errors. pipefail and the response checks prevent a failed request
from appearing as a zero count or a null identifier:
set -euo pipefail
: "${DUCTOR_API_KEY:?Set your ordinary tenant API key}"
BASE=${DUCTOR_API_URL:-http://localhost:8080}
ductor_api() {
curl --fail-with-body --silent --show-error --max-time 30 \
-H "Authorization: Bearer $DUCTOR_API_KEY" \
-H "X-Tenant-ID: $DUCTOR_TENANT_ID" "$@"
}
ductor_api "$BASE/api/workflows/runs" | jq -e '
if (.metadata.fetched_at | type) != "string" then
error("missing observation metadata")
else
(if has("data") then .data else [] end | arrays | length)
end'The seeded tenant should show demo runs. A successful empty list omits the
protobuf data field; the check accepts that only after a successful HTTP request
with observation metadata. An HTTP error or malformed response is a failed
observation, not an empty backlog. Stop and resolve that error before creating
work.
Two things to internalize from this call:
- REST paths are
/api/...(and/api/v2/...for newer services), not/v1/.... The REST surface is generated from the same protobuf definitions as the Connect-RPC API, so anything you can do over gRPC you can do with plain JSON — all snake_case. - Authenticated calls derive the tenant from the token. When supplied,
X-Tenant-IDmust match it; these examples include the seeded UUID explicitly.
Prefer typed clients? Point Connect or gRPC clients at the shared :8080 Compose listener (or
your configured api.addr). The full request/response reference lives in the API
Reference.
Run your first workflow
Now make the engine do its job. You'll define a small two-step DAG that needs no
connectors — a dataflow step (a deterministic in-coordinator transform) feeding
a wait step (a durable delay) — then publish it, trigger a run, and poll it to
completion.
Define a draft. Create a workflow definition with two steps and one edge.
Step types accept the protobuf enum name (STEP_TYPE_DATAFLOW) and edges accept
EDGE_TYPE_SUCCESS; args is free-form JSON:
DEF=$(ductor_api -X POST "$BASE/api/workflow-definitions" \
-H "Content-Type: application/json" -d '{
"environment": "default",
"family_slug": "hello-world",
"title": "Hello World",
"steps": [
{"ref": "greet", "type": "STEP_TYPE_DATAFLOW", "title": "Build greeting",
"args": {"operation": "project", "input_expr": "TRIGGER",
"redaction_policy": "none", "project": {"message": "\"hello\""}}},
{"ref": "pause", "type": "STEP_TYPE_WAIT", "title": "Short delay",
"args": {"duration": "2s"}}
],
"edges": [{"source_ref": "greet", "target_ref": "pause", "type": "EDGE_TYPE_SUCCESS"}]
}')
ID=$(echo "$DEF" | jq -er '.data.id | strings | select(length > 0)')
echo "definition: $ID"The dataflow step's input_expr selects its input (TRIGGER is the run input).
Its project operation maps output fields to CEL expressions (here, the literal
string "hello"). redaction_policy: "none" makes this synthetic greeting
inspectable; keep the default hashed samples when processing sensitive data.
The wait step takes exactly one of
duration, duration_ms, until, cron, or until_expr; duration is a Go
duration string such as "2s" or "5m".
Publish it. Publication returns a new immutable version id under
published.id; the original id remains the editable draft. Capture the published
id before triggering:
PUBLISHED=$(ductor_api -X POST "$BASE/api/workflow-definitions/$ID/publish")
ID=$(echo "$PUBLISHED" | jq -er '.published.id | strings | select(length > 0)')Trigger a run. The trigger endpoint returns the new run_id:
RUN=$(ductor_api -X POST "$BASE/api/v2/workflow-definitions/$ID:trigger" \
-H "Content-Type: application/json" \
-d '{"subject_type": "demo", "subject_id": "1", "input": {"name": "world"}}')
RID=$(echo "$RUN" | jq -er '.run_id | strings | select(length > 0)')
echo "run: $RID"If you'd rather block until the run reaches a terminal checkpoint instead of
polling, add "wait_for_checkpoint": true, "checkpoint_timeout_seconds": 10 to
the trigger body.
Poll it to completion. The run moves pending → running → completed. Read the
status, then inspect the per-step attempt list:
ductor_api "$BASE/api/workflows/runs/$RID" | jq -e '.data.summary.status | strings'
ductor_api "$BASE/api/workflows/runs/$RID/steps" | jqRepeat the status read until it says "completed". Inspect the coordinator's
node snapshot to verify the greeting; in-coordinator dataflow output is available
there even when it is absent from the attempt list:
ductor_api "$BASE/api/v2/workflow-runs/$RID/snapshot" > hello-world.snapshot.json
jq -e '.nodes.greet.latest_output_json | fromjson | .result | select(.message == "hello")' \
hello-world.snapshot.jsonThe output should be {"message":"hello"}. The workflow has completed and its
visible result matches the intended greeting.
Watch it survive a crash
Durable execution means a run's progress lives in Postgres, not in process
memory — so killing the engine mid-run doesn't lose the run. Prove it by parking
a run on a longer wait, crashing the container, and watching it resume.
This restarts Ductor while retaining Postgres and cache data. It is not a
database restore test; backup recovery requires a separate restore drill against
your deployment's storage and encryption keys.
Publish a run with a long wait. Reuse the two-step shape from above, but set
the wait duration to 60s so you have a comfortable window to crash it:
DEF=$(ductor_api -X POST "$BASE/api/workflow-definitions" \
-H "Content-Type: application/json" -d '{
"environment": "default",
"family_slug": "durable-demo",
"title": "Durable Demo",
"steps": [
{"ref": "greet", "type": "STEP_TYPE_DATAFLOW", "title": "Build greeting",
"args": {"operation": "project", "input_expr": "TRIGGER",
"redaction_policy": "none", "project": {"message": "\"hello\""}}},
{"ref": "pause", "type": "STEP_TYPE_WAIT", "title": "Long delay",
"args": {"duration": "60s"}}
],
"edges": [{"source_ref": "greet", "target_ref": "pause", "type": "EDGE_TYPE_SUCCESS"}]
}')
ID=$(echo "$DEF" | jq -er '.data.id | strings | select(length > 0)')
PUBLISHED=$(ductor_api -X POST "$BASE/api/workflow-definitions/$ID/publish")
ID=$(echo "$PUBLISHED" | jq -er '.published.id | strings | select(length > 0)')
RID=$(ductor_api -X POST "$BASE/api/v2/workflow-definitions/$ID:trigger" \
-H "Content-Type: application/json" \
-d '{"subject_type": "demo", "subject_id": "2"}' | jq -er '.run_id | strings | select(length > 0)')
echo "run: $RID"Confirm the run is still "running" and retain its wakeup evidence before
killing the process. A running status alone does not prove it reached the wait:
ductor_api "$BASE/api/workflows/runs/$RID" | tee durable-before.json | jq -e '.data.summary.status | strings'
ductor_api "$BASE/api/workflows/runs/$RID/wakeups" | tee durable-wakeups-before.json | jqWait until the wakeup list shows the delay for this run, and keep the run id.
Kill the engine, then bring it back. While the run is still waiting:
ductor_compose kill ductor
ductor_compose up -d ductor
until curl -fsS --max-time 5 http://localhost:8080/ready >/dev/null 2>&1; do sleep 2; doneWatch it resume and complete. Poll again once the delay has elapsed:
ductor_api "$BASE/api/workflows/runs/$RID" | jq -e '.data.summary.status | strings'The same run id reaches "completed". Inspect its steps and wakeups again and
compare them with the retained evidence to confirm the persisted run finished.
This proves recovery of this deterministic wait workflow; output quality and
external action effects need their own checks.
Why this works. A wait step is persisted with a durable wakeup, not an
in-memory timer, so a parked run is just a row waiting to be ticked. On restart,
the workflow runtime coordinator scans for runs due for a tick every
poll_interval (5s by default) and reclaims any run whose previous coordinator
lease went stale after recovery_stale_after (30s by default). It only picks up
non-terminal runs; completed runs are skipped. This does not establish
exactly-once external effects: action retries and ambiguous provider outcomes
still require the connector's idempotency and recovery contract.
Concretely, a cleanly-parked run resumes within roughly the poll interval, or up
to the stale-after window if the previous coordinator's lease has to expire
first.
Next steps
Understand the execution model
How the Coordinator and Step Workers divide responsibility, tick by tick.
Learn the workflow vocabulary
Steps, edges, scatter/gather, sub-workflows, waits, and ContinueAsNew.
Route your first event
Walk the Validate → Enrich → Filter → Select → Assign pipeline.
See the architecture
The layered design, data flow, and how it all fits together.
Going further: enable the inbound MCP server and in-product chat agent under AI & agents, or define your workflows in code with the Go and Python SDKs instead of authoring DAGs through the API.