# Routing & Work Experiments (/docs/management/experiments)
Experiments let you compare live behavior without losing the connection between
the variant Ductor selected and the outcome that arrived later. The experiment
service owns the configuration and lifecycle, assigns subjects consistently,
persists an immutable exposure receipt for workflow and agent executions, and
accepts delayed scores against that exact receipt.
Use an experiment when a variant is allowed to affect live behavior for a
controlled slice of traffic. Use a
[shadow campaign](/docs/strategies/governance#strategy-shadow-campaigns) when
you need counterfactual evidence with no authoritative side effects.
Every operation is tenant-scoped. Reads require `experiment:read`; creates,
assignments, lifecycle changes, scores, and deletion require
`experiment:write`.
## Supported experiment surfaces [#supported-experiment-surfaces]
`subject_scope` tells Ductor what the variants control. `subject_ref` anchors
the experiment to the exact resource being changed.
| `subject_scope` | `subject_ref` | Typical use |
| ------------------ | ------------------------------- | ----------------------------------------------------------------------------- |
| `routing_strategy` | Pool identifier | Compare strategy implementations or strategy options for routing decisions. |
| `workflow_step` | Stable workflow step identifier | Compare prompts, models, tools, thresholds, or other step behavior. |
| `agent_definition` | Agent definition identifier | Compare an agent's model, instructions, tools, or policy-bound configuration. |
Routing experiments also require `pool_id`. If `subject_scope` is omitted,
Ductor treats the experiment as `routing_strategy` and uses `pool_id` as the
subject reference.
For routing strategies, Ductor's routing pipeline can apply the selected
strategy and attribute committed outcomes. For workflow-step and agent
experiments, call the assignment API with the run, step, or agent-session
identity, then apply the returned variant in your authored runtime. Preserve
the returned exposure id for any delayed score.
## Lifecycle [#lifecycle]
An experiment is created as `draft`. Start it to admit assignments, pause it to
temporarily return traffic to the default behavior, and either complete or stop
it when data collection ends.
```mermaid
stateDiagram-v2
[*] --> draft
draft --> running: start
running --> paused: pause
paused --> running: start
running --> completed: complete
running --> stopped: stop
paused --> completed: complete
paused --> stopped: stop
```
* **`draft`** — configuration state before activation; no live assignments.
* **`running`** — the only state that accepts experiment traffic.
* **`paused`** — temporarily inactive and restartable.
* **`completed`** — concluded with results retained for analysis.
* **`stopped`** — manually terminated with results retained.
Starting a routing-strategy experiment validates every named strategy and its
options against the live strategy registry. Strategy variants must use
deterministic splitting so assignments remain reproducible across retries,
replay, and process restarts.
## Create an experiment [#create-an-experiment]
The following experiment compares two routing strategies for half of a pool's
traffic. Variant weights are relative: equal weights produce an even split
inside the admitted 50% traffic slice.
```bash
curl -s -X POST "https://api.ductor.io/api/experiments" \
-H "Authorization: Bearer $DUCTOR_API_KEY" \
-H "X-Tenant-ID: $DUCTOR_TENANT" \
-H "Content-Type: application/json" \
-d '{
"name": "Lead assignment quality",
"description": "Compare the current strategy with the candidate on qualified conversion",
"pool_id": "550e8400-e29b-41d4-a716-446655440000",
"subject_scope": "routing_strategy",
"subject_ref": "550e8400-e29b-41d4-a716-446655440000",
"traffic_split_method": "hash",
"traffic_percentage": 50,
"variants": [
{
"id": "control",
"name": "Current strategy",
"strategy": "swrr",
"weight": 1,
"is_control": true
},
{
"id": "candidate",
"name": "Learning strategy",
"strategy": "epsilon_greedy",
"strategy_options": { "epsilon": 0.1 },
"weight": 1
}
],
"metrics": [
{
"name": "qualified_conversion",
"type": "gauge",
"minimum_detectable_effect": 0.05
}
]
}'
```
Creation rejects configurations with fewer than two variants, no control
variant, duplicate variant ids, no positive variant weight, an invalid traffic
percentage, or an unavailable routing strategy. Omitted variant ids are
generated; omitted variant names default to their ids.
Start the experiment only after the draft matches the hypothesis you intend to
measure:
```bash
curl -s -X POST \
"https://api.ductor.io/api/experiments/$EXPERIMENT_ID/start" \
-H "Authorization: Bearer $DUCTOR_API_KEY" \
-H "X-Tenant-ID: $DUCTOR_TENANT"
```
## Assign a variant and capture exposure [#assign-a-variant-and-capture-exposure]
`POST /api/experiments/{experiment_id}/assign` always requires a stable
`entity_id`. Ductor uses that identity for sticky bucketing, so retries for the
same entity return the same arm.
For a workflow or agent experiment, also send at least one execution identity:
`workflow_run_id`, `workflow_step_id`, or `agent_session_id`. Ductor then returns
an `exposure` alongside the variant.
```bash
curl -s -X POST \
"https://api.ductor.io/api/experiments/$EXPERIMENT_ID/assign" \
-H "Authorization: Bearer $DUCTOR_API_KEY" \
-H "X-Tenant-ID: $DUCTOR_TENANT" \
-H "Content-Type: application/json" \
-d '{
"entity_id": "account-42",
"workflow_run_id": "run-8a12",
"workflow_step_id": "summarize-lead"
}'
```
```json
{
"variant": {
"id": "candidate",
"name": "Concise prompt",
"weight": 1,
"is_control": false
},
"exposure": {
"id": "a593d4f0b8c6f48202d78411eeb901da",
"experiment_id": "exp-7e0f",
"variant_id": "candidate",
"entity_id": "account-42",
"subject_scope": "workflow_step",
"subject_ref": "summarize-lead",
"workflow_run_id": "run-8a12",
"workflow_step_id": "summarize-lead",
"assigned_at": "2026-08-23T20:14:12Z"
}
}
```
The exposure is immutable and tenant-bound. Its identity is derived from the
experiment plus the run, step, and agent-session coordinates. Replaying the
same execution returns the original receipt even if the experiment
configuration changed later. That makes the receipt the durable attribution
boundary—not the experiment's current configuration.
Delayed scoring requires the exact `exposure.id`. Do not reconstruct variant
attribution from the current traffic split, and do not attach a score only to
an entity id. The experiment may have changed between exposure and outcome.
## Record a delayed score [#record-a-delayed-score]
Real outcomes often arrive after the run that produced them: a conversion,
human rating, refund, quality review, latency observation, or model-judge score.
Attach that value to its exposure with `POST
/api/experiments/{experiment_id}/scores`.
```bash
curl -s -X POST \
"https://api.ductor.io/api/experiments/$EXPERIMENT_ID/scores" \
-H "Authorization: Bearer $DUCTOR_API_KEY" \
-H "X-Tenant-ID: $DUCTOR_TENANT" \
-H "Content-Type: application/json" \
-d '{
"exposure_id": "a593d4f0b8c6f48202d78411eeb901da",
"metric_name": "qualified_conversion",
"value": 1,
"idempotency_key": "conversion:order-9182"
}'
```
The metric name must be declared on the experiment, the value must be finite,
and the exposure must belong to the experiment and authenticated tenant. The
idempotency key is part of the observation identity. Retrying the same exposure,
metric, and key returns the original observation instead of counting the
outcome twice.
Use an idempotency key that names the real-world fact, such as
`conversion:`, `rating:`, or `judge:`. Do
not use a random key on every retry.
## Read results [#read-results]
Results remain readable while an experiment is running and after it is paused,
completed, or stopped:
```bash
curl -s \
"https://api.ductor.io/api/experiments/$EXPERIMENT_ID/results" \
-H "Authorization: Bearer $DUCTOR_API_KEY" \
-H "X-Tenant-ID: $DUCTOR_TENANT"
```
The response groups observations by variant and returns `sample_count` plus a
`metric_means` map. Compare only metrics with enough representative samples;
an observed mean is evidence, not an automatic promotion decision. Keep
business, fairness, and safety guardrails separate from the primary objective.
## API surface [#api-surface]
| Method and path | Purpose |
| ------------------------------------------------ | -------------------------------------------------------------------------- |
| `POST /api/experiments` | Create a draft experiment. |
| `GET /api/experiments` | List experiments; filter by `pool_id` or lifecycle `status`. |
| `GET /api/experiments/{experiment_id}` | Read configuration and lifecycle state. |
| `POST /api/experiments/{experiment_id}/start` | Start a draft or resume a paused experiment. |
| `POST /api/experiments/{experiment_id}/pause` | Pause live assignment. |
| `POST /api/experiments/{experiment_id}/complete` | Conclude the experiment and retain results. |
| `POST /api/experiments/{experiment_id}/stop` | Manually stop the experiment and retain results. |
| `DELETE /api/experiments/{experiment_id}` | Delete the experiment configuration. |
| `POST /api/experiments/{experiment_id}/assign` | Return a sticky variant and, with execution identity, an exposure receipt. |
| `POST /api/experiments/{experiment_id}/scores` | Attach an idempotent delayed score to an exposure. |
| `GET /api/experiments/{experiment_id}/results` | Read per-variant sample counts and metric means. |
## Production checklist [#production-checklist]
* Write the hypothesis, primary metric, guardrails, and stopping rule before
starting.
* Use stable entity ids and deterministic splitting for any strategy variant.
* Keep one control variant and use relative weights that match the intended
allocation.
* Persist exposure ids with workflow, agent-session, or decision lineage.
* Use deterministic idempotency keys for delayed outcomes.
* Separate score collection from promotion approval; inspect sample quality and
guardrails before adopting a winner.
* Pause first when results or instrumentation look wrong. Complete only when the
planned analysis is finished; stop when the run is being terminated.
## Related [#related]
Shadow, certify, experiment, canary, and promote routing changes safely.
Trigger runs with durable correlation labels and operate their lifecycle.
Attribute AI cost, latency, and errors to experiments and assignments.