Durable Chat Agent

The chat agent is Ductor's operator sidekick. It can inspect workflows, runs, connectors, and pools through governed tools, but it does not run as a browser-owned LLM loop. Every turn is admitted to the same journal-backed agent runtime used by workflow agents before provider work begins.

That distinction matters operationally: closing a tab, losing a network connection, or restarting an SSE request does not cancel accepted work. The browser reconnects to the existing session and asks for events after its last committed sequence.

Enable the canonical runtime

Chat is off by default. An enabled deployment must opt into the durable runtime and pin a model:

aichat:
  enabled: true
  durable_runtime: true
  model: gemini-2.5-flash-lite
  max_iterations: 8
  max_tool_calls: 16

Startup fails closed unless all of these are available:

• ai_inference.enabled=true with at least one provider that serves the pinned model;
• Postgres with the migrations included in your Ductor release applied;
• the agent-tool policy service and a ready operational MCP backend;
• the configured provider credential environment variable.

The legacy in-process chat loop is not a valid production configuration: aichat.enabled=true requires aichat.durable_runtime=true.

1. Admit a turn

Admission and event delivery are separate HTTP operations. turn_id is required and is the caller's idempotency key; reuse it only when retrying the exact same user turn.

POST /api/v1/chat/turns
Content-Type: application/json

{
  "conversation_id": "optional-existing-conversation",
  "turn_id": "turn_01JZ6Y8J4X0YH1K1Q9A8M3S7TD",
  "entity_type": "run",
  "entity_id": "run_01JZ6X...",
  "agent_preset_id": "optional-tenant-preset-id",
  "message": "Why did this run fail, and what is safe to retry?"
}

Successful admission returns 202 Accepted before the detached worker calls a provider:

{
  "conversation_id": "01JZ6YB3D5...",
  "session_id": "01JZ6YB3D5...",
  "turn_id": "turn_01JZ6Y8J4X0YH1K1Q9A8M3S7TD",
  "sequence": 1,
  "state": "accepted"
}

The journal rejects a reused session/turn identity whose immutable input, definition, manifest, policy receipt, or budget differs. Durable user and assistant transcript rows are also unique by tenant, conversation, turn, and role, so a completed retry does not duplicate the transcript.

List and inspect sessions

The durable chat API has its own cursor-based session projection for reconnecting a dashboard client:

List results include state and the latest committed cursor. Detail adds the current turn, resync_cursor, pending durable input, and a backward transcript cursor. Use these projections to rebuild UI state; use the event stream for incremental progress.

The separately generated /api/v2/agent-sessions service remains the general tenant/preset transcript catalog. /api/v1/chat/sessions is the canonical dashboard chat protocol and carries live state plus replay cursors.

2. Subscribe or reconnect

GET /api/v1/chat/sessions/{session_id}/events
Accept: text/event-stream
Last-Event-ID: 17

Each public event with a durable sequence includes the same value in both the SSE id: field and JSON sequence. Persist it only after processing that frame. On reconnect, send it as Last-Event-ID; clients that cannot set request headers may use ?after=17.

id: 18
event: text_delta
data: {"type":"text_delta","delta":"The retry was blocked","sequence":18}

id: 21
event: done
data: {"type":"done","final_message":{"role":"assistant","content":"..."},"sequence":21}

The server replays events strictly after the cursor, then hands off to live delivery without a gap. A cursor ahead of the session returns 409 Conflict. Reconnecting at the terminal sequence returns no duplicate done event. Slow readers cannot block provider or tool work: notifications are only wake-ups and the journal remains the source of truth.

Agent events are retained indefinitely by the canonical schema unless an operator installs an external archival policy. If pruning creates a gap, a cursor older than the earliest available event returns 410 Gone:

{
  "error": "resync_required",
  "requested_sequence": 41,
  "earliest_available_sequence": 80,
  "latest_sequence": 126
}

Fetch a current session projection and reopen after its committed sequence. Never guess that a missing range completed successfully.

3. Approve or reject a tool

When a policy requires approval, the runtime commits input_required and stops before tool execution. Answer the exact request with the exact journal sequence:

POST /api/v1/chat/sessions/{session_id}/inputs/{request_id}
Content-Type: application/json

{
  "expected_sequence": 12,
  "approved": true,
  "response": {
    "reason": "Validated against ticket INC-2041"
  }
}

The response is single-use. Stale sequences, mismatched request IDs, and cross-tenant session IDs fail before execution. Approval resumes the already-authorized invocation; it does not ask the model to propose a new one. Rejection becomes a durable tool-result message so the next model request has a protocol-valid explanation that no mutation occurred.

Actor identity and roles come from the authenticated principal, never the JSON body. The same endpoint carries schema-validated question, budget_extension, external_auth, and reconciliation responses. Do not send OAuth tokens or provider keys; store secrets through the credential service and return only a non-secret reference.

4. Cancel accepted work

Closing an event stream or aborting its HTTP request only disconnects the client. To cancel an admitted turn, send an explicit, idempotent durable command:

POST /api/v1/chat/sessions/{session_id}/turns/{turn_id}/cancel
Content-Type: application/json
Ductor-Agent-Protocol-Version: 1

{
  "command_id": "cancel_01JZ7A2XKQ9X2R4J6M8N"
}

Reuse command_id only when retrying the same cancellation. The result reports cancelled, duplicate, or too_late, together with current state and committed sequence. Cancellation is durable and cooperative: it prevents later transitions, but cannot prove that an external provider or tool effect already in flight did not occur. Ambiguous effects still require reconciliation.

Provider streaming and usage

The runtime calls the AI inference vertical, which owns provider routing, translation, circuit protection, tenant budget reservations, and usage accounting. OpenAI-compatible SSE is normalized into provider-neutral content, tool, usage, and terminal events. Content fragments are journaled before being published to chat clients; provider reasoning or raw error bodies are not persisted.

A disconnect affects only the subscription. Accepted work continues, and provider usage is reconciled once by the inference vertical. Replaying an event cursor never issues another provider request.

Durable usage events can correlate requests, tokens, cost, latency, and errors with agent_definition_id, agent_session_id, workflow identity, and an experiment exposure. See Durable usage and attribution for the supported metadata keys and pricing provenance.

Production client checklist

1. Send Ductor-Agent-Protocol-Version: 1 on every request.
2. Generate a unique turn_id before admission and retain it across HTTP retries.
3. Store session_id and the highest fully processed event sequence.
4. Reconnect with exponential backoff and Last-Event-ID (or ?after=).
5. Treat 409 as a stale/future cursor or input conflict; treat 410 as an explicit resynchronization requirement.
6. Render input_required as an explicit approval decision; never auto-approve in the UI.
7. Use the cancel endpoint—not stream abort—to stop accepted work.
8. Stop reconnecting after done, error, or an input_required frame until the user acts.
9. Do not log request bodies, provider payloads, or approval responses without redaction.

Related