Agent Work Runtime

Ductor's agent runtime separates accepted work from connected clients. A chat request or workflow node first admits an immutable turn to an append-only journal. Only then can a runtime lease execute provider and tool work. Chat and workflow are host adapters around that same state machine.

Durable invariants

The journal stores normalized events: accepted turns, runtime leases and state changes, model deltas, tool invocations and redacted results, input requests and responses, usage, completion, and terminal failures. Raw provider reasoning and credential-bearing payloads do not belong in it.

One runtime, three hosts

• The chat host admits an operator turn and projects committed journal events into SSE. Reconnect starts strictly after Last-Event-ID.
• The workflow host derives a stable agent session from run/node identity and maps runtime outcomes to node state. Approval requests become durable workflow waits.
• The external conversation bridge verifies provider requests and explicit participant bindings, durably orders each thread, admits turns to the same runtime, and sends the terminal result back through the governed connector action path.

Neither host owns provider-specific parsing or tool execution. The inference vertical normalizes provider streams and accounts for usage; the tool broker resolves a pinned tool definition, enforces policy, executes through the connector registry, and returns a redacted receipt.

Definitions and presets

At admission, the runtime resolves an agent definition: model, instructions, output schema, tools, limits, model policy, ordered processors, memory scope, delegation policy, and policy context. Tenant presets can be version-pinned. Tool definitions are immutable for the lifetime of a turn and include the canonical name, version, and schema hash. A later catalog change therefore causes an explicit mismatch instead of silently changing an in-flight agent.

The model decision receipt records eligible candidates, the selected provider/model, policy version, scope, and reason. Every proxy request is strict: a missing or prohibited provider, or a response attributed to another provider, fails closed. An allowed provider transition must become a new durable decision; retries inside an adapter cannot hide a model-policy change.

Context, processors, and memory

The immutable transcript is audit history; the context projection is the bounded request sent to a model. Projection accounts for the selected model ceiling, reserved output tokens, tool schemas, complete assistant/tool batches, and a conservative token count. When compaction is needed, the summary records its source sequence range, generating model, prompt hash, token metadata, and supersession lineage.

Definitions pin ordered input, request, output, and error processors. The default pipeline bounds input, records injection signals, redacts secrets—including values split across stream frames—prevents tool expansion, validates the pinned model decision, supports fail-closed moderation, checks structured output, and emits a stable error taxonomy. Processor receipts contain hashes and bounded signals, not redacted content.

Durable memory is opt-in. The Postgres backend requires exact tenant, authenticated subject, entity type, and entity id; it returns only non-expired records marked model_safe, within item and byte ceilings. The projector rechecks scope and provenance and injects recall as explicitly untrusted user-role context. There is no public free-form memory insertion endpoint.

Typed input and specialist delegation

Approval is one kind of a general durable input protocol. approval, question, budget_extension, external_auth, and reconciliation requests carry an exact tenant/session/turn/sequence binding, response JSON Schema hash, actor/role policy, expiry, and resume state. A valid response resolves once; its redacted form and hash are auditable across restart.

A parent definition may declare named specialists with exact child definition refs, tool allowlists, per-child budgets, output schemas, and depth/fanout/concurrency ceilings. The runtime exposes a generated delegate_<name> tool; the model supplies only bounded {id,prompt} tasks. Server code derives child identity, intersects tools, inherits deadlines, enforces the parent's remaining aggregate budget, admits children through the same journal/runtime, validates outputs, and joins results in stable task order. Ductor does not expose an arbitrary recursive agent-spawn path.

Structured output and multi-tool turns

A provider iteration may emit content, multiple tool calls, usage, and a terminal reason. Tool-call fragments are assembled before validation. Each complete call is authorized independently and committed in deterministic order. When a definition declares a JSON output schema, final output must validate before completion; malformed output fails the turn instead of being stored as a successful result. The schema is also sent to compatible providers as a strict structured-output request, including OpenAI-compatible endpoints and Gemini, but local validation remains authoritative.

Recovery boundary

The committed sequence is the recovery boundary. A subscriber can always replay from the journal without re-calling a provider. An execution worker must acquire the session lease before advancing state, and duplicate attempts observe the already committed transition.

Each chat replica performs a bounded recovery scan at startup and every five seconds. The scan selects only non-terminal sessions with no live lease; every candidate still has to win the normal atomic claim. Safe persisted states resume. A recovered model_running or tool_running state is an ambiguous external outcome and parks as needs_reconciliation instead of repeating possible work.

Tool authorization is itself a recovery artifact. The live authenticated context is checked before the authorization receipt is committed. The receipt binds the tenant, session, invocation, manifest, schema, argument hash, effective arguments, execution mode, limits, and redaction policy. A recovered worker can therefore execute that same authority without relying on the original request context or accepting a new caller identity.

Subscriptions also preflight cursor continuity. A future cursor is a conflict. If an operator's external archive policy has removed a requested prefix, chat returns HTTP 410 resync_required with the requested, earliest, and latest sequences. The default schema does not prune agent events.

Production limits

Set the narrowest useful model, tools, request/tool bounds, and tenant budget. Treat an exceeded bound as a typed operational result, not permission to retry the whole turn. Keep approval required for mutations and destructive actions. Keep read-only diagnostics in the default safe policy, with output redaction and byte limits.

Apply the database migrations included with your Ductor release before enabling the runtime.

Tracing, feedback, and evaluation

Content capture is off by default. The trace hierarchy restores a persisted trace id across worker restart and emits only durable identifiers, definition/manifest/policy ids, provider/model/tool names, approval class, usage, outcome, and stable error type. Authenticated feedback binds to tenant, turn, step, and trace; only the note hash is retained.

The evaluation plane stores versioned datasets, trajectory assertions, scores, and experiments. Promotion policies can require schema, security, recovery, tool-trajectory, and isolation checks. Model-judge scores can supplement those checks but do not override them.

Related