# Connector Architecture (/docs/connectors/architecture)



The connector subsystem is the machinery behind the **action inventory** — the
governed, typed actions the *executed* stage of the clearing lifecycle draws on.
It is built from a small set of registries and one central executor. Everything
else — workflow steps, the management API, routing hooks, and agent tool calls —
is a caller that funnels into that executor. This page traces the machinery from
registration through dispatch to the durable attempt model.

## The two registries [#the-two-registries]

A provider is described by two registries, both singletons wired via `fx` in
`cmd/ductor/fx_connector.go` (`ConnectorRegistryModule`):

* **`ProviderRegistry`** (`application/connector/provider_registry.go`) maps a
  `providerKey` → a `Provider` definition. `Get(key)` returns the provider or
  `ErrProviderNotFound`. Keys must match `^[a-z][a-z0-9_]*$`. The registry keeps
  a `byCategory` index so the catalog can list providers by category.
* **`ActionRegistry`** (`application/connector/action_registry.go`) maps a
  `(providerKey, actionKey)` pair → an `ActionSpec`. `Get(providerKey, actionKey)`
  returns a deep clone of the spec or `ErrActionNotFound`.

There are four more registries alongside these — `TriggerRegistry`,
`ModelRegistry`, `MapperRegistry`, and `ResolverRegistry` — all built on the same
generic `specRegistry[S]`, whose keys allow multi-segment names like
`connector.sync.lookup_record`.

<Callout type="info" title="Registration goes through one interface">
  Providers never touch the registries directly. They call the `Registrar`
  interface (`application/connector/registrar.go`), which has one method per
  registry — `Provider`, `Action`, `Trigger`, `Model`, `Mapper`, `Resolver`,
  plus bulk variants. `ActionRegistry.Register` validates the spec's semantics
  and stamps its derived annotations before storing it, so an invalid action
  fails loudly at startup, not at dispatch.
</Callout>

## The Provider definition [#the-provider-definition]

A `Provider` (`domain/connector/provider.go`) is a compiled, versioned
description. The load-bearing fields:

| Field                                                                     | Purpose                                                                                                                                                |
| ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `Key`, `DisplayName`, `Description`                                       | Identity and catalog display.                                                                                                                          |
| `BaseURL`                                                                 | A **redacted authoring template** — no tokens, headers, or secrets. Used to build the generic API-call action.                                         |
| `Categories []Category`                                                   | Catalog taxonomy (see [The Catalog](/docs/connectors/catalog)).                                                                                        |
| `AuthTypes []AuthType`                                                    | Which auth types this provider supports; the first is the default the builder suggests.                                                                |
| `CredentialSchema`                                                        | The connection form for secret-text / basic / custom auth (nil for OAuth2).                                                                            |
| `ActionKeys`, `TriggerKeys`, `ModelKeys`, `MapperKeys`                    | Fully-qualified capability keys the provider registers.                                                                                                |
| `Hooks ProviderHooks`                                                     | Provider-level Go callbacks: `TestAuth`, `ExtractIdentity`, `RefreshToken`, `InstallAppWebhook`, `UninstallAppWebhook`, `ResolveAppWebhookConnection`. |
| `OAuth2Config`, `TwoStepConfig`, `OAuth1Config`, `JWTConfig`, `MCPConfig` | Per-auth-type config, non-nil only for the auth types the provider uses.                                                                               |
| `ReleaseStage`, `Deprecated`, `Compliance`, `RateLimit`, `Quota`          | Lifecycle and governance metadata.                                                                                                                     |

## The ActionSpec [#the-actionspec]

An `ActionSpec` (`domain/connector/action.go`) is "the complete, immutable
description of one operation... registered once at startup, read-only
thereafter." Key fields:

* `Key` (fully-qualified, e.g. `hubspot.create_contact`), `Provider`, `Auth`.
* `Properties []Property` — the input form (see [Building a Provider](/docs/connectors/building-a-provider)).
* `OutputSchema json.RawMessage` — a JSON Schema for the result.
* `Semantics ActionSemantics` — the canonical effect/idempotency/retry contract.
* `Execute ActionExecuteFunc` — the function that does the work.
* `DefaultTimeout`, `DefaultFailurePolicy`, `Pagination`, `RequiredScopes`, `Quota`.

The execute signature is uniform across every action:

```go
type ActionExecuteFunc func(ctx context.Context, in ActionInput) (ActionOutput, error)
```

`ActionInput` carries the resolved `Auth *AuthContext`, the caller `Args`, and the
full execution context (`TenantID`, `ExecutionID`, `RunID`, `StepRef`, `AttemptNo`,
`ExecutionSurface`, `BudgetDeadline`, …). `ActionOutput` carries a `Result`
(`json.RawMessage`), an optional normalized result, and — critically — an
`EffectPatch`.

<Callout title="Actions return effects, they never mutate shared state">
  An action's output "must contain an effect patch; must not mutate shared state
  directly." An `EffectPatch` is a list of `EffectOp{Op, Path, Value}` where `Op`
  is one of `set`, `merge_object`, `emit_request`, or `search_attributes_merge`.
  The workflow coordinator applies these ops transactionally when it folds the
  attempt result back into run state. This is what keeps dispatch idempotent and
  the coordinator the sole writer of run state.
</Callout>

## Action semantics [#action-semantics]

Every action declares an `ActionSemantics` (`domain/connector/action_semantics.go`)
— the machine-readable contract that governs whether the engine may retry it,
whether it needs approval, and how to make it idempotent. The dimensions:

| Dimension          | Values                                                                                                                                |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| `EffectClass`      | `read`, `create`, `update`, `upsert`, `delete`, `send`, `custom_write`, `multi_step`, `unknown`                                       |
| `MutationScope`    | `none`, `single_record`, `collection`, `provider_account`, `external_delivery`, `custom`, `unknown`                                   |
| `DestructiveClass` | `none`, `soft_delete`, `hard_delete`, `overwrite`, `irreversible_send`, `unknown_destructive`                                         |
| `IdempotencyKind`  | `naturally_idempotent`, `provider_idempotency_key`, `client_supplied_external_key`, `ductor_ledger_only`, `not_idempotent`, `unknown` |
| `RetrySafety`      | `safe_retry`, `retry_before_provider_call`, `retry_with_idempotency_key`, `no_automatic_retry`, `manual_review`                       |

`Validate()` enforces the relationships between them — a `read` cannot be
destructive and must be naturally idempotent and safe to retry; a destructive
action requires approval; a non-idempotent action cannot be marked safe-retry.
The shared gate `ValidateActionExecutionPolicy(sem, surface, attemptNo, key, policy)`
runs before side effects on the `mcp_tool`, `async_action_job`, and
`unified_operation` surfaces, so a retry of a not-idempotent action without a key
is refused rather than silently duplicating a downstream write.

This is what makes the inventory *governed* rather than merely callable: because
every action's mutation class, idempotency, and retry safety are declared, the
runtime — and any worker it dispatches to, human or agent — knows what is safe to
retry and what needs a human, without reading the provider's source.

## Dispatch: the central executor [#dispatch-the-central-executor]

All three call surfaces converge on `ActionExecutorService.ExecuteActionWithRequest`
(`application/connector/action_executor.go`), which takes an
`ActionExecutionRequest` and does, in order:

1. `spec, err := actions.Get(providerKey, actionKey)` — resolve the action, or fail closed.
2. Simulator gate — a simulator-only provider is not routable unless explicitly allowed.
3. Merge required scopes and resolve the execution budget/deadline.
4. `StartExecution` on the execution ledger.
5. The install gate (is the provider config actually set up?).
6. Resolve auth via `ConnectionService.Resolve` (see below).
7. Run the interceptor chain (rate limit, audit, circuit breaker, …).
8. Call `spec.Execute`, then the output mapper projection.
9. `FinishExecution` on the ledger.

The interceptor chain is contributed through the `connector_interceptors` `fx`
option group, which is the same extension pattern routing strategies use.

### Resolving the connection [#resolving-the-connection]

`ConnectionService` (`application/connector/connection_service.go`) has two
methods: `Get` (the raw `Connection`) and `Resolve` (a per-attempt, decrypted
`AuthContext`). The executor calls `Resolve` at dispatch time. That flow —
cache, decrypt, per-auth decoration, OAuth2 refresh — is detailed on the
[Authentication](/docs/connectors/authentication) page.

## The durable attempt model [#the-durable-attempt-model]

The synchronous workflow path (`application/workflow/dag/worker_connector_action.go`,
`StepWorker.dispatchConnectorAction`) calls the same `ExecuteActionWithRequest`
with `ExecutionSurface = workflow_action` and returns a `StepExecutionResult`
tagged with an `AttemptID`. The coordinator applies the returned `EffectPatch`
when it commits the tick. Because dispatch is **at-least-once**, the same action
can execute more than once across a crash; the effect patch and any provider-side
idempotency key are what make that safe. See
[Coordinator & Step Workers](/docs/concepts/coordinator-workers) and
[Idempotency & Exactly-Once](/docs/concepts/idempotency).

## The async action-job model [#the-async-action-job-model]

For work that shouldn't block a synchronous request or a workflow tick, Ductor
has a separate durable job surface (`ExecutionSurface = async_action_job`),
modelled by `ActionJob` (`domain/connector/action_job.go`) and driven by the
`ConnectorActionJobService` (`application/connector/action_job_service.go`).

A job moves through an explicit state machine:

```mermaid
stateDiagram-v2
    [*] --> queued
    queued --> running
    running --> succeeded
    running --> failed
    running --> canceled: cancel_requested
    running --> expired
    running --> dead_lettered: attempts exhausted
    succeeded --> [*]
    failed --> [*]
    canceled --> [*]
    expired --> [*]
    dead_lettered --> [*]
```

The mechanics that make it safe:

* **Idempotent submission.** `SubmitConnectorActionJob` computes an
  `IdempotencyScope` from the tenant, provider, action, connection, and route,
  then looks up any existing job by `(scope, idempotencyKey)`. A resubmission
  with the **same** input digest returns the existing job (`Deduplicated=true`);
  a resubmission with a **different** digest under the same key returns
  `ErrConflict`.
* **Retries require a key.** A job with `MaxAttempts > 1` is rejected at creation
  unless it carries an idempotency key, and `ScheduleRetry` errors without one.
  On a retryable failure the job returns to `queued` with a backoff; once
  `Attempts >= MaxAttempts` it is `dead_lettered`.
* **Lease fencing.** A worker leases a due job (`MarkLeased`), executes it, and
  writes the terminal result through `UpdateConnectorActionJobFenced(job, leaseOwner, running)`.
  If the fence fails — the lease was reclaimed and re-leased by another worker —
  the worker **discards** its result rather than double-finalizing.
* **Budget inheritance.** The persisted `DeadlineAt` is stamped onto the
  execution context so a deferred provider call still respects the originating
  run's deadline.

The worker itself (`cmd/ductor/fx_connector_action_job_worker.go`) is
**default-off**: it runs only when `connector.async_action_job_worker.enabled` is
set with a positive interval, batch limit, lease duration, and a non-empty tenant
list. Each tick reclaims expired leases, expires due jobs, then runs due jobs up
to the batch limit.

<Callout type="info" title="Both surfaces, one executor">
  The synchronous workflow step and the async job are two execution *surfaces*
  over the same `ExecuteActionWithRequest`. The workflow path carries the
  coordinator's `AttemptID`; the async job carries its own `Attempts` count and
  idempotency key. Both emit `EffectPatch` ops and both flow through the same
  execution ledger, so observability is uniform regardless of how the action was
  dispatched.
</Callout>

## Governed proxy requests [#governed-proxy-requests]

Not every provider call is worth modeling as a stable action. For one-off HTTP
calls, `ProxyConnector` (`docs/connectors/proxy-requests.md`) runs the request
through the **full connector boundary** — tenant provider policy, marketplace
install gates, direct/route connection selection, auth injection, egress policy,
circuit breakers, interceptors, metrics, audit, and request/response limits. It is
not raw HTTP: `endpoint` must be a relative path beginning with `/`, and the legacy
`provider.custom_api_call` action is now just an action-shaped wrapper over the
same governed engine.

* **Direct mode** (`connection_mode: direct`) names a `connection_id`.
* **Route mode** (`connection_mode: route`, optional `connection_route_provider`)
  lets the routing interceptor select the concrete connection and stamp
  `resolved_connection_id` in the response.

<Callout type="warn" title="base_url_override is denied by default">
  A proxy request cannot point itself at an arbitrary origin: `base_url_override`
  is denied unless a provider or connection policy explicitly allows it. This is
  what keeps a proxy request from degrading into arbitrary raw HTTP egress. Unlike
  `custom_api_call`, the proxy keeps non-2xx provider responses inside the response
  envelope so callers can inspect provider validation errors.
</Callout>

## The connector execution runtime [#the-connector-execution-runtime]

Four domain models back the observability, control, and safety of every execution,
across all surfaces:

* **Execution ledger** (`ExecutionRecord`, `docs/connectors/execution-ledger.md`)
  is the canonical, **redacted** operational history for *all* connector activity —
  direct `ExecuteAction`, workflow actions, sync runs, routing hooks, proxy
  requests, lifecycle/MCP operations, and mapper executions. Each row has a stable
  `execution_id`, and secret-shaped keys (`authorization`, `token`,
  `refresh_token`, `client_secret`, …) are stripped at any nesting depth.

  <Callout type="info" title="The ledger is observational, not the dedupe boundary">
    The execution ledger records *what happened*; it is **not** the idempotency
    boundary. Side-effect deduplication remains the workflow idempotency table
    `eec_connector_action_invocation` (the `eec_` prefix marks Ductor's durable
    *Enterprise Eventing Core* tables). Never use the ledger as a dedupe key —
    `ExecuteActionResponse` and `ProxyConnectorResponse` expose `execution_id` only
    so you can pivot into the execution/log APIs for triage.
  </Callout>

* **Execution lifecycle control** (`ConnectorExecutionLifecycle`,
  `docs/connectors/execution-lifecycle.md`) is a per-attempt record for one
  connector execution unit (async attempt, function invocation, sync, mapper).
  Statuses are explicit: `queued`, `starting`, `running`, `interrupting`,
  `abort_requested`, `aborting`, `cleanup_running`, then a terminal `succeeded`,
  `failed`, `cancelled`, `timed_out`, `killed`, `orphaned`, or `cleanup_failed`.
  **Cancel** stops queued/running work cooperatively; **interrupt** finishes the
  current safe operation then stops; **kill** is reserved for deadline or
  worker-loss cases — none claims to undo provider side effects that already
  happened. Running attempts record **heartbeats**, and orphan sweeps mark an
  attempt `orphaned` when the lease expires without one.

* **Execution locks** (`ConnectorExecutionLock`,
  `docs/connectors/execution-locks.md`) are short-lived, non-blocking
  critical-section leases a connector function acquires with
  `ductor.tryAcquireLock({ key, ttlMs })` (TTL bounded 1s–15min). They protect a
  small provider-scoped mutation, cursor rebuild, or webhook-dedupe window — they
  are **not** routing admission, and connector code never chooses the lock owner
  (Ductor derives it from the trusted execution context). The runner calls
  `releaseAllLocks()` in cleanup after both success and failure.

* **Async action jobs** (`ActionJob`) run long-running actions off the synchronous
  path with the same lifecycle control. The worker
  (`cmd/ductor/fx_connector_action_job_worker.go`) is **default-off**
  (`connector.async_action_job_worker.enabled`); see
  [the async action-job model](#the-async-action-job-model) above.

## Where to go next [#where-to-go-next]

* [Authentication](/docs/connectors/authentication) — the `AuthContext` resolution flow and credential encryption.
* [Connections](/docs/connectors/connections) — direct vs. route mode and the management API.
* [Building a Provider](/docs/connectors/building-a-provider) — author actions and register a provider.