# Certification & Testing (/docs/connectors/certification-and-testing)



An action in the inventory is only as trustworthy as the evidence that it
actually works — and trust is what lets a <Term name="Route" /> or an agent
dispatch to it unattended. Ductor has a layered set of quality surfaces: a
workbench that turns tests into promotable scorecards, canaries that keep probing
production paths, drift intelligence that watches for provider-side API changes,
and credential-free dev tooling to exercise connectors without a live upstream.

<Callout type="info" title="Evidence here, the declaration next door">
  Everything on this page — the workbench, scorecards, canaries, and drift
  intelligence — ships today and produces real evidence of quality. That evidence
  is now recorded as a declaration on the
  [assurance axis](/docs/connectors/assurance): a certified claim carries its
  contract-test result, conformance run, canary verdict, author, and an expiry,
  and both expiry and revocation downgrade an action to community at read time.
  Assurance ships opt-in via `connector_assurance.enabled`.
</Callout>

## The certification workbench [#the-certification-workbench]

The **certification workbench** (`docs/connectors/certification-workbench.md`)
turns coverage and conformance tests into durable, **redacted** scorecards that
routing, marketplace installs, promotion, and headless tools evaluate *before*
production traffic reaches a capability. Status is ordered from no evidence to
strongest provider-backed evidence:

```mermaid
flowchart LR
    A[uncertified] --> B[metadata_only]
    B --> C[mocked_pass]
    C --> D[recorded_replay_pass]
    D --> E[sandbox_pass]
    E --> F[live_pass]
```

(plus the out-of-band states `blocked`, `stale`, and `waived`). Production
deployment activation in the enterprise profile requires `recorded_replay_pass` by
default for connector actions unless a more specific requirement is supplied.
Normal CI only runs mocked and recorded-replay suites; sandbox and live suites
require explicit operator opt-in and provider allowlists.

<Callout type="warn" title="Evidence is redacted or it is rejected">
  Scorecard evidence stores labels, bounded metadata, hashes, and sizes — never
  raw provider payloads. Fixtures are scanned for forbidden content
  (`Authorization`, `Bearer `, `xoxb-`, `ya29.`, `refresh_token`, `client_secret`,
  `access_token`); PII must be minimized before a payload becomes a fixture. Prefer
  synthetic values, deterministic ids, and schema-shaped records over copied
  customer data.
</Callout>

Marketplace manifests can declare `runtime.certification_requirements`, and
enterprise installs plus deployment promotion evaluate those against tenant-scoped
scorecards and **fail closed** when a requirement is missing, stale, blocked, or
below threshold. Waivers exist for bounded operator exceptions — they require an
actor, reason, and expiry, and are meant to be replaced with fresh evidence, not
extended.

## Synthetic canaries [#synthetic-canaries]

Certification proves a capability *at promotion*; a **synthetic canary** (Plan 105,
`docs/connector-system/synthetic-canaries.md`) keeps answering, on a schedule, &#x2A;is
this deployed tenant/env/provider/action path healthy right now?* It reuses the
canonical `ExecuteActionWithRequest` boundary on a distinct `synthetic_canary`
execution surface, so it inherits **every** production gate — installation,
registry lock, connector policy, setup readiness, execution admission/fairness,
budget, circuit breaker, and the execution ledger. Canaries never bypass a gate.

Eligibility is derived from canonical action semantics and is fail-closed:

* **`read_only`** — eligible by default.
* **`idempotent_write`** — eligible only with an explicit governed opt-in
  (`safety_policy.allow_idempotent_write = true`) **and** a bound sandbox object.
* **`destructive_forbidden`** — any destructive, irreversible, or unknown-effect
  action is **never** eligible; no policy can escalate it. An ineligible target is
  surfaced as `misconfigured` and never reaches the provider.

The scheduler is disabled by default (`connector.synthetic_canaries.enabled`) and
guarded by a durable schedule lease, so many workers can run it without
double-firing. Evidence defaults to `redacted_summary` (secret-redacted,
size-bounded body); modes `none` and `hash_only` store even less. Canary rollups
feed availability readiness, and can be adapted into rollout evidence packs,
provider-drift findings, and certification-staleness signals — but the canary
runtime never pushes into those systems directly.

## Provider API drift intelligence [#provider-api-drift-intelligence]

**Provider API drift intelligence** (Plan 090,
`docs/connectors/provider-api-drift-intelligence.md`) compares Ductor connector
evidence against pinned or governed provider metadata. It is an **evidence and
governance plane only**: drift runs produce snapshots, findings, advisories,
impact previews, and promotion-block decisions, but they **never edit** live
connector definitions, workflows, routing bundles, mappings, permissions, or
credentials.

Deterministic local sources (the Ductor action corpus, bootstrap manifests,
operator-pinned integration-template metadata, local OpenAPI files) are
safe for normal CI. Live docs/changelog/OpenAPI probes are disabled by default and,
when enabled, require tenant/environment scope, an egress policy ref, rate limits,
timeouts, and **credential-free** requests. Findings classify endpoint, method,
path-template, schema, scope, auth-mode, API-version, rate-limit, deprecation, and
sunset changes; promotion preflight blocks unacknowledged breaking or sunset
findings.

## Dev and test tooling [#dev-and-test-tooling]

Two credential-free harnesses let provider authors exercise connectors without a
live upstream:

* **Connector simulator** (`infrastructure/connector/simulator`) *is* the
  provider — a credential-free, deterministic in-memory `Store` implementing the
  real connector operation contracts, generating sync, event, and failure behavior
  from check-in-able scenarios. Its `FailureMode` vocabulary (`rate_limit`,
  `timeout`, `provider_error`, `validation_error`, `auth_expired`, `stale_cursor`,
  `malformed_payload`) maps onto the real execution-error constructors, so
  simulated failures classify exactly like real ones.
* **Fixture harness** (`infrastructure/connector/fixtureharness`) validates a
  provider's HTTP *behavior* against a declarative mock server. Scenario builders
  cover `Fixed`, `Conditional`, `Switch`, `Sequence`, `Paginated`, `RateLimited`,
  `Malformed`, `Timeout`, and `ProviderError`; comparators assert only the request
  shape, result subset, error envelope, or effect-ledger receipt that matters.

<Callout type="warn" title="The simulator is never routable in production">
  The simulator is recognized by `domain/connector.IsSimulatorProviderKey`, and
  both `ExecuteActionWithRequest` and `ConnectorRouter.Select` reject it with
  `ErrSimulatorProviderNotRoutable` unless the caller explicitly sets
  `AllowSimulator`. It is wired only into tests, certification, conformance, and
  sandbox harnesses — production routing can never select it by accident. A
  simulator pass ranks **15** on the certification ladder (above `metadata_only`,
  below `mocked_pass`), so it can never graduate a connector on its own.
</Callout>

Both harnesses bridge into the certification workbench (a fixture pass lands as
`mocked_pass`), and `connectortest` separately validates provider *registration*
metadata. The **external component corpus** intake is a read-only reference set
— Ductor does not import, execute, alias, or translate third-party source
automatically; those candidates stay `blocked_by_policy`.

## Where to go next [#where-to-go-next]

* [Marketplace & Deployments](/docs/connectors/marketplace-and-deployments) — how certification requirements gate installs and promotion.
* [Building a Provider](/docs/connectors/building-a-provider) — authoring the actions these harnesses exercise.
* [Policies, Quotas & Audit](/docs/connectors/policies-quotas-audit) — the production gates a canary inherits.