# Provider-Aware Prompt Caching (/docs/ai/prompt-caching)



<StoryFrame label="Cache invariant" title="Reuse the prefix. Never loosen the tenant, lineage, or prompt contract." accent="commit">
  <StoryFact label="Identity">
    Tenant + immutable definition + exact projected prefix
  </StoryFact>

  <StoryFact label="Semantics">
    Cache hit and miss produce the same model request
  </StoryFact>

  <StoryFact label="Accounting">
    Read, write, input, output, and fixed costs settle separately
  </StoryFact>
</StoryFrame>

Provider prompt caching can reduce latency and input cost without changing the
model-visible prompt. Ductor enables it only when an immutable definition proves a
stable, tenant-scoped prefix. The feature is off by default.

## Definition policy [#definition-policy]

```json
{
  "ref": { "id": "support-agent", "version": 7 },
  "provider": "openai",
  "model": "gpt-4.1",
  "model_policy": {
    "prompt_cache": {
      "enabled": true,
      "eligible_segments": ["system", "tools"],
      "min_prefix_tokens": 1024,
      "ttl_seconds": 300,
      "fail_open": true
    }
  }
}
```

TTL must be 300, 3600, or 86400 seconds. Ductor currently requires `fail_open:
true`: an adapter without cache support sends the unchanged request without a cache
hint. Invalid policy, mutable definition identity, or missing tenant identity fails
before inference.

The cache identity covers tenant, definition and version, provider/model, exact
projected leading system messages, tool definitions, skill and processor pins,
projection version, and compaction lineage. Any meaningful change produces a new
identity.

## Provider behavior [#provider-behavior]

| Provider  | Ductor behavior                                                                     |
| --------- | ----------------------------------------------------------------------------------- |
| OpenAI    | Sends `prompt_cache_key`; 24-hour policy also sends `prompt_cache_retention: "24h"` |
| Anthropic | Emits request-level ephemeral cache control for 5-minute and 1-hour policies        |
| Gemini    | Uses provider-managed implicit caching and reads cached-content token usage         |
| Bedrock   | Ignores unsupported hints while preserving the original request                     |

Cache hit and miss must produce identical messages, tools, structured output, and
replay behavior.

## Usage, pricing, and budgets [#usage-pricing-and-budgets]

Ductor normalizes total input tokens plus separate cache-read and cache-write tokens.
Model pricing may declare `cache_read_price_per_token` and
`cache_write_price_per_token`. If absent, the normal input rate is charged
conservatively; an explicit zero is honored.

Budget settlement includes uncached input, cache reads, cache creation, output, and
fixed request cost. Durable rollups include `ai.tokens.cache_read`,
`ai.tokens.cache_write`, `ai.cache_read.cost.usd`, and `ai.cache_write.cost.usd`.

<Callout type="warn" title="Caching is a data-governance decision">
  Ductor stores only hashes and accounting metadata, but provider-side caches still process prompt
  content. Confirm retention, region, zero-data-retention, and contractual controls before enabling
  caching for a definition.
</Callout>

## Safe rollout [#safe-rollout]

1. Pin cache read/write prices and approve provider data handling.
2. Enable one immutable definition version with a conservative minimum prefix.
3. Compare normalized usage and cost with provider invoices.
4. Prove hit/miss semantic equivalence and expected invalidation.
5. Promote or disable with a new immutable definition version.

## Related [#related]

<Cards>
  <Card title="Inference Proxy" href="/docs/ai/inference-proxy">
    Provider routing, usage, cost, health, and circuit breaking.
  </Card>

  <Card title="Durable Runtime" href="/docs/ai/durable-agent-runtime">
    Deterministic projection and compaction lineage.
  </Card>
</Cards>