Semantic Routing

semantic_choice is an optional routing strategy backed by the shared typed decision plane. It is useful when rules can determine who is eligible but structured and textual context is needed to distinguish the best fit.

The strategy cannot add a recipient. It sees only candidates already admitted by normal routing, returns only their opaque IDs (or __no_match__), and is revalidated before the decision is persisted.

Governed profiles

A route supplies one profile reference. Models, questions, field paths, thresholds, endpoints, and credentials cannot be authored in route options.

ai:
  decision:
    semantic_routing_profiles:
      buyer-fit@v1:
        id: buyer-fit
        version: v1
        model: typesafe/jev-1.13-20260917
        question_set_version: buyer-fit-questions-v1
        question: Choose the best eligible recipient for this routable.
        routable_fields: [type, attributes.intent, attributes.location]
        candidate_fields: [name, metadata.specialties, metadata.territories]
        data_class: internal
        max_candidates: 16
        max_state_bytes: 16384
        min_confidence: 0.8
        min_winner_margin: 0.1
        shortlist_mode: stable_id
        failure_policy: pipeline_fallback
        evaluate_single_candidate: false

Profile keys are exact id@version identities. Mutable aliases are rejected. The projector reads only declared fields, treats embedded text as untrusted evidence, canonicalizes state, and enforces candidate and byte limits before egress. Its data_class must also appear in the deployment-level ai.decision.allowed_data_classes allowlist or startup fails and the request cannot reach a provider.

Route pipeline

strategy_pipelines:
  semantic_buyer_fit_v1:
    stages:
      - id: semantic_fit
        type: select
        strategy: semantic_choice
        options:
          profile: buyer-fit@v1
        on_error: stage:deterministic_fallback
        on_no_candidates: stage:deterministic_fallback
      - id: deterministic_fallback
        type: fallback
        strategy: smooth_weighted_round_robin

Zero candidates and, by default, one candidate make no provider call. An over-limit set is deterministically shortlisted or follows the profile's failure policy. Exclusive choices are never split into independent batches because probability distributions over different option sets are not comparable.

Durable evidence and replay

Accepted evidence includes the resolved evaluator model, profile and question versions, state/candidate/request/result hashes, selected option, complete probability map, confidence, and bounded outcome/fallback class. Raw state and arbitrary strategy metadata are excluded.

The routing selector verifies that the evidence matches the exact eligible snapshot and ranked slate. The decision store persists it in a dedicated JSONB column.

original_lock replay reads that stored decision and makes zero evaluator calls. Latest, diff, and what-if modes are explicit re-evaluations and report semantic drift rather than rewriting history.

Rollout

Start with a deterministic fallback and shadow evidence. Promote one exact profile/model version only after false-route rate, no-match rate, low-confidence rate, latency, spend, and provider-failure behavior meet the route's release criteria. Rollback is removing the semantic stage or disabling the decision plane; historical evidence remains readable.

Related