# Events & Event Sourcing (/docs/concepts/events)
Ductor uses the word "event" for three genuinely different things. Conflating
them is the fastest way to misunderstand the system, so this page keeps them
separate: aggregate **event sourcing**, the **append-only bus**, and the
workflow **transition journal**.
## 1. Aggregate event sourcing [#1-aggregate-event-sourcing]
Some domain aggregates — pools, rules, and similar configuration objects — are
**event-sourced**: their current state is a fold over an append-only log of
immutable events rather than a mutable row you overwrite.
An event carries an aggregate ID, an aggregate type, a type string (like
`pool.created`), a sequential per-aggregate `version`, a timestamp, an opaque
JSON `data` payload, and correlation/causation IDs. Events are appended, never
edited.
The `EventStore` interface enforces **optimistic concurrency** at append time:
```go
Append(ctx, aggregateID, expectedVersion, events)
```
If `expectedVersion` doesn't match the aggregate's current version, the append
is rejected with a concurrency-conflict error. In Postgres this is enforced by a
`UNIQUE (aggregate_id, version)` constraint on the `domain_events` table — two
writers can't both claim version *N*. Reconstruction is a replay: load events
from a version and `Apply` each in order. **Snapshots** (`aggregate_snapshots`)
let you skip replaying from zero for long-lived aggregates.
This is the audit-trail-and-time-travel layer: you can reconstruct exactly what
a pool or rule looked like at any point, and *why* it changed.
## 2. The append-only bus [#2-the-append-only-bus]
The **append-only bus** is the canonical event *plane* — how a state change
fans out to the rest of the system. Its defining property is in the name:
**it appends durably before it fans out.**
When something publishes an event (for example, a pool config change), the bus:
1. Normalizes and validates the event envelope.
2. **Appends it durably to Postgres** (with a per-event idempotency key).
3. *Then* attempts the optional Redis pub/sub fanout to other pods.
The order matters. Because the durable append happens first, an event is never
lost just because the fanout hop failed. A **fanout policy** decides what a
fanout failure means:
* `best_effort` (default) — the durable append is authoritative; fanout errors
are observed and swallowed.
* `required` — a fanout failure is returned to the caller, so a system that
must fan out can fail closed.
This is the mechanism behind cache invalidation: a config change is persisted,
then a Redis pub/sub message tells every pod's cache invalidator to drop the
stale entry. If the message is dropped, the durable record is still there to
reconcile against.
The bus never fans out an event it hasn't already durably recorded. That single
ordering rule is what lets Ductor treat pub/sub as a best-effort accelerator
rather than a source of truth — a dropped message degrades latency, not
correctness.
## 3. The workflow transition journal [#3-the-workflow-transition-journal]
A workflow run keeps its own history — a durable, append-then-compact
**transition journal** on `eec_workflow_run` (the `eec_` prefix marks Ductor's
durable *Enterprise Eventing Core* tables) and its transition rows. This is
*not* the aggregate event store; it's the coordinator's record of every state
transition a run made, which underpins replay, debugging, and audit.
Because journal payloads can be large or sensitive, they're governed by two
policies:
* **Payload budget** — each journal entry has a payload class
(`inline_redacted` by default, `inline_public`, `external_artifact_ref`,
`summary_only`, `discarded_by_policy`) and an overflow action. Oversized
payloads are externalized to the archival backend rather than bloating the run
row.
* **Compaction** — old transitions can be pruned, but never silently: compaction
writes a durable `CompactionReceipt` recording the sequence range removed, the
policy that removed it, the retained checkpoint, and an evidence artifact
reference and hash. The receipt still anchors replay, so history stays
reconstructable even after it's compacted.
## Which one am I looking at? [#which-one-am-i-looking-at]
| System | Purpose | Backing |
| ------------------------ | ------------------------------------------------ | -------------------------------------- |
| Aggregate event sourcing | Rebuild & audit config aggregates (pools, rules) | `domain_events`, `aggregate_snapshots` |
| Append-only bus | Fan a durable state change out to all pods | Postgres append + Redis pub/sub |
| Transition journal | A run's own replayable, compactable history | `eec_workflow_run` + transition rows |
## The trigger plane [#the-trigger-plane]
None of the three systems above is how an external event *starts* a workflow.
That is the job of the **trigger plane** — the ingress boundary every external
source passes through before it can spawn or wake a run. It is not part of event
sourcing; it is the front door.
```mermaid
flowchart LR
W["Webhook"] --> N["Normalize"]
S["Schedule"] --> N
E["Event"] --> N
G["Signal"] --> N
M["Manual"] --> N
C["Connector"] --> N
N --> EN["Canonical envelope"]
EN --> R["Persist receipt"]
EN --> RN["Spawn / wake run"]
```
Every external ingest surface — webhook, schedule, event, signal, manual (API/
operator), and connector sources — normalizes what it received into one
canonical envelope before anything downstream sees it:
```go title="domain/workflow/event/envelope.go"
type Envelope struct {
ID string // caller-facing event ID
Source Source // webhook | schedule | event | signal | manual | connector
Name string // event name, e.g. "stripe.invoice.paid"
TenantID string
Data json.RawMessage // opaque payload
ReceivedAt time.Time
InternalID string // deterministic internal ID
DedupeKey string // optional, for spawn suppression
ContextRef ContextRef // {type, id} the event concerns
}
```
Two properties matter. First, `Data` is **hard-capped at `MaxDataBytes` (1 MiB)**
at the ingress boundary — an oversized payload is rejected there, not carried
into a run. Second, ingest **persists a receipt** for every accepted envelope, so
external ingest is auditable independently of whatever run it did or didn't
spawn.
### Wildcard event routing [#wildcard-event-routing]
Event-name triggers can be literal or wildcard. When you **publish** a workflow
whose event trigger declares a name pattern, the publish path upserts a row into
an `EventTriggerIndex` keyed `(tenantID, pattern, definitionID)`. Patterns are
either a literal name (`stripe.invoice.paid`) or a suffix wildcard
(`stripe.invoice.*`).
At **ingest** time the dispatcher calls `pkg/eventpattern.Expand` on the concrete
event name — expanding `stripe.invoice.paid` into `["stripe.invoice.paid",
"stripe.invoice.*", "stripe.*", "*"]` — and walks the index to find every
definition that should spawn. Fan-out to N subscribers is a single index lookup,
not a scan.
The feature flag `ductor.events.wildcards.enabled` defaults to **OFF**. The
index is still populated on publish, but the dispatcher **refuses to spawn**
until an operator flips the flag — so a freshly published wildcard trigger can
look wired-up yet never fire. Separately, per-trigger rate limiting
(`ductor.events.triggers.rate_limit.enabled`) is **lossy**: on a budget breach
the fan-out is *dropped* with a structured log and metric, not queued for
later. Do not use it as backpressure for events you cannot afford to lose.
## Which one am I looking at (again)? [#which-one-am-i-looking-at-again]
The trigger plane *feeds* the systems above — a spawned run then writes its own
transition journal, and config changes it makes fan out over the bus. If you
need a run to pause mid-flight and fan in a *set* of correlated events, that is
the [`event_set` step](/docs/concepts/dag-workflow-model#steps), which is part of
the DAG model, not the trigger plane.
## Where to go next [#where-to-go-next]
* [Idempotency & Exactly-Once](/docs/concepts/idempotency) — how durable append plus dedup keys give effectively-once delivery.
* [Data Flow](/docs/architecture/data-flow) — where events sit in the system.
* [The DAG Workflow Model](/docs/concepts/dag-workflow-model#steps) — what a run's transition journal records, and the `event_set` in-run wait.
* [Managing Workflows](/docs/management/workflows) — publishing triggers and enabling wildcard fan-out.