# Simulate a lead marketplace (/docs/guides/lead-marketplace)



A lead marketplace connects publishers to buyers with independent filters,
budgets, and delivery requirements. Ductor's optional ping/post service can
solicit bids using partial lead data, select an exclusive buyer or several
shared buyers, and deliver the full lead to winners.

This guide describes a tested marketplace slice. Boberdoo's
[buyer matching](https://www.boberdoo.com/how-leads-match-to-buyers) and
[distribution logic](https://www.boberdoo.com/lead-distribution-logic) provide
the comparison use cases. A full clone also needs publisher and buyer portals,
pricing and billing operations, reporting, consent handling, and delivery
reconciliation.

## Follow one lead through the marketplace [#follow-one-lead-through-the-marketplace]

1. Receive a publisher's inquiry with stable source and lead references.
2. Match geography and product to a bounded buyer candidate set.
3. Exclude inactive buyers and buyers without enough remaining budget.
4. Ping candidates with partial data that excludes configured PII fields.
5. Collect qualifying bids and explicit declines before closing the auction.
6. Select one buyer for exclusive delivery or top-N buyers for shared delivery.
7. Post the full lead to winners and record each buyer's delivery result.
8. Reconcile rejection, asynchronous acceptance, and uncertain transport results.
9. Correlate later business outcomes to the original lead and buyer decision.

Keep matching, auction selection, delivery, and settlement as observable stages.
An auction winner does not prove delivery, and delivery does not prove a closing.
The [routing-market model](/docs/strategies/markets) and its settlement surfaces
are separate contracts; closing a ping/post auction does not by itself exercise
those settlement guarantees.

## Choose the distribution policy deliberately [#choose-the-distribution-policy-deliberately]

| Marketplace behavior            | Ductor use-case boundary                                                                              |
| ------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Geographic and product matching | Build an eligible candidate set before starting the auction.                                          |
| Exclusive sale                  | Select one winner and post full data only to that winner.                                             |
| Shared sale                     | Bound the number of winners and retain a result per buyer.                                            |
| Highest bid                     | Auction policy ranks valid bids; tests should use distinct prices before testing ties.                |
| Priority or round robin         | Use the corresponding routing strategy; do not infer it from auction behavior.                        |
| Weighted buyer targets          | Specify target accounting and state explicitly; static weights alone do not prove target fulfillment. |
| Waterfall after buyer rejection | Requires an explicit recovery policy with delivery and charge reconciliation.                         |
| Earnings-per-lead optimization  | Requires observed net earnings, refunds, and attribution; nominal bid price is insufficient.          |

Before selecting a policy, read [eligibility](/docs/strategies/eligibility),
[pool management](/docs/management/pools-and-recipients), and
[outcome feedback](/docs/concepts/deduplication-and-outcomes). A buyer's
qualification, budget, bid, and historical conversion performance describe
different constraints.

## Preserve uncertain delivery [#preserve-uncertain-delivery]

Per-winner post results expose four states:

| State      | Meaning                                                        |
| ---------- | -------------------------------------------------------------- |
| `accepted` | Buyer acceptance is recorded.                                  |
| `rejected` | Buyer rejection is recorded.                                   |
| `pending`  | Awaiting the ordinary asynchronous buyer callback.             |
| `unknown`  | A transport failure did not establish acceptance or rejection. |

Authenticated buyer callbacks can resolve `pending` or `unknown` to a terminal
result. The ordinary pending callback timeout does not automatically accept
`unknown`. Existing charges remain until reconciliation according to the owning
pricing flow; transport uncertainty does not justify either blind redelivery
or an automatic refund.

Expose unknown results to operations with the lead, session, buyer, and delivery
attempt references. A dedicated business reconciliation workflow still needs
to establish what the buyer received. Do not label an unknown result as a
successful delivery in a publisher portal.

## Reproduce the simulation [#reproduce-the-simulation]

From the Ductor backend repository:

```bash
./scripts/dev-cache.sh run -- go test -race ./modules/services/pingpost/... -count=1
./scripts/dev-cache.sh run -- go test ./modules/services/pingpost -run TestBoberdoo -bench BenchmarkBoberdoo -benchtime=10x -count=1 -v -benchmem
```

The fixture calls the production ping/post service with synthetic buyer adapters
and in-memory repositories. It verifies active and funded buyer filtering,
declines, exclusive and shared winner counts, ping PII suppression, buyer
rejection, solicitation failure, and uncertain-delivery callbacks.

## Interpret the scale result [#interpret-the-scale-result]

The requested target was a 10,000-buyer catalog and 1,000 leads per second.
The latest local measurements on an Apple M1 Ultra include durable attempt
bookkeeping and canonical pricing lookups in the in-memory fixture:

| Scenario                                            | Measured result                                                              | What it establishes                                                                               |
| --------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| All 10,000 buyers solicited per lead                | 63.245 ms mean lifecycle; 15.81 serial leads/s; 32.085 MB allocated per lead | Full fan-out misses the target even with zero network or database I/O.                            |
| Catalog manually partitioned into 100-buyer cohorts | 60,000 offered, started and completed leads at 1,000.0 leads/s; zero errors  | A one-minute synthetic run sustains the offered target rate during concurrent development builds. |
| Same bounded run, scheduled arrival to completion   | p99 15.165 ms; maximum sampled queue depth 35                                | Includes producer scheduling and queue residence in this fixture.                                 |

The bounded run excludes production candidate matching, HTTP, PostgreSQL,
Redis and ledger mutations. It uses in-memory durable-receipt semantics and
synthetic buyer senders. It is not a production throughput certification.
Cumulative allocation was 19.779 GB over the minute; the ending Go heap was
17.421 MB. Neither figure measures peak process memory or infrastructure cost.

Candidate IDs have a persisted sorted, distinct representation. Binary search
replaces a linear scan for each submitted bid. The earlier membership-only
optimization improved the 10,000-candidate lifecycle from 140.9 ms to 25.3 ms;
the latest 63.245 ms includes delivery bookkeeping, scoped lead snapshots and
canonical pricing lookups. Production currently reads pricing per candidate;
these in-memory lookups do not establish database matching capacity. Nonfinite prices
are rejected and duplicate bid conflicts remain observable.

## Preserve attempts across recovery [#preserve-attempts-across-recovery]

Each recipient has a persisted attempt identity for ping and post. A worker
records an uncertain attempt before external I/O. Another worker skips both
completed and uncertain attempts, so a restart cannot authorize blind resend.
Independent buyer evidence is needed to resolve uncertainty; these receipts do
not establish exactly-once effects at an external buyer.

Real PostgreSQL regression tests separately exercise concurrent dispatch
claimants, duplicate bids, budget races, stale attempt acknowledgments,
immutable original charges, and rejection refunds. Result, refund and receipt
updates commit together. Those correctness tests are separate from the scale
measurement above.

Production capacity still requires real candidate selection, HTTP buyer
latency, PostgreSQL/Redis load, and sustained saturation testing. Track offered,
started, completed, failed and queued work independently so delayed admission
cannot make latency appear artificially low.

## Configure and verify real buyer delivery [#configure-and-verify-real-buyer-delivery]

Author buyer URLs through authenticated admin `PUT` and `GET`
`/api/pingpost/recipients/{recipient_id}/delivery-config`. These settings are
persisted independently of pricing. Paid auction admission reads saved buyer
pricing, excludes disabled or underfunded buyers, and fails closed when that
configuration cannot be read. Enable the production sender with its
callback origin, private signing secret, timeout, and governed egress policy.
Buyer requests carry the immutable attempt UUID as `Idempotency-Key`; they do
not receive the service credential.

Operators can inspect `GET /api/pingpost/sessions/{session_id}/deliveries`
and resolve uncertain delivery through
`POST /api/pingpost/sessions/{session_id}/deliveries/{recipient_id}/reconcile`.
Reconciliation requires the original attempt identity and independent buyer
evidence. Public receipts omit callback tokens; rejection, refund, and the
terminal receipt commit together.

The self-hosted laboratory passed seven functional scenarios using actual HTTP
buyers and PostgreSQL: synchronous acceptance, asynchronous callback acceptance,
synchronous rejection, and lost acknowledgments reconciled as accepted or
rejected, timeout-only auction closure, and funded/disabled/underfunded buyer
admission. Each auction created one charge; rejected outcomes created one refund.
Excluded buyers received no full lead and incurred no charge. Duplicate terminal
operations conflicted. All 14 buyer endpoint configurations survived a controlled
service restart. This functional evidence is separate
from the synthetic throughput result above.

Run `python3 scripts/validate-pingpost-lab.py` in the backend repository with an
owned laboratory and a private, short-lived admin credential. Its report keeps
fixture identities and assertion results without credentials or callback tokens.
The backend's `docs/developers/pingpost-live-validation.md` describes setup and
restart verification.