Balance

Balance strategies answer the everyday routing question: spread this work across recipients without overloading anyone. They range from stateless and near-free (random_weighted) to precise and shared-state (fair_catchup), and they are the strategies most pools start with.

All seven are built in and selectable in a pool.

Smooth weighted round-robin

Modes: weighted · Aliases: round_robin, weighted_round_robin

The default distribution strategy. An Nginx-inspired smooth weighted round-robin (SWRR) that hands out assignments proportionally to recipient weight while avoiding the bursty clumping of naive weighted rotation — a recipient with weight 5 gets five-in-eight of the traffic, but spread evenly, not five in a row.

Why use it. It is the safe default: fair, low-variance, and cheap. Choose it when you simply want traffic split by weight and you have no stronger signal (load, SLA, geography) to route on. It is also the default tie-breaker and fallback for most scoring strategies.

How to configure it.

With atomic: false, each pod keeps a process-local cursor — fine for a single pod, but distribution only converges globally with the shared cursor. With atomic: true, the cursor lives in Redis so every pod advances the same rotation.

{ "strategy": "smooth_weighted_round_robin", "options": { "atomic": true } }

Where it fits. The Select stage. It is stateful (the cursor) and weighted (honors Candidate.Weight). Because it is deterministic and well-behaved, it is the standard fallback_strategy / base_strategy value referenced by scoring strategies.

Random weighted

Modes: weighted

Selects a recipient at random, biased by weight. It builds a cumulative-weight array and does a single binary search per call — the cheapest stateless way to spread load when exact fairness is not required.

Why use it. When you want weight-proportional distribution with zero shared state and don't care that any individual short window may look lumpy. Over many decisions it converges to the weight distribution; over a handful it may not. Ideal for high-throughput, low-coordination routing.

How to configure it. No params — weight comes from each Candidate.Weight (a non-positive weight is treated as 1.0).

Where it fits. The Select stage. It is weighted but not stateful, so it needs no cursor and no Redis, making it trivially correct across any number of pods.

Power of two choices

Modes: single-winner · Aliases: p2c

Envoy's default load balancer. Samples two candidates at random and routes to the one with lower current load. This "power of two choices" trick gets near-optimal balancing without the herding problem of global least-loaded — at constant O(1) cost, regardless of pool size.

Why use it. When you have a live load signal and a large pool. Global least-loaded scans every candidate and tends to herd new work onto whichever recipient briefly looked freest; sampling two avoids that thundering-herd while still strongly favoring the less-loaded side. It is the sweet spot between random and least-loaded.

How to configure it. No params. It reads Candidate.CurrentLoad.

Where it fits. The Select stage. Marked stateful because it reasons over live load, though it keeps no cursor of its own.

Least loaded

Modes: single-winner

Selects the recipient with the lowest current load outright — it packs work onto the freest workers.

Why use it. When your load signals are accurate and low-latency and you genuinely want to fill the freest recipient first (for example, draining a backlog onto whoever has spare capacity). The trade-off is herding: if many decisions happen faster than load updates propagate, they can all pile onto the same "freest" recipient. If that is a risk, prefer power of two choices.

How to configure it. No params. It reads Candidate.CurrentLoad (and respects MaxConcurrent).

Where it fits. The Select stage. stateful in that it depends on live load facts.

Consistent hash

Modes: single-winner

Hashes the routable's key onto a ring so that identical keys always map to the same recipient across runs, with minimal reshuffling when the pool membership changes. This is the basis for affinity and cache locality.

Why use it. When stickiness matters more than balance — session affinity, per-customer routing, cache warmth, or any case where "the same input should keep going to the same place." Adding or removing one recipient only remaps its share of keys, not the whole space, so churn is bounded.

How to configure it. No tunable params in the product surface; it is weighted (weights bias each recipient's arc on the ring) and stateful (the ring). The hash key is derived from the routable.

Where it fits. The Select stage. Pair it with a fallback for keys that have no natural hash input.

Failover

Modes: single-winner · Aliases: active_passive

Ordered primary → secondary fallthrough. Sends everything to the active recipient and promotes the next tier the moment health checks fail — classic active-passive high availability, with intent.

Why use it. When you want redundancy, not distribution: one recipient should take all the traffic while it is healthy, and a standby should take over instantly if it is not. Think of a primary vendor with a backup vendor, or a main queue with an overflow queue.

How to configure it. No product-surface params; ordering comes from candidate tiers, and it uses a sub-strategy (default smooth_weighted_round_robin) to distribute within a tier when a tier has more than one member. It advertises health_check, so it consults recipient health before promoting.

Where it fits. The Select stage. Pair it with reliability-weighted routing when recipient health should influence failover.

Fair catch-up

Modes: single-winner · Aliases: catch_up

Fair distribution with catch-up compensation: recipients that missed earlier assignments (say, during an outage) are boosted until their share evens out across a window. It keeps the long-run distribution fair even after disruptions that a plain round-robin would never repay.

Why use it. When fairness must survive outages. If a recipient was unavailable and dropped out of rotation, SWRR simply moves on and never makes it whole; fair_catchup tracks the deficit and preferentially routes to the under-served recipient once it returns, until the books balance.

How to configure it. No product-surface params. It is weighted and stateful (it tracks per-recipient fairness debt over a window), and its factory resolves a base strategy through the registry.

Where it fits. The Select stage. The durable form of its fairness-debt state is described by the strategy state plane.

Related