8.3.6 Advanced Extensions

Basic Prerequisites: 03 · Lock-Free Structures and 05 · Failure Modes.

---

1. Intuition & Practical Objective

Once the SPSC ring is correct and fast, the next question is throughput. Two production ideas dominate:

1. Batching (the Disruptor’s core insight). The per-message cost has a fixed component - the publish/handshake/store-release - that is the same whether you move one message or one hundred. If the producer accumulates a batch of $B$ messages and publishes them with one handshake, the per-message overhead drops by a factor of $B$. This is why LMAX’s Disruptor beat bounded queues: not by making a single handshake faster, but by paying it B times less often.
2. Single-writer design. Keep every counter owned by one thread so no CAS is ever needed. The Disruptor is explicitly single-writer; consumers coordinate through a read-mostly cursor. This is a design decision that eliminates the ABA problem and CAS contention before they exist.
3. Wait-free progress. For hard real-time paths - a kill switch that must fire within a deadline - lock-free is not enough, because an individual thread could theoretically starve. Wait-free bounds every thread’s steps. This matters for Production Risk Guards & Kill Switches.

The one-sentence essence. “After correctness, throughput is bought by amortising the fixed publish cost across many messages (batching) and by designing the conflict out with single-writer ownership - the Disruptor, not a faster lock, is the endpoint of the lock-free path.”

---

2. Mathematical Ground Truth & Derivations

Batching amortises the handshake. Let each message cost base ns of work and each publish (regardless of how many messages it carries) cost handshake ns. Pushing $N$ messages in batches of $B$ gives $N/B$ publishes, so total time and throughput:

$$T(B)=N\cdot \text{base}+\frac{N}{B}\cdot \text{handshake},\lambda (B)=\frac{N}{T(B)}=\frac{1}{\text{base}+\frac{\text{handshake}}{B}}.$$

The per-message overhead $\text{handshake}/B$ shrinks linearly in $B$ until it is negligible against base. For base=20 ns and handshake=100 ns: at $B=1$ the handshake is 5x the work; at $B=64$ it is down to 1.6 ns - a ~5.6x throughput gain from scheduling alone.

Latency vs throughput in batching. Batching adds latency: a message waits up to a batch’s worth before being published. The tension is the latency–throughput trade captured by Little’s law ($L=\lambda W$): batch large for throughput, batch small for latency. A market-data fan-out that can tolerate a few hundred ns of batching gets the throughput; an order path cannot, and stays at $B=1$. Choose the batch size per channel’s latency budget.

Progress hierarchy. Blocking (a thread can wait forever) ⊂ lock-free (system-wide progress) ⊂ wait-free (per-thread bounded steps). Disruptor is lock-free; a single fetch_add counter or a bounded stack with helping is wait-free. The difference is whether your worst-case latency is provably bounded - the property a kill switch needs.

---

3. Computational Implementation - batching in numbers

Standard library only. 1 000 000 messages, 20 ns of work each, 100 ns per publish handshake - swept over batch size $B$, showing the per-message overhead collapsing and throughput rising toward its ceiling.

Read the table. Moving from $B=1$ to $B=64$ is a 5.6x throughput gain with no algorithmic change - just amortising the fixed publish cost. Beyond $B=256$ the handshake is negligible (0.4 ns/message) and throughput approaches its work-bound ceiling of $1/20\text{ns}=50$M msg/s. The Disruptor’s reported ~6M msg/s at sub-µs latency is exactly this batching economics applied to a real engine - the number the corpus cites. And the progress table is the map for when lock-free is enough vs when you need wait-free (a kill switch).

---

4. Failure Modes & First-Principles Breakdowns

1. Batching on a latency-critical path. Batching adds up to a batch of latency; applying it to an order path whose budget is microseconds adds unearned decay (see Queue Position & Fill Probability). Batch the market-data fan-out, keep the order path at $B=1$.
2. “Single-writer” broken by growth. Adding a second producer to an SPSC ring turns it into an MPMC that needs CAS + ABA handling - silently losing the property that made it fast. Redesign, don’t extend.
3. Lock-free mistaken for wait-free. A kill switch or risk guard that must fire within a deadline cannot rely on “some thread makes progress” - a specific thread might starve. It needs wait-free or a bounded helping scheme (see Production Risk Guards & Kill Switches).
4. Unbounded ring growth. Converting an overloaded bounded ring into an unbounded one trades latency for memory and hides the backlog - the throughput failure of Pillar 2 · 04 Lock-Free & Ring Buffers in a new guise.

---

5. References

• LMAX, The Disruptor (paper) and Fowler, Martin, The LMAX Architecture
• Thompson, Martin - Mechanical Sympathy
• Herlihy & Shavit, The Art of Multiprocessor Programming
• Preshing, Jeff - An Introduction to Lock-Free Programming
• Williams, C++ Concurrency in Action

---

6. Connected Graph Bridges

• Back: 03 · Lock-Free Structures · 05 · Failure Modes · Index Hub
• Cross-pillar (architecture): 04 · Lock-Free & Ring Buffers · Low-Latency Systems Architecture
• Application: Event-Driven Backtesting Engines (the event loop as a single-writer queue) · Tick Databases & Time-Series (ingestion fan-out)
• Hard-real-time: Production Risk Guards & Kill Switches (why wait-free beats lock-free)
• Hardware: Hardware Acceleration & FPGA (where batching moves off-CPU entirely)