Kwant Atlas

Explorer

2.9.6 Advanced Extensions

Basic Prerequisites: 05 · Failure Modes and 02 · The Latency Hierarchy.

1. Intuition & Practical Objective

Once the software path is clean - no allocation, lock-free channels, sequential memory - the remaining latency lives outside the program: in the operating system, in the memory topology, and in the fact that a general-purpose CPU must interpret an instruction stream at all. This page is the launchpad to the three extensions that remove those costs:

  1. NUMA & memory placement - stop paying the cross-socket tax and get the hot path’s data next to the core that uses it.
  2. Kernel bypass & OS isolation - take the operating system out of the receive path entirely (DPDK, Solarflare OpenOnload/EF_VI, isolcpus, nohz_full, huge pages, busy-polling).
  3. The silicon tier - move the feed handler into an FPGA so the parse itself costs tens of nanoseconds (see Hardware Acceleration & FPGA).

The through-line is the same “do less work” law from 01: each extension removes a layer (a memory hop, a kernel hop, an instruction-fetch hop) rather than making the current layer faster. The economics - is it worth it? - are the equilibrium-fast-trading question (Biais–Foucault–Moinas), and the physical floor (colocation, microwave) is the ceiling on how much any of it can buy.

The one-sentence essence. “The last microseconds are removed by reconfiguring the machine around the hot path - pin the core, place the memory, bypass the kernel, pre-fault the pages - and the final nanoseconds by moving the logic into silicon; but never forget that speed bought below the physical floor is money burned for nothing.”

2. Mathematical Ground Truth & Derivations

2.1 NUMA: the hidden per-access tax

A two-socket box has two memory controllers. A core on socket 0 accessing memory attached to socket 1 crosses the interconnect (QPI/UPI), costing roughly – ns per access in addition to the ~60–100 ns DRAM cost. If a fraction of a thread’s accesses are remote, the effective memory latency is

At the tax is ~80 % on every load - as large as moving a struct from L3 to DRAM. The fix is threefold: pin the thread (sched_setaffinity/numactl) to a core on the NIC’s socket, allocate the hot-path memory on that node (numactl --membind/mbind), and make sure the two agree. Verify with /sys/devices/system/node/node*/numastat and per-node counters.

2.2 Kernel bypass: removing a fat-tailed hop

The standard receive path is: NIC raises an interrupt → softirq/NAPI → sk_buff allocation → protocol processing → wake/schedule the socket-owning thread → copy into user space. Each step can wait, so the path is both slow and high-variance. Kernel bypass (DPDK poll-mode drivers, Solarflare OpenOnload/EF_VI) maps the NIC’s DMA descriptor rings directly into user space and replaces the whole chain with a user-space poll loop that reads completed descriptors. The result is not just a lower mean but a thin distribution - fewer stages means fewer places for variance to enter (and fewer blockers).

2.3 OS isolation: making the hot thread unhurried

Even with bypass, the scheduler can preempt the hot thread. The standard configuration (root-level; see Concurrency & Lockless Programming for the safe application of privilege):

  • isolcpus=2-7 nohz_full=2-7 rcu_nocbs=2-7 - remove cores from the general scheduler, stop the periodic tick on them, move RCU callbacks off them.
  • IRQ affinity - steer NIC IRQs and other interrupts away from the hot cores (/proc/irq/*/smp_affinity).
  • mlockall(MCL_CURRENT|MCL_FUTURE) - prevent page faults from swapping the hot path’s memory out.
  • Huge pages (2 MB / 1 GB) - shrink the TLB-miss rate dramatically (one 1 GB page covers what 512× 2 MB, or 262 144× 4 KB, pages would).
  • Pin the busy-poll loop to an isolated core and never yield it (the wake-up tax measured in 04).

2.4 The physical floor (and why the silicon tier ends the ladder)

Round-trip latency has an irreducible geographic term: , e.g. an NJ–Chicago corridor (≈1 200 km) is ≈12 ms of fibre RTT (≈6 ms one-way), cut by microwave to ≈8 ms RTT (straighter path, ~4 ms one-way). Colocation removes the “last mile”. No software optimization competes with walking the box closer; the FPGA tier (below 200 ns wire-to-wire) is the end of the CPU ladder and the doorway to the hardware topic-folder.

3. Computational Implementation - the tail fix, quantified

Standard library only. We hold the downstream work fixed (parse, strategy, serialize, transmit) and vary only the receive path: an epoll/syscall stack, a busy-poll socket (SO_BUSY_POLL), and a kernel-bypass (EF_VI-style DMA ring). The point is that the receive path’s variance - not its mean - is what kernel bypass removes.

Read the result. The median moves 1 947 → 860 ns (2.3x), but the p99 moves 5 569 → 1 087 ns (5.1x) and the tail ratio collapses from 2.86x to 1.26x. Kernel bypass is a distribution fix: it removes the fat-tailed kernel hop, so the residual path is nearly as fast at its worst case as at its best. That collapse in the ratio - not the mean - is what wins races, and it is the quantitative statement of the failure-mode analysis in 05.

4. Failure Modes & First-Principles Breakdowns

  1. Kernel bypass is an operational commitment, not a flag. DPDK/EF_VI poll-mode drivers bypass the kernel’s safety net: you lose the kernel’s packet filtering, you must manage the rings yourself, and a bug can wedge the NIC. Budget engineering time, not just configuration.
  2. Isolation can starve the system. Over-isolating cores (isolcpus everything, nohz_full everything) can leave the machine unable to run housekeeping tasks, causing worse stalls elsewhere. Isolate the exact cores the hot path needs.
  3. Huge pages without pre-faulting. Huge pages only help if the memory is actually backed and resident; a lazy 1 GB mapping faults on first touch and stalls the loop. Touch/pre-fault at startup.
  4. NUMA misplacement silently cancels the gain. A bypass NIC on socket 0 with strategy memory on socket 1 pays the cross-socket tax on every access, erasing much of the bypass benefit. Pin the thread, bind the memory, verify with numastat.
  5. Optimizing below the floor. If the competitive gap is geographic (milliseconds), sub-microsecond tuning is noise. Always compare any latency investment against the physical floor and the competition’s position.
  6. The silicon tier has its own ceiling. FPGA wire-to-wire is ~30–150 ns, but designing, verifying, and shipping logic in hardware is orders of magnitude more expensive than software; it is the right move only when the CPU ladder is exhausted.

5. References

  • DPDK documentation (dpdk.org). Poll-mode drivers, the user-space receive path, and the getting-started/kernel-bypass guide. (Primary documentation.)
  • Solarflare/Onload & EF_VI documentation. The canonical production kernel-bypass API for trading NICs; zero-copy receive and busy-polling.
  • Benvenuti, Christian - Understanding Linux Network Internals (2005) and Rosen, Rami - Linux Kernel Networking (2013). Where the microseconds you are bypassing actually come from.
  • Kerrisk, Michael - The Linux Programming Interface (2010). mlockall, CPU affinity, huge pages, real-time scheduling
  • Databento - Low-Latency Tuning Guide for Linux and Trading Systems. Practitioner-grade tuning checklist (NIC multi-queue/RSS/RFS, IRQ affinity, busy-polling, NUMA-local everything) specific to trading hosts.
  • De Schryver, Christian (ed.) - FPGA Based Accelerators for Financial Applications (Springer, 2015) and Leber, Geib & Litz - “High Frequency Trading Acceleration Using FPGAs” (FPL 2011). The silicon tier, cross-referenced from Hardware Acceleration & FPGA.
  • Biais, Foucault & Moinas - “Equilibrium fast trading,” JFE 116(2) (2015) and Budish, Cramton & Shim - “The High-Frequency Trading Arms Race,” QJE (2015). Whether the arms race is worth running.
  • MacKenzie, Donald - Trading at the Speed of Light (2021). Colocation, microwave, and the geography of latency.

6. Connected Graph Bridges

Graph View

Backlinks