Basic Prerequisites: Computer architecture (CPU caches, memory hierarchy, OS system calls) and Market Microstructure & Order Types. (these are the folder-level prerequisites for pages 02–06; page 01 states its own, smaller, entry requirements)
1. Intuition & Practical Objective
In a latency-sensitive strategy the speed of the software is not an optimization of the strategy - it is part of the strategy. When two quotes for the same instrument exist and one is stale, the first engine to send the cancel or the aggressive order wins the difference and the slower engine wears it as adverse selection. So the objective of this topic-folder is narrow and measurable: build a deterministic path from an inbound market-data packet to an outbound order packet, quantify every nanosecond of it, and kill everything that makes the path nondeterministic.
This folder is the low-latency-systems topic-folder for Pillar 2. It is a hub: it gives you (a) the fast latency lookup below (job #1) and (b) routes you to six sub-pages that go from zero-knowledge intuition, through the nano-to-milli latency hierarchy and the canonical three-stage HFT architecture, to ring buffers and lock-free queues, the failure modes that turn a fast system into an unpredictable one, and the modern extensions (NUMA, kernel bypass, FPGA).
The one-sentence essence. “Latency is a budget of physical and mechanical costs - nanoseconds in L1, tens of nanoseconds to DRAM, hundreds to a system call, microseconds to the kernel network stack - and the whole discipline is doing less work per message, never more: you do not optimize the average, you eliminate the tail.”
Scope note (vs the siblings and Pillar 8). This folder is the systems-architecture view: how the money path is laid out and why. The language-level mechanics of the same patterns (C++ atomics, memory ordering, the Disruptor implementation) live in Concurrency & Lockless Programming and High-Performance C++ for Trading - this folder links there rather than duplicating C++. The silicon endpoint (parsing feeds in hardware) is Hardware Acceleration & FPGA; the economic question of whether the speed is worth building is the equilibrium-fast-trading literature below.
Primary sources: Drepper, What Every Programmer Should Know About Memory; LMAX Disruptor paper and Fowler’s LMAX Architecture; Martin Thompson, Mechanical Sympathy; Hasbrouck & Saar (2013), “Low-latency trading”; Biais, Foucault & Moinas (2015), “Equilibrium fast trading”; DPDK and Solarflare OpenOnload/EF_VI documentation; exchange ITCH/FIX specifications.
2. Mathematical Ground Truth & Latency Lookup
Notation. tick-to-trade latency; the -th pipeline stage; message arrival rate; service time; utilization; sojourn time; work-in-process; coefficient of variation of service time; a per-hop tail probability; the number of hops.
Quick-Reference Lookup (job #1). Illustrative reference figures; treat them as orders of magnitude, not benchmarks of your box.
| Level | Approx. latency | Cycles @ 3.5 GHz | Comment |
|---|---|---|---|
| L1 cache hit | ns | the floor; keep the hot path here | |
| L2 cache hit | ns | ||
| L3 cache hit | ns | shared, contended | |
| DRAM (local NUMA) | ns | budget ~100 ns, not 1 | |
| DRAM (remote NUMA) | ns | - | crossing QPI/UPI to the other socket |
| TLB miss / page walk | ns | - | huge pages remove most of it |
| Function call + branch mispredict | ns | - | the tax of “just one more layer” |
malloc/new in hot path | ns | - | heap search + possible lock |
System call (read/write) | ns | - | plus copies; #1 thing to remove |
| Context switch | s | - | scheduler + cache pollution |
| Linux TCP/IP receive path | s | - | interrupts + copies + scheduling |
| Kernel bypass (EF_VI/DPDK) RX | ns | - | user-space DMA ring |
| FPGA wire-to-wire | ns | - | see Hardware Acceleration & FPGA |
The latency budget (additive). Total tick-to-trade is the sum of the stage costs:
Because it is a sum, the slowest stages dominate the mean and the tails dominate the percentiles (§3).
Physics floor. Light travels cm/ns in vacuum and cm/ns in optical fibre, so ~5 ns per metre one way. A 1 200 km route (e.g. New Jersey to Chicago) is therefore ms of one-way fibre latency - no software makes that faster. This is why microwave towers beat fibre on that corridor (shorter straight-line path) and why colocation, not code, is the first lever.
Queueing - where a “fast” stage still loses. A single-threaded handler is a queue; its waiting time is the Pollaczek–Khinchine formula:
Two facts follow. (i) As , : an engine running at 90 % utilization has nine times the queueing delay of one at 50 %. (ii) Jitter multiplies it: a stage with queues twice as long as a constant-cost stage at the same load. Determinism, not just speed, is what keeps small.
Tail composition. If each of independent hops is in its worst top- fraction with probability , the chance at least one hop is in the tail is
A 1-in-1000 per-hop event becomes ~6-in-1000 across six hops. Rare per-stage faults are routine at the system level - the structural reason low-latency work is about tails.
3. Computational Implementation - the latency-budget engine
Standard library only. Six log-normal hops summed into an end-to-end tick-to-trade distribution, then read off as percentiles (the numbers that decide races).
The mean (2 539 ns) sits above the median (2 311 ns) and the p99 is ~2.6x the median: the budget is skewed by the fat-tailed kernel/stack hop. Median tells you how the engine usually behaves; p99 is what decides whether you usually win. Full decomposition in 02 · The Latency Hierarchy.
4. Failure Modes & First-Principles Breakdowns
Hub signposts - the folder’s failure-mode analysis lives in 05 · Failure Modes & Practice. In one line each:
- You optimize the mean and lose on the tail. The 1-in-1000 event is the one that costs money; the median is a vanity metric.
- Allocation in the hot path. Any
malloc/new/GC pause injects a stop-the-world event of – ns into a path measured in hundreds of ns - the tail explodes while the median is untouched. - You fight the cache. Random access to a large structure misses L1/L2 on every message; mechanical sympathy (contiguous, streaming, predictable) beats clever code.
5. References
- Drepper, Ulrich - What Every Programmer Should Know About Memory (Red Hat, 2007). The authoritative free treatment of the cache/memory hierarchy that every latency number in this folder comes from.
- LMAX - The Disruptor (paper + library) and Fowler, Martin - The LMAX Architecture. The canonical treatment of ring-buffer batching and mechanical sympathy.
- Thompson, Martin - Mechanical Sympathy (blog/talks). Cache lines, false sharing, memory-mapped I/O, low-latency design.
- Hasbrouck, Joel & Saar, Gideon - “Low-latency trading,” Journal of Financial Markets 16(4), 646–679 (2
- Biais, Bruno; Foucault, Thierry; Moinas, Sophie - “Equilibrium fast trading,” J. Financial Economics 116(2), 292–313 (2015). When speed is privately profitable but socially wasteful
- MacKenzie, Donald - Trading at the Speed of Light (Princeton, 2021). The best single read on the end-to-end low-latency stack and its economics.
- Hasbrouck, Joel - Empirical Market Microstructure
- DPDK documentation and Solarflare/OpenOnload & EF_VI docs. Primary references for kernel-bypass receive paths.
6. Connected Graph Bridges
- Foundational: Numerical Methods · Probability & Measure Theory (queueing/percentiles)
- Sibling topics: Market Microstructure & Order Types · Queue Position & Fill Probability (why cancel speed is P&L)
- Cross-pillar (do not duplicate): Concurrency & Lockless Programming · High-Performance C++ for Trading · FIX Protocol & Exchange Connectivity
- Sub-pages (in-folder): 01 From Zero · 02 The Latency Hierarchy · 03 System Architecture · 04 Lock-Free & Ring Buffers · 05 Failure Modes · 06 Advanced Extensions