8.2.4 Zero-Cost Abstraction

Basic Prerequisites: 03 · Memory & Cache and 02 · Why C++.

---

1. Intuition & Practical Objective

“Zero-cost abstraction” is Stroustrup’s promise: a well-designed C++ abstraction should cost nothing at runtime compared with the hand-written code it replaces. That promise is not automatic - it is a property you engineer, and this page covers the three tools that deliver it: templates (compile-time dispatch, no vtable), move semantics (transfer ownership instead of copying), and pre-allocated storage (no allocator in the hot path).

The central tension: readability wants abstraction (a generic Signal<Fast> type, a PriceSource interface); latency wants none of the runtime machinery abstraction usually drags in (virtual calls, heap allocation, refcount atomics). C++ resolves it by making those mechanics compile-time: the template specialises for the concrete type, the call is inlined, and the binary contains no runtime dispatch at all.

Three “aha”s:

1. Abstraction can be free or expensive - you choose which. template<class T> T square(T x){return x*x;} with T=double compiles to a single multiply. A virtual double square(double) compiles to a vtable load + indirect call that may mispredict. Same source intent, different machine code.
2. Move ≠ copy. Copying $n$ elements is $O(n)$; moving is $O(1)$ (a few pointer writes). Returning a big buffer by value is only cheap because move makes it a pointer handoff.
3. The allocator is the tax collector of “convenient” code. Every implicit new (a std::vector growing, a std::function boxing a lambda, a shared_ptr creating a control block) adds run time the source line conceals. Zero-cost engineering removes the call, not just the frequency.

One-sentence essence. “Write code as if abstraction were free, then verify with the profiler that it actually is - and where it isn’t, move the cost to compile time or out of the hot path.”

---

2. Mathematical Ground Truth & Derivations

2.1 Templates: dispatch cost collapses to zero

A virtual call traverses: load vptr → load function pointer from vtable → indirect call (often mispredicted, ~15–20 cycles flush) → execute. Its cost is

$$t_{\text{virtual}}=t_{\text{vptr load}}+t_{\text{vtable load}}+t_{\text{mispredict}}\approx 20\text{–}40\text{cycles},$$

whereas a template instantiation resolves the concrete function at compile time and inlines it:

$$t_{\text{template}}=t_{\text{body}}(\text{the call disappears}).$$

In a hot loop running $M$ times, the difference is $M\cdot t_{\text{virtual}}$ - pure, avoidable overhead. Templating also unlocks inlining across the call boundary, which is what lets the compiler auto-vectorise (03 · Memory & Cache, §2.3).

2.2 Move semantics: $O(n)\to O(1)$

Copying a container of $n$ elements of size $s$ costs

$$t_{\text{copy}}=n\cdot c_{\text{elem}},c_{\text{elem}}\approx (t_{\text{load}}+t_{\text{store}})\sim \text{a few cycles},$$

so $t_{\text{copy}}=O(n)$. A move constructor transfers the internal pointer, capacity, and size instead:

$$t_{\text{move}}=\underset{\text{a handful of writes}}{\underbrace{c_{\text{ptr}}+c_{\text{size}}}}=O(1),$$

independent of $n$. This is why return big_vector; is cheap (return-value optimisation and/or move) and why std::vector<std::vector<double>> operations are pointer-hops, not data floods. A shared_ptr copy is a special trap: it is not $O(n)$ data, but it is an atomic refcount increment

$$t_{\text{shared\_ptr copy}}=t_{\text{atomic inc}}\sim 10\text{–}20\text{cycles}+\text{lock-prefix contention},$$

i.e., a hidden synchronisation point on the hot path - prefer unique_ptr or, better, value types.

2.3 Allocation: amortised vs bounded (the reserve inequality)

Appending $N$ elements to a doubling vector triggers reallocations when capacity is exhausted. Element copies total

$$C(N)=\sum _{k=0}^{\lfloor {\log }_{2}N\rfloor }{2}^k\approx N,$$

so the amortised per-append cost is $O(1)$ - but the worst single append copies up to $N/2$ elements at an unpredictable moment. Reserve in advance and both terms vanish:

$$C_{\text{reserve}}(N)=0,\text{reallocations}=0.$$

The tail paragraph is the point: amortised $O(1)$ and bounded $O(1)$ are different guarantees, and the hot path needs the second (this is the theme of 05 · Failure Modes).

2.4 The per-tick allocation tax

If a hot path allocates once per message at rate $\lambda$, the expected allocator time per second is $\lambda \cdot \mathbb{E}[t_{\text{alloc}}]$, with $\mathbb{E}[t_{\text{alloc}}]$ as in 02 · Why C++, §2.3. §3 measures the fast-path component of that term in a language-agnostic proxy: the reused buffer is strictly cheaper than fresh allocation by a factor of ~3×.

---

3. Computational Implementation - allocation cost, measured

We cannot call a C++ allocator from these stdlib-only examples, but the mechanism (ask the runtime for a fresh buffer vs write into existing storage) is language-agnostic and reproduces the same order of magnitude. The example times both patterns, warm, over $10^6$ iterations.

Reading the result. Reusing pre-allocated storage is $\approx 3.37\times$ cheaper per operation (26.8 vs 90.4 ns/op) for identical work - the difference is entirely allocator/runtime overhead. The absolute numbers are interpreter-specific, but the ratio and the direction are the C++ rule stated as arithmetic: allocate once before the hot path, then write into the buffer. (In C++ the same experiment with std::vector + reserve vs push_back shows the §2.3 inequality directly - see 05 · Failure Modes for the growth-cost model.)

C++ form (prose). The engine that matches this experiment: a std::vector<Price> book; book.reserve(MAX_LEVELS); sized before the session starts, then book[i] = p; (or emplace_back bounded by the reserve) inside the callback - no new, no push_back past capacity, no std::function boxing. Templates specialise the strategy’s on_tick<Venue> per venue so the dispatch is inlined; moves pass the order batch between components as a pointer handoff.

---

4. Failure Modes & First-Principles Breakdowns

1. The abstraction that isn’t zero-cost. std::function may heap-allocate its target; virtual calls mispredict; shared_ptr copies do atomic refcounts. Zero-cost is a property to verify (inspect the assembly / profile), not a property to assume. Prefer templates, unique_ptr/value types, and const&.
2. Allocating in the hot path. The §3 gap plus the growth spike (§2.3) are why every production hot path is pre-allocated and pool-based. A single push_back past capacity can memcpy hundreds of thousands of elements at an arbitrary instant.
3. Move mistakes. Forgetting std::move (silent copy) or moving from an object you then use (moved-from state is unspecified) and - the subtle one - capturing by value in a lambda that then copies a large container. Move correctness is a lifetime discipline, not just a speed trick.
4. Rule-of-five erosion. A type with a user-declared destructor but no move constructor loses its implicit move and silently copies. This is a classic hidden-regression source; follow the rule of five (or zero) rigorously.
5. Template bloat / compile time. Zero runtime cost is not zero build cost; heavy instantiation balloons the binary and compile time, and code-bloat can hurt the instruction cache. Specialise deliberately, not indiscriminately.

---

5. References

• Meyers, Scott: Effective Modern C++
• Meyers, Scott: Effective C++
• Stroustrup, Bjarne: The C++ Programming Language / A Tour of C++
• Ghosh, Sourav: Building Low Latency Applications with C++

---

6. Connected Graph Bridges

• Back: 03 · Memory & Cache · Index Hub
• Continue: 05 · Failure Modes · 06 · Advanced Extensions
• Sibling: Concurrency & Lockless Programming (atomics vs locks; the cost model of shared_ptr refcounts)
• Base: Numerical Methods (floating-point precision when choosing value types)