2.6.1 Execution Backtesting from Zero

Basic Prerequisites: Market Microstructure & Order Types.

---

1. Intuition & Practical Objective

This page builds the why of execution backtesting with no prior knowledge needed. The objective is one idea: a trade price and a fill are not the same thing. A signal backtest treats the price series as something you can transact at on demand. An execution backtest cannot - because the price series is the output of a matching process, and whether you get to participate in that process is a separate, harder question.

Start with the dumbest question: why can’t I just test my execution algorithm by backtesting on prices? Because “buy 10,000 shares at 10:00” is not an instruction a price series can honour. To turn it into a fact you must answer three things the price series never asked:

1. Did I fill at all? A market order fills (minus size you walk through), but a limit order only fills if the queue ahead of you is consumed. Book that as a coin that may come up “no.”
2. At what price relative to the decision price? You buy at the ask, not the mid - you pay the half-spread on entering and again on exiting.
3. Did the market move because of me, or against me? Your own order has impact, and your passive fills are adversely selected.

Three “aha”s:

1. The mid is a theory; the trade is a price. The Roll model (Hasbrouck Ch 3) writes the trade price as $p_t=m_t+q_{t}c$: the efficient price plus a $\pm c$ bounce at half the spread. A backtest that fills at $m_t$ has silently deleted $c$ per share on every trade.
2. A fill is a queue event, not a price event. “The market printed at my price” $\ne$ “I filled.” You fill when the cumulative outflow ahead of you passes your position: $\xi \ge x$. Ignoring $x$ is the single largest source of fake edge.
3. Execution cost is a separate return stream. Perold’s implementation shortfall decomposes realised vs paper P&L into an execution cost (how badly you paid) and an opportunity cost (what you failed to execute). A signal backtest sets the first to zero and the second to zero by assumption.

The one-sentence essence. “A signal backtest tells you whether the idea was right; an execution backtest tells you whether the idea survives contact with the order book - and these two answers can have opposite signs for the same alpha.”

---

2. Mathematical Ground Truth & Derivations

The two experiments, side by side. Suppose a signal earns an expected favourable move ${\alpha }_H$ over a holding horizon of $H$ ticks, and the mid moves $\sigma \sqrt{H}$ of noise. Then

• Signal backtest P&L per trade (fill at mid, both legs, full size): $g={\alpha }_H+\epsilon ,\epsilon \sim \mathcal{N}(0,{\sigma }^{2}H).$
• Execution backtest P&L per trade (you cross the spread in and out, and pay impact per leg): $g_{\text{exec}}={\alpha }_H-s-2h,$ where $s$ is the quoted spread and $h$ the per-leg temporary impact.

The Sharpe ratios are the same denominator $\sigma \sqrt{H}$; the cost terms subtract expected value only. So

$${\text{SR}}_{\text{signal}}=\frac{{\alpha }_H}{\sigma \sqrt{H}},{\text{SR}}_{\text{exec}}=\frac{{\alpha }_H-s-2h}{\sigma \sqrt{H}},$$

and the whole game is the sign and size of $s+2h$ relative to ${\alpha }_H$.

Perold’s implementation shortfall (Hasbrouck Ch 14, eq 14.1) is the accounting identity that makes the omission precise. With $n_0$=initial position, $v$=desired position, $n_1$=final position, decision/arrival price ${\pi }_0$, terminal price ${\pi }_1$:

$$\text{IS}=\underset{\text{execution cost}}{\underbrace{(n_1-n_0{)}^{\prime }(p-{\pi }_0)}}+\underset{\text{opportunity cost}}{\underbrace{(v-n_1{)}^{\prime }({\pi }_1-{\pi }_0)}},$$

A signal backtest that assumes you always trade the full size at ${\pi }_0$ forces both terms to zero. An execution backtest must simulate $p$ (the realised fill price, from the book) and $n_1$ (the realised filled quantity, from the queue).

Effective vs realized cost (Hasbrouck Ch 14, eq 14.2) splits what you paid into spread and impact:

$$p_t-m_t=\underset{\text{realized cost}}{\underbrace{(p_t-m_{t+5})}}+\underset{\text{price impact}}{\underbrace{(m_{t+5}-m_t)}}.$$

A backtest that records only $p_t$ still misses the $m_{t+5}-m_t$ impact term - the part of the move that is yours.

---

3. Computational Implementation - the same alpha, two verdicts

One alpha, evaluated two ways. The signal backtest fills at the mid; the execution backtest pays the spread on both legs plus impact. Stdlib only.

The signal backtest is identical in all three rows ($\text{SR}=0.671$): it never sees cost. The execution backtest takes the same alpha to $\text{SR}=0.224$ in a tight market, to $\text{SR}=-0.222$ (a loser) in a wide market, and back to $0.570$ where costs are tiny. The alpha did not change; the verdict did. This is why “backtested Sharpe” is meaningless without the fill and cost model underneath it.

---

4. Failure Modes & First-Principles Breakdowns

1. Filling at the mid (deleting the spread). The Roll model says the trade price is $m_t\pm c$. A mid-fill backtest earns a half-spread it can never collect; on a strategy that trades often, this is a pure fiction with an unbounded cumulative cost.
2. Assuming full size fills. A signal backtest assumes $n_1=v$ in Perold’s identity. Real limit orders are censored: only a minority of submitted limit orders ever execute (Hasbrouck Ch 15 reports roughly $13\%$ in the Hasbrouck–Saar data). Unfilled size is the opportunity cost term, invisible to the naive test.
3. Ignoring your own impact. Treating $p={\pi }_0$ drops the $m_{t+5}-m_t$ term; for any size beyond a rounding lot the omitted cost is linear in the metaorder and can exceed the alpha entirely.
4. Benchmark confusion. VWAP, arrival price, and close are different benchmarks; a strategy can look great vs VWAP and terrible vs arrival (Hasbrouck Ch 14). The “execution backtest” number is only as good as the benchmark it is measured against.
5. One-way thinking. Costs are subtracted on both legs and on every reprice. A backtest that charges entry only, or only charges when the trade is a taker, will systematically flatter the strategy.

---

5. References

• Perold, André F. - “The implementation shortfall: paper vs. reality,” Journal of Portfolio Management 14(3), 4–9 (1988). The origin of every execution benchmark; the execution/opportunity-cost decomposition.
• Hasbrouck, Joel - Empirical Market Microstructure (2007)
• Almgren, Robert; Chriss, Neil - “Optimal execution of portfolio transactions,” Journal of Risk 3(2) (2000)
• Harris, Larry - Trading and Exchanges (2003)

---

6. Connected Graph Bridges

• Base: Market Microstructure & Order Types · Backtesting Hygiene (the signal-backtest half)
• Continue: 02 · Why Execution Backtests Lie · Index Hub
• Cost model to calibrate: Optimal Execution & Almgren–Chriss · Market Impact & Depth