5.8.5 Failure Modes & Practice

Basic Prerequisites: 04 · Turnover & the Multi-Period Trade-Off.

---

1. Intuition & Practical Objective

Every failure in this folder is a discrepancy between the objective you optimized and the objective you were actually paid on. There are exactly three places that discrepancy enters, and each has a name in industry:

1. Cost underestimation - you optimized with $\hat{c}$ but the market charged $c_{\text{true}}>\hat{c}$. The optimizer then trades too much, and the extra trades earn less than they cost. This is the reason strategies die between backtest and production.
2. Turnover explosion - your alphas are noisy, your optimizer is linear in ${\Sigma }^{-1}$, and every fresh estimate re-shuffles the whole book. The portfolio becomes a perpetuum mobile of commissions: high gross, negative net.
3. Constraint interaction & tracking drift - a turnover budget or a tight cap does not just slow you down; it silently changes where you can be, so you accumulate an unmanaged gap to the target that you never fully close.

The practical objective of this page is diagnostic: given a strategy that looks good on paper and loses money live, decide in five minutes which of these three is happening, because the fixes are completely different (re-estimate costs / shrink the alphas / loosen the budget respectively).

The one-sentence essence. “Optimizing with the wrong cost is a bias; optimizing a noisy alpha with no cost term is a variance; enforcing a budget without tracking the residual is a drift - and a strategy must survive all three.”

---

2. Mathematical Ground Truth & Derivations

2.1 Cost underestimation is a bias you can bound

Let $w(\hat{c})$ solve the cost-aware problem with assumed cost $\hat{c}$ and true cost $c_{\text{true}}$. Realized certainty equivalent is

$$U(\hat{c})={\mu }^{\top }w(\hat{c})-\frac{\delta }{2}w(\hat{c}{)}^{\top }\Sigma w(\hat{c})-c_{\text{true}}\Vert w(\hat{c})-w_0{\Vert }_1.$$

Because $w(\hat{c})$ is the maximizer of the objective with $\hat{c}$, the envelope theorem gives

$$\frac{dU}{d\hat{c}}{|}_{\hat{c}=c_{\text{true}}}=0,$$

so $U$ is maximized exactly at the true cost - and mistakes are locally second-order but quadratic in the error. In practice the curve is asymmetric: assuming $\hat{c}=0$ (a cost-free backtest) is a far bigger error than assuming $\hat{c}=2c_{\text{true}}$, because the zero-cost problem has no no-trade region at all and therefore trades continuously. §3(A) quantifies it.

2.2 Turnover explosion is the estimation-error amplifier again

From the unconstrained first-order condition, the weight sensitivity to the alpha signal is

$$w^{\star }=\frac{1}{\delta }{\Sigma }^{-1}\mu ⟹d{w}^{\star }=\frac{1}{\delta }{\Sigma }^{-1}d\mu ,$$

so an alpha revision $d\mu$ in the small-eigenvalue direction of $\Sigma$ moves weights by a large multiple. If alphas are re-estimated every period with noise of size $s$, the expected per-period turnover scales like $\frac{1}{\delta }\Vert {\Sigma }^{-1}\Vert s$ - the same ${\Sigma }^{-1}$ amplification that makes naive MVO an estimation-error maximizer (02), now paid in trading costs rather than in risk. Two structural facts matter:

• The ${\ell }_1$ no-trade band is the only brake. Without it, turnover is linear in the noise; with it, small alpha revisions produce zero trade until the gap clears $\theta =c/(\delta {\sigma }^2)$.
• The cost-blind annualized turnover is a property of the signal’s noise, not of the market. §3(B) shows a cost-blind optimizer at $10$ bp/month alpha noise turning over $3.83\times$/yr; the same optimizer on a noisier signal would turn further.

2.3 A budget truncates, and the residual accumulates

If you cap per-period turnover at $\tau$ by scaling the trade, then whenever the required move exceeds $\tau$ you leave a residual gap

$$g_t=(w_t^{\text{target}}-w_t),\Vert g_t{\Vert }_1\text{persists and accumulates until the target stops moving}.$$

Over a horizon in which the target keeps revising, the steady-state gap is roughly the ratio of the per-period revision rate to the budget - you are chasing a moving object with a speed limit. §3(C) measures it. The first-principles warning: a turnover budget is a tracking-error budget in disguise, and it should be chosen with that in mind rather than set at “whatever the risk committee last used.”

---

3. Computational Implementation - the three failure modes, measured

(A) assume $\hat{c}\in \{0,5,10,25,50\}$ bp while the true cost is $25$ bp and watch realized utility; (B) a $24$-month backtest with noisy alphas, cost-blind vs cost-aware; (C) truncate turnover at a per-period budget and measure the accumulated tracking gap. numpy + scipy.

Four verified readings:

• (A) The optimizer is honest about cost - you have to be honest too. Utility peaks at $\hat{c}=25$ bp $=c_{\text{true}}$ ($0.14850$), exactly the envelope-theorem prediction. Assuming zero cost costs $12.3\%$ of the achievable certainty equivalent ($0.14850\to 0.13017$); assuming double ($50$ bp) costs $4.2\%$ ($0.14225$) and drives turnover to literally zero ($\text{TO}=0.0000$) - you leave the entire book untouched even though trading is profitable. The asymmetry is the lesson: underestimating costs overtrades, overestimating them paralyzes.
• (B) Cost-awareness cuts turnover by $76\%$ and raises net alpha. Cost-blind: annualized turnover $3.83\times$, gross $0.2035$, net $0.1958$. Cost-aware: turnover $0.91\times$, gross $0.2052$, net $0.2034$. Gross alpha is essentially unchanged ($0.2035\to 0.2052$) - the cost-aware optimizer gives up nothing - yet $\approx 77\%$ of the annual cost bill disappears. This is the clearest possible refutation of the folk belief that “cost-aware optimization costs you alpha.”
• (C) A turnover budget leaves a permanent tracking gap. With $\tau =0.25$/period the residual gap is $0.3517$ in ${\ell }_1$; tighten to $\tau =0.05$ and the gap balloons to $0.8348$ - the portfolio is more than $80\%$ of notional away from its own target, on average, forever. The gap is not a transient; it is the steady state of chasing a moving target under a speed limit.
• (C) The budget’s benefit is invisible in the gap. Look at what the budget actually purchased: at $\tau =0.25$ you spend half the turnover ($0.1250$ vs $0.2425$) of the cost-blind policy for a gap of $0.35$. Whether that is a good trade requires the cost model - which is why a budget without a cost estimate is a blind policy.

---

4. Failure Modes & First-Principles Breakdowns

1. Backtesting at zero cost. The most common single error in quantitative finance. It removes the no-trade region entirely, so the backtest trades continuously and reports an alpha that exists only in the absence of a market. Always backtest with the assumed cost inside the optimizer and the true (conservatively high) cost applied to realized returns.
2. Underestimating impact for large notional. Proportional costs are known; impact is estimated. When AUM grows, the same weights imply larger trades, $\eta$ rises with size, and a strategy that worked at $10M stops working at $1B - not because the signal decayed but because the cost function changed. Re-estimate $\eta$ as a function of your own AUM, not from a vendor deck.
3. Treating noise in alphas as signal to trade on. Every fresh estimate looks like an opportunity to a cost-blind optimizer. The remedy is two-sided: shrink the alphas (Covariance Shrinkage & RMT) and price the trades.
4. Setting the turnover budget without a tracking-error target. §3(C) shows the budget is a tracking-error budget. A risk committee that sets $\tau =0.05$ “to be safe” is silently choosing an $0.83$ tracking gap per period. Make the substitution explicit.
5. Ignoring the constraint price when combining constraints. A per-name cap plus a group cap plus a turnover budget interact (§02): after enough constraints, the marginal one is free and the binding one is invisible. Always report the active set and the multipliers, not just the weights.

---

5. References

• Grinold & Kahn (2000), Active Portfolio Management, 2nd ed.
• Lobo, Fazel & Boyd (2007), Annals of OR 152:341–365
• Almgren & Chriss (2000/01), Journal of Risk 3(2)
• Clarke, de Silva & Thorley (2002), FAJ 58(5):48–66
• DeMiguel, Garlappi & Uppal (2009), RFS 22(5):1915–1953
• Hasbrouck (2007), Empirical Market Microstructure

---

6. Connected Graph Bridges

• Back: 04 · Turnover & the Multi-Period Trade-Off
• Siblings: Robust Optimization · 05 (input-error failure modes) · MVO · Estimation-Error Maximizers · Covariance Shrinkage & RMT
• Execution bridge: Optimal Execution (Almgren–Chriss) · POV
• Continue: 06 · Advanced Extensions
• Hub: Index Hub