5.6.2 The Estimation-Error Problem

Basic Prerequisites: 01 · From Zero.

---

1. Intuition & Practical Objective

Robust optimization exists because of one empirical fact: the numbers you feed a Markowitz optimizer are estimates, and the optimizer’s sensitivity to those estimates is so extreme that the “optimal” portfolio is mostly a portrait of your sample’s luck. This page makes the estimation error quantitative - where it comes from, how it propagates, which estimate hurts most, and how badly the in-sample promise decays out of sample.

The chain of reasoning is short:

1. Every input is a random variable. The sample mean $\hat{\mu }=\frac{1}{T}{\sum }_t{r}_t$ is itself random, centered on the truth with covariance $\Sigma /T$. The sample covariance $\hat{\Sigma }$ is a Wishart draw, badly conditioned when $N$ approaches $T$.
2. The optimizer maps those random variables non-linearly into weights. $w^{\star }=\frac{1}{\delta }{\hat{\Sigma }}^{-1}\hat{\mu }$ therefore inherits a heavy-tailed, wildly dispersed distribution. Two samples from the same market can give you portfolios that look nothing alike.
3. Errors in the mean dominate everything. Chopra & Ziemba (1993) - a canonical, heavily-cited result - found errors in means cost roughly $11\times$ as much as errors in variances (and $\sim 21\times$ as much as errors in covariances), while errors in variances cost $\sim 2\times$ errors in covariances, measured by certainty-equivalent loss. The means, not the covariance, are where robustness must be spent.

The practical objective is a mindset: stop treating the optimizer’s output as an answer, and start treating the inputs as the real problem. Every fix downstream (shrinkage, constraints, resampling, robust sets) is an attempt to make the portfolio depend less on a fragile point estimate.

---

2. Mathematical Ground Truth & Derivations

Sampling distribution of the inputs. For i.i.d. returns $r_t\sim (\mu ,\Sigma )$, the sample moments satisfy

$$\hat{\mu }\sim \mathcal{N}\left(\mu ,\frac{1}{T}\Sigma \right),T\hat{\Sigma }\sim \mathrm{Wishart}(T-1,\Sigma ).$$

So the standard error of the mean of asset $i$ is ${\sigma }_i/\sqrt{T}$. With $T=60$ months, a $20\%$-vol asset has a monthly mean estimate whose $1\sigma$ error is $0.20/\sqrt{12}/\sqrt{60}\approx 0.0074/\dots$ - comparable to or larger than the true monthly mean it is trying to measure. That is the heart of the matter: the signal is smaller than its own error bar.

Error propagation. Write $\hat{\mu }=\mu +\eta$ with $\mathrm{Cov}(\eta )=\Sigma /T$. To first order,

$$d{w}^{\star }=\frac{1}{\delta }{\Sigma }^{-1}\eta ,\mathrm{Cov}(w^{\star })\approx \frac{1}{{\delta }^2}{\Sigma }^{-1}\Sigma {\Sigma }^{-1}\cdot \frac{1}{T}=\frac{1}{{\delta }^{2}T}{\Sigma }^{-1}.$$

The weight covariance is proportional to ${\Sigma }^{-1}$ - again the inverse covariance. The directions where the market has little risk (${\lambda }_i$ small) are precisely where the estimator of $\mu$ is most uncertain and where the weights are most volatile.

Bias–variance view (ESL 2nd ed.). Any estimator’s expected test error decomposes as

$$\mathrm{Err}(x_0)={\sigma }_{\epsilon }^2+{\mathrm{Bias}}^2(\hat{f}(x_0))+\mathrm{Var}(\hat{f}(x_0)),$$

three terms (the irreducible noise ${\sigma }_{\epsilon }^2$ form is ESL eq. 2.46 / eq. 7.9). A portfolio built from noisy $\hat{\mu }$ is a high-variance estimator $\hat{f}$: great in-sample (it was fit to that sample), poor out-of-sample. Shrinkage and robustness trade a little bias for a large variance reduction - exactly the ridge/smoothing idea of ESL Ch. 3 and §5.

Chopra–Ziemba ranking (1993). Using certainty-equivalent loss on the objective ${\mu }^{\top }w-\frac{\delta }{2}{w}^{\top }\Sigma w$, the damage ranks

$$\text{errors in }\mu >\text{errors in }\Sigma \gtrsim \text{errors in }\mathrm{diag}(\Sigma ),$$

with means dominating covariances by an order of magnitude. §3 reproduces the ranking on our universe.

---

3. Computational Implementation - measuring the damage

Three experiments on the shared universe: (A) how dispersed the optimizer is across re-draws of the same market, (B) the in-sample → out-of-sample Sharpe collapse, (C) mean-errors vs covariance-errors via certainty-equivalent loss.

Three verified findings:

• (A) The optimizer is not stable. Redrawing the same market’s history of the same length produces weights with standard deviation $1.0$–$2.1$ per asset - larger than the weights themselves. Gross exposure roams from $1.45$ to $22.05$ across re-draws. A portfolio whose leverage is a coin flip is not a portfolio.
• (B) The in-sample promise decays. Naive MVO’s Sharpe falls from $1.93$ in-sample to $0.63$ out-of-sample - a $67\%$ collapse - while $1/N$ barely moves ($1.24\to 0.66$). The optimizer spent its effort fitting the sample.
• (C) Means dominate covariance. Using only sample means (true covariance) costs $0.02327$ in certainty equivalent; using only the sample covariance (true means) costs $0.00083$ - a factor of $\sim 28$. The direction and order of magnitude match Chopra & Ziemba (1993).

---

4. Failure Modes & First-Principles Breakdowns

1. Ignoring the estimator’s sampling distribution. People treat $\hat{\mu }$ as data, not as a draw. The correct object is a distribution over $\mu$; every robust method is an explicit model of that distribution (§03’s uncertainty sets are confidence regions).
2. Spending robustness on the wrong input. Covariance regularization is popular because $\hat{\Sigma }$ is visibly ill-conditioned, but the certainty-equivalent loss is dominated by the mean. Shrinking only $\Sigma$ leaves the main leak open.
3. “More data will fix it.” DeMiguel, Garlappi & Uppal (2009) show that even $14$ sophisticated optimizers fail to beat $1/N$ out of sample across seven datasets, needing on the order of $3,000$–$6,000$ months of data to win - decades of history the market does not provide. Time is not the remedy; regularization is.
4. $N\gtrsim T$ catastrophic conditioning. When assets approach periods, $\hat{\Sigma }$ becomes singular or near-singular and ${\hat{\Sigma }}^{-1}$ explodes. The first-principles fix is to impose structure (shrinkage toward a factor/diagonal target), not to drop assets one by one.

---

5. References

• Chopra, Vijay & Ziemba, William: The Effect of Errors in Means, Variances, and Covariances on Optimal Portfolio Choice, JPM 19(2):6–11, 1993
• Best & Grauer (1991), RFS 4(2)
• DeMiguel, Garlappi & Uppal (2009), Optimal Versus Naive Diversification, RFS 22(5):1915–1953
• Hastie, Tibshirani & Friedman, The Elements of Statistical Learning, 2nd ed.
• Jorion, Philippe: Bayes–Stein Estimation for Portfolio Analysis, JFQA 21(3):279–292, 1986

---

6. Connected Graph Bridges

• Back: 01 · From Zero · 05 · Estimation-Error Maximizers
• Statistics base: Point Estimation · Bias–Variance & Validation
• Forward: 03 · Robust Formulations · Covariance Shrinkage & RMT · Index Hub