7.1.1 Financial ML Pitfalls & Low SNR from Zero

Basic Prerequisites: Statistics & Inference.

---

1. Intuition & Practical Objective

This page builds the why of the whole folder with no prior ML or finance background needed. The objective is one idea: when the signal you are trying to learn is far weaker than the noise around it, a flexible model will “learn” the noise instead - and the more flexible it is, the more noise it memorizes. That is the fundamental reason most financial ML fails.

Start with a dumb question: why can a computer recognize a cat? Because a cat image is mostly signal - structure that is stable across all the photos. Now ask: why does an ML model for daily stock returns fail? Because a day’s return is almost all noise - the tiny predictable part (the signal) is drowned out. A model that fits the training data perfectly is therefore not “smart,” it is memorizing the particular noise pattern of those days, which will never repeat.

The classic picture is the bias–variance tradeoff (ESL eq. 7.9):

$$\text{MSE}=\underset{\text{irreducible noise}}{\underbrace{{\sigma }_{\epsilon }^2}}+\underset{\text{rigidity error}}{\underbrace{{\text{Bias}}^2}}+\underset{\text{overfit error}}{\underbrace{\text{Var}}}.$$

Three ideas, three “aha”s:

1. There is a floor you cannot remove. Even a perfect model still carries ${\sigma }_{\epsilon }^2$ - the noise in the data. In finance that floor is almost everything: $>99\%$ of daily return variance is noise.
2. Flexibility buys variance. A simple (high-bias) model can’t capture the signal but won’t overfit; a complex (low-bias) model captures the signal but also memorizes noise, and its out-of-sample error rises past a point.
3. Out-of-sample performance is the only truth. In-sample $R^2$ can be made arbitrarily high by adding parameters - a 25-degree polynomial can fit pure noise to $R^2\approx 0.07$ while predicting worse than guessing out-of-sample. That is exactly what an overfit quant backtest looks like.

---

2. Mathematical Ground Truth & Derivations

The bias–variance decomposition. For a target $Y=f(X)+\epsilon$ with $\epsilon$ independent noise of variance ${\sigma }_{\epsilon }^2$, and an estimator $\hat{f}$ fit on a training set, the expected squared error at a fixed point $x_0$ is (ESL §2.9 / eq. 7.9):

$$\text{MSE}(x_0)={\sigma }_{\epsilon }^2+\underset{{\text{Bias}}^2}{\underbrace{{\left(\mathbb{E}[\hat{f}(x_0)]-f(x_0)\right)}^2}}+\underset{\text{Var}}{\underbrace{\mathbb{E}\left[{\left(\hat{f}(x_0)-\mathbb{E}[\hat{f}(x_0)]\right)}^2\right]}}.$$

The three terms move in opposite directions as model complexity grows: complexity lowers Bias but raises Var. In a high-noise problem the Var term grows fast, so the total MSE is minimized by a deliberately simple model. This is why the industry default for tabular factor data is shallow gradient-boosted trees with aggressive regularization, not deep networks - deep networks win in high-SNR domains (vision, speech) and lose in low-SNR finance unless heavily regularized.

Where overfitting shows up in numbers. For a linear model with $p$ parameters fit on $N$ independent samples, the in-sample error understates the true error by roughly the optimism term (ESL §7.4):

$$\mathbb{E}[{\text{Err}}_{\text{in}}]\approx \mathbb{E}[{\text{Err}}_{\text{test}}]-\frac{2p}{N}{\sigma }_{\epsilon }^2.$$

With $N$ small and $p$ large, the gap $2p{\sigma }_{\epsilon }^2/N$ is huge - the model can drive in-sample error to zero while test error stays at (or above) the noise floor. In low-SNR finance this gap is the whole story: the fewer genuinely independent samples you have, the faster a flexible model locks onto noise.

---

3. Computational Implementation - watch a polynomial memorize noise

A single experiment makes the entire intuition concrete: generate low-SNR finance-like data (a mild hidden curve buried in noise), fit polynomials of increasing degree on a small training set, and compare in-sample vs out-of-sample $R^2$. Stdlib only.

The linear model (degree 1) is the only one that generalizes ($+0.016$ OOS). Each extra degree lifts in-sample $R^2$ (to $0.107$ at degree 15) while collapsing OOS - down to $-0.065$ at degree 15 and a catastrophic $-2.1$ at degree 25, i.e. worse than predicting the mean. That rising in-sample score is the illusion every overfit backtest sells you.

---

4. Failure Modes & First-Principles Breakdowns

1. “Higher in-sample $R^2$ = better model.” False in low SNR. The experiment above shows in-sample $R^2$ rising while OOS falls. Judge models only on honest out-of-sample error.
2. Using the test set for tuning. Every time you look at test performance and adjust, the test set becomes training data and its score inflates (ESL §7.1: keep the test set “in a vault”). In finance this is how people “discover” strategies that never survive live.
3. Ignoring the noise floor. Because ${\sigma }_{\epsilon }^2$ dominates, even a perfect model can only explain $\sim 0.25\%$ of daily return variance (see 03 · The Low-SNR Problem). Any claim of large OOS $R^2$ is leakage, not skill.

---

5. References

• Hastie, Tibshirani & Friedman, The Elements of Statistical Learning
• López de Prado, Advances in Financial Machine Learning

---

6. Connected Graph Bridges

• Base: Statistics & Inference · 05 · Bias–Variance & Model Selection
• Continue: 02 · Why Finance Is Different · Index Hub