7.1 Financial ML Pitfalls & Low SNR

Basic Prerequisites: Statistics & Inference (bias-variance, model selection) and Econometrics & Time Series (stationarity). (these are the folder-level prerequisites for pages 02–06; page 01 states its own, smaller, entry requirements)

1. Intuition & Practical Objective

In computer vision a cat is a cat: the physical generator of a cat image does not change from 2010 to 2024, so the Signal-to-Noise Ratio (SNR) is enormous and a model that memorizes structure generalizes. In finance the situation is inverted: the signal is microscopic and the noise dominates. A real quantitative edge on daily data is an Information Coefficient (IC) of roughly $0.05$ , which means the best possible out-of-sample $R^{2}$ a model can achieve is $0.0 5^{2} = 0.25%$ of variance. Everything else a backtest shows is, by construction, overfitting, leakage, or selection bias.

This folder is the cornerstone of Pillar 7 and the hub for everything else in the pillar. Its job: (a) give you the fast signal/hygiene lookup below (job #1), and (b) route you through six sub-pages that build the intuition from zero, prove why finance is different, quantify the low-SNR ceiling, expose non-stationarity and non-IID samples, name the failure modes, and hand you the advanced toolkit (Deflated Sharpe, fractional differentiation, purged CV).

The one-sentence essence. “Finance is not normal ML: the signal-to-noise ratio is microscopic, the data-generating process is non-stationary, the samples are non-IID and overlap in time, and the market actively adapts to destroy your edge the moment you deploy it - so almost every impressive in-sample result is a false positive produced by overfitting, look-ahead, or selection bias.”

2. Mathematical Ground Truth & Lookup

All numbers in the check column were re-executed and reproduced exactly from the verified code in §3 (stdlib/numpy only).

Notation: $r_{t}$ return, $s_{t}$ signal, $ε_{t}$ noise, $σ_{s}, σ_{ε}$ signal/noise std, $T$ number of observations, $ρ$ lag-1 autocorrelation, $N$ number of trials/strategies tried, $q$ observations per year.

Quantity	Formula	Verified check
Return decomposition	$r_{t} = s_{t} + ε_{t}$ , $s_{t} ⊥ ε_{t}$	-
Signal-to-noise ratio	$SNR = \frac{σ _{s}}{σ _{ε}}$	-
$R^{2}$ from SNR (Gaussian)	$R^{2} = \frac{σ _{s}^{2}}{σ _{s}^{2} + σ _{ε}^{2}} = \frac{SNR ^{2}}{1 + SNR ^{2}}$	$SNR = 0.0501 \Rightarrow R^{2} = 0.00250$
IC ceiling on OOS $R^{2}$	$R_{OOS}^{2} \leq IC^{2}$	$IC = 0.05 \Rightarrow R^{2} = 0.25%$
Effective sample size (AR(1))	$N_{eff} = T \frac{1 - ρ}{1 + ρ}$	$ρ = 0.9, T = 1000 \Rightarrow N_{eff} = 52.6$
Expected max of $N$ luck strategies	$E [max] \approx 2 ln N$ (std Normals)	$N = 1000 \Rightarrow 3.717$
Bias–variance (ESL eq. 7.9)	$MSE = σ_{ε}^{2} + Bias^{2} + Var$	-
Minimum backtest length	$y \geq \frac{2 ln N}{SR _{IS}^{2}}$ years	$N = 45, SR = 1 \Rightarrow y \geq 7.6$

The critical scaling caveat. The IC ceiling ( $R^{2} \leq IC^{2}$ ) is why any model claiming an out-of-sample $R^{2} > 10%$ on daily asset returns is mathematically guaranteed to be contaminated by look-ahead or leakage - no legitimate model of daily cross-sections can get there.

3. Computational Implementation - the signal & selection-bias engine

This runs on the standard library only. It reproduces every verified number above and is the mental template for the whole folder: SNR↔ $R^{2}$ bookkeeping, and the selection-bias floor on any backtest.

4. Failure Modes & First-Principles Breakdowns

Hub signposts - the full treatment lives in 05 · Failure Modes & Practice. In one line each:

Overfitting with few effective samples - flexible models memorize noise; $N_{eff}$ (not the raw row count) is what a complex model can actually “use.”
Random K-fold CV on financial data - shuffling non-IID, overlapping-label series leaks the future across folds (spurious $+ 0.13$ $R^{2}$ on pure noise, see 02/05).
Look-ahead & point-in-time leakage - using information not yet available at the timestamp (reported earnings, survivorship-filtered universes).
Selection bias / multiple testing - the best of $N$ tried strategies has expected Sharpe $\approx 2 ln N$ , not $0$ ; that “edge” is luck.
Non-stationarity - a model fit in one regime dies in the next (sign flips, vol breaks, structural shifts).

5. References

López de Prado, Marcos: Advances in Financial Machine Learning (Wiley, 2018)
López de Prado, Marcos: Machine Learning for Asset Managers (Cambridge Elements, 2020)
Hastie, Tibshirani & Friedman: The Elements of Statistical Learning (2nd ed., 2009)
Bailey, David H. & López de Prado, Marcos: “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance,” Notices of the AMS 61(5), 2014
Gu, Shihao; Kelly, Bryan; Xiu, Dacheng: “Empirical Asset Pricing via Machine Learning,” RFS 33(5), 2020

6. Connected Graph Bridges

Foundational base: Statistics & Inference · 05 · Bias–Variance & Model Selection · Econometrics & Time Series (stationarity) · Probability & Measure Theory
Sibling topic: Tree-Based Factor Ranking & Purged CV (the algorithm-side answer to the pitfalls)
Backtesting hygiene: Backtesting Hygiene & Deflated Sharpe
Sub-pages (in-folder): 01 From Zero · 02 Why Finance Is Different · 03 The Low-SNR Problem · 04 Non-Stationarity & Samples · 05 Failure Modes & Practice · 06 Advanced Extensions

Beginner: start at 01 · Practitioner: start at 05

Kwant Atlas

Explorer

7.1 Financial ML Pitfalls & Low SNR

1. Intuition & Practical Objective

2. Mathematical Ground Truth & Lookup

3. Computational Implementation - the signal & selection-bias engine

4. Failure Modes & First-Principles Breakdowns

5. References

6. Connected Graph Bridges

7.1.1 Financial ML Pitfalls & Low SNR from Zero

7.1.2 Why Finance Is Not Normal ML

7.1.3 The Low-Signal-to-Noise Problem

7.1.4 Non-Stationarity and Non-IID Samples

7.1.5 Failure Modes & Real-World Practice

7.1.6 Advanced Extensions