7.1.6 Advanced Extensions

Basic Prerequisites: 05 · Failure Modes and Bias–Variance & Model Selection.

---

1. Intuition & Practical Objective

Every failure mode in this folder has a tool that fixes it. This page is the launchpad for the pillar’s advanced toolkit - the techniques that turn “financial ML is hard” from a warning into a workflow. Three extensions, each targeting one failure:

1. The Deflated Sharpe Ratio (DSR) - corrects a reported Sharpe for the number of strategies you actually tried, killing the selection-bias illusion (Bailey & López de Prado 2014; AFML Ch. 14).
2. Fractional differentiation - replaces the crude integer difference ($d=1$) with a fractional one ($0<d<1$) so features are stationary without destroying the long memory that carries the signal (AFML Ch. 5).
3. Purged & embargoed / combinatorial purged cross-validation - the honest CV that respects label horizons (AFML Ch. 7; the dedicated sibling folder).

Why these three first? They are the minimal complete answer to the folder’s core question - “is this result real?” DSR says whether the Sharpe is real given how hard you searched; fractional differentiation makes features usable at all; purged CV makes the evaluation honest.

---

2. Mathematical Ground Truth & Derivations

2.1 The Deflated Sharpe Ratio (Bailey & López de Prado 2014; AFML §14.5.2)

Under the null of zero skill, the expected maximum in-sample Sharpe of $N$ independently tried strategies is $\approx \sqrt{2\ln N}$ (in units scaled by track length). The DSR measures whether a reported Sharpe $\widehat{\mathrm{SR}}$ exceeds that luck floor. For returns with per-period estimated Sharpe $\widehat{\mathrm{SR}}$, skewness ${\gamma }_3$, excess-kurtosis-corrected ${\gamma }_4$, and $n$ per-period observations, the deflated probability is

$$\mathrm{DSR}=\Phi \left[\frac{(\widehat{\mathrm{SR}}-{\mathrm{SR}}_0)\sqrt{n-1}}{\sqrt{1-{\gamma }_3\widehat{\mathrm{SR}}+\frac{{\gamma }_4-1}{4}{\widehat{\mathrm{SR}}}^2}}\right],$$

where ${\mathrm{SR}}_0=\sqrt{2\ln N}/\sqrt{n}$ is the expected-maximum benchmark. For Normal returns (${\gamma }_3=0,{\gamma }_4=3$) the denominator is $\sqrt{1+\frac{1}{2}{\widehat{\mathrm{SR}}}^2}$. The DSR is the probability that the true Sharpe exceeds the best of $N$ pure-luck strategies - the honest, selection-bias-corrected score.

2.2 Fractional differentiation (AFML Ch. 5)

A price is $I(1)$; differencing once ($d=1$) makes it stationary but wipes out memory. The fractional difference with order $0<d<1$ uses weights

$$w_k=(-1{)}^k(\genfrac{}{}{0ex}{}{d}{k}),(\genfrac{}{}{0ex}{}{d}{k})=\prod _{i=1}^k\frac{d-i+1}{i},X_t^{(d)}=\sum _{k=0}^l{w}_k{X}_{t-k}.$$

Because $|w_k|$ decays like $k^{-d-1}$ (unlike the sharp $d=1$ truncation), a fractionally-differenced series stays stationary while retaining long-memory dependence - the exact tradeoff AFML calls stationarity with maximum memory preservation.

2.3 Purged & embargoed CV (AFML Ch. 7)

If a label spans the forward window $[t_0,t_1]$, any training sample whose label window overlaps the test window must be purged (removed), and samples immediately after the test window must be embargoed (excluded) to kill auto-regressive residual memory. Combinatorial purged CV (CPCV) further averages over many purged split paths to estimate variance. These are the direct fix for the leaky-CV bias measured in 02.

---

3. Computational Implementation - the Deflated Sharpe in action

A single, decisive experiment: one backtest (3 years of daily returns, reported annualized Sharpe $1.00$) judged under increasing numbers of strategies tried. Stdlib only.

The same backtest (Sharpe $1.00$) is highly significant if you tried one strategy (DSR $=0.96$) and statistically dead if you tried 1,000 (DSR $=0.02$). This is why a strategy memo that does not report the number of trials $N$ makes the result uninterpretable (Bailey & López de Prado 2014; AFML Ch. 11). Always deflate before believing.

---

4. Failure Modes & First-Principles Breakdowns

1. Reporting Sharpe without $N$. A Sharpe of $1.0$ means opposite things at $N=1$ vs $N=1000$ (Experiment above). Selection bias is invisible unless the search space is disclosed.
2. Over-differencing. The integer difference $d=1$ destroys the long-memory component that may be the signal. Fractional differentiation is the correct middle ground (AFML Ch. 5).
3. Purged CV is not “more folds.” Purging the training set by the label horizon and embargoing the post-test window are structural changes, not tuning knobs; skipping them re-introduces the $+0.13$ $R^2$ leakage of page 02.
4. DSR assumes $N$ independent trials. Correlated trials (similar strategy families) reduce the effective $N$; the correction is conservative when trials are dependent.

---

5. References

• Bailey, David H. & López de Prado, Marcos, “The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality,” Journal of Portfolio Management 40(5), 2014
• López de Prado, Advances in Financial Machine Learning
• Bailey, Borwein, López de Prado & Zhu, “The Probability of Backtest Overfitting,” Journal of Computational Finance 20(4), 2017
• Hastie, Tibshirani & Friedman, The Elements of Statistical Learning

---

6. Connected Graph Bridges

• Back: 05 · Failure Modes · Index Hub
• Sibling topic-folder pages: Purged & Embargoed CV for Trees · Backtesting Hygiene & Deflated Sharpe · Feature Engineering & Target Labeling (fractional differentiation, labels)
• Base: Statistics & Inference · Econometrics & Time Series