5.5.6 HRP Extensions

Basic Prerequisites: 04 · Recursive Bisection and Risk Parity & ERC.

---

1. Intuition & Practical Objective

HRP is a recipe, and the recipe has two ingredients you can swap: (i) how you measure a cluster’s risk, and (ii) what you do inside a cluster. HRP uses the naive-risk-parity (inverse-variance) sub-portfolio variance and recurses all the way to single assets. Every sensible variant changes one of those two choices:

• Hierarchical Equal Risk Contribution (HERC) - replace the naive-RP variance with the ERC solution, and stop the tree at $K$ clusters: allocate by ERC within each cluster and by inverse cluster risk across them. This makes “risk parity” true at both levels.
• Nested Clustered Optimization (NCO) - keep the clustering, but replace the recursion with a small mean–variance (or min-variance) solve inside each cluster and again across clusters. This brings expected returns back in without ever inverting the full $\Sigma$ - the natural bridge between HRP and MVO.
• Denoised-HRP - run the whole tree and its splits on a shrunk / RMT-denoised covariance rather than the sample matrix, attacking the input noise of 05 · Failure Modes.

This page builds HERC concretely (with numbers), states NCO and the Ward-linkage variant, and maps the research frontier.

The one-sentence essence. “Keep HRP’s tree, but upgrade the risk measure (naive-RP → ERC), the objective (bisection → nested MVO), or the input (sample $\Sigma$ → denoised $\Sigma$) - and the covariance inverse stays safely out of the loop.”

---

2. Mathematical Ground Truth & Derivations

2.1 HERC - HRP with ERC at every level

Cut the dendrogram at $K$ clusters, i.e. perform only the first $N-K$ merges; the $K$ surviving active clusters are $\{{\mathcal{C}}_1,\dots ,{\mathcal{C}}_K\}$ (in the worked universe these are exactly the economic blocks: equity, rates, commodity). Then:

Within a cluster ${\mathcal{C}}_k$, solve the ERC problem: find $w^{(k)}>0,1^{\top }{w}^{(k)}=1$ such that every asset contributes equal risk,

$$R{C}_i=w_i^{(k)}\frac{({\Sigma }_{{\mathcal{C}}_k}{w}^{(k)}{)}_i}{\sqrt{w^{(k)\top }{\Sigma }_{{\mathcal{C}}_k}{w}^{(k)}}}=\frac{\sigma (w^{(k)})}{|{\mathcal{C}}_k|}\forall i\in {\mathcal{C}}_k.$$

Across clusters, allocate by inverse cluster risk (naive risk parity at the cluster level), using each cluster’s ERC portfolio variance $V_k=w^{(k)\top }{\Sigma }_{{\mathcal{C}}_k}{w}^{(k)}$:

$${\beta }_k=\frac{1/V_k}{{\sum }_{l}1/V_l},w_i={\beta }_{k(i)}{w}_i^{(k(i))}.$$

The difference from HRP is precise: HRP’s cluster risk $V_{\mathcal{C}}$ is the variance of the inverse-variance portfolio; HERC’s $V_k$ is the variance of the ERC portfolio. ERC weights differ from inverse-variance whenever intra-cluster correlations are non-constant, so the two allocators genuinely diverge.

2.2 Ward linkage - the variance-minimizing merge

Ward’s method merges the pair whose fusion increases within-cluster sum of squares the least. Its Lance–Williams coefficients (with $n_i$ the size of cluster $i$ and $k$ the other cluster) are

$${\alpha }_i=\frac{n_i+n_k}{n_i+n_j+n_k},{\alpha }_j=\frac{n_j+n_k}{n_i+n_j+n_k},\beta =\frac{-n_k}{n_i+n_j+n_k},\gamma =0,$$

so

$$d(u,k)=\frac{(n_i+n_k)d(i,k)+(n_j+n_k)d(j,k)-n_{k}d(i,j)}{n_i+n_j+n_k}.$$

Ward produces compact, equal-sized clusters and, on many equity universes, higher cophenetic correlation than single/complete (see §03). It is the standard alternative when single-linkage chaining is a concern - but it is not a silver bullet; validate on the cophenetic correlation and out-of-sample risk.

2.3 Nested Clustered Optimization (NCO)

NCO (López de Prado, AFML Ch. 16) uses the same tree but computes, at each cluster, the mean–variance (or min-variance) weights of that cluster, then treats each cluster’s portfolio as a “super-asset” and solves a small MV problem across clusters. Formally, for cluster ${\mathcal{C}}_k$ with expected returns ${\mu }_{{\mathcal{C}}_k}$,

$$w^{(k)}=\frac{{\Sigma }_{{\mathcal{C}}_k}^{-1}{\mu }_{{\mathcal{C}}_k}}{1^{\top }{\Sigma }_{{\mathcal{C}}_k}^{-1}{\mu }_{{\mathcal{C}}_k}},\text{then across clusters: }\beta =\frac{{\Sigma }_{\text{cl}}^{-1}{\mu }_{\text{cl}}}{1^{\top }{\Sigma }_{\text{cl}}^{-1}{\mu }_{\text{cl}}},$$

where ${\Sigma }_{\text{cl}}$ and ${\mu }_{\text{cl}}$ are the covariance/means of the $K$ cluster portfolios. The key benefit: the matrices inverted are size $|{\mathcal{C}}_k|\times |{\mathcal{C}}_k|$ and $K\times K$ - small and well-conditioned - never the $N\times N$ sample covariance. NCO therefore reintroduces return forecasts (fixing HRP’s structural blindness to alpha) while retaining the dimensionality reduction that makes the inversion safe.

---

3. Computational Implementation - HERC vs HRP on the 8-asset universe

Runs on numpy. We agglomerate with average linkage, cut the dendrogram into $K=3$ clusters, build HERC (ERC within clusters, inverse cluster risk across), and compare against classic HRP. We also print the average-linkage merge heights and the resulting quasi-diagonal order.

Read it:

• The $K=3$ cut recovers the economic blocks exactly - $\{$EQ1,EQ2,EQ3$\}$, $\{$BD1,BD2$\}$, $\{$CM1,CM2,CM3$\}$ - with no labels supplied. The clustering found the asset classes from correlations alone. That, not any single weight, is HRP’s real product: an unsupervised taxonomy of the universe.
• HERC and HRP agree on the big picture, differ in detail. Both load the bond block heavily (HERC $0.4270+0.3660=79.3\%$, HRP $74.9\%$), but HERC spreads the bond pair more evenly ($0.4270$ vs $0.3660$, ratio $1.17$) than HRP ($0.4320$ vs $0.3174$, ratio $1.36$) - because ERC equalizes the two bonds’ risk contributions while inverse-variance does not. The largest single-asset difference is $\max |\Delta w|=0.0486$ on BD2. The “risk measure” choice moves weights by up to ~5 percentage points.
• The average-linkage heights are cleanly separated ($0.316$, $0.387$, $0.400$ within blocks; $0.524$, $0.559$, $0.652$, $0.707$ across), which is why the $K=3$ cut is unambiguous here and why this universe is linkage-insensitive.

---

4. Failure Modes & First-Principles Breakdowns

1. HERC’s $K$ is another hyperparameter. Choosing $K$ by an economic story is convenient but unvalidated; choosing it from the dendrogram (gap in merge heights) is better; validating it out-of-sample is best. Too-large $K$ degenerates toward HRP, too-small $K$ toward block-level risk parity.
2. ERC sub-solvers can stall in the worst regimes. ERC uses a fixed-point/coordinate-descent iteration (the $O(\text{sweeps}\cdot N)$ loop above); on nearly singular sub-blocks it converges slowly or needs a damping/regularization term. Always check convergence (risk contributions equal to tolerance) before trusting the output.
3. NCO reintroduces the estimation problem it was meant to dodge. Inverting small sub-matrices is far safer than the full $\Sigma$, but it is still an inversion: if a cluster is itself ill-conditioned, NCO’s within-cluster solve can be as unstable as MVO. Keep clusters small and add shrinkage to the sub-blocks.
4. NCO needs forecasts, which are the hardest input. Adding $\mu$ buys return-awareness at the cost of the single most error-prone input (Chopra–Ziemba: mean errors dominate covariance errors $\sim 20\times$). NCO is only as good as its $\mu$; use it with shrunk/Bayesian forecasts.
5. Variants can overfit the clustering. Every extra choice (linkage, $K$, distance, window, risk measure) is a degree of freedom you can tune on a backtest. Where HRP’s power is its near-parameter-freeness, HERC/NCO trade some of that away - hold them to a higher out-of-sample bar.
6. All variants share the tree’s failure modes. Clustering instability, non-stationarity, and noisy $\hat{\rho }$ (§05) propagate unchanged. Improving the objective does not fix the input.

---

5. References

• López de Prado, M. (2018). Advances in Financial Machine Learning, Wiley
• López de Prado, M. (2016). “Building Diversified Portfolios that Outperform Out of Sample.” J. Portfolio Management 42(4):59–69
• Raffinot, T. (2017/18). “Hierarchical Clustering-Based Asset Allocation.” J. Portfolio Management 44(2):89–99
• Maillard, Roncalli & Teïletche (2010). “The Properties of Equally Weighted Risk Contribution Portfolios.” J. Portfolio Management 36(4):60–70
• Hastie, Tibshirani & Friedman (2009). The Elements of Statistical Learning
• Ledoit & Wolf (2004). “A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices.” J. Multivariate Analysis 88(2):365–411

---

6. Connected Graph Bridges

• Back: 05 · Failure Modes · Index Hub
• Sibling topics: Risk Parity & ERC (HERC’s inner solver) · Covariance Shrinkage & RMT Denoising (the denoised input) · Mean–Variance (NCO’s inner objective) · Black–Litterman (views for NCO’s $\mu$)
• Cross-pillar: Machine Learning & Alt-Data · Quantitative Research