1.8.4 Hidden Markov Models

Basic Prerequisites: 02 · Markov-Switching Models and Probability & Measure Theory (Bayes, conditioning).

---

1. Intuition & Practical Objective

A Hidden Markov Model (HMM) is the formal, machine-learning-side statement of Hamilton’s Markov-switching model (the two are the same object; the traditions differ in emphasis). An HMM has three parts:

1. Hidden states $s_t\in \{1,\dots ,K\}$ following a first-order Markov chain with transition matrix $P$.
2. Emission densities $f(y_t\mid s_t=j)$ - e.g., Gaussian with regime mean ${\mu }_j$ and vol ${\sigma }_j$ (this is the HMM’s connection to Gaussian Mixture Models / GMM when the time-dependence is removed).
3. The initial distribution $\pi$.

You observe only $y_{1:T}$ (returns). The three canonical inference problems, each with a clean algorithm:

• Filtering / smoothing - $\mathbb{P}[s_t\mid y_{1:T}]$ for each $t$ → the forward–backward algorithm.
• Decoding - the single most-likely state sequence $\arg {\max }_{s_{1:T}}\mathbb{P}[s_{1:T}\mid y_{1:T}]$ → Viterbi.
• Learning - estimate $(P,\mu ,\sigma ,\pi )$ from data → EM / Baum–Welch (exactly the EM of page 02, formalized).

The practical objective: an HMM is the machinery you use when the regime structure is genuinely latent and you want smoothing (best estimate of past regimes), decoding (a dating of the full regime history), and clean parameter learning all from one estimable object.

---

2. Mathematical Ground Truth & Derivations

Forward pass (filtering). Define ${\alpha }_t(j)=f(y_{1:t},s_t=j)$, the joint probability of the data up to $t$ and being in state $j$ at $t$. It satisfies the recursion (with scaling factor $c_t$ for numerical stability):

$${\alpha }_1(j)={\pi }_{j}f(y_1\mid j),{\alpha }_t(j)=f(y_t\mid j)\sum _i{\alpha }_{t-1}(i)P_{ij}.$$

The scaled version stores ${\hat{\alpha }}_t(j)={\alpha }_t(j)/c_t$ with $c_t={\sum }_j{\alpha }_t(j)$; then ${\sum }_t\ln c_t$ is the log-likelihood.

Backward pass (smoothing). Define ${\beta }_t(j)=f(y_{t+1:T}\mid s_t=j)$, computed backward:

$${\beta }_T(j)=1,{\beta }_t(j)=\sum _k{P}_{jk}f(y_{t+1}\mid k){\beta }_{t+1}(k).$$

Smoothing - the marginal posterior probability of state $j$ at time $t$ given all data:

$${\gamma }_t(j)=\mathbb{P}[s_t=j\mid y_{1:T}]\propto {\alpha }_t(j){\beta }_t(j),$$

normalized over $j$. This is the “smoothed regime probability” - the best answer to “which regime were we in?”

Decoding (Viterbi). Instead of the marginal, find the joint-MAP path via dynamic programming:

$${\delta }_1(j)={\pi }_{j}f(y_1\mid j),{\delta }_t(j)=f(y_t\mid j)\underset{i}{\max }[{\delta }_{t-1}(i)P_{ij}],$$

recording the argmax ${\psi }_t(j)$ at each step and backtracking from $\arg {\max }_j{\delta }_T(j)$.

Learning (Baum–Welch EM). The E-step computes responsibilities ${\gamma }_t(j)$ and pairwise transition responsibilities ${\xi }_t(i,j)\propto {\alpha }_t(i)P_{ij}f(y_{t+1}\mid j){\beta }_{t+1}(j)$; the M-step re-estimates:

$${\mu }_j=\frac{{\sum }_t{\gamma }_t(j)y_t}{{\sum }_t{\gamma }_t(j)},{\sigma }_j^2=\frac{{\sum }_t{\gamma }_t(j)(y_t-{\mu }_j{)}^2}{{\sum }_t{\gamma }_t(j)},P_{ij}=\frac{{\sum }_{t<T}{\xi }_t(i,j)}{{\sum }_{t<T}{\gamma }_t(i)},{\pi }_j={\gamma }_1(j).$$

Each EM iteration is guaranteed not to decrease the likelihood (ESL Ch 8.5’s EM treatment; Tsay Ch 12’s MCMC approach is the Bayesian alternative).

---

3. Computational Implementation - smooth, decode, and reconstruct an HMM

Generate a 2-state Gaussian HMM, run forward–backward smoothing, Viterbi decoding, and compare both to the planted states. Stdlib only.

Smoothing (88.6%) beats Viterbi (77.8%) - a genuinely important fact: using all the data (past and future) to estimate each regime is more reliable than committing to a single joint-MAP path. The smoothed probabilities are high and stable during the (truly bullish) opening stretch, showing the smoothing knows the early data was bull with near-certainty.

---

4. Failure Modes & First-Principles Breakdowns

1. Smoothing vs filtering vs decoding are different questions. Filtering ($\mathbb{P}[s_t\mid y_{1:t}]$, causal) is what a trading decision can use; smoothing ($\mathbb{P}[s_t\mid y_{1:T}]$, non-causal) is better for dating regimes but peeks into the future. Viterbi gives the joint-MAP path, not the marginal probabilities - and (as shown) is less accurate than smoothing. Use each for its correct job.
2. Overfitting via too many states. EM only raises the likelihood; a 3-state HMM on 2-state data splits a real state into two clones. Use BIC - see 05 · Failure Modes.
3. Label switching / non-identifiability. Permuting state labels leaves the likelihood identical; EM may converge to a swapped labeling. Fix an ordering constraint on a parameter (e.g. ${\mu }_1>{\mu }_0$).
4. Scale/numerics. Unscaled forward probabilities underflow exponentially in $T$; always use the scaled ($c_t$) version - the code above scales both passes.

---

5. References

• Rabiner, Lawrence R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proc. IEEE 77(2), 257–286 (1989)
• Bishop, Christopher M.: Pattern Recognition and Machine Learning
• ESL (Hastie, Tibshirani & Friedman):
• Tsay, Analysis of Financial Time Series
• Hamilton (1989)

---

6. Connected Graph Bridges

• Back: 02 · Markov-Switching · Index Hub
• Forward: 05 · Failure Modes · 06 · Advanced Extensions
• ML cross-link: Regime Classification: HMM & GMM (Baum–Welch + Viterbi in the ML pillar)
• Contrast: 03 · Threshold Models (observed-state regimes vs latent-state HMM)