1.5.3 The Kalman Filter

Basic Prerequisites: 02 · State-Space Models.

---

1. Intuition & Practical Objective

The Kalman filter is the predictor–corrector loop for a Gaussian state-space model. Each cycle does two things and only two things:

1. Predict - push the current belief about the state forward through the transition dynamics, inflating its uncertainty (the state may have moved, and we weren’t looking).
2. Correct - when a new observation arrives, form the innovation (how surprising the observation is given what we predicted), and move the belief toward the data by a fraction - the Kalman gain - that is exactly the signal-to-noise ratio at that moment.

Two properties make it the workhorse of quantitative finance:

• It is recursive and online. It never re-runs the whole history; it stores only the current state estimate and its covariance. This makes it $O(m^3)$ per step regardless of how long the series is, and directly deployable in a live system.
• It is optimal. Under linearity and Gaussianity it is the minimum mean-squared-error estimator - no other estimator (linear or nonlinear) can do better. When the assumptions fail, it is still the best linear estimator (the orthogonal-projection view).

The practical objective is to give the practitioner the recursion in a form they can drop into code, with the conventions pinned down and the numerics made stable.

---

2. Mathematical Ground Truth & Derivations

2.1 The recursion, step by step

Given $(s_{t\mid t-1},{\Sigma }_{t\mid t-1})$ (the a-priori belief at time $t$), the filter processes $y_t$ and produces $(s_{t+1\mid t},{\Sigma }_{t+1\mid t})$:

$$\text{Innovation:}{v}_t=y_t-c_t-Z_t{s}_{t\mid t-1}$$ $$\text{Innovation covariance:}{V}_t=Z_t{\Sigma }_{t\mid t-1}{Z}_t^{\top }+H_t$$ $$\text{Kalman gain:}{K}_t=T_t{\Sigma }_{t\mid t-1}{Z}_t^{\top }{V}_t^{-1}$$ $$\text{State update:}{s}_{t+1\mid t}=d_t+T_t{s}_{t\mid t-1}+K_t{v}_t$$ $$\text{Covariance update:}{\Sigma }_{t+1\mid t}=T_t{\Sigma }_{t\mid t-1}{L}_t^{\top }+R_t{Q}_t{R}_t^{\top },L_t=T_t-K_t{Z}_t.$$

(Tsay Eq. 11.64, verified in the corpus.) The gain is a matrix version of the scalar weight $P^{-}/(P^{-}+r)$: $Z_t\Sigma Z_t^{\top }$ is the predicted observation variance (signal), $V_t$ adds the measurement variance (signal + noise), and $T_t\Sigma Z_t^{\top }$ maps it back to state space. When the observation is uninformative ($V_t$ large), $K_t\to 0$ and the state just follows its prediction.

2.2 Where the gain comes from (three equivalent routes)

• Bayes / Gaussian conditioning. The joint of $(s_{t+1},y_t)$ is Gaussian; conditioning on $y_t$ gives a posterior mean $=\text{prediction}+K\cdot \text{innovation}$ and covariance $=(I-KZ){\Sigma }^{-}$ with $K={\Sigma }^{-}{Z}^{\top }(Z{\Sigma }^{-}{Z}^{\top }+H{)}^{-1}$. Algebra, not magic.
• Orthogonal projection (least squares). Among all linear functions of the data, the minimum-MSE state estimate is the orthogonal projection onto the observation space; solving the normal equations reproduces exactly the same $K$. This is why the filter remains the best linear estimator even when the model is non-Gaussian.
• Weighted average of two beliefs. The predict step and the observation are two Gaussian “estimates” of the same quantity; the posterior is their variance-weighted average. $K$ is the weight on the data.

2.3 The scalar local-level case (Tsay 11.14), for cross-checking

With $T=Z=R=1$, $d=c=0$, $Q={\sigma }_{\eta }^2$, $H={\sigma }_e^2$:

$$v_t=y_t-{\mu }_{t\mid t-1},V_t={\Sigma }_{t\mid t-1}+{\sigma }_e^2,K_t=\frac{{\Sigma }_{t\mid t-1}}{V_t},{\mu }_{t+1\mid t}={\mu }_{t\mid t-1}+K_t{v}_t,{\Sigma }_{t+1\mid t}={\Sigma }_{t\mid t-1}(1-K_t)+{\sigma }_{\eta }^2.$$

This is the form to verify a matrix implementation against - it is small enough to compute by hand.

2.4 Steady state and the Riccati recursion

For a time-invariant model, the covariance recursion converges: ${\Sigma }_t\to {\Sigma }_{\infty }$ solving the algebraic Riccati equation, after which $K_t\to K_{\infty }$ is a constant. In steady state the filter is a fixed-coefficient linear filter, and this is the origin of the famous result that the optimal filter satisfies a Riccati (not a Wiener–Hopf) equation. Practically: you can pre-compute $K_{\infty }$ and run a gloriously cheap constant-gain filter - at the cost of losing adaptivity.

2.5 Maximum likelihood for free

The prediction-error decomposition (Tsay Eq. 11.25) gives the exact log-likelihood of the observations as a by-product of the recursion:

$$\ln L=-\frac{T}{2}\ln (2\pi )-\frac{1}{2}\sum _t[\ln V_t+\frac{v_t^2}{V_t}].$$

So parameters $({\sigma }_e^2,{\sigma }_{\eta }^2,\dots )$ are estimated by simply maximizing this - no external likelihood machinery, and the innovations $v_t$ double as a model-diagnostic series (page 05).

---

3. Computational Implementation - the filter as a function, verified two ways

The general matrix filter is implemented exactly as §2.1. It is then run on a local-level model and its output is checked coefficient-by-coefficient against the scalar recursion - the strongest possible test that the matrix code is correct. Finally the Riccati fixed point is computed and compared to the filter’s converged gain.

The matrix implementation matches the hand-checkable scalar recursion to $3.5\times 10^{-12}$ (the residual is the diffuse initial $P_0=10^7$; the two recursions are algebraically identical), and its converged gain $K_{\infty }=0.417891$ equals the algebraic-Riccati fixed point computed independently - two implementations, one number.

---

4. Failure Modes & First-Principles Breakdowns

1. Covariance blow-up / loss of symmetry. The update $(I-KZ)\Sigma$ is only mathematically symmetric; floating-point round-off can drift it into asymmetry and even negative variance, after which the filter oscillates or diverges. Fix: use the Joseph form $\Sigma =(I-KZ){\Sigma }^{-}(I-KZ{)}^{\top }+KH{K}^{\top }$, which is symmetric and positive-semi-definite by construction, or use a square-root (Cholesky) implementation.
2. Numerically singular $V_t$. $V_t^{-1}$ is computed every step; if a regressor is nearly collinear with another, $V_t$ becomes ill-conditioned and $K_t$ explodes. In practice solve the linear system rather than inverting, and consider a small jitter on $H$.
3. Diffuse initialization. A genuinely unknown initial state needs $P_0\to \infty$ (S-Plus uses an mSigma = −1 sentinel). Using a large finite $P_0$ instead leaves a transient bias; using a large inverse is numerically disastrous. Tsay §11.1.6 treats this explicitly.
4. Mixing the two recursion forms. The canonical (11.64) form uses $L_t=T_t-K_t{Z}_t$ and puts the $T_t$ factor in $K_t$; the contemporaneous form updates $\Sigma$ as $T{\Sigma }_{t\mid t}{T}^{\top }+RQ{R}^{\top }$. Both are correct internally, but splicing a line from one into the other silently off-by-ones the covariance (flagged in the corpus verification of Tsay Ch 11).

---

5. References

• Tsay, Analysis of Financial Time Series (3rd ed.)
• Durbin & Koopman, Time Series Analysis by State Space Methods
• Welch & Bishop, An Introduction to the Kalman Filter (TR 95-041)
• Särkkä, Bayesian Filtering and Smoothing
• Hamilton, Time Series Analysis

---

6. Connected Graph Bridges

• Back: 02 · State-Space Models
• Continue: 04 · Time-Varying Beta · Index Hub
• Base: Linear Algebra (matrix inverse, SPD) · Probability (Gaussian conditioning)
• Robustness: 05 · Failure Modes · 06 · Smoothing & Particle Filters