M.1 Linear Algebra & Matrices

Basic Prerequisites: High-school algebra and introductory vector geometry. This node is the entry point of the Foundations - every other foundations node (Calculus & Optimization, Probability & Measure Theory, Econometrics & Time Series, Stochastic Calculus) assumes it, and every factor model, covariance-denoiser, and mean-variance optimizer in the operational pillars is built on top of it. (these are the folder-level prerequisites for pages 02–06; page 01 states its own, smaller, entry requirements)

1. Intuition & Practical Objective

Assets rarely move in isolation. A book of 500 equities, a multi-currency yield curve, or a cross-market order book is a high-dimensional coordinate system, and linear algebra is the language in which you rotate (change basis), project (extract the part of a return you can explain), compress (keep only the dominant risk dimensions), and denoise (separate signal eigenvalues from noise) that system. Its claim is sharp: almost every quantitative object you will touch - a covariance matrix, a beta, a factor loading, a principal component - is a linear-algebra object, and every failure (singular optimizers, Cholesky crashes, $N>T$ instability) is a linear-algebra failure first.

This page is a hub: it (a) gives the fast decomposition-and-fact lookup below (job #1), and (b) routes you to six sub-pages that walk from raw intuition through vector spaces, linear systems, eigenvalues & covariance, SVD/PCA/regression, and finally random-matrix and numerical extensions.

The one-sentence essence. “Every symmetric matrix can be diagonalized (spectral theorem) - a covariance matrix is a sum of uncorrelated risk factors ${\lambda }_i{q}_i{q}_i^{\prime }$ - and every matrix can be decomposed (SVD) into its three essential actions: rotate, scale, rotate - the engine behind PCA, least squares, and covariance denoising.”

2. Mathematical Ground Truth & Derivations

Quick-Reference Lookup (job #1). All formulas below are transcribed from ESL (2nd ed.) Ch 3 & 14, Tsay (2nd ed.) §8–9, and Glasserman Ch 2–3, and the numbers in the check column were re-executed and reproduced exactly by from-scratch code (see §3).

Notation: $X\in {\mathbb{R}}^{T\times N}$ returns/data ($T$ observations, $N$ assets), $\Sigma$ covariance, $w$ portfolio weights, $q_i$ eigenvector, ${\lambda }_i$ eigenvalue, ${\sigma }_i$ singular value, $S=U\Sigma V^{\prime }$ SVD, ${\kappa }_2(A)={\sigma }_{\max }/{\sigma }_{\min }$ the 2-norm condition number.

Decomposition dictionary (one column = what exists when + why you reach for it):

Key facts & formulas:

Scaling/interpretation caveat (common lookup error). PCA loadings are only identified up to sign and the eigen/singular vectors of a covariance built on correlations (Tsay: eigenvalues sum to $k$) differ from those of the variance–covariance matrix (eigenvalues sum to total variance). Always state which matrix you diagonalized; a “market factor” read off the correlation matrix is a standardized object.

3. Computational Implementation - the decomposition engine

This runs on the standard library only - the SVD is built from scratch by diagonalizing $X^{\prime }X$ (Jacobi rotations) and the eigen-decomposition by the same Jacobi routine, so nothing depends on numpy/scipy. It reproduces the verified numbers above.

4. Failure Modes & First-Principles Breakdowns

Hub signposts - the folder’s failure-mode analysis lives in 06 · Advanced Extensions (RMT & Numerics). In one line each:

1. Singular / non-PSD covariance - a singular matrix has a zero eigenvalue; a covariance must be PSD. When the sample matrix isn’t, optimizers diverge and Cholesky throws. Root cause is usually $N>T$ or pairwise-correlation construction.
2. Ill-conditioning - a large ${\kappa }_2(A)$ means tiny input perturbations blow up in the output; solving with a near-singular normal-equation matrix silently destroys accuracy.
3. The $N>T$ trap - sample covariance rank $\le T$; at least $N-T$ eigenvalues are exactly zero (noise), so naive eigen-risk is nonsense.

5. References

• Hastie, Tibshirani, Friedman: The Elements of Statistical Learning (2nd ed., 2009)
• Tsay, Ruey S.: Analysis of Financial Time Series (2nd ed.)
• Glasserman, Paul: Monte Carlo Methods in Financial Engineering
• Strang, Gilbert: Introduction to Linear Algebra (5th ed., 2016)
• Horn & Johnson: Matrix Analysis (2nd ed., 2013)

6. Connected Graph Bridges

• Foundational base: self-contained entry node; feeds Multivariable Calculus & Optimization · Probability & Measure Theory · Econometrics & Time Series · Stochastic Calculus
• Sibling toolbox node: Numerical Methods (05 · Numerical Linear Algebra is this folder’s computational shadow)
• Sub-pages (in-folder): 01 From Zero · 02 Vectors, Spaces & Matrices · 03 Linear Systems & Decompositions · 04 Eigenvalues & Covariance · 05 SVD, PCA & Regression · 06 Advanced Extensions (RMT & Numerics)

Beginner: start at 01 · Practitioner: start at 05