M.2 Calculus & Optimization

Basic Prerequisites: Linear Algebra & Matrices (vectors, matrices, quadratic forms, eigenvalues). Note: this folder builds the multivariable/optimization machinery the linear-algebra page assumes is available. (these are the folder-level prerequisites for pages 02–06; page 01 states its own, smaller, entry requirements)

1. Intuition & Practical Objective

Quantitative finance is an optimization discipline dressed up as a probability discipline. Portfolio construction minimises variance subject to a return floor; a market maker’s quote minimises inventory risk subject to a spread the client will pay; a model is fit by minimising a loss; a parameter is calibrated by solving ${\nabla }_{\theta }\ell (\theta )=0$. Every one of those sentences is calculus plus constraints.

This folder is the calculus-and-optimization toolbox for the Atlas: the single-variable foundation, the multivariate apparatus (gradients, Jacobians, Hessians), the theory of constrained optima (Lagrange and KKT), the notion of convexity that decides whether a stationary point is the answer, and the iterative algorithms (gradient descent, Newton) that actually find the answer when no closed form exists.

Everything reduces to five primitive ideas:

1. A derivative is a local linear map → the best linear approximation of a function at a point. → Single-Variable Calculus.
2. The gradient stacks the partials; the Hessian stacks the second-order curvature → the multivariable Taylor model. → Multivariable Calculus.
3. Constraint borders bend the first-order condition → the Lagrangian multiplier is the shadow price. → Constrained Optimization.
4. Convexity turns “stationary” into “globally optimal” → why the practitioner’s life is easy (or not). → Convexity & Applications.
5. When no closed form exists, iterate → gradient descent’s linear rate, Newton’s quadratic rate. → Gradient & Newton Methods.

This page is the hub: it gives the fast formula and condition lookup below, then routes you to six sub-pages that build each primitive from first principles, with working code and failure modes.

The one-sentence essence. “Differentiate to get a local linear model, set the (constrained) derivative to zero to characterise an optimum, and use convexity to know whether that optimum is the global one - every finance optimisation is this sentence in a different costume.”

2. Mathematical Ground Truth & Derivations

Lookup 1 - the calculus dictionary, single-variable → several variables → finance. Notation: $f:{\mathbb{R}}^n\to \mathbb{R}$ is $C^2$ unless stated; $\nabla f$ the gradient (column of partials), ${\nabla }^{2}f$ the Hessian, $J$ the Jacobian of a vector map.

Lookup 2 - optimality conditions (unconstrained and constrained). The whole of static optimisation is this table (Simon & Blume Ch 17–19; Boyd §5.5.3).

The KKT conditions (Boyd eq. 5.49; Simon & Blume §18.6). For $\min f_0(x)$ s.t. $f_i(x)\le 0,h_j(x)=0$:

The four blocks are primal feasibility · dual feasibility · complementary slackness · stationarity. Dropping the inequalities leaves exactly the Lagrange system. Under convexity (and Slater), KKT is necessary and sufficient (Boyd §5.5.3).

The multiplier is a shadow price (Simon & Blume §19.1–19.2, envelope theorem). Perturb the constraint to $h(x)=b$; then $\frac{d{f}^{\ast }}{db}={\nu }^{\ast }$ at the optimum. In portfolio terms, ${\lambda }^{\ast }$ on "${\mu }^{\top }w=r_0$" is the marginal variance bought per unit of extra required return.

Lookup 3 - convexity decides everything. A set $C$ is convex if $x,y\in C\Rightarrow \theta x+(1-\theta )y\in C$. $f$ is convex if $f(\theta x+(1-\theta )y)\le \theta f(x)+(1-\theta )f(y)$; for $C^2$ functions this is equivalent to ${\nabla }^{2}f⪰0$ everywhere. Convex objective $+$ convex feasible set $\Rightarrow$ every stationary point is a global minimum - the reason Markowitz, regression and LP are solvable while “pick 10 stocks” is not. Strong convexity sharpens the geometry: ${\nabla }^{2}f(x)⪰mI$ (Boyd eq. 9.7).

Lookup 4 - the two workhorse algorithms.

The condition number $M/m=\kappa$ is the whole story: gradient descent needs $\sim \kappa \log (1/ϵ)$ iterations (Boyd eq. 9.19), Newton is affine-invariant and independent of $\kappa$ (Newton decrement $\lambda (x{)}^2=\nabla f^{\top }{\nabla }^2{f}^{-1}\nabla f$ measures suboptimality as $f(x)-p^{\ast }\approx \frac{1}{2}{\lambda }^2$).

3. Computational Implementation - one screen, all five primitives

Stdlib only. It differentiates, expands, and optimises - the three things this folder actually does.

(The KKT solve reproduces the full worked system of 04 · Constrained Optimization: ${\lambda }_1=0.288000$ equals the numeric $d(\frac{1}{2}{w}^{\top }Sw)/d{r}_0$.)

4. Failure Modes & First-Principles Breakdowns

Hub signposts - the full first-principles analysis lives on the sub-pages. In one line each:

1. "$\nabla f=0$" is only necessary, not sufficient. A saddle point satisfies the first-order condition; without a sign check on ${\nabla }^{2}f$ you can report a maximum as a minimum (page 03).
2. Constrained optima need a constraint qualification. If the constraint gradients are linearly dependent at $x^{\ast }$, the Lagrange/KKT multipliers may fail to exist and the first-order system is misleading (Simon & Blume §19.5).
3. Non-convexity makes the solver’s answer a local one. Adding a cardinality cap (“hold at most 10 names”) or fixed costs destroys convexity; gradient methods then report a stationary point that depends on where they started (page 06).
4. Ill-conditioning kills first-order methods and amplifies second-order ones. Gradient descent needs $\sim \kappa \log (1/ϵ)$ iterations; Newton’s step solves a system with $\kappa ({\nabla }^{2}f)$ and loses digits to round-off (pages 05, 06).
5. Finite differences have a round-off floor. The centred derivative of $x^3$ at $2$ reaches $\sim 2\times 10^{-10}$ accuracy at $h\approx 10^{-5}$ then degrades back to $\sim 10^{-6}$ at $h=10^{-10}$ - you cannot out-shrink floating point (page 02).

5. References

• Simon, Carl P. & Blume, Lawrence: Mathematics for Economists (W. W. Norton, 1994)
• Boyd, Stephen & Vandenberghe, Lieven: Convex Optimization (Cambridge University Press, 2004)
• Spivak, Michael: Calculus (4th ed.)
• Stewart, James, Clegg, Daniel & Watson, Saleem: Calculus: Early Transcendentals (9th ed., 2020)
• Hastie, Tibshirani & Friedman: The Elements of Statistical Learning (2nd ed., 2009)
• Tsay, Ruey S.: Analysis of Financial Time Series (3rd ed.)
• Bernstein, D. J.: Calculus for Mathematicians (1997 draft) - internal foundation note. Scope caveat: this is a compact, proof-based single-variable only text (continuity, Carathéodory derivatives, completeness, MVT, Kurzweil–Henstock integration, limits/L’Hôpital). It contains no multivariable calculus, no Taylor/power series, and no optimisation beyond Fermat’s interior-extremum principle. It underwrites page 02 only; pages 03–06 are built from Simon & Blume, Boyd and Stewart.

6. Connected Graph Bridges

• Foundational base: Linear Algebra & Matrices (quadratic forms, eigenvalues/definiteness) · Probability & Measure Theory (expectations as integrals)
• Applied destination - derivative pricing: The Greeks & Dynamic Hedging (the Greeks are the partials of $V$), BSM · PDE & Derivation (chain rule/Itô).
• Applied destination - optimising under constraints: Mean–Variance Optimization (the QP of page 04) · Almgren–Chriss Optimal Execution.
• Applied destination - model fitting: Machine Learning (gradient descent, IRLS), Numerical Optimization.
• Related foundation folder: Linear Algebra · Numerical Methods · Econometrics & Time Series
• Sub-pages (in-folder): 01 From Zero · 02 Single-Variable Calculus · 03 Multivariable Calculus · 04 Constrained Optimization · 05 Gradient & Newton · 06 Convexity & Applications

Beginner: start at 01 · Practitioner: start at 05