M.3.1 Probability & Measure Theory from Zero

Basic Prerequisites: Elementary set theory and arithmetic.

---

1. Intuition & Practical Objective

This page builds the why of probability with no prior probability theory needed. The objective is one idea: probability is a way of assigning “how much of the space” a set of outcomes takes up, and the tools that make finance rigorous - expectations, conditioning, martingales - are all just disciplined counting/summing over sets.

Start with the dumbest question: why does the coin you flip forever not give each outcome a probability? Because there are uncountably many infinite sequences of Heads/Tails (${\Omega }_{\infty }$ is uncountable, Shreve II §1.1). You cannot give positive mass to a single path in a continuous space (a stock price path, a return in continuous time) - the only sensible questions are about sets of paths (“does the price stay above $90$?”), and you measure those sets. That single shift - from counting outcomes to measuring sets - is the whole subject in one move.

Four “aha”s:

1. Probability is a measure of sets. A probability space $(\Omega ,\mathcal{F},\mathbb{P})$ assigns a number $\mathbb{P}(A)\in [0,1]$ to every event $A$ in a family $\mathcal{F}$ that is closed under complements and countable unions (a $\sigma$-algebra). Countable additivity - $\mathbb{P}(\cup A_i)=\sum \mathbb{P}(A_i)$ for disjoint $A_i$ - is the “mass doesn’t disappear” rule.
2. Expectation is a weighted average that the Law of Large Numbers makes real. If $X$ is a payoff, $\mathbb{E}[X]$ is its probability-weighted average; the average of many iid draws converges to it (SLLN), and their error is $O(1/\sqrt{n})$ (CLT). That is the entire engine of Monte Carlo pricing (Glasserman Ch 1).
3. Conditional expectation is “best prediction given information.” $\mathbb{E}[X\mid \mathcal{G}]$ is the $\mathcal{G}$-measurable random variable closest to $X$ in mean-square (the regression function $f(x)=\mathbb{E}[Y\mid X=x]$ of ESL Ch 2) - not a guess, a projection.
4. A martingale is a fair game. If the expected next value given all history equals the present value, $\mathbb{E}[M_{t+1}\mid {\mathcal{F}}_t]=M_t$, there is no edge to exploit - and “prices under the risk-neutral measure are martingales” is how no-arbitrage pricing is stated.

---

2. Mathematical Ground Truth & Derivations

The probability space (Shreve II Def 1.1.1–1.1.2). A $\sigma$-algebra $\mathcal{F}$ on nonempty $\Omega$ contains $\varnothing$, is closed under complements and countable unions (hence countable intersections and $\Omega \in \mathcal{F}$). A probability measure is $\mathbb{P}:\mathcal{F}\to [0,1]$ with $\mathbb{P}(\Omega )=1$ and countable additivity for disjoint events:

$$\mathbb{P}\left(\bigcup _n{A}_n\right)=\sum _n\mathbb{P}(A_n)\Rightarrow \mathbb{P}(A^c)=1-\mathbb{P}(A),\mathbb{P}(\varnothing )=0.$$

On a finite space one takes $\mathcal{F}=$ all subsets; on $[0,1]$ the uniform (Lebesgue) measure is defined first on intervals by $\mathbb{P}(a,b]=b-a$ and extended to Borel sets by countable additivity (Shreve II Ex 1.1.3).

Expectation = the “standard machine” (Shreve II §1.4–1.5). Integrals on uncountable spaces are built in four steps - indicator $\to$ simple $\to$ nonnegative $\to$ general - and computed via densities: $\mathbb{E}[h(X)]=\int h(x)f_X(x)dx$ (Shreve I Ch 11 Thm 3.32). The limit theorems - Monotone Convergence, Fatou, Dominated Convergence - justify passing limits through expectations (Shreve II Thms 1.4.5/1.4.9), which is exactly what makes “stop the martingale and let $t\to \infty$” legal in Ch 3.

The two theorems that run Monte Carlo (Glasserman §1.1). For iid $U_i$ with $\alpha =\mathbb{E}[f(U)]$:

$${\hat{\alpha }}_n=\frac{1}{n}\sum _{i=1}^{n}f(U_i)\stackrel{\text{a.s.}}{\to }\alpha \text{(SLLN)},{\hat{\alpha }}_n-\alpha \approx N(0,\frac{{\sigma }_f}{\sqrt{n}})\text{(CLT)}.$$

The standard error ${\sigma }_f/\sqrt{n}$ is independent of dimension - the raison d’être of MC in high-dimensional pricing (Glasserman §1.1; the error rate is $O(n^{-1/2})$ in any $d$).

Conditional expectation and martingales previewed. $\mathbb{E}[X\mid \mathcal{G}]$ is $\mathcal{G}$-measurable and satisfies partial averaging ${\int }_A\mathbb{E}[X\mid \mathcal{G}]d\mathbb{P}={\int }_{A}Xd\mathbb{P}$ ∀$A\in \mathcal{G}$ (Shreve II Def 2.3.1). A martingale is adapted + integrable with $\mathbb{E}[M_t\mid {\mathcal{F}}_s]=M_s$ (Shreve I §2.4). Both get full pages below (04 · Conditional Expectation, 05 · Martingales).

---

3. Computational Implementation - see the LLN and the CLT

Simulate a fair coin and watch the sample mean converge to $0.5$; then simulate many sample means of $k$ uniforms and check their spread and the 68–95–99.7 rule. Stdlib only.

---

4. Failure Modes & First-Principles Breakdowns

1. Counting outcomes instead of measuring sets. On an uncountable space, “probability of the path $S_t=100.000$” is $0$ for each path yet the union of paths has probability $1$. Assigning mass to outcomes, not sets, is exactly what $\sigma$-algebras prevent - the root motivation for the whole subject (Shreve II §1.1).
2. Treating $\mathbb{E}[\cdot ]$ as “the value it will take.” The LLN gives convergence in average, not a prediction for any single draw. A martingale can have huge variance and still be perfectly “fair” - see 05 · Martingales.
3. Believing the sample mean is exact. The CLT error ${\sigma }_f/\sqrt{n}$ is probabilistic; halving the error needs $4\times$ samples, one extra decimal needs $100\times$. With no convergence theorem you cannot even justify the MC estimate (Glasserman §1.1).
4. Conditioning on zero-probability events via the schoolbook ratio. $\mathbb{P}(A\mid B)=\mathbb{P}(A\cap B)/\mathbb{P}(B)$ is undefined for $\mathbb{P}(B)=0$; only partial averaging defines conditioning there. See 04 · Conditional Expectation.

---

5. References

• Shreve, Stochastic Calculus for Finance II
• Shreve, Stochastic Calculus for Finance I
• Glasserman, Monte Carlo Methods in Financial Engineering

---

6. Connected Graph Bridges

• Base: none (this is the entry point) · Multivariable Calculus (expectations as integrals)
• Continue: 02 · Probability Spaces & Filtrations · Index Hub
• Forward: Stochastic Calculus · No-Arbitrage & Binomial Trees