Measure Theory

Why naive length breaks as a probability definition

Setup: Throw a perfectly random dart at the interval $[0,1]$, and call the landing point $X$. Because no region is favored over any other, the probability of landing in an interval $[a,b]$ is simply its length:

$$P(X \in [a,b]) = b - a \qquad (0 \le a \le b \le 1)$$

If we add the rule that probabilities of non-overlapping pieces add up, we can measure any event made of finitely many intervals:

$$P\big(X \in [0, 0.1] \cup [0.5, 0.7]\big) = 0.1 + 0.2 = 0.3$$

This can handle most of the basic questions we might have about the dart. But, the machinery breaks the moment we ask a seemingly simple question about a set that isn’t a finite collection of intervals:

$$\text{What is } P(X \text{ is a rational number})?$$

Countable vs. uncountable infinity

At first glance, it feels like $P(X \in \mathbb{Q})$ should be greater than $0$. The fractions $p/q$ are dense in $[0,1]$, meaning mathematically that for any real numbers $a, b$ with $a < b$, there exists some $q \in \mathbb{Q}$ such that $a < q < b$, and nearly every number we write down in daily life is rational.

However, density is a statement about arrangement, not about quantity. We are talking about two different sizes of infinity here:

1. The rationals $\mathbb{Q}$ are countable (listable): Even though there are infinitely many fractions in $[0,1]$, we can line them up in a single sequence $q_1, q_2, q_3, \dots$ without missing any. Simply order them by their denominator, and then by their numerator (skipping duplicates like $2/4$): $$\frac{0}{1}, \frac{1}{1}, \frac{1}{2}, \frac{1}{3}, \frac{2}{3}, \frac{1}{4}, \frac{3}{4}, \frac{1}{5}, \frac{2}{5}, \frac{3}{5}, \frac{4}{5}, \dots$$ Every rational number has a specific index $n$ where it eventually appears in this list.

   import math
   
   def get_nth_rational(n: int) -> str:
       """Returns the n-th rational in 0/1, 1/1, 1/2, 1/3, 2/3, 1/4, 3/4, ..."""
       if n <= 0:
           raise ValueError("n must be a positive integer (1-indexed).")
       if n == 1:
           return "0/1"
       if n == 2:
           return "1/1"
   
       remaining = n - 2  # the first two terms (d = 1) are handled above
       d = 2
       while True:
           # at denominator d, the valid numerators are the k < d coprime to d
           # this is because for coprimes we can't cancel a common factor in both numerator and denominator
           coprimes = [k for k in range(1, d) if math.gcd(k, d) == 1]
           if remaining <= len(coprimes):
               return f"{coprimes[remaining - 1]}/{d}"
           remaining -= len(coprimes)  # skip this denominator's block, keep counting
           d += 1

2. The full interval $[0,1]$ is uncountable (unlistable): By Cantor’s diagonal argument, no sequential list can ever hold all real numbers in $[0,1]$.

ProofCantor’s Diagonal Argument: why $[0,1]$ cannot be listed

Suppose someone claims to have a complete list $r_1, r_2, r_3, \dots$ of every real number in $[0,1]$, written as infinite decimals:

• $r_1 = 0.\mathbf{5}1492\dots$
• $r_2 = 0.3\mathbf{8}017\dots$
• $r_3 = 0.92\mathbf{5}61\dots$

We can build a brand-new number $x \in [0,1]$ by walking down the diagonal (the $n$-th digit of the $n$-th number) and picking a different digit at each step, say, pick $4$ unless the diagonal digit is $4$, in which case pick $5$.

• For the list above, $x$ starts $0.\mathbf{444}\dots$

By construction, $x$ differs from $r_1$ in the 1st decimal place, from $r_2$ in the 2nd decimal place, and from $r_n$ in the $n$-th decimal place. Thus $x$ is not on the list, contradicting the claim that the list was complete.

∎

Because the entire interval $[0,1]$ cannot be listed, while the rationals can, the irrationals (like $\pi/4$ or $\sqrt{2}/2$) must make up almost the entirety of the line. The rationals are merely a countable dusting of points inside an uncountable continuum, so $P(X \in \mathbb{Q})$ ought to be $0$.

Two failed attempts using finite intervals

When we try to prove $P(X \in \mathbb{Q}) = 0$ using only finite intervals, we hit a wall from both directions:

Approach	How it works	Why it fails
1. Point-by-point (from inside)	A single point $\{q\}$ is the degenerate interval $[q,q]$ of length $q - q = 0$. Since $\mathbb{Q} \cap [0,1]$ is the list $\{q_1, q_2, \dots\}$, we add up the lengths: $0 + 0 + 0 + \cdots = 0$.	Our rules only allow adding finitely many pieces. Adding across an infinite sequence requires a new axiom (countable additivity), which we haven’t justified yet.
2. Trapping in boxes (from outside)	Cover all rationals in $[0,1]$ using a finite collection of intervals, and sum their lengths to get an upper bound.	Because rationals sit inside every gap of positive width, any finite set of intervals that covers every rational in $[0,1]$ must cover the entire segment $[0,1]$ (except at most finitely many single points). This only proves $P(X \in \mathbb{Q}) \le 1$, which is useless.

Converting probability to integration: why Riemann also fails

Since trying to trap the rationals in finite external boxes failed to give an answer, we need a more powerful tool which we usually use to “measure” complex sets. We can do this by turning a set into a shape and calculating its area via integration.

To build an intuition for how we measure complex, non-interval sets, consider a famous fractal: the Cantor set.

• Imagine starting with the interval $[0,1]$ and repeatedly removing the open middle third of every remaining piece infinitely. What is left behind is a “dust” of points, infinitely many of them, and we want to find their total length (probability of the dart landing inside the cantor set). If you try to use a finite collection of external boxes to cover all the remaining pieces as we tried above, any collection of finite boxes spanning from the first piece to the last will inevitably bridge the gaps, leaving you with a useless upper bound of $1$.
• The Formal Problem: Let $C_0 = [0,1]$. At each step $n$, $C_n$ is formed by removing the open middle third of each interval in $C_{n-1}$. The Cantor set is the intersection of all these stages: $C = \bigcap_{n=0}^{\infty} C_n$. We want to compute its total length (measure) $m(C)$.

ProofMeasuring the Cantor set via shrinking approximations

At step $n$, the set $C_n$ consists of $2^n$ disjoint closed intervals, each of length $\left(\frac{1}{3}\right)^n$.

The total length (measure) of $C_n$ is the sum of the lengths of these intervals: $$m(C_n) = 2^n \cdot \left(\frac{1}{3}\right)^n = \left(\frac{2}{3}\right)^n$$

Because the Cantor set $C$ is contained inside every approximation $C_n$ for all $n$, its measure must be less than or equal to $m(C_n)$. By taking the limit as $n \to \infty$: $$m(C) = \lim_{n \to \infty} \left(\frac{2}{3}\right)^n = 0$$

Even though the Cantor set contains an uncountable infinity of points (just as many as the entire real line), its total measure is strictly $0$ because we can trap it inside a sequence of outer approximations whose lengths shrink to zero.

∎

How this helps us solve the rationals problem: This exact strategy, approximating a complex set from the outside using an infinite sequence of shrinking pieces rather than a single finite collection, looks like it can be used to measure the rationals. Because the rationals are countable, we can place a tiny box of length $\frac{\varepsilon}{2^n}$ around each rational $q_n$ in our list. By summing these lengths across all infinitely many boxes, we get a total outer measure that can be made arbitrarily small, proving that $P(X \in \mathbb{Q}) = 0$.

To measure the probability of landing in a set $A$, we can define an indicator function $\mathbf{1}_A(x)$ that outputs $1$ if $x \in A$ and $0$ otherwise. For our question about the rationals, this creates the Dirichlet function: $$D(x) = \mathbf{1}_{\mathbb{Q}}(x) = \begin{cases} 1 & \text{if } x \text{ is rational} \\ 0 & \text{if } x \text{ is irrational} \end{cases}$$

The total area under this function from $0$ to $1$ represents the expected value, which is our exact probability: $$\mathbb{E}[D(X)] = 1 \cdot P(X \in \mathbb{Q}) + 0 \cdot P(X \notin \mathbb{Q}) = P(X \in \mathbb{Q}) = \int_0^1 D(x)\,dx$$

If standard calculus can evaluate $\int_0^1 D(x)\,dx$, our probability problem is solved. Let’s try using the standard Riemann integral:

1. Chop $[0,1]$ into finitely many narrow vertical strips.
2. On each strip, find the upper sum (tallest height $\times$ width) and lower sum (lowest height $\times$ width).
3. Squeeze the strip widths to zero to see if they converge to the same area.

For $D(x)$, this completely breaks down: every subinterval, no matter how microscopic, contains both a rational ($D=1$) and an irrational ($D=0$).

• Every upper sum evaluates to $1 \times (\text{total width}) = 1$.
• Every lower sum evaluates to $0 \times (\text{total width}) = 0$.

Because the Riemann integral relies on finitely many intervals at each step, it suffers from the exact same flaw as our earlier box-trapping approach: it cannot isolate a countable set of points from the continuum.

Why taking limits breaks the Riemann integral

You might try to bypass this wall by building the Dirichlet function through a sequence of approximations. Let $f_n(x) = \mathbf{1}_{\{q_1, \dots, q_n\}}(x)$ be a function that equals $1$ on just the first $n$ rationals in our countable list, and $0$ everywhere else.

This introduces a critical conflict when we try to pass limits through our integral:

1. Limit Outside ($\lim_{n \to \infty} \int_0^1 f_n(x)\,dx = 0$): Here, we integrate each finite approximation first. Because any finite set of $n$ rational points has total length zero, $\int_0^1 f_n(x)\,dx = 0$ for every fixed $n$. Taking the limit as $n \to \infty$ gives $0$.

2. Limit Inside ($\int_0^1 \big(\lim_{n \to \infty} f_n(x)\big)\,dx$): Here, we take the limit of the functions first. As $n \to \infty$, $f_n(x)$ pointwise converges to the Dirichlet function $D(x)$. But when we then try to compute $\int_0^1 D(x)\,dx$, the integral does not exist because the Riemann upper and lower sums never meet.

Why this failure is fatal for probability

Probability theory is built entirely on limits, such as the Law of Large Numbers, infinite coin tosses, and Markov chain convergence, where we constantly need to swap limits and expectations (integrals).

To see how explicitly this breaks down, consider one of the foundational theorems of probability: Continuity of Probability (or Monotone Convergence for Events).

Suppose we have an expanding sequence of nested events $E_1 \subset E_2 \subset E_3 \subset \dots$, and we want to find the probability of their infinite union $E = \bigcup_{n=1}^\infty E_n$. By definition, we compute this via a limit of finite probabilities: $$P\left(\bigcup_{n=1}^\infty E_n\right) = \lim_{n \to \infty} P(E_n)$$

Expressed as an integral of indicator functions $f_n(x) = \mathbf{1}_{E_n}(x)$, this theorem relies entirely on the ability to safely swap the limit and the integral: $$\lim_{n \to \infty} \int_0^1 f_n(x)\,dx = \int_0^1 \left( \lim_{n \to \infty} f_n(x) \right) dx$$

• Where Riemann fails: As we saw with our sequence of rational approximations $f_n \to D$, the left side evaluated to $0$, while the right side (integrating the limit function $D(x)$) completely failed to exist under Riemann rules.
• Why this is a problem: If an integration theory forbids this swap, you cannot even define the probability of infinite events. You would be unable to prove that the probability of an infinite union equals the limit of its finite parts, leaving basic axioms of probability (like countable additivity) unprovable and making it impossible to analyze infinite stochastic processes or limits of random variables.

Where this goes next: All three failures, measuring $\mathbb{Q}$ with points, integrating $D(x)$ with Riemann strips, and swapping limits, stem from being restricted to finitely many pieces. To fix probability, we need a modern measure theory that can handle countably many pieces natively.

---

The wishlist for “size”

We want a size function $\lambda$ (Greek lambda, for length) that assigns a length to subsets of the real line $\mathbb{R}$.

Component	Detail
Signature	$\lambda : \text{Subsets of } \mathbb{R} \rightarrow [0, \infty]$
Domain	Ideally, any subset $A \subseteq \mathbb{R}$.
Codomain	Non-negative real numbers plus $\infty$ (since sets like $[0, \infty)$ are infinitely long, with $a + \infty = \infty$).

To solve our dart problems, $\lambda$ must obey four foundational rules:

1. (W1) Intervals get their length: $\lambda([a,b]) = b - a$. Why: Size must agree with basic geometry on simple intervals.
2. (W2) Countable additivity: If $A_1, A_2, \dots$ are pairwise disjoint (non-overlapping, so $A_i \cap A_j = \varnothing$ for $i \neq j$), then: $$\lambda\left(\bigcup_{n=1}^\infty A_n\right) = \sum_{n=1}^\infty \lambda(A_n)$$ Why countable: This is the exact upgrade that gives $\lambda(\mathbb{Q}) = 0 + 0 + \cdots = 0$ and allows limits to work. Why not uncountable: $[0,1]$ is the disjoint union of its uncountably many individual points $\{x\}$. If we allowed uncountable sums, we would get $1 = \lambda([0,1]) = \sum_{x \in [0,1]} \lambda(\{x\}) = \sum 0 = 0$, a contradiction. Countable additivity is the sharpest rule possible without running into contradictions like this.
3. (W3) Translation invariance: $\lambda(A + t) = \lambda(A)$, where $A + t = \{a + t : a \in A\}$ shifts every point in $A$ by $t$. Why: A uniform dart has no favorite region, sliding a set sideways cannot alter its length.
4. (W4) Universal domain: $\lambda(A)$ is defined for every subset $A \subseteq \mathbb{R}$. Why: Ideally, we never want to check if an event is “valid” before asking for its probability.

---

The Vitali impossibility: why W1–W4 cannot coexist

Proof Strategy: We are trying to cut the 1-unit interval $[0,1)$ into a countably infinite number of identical, non-overlapping clones that can perfectly reassemble the original interval.

Think of every number in $[0,1)$ as a person. We assign everyone to a “club” based on one rule: two people are in the same club if the exact distance between them is a fraction (a rational number).

• Person $0.1$ and Person $0.6$ are in the same club (distance is $0.5$, which is $1/2$).
• Person $\pi - 3$ and Person $0.1$ are in different clubs.

Because there are “more” irrational numbers than rational ones, the interval splits into an uncountable number of different clubs. Inside each club, the members are perfectly spaced apart by exact fractions.

We then create a “VIP guest list”, the Vitali set $V$, by picking exactly one person from every single club.

If we take everyone on this list $V$ and shift them by every possible fraction $r_n \in [0,1)$ (wrapping around if they pass 1), we get a countable list of shifted copies: $V \oplus r_1, V \oplus r_2, \dots$

Because everyone in a club is separated by fractions, shifting our VIP list by fractions just systematically swaps in different members of the same clubs. The lists never overlap, and together they perfectly tile the entire interval:

$$[0,1) = (V \oplus r_1) \cup (V \oplus r_2) \cup (V \oplus r_3) \cup \cdots$$

By shift invariance (W3), every clone has the exact same length $\lambda(V)$. By countable additivity (W2), the total length of $[0,1)$ (which is $1$) must equal $\lambda(V)$ added to itself infinitely many times:

$$1 = \lambda(V) + \lambda(V) + \lambda(V) + \dots$$

• If $\lambda(V) = 0$, the infinite sum is $0 \neq 1$.
• If $\lambda(V) > 0$, the infinite sum is $\infty \neq 1$.

Either way, $\lambda(V)$ cannot equal any number in $[0, \infty]$, breaking W4. Here is the formal 3-step construction of $V$:

ProofStep-by-step construction of the Vitali set $V$

Step 1: Define “wrap-around shifting” ($\oplus$) so shifted sets stay inside $[0,1)$ If we shift a subset of $[0,1)$ to the right by $t$, part of it spills past $1$. To keep everything inside $[0,1)$, any part that goes past $1$ simply wraps around to $0$ (addition modulo $1$, like a clock): $$x \oplus t = \begin{cases} x + t & \text{if } x + t < 1 \\ x + t - 1 & \text{if } x + t \ge 1 \end{cases}$$ Does $\oplus$ preserve length? Yes. Shifting a set $A \subseteq [0,1)$ with wrap-around just cuts $A$ into two disjoint pieces, the left piece $A_{\text{left}} = A \cap [0, 1-t)$ that shifts normally by $+t$, and the right piece $A_{\text{right}} = A \cap [1-t, 1)$ that shifts by $t-1$. By W3 and W2: $$\lambda(A \oplus t) = \lambda(A_{\text{left}} + t) + \lambda(A_{\text{right}} + t - 1) = \lambda(A_{\text{left}}) + \lambda(A_{\text{right}}) = \lambda(A)$$

Step 2: Partition $[0,1)$ into “rational-difference buckets” and pick one representative from each Put two numbers $x, y \in [0,1)$ into the same bucket (equivalence class) if the distance between them, $x - y$, is a rational number.

• Every bucket is countable (it consists of one starting number $x$ plus all rational wrap-around shifts $x \oplus r$ for $r \in \mathbb{Q} \cap [0,1)$).
• Because $[0,1)$ is uncountable and each bucket is only countable, there are uncountably many disjoint buckets that together fill $[0,1)$.

Now form the Vitali set $V$ by reaching into each bucket and picking exactly one number from it. (Because there is no formula to pick a “first” number from each uncountable collection of buckets, this step invokes the Axiom of Choice, so $V$ exists mathematically, though no one can draw it.)

Step 3: Shift $V$ by every rational $r_1, r_2, \dots$ in $[0,1)$ to tile $[0,1)$ List all rationals in $[0,1)$ as $r_1, r_2, r_3, \dots$ and form the shifted copies $V_n = V \oplus r_n$. We verify two facts:

1. The copies $V_n$ cover every point in $[0,1)$: Pick any number $x \in [0,1)$. It belongs to some bucket, and $V$ contains that bucket’s unique representative $v$. Because $x$ and $v$ are in the same bucket, their wrap-around difference is some rational $r_n$, meaning $x = v \oplus r_n \in V_n$.
2. The copies $V_n$ never overlap (pairwise disjoint): Suppose $V_i$ and $V_j$ shared a point. Then $v_1 \oplus r_i = v_2 \oplus r_j$ for some $v_1, v_2 \in V$. Rearranging shows $v_1 - v_2$ is rational, meaning $v_1$ and $v_2$ live in the same bucket. Since $V$ only picked one number per bucket, $v_1$ and $v_2$ must be the exact same number, which forces $r_i = r_j$ (so $V_i$ and $V_j$ were the same copy all along).

Conclusion: $[0,1) = \bigcup_{n=1}^\infty (V \oplus r_n)$ is a countable disjoint union. Applying W2 and Step 1 gives: $$1 = \lambda([0,1)) = \sum_{n=1}^\infty \lambda(V \oplus r_n) = \sum_{n=1}^\infty \lambda(V) \in \{0, \infty\}$$ A contradiction ($1 \neq 0$ and $1 \neq \infty$). Thus $V$ cannot have a length. $\blacksquare$

∎

Which rule do we sacrifice?

We cannot keep all four wishes.

• Dropping W1 (interval length) or W3 (shift invariance) destroys basic geometry.
• Dropping W2 (countable additivity) leaves us stuck with the Riemann/rational failures of Section 1.
• Dropping W4 (every set has a size) costs us almost nothing in practice: the only sets that break $\lambda$, like the Vitali set $V$, require the non-constructive Axiom of Choice to scatter points so pathologically that no formula can ever define them. Here is what this sentence means:
  • What is the Axiom of Choice? Normally, to define a set, you use a rule or a formula (like ”$x$ is a number between $0$ and $1$”). But the Axiom of Choice is a rule that says: “Even if you have an infinite number of groups and no formula or rule can tell you how to pick an item from each group, you are legally allowed to just imagine that you reached into all of them simultaneously and grabbed one item anyway.”
  • What does the Vitali set look like? Because it was built using that “no-formula” rule, the Vitali set is not a normal geometric shape you can draw, graph, or measure in a physics lab. It is a completely chaotic, infinitely scattered dust cloud where every point is isolated from any recognizable pattern.
  • Why it doesn’t hurt us: In real-world math, physics, engineering, and statistics, you never encounter these bizarre, unformulaic dust clouds. Every set you can actually write down, describe, or use to model a real problem is “well-behaved” and gets a valid length just fine. Dropping W4 just bans these ghost-like mathematical anomalies from having a length, leaving all normal math completely untouched.

Our resolution: Keep W1–W3, drop W4, and restrict $\lambda$ to a domain of “well-behaved” subsets called measurable sets.

---

σ-algebras: deciding which sets get a size

Motivation: Having dropped W4, we cannot let our event space be the power set $\mathcal{P}(\Omega)$ (all subsets of $\Omega$). Instead, on any sample space $\Omega$ of outcomes, we must select a restricted club of “measurable” subsets $\mathcal{F} \subseteq \mathcal{P}(\Omega)$ that are allowed to have a size. What closure rules must this club obey so we never accidentally manufacture an unmeasurable set like $V$ while doing probability?

Rule	Mathematical Statement	Probabilistic Meaning & Why We Need It
(S1) Contains the whole space	$\Omega \in \mathcal{F}$	”Something happens” must be a valid event (so we can set $P(\Omega) = 1$).
(S2) Closed under complements	$A \in \mathcal{F} \implies A^c = \Omega \setminus A \in \mathcal{F}$	If we can find the probability that $A$ happens, we must be able to find the probability that $A$ does not happen ($1 - P(A)$).
(S3) Closed under countable unions	$A_1, A_2, \dots \in \mathcal{F} \implies \bigcup_{n=1}^\infty A_n \in \mathcal{F}$	Required by W2: countable additivity $\mu(\bigcup A_n) = \sum \mu(A_n)$ only makes sense if the countable union $\bigcup A_n$ is itself inside the allowed domain $\mathcal{F}$!

The Borel $\sigma$-algebra: solving the dart board

Now return to the real line $\Omega = \mathbb{R}$. Which specific $\sigma$-algebra should we use?

• We must include all open intervals $(a,b)$ so we can measure basic lengths (W1).
• We must not include unnecessary garbage like the Vitali set $V$.

Therefore, we want the smallest possible $\sigma$-algebra that contains all open intervals.

Because $\mathcal{B}(\mathbb{R})$ is closed under complements, countable unions, and countable intersections, starting with just open intervals $(a,b)$ automatically generates every concrete set we needed in Section 1:

• Single points $\{x\}$: Squeeze open intervals down onto $x$: $\{x\} = \bigcap_{n=1}^\infty \left(x - \frac{1}{n},\, x + \frac{1}{n}\right) \in \mathcal{B}(\mathbb{R})$.
• Closed intervals $[a,b]$ and half-open intervals $(a,b]$: $[a,b] = (a,b) \cup \{a\} \cup \{b\} \in \mathcal{B}(\mathbb{R})$.
• The rationals $\mathbb{Q}$: A countable union of single points: $\mathbb{Q} = \bigcup_{n=1}^\infty \{q_n\} \in \mathcal{B}(\mathbb{R})$.
• The irrationals $\mathbb{R} \setminus \mathbb{Q}$: The complement of a Borel set: $\mathbb{Q}^c \in \mathcal{B}(\mathbb{R})$.

Every set you can ever write down explicitly with formulas and limits is a Borel set. The Vitali set $V$ is not.

---

Measures: the size function, formalized

With the domain restricted from all subsets to a $\sigma$-algebra $\mathcal{F}$, we can formalize our size function.

Component	Detail
Signature	$\mu : \mathcal{F} \rightarrow [0, \infty]$
Domain	Only the measurable sets $A \in \mathcal{F}$ (where W4 was dropped).
Codomain	Non-negative reals plus $\infty$. If $\mu(\Omega) = 1$, $\mu$ is a probability measure $P$.

Core properties that come for free from M1 & M2

Every measure automatically satisfies four foundational behaviors:

1. Monotonicity ($A \subseteq B \implies \mu(A) \le \mu(B)$): Since $B = A \cup (B \setminus A)$ is disjoint, $\mu(B) = \mu(A) + \mu(B \setminus A) \ge \mu(A)$.
2. Countable subadditivity (even for overlapping sets): $\mu\left(\bigcup_{n=1}^\infty A_n\right) \le \sum_{n=1}^\infty \mu(A_n)$. (Shave off overlaps by defining disjoint pieces $B_n = A_n \setminus (A_1 \cup \cdots \cup A_{n-1}) \subseteq A_n$ and apply M2).
3. Continuity from below (growing sets): If $A_1 \subseteq A_2 \subseteq \cdots$ grows to $A = \bigcup_{n=1}^\infty A_n$, then $\mu(A_n) \to \mu(A)$.
4. Continuity from above (shrinking sets): If $A_1 \supseteq A_2 \supseteq \cdots$ shrinks to $A = \bigcap_{n=1}^\infty A_n$ and $\mu(A_1) < \infty$, then $\mu(A_n) \to \mu(A)$.

flowchart LR
  subgraph "Continuity from Below: A₁ ⊆ A₂ ⊆ A₃ ..."
    direction LR
    B1["B₁ = A₁"] --> B2["+ ring B₂ = A₂ \ A₁"] --> B3["+ ring B₃ = A₃ \ A₂"] --> Sum["Disjoint sum = lim μ(Aₙ)"]
  end

ProofWhy Countable Additivity forces Continuity of Measure

To prove continuity from below, turn the nested sequence $A_1 \subseteq A_2 \subseteq \cdots$ into disjoint “onion rings”: $$B_1 = A_1, \qquad B_2 = A_2 \setminus A_1, \qquad \dots, \qquad B_n = A_n \setminus A_{n-1}$$ The rings $B_k$ are pairwise disjoint, the first $n$ rings form $A_n = \bigcup_{k=1}^n B_k$, and all rings together form $A = \bigcup_{k=1}^\infty B_k$. By countable additivity (M2) and the definition of an infinite series: $$\mu(A) = \mu\left(\bigcup_{k=1}^\infty B_k\right) = \sum_{k=1}^\infty \mu(B_k) = \lim_{n \to \infty} \sum_{k=1}^n \mu(B_k) = \lim_{n \to \infty} \mu(A_n)$$ (For continuity from above, apply this to $A_1 \setminus A_n \uparrow A_1 \setminus A$. We require $\mu(A_1) < \infty$ so we never subtract $\infty - \infty$; for example, $A_n = [n, \infty)$ has $\lambda(A_n) = \infty$ for all $n$, yet $\bigcap A_n = \varnothing$ has measure $0$.)

∎

Immediate payoff: $P(X \in \mathbb{Q}) = 0$ and “Almost Everywhere”

Suppose we have a measure $\lambda$ on $\mathcal{B}(\mathbb{R})$ that assigns intervals their length ($\lambda([a,b]) = b-a$).

• Every single point $\{x\}$ has length $\lambda(\{x\}) = \lambda([x,x]) = 0$.
• By countable additivity (M2), the rationals have length: $$\lambda(\mathbb{Q}) = \lambda\left(\bigcup_{n=1}^\infty \{q_n\}\right) = \sum_{n=1}^\infty \lambda(\{q_n\}) = 0 + 0 + 0 + \cdots = 0$$

---

Constructing Lebesgue measure: Carathéodory’s extension

The Problem: Section 5 assumed a length measure $\lambda$ on $\mathcal{B}(\mathbb{R})$ exists. How do we prove it actually exists? We know the length of a half-open interval is $\ell((a,b]) = b-a$, and we can add lengths of finitely many disjoint intervals. But $\mathcal{B}(\mathbb{R})$ contains unimaginably complex sets built by infinite intersections and unions of intervals. You cannot write an explicit formula for every Borel set directly.

In 1914, Constantin Carathéodory solved this with a brilliant 3-step pipeline that builds Lebesgue measure from the outside in:

flowchart LR
  S1["Step 1: Elementary length ℓ\non finite unions of (a,b]"] --> S2["Step 2: Outer measure λ*\nInfinite shrink-wrap on ALL sets"] --> S3["Step 3: Carathéodory cut test\nKeep only sets that split cleanly"]

Step 1: Length on elementary intervals (the Premeasure)

Start with the simple collection $\mathcal{A}$ of finite disjoint unions of half-open intervals $(a,b]$. (We use half-open intervals because they tile cleanly without overlap: $(a,b] \cup (b,c] = (a,c]$). On $\mathcal{A}$, define naive length:

$$\ell\left(\bigcup_{i=1}^k (a_i, b_i]\right) = \sum_{i=1}^k (b_i - a_i)$$

Using the compactness of closed intervals on the real line (the Heine–Borel theorem), one can verify that if an interval $(a,b]$ is sliced into a countable disjoint union of smaller intervals $\bigcup_{n=1}^\infty (a_n, b_n]$, its length still equals $\sum_{n=1}^\infty (b_n - a_n)$. A size function on a simple collection $\mathcal{A}$ that obeys countable additivity whenever the union happens to land back inside $\mathcal{A}$ is called a premeasure.

Step 2: Outer measure $\lambda^*$ (“infinite shrink-wrapping”)

Next, we define an approximate size, called outer measure $\lambda^*(A)$, for every subset $A \subseteq \mathbb{R}$.

How do you measure an arbitrary, jagged set $A$ using only simple intervals? You cover $A$ with a countable sequence of intervals $(a_1, b_1], (a_2, b_2], \dots$ such that $A \subseteq \bigcup_{n=1}^\infty (a_n, b_n]$, add up the lengths of those intervals, and take the tightest possible fit (the infimum / greatest lower bound over all valid covers):

Step 3: The Carathéodory cut condition

Outer measure $\lambda^*(A)$ is defined on every subset of $\mathbb{R}$, matches $b-a$ on intervals, and is shift-invariant. Therefore, by Vitali’s theorem (§3), $\lambda^*$ cannot be countably additive on all sets.

Specifically, when two disjoint sets (like two shifted Vitali sets $V_1$ and $V_2$) are pathologically intertwined like Velcro, any interval cover around $V_1$ inevitably overlaps $V_2$, causing the sum of their individual shrink-wraps to exceed the shrink-wrap of their union: $\lambda^*(V_1 \cup V_2) < \lambda^*(V_1) + \lambda^*(V_2)$.

To turn $\lambda^*$ into a true countably additive measure, Carathéodory simply filtered out the Velcro sets: keep only the sets $A$ that slice every other set $E$ cleanly without any overlap penalty.

---

Measurable functions and random variables

Motivation: In probability, outcomes $\omega \in \Omega$ are physical occurrences (coin flips, weather states, dart positions), and we use a function $X : \Omega \rightarrow \mathbb{R}$ (a random variable) to turn those outcomes into numerical payoffs. Next, we will want to compute the average of $X$, which requires asking: “What is the probability that the output $X(\omega)$ falls inside some target range $B \subseteq \mathbb{R}$?”

For that probability $P(\{\omega \in \Omega : X(\omega) \in B\})$ to exist, the set of inputs $\omega$ that land inside $B$ must belong to our measurable $\sigma$-algebra $\mathcal{F}$.

Component	Detail
Signature	$f : (\Omega, \mathcal{F}) \rightarrow (\mathbb{R}, \mathcal{B}(\mathbb{R}))$
Preimage $f^{-1}(B)$	$\{\omega \in \Omega : f(\omega) \in B\}$, the set of inputs in $\Omega$ that map into the target set $B \subseteq \mathbb{R}$.
Requirement	Every Borel target set $B \in \mathcal{B}(\mathbb{R})$ must pull back to a measurable event $f^{-1}(B) \in \mathcal{F}$.

flowchart RL
  B["Target Borel set B ∈ ℬ(ℝ)\ne.g. (-∞, a]"] -- "Preimage f⁻¹(B)\npulls back cleanly" --> A["Event {ω : f(ω) ∈ B}\nMUST lie in ℱ"]

Why this class of functions is bulletproof

Unlike Riemann-integrable functions, the family of measurable functions is closed under virtually every operation in analysis:

• Indicators: $\mathbf{1}_A$ is measurable $\iff A \in \mathcal{F}$. (Thus the Dirichlet function $D = \mathbf{1}_{\mathbb{Q}}$ is Borel-measurable!)
• Algebra & Calculus: Continuous functions, sums $f+g$, products $f \cdot g$, and compositions of measurable functions are all measurable.
• Pointwise Limits: If $f_1, f_2, \dots$ are measurable, then $\sup_n f_n$, $\inf_n f_n$, $\limsup_n f_n$, $\liminf_n f_n$, and $\lim_{n \to \infty} f_n$ are always measurable (because $\{\sup_n f_n \le a\} = \bigcap_{n=1}^\infty \{f_n \le a\}$, which is a countable intersection of measurable sets).

---

The Lebesgue integral: slicing the y-axis

Motivation: Why did the Riemann integral choke on the Dirichlet function $D = \mathbf{1}_{\mathbb{Q}}$ in Section 1? Because Riemann slices the input axis ($x$-axis) into narrow intervals $[x_{i-1}, x_i]$ and expects $f(x)$ to stay roughly flat inside each interval. When a function jumps wildly between $0$ and $1$ inside every interval, $x$-axis slicing fails.

Henri Lebesgue reversed the geometry: slice the output axis ($y$-axis) into horizontal height levels, collect all inputs $\omega$ where $f(\omega)$ hits a given height $a_i$, and multiply that height by the measure $\mu$ of that input set.

We build the Lebesgue integral $\int_\Omega f\,d\mu$ in three progressive stages:

Stage 1: Simple functions (finitely many output heights)

A simple function $\varphi : \Omega \rightarrow [0, \infty)$ is a measurable function that takes only finitely many non-negative values $a_1, \dots, a_k$:

$$\varphi(\omega) = \sum_{i=1}^k a_i\,\mathbf{1}_{A_i}(\omega) \qquad \text{where } A_i = \{\omega : \varphi(\omega) = a_i\} \in \mathcal{F}$$

Its Lebesgue integral is simply height $\times$ measure of the level set:

Immediate Payoff (Solving Problem 2 from §1): The Dirichlet function $D = 1 \cdot \mathbf{1}_{\mathbb{Q}} + 0 \cdot \mathbf{1}_{\mathbb{Q}^c}$ is already a simple function! On $[0,1]$, its Lebesgue integral is:

$$\int_{[0,1]} D\,d\lambda = 1 \cdot \lambda(\mathbb{Q} \cap [0,1]) + 0 \cdot \lambda(\mathbb{Q}^c \cap [0,1]) = 1 \cdot 0 + 0 \cdot 1 = 0$$

Stage 2: Non-negative measurable functions ($f \ge 0$)

For any measurable function $f : \Omega \rightarrow [0, \infty]$, we approximate it from below using simple “staircase” functions $\varphi$ that sit underneath $f$ ($0 \le \varphi \le f$), and take the supremum:

Specifically, we can always construct an explicit sequence of simple functions $\varphi_n \uparrow f$ by slicing the $y$-axis into a grid of spacing $1/2^n$ (up to height $n$) and rounding $f(\omega)$ down to the nearest grid line. Because $f$ is measurable, each horizontal slice $A_k = \{\omega : k 2^{-n} \le f(\omega) < (k+1) 2^{-n}\}$ is guaranteed to be a measurable set in $\mathcal{F}$!

Stage 3: General signed functions ($f : \Omega \rightarrow \mathbb{R}$)

To handle functions that dip below the axis, split $f$ into its positive part $f^+(\omega) = \max(f(\omega), 0) \ge 0$ and negative part $f^-(\omega) = \max(-f(\omega), 0) \ge 0$, so that $f = f^+ - f^-$ and $|f| = f^+ + f^-$.

Key properties of the Lebesgue integral

• Extends Riemann: Every bounded Riemann-integrable function on $[a,b]$ is Lebesgue integrable and gives the exact same number.
• Blind to Null Sets: Changing $f(\omega)$ on a set of measure $0$ does not change $\int_\Omega f\,d\mu$ at all (because a simple function step over a set of measure $0$ adds $a_i \cdot 0 = 0$). Furthermore, if $f \ge 0$ and $\int_\Omega f\,d\mu = 0$, then $f = 0$ almost everywhere.
• Probability Expectation: When $(\Omega, \mathcal{F}, P)$ is a probability space, the Lebesgue integral of a random variable $X$ is its expected value: $\mathbb{E}[X] = \int_\Omega X\,dP$.

---

Convergence theorems: when can we swap $\lim$ and $\int$?

Motivation: We now return to Problem 3 from Section 1: given a sequence of measurable functions $f_n(\omega) \to f(\omega)$ pointwise, when can we guarantee that the limit of the integrals equals the integral of the limit?

$$\lim_{n \to \infty} \int_\Omega f_n\,d\mu \stackrel{?}{=} \int_\Omega \left(\lim_{n \to \infty} f_n\right) d\mu$$

Even with the Lebesgue integral, you cannot always swap $\lim$ and $\int$, because “area (mass) can escape to infinity” before the limit finishes! There are two classic ways area escapes while $f_n(x) \to 0$ at every point $x$:

Failure Mode	Formula ($f_n \to 0$ pointwise)	Visual Behavior	Why Swap Fails
1. Escape Horizontally (Moving Bump)	$f_n(x) = \mathbf{1}_{[n,\, n+1]}(x)$ on $\mathbb{R}$	A box of height $1$ and width $1$ slides off to $+\infty$. At any fixed $x$, the box eventually passes by, so $f_n(x) \to 0$.	$\int f_n = 1$ for all $n$, so $\lim \int f_n = 1 \neq \int 0 = 0$.
2. Escape Vertically (Witch’s Hat Spike)	$f_n(x) = n\,\mathbf{1}_{(0,\, 1/n)}(x)$ on $[0,1]$	A rectangle of height $n$ and width $1/n$ squeezes against $0$. At $x=0$, $f_n(0)=0$; for any $x > 0$, $1/n < x$ eventually, so $f_n(x) \to 0$.	$\int f_n = n \cdot \frac{1}{n} = 1$ for all $n$, so $\lim \int f_n = 1 \neq \int 0 = 0$.

Measure theory provides three master theorems that tell us exactly what happens to the area under a limit:

1. Monotone Convergence Theorem (MCT): upward growth never loses mass

If the functions only ever grow upward ($0 \le f_1 \le f_2 \le \cdots$), mass cannot slide away or disappear, new area is only ever stacked on top of old area.

2. Fatou’s Lemma: mass can escape in the limit, but can never be created out of nothing

What if $f_n \ge 0$ does not increase monotonically (like the Moving Bump or Witch’s Hat)? Then mass might escape on the way to the limit, making the integral of the limit smaller than the limit of the integrals. Can mass ever spontaneously appear in the limit? No.

(Recall $\liminf_{n\to\infty} a_n = \lim_{n\to\infty} (\inf_{k \ge n} a_k)$ is the “eventual floor” of a sequence, which always exists even if $a_n$ oscillates.)

Proof3-line proof of Fatou’s Lemma from MCT

Define the running floor $g_n(\omega) = \inf_{k \ge n} f_k(\omega)$. As $n$ grows, we drop terms from the infimum, so $0 \le g_1 \le g_2 \le \cdots$ increases monotonically to $\liminf_n f_n$.

1. For every $k \ge n$, we have $g_n \le f_k$, so $\int g_n \le \int f_k$. Taking the infimum over $k \ge n$ gives $\int g_n \le \inf_{k \ge n} \int f_k$.
2. Take $\lim_{n \to \infty}$ on both sides and apply MCT to the left side ($\int g_n \uparrow \int \liminf f_n$) to get $\int (\liminf f_n) \le \liminf \int f_n$. $\blacksquare$

∎

3. Dominated Convergence Theorem (DCT): trapping functions under a finite roof

How do we prevent both horizontal escape and vertical spikes when $f_n$ can oscillate up and down? Trap the entire sequence $|f_n|$ underneath a single integrable roof function $g$ with finite total area ($\int_\Omega g\,d\mu < \infty$).

• Because $\int g < \infty$, the roof must taper down to zero at $\pm\infty$ (blocking horizontal escape).
• Because $\int g < \infty$, the roof cannot enclose an infinite-area vertical spike (blocking vertical escape; for the Witch’s Hat $n\mathbf{1}_{(0,1/n)}$, the lowest roof covering all spikes is $g(x) = 1/x$, which has $\int_0^1 \frac{1}{x}\,dx = \infty$, so DCT rightly refuses to apply!).

---

$L^p$ spaces: geometry and completeness of functions

Motivation: In probability and statistics, we constantly measure the “distance” between two random variables $X$ and $Y$, for instance, measuring the mean-squared error $\mathbb{E}[|X_n - X|^2]$ of an estimator $X_n$. The Lebesgue integral lets us organize functions into geometric spaces based on their $p$-th power average size.

Component	Detail
$p$-norm ($1 \le p < \infty$)	$|f|p = \left(\int\Omega
$\infty$-norm ($p = \infty$)	$|f|_\infty = \inf{M \ge 0 :
Space $L^p(\mu)$	The space of all measurable functions with $\|f\|_p < \infty$, where two functions equal a.e. ($f = g$ a.e.) are treated as the exact same point.

Two foundational inequalities and completeness

1. Hölder’s Inequality ($\frac{1}{p} + \frac{1}{q} = 1$): $\int_\Omega |f g|\,d\mu \le \|f\|_p\,\|g\|_q$.
   • For $p = q = 2$, this is the Cauchy–Schwarz Inequality $\mathbb{E}[|XY|] \le \sqrt{\mathbb{E}[X^2]\,\mathbb{E}[Y^2]}$, which guarantees that correlation always lies in $[-1, 1]$ and turns $L^2(P)$ into a Hilbert space with dot product $\langle X, Y \rangle = \mathbb{E}[XY]$ (where conditional expectation $\mathbb{E}[X \mid \mathcal{G}]$ is simply an orthogonal projection!).
   • On a probability space ($P(\Omega) = 1$), applying Hölder with $g = 1$ proves that higher norms dominate lower norms: $\|X\|_p \le \|X\|_q$ for $p \le q$, meaning $L^q(P) \subseteq L^p(P)$ (e.g., finite variance $X \in L^2(P)$ automatically implies finite mean $X \in L^1(P)$).
2. Minkowski’s Inequality ($p \ge 1$): $\|f + g\|_p \le \|f\|_p + \|g\|_p$ (the Triangle Inequality, proving $\|f - g\|_p$ is a valid metric distance).
3. Completeness (The Riesz–Fischer Theorem): Every Cauchy sequence in $L^p(\mu)$ converges to a limit function inside $L^p(\mu)$.

---

Translation Dictionary & Summary

Measure Theory Concept	Probability Meaning	Role in the Architecture
Sample space $\Omega$	Set of all physical outcomes $\omega$	The underlying universe before numbers are assigned.
$\sigma$-algebra $\mathcal{F}$	Allowed events / observer’s information	Excludes Vitali sets so countable additivity never contradicts geometry.
Borel $\sigma$-algebra $\mathcal{B}(\mathbb{R})$	$\sigma(\{(a,b)\})$ on the real line	Contains intervals, points, $\mathbb{Q}$, and all limits, while locking out $V$.
Measure $\mu$ (with $\mu(\Omega)=1$)	Probability measure $P$	Countably additive size function; gives $P(A_n) \to P(A)$ on nested events.
Null set ($\mu(N)=0$) & a.e.	Probability-$0$ event & almost surely (a.s.)	Allows sets like $\mathbb{Q}$ to be non-empty yet carry zero weight.
Carathéodory extension	A CDF $F(x)$ uniquely defines a distribution	Extends interval lengths $F(b)-F(a)$ via outer shrink-wrapping $\lambda^*$.
Measurable map $X:\Omega\to\mathbb{R}$	Random variable $X$	Guarantees $\{X \in B\} \in \mathcal{F}$ for every Borel set $B \in \mathcal{B}(\mathbb{R})$.
Lebesgue integral $\int_\Omega X\,dP$	Expected value $\mathbb{E}[X]$	Slices the $y$-axis into simple staircases so jagged functions integrate cleanly.
MCT / Fatou / DCT	Rules for $\lim \mathbb{E}[X_n] = \mathbb{E}[\lim X_n]$	Prevents probability mass from escaping to $\infty$ during a limit.
$L^1(P)$ and $L^2(P)$	Finite mean ($\mathbb{E}[\lvert X\rvert]<\infty$) & finite variance	Complete geometric spaces where $L^2$ dot products represent covariance.