Reminder: This post contains 3812 words
· 11 min read
· by Xianbin
I feel it is necessary to learn measure-theory to help me understand some fundamental conceptions.
The motivation of the incoming series of measure-theory posts is that I want to know the following question?
- What is a random variable?
- What is randomness?
- What can influence a probability?
- Why we can guarantee that random variables are independent of each other?
1. What is Measure?
\(\textbf{Definition}(\sigma\text{-algebra})\). Let \(Z\) be a set. A \(\sigma\)-algebra \(\zeta\) is a collection of subsets of \(Z\) such that it satisfies the following three conditions:
-
We have \(\emptyset \in \zeta\);
-
If \(A\in \zeta\), then \(A^c \in \zeta\);
-
\(\bigcup X_i\in \zeta\) where \(X_i\in \zeta\).
To better understand the definitions, we should do something based on the definition.
\(\textbf{Fact 1}\). \(\bigcap X_i \in \zeta\).
By the definition and De-Morgan’s law, we can prove Fact 1.
We say that \((Z,\zeta)\) is a measure space.
2. The Measure Function
\(\textbf{Definition}(\text{Measure Function})\). A measure is a function \(\mu: \zeta \to [0, \infin)\) that satisfies:
- \(\mu(\empty) = 0\);
- Countable additivity:
For any disjoint sequence \(X_i\) in \(\zeta\), we have
\[\mu(\bigcup_{i=1}^\infin X_i) = \sum_{n=1}^\infin \mu(X_i)\]
We also use \((Z, \zeta, \mu)\) to describe a measure space.
3. Smallest \(\sigma\)-algebra
\(\textbf{Definition}(\text{Generator of } \sigma\text{-algebra})\). Let \(B\) be a set and let \(P(Z)\) be the set of all subsets of \(Z\). We define \(\sigma(B)\) as follows:
\[\sigma(B) = \{ B\subseteq Z: B\in \zeta \text{ for all } \sigma\text{-algebra that contains } B \}\]
Example 1: \(Z = \mathbb{Z}\), and \(B = \{\{x\}: x\in \mathbb{Z}\}\). Then, \(\sigma(B) = P(Z)\).
Borel \(\sigma\text{-algebra}\)
\(\textbf{Definition}(\text{Borel } \sigma\text{-algebra})\). Let \(Z= \mathbb{R}\), and \(B = \{ U\subseteq \mathbb{R}: U \text{ is open} \}\). Then, \(\sigma(B)\) is called Borel\(\sigma\text{-algebra}\).
Why do we need Borel \(\sigma\text{-algebra}\)
One reason is that not all events have probability. To have a probability, we define a space that is \(\sigma\)-algebra.
Examples
It is very common to see \(X\sim [0,1]\) (uniform distribution). What does it mean precisely?
We believe that (Property 1)
\[P(0\leq X \leq 1) = 1,P(0\leq X < 1/2) = 1/2\]
In general, we believe that
\[P([a,b]) = P((a,b]) = P((a,b)) = b-a\]
Also, for disjoint set \(A\) and \(B\), we have (Property 2)
\[P(A\cup B) = P(A) + P(B)\]
The above is called countable additivity.
we cannot say that
\[P([0,1]) = \sum_{x\in[0,1]}P(\{x\})\]
LHS = 1, RHS = 0.
Another property that we want is “shifting”. Let \(r\in[0,1]\).
Let \(A \oplus r = \{a+r: a\in A, a+r \leq 1\}\cup \{ a+r-1: a\in A, a+r > 1\}\)
where \(a\leq a+r \leq 1\) and \(0\leq a+r-1 \leq a\).
We write the above as (Property 3)
\[P(A\oplus r) = P(A)\]
The interesting thing comes!
\(\textbf{Lemma 1}\). There does not exist a definition of \(P(A)\) defined for all subsets \(A\subseteq [0,1]\), satisfying the above properties.
Proof. To prove Lemma 1, we need Axiom of Choice. Basically, it means that given a collection of non-emptyset, we can choose one element from each set. We also need to know what is countable. Here, we just use the math that rational numbers are countable. For details, please look at the work of Georg Cantor (He is also one of my favorite mathematicians).
Now, let us start to prove this lemma. We define an equivalence relation on \([0,1]\) by \(x\sim y\) if and only if the difference \(y - x\) is rational. Then, this relation partitions the interval \([0,1]\) into disjoint union of equivalence classes. Let \(Z\) be a subset of \([0,1]\) consisting of exactly one element from each equivalence class (by Axiom of Choice, we know that we can do this). For definiteness, we assume that \(0\not \in Z\).
Now, since \(Z\) contains an element of each equivalence class, we see that each point in \([0,1]\) is contained in the union
\[\bigcup_{r\in[0,1)} (Z\oplus r)\]
where \(r\) is rational.
Since rational numbers are countable and these sets \(Z\oplus r\) are disjoints (we only select one point), we have
\[1= P((0,1]) = \sum_{r\in[0,1)} P(Z\oplus r) = \sum_{r\in[0,1)} P(Z)\]
A countable infinite sum of the same quantity can only be 0 or infinite, but it never equals 1. A contradiction.
Reference
[1]. Part II — Probability and Measure (Based on lectures by J. Miller).
[2]. Rosenthal, J.S., 2006. First look at rigorous probability theory, A. World Scientific Publishing Company.