Lecture 4 — Beginner tutorial

Distribution function / CDF as pushforward of P

Lecture 4 Tutorial — Distribution of a random variable (Beginner)

Based on Lec 04.

Goal

See the formula

\[P_X(x) = P(\{\omega : X(\omega) \le x\})\]

on a tiny example (coin), then estimate a CDF from samples.

Run with Shift+Enter.

Step 0 — Imports

import numpy as np
import matplotlib.pyplot as plt

print('Ready')

Step 1 — Abstract world: Omega, P, and X

Fair coin:

• Omega = {H, T}
• P({H}) = P({T}) = 1/2
• Random variable X: X(H)=1, X(T)=0

Omega = {"H", "T"}

def P(event):
    event = set(event)
    return sum(0.5 for o in event if o in Omega)

def X(omega):
    return 1.0 if omega == "H" else 0.0

print("P({H}) =", P({"H"}))
print("X(H), X(T) =", X("H"), X("T"))

Step 2 — Distribution function by inverse image

For a number x, look at the event {omega : X(omega) <= x}, then apply P.

That is exactly P_X(x) from the lecture.

def P_X(x):
    """CDF / distribution function of X at real number x."""
    event = {omega for omega in Omega if X(omega) <= x}
    return P(event)

for x in [-1, 0, 0.5, 1, 2]:
    print(f"P_X({x}) = P(X <= {x}) = {P_X(x)}")

What you should see

• For x < 0: nothing maps to <= x → probability 0
  
• For 0 <= x < 1: only T (value 0) → probability 1/2
  
• For x >= 1: both outcomes → probability 1

That step function is the pushforward / distribution of X.

Step 3 — Estimate the same CDF from samples

In ML we rarely know P. We observe many values of X (range samples) and estimate.

rng = np.random.default_rng(0)
n = 5000
# simulate abstract outcomes, push through X (like a sensor)
samples = np.array([X(o) for o in rng.choice(["H", "T"], size=n)])

xs = np.linspace(-0.5, 1.5, 200)
# empirical CDF: fraction of samples <= x
ecdf = np.array([(samples <= x).mean() for x in xs])
true = np.array([P_X(x) for x in xs])

plt.figure(figsize=(7, 4))
plt.step(xs, true, where="post", label="true P_X (from Omega)")
plt.plot(xs, ecdf, label="empirical CDF from samples")
plt.xlabel("x")
plt.ylabel("P(X <= x)")
plt.title("Pushforward distribution of coin RV X")
plt.legend()
plt.show()

Step 4 — Mini exercises

# Exercise A: unfair coin P(H)=0.7, same X coding.
# Recompute P_X at x = -1, 0, 0.5, 1.

def P_unfair(event, p_h=0.7):
    event = set(event)
    mass = {"H": p_h, "T": 1 - p_h}
    return sum(mass[o] for o in event)

def P_X_unfair(x):
    event = {o for o in Omega if X(o) <= x}
    return P_unfair(event)

for x in [-1, 0, 0.5, 1]:
    print(x, P_X_unfair(x))

# Exercise B: die, X(omega)=omega.
# What is P_X(3.5)? (should equal P(X <= 3.5) = P({1,2,3}) = 3/6)

Omega_die = {1, 2, 3, 4, 5, 6}

def P_die(event):
    return len(set(event) & Omega_die) / len(Omega_die)

def P_X_die(x):
    event = {o for o in Omega_die if o <= x}
    return P_die(event)

print("P_X(3.5) =", P_X_die(3.5))

Recap

Lecture idea	In this notebook
Inverse image	{omega : X(omega) <= x}
Pushforward / P_X	P of that event
Work with samples	empirical CDF approximates P_X
Course goal	estimate distributions from range samples

Next: several random variables, conditioning, then ML as distribution learning.