Probability

Basics

An experiment has a set of possible outcomes, the sample space S. An event is a subset of S. When all outcomes are equally likely,

P(A) = (number of outcomes in A) / (number of outcomes in S).

Rolling two dice gives 36 equally likely outcomes; the event "the sum is 7" contains 6 of them, so P = 6/36 = 1/6.

The axioms: P(A) ≥ 0, P(S) = 1, and P(A ∪ B) = P(A) + P(B) when A and B are disjoint. From them:

• P(Ā) = 1 − P(A) (the complement rule, often the easy way: "at least one" = 1 − "none").
• P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
• P(∅) = 0, and A ⊆ B ⇒ P(A) ≤ P(B).

Conditional probability and independence

The probability of A given that B has happened:

P(A | B) = P(A ∩ B) / P(B), so P(A ∩ B) = P(B) · P(A | B) (the multiplication rule).

A and B are independent if P(A ∩ B) = P(A) · P(B), equivalently P(A | B) = P(A). Independent is not the same as disjoint: two disjoint events with positive probabilities are never independent (if one happens, the other cannot).

Examples:

• Drawing 2 cards without replacement, P(both aces) = 4/52 × 3/51 = 1/221.
• Rolling a die 4 times, P(at least one six) = 1 − (5/6)⁴ = 671/1296 ≈ 0.518.

Total probability and Bayes' theorem

If B₁, …, Bₙ partition the sample space (exactly one of them happens):

• Law of total probability: P(A) = Σ P(Bᵢ) P(A | Bᵢ).
• Bayes' theorem: P(Bᵢ | A) = P(Bᵢ) P(A | Bᵢ) / P(A): it turns "probability of the evidence given the cause" into "probability of the cause given the evidence".

Example (medical test). A disease affects 1% of people. A test detects it in 99% of sick people, but also gives a positive result for 5% of healthy people. If someone tests positive, how likely are they to be sick?

P(sick | +) = (0.01 × 0.99) / (0.01 × 0.99 + 0.99 × 0.05) = 0.0099 / 0.0594 = 1/6 ≈ 0.167.

Most positives come from the large healthy group, so a positive result still means only a 1-in-6 chance of being sick.

Exact computations with fractions, by enumerating outcomes:

from fractions import Fraction
from itertools import product

dice = list(product(range(1, 7), repeat=2))
def P(event):
    return Fraction(sum(1 for d in dice if event(d)), len(dice))

print("P(sum = 7) =", P(lambda d: sum(d) == 7))
print("P(sum = 8 | first die even) =", P(lambda d: sum(d) == 8 and d[0] % 2 == 0) / P(lambda d: d[0] % 2 == 0))
print("P(larger die is 4) =", P(lambda d: max(d) == 4))
print("P(at least one six in 4 rolls) =", 1 - Fraction(5, 6) ** 4)

prior, sensitivity, false_positive = Fraction(1, 100), Fraction(99, 100), Fraction(5, 100)
posterior = prior * sensitivity / (prior * sensitivity + (1 - prior) * false_positive)
print("P(sick | positive) =", posterior)

p_no_shared = 1.0
for i in range(23):
    p_no_shared *= (365 - i) / 365
print(f"birthday problem, 23 people: P(some shared birthday) = {1 - p_no_shared:.4f}")

P(sum = 7) = 1/6
P(sum = 8 | first die even) = 1/6
P(larger die is 4) = 7/36
P(at least one six in 4 rolls) = 671/1296
P(sick | positive) = 1/6
birthday problem, 23 people: P(some shared birthday) = 0.5073

The birthday problem surprises most people: with only 23 people, a shared birthday is more likely than not.

Practice questions