§ 01What Is Probability?
Probability is the mathematical language for uncertainty: a way of assigning numbers to how likely events are to occur.
Every time you wonder whether it will rain tomorrow, what your chances are of passing an exam, or whether a clinical trial drug works, you are asking a probability question. At its core, probability is a function that maps events to numbers between 0 and 1, where 0 means impossible and 1 means certain.
The modern, rigorous foundation was laid by Andrei Kolmogorov in 1933. His three axioms are simple enough to state in a few lines, yet powerful enough to derive every result in this entire section from them.
Axiom 1 — Non-negativity: P(A) ≥ 0 for every event A
Axiom 2 — Normalisation: P(Ω) = 1
Axiom 3 — Countable Additivity: If A₁, A₂, … are mutually exclusive, then P(A₁ ∪ A₂ ∪ …) = P(A₁) + P(A₂) + …
From these three axioms alone (without any other assumptions) we can derive the complement rule, the addition rule, Bayes' theorem, and the properties of every distribution covered in this section. The axioms are minimal and beautiful.
-
CComplement RuleP(Aᶜ) = 1 − P(A)
The probability that A does not occur equals one minus the probability that it does.
-
∪Addition RuleP(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Subtract the intersection to avoid double-counting events in both A and B.
-
∩Multiplication RuleP(A ∩ B) = P(A) · P(B|A)
The probability that both A and B occur equals the probability of A times the conditional probability of B given A.
-
⊥IndependenceA ⊥ B ⟺ P(A ∩ B) = P(A) · P(B)
Events are independent when knowing one occurs tells you nothing about the other. Do not confuse independence with mutual exclusivity, they are almost opposites.
§ 02All Topics in This Section
Nine self-contained pages, each with full theory, worked examples, and an interactive quiz. They are ordered by dependency: earlier topics are used in later ones.
Sample spaces, events, the three Kolmogorov axioms, equally likely outcomes, and counting techniques. The vocabulary and rules every subsequent topic depends on, start here if you are new to probability.
P(A|B): the probability of A given that B has occurred. Tree diagrams, the multiplication rule for dependent events, and the independence condition. Essential before Bayes' theorem.
Reversing conditional probabilities: from P(B|A) to P(A|B). Prior, likelihood, posterior. The medical testing problem, false positives, and the law of total probability.
Probability mass functions, cumulative distribution functions, and the definitions of E[X] and Var(X) for discrete random variables. The gateway to all named discrete distributions.
Counting successes in n independent trials. The PMF C(n,k)pᵏ(1−p)ⁿ⁻ᵏ, mean np, variance np(1−p), and the normal approximation with continuity correction.
Modelling rare events: calls to a helpdesk, radioactive decay, traffic arrivals. P(X=k) = λᵏe⁻λ/k!, mean = variance = λ, and the Poisson limit of the binomial.
The bell curve: the PDF formula, 68–95–99.7 rule, standardising to Z-scores, reading z-tables, inverse normal, and why so many natural phenomena follow this shape.
PDFs and CDFs for continuous random variables. Uniform distribution, exponential distribution, and the relationship between them. Computing probabilities as areas under the density curve.
E[X] = Σx·P(X=x). Var(X) = E[X²] − (E[X])². Linearity of expectation, properties of variance, standard deviation, covariance, correlation coefficient, and the Cauchy-Schwarz inequality.
§ 03The Distributions at a Glance
A quick-reference card for each named distribution: its formula, mean, and variance: so you can identify which to use at a glance.
§ 04Recommended Learning Path
If you are working through probability for the first time, follow this order. Each topic builds directly on the one before it.
Start here. Learn sample spaces, events, and the three axioms. Without this, nothing else makes sense.
Learn P(A|B) and why dependence between events matters. Includes tree diagrams and the multiplication rule.
The most important formula in probability. Used everywhere from spam filters to medical testing.
The framework: PMF, CDF, E[X], Var(X). Read this before any named discrete distribution.
The workhorse of discrete probability. Counting successes in repeated independent trials.
Rare events over time or space. Follows naturally from the binomial as p → 0 and n → ∞.
Continuous probability and the Central Limit Theorem. The bridge between discrete models and statistical inference.
PDFs, CDFs, and the uniform and exponential distributions. Extends the discrete framework to continuous random variables.
The theoretical capstone. Linearity of expectation, covariance, correlation, the tools used in statistics and machine learning.
§ 05Key Formulae at a Glance
A rapid-reference collection of the most important formulas across the entire probability section. Useful when revising before an exam.
§ 06Where Probability Meets Statistics
Probability is the theory; statistics is the application. Once you master distributions and expected values, the following statistics topics open up naturally.
The normal distribution and the Central Limit Theorem are the bridge between probability and inferential statistics. When you understand that a sample mean follows an approximately normal distribution regardless of the underlying population shape (for large enough samples), hypothesis testing and confidence intervals become straightforward applications of z-scores and t-scores.