Statistics — Complete Varsity Guide | MathsTricks & Calculus

§ 01

What Is Statistics — and Why Does It Matter?

Probability tells you what to expect from a known process. Statistics reverses the question: given data you have already collected, what can you infer about the process that generated it?

Every scientific paper, every clinical trial, every opinion poll, and every quality-control system in a factory relies on statistical reasoning. The core problem is always the same: you have a sample (a finite, imperfect window onto a much larger population) and you want to draw reliable conclusions despite the noise.

Statistics gives you the tools to do this honestly. Descriptive statistics summarise what you actually observed: means, medians, spreads, and shapes. Inferential statistics go further: they let you make claims about the whole population with a quantified level of confidence, and they tell you whether a pattern in your data is likely to be real or just the result of random chance.

At varsity level, the central limit theorem is the engine underneath most of these tools. It guarantees that sample means follow a normal distribution, regardless of what the underlying population looks like, as long as the sample is large enough. This single result is why the normal distribution appears in almost every inferential procedure.

Prerequisites. You should be comfortable with basic probability — particularly the normal distribution and the concept of expected value — before working through the inferential topics in this section. The Probability hub is the right place to start if you need a refresher.

Statistics vs Probability — the two directions of reasoning

🎲 Probability

Known model → predict outcomes
Deductive reasoning
P(data | model)
e.g. If a coin is fair, what is P(≥ 7 heads in 10 flips)?
Covered in the Probability hub

📊 Statistics

Observed data → infer model
Inductive reasoning
P(model | data)
e.g. We got 7 heads in 10 flips: is the coin fair?
Covered right here

§ 02

All 5 Topics in This Section

The topics follow a natural progression, from describing data you have, to making inferences about data you have not seen, to modelling relationships between variables.

Topic 01 · Foundation

Descriptive Statistics

Summarise any dataset with numbers. Mean, median, mode, range, interquartile range, variance, standard deviation, skewness, and box plots — everything you need to describe what you see before making any inferences.

Mean & MedianStd DeviationBox PlotsSkewness

Topic 02 · Inference Foundation

Sampling & Estimation

Move from sample to population. Point estimators, unbiasedness, consistency, the Central Limit Theorem, and sampling distributions — the theoretical bedrock of all inferential statistics.

CLTPoint EstimatorsSampling Dist.Unbiasedness

Topic 03 · Inference

Hypothesis Testing

Decide whether data supports a claim. Null and alternative hypotheses, Type I & II errors, significance level α, p-values, z-tests and t-tests, one-tailed vs two-tailed tests — the complete framework.

p-valuez-testt-testType I & II Errors

Topic 04 · Inference

Confidence Intervals

Estimate population parameters with a margin of error. The CI formula x̄ ± z*(σ/√n), interpreting what 95% confidence actually means, t-distribution for small samples, and margin of error calculations.

95% CIMargin of Errort-dist.z-scores

Topic 05 · Modelling

Regression Analysis

Fit a straight line through data using least squares. Derive the β₀ and β₁ formulas, measure goodness of fit with R², interpret residuals, and distinguish prediction from interpolation.

Least SquaresR²ResidualsLinear Model

§ 03

Key Formulas at a Glance

These are the formulas that appear most frequently across all five topics. Knowing them cold will save you time in exams.

Sample Mean

x̄ = (1/n) Σ xᵢ

Sum of all observations divided by the count. The most natural measure of centre.

Sample Variance

s² = Σ(xᵢ − x̄)² / (n−1)

Dividing by n−1 (not n) makes s² an unbiased estimator of the population variance σ².

Standard Error

SE = σ / √n (or s / √n)

How much the sample mean varies across repeated samples. Shrinks as sample size grows.

z-Score

z = (x̄ − μ₀) / (σ/√n)

How many standard errors the sample mean sits from the hypothesised population mean.

Confidence Interval (z)

x̄ ± z* · (σ/√n)

Use z* = 1.96 for 95%, z* = 2.576 for 99%. Switch to t* when σ is unknown.

Least-Squares Slope

β̂₁ = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)²

The slope of the best-fit line. Paired with β̂₀ = ȳ − β̂₁x̄ for the intercept.

Coefficient of Determination

R² = 1 − SSres/SStot

Proportion of variance in y explained by the regression. R² = 1 is a perfect fit.

Central Limit Theorem

X̄ ~ N(μ, σ²/n) as n → ∞

The sample mean is approximately normally distributed for large n, regardless of the population shape.

§ 04

The Normal Distribution — Interactive Explorer

The normal (Gaussian) distribution underpins almost every formula in this section. Adjust the mean μ and standard deviation σ below to see how the shape changes, and observe where the 68%, 95%, and 99.7% regions fall.

Normal Curve Explorer

Drag the sliders, the shaded region shows ±1σ, ±2σ, and ±3σ from the mean.

Mean μ = 0

Std Dev σ = 1.0

Shade region

The 68-95-99.7 rule is worth memorising: approximately 68% of data falls within 1 standard deviation of the mean, 95% within 2 standard deviations, and 99.7% within 3. These numbers drive almost all hypothesis testing and confidence interval calculations: they tell you exactly how unusual a given result is.

§ 05

Recommended Learning Path

If you are working through this section for the first time, follow this order. Each step builds directly on the last.

Descriptive Statistics — Summarising Data with Numbers

Start here to understand how to describe a dataset before making any inferences. Mean, variance, IQR, and box plots.

Sampling & Estimation — From Sample to Population

Learn the Central Limit Theorem and why sample means follow a normal distribution: the engine of all inferential statistics.

Hypothesis Testing — Making Decisions from Data

The full framework: H₀ and H₁, p-values, z-tests and t-tests, Type I & II errors, significance levels.

Confidence Intervals — Estimating Parameters with Uncertainty

The interval estimate counterpart to hypothesis testing. Understand exactly what 95% confidence means: and what it does not.

Regression Analysis — Fitting a Line Through Data

Move from summarising a single variable to modelling the relationship between two. Least squares, R², residuals.

§ 06

How Statistics Connects to the Rest of the Curriculum

Probability is the direct prerequisite. The normal distribution, expected value, variance, and the concept of a random variable are all covered in the Probability hub and are used constantly here. Hypothesis testing is essentially an application of conditional probability, and the p-value is a probability statement.

Integration underlies every probability density function. The area under a normal curve between two points is computed by integration: in practice you use z-tables, but understanding where those values come from requires improper integrals. The expected value E[X] = ∫ x f(x) dx is a direct application of integration to statistics.

Linear algebra becomes essential once you move beyond simple linear regression to multiple regression or principal component analysis. Regression in matrix form is Ax = b solved by the normal equations, and PCA relies on eigenvalues and eigenvectors: both covered in the Linear Algebra hub.

Calculus of optimisation is what produces the least-squares formulas: you minimise the sum of squared residuals by taking derivatives and setting them to zero, exactly as in the Optimisation page.

Exam tip. The two most commonly tested topics at first-year level are hypothesis testing (especially choosing the right test statistic and interpreting p-values) and confidence intervals. Make sure you understand the difference between a z-interval (σ known) and a t-interval (σ unknown), and know when to use each.

Related sections on this site

§ 07

Frequently Asked Questions

The p-value is the probability of observing a test statistic at least as extreme as the one you computed, assuming the null hypothesis H₀ is true. A small p-value (typically p < 0.05) means your data would be very unusual if H₀ were true — so you reject H₀. Crucially, the p-value is not the probability that H₀ is true, and it is not the probability that your result occurred by chance. It is a conditional probability about the data, given the model. Full details in Hypothesis Testing.

A 95% CI does not mean "there is a 95% probability that μ lies in this interval." The true population mean μ is fixed (not random) — it either is or is not in the interval. What the 95% refers to is the procedure: if you repeated the sampling and interval construction many times, approximately 95% of the intervals produced would contain μ. Any single interval either covers μ or it does not. See Confidence Intervals for a full worked explanation.

Use a z-test when the population standard deviation σ is known, or when the sample size is large (n > 30) so that the sample standard deviation s is a good estimate of σ. Use a t-test when σ is unknown and the sample is small (n ≤ 30). The t-distribution has heavier tails than the normal to account for the extra uncertainty — as the degrees of freedom increase, the t-distribution converges to the standard normal.

When you compute Σ(xᵢ − x̄)², you are measuring deviations from the sample mean x̄, not the true population mean μ. The sample mean x̄ is pulled towards the data in a way that μ is not — it minimises the sum of squared deviations. This causes the sum to systematically underestimate the true variability. Dividing by n−1 instead of n corrects for this bias, giving E[s²] = σ². The n−1 is called the degrees of freedom.

Correlation (Pearson's r) measures the strength and direction of a linear relationship between two variables — it is symmetric (r(x,y) = r(y,x)) and has no units. Regression goes further: it quantifies exactly how much y changes for a one-unit increase in x (the slope β₁), and produces a model you can use for prediction. Regression is asymmetric — swapping x and y gives a different line. A high correlation justifies fitting a regression line; the regression line tells you the exact nature of the relationship. See Regression Analysis.

The Central Limit Theorem (CLT) states that the sampling distribution of the sample mean X̄ approaches a normal distribution N(μ, σ²/n) as the sample size n increases — regardless of the shape of the original population. In practice, n ≥ 30 is usually sufficient. This is the reason the normal distribution dominates statistics: even if individual observations are skewed, uniform, or exponentially distributed, their averages are approximately normal for reasonably large samples. Without the CLT, the z-test, t-test, and confidence interval formulas would only work for normally distributed populations.