Probability · Random Variables

Expected Value
& Variance

The mean, the spread, and everything in between, a complete treatment of E[X], Var[X], covariance, correlation, and the properties that make them so powerful.

E[X] & Var[X] Linearity of Expectation Covariance Correlation Standard Deviation
Share this page
Contents
  1. What is Expected Value?
  2. Expected Value, Formal Definition & Properties
  3. Variance & Standard Deviation
  4. Interactive Visualiser
  5. Computational Formulas & Quick Reference
  6. Covariance
  7. Correlation
  8. Worked Examples
  9. Common Mistakes
  10. 10-Question Quiz
  11. Continue in Probability

§ 01What Is Expected Value?

Expected value is the long-run average. It answers: if you repeated this experiment infinitely many times and averaged all the outcomes, what number would you converge to?

Suppose you roll a fair six-sided die. The possible outcomes are 1, 2, 3, 4, 5, 6, each equally likely. If you roll 600 times, you expect roughly 100 of each. The average score you expect is (1+2+3+4+5+6)/6 = 3.5. This is the expected value E[X], the probability-weighted average of all possible outcomes.

Expected value is also called the mean, the population mean, or the first moment of the distribution. It is denoted E[X], μ (mu), or ⟨X⟩ depending on the textbook. Throughout this page we use E[X] and μ interchangeably.

Intuition: A centre of gravity Think of the PMF (or PDF) as a mass distribution along the number line. The expected value E[X] is the centre of gravity — the point at which the distribution would balance perfectly if placed on a fulcrum. A distribution skewed right has its E[X] pulled toward the right tail; a symmetric distribution has E[X] at its axis of symmetry.

One important subtlety: the expected value need not be a value that X can actually take. The die example gives E[X] = 3.5, which is not a possible roll. Expected value describes the distribution as a whole, not any single outcome.

Expected value is defined for both discrete and continuous random variables. The formulas look different, a sum versus an integral, but the underlying idea is identical: a probability-weighted average.

§ 02Expected Value — Definition & Properties

Discrete random variable

For a discrete random variable X taking values in a countable set:

Definition — E[X] (Discrete)
E[X] = μ = Σx x · P(X = x)

The sum runs over every value x in the range of X. Each value x is weighted by its probability P(X = x).

Continuous random variable

For a continuous random variable X with probability density function (PDF) f(x):

Definition — E[X] (Continuous)
E[X] = μ = ∫−∞∞ x · f(x) dx

The integral plays the role of the sum: it weights each value x by its density f(x). You will need this form when studying the normal distribution, exponential distribution, and others.

Expected value of a function — LOTUS

A theorem called the Law of the Unconscious Statistician (LOTUS) lets us compute E[g(X)] without first finding the distribution of g(X):

LOTUS — Discrete
E[g(X)] = Σx g(x) · P(X = x)

This is essential for computing variance (where g(x) = (x − μ)² or g(x) = x²) and for moments in general.

Linearity of Expectation

The single most important property of expected value is its linearity. It holds regardless of whether X and Y are independent:

Linearity of Expectation
E[aX + b] = a·E[X] + b E[X + Y] = E[X] + E[Y] E[aX + bY + c] = a·E[X] + b·E[Y] + c
Why linearity is so powerful Linearity of expectation means you can break a complicated random variable into simple pieces, compute each piece's expectation, and add them up. You never need to worry about dependence between the pieces. This technique — sometimes called the "indicator trick" — is behind elegant solutions to problems like the expected number of fixed points in a random permutation.

Non-linearity for products

Expectation is not linear for products unless X and Y are independent:

E[XY] — Independence Required
E[XY] = E[X] · E[Y]   only if X ⊥ Y (independent)

In general, E[XY] = E[X]E[Y] + Cov(X, Y), where Cov is the covariance (covered in §06).

§ 03Variance & Standard Deviation

Variance measures how spread out a distribution is, how far values of X tend to stray from the mean. Standard deviation is its square root, expressed in the same units as X.

Definition

Definition — Variance
Var[X] = σ² = E[(X − μ)²]

Expanded using LOTUS for a discrete X:

Variance — Discrete Sum Form
Var[X] = Σx (x − μ)² · P(X = x)

The Computational Shortcut

Expanding (x − μ)² = x² − 2μx + μ² and applying linearity of expectation yields the most practical formula:

Computational Formula — always faster
Var[X] = E[X²] − (E[X])²

In words: mean of the squares minus the square of the mean. This avoids computing each (x − μ)² separately and is the standard approach in virtually all exam calculations.

Standard Deviation

Standard Deviation
σ = SD[X] = √Var[X] = √(E[X²] − (E[X])²)

Standard deviation is always non-negative. It equals zero if and only if X is a constant (a degenerate random variable with no randomness). The standard deviation is in the same units as X, making it the preferred measure of spread when communicating results.

Properties of Variance

Variance Properties
Var[aX + b] = a² · Var[X] Var[X + Y] = Var[X] + Var[Y]   (only if X ⊥ Y) Var[X + Y] = Var[X] + Var[Y] + 2Cov(X,Y)   (general) Var[X] ≥ 0   always Var[c] = 0   for any constant c

The most important point above: adding a constant does not change variance. Only multiplying by a constant changes spread: and the effect is squared (multiply by a, variance multiplies by a²).

Var[X + Y] ≠ Var[X] + Var[Y] in general This formula only holds when X and Y are independent. For dependent random variables, the covariance term appears. Forgetting this is one of the most common errors in probability exams.

Variance for Continuous Random Variables

Variance — Continuous Form
Var[X] = ∫−∞∞ (x − μ)² f(x) dx = E[X²] − (E[X])²

The computational shortcut E[X²] − (E[X])² applies equally for continuous random variables: only the method of computing E[X²] changes (integral instead of sum).

§ 04Interactive Visualiser

The chart below shows a custom discrete distribution over the values 1 through 8. Adjust the sliders to reshape the distribution and watch how E[X], Var[X], and σ respond in real time. Notice how spreading probability into the tails increases variance while keeping E[X] fixed when the distribution remains symmetric.

Interactive · Discrete Distribution Visualiser

Reshape the distribution — observe E[X] and Var[X] live

E[X] — Mean
—
E[X²]
—
Var[X] = E[X²]−μ²
—
σ — Std Dev
—

Experiment: set all weights equal to see a uniform distribution. Then set all weight on a single value to see a degenerate distribution with Var[X] = 0. Try concentrating probability at the extremes (values 1 and 8) to maximise variance.

§ 05Quick Reference — Formulas for Named Distributions

Here are the E[X] and Var[X] formulas for the most common discrete distributions you will encounter. Each is derived by applying the definitions above to the specific PMF.

Distribution Parameters E[X] Var[X]
Discrete Uniform {a, a+1, …, b} (a + b) / 2 ((b−a+1)² − 1) / 12
Binomial n, p np np(1−p)
Poisson λ λ λ
Geometric p 1/p (1−p) / p²
Negative Binomial r, p r/p r(1−p) / p²
Hypergeometric N, K, n nK/N nK(N−K)(N−n) / N²(N−1)
Uniform (cts) [a, b] (a + b) / 2 (b − a)² / 12
Exponential (cts) λ 1/λ 1/λ²
Normal (cts) μ, σ² μ σ²
E[X] — Discrete
Σ x · P(X = x)
Sum over all values in the range of X.
E[X] — Continuous
∫ x · f(x) dx
Integral over the support of the PDF.
Var[X] — Shortcut
E[X²] − (E[X])²
Always use this in calculations — it's fastest.
Linear Transform
E[aX+b] = aμ+b
Var[aX+b] = a²σ²
Constant b shifts E[X], does not affect Var[X].
Sum of Independent RVs
E[X+Y] = E[X]+E[Y]
Var[X+Y] = Var[X]+Var[Y]
Variance additivity requires independence.
LOTUS
E[g(X)] = Σ g(x)·P(X=x)
No need to find the distribution of g(X) first.

§ 06Covariance — How Two Variables Move Together

Covariance measures the degree to which two random variables X and Y move in the same direction. Positive covariance means they tend to be above their means together; negative covariance means one tends to be above when the other is below.

Definition — Covariance
Cov(X, Y) = E[(X − μX)(Y − μY)]

Computational formula for covariance

Expanding the definition using LOTUS yields the computationally convenient form:

Cov — Computational Shortcut
Cov(X, Y) = E[XY] − E[X]·E[Y]

Notice that if X and Y are independent, then E[XY] = E[X]·E[Y] and so Cov(X, Y) = 0. The converse is not always true: zero covariance does not imply independence (it only implies uncorrelatedness).

Properties of Covariance

Covariance Properties
Cov(X, X) = Var[X] Cov(X, Y) = Cov(Y, X) Cov(aX + b, cY + d) = ac · Cov(X, Y) Var[X + Y] = Var[X] + Var[Y] + 2Cov(X,Y) Var[X − Y] = Var[X] + Var[Y] − 2Cov(X,Y)

The first property is elegant: the covariance of a variable with itself is its variance. The last two generalize the variance addition rule for sums and differences of any two random variables, dependent or not.

General sum formula

For a linear combination of n random variables X₁, X₂, …, Xₙ:

Variance of a General Sum
Var[Σ aᵢXᵢ] = Σ aᵢ² Var[Xᵢ] + 2 Σi<j aᵢaⱼ Cov(Xᵢ,Xⱼ)

When all variables are mutually independent, all covariance terms vanish and the formula reduces to Var[Σ aᵢXᵢ] = Σ aᵢ² Var[Xᵢ].

Covariance has units If X is measured in metres and Y in seconds, then Cov(X, Y) is in metre-seconds. This makes covariance hard to interpret directly. Correlation (§07) solves this by standardising covariance to a dimensionless number between −1 and 1.

§ 07Correlation — Standardised Association

The Pearson correlation coefficient ρ (rho) standardises covariance so it lies between −1 and 1, making it directly interpretable as a measure of linear association:

Definition — Pearson Correlation
ρ(X, Y) = Cov(X, Y) / (σX · σY)

Interpreting ρ

Value of ρInterpretation
ρ = +1Perfect positive linear relationship: Y = aX + b with a > 0
0 < ρ < 1Positive association: X above its mean tends to go with Y above its mean
ρ = 0No linear association (but could still be a nonlinear relationship)
−1 < ρ < 0Negative association: X above its mean tends to go with Y below its mean
ρ = −1Perfect negative linear relationship: Y = aX + b with a < 0

Properties of Correlation

  • −1 ≤ ρ(X, Y) ≤ 1 always (follows from the Cauchy-Schwarz inequality)
  • ρ(X, Y) = ρ(Y, X), it is symmetric
  • ρ(aX + b, cY + d) = sign(ac) · ρ(X, Y): correlation is unaffected by linear transformations except for sign changes
  • If X and Y are independent, then ρ = 0. The converse is false in general.
Correlation ≠ Causation A high correlation between X and Y tells you they move together linearly. It says nothing about which causes which, or whether a third variable drives both. This distinction is critical in statistical reasoning and is covered in depth in Regression Analysis.
✦   ✦   ✦

§ 08Worked Examples

Twelve fully worked examples covering every formula and technique on this page.

Example 01 — Basic E[X] from a table
A random variable X has distribution: P(X=0)=0.1, P(X=1)=0.4, P(X=2)=0.3, P(X=3)=0.2. Compute E[X].
  1. E[X] = Σ x · P(X=x)
  2. E[X] = 0(0.1) + 1(0.4) + 2(0.3) + 3(0.2)
  3. = 0 + 0.4 + 0.6 + 0.6 = 1.6
E[X] = 1.6
Example 02 — Variance using the shortcut
Using the distribution from Example 1, compute Var[X] and σ.
  1. First compute E[X²] via LOTUS (g(x) = x²):
    E[X²] = 0²(0.1) + 1²(0.4) + 2²(0.3) + 3²(0.2) = 0 + 0.4 + 1.2 + 1.8 = 3.4
  2. Apply the shortcut:
    Var[X] = E[X²] − (E[X])² = 3.4 − (1.6)² = 3.4 − 2.56 = 0.84
  3. Standard deviation: σ = √0.84 ≈ 0.917
Var[X] = 0.84,   σ ≈ 0.917
Example 03 — Linearity of expectation
Given E[X] = 3 and E[Y] = 7, find E[4X − 2Y + 5].
  1. Apply linearity of expectation (no independence required):
  2. E[4X − 2Y + 5] = 4·E[X] − 2·E[Y] + 5
  3. = 4(3) − 2(7) + 5 = 12 − 14 + 5 = 3
E[4X − 2Y + 5] = 3
Example 04 — Variance of a linear combination (independent)
X and Y are independent with Var[X] = 4 and Var[Y] = 9. Find Var[3X − 2Y + 1].
  1. Var[aX + bY + c] = a²Var[X] + b²Var[Y] when X, Y independent (constants vanish):
  2. Var[3X − 2Y + 1] = 3²·Var[X] + (−2)²·Var[Y]
  3. = 9(4) + 4(9) = 36 + 36 = 72
  4. Note: the +1 contributes nothing to variance.
Var[3X − 2Y + 1] = 72
Example 05 — Variance with covariance (dependent)
Var[X] = 5, Var[Y] = 3, Cov(X,Y) = −2. Find Var[X + Y] and Var[X − Y].
  1. Var[X + Y] = Var[X] + Var[Y] + 2Cov(X,Y):
    = 5 + 3 + 2(−2) = 8 − 4 = 4
  2. Var[X − Y] = Var[X] + Var[Y] − 2Cov(X,Y):
    = 5 + 3 − 2(−2) = 8 + 4 = 12
Var[X+Y] = 4,   Var[X−Y] = 12
Example 06 — Computing covariance from a joint table
X and Y have joint PMF: P(0,0)=0.1, P(0,1)=0.2, P(1,0)=0.3, P(1,1)=0.4. Compute Cov(X,Y).
  1. E[X] = 0(0.1+0.2) + 1(0.3+0.4) = 0 + 0.7 = 0.7
  2. E[Y] = 0(0.1+0.3) + 1(0.2+0.4) = 0 + 0.6 = 0.6
  3. E[XY] = Σ xy·P(X=x,Y=y) = 0(0.1)+0(0.2)+0(0.3)+1·1(0.4) = 0.4
  4. Cov(X,Y) = E[XY] − E[X]E[Y] = 0.4 − (0.7)(0.6) = 0.4 − 0.42 = −0.02
Cov(X,Y) = −0.02
Example 07 — Correlation coefficient
Cov(X,Y) = 6, Var[X] = 9, Var[Y] = 16. Find ρ(X,Y).
  1. σX = √9 = 3,   σY = √16 = 4
  2. ρ(X,Y) = Cov(X,Y) / (σX·σY) = 6 / (3·4) = 6/12 = 0.5
ρ(X,Y) = 0.5 — moderate positive linear association
Example 08 — E[X²] and the indicator trick
X is the score on a fair 4-sided die (values 1, 2, 3, 4 equally likely). Find E[X²] and Var[X].
  1. E[X] = (1+2+3+4)/4 = 10/4 = 2.5
  2. E[X²] = (1²+2²+3²+4²)/4 = (1+4+9+16)/4 = 30/4 = 7.5
  3. Var[X] = E[X²] − (E[X])² = 7.5 − 6.25 = 1.25
  4. σ = √1.25 = √(5/4) = √5/2 ≈ 1.118
E[X²] = 7.5,   Var[X] = 1.25,   σ = √5/2
Example 09 — Standardising a random variable
X has E[X] = 10 and Var[X] = 25. Define Z = (X − 10)/5. Find E[Z] and Var[Z].
  1. E[Z] = E[(X−10)/5] = (E[X] − 10)/5 = (10−10)/5 = 0
  2. Var[Z] = Var[(X−10)/5] = (1/5)²·Var[X] = (1/25)·25 = 1
  3. Z is the standardised version of X: mean 0, variance 1. This is the foundation of z-scores used in the normal distribution.
E[Z] = 0,   Var[Z] = 1
Example 10 — Sum of independent Binomials
X ~ B(10, 0.3) and Y ~ B(15, 0.3) are independent. Find E[X+Y] and Var[X+Y].
  1. E[X] = np = 10(0.3) = 3;   E[Y] = 15(0.3) = 4.5
  2. E[X+Y] = E[X]+E[Y] = 3 + 4.5 = 7.5
  3. Var[X] = np(1−p) = 10(0.3)(0.7) = 2.1
  4. Var[Y] = 15(0.3)(0.7) = 3.15
  5. Since X and Y are independent: Var[X+Y] = 2.1 + 3.15 = 5.25
  6. Note: X+Y ~ B(25, 0.3), the sum of independent Binomials with the same p is also Binomial.
E[X+Y] = 7.5,   Var[X+Y] = 5.25
Example 11 — LOTUS applied to an exponential function
X takes values 1, 2, 3 with equal probability 1/3. Find E[2X].
  1. Apply LOTUS with g(x) = 2x:
  2. E[2X] = 2¹·(1/3) + 2²·(1/3) + 2³·(1/3)
  3. = (2 + 4 + 8)/3 = 14/3 ≈ 4.667
  4. Note: E[2X] ≠ 2E[X] = 2² = 4. Jensen's inequality says E[g(X)] ≥ g(E[X]) for convex g.
E[2X] = 14/3 ≈ 4.667
Example 12 — Full analysis: E[X], Var[X], Cov, ρ from scratch
A game pays £X where X takes value 2 with probability 0.6 and value −1 with probability 0.4 (a loss). Y = 3X + 5. Find E[X], Var[X], E[Y], Var[Y], and ρ(X,Y).
  1. E[X] = 2(0.6) + (−1)(0.4) = 1.2 − 0.4 = 0.8
  2. E[X²] = 4(0.6) + 1(0.4) = 2.4 + 0.4 = 2.8
  3. Var[X] = 2.8 − 0.64 = 2.16;   σX = √2.16 ≈ 1.470
  4. E[Y] = E[3X+5] = 3(0.8)+5 = 7.4
  5. Var[Y] = Var[3X+5] = 9·Var[X] = 9(2.16) = 19.44;   σY = 3σX ≈ 4.409
  6. Cov(X,Y) = Cov(X, 3X+5) = 3·Cov(X,X) = 3·Var[X] = 3(2.16) = 6.48
  7. ρ(X,Y) = Cov(X,Y)/(σX·σY) = 6.48/(1.470·4.409) ≈ 6.48/6.48 = 1
  8. ρ = 1 because Y = 3X+5 is a perfect positive linear function of X: no surprise!
E[X]=0.8, Var[X]=2.16, E[Y]=7.4, Var[Y]=19.44, ρ(X,Y)=1

§ 09Common Mistakes

Mistake 1 — Forgetting to square E[X] in the variance shortcut The formula is Var[X] = E[X²] − (E[X])². The square applies to the entire expected value, not just to X. Writing E[X²] − E[X²] or E[X²] − E[X]² (misread as E[X]·2) will cost marks.
Mistake 2 — Adding variances for dependent variables Var[X+Y] = Var[X] + Var[Y] only when X and Y are independent. If there is dependence, you must include the covariance term: Var[X+Y] = Var[X] + Var[Y] + 2Cov(X,Y).
Mistake 3 — Applying linearity to products: E[XY] ≠ E[X]·E[Y] Linearity of expectation gives E[X+Y] = E[X]+E[Y] always. But E[XY] = E[X]·E[Y] only holds under independence. When independence is not stated, compute E[XY] directly from the joint distribution.
Mistake 4 — Claiming ρ = 0 implies independence Zero correlation means no linear association. Two variables can be highly dependent yet have ρ = 0 (e.g., Y = X² when X is symmetric around zero). Always distinguish between uncorrelated and independent.
Mistake 5 — E[g(X)] = g(E[X]) for nonlinear g This equality only holds for linear g. For any convex function g, Jensen's inequality says E[g(X)] ≥ g(E[X]). For example, E[X²] ≥ (E[X])² — which is just the statement that variance is non-negative.

§ 1010-Question Quiz

Select one answer per question, then click Check All Answers to see your score and full explanations.

Score: 0 / 10
Question 01
X has P(X=1)=0.2, P(X=2)=0.5, P(X=3)=0.3. What is E[X]?
Question 02
If E[X] = 5 and E[X²] = 30, what is Var[X]?
Question 03
E[X] = 3, E[Y] = −2. What is E[5X + 3Y − 4]?
Question 04
Var[X] = 9. What is Var[4X − 7]?
Question 05
X and Y are independent with Var[X] = 4, Var[Y] = 6. What is Var[X − Y]?
Question 06
Var[X] = 5, Var[Y] = 3, Cov(X,Y) = 2. What is Var[X + Y]?
Question 07
Which statement about Cov(X,Y) = 0 is always true?
Question 08
Cov(X,Y) = −4, σX = 2, σY = 4. What is ρ(X,Y)?
Question 09
X takes values 1, 2, 3 each with probability 1/3. What is E[X²]?
Question 10
If Y = −2X + 3, what is ρ(X,Y)?
—

§ 11Continue in Probability & Beyond

Expected value and variance are the foundation of almost every topic that follows: from named distributions to statistical inference and regression. Here are the most important next steps.

You are here Expected Value & Variance E[X], Var[X], covariance, correlation ← Prerequisite Discrete Distributions PMF, CDF — where E[X] and Var[X] come from Probability Binomial Distribution E[X]=np, Var[X]=np(1−p) applied Probability Poisson Distribution E[X]=Var[X]=λ — a unique property Probability Normal Distribution Parameters μ and σ² are exactly E[X] and Var[X] Probability Continuous Distributions E[X] and Var[X] via integration (LOTUS) Probability Basic Probability Sample spaces and probability axioms Probability Bayes' Theorem Updating probability with evidence Statistics → Hypothesis Testing E[X] and σ underpin every statistical test Statistics → Confidence Intervals Standard error = σ/√n — directly uses Var[X] Statistics → Linear Regression Slope β₁ = Cov(X,Y)/Var[X] Probability Conditional Probability Leads to conditional expectation E[X|Y]

Cookie Settings