Probability Cheat Sheet — Bayes, Distributions, Expected Value | Dataplexa

Probability

Bayes' theorem · conditional probability · distributions · expected value · counting · Bayesian vs frequentist

Sheet 2 of 6 Stats & Math Beginner–Inter. Printable

Probability Fundamentals

must know first

Basic Probability

P(A) = favourable outcomes total outcomes

Range: 0 ≤ P(A) ≤ 1 · P(impossible) = 0 · P(certain) = 1

Complement Rule

P(A') = 1 − P(A)

The probability of A NOT happening. P(A) + P(A') = 1 always.

Addition Rule

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Mutually exclusive: P(A ∩ B) = 0, so P(A ∪ B) = P(A) + P(B).

Multiplication Rule

P(A ∩ B) = P(A) · P(B|A)

Independent events: P(A ∩ B) = P(A) · P(B).

Conditional Probability

P(B|A) = P(A ∩ B) P(A)

Probability of B given A has already occurred. P(A) must be > 0.

Independence

A ⊥ B ⟺ P(B|A) = P(B)

Knowing A happened gives no information about B. Coin flips are independent.

Quick Example

Deck of 52 cards:

P(Ace)        = 4/52  = 0.077
P(Heart)      = 13/52 = 0.25
P(Ace∩Heart) = 1/52
P(Ace∪Heart)
  = 4/52 + 13/52 - 1/52
  = 16/52 ≈ 0.308

P(Ace|Heart)
  = P(Ace∩Heart) / P(Heart)
  = (1/52) / (13/52)
  = 1/13 ≈ 0.077

Mutually exclusive vs Independent: Mutually exclusive means they can't both happen (P(A∩B)=0). Independent means one doesn't affect the other. Mutually exclusive events are never independent (unless one has P=0).

Bayes' Theorem

prior · likelihood · posterior

Bayes' Theorem

P(A|B) = P(B|A) · P(A) P(B)

P(A) = prior · P(B|A) = likelihood · P(A|B) = posterior · P(B) = evidence

Total Probability (expand P(B))

P(B) = P(B|A) · P(A) + P(B|A') · P(A')

Law of Total Probability — when you have two exhaustive, mutually exclusive scenarios.

Medical Test Example

Disease prevalence: 1% → P(D)=0.01
Test sensitivity:  99% → P(+|D)=0.99
Test specificity:  95% → P(-|D')=0.95
  → P(+|D')=0.05 (false positive rate)

P(+) = P(+|D)·P(D) + P(+|D')·P(D')
     = 0.99×0.01 + 0.05×0.99
     = 0.0099 + 0.0495 = 0.0594

P(D|+) = P(+|D)·P(D) / P(+)
       = 0.0099 / 0.0594
       ≈ 16.7%

Even with a positive test, only ~17%
chance of disease (base rate matters!)

Term	Meaning	In example
Prior	Belief before seeing evidence	P(D) = 1%
Likelihood	How probable is evidence given H	P(+\|D) = 99%
Evidence	Marginal probability of data	P(+) = 5.94%
Posterior	Updated belief after evidence	P(D\|+) ≈ 17%

Base rate neglect is a common error — always consider the prior probability. A rare disease + high-sensitivity test still means most positives are false.

Counting — Permutations & Combinations

n! · P(n,r) · C(n,r)

Factorial

n! = n × (n−1) × ··· × 1 0! = 1

5! = 5×4×3×2×1 = 120 · Number of ways to arrange n distinct objects.

Permutations — order matters

P(n,r) = n! (n−r)!

Arrange r items from n. Order matters: ABC ≠ BAC. P(5,3) = 60.

Combinations — order doesn't matter

C(n,r) = n! r!(n−r)!

Choose r items from n. Order doesn't matter: ABC = BAC. C(5,3) = 10. Also written ⁿCᵣ or (n choose r).

Expected Value & Variance

E(X) · Var(X) · Std Dev

Expected Value (Discrete)

E(X) = Σ x_i · P(X=x_i)

Weighted average of all possible values — the long-run average outcome.

Variance of Random Variable

Var(X) = E[(X−μ)²] = E(X²) − [E(X)]²

Standard deviation of X = √Var(X). Measures spread of the distribution.

Example — Fair Die

X = outcome of rolling 1 die
P(X=k) = 1/6 for k = 1,2,3,4,5,6

E(X) = 1×1/6 + 2×1/6 + ... + 6×1/6
     = 21/6 = 3.5

E(X²) = 1+4+9+16+25+36/6 = 91/6
Var(X) = 91/6 - 3.5² = 2.917
SD(X)  = √2.917 ≈ 1.71

Probability Distributions

discrete · continuous

Binomial B(n, p)

P(X=k) = C(n,k) · p^k(1−p)^n−k

n trials, p = prob of success. Discrete. E(X) = np · Var(X) = np(1−p)

Binomial Example

10 coin flips, P(heads)=0.5
P(exactly 3 heads)?
= C(10,3) × 0.5³ × 0.5⁷
= 120 × 0.125 × 0.0078
≈ 0.117 (11.7%)

E(X) = np = 10×0.5 = 5

Poisson Pois(λ)

P(X=k) = λ^ke^−λ k!

Counts events in fixed interval. E(X) = Var(X) = λ. Use for rare events.

Normal N(μ, σ²)

X ~ N(μ, σ²)
Symmetric, bell-shaped
68-95-99.7 rule applies

Standard normal Z ~ N(0,1). Standardise: Z = (X−μ)/σ.

Distribution	Use for	Key params
Bernoulli	Single binary trial	p
Binomial	n binary trials	n, p
Geometric	Trials until first success	p
Poisson	Rare events per interval	λ
Uniform	All outcomes equally likely	a, b
Normal	Many natural phenomena	μ, σ²
Exponential	Time between events	λ
t-dist	Small sample means	df
Chi-squared	Categorical test statistic	df

Bayesian vs Frequentist

two schools of thought

	Frequentist	Bayesian
Probability	Long-run frequency of events	Degree of belief / uncertainty
Parameters	Fixed but unknown constants	Random variables with distributions
Prior info	Not incorporated	Prior distribution P(θ)
Output	p-value, CI (95% of CIs contain true θ)	Posterior P(θ\|data) — probability θ is value
Tools	t-test, ANOVA, chi-square	MCMC, Stan, PyMC, conjugate priors
Best for	Large samples, no prior knowledge	Small samples, prior domain knowledge

Key insight: A Bayesian 95% credible interval means "95% probability the parameter is in this range." A frequentist 95% CI means "if we repeated this experiment many times, 95% of such intervals would contain the true value."

Properties of E(X) & Var(X)

linearity · rules

Linearity of Expectation

E(aX + b) = aE(X) + b
E(X + Y) = E(X) + E(Y)

Always true — even if X and Y are dependent.

Variance Rules

Var(aX + b) = a²Var(X)
Var(X + Y) = Var(X) + Var(Y) (if independent)
Var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y)

A constant b shifts the distribution but doesn't change spread. Multiplying by a scales variance by a².

Covariance & Correlation

Cov(X,Y) = E[(X−μ_X)(Y−μ_Y)]
ρ = Cov(X,Y) σ_X · σ_Y

Probability Mastery Checklist

sheet 2 complete

Fundamentals	Key point
Apply addition rule	P(A∪B) = P(A)+P(B)−P(A∩B)
Compute conditional prob	P(B\|A) = P(A∩B)/P(A)
Test independence	P(B\|A) = P(B)?
Permutations vs combinations	order matters vs not

Bayes & Distributions	Key point
Apply Bayes' theorem	P(A\|B) = P(B\|A)·P(A)/P(B)
Identify correct distribution	binomial / Poisson / normal
Compute expected value	Σ x·P(X=x)
Know 68-95-99.7 rule	±1σ / ±2σ / ±3σ

Advanced	Key point
Distinguish Bayesian vs frequentist	prior vs long-run freq
Use linearity of expectation	E(X+Y) = E(X)+E(Y)
Variance of sum (independent)	Var(X+Y) = Var(X)+Var(Y)
Standardise to Z-score	(X−μ)/σ

Next up → Sheet 3: Linear Algebra · vectors · matrices · eigenvalues · dot product · matrix operations · SVD — the mathematical foundation of machine learning.

Back to Cheat Sheets