A/B Testing
Course Index
28 Lessons · 3 Sections
Master A/B Testing — the gold standard of experimentation, comparing control and treatment variants statistically to determine what performs better. Covers hypothesis formation, sample size calculation, p-values & Bayesian alternatives across Optimizely . Includes 4 builds: conversion rate A/B test, sample size & power analysis, Python results analysis & Bayesian A/B test from scratch.
28Lessons
3Sections
4Projects
FreeAccess
Section 1A/B Testing FundamentalsLessons 1–10
Lesson 1
What is A/B Testing? Experimentation Overview
Lesson 2
Hypothesis Formation & Experiment Design
Lesson 3
Statistical Significance & p-Values
Lesson 4
Sample Size Calculation & Power Analysis
Lesson 5
Randomisation & Splitting Traffic
Lesson 6
Choosing the Right Metric: Primary & Guardrail
Lesson 7
Running an A/B Test: End-to-End Process
Lesson 8
Interpreting Results: Confidence Intervals & Effect Size
Lesson 9
Common A/B Testing Mistakes & Pitfalls
Lesson 10
A/B Testing with SQL: Analysing Results
Section 2Advanced Methods & ToolsLessons 11–22
Lesson 11
Frequentist vs Bayesian A/B Testing
Lesson 12
Multi-Armed Bandit: Beyond A/B Testing
Lesson 13
Multivariate Testing (MVT) vs A/B Testing
Lesson 14
A/B Testing Tools: Optimizely, VWO & LaunchDarkly
Lesson 15
A/B Testing with Google Analytics 4 & Firebase
Lesson 16
Feature Flags & A/B Testing in Engineering
Lesson 17
A/B Testing for Emails & Push Notifications
Lesson 18
Pricing & Revenue A/B Tests
Lesson 19
A/B Testing at Scale: Platform Design
Lesson 20
Causal Inference Beyond A/B: DiD & RDD
Lesson 21
A/B Testing with Python: scipy & statsmodels
Lesson 22
A/B Testing in 2026: AI-Powered Experiments
Section 3Anti-Patterns, Best Practices & ProjectsLessons 23–28