Skip to main content
013.003

Statistical Significance

How to know if your test results are "real." This section explains statistical significance—the "magic" math that tells you if your "winner" actually won, or if you just got "lucky" due to a small sample size. It's the 'sea-trial' that proves your ship is truly faster.

6 questions in this topic

Questions about Statistical Significance

What is statistical significance in A/B testing? What is a p-value? What confidence level should I aim for (e.g., 95%)? How long should I run an A/B test? Can I stop a test early if results look clear? What tools calculate statistical significance?