Skip to main content

How long should I run an A/B test?

Test duration depends on reaching statistical significance, not arbitrary time periods:

Calculate required sample size before starting based on your baseline metrics, expected effect size, and desired confidence level.

Estimate duration by dividing required sample by your typical send volume. If you need 10,000 recipients per variant and send to 5,000 daily, expect roughly 4 days.

Minimum duration should span at least one full week to capture day-of-week variations in recipient behavior. Two weeks is safer for most campaigns.

Never stop based solely on time if significance has not been reached. Either continue until significant or accept the test as inconclusive.

Account for external factors by avoiding tests that span major holidays or unusual periods.

Patience produces truth. Rushing to conclusions based on incomplete data wastes the effort you invested in setting up the test.