What sample size is needed for MVT?
MVT sample size requirements depend on the number of combinations being tested and the effect size you need to detect.
A basic rule: each combination needs enough traffic to achieve significance independently. If an A/B test needs 5,000 recipients per variant, an 8-combination MVT needs approximately 40,000 total.
Factors affecting requirements:
Number of combinations: 2 variables × 2 options = 4 combinations. 3 variables × 2 options = 8 combinations. It multiplies quickly.
Baseline conversion rate: Lower baseline rates need larger samples.
Minimum detectable effect: Detecting small differences requires larger samples than detecting large ones.
Desired confidence level: Higher confidence (99% vs 95%) increases requirements.
Use MVT calculators specific to factorial designs rather than simple A/B calculators.
Insufficient sample size is the most common MVT mistake. If you cannot meet the requirements, run sequential A/B tests instead.
Was this answer helpful?
Thanks for your feedback!