Skip to main content

What sample size is needed for MVT?

MVT sample size requirements depend on the number of combinations being tested and the effect size you need to detect.

A basic rule: each combination needs enough traffic to achieve significance independently. If an A/B test needs 5,000 recipients per variant, an 8-combination MVT needs approximately 40,000 total.

Factors affecting requirements:

Number of combinations: 2 variables × 2 options = 4 combinations. 3 variables × 2 options = 8 combinations. It multiplies quickly.

Baseline conversion rate: Lower baseline rates need larger samples.

Minimum detectable effect: Detecting small differences requires larger samples than detecting large ones.

Desired confidence level: Higher confidence (99% vs 95%) increases requirements.

Use MVT calculators specific to factorial designs rather than simple A/B calculators.

Insufficient sample size is the most common MVT mistake. If you cannot meet the requirements, run sequential A/B tests instead.