How long does this test actually need to run?
Calculate it before launching and commit to the number. Stopping a test the moment it looks positive is the single most common way false wins get shipped.
Your numbers
Relative. 15% on 2.4% means detecting a move to 2.76%.
Including the control. Every extra variant splits traffic further.
95% is standard. 90% detects smaller effects faster with more false positives.
80% is standard — an 20% chance of missing a real effect.
Test duration required
6.8 weeks
Commit to this before launching. Stopping early is how false wins happen.
Sample per variant
30,450
Total sample needed
60,900
Feasible: 6.8 weeks at your current traffic. Set the end date now and don't look at significance until you reach it.
Two-proportion test, two-sided, using the normal approximation. Testing a bigger change is almost always the better fix for an infeasible duration — button colours need huge samples because they produce tiny effects.
Calculated in your browser — nothing is sent anywhere, and nothing is stored.
The maths, so it isn't a black box.
Two-proportion test, two-sided
Standard normal approximation using your baseline rate, the minimum lift worth detecting, confidence level, and statistical power. The same maths any testing platform uses.
Lift is relative, not absolute
A 15% lift on a 2.4% baseline means detecting a move to 2.76%, not to 17.4%. Relative is how test results are normally reported and how platforms display them.
Traffic is split across variants
Every additional variant divides your traffic further, so a four-way test takes roughly twice as long as an A/B at the same power. That cost is usually underestimated.
It flags infeasible tests
If the duration exceeds about eight weeks, the test will be disrupted by seasonality or a site change before it concludes. Better to know that before you build the variant.
Questions about this calculation
Because checking repeatedly and stopping at the first significant result inflates your false positive rate dramatically — well above the 5% the test is supposed to guarantee. Either commit to the calculated duration, or use a sequential testing method designed for continuous monitoring.
Terms used here
Other calculators
Where we do this work
We'll build this model on your real data.
A free 30-minute teardown where we build your contribution margin model from actual figures and show you where the funnel leaks. Yours to keep either way.