A/B test sample size and duration calculator, with the formula

n per variant = (z(1 − α/2) × √(2p(1 − p)) + z(power) × √(p(1 − p) + p2(1 − p2)))² ÷ (p2 − p)²
p2 = p × (1 + lift)
days = n × 2 ÷ visitors a day, rounded up

An A/B test sample size calculator turns four inputs into the number of visitors each variant needs: baseline conversion rate, minimum detectable effect (MDE), significance level and power. A page converting at 5% needs 31,234 visitors per variant to detect a 10% relative lift, using a two-sided test at 5% significance and 80% power. Divide the total by daily eligible traffic to get the run time in days.

How many users do you need for an A/B test

Here is the short answer. A page converting at 5% needs 31,234 visitors per variant to detect a 10% relative lift, using a two-sided test at 5% significance and 80% power. Four inputs set that number: the baseline conversion rate, the minimum detectable effect (MDE), the significance level and the power. Lower baselines and smaller lifts push it up quickly.

At a 10% relative lift, the per-variant sample falls from 80,682 at a 2% baseline to 14,751 at a 10% baseline.

Visitors per variant to detect a 10% relative liftvisitors per variant

Visitors per variant to detect a 10% relative lift

Item	Value, visitors per variant
Baseline 2%	80,682 visitors per variant
Baseline 5%	31,234 visitors per variant
Baseline 10%	14,751 visitors per variant

Baseline conversion	5% relative lift	10% relative lift	20% relative lift
2%	315,206	80,682	21,109
5%	122,124	31,234	8,158
10%	57,763	14,751	3,841

These are visitors per variant.

The formula behind the numbers

The standard two-proportion formula is n = (Zα/2+Zβ)2 * (p1(1-p1)+p2(1-p2)) / (p1-p2)2, where p1 and p2 are the expected rates in each group (Select Statistical Consultants). At a 95% confidence level the critical value is 1.96. At 80% power it is 0.84 (Select Statistical Consultants).

The two error rates are worth stating plainly to your team. A 95% confidence level means a 5% chance of a false positive. An 80% power means a 20% chance of missing a real difference (Select Statistical Consultants). Large jumps need far fewer people. Moving uptake from 15% to 25% takes only 248 customers in each group (Select Statistical Consultants).

What the calculator asks for and returns

You enter your baseline rate, which is the control's current conversion. Then you set the MDE, the smallest effect the test will catch at your chosen power (evanmiller.org). Choose whether the MDE is absolute (percentage points) or relative (a share of the baseline). Then set alpha (significance), power, and one-sided or two-sided. Finally, enter the split ratio between control and variants, your daily eligible traffic and the number of variants.

The calculator returns three things: the sample size per variant, the total sample across all variants, and the number of days at your traffic.

Worked example: from sample to days

Take a 5% baseline and a 10% relative lift. At 31,234 visitors per variant across two variants, the test needs 62,468 visitors. With an example 2,000 eligible visitors a day, 62,468 visitors takes just over 31 days. In our practice we round up to whole weeks so every weekday appears equally often.

If that runtime is too long, test a bigger change or move the test to a higher-converting step. Log both options in the growth experiment backlog template.

The two-proportion sample size formula, with every input defined

The formula from the previous section has five inputs. Each one is easy to fill in once you know where it comes from. Select Statistical Consultants traces it to Wang and Chow's 2007 entry in the Wiley Encyclopedia of Clinical Trials, pages 3-4, section 3.1, "Test for Equality."

Term	What it is	Where you get it (our reading)
p1	Expected conversion rate of the control	Last 4 to 8 weeks of your own data
p2	Expected conversion rate of the variant	p1 plus the smallest lift worth shipping
Zα/2	Critical value of the Normal distribution at α/2	Set by your confidence level
Zβ	Critical value of the Normal distribution at β	Set by your power
n	Users needed in each group	The output

In the source's wording, p1 and p2 are the expected sample proportions of the two groups. They can come from a previous survey or a small pilot study. For a product team, the pilot is usually your current funnel. Our free-to-paid conversion benchmarks help you sanity-check a baseline you have never measured.

The two Z terms are fixed once you pick confidence and power. ClinCalc states the link plainly: Power = 1 - β.

Why low baselines and small effects inflate n

Look at the denominator. It is the gap between p1 and p2, squared. A small gap gives a very small denominator, so n climbs fast. ClinCalc puts the same point in plain terms: if the difference between two treatments is small, more subjects are required.

Low baselines cause a related problem. ClinCalc notes that an outcome that occurs infrequently needs many more subjects. We think the practical reason is that a relative lift on a low rate is a tiny absolute gap. A 10% relative lift on a 2% signup rate moves the rate by less than a quarter of a point, and that sliver is what gets squared.

Three checks before you trust the output

1. n is per group. It is the minimum sample size for each group. For example, if the formula returns 1000, a two-arm test needs 1000 x 2 = 2000 users.
2. Extreme rates break the math. The formula uses a Normal approximation, and if one or both proportions are close to 0 or 1, the approximation is not valid. Very rare events, such as enterprise demo requests, need a different method.
3. Pad for dropouts. If non-response is a possibility, increase the sample size accordingly. In product tests, bot traffic and users who never load the variant fall into this group.

If you have no idea what p1 will be, the source suggests using proportions near to 50%, which is conservative and gives the largest sample size. Log each test's inputs in your growth experiment backlog, so the next estimate starts from real data.

What a minimum detectable effect is, and absolute versus relative

The previous section defined the minimum detectable effect (MDE) as the smallest effect the test will detect at the chosen power. The part that trips teams up is the unit. Most calculators read the MDE as a relative change. Optimizely describes it as "the relative minimum improvement over the baseline you are willing to detect in an experiment to a certain degree of statistical significance", and Statsig's calculator labels its MDE field "The relative minimum difference in conversion rate between the test and control group we want to observe."

Converting relative to absolute

Absolute MDE equals baseline times relative MDE. As an example, not a benchmark: a 10% relative MDE on a 5% baseline is 5% × 10% = 0.5 points. The variant has to reach 5% + 0.5 points = 5.5% before the test reliably sees it.

Optimizely's checkout example works the same way: the baseline conversion rate is 10%, the estimated improvement is at least 5%, and the variant has to reach 10.5% or greater.

The formula from the previous section takes p1 and p2 as absolute proportions. Convert first, then plug in 0.05 and 0.055. Typing "10" where the formula expects a proportion gap gives a nonsense sample size.

Why a small MDE costs traffic

The MDE is also a floor. If the real improvement comes in below it, the test does not reach statistical significance within the planned sample size.

Picking an MDE from what the change is worth

Do not set the MDE to the lift you hope for. Set it to the smallest lift that pays for the work. Use these steps:

1. Count the absolute gain. Example inputs: 5000 trial signups a month at a 5% baseline gives 5000 × 5% = 250 paid conversions. A 0.5 point lift on 5000 signups adds 5000 × 0.5% = 25 customers a month.
2. Put a value on that gain. Multiply the extra customers by what one customer is worth to you, using your own revenue data.
3. Compare it with the cost. Add build time and the weeks of traffic the sample size requires.
4. Decide. If the gain does not cover the cost, either accept a larger MDE or move the idea down your experiment backlog.

Optimizely notes that your desire for a lower MDE may increase if a conversion event is directly connected to revenue. In our reading, that means a paywall or checkout test can justify a long run, while a cosmetic change on a low-value page usually cannot. To sanity check your baseline before you start, see our free-to-paid conversion benchmarks.

Which significance level, power and test direction to use

Two settings decide how cautious your test is. Significance, written alpha, is your tolerance for false positives. Power is your chance of catching a real effect. Every A/B test calculator asks for both, plus a test direction and a split ratio.

Alpha: how often you accept a false win

Alpha is the probability of declaring a difference when none exists. Statsig notes this is commonly referred to as a type 1 error or a false positive. Its calculator sets the default at 0.05, with a range of 0.01 to 0.1. The same convention holds well outside marketing: most medical literature uses an alpha cut-off of 5% (0.05).

The trade is direct. Lowering alpha allows for higher confidence that the difference observed isn't due to chance, but requires larger sample sizes.

Power: how often you catch a real effect

Power is the probability that a test correctly rejects a false null hypothesis. In plain terms, if your change really works, power is the chance the test shows it. Statsig's default is 0.8, adjustable between 0.65 and 0.95, and medical studies commonly use a beta cut-off of 20% (0.2). Raising power cuts false negatives but requires a larger sample size.

One-sided or two-sided

A one-sided test asks if the test variation is better than the control. A two-sided test asks if the test variation is simply different than the control.

We think the choice should follow the decision you will make. If you will ship only on a lift and treat "no change" and "worse" the same way, one-sided fits and needs fewer users at the same alpha. If a drop would change what you do, for example a pricing page where a loss means rolling back and investigating, use two-sided.

Split ratio

Statsig's default of 0.5 sets the A/B test to split test users versus control users 50%/50%. In our reading, an even split gives the smallest total sample for a given effect. Any tilt toward one arm makes the smaller arm the bottleneck, so the total rises. Tilt only when you have a reason, such as limiting exposure to a risky variant.

Setting	Common default	When to change it (our reading)
Significance (alpha)	0.05	Go lower for costly, hard-to-reverse launches
Power	0.8	Go higher when missing a real win is expensive
Direction	One- or two-sided	Two-sided when a loss changes your action
Split ratio	0.5	Uneven only to cap risk on a variant

Record all four settings before launch in your growth experiment backlog, so no one adjusts them after seeing results.

Worked examples computed from the formula

Two published examples make good checks on any spreadsheet you build. Run them first, then put in your own numbers.

Example 1: 15% to 25%

Select Statistical Consultants describes a supermarket promotion where currently 15% of customers buy a product and uptake needs to reach 25%. At 95% confidence the critical value is 1.96, and at 80% power it is 0.84, per the same calculator page.

Step by step:

1. Adding 1.96 and 0.84 gives 2.8, and 2.8 squared is 7.84.
2. For the first group, 0.15 × 0.85 = 0.1275. For the second, 0.25 × 0.75 = 0.1875.
3. Adding 0.1275 and 0.1875 gives 0.315.
4. The gap, 0.25 minus 0.15, is 0.10, and 0.10 squared is 0.01.
5. Multiplying 7.84 by 0.315 gives 2.4696, and 2.4696 divided by 0.01 is 246.96.

The published answer is 248 customers who received the promotion and 248 who did not. In our reading, the small gap comes from rounding the critical values. Unrounded values push the result slightly higher, and a sample size always rounds up to the next whole person.

Example 2: a 3-point lift on a 10.2% baseline

Evan Miller's calculator shows a baseline conversion rate of 10.2% and a minimum detectable effect range up to 13.2%. It returns 2,545 per variation. The effect is 13.2% minus 10.2%, which is 3 percentage points, an absolute change.

In our reading, tools handle the variance term differently. In our reading, the formula above and this calculator will not agree to the unit. Pick one method and use it for every test in your experiment backlog.

A startup case, from sample to weeks

Measuring a 0.5% absolute increase on a 5% base conversion rate needs a sample of roughly 62000. Now take an example input, not a benchmark: 5000 visitors a week enter the test. At 5000 visitors a week, 62000 visitors take 12.4 weeks.

If 12.4 weeks does not fit your plan, raise the MDE or move the test to a page with more traffic.

The law of successes

For small proportions, the same post treats the sample size times the proportion as the number of successes. Detecting a given relative increase takes at least a set number of conversions. For example, measuring a 5% increase in upgraded teams requires 12500 upgraded teams.

For a low-rate metric like free-to-paid upgrades, size the test in conversions, not visitors. Take another example input: a product with 1000 upgrades a month needs 12.5 months to collect 12500 upgrades. That number tells you this week whether the test is worth starting.

How long an A/B test should run

Duration is total sample divided by the visitors who actually enter the test each day. Total sample is the per-variant figure times the number of variants: Optimizely's documentation defines it as the number of variations multiplied by the sample size per variation. Daily eligible traffic is the visitors who reach the tested page or step, times the share you allocate to the experiment.

At a 5% baseline and a 10% relative lift, the same 62,468 visitors take 63 days at 1,000 daily visitors and 4 days at 20,000.

Days to finish a two-variant test at a 5% baseline and 10% relative liftdays

Days to finish a two-variant test at a 5% baseline and 10% relative lift

Item	Value, days
1,000 visitors a day	63 days
2,000 visitors a day	32 days
5,000 visitors a day	13 days
20,000 visitors a day	4 days

Days to finish = total sample ÷ (daily eligible visitors × allocation)

The table uses a 5% baseline and a 10% relative lift, computed with the formula from earlier sections. The daily traffic figures are example inputs, not benchmarks.

Daily visitors to the test	Visitors needed (two variants)	Days to finish
1,000	62,468	63
2,000	62,468	32
5,000	62,468	13
20,000	62,468	4

Allocation changes which row you are on. If a page gets 2,000 visitors a day and you send half of them into the test, 1,000 a day reach it, and you are on the 63-day row. Before you launch, check the analytics count for the exact page or step in the test. Do not use sitewide traffic.

Run full weeks

Optimizely's guidance is direct: "Since most business metrics run weekly, consider rounding up the time you calculate for running an experiment to whole weeks." Its own example divides 60,000 visitors by 40,000 weekly visitors to get 1.5 weeks, then plans for approximately 2. Apply the same rule to the table. At 2,000 daily visitors, 32 days is four full weeks plus four days, so plan for five weeks. In our reading, a test that stops midweek over-weights whichever weekdays it happened to include.

Each added variant stretches the run

Every variant needs its own per-variant sample, so total sample grows with each one you add. In the example above, 62,468 visitors split across two variants is 31,234 each. Three variants need 3 times 31,234, or 93,702 visitors. At 2,000 daily visitors, 93,702 takes about 47 days.

Optimizely shows the same arithmetic with four variations: 8,000 visitors per variation x 4 variations = 32,000 visitors, and at 10,000 weekly visitors that comes to 3.2 weeks.

As one analysis puts it, "If we want more sample size, usually the only option is to wait longer." That leaves you three levers on a low-traffic page:

1. Cut variants.
2. Raise allocation toward the full eligible audience.
3. Accept a larger minimum detectable effect.

Write the planned end date in your growth experiment backlog before launch, so nobody stops the test early.

Why you cannot stop a test early when it looks significant

Every sample size you computed earlier rests on one condition. As Evan Miller puts it, the significance calculation assumes the sample size was fixed in advance.

The reason is simple. Each look is another chance for random noise to cross the line, and you stop on the first crossing. In our reading, Peeking only ever pushes errors in one direction: more false winners.

How much peeking costs

Statsig puts it this way: checking 20 times instead of once can inflate the false positive rate from 5% to potentially 40% or higher. Miller's worst case, a test checked after every observation and capped at 150 observations, wrongly found a winner 26.1% of the time when the change did nothing.

If you must look, the bar has to rise. Miller's table shows the reported significance you need for 5% actual significance:

Times you peeked	Reported significance needed
1	2.9%
2	2.2%
3	1.8%
5	1.4%
10	1.0%

Worked example, not a benchmark: you plan 5000 users per arm and check every 1000 users, which is five looks. By the table, a 5% result at one of those looks is not a 5% result.

The fix: commit, or use a method built for watching

Option one is Miller's: decide on a sample size in advance and wait until the experiment is over. Write the number into your experiment backlog before launch. If someone asks for progress, report the effect the current sample can detect, not the running p-value.

Option two is sequential testing, which is A/B testing that expects you to peek. It applies adjustments that raise the bar for significance as you check more often. The cost is that you are trading some statistical power for the ability to make faster decisions. Statsig lists high-traffic experiments where data accumulates quickly among its uses, along with catching bugs and regressions.

Two other ways to fool yourself

Tiny traffic. Neither method rescues a test that cannot reach its sample. In our reading, if the run time from the earlier section stretches past a quarter, test a bigger change or a higher-traffic page instead.

Too many variants and segments. Each added variant raises the total sample you need, as covered above, and each extra comparison is another draw at a false winner. Statsig's rule: don't peek at segments unless you've adjusted for multiple comparisons. We keep startup tests to one challenger and one primary metric.

What this page does not cover

This page does not cover methods for very rare events where proportions sit close to 0 or 1 and the Normal approximation is not valid, nor the detailed math of sequential testing or multiple comparison adjustments for segments. It is not financial advice: the value of an extra customer and whether a test pays for itself depend on your own revenue data.

Related on Race to Scale: Growth experiment template with ICE scoring, a weekly cadence and a backlog you can copy; What is a good activation rate? Benchmarks by product type and how to define yours; Free-to-paid conversion rate benchmarks for freemium, opt-in trials and card-required trials.