Power and sample size calculations for randomized evaluations
Leave these blank and cost stays out of the results. Prices are unit-free: enter them all in the same currency and the total comes back in that currency.
A two-arm or multi-arm comparison of means, for continuous or binary outcomes, with individual or clustered randomization. It solves in any of three directions: the power of a design you have, the smallest effect that design can detect, or the sample size a target effect requires.
The analysed sample required is n = (qα + qpower)² · σ²(1 − R²) · A / (MDE² · (take-upT − take-upC)²), where σ² = SD² for a continuous outcome and A is the allocation factor — 1/(P(1−P)) for a single treatment arm, or 2(T+1) when the sample splits equally across control and T treatment arms. Covariates reduce the residual variance by (1 − R²). Take-up rescales an intention-to-treat design to the effect on compliers. Binary outcomes do not take this form: the two quantiles multiply different variances and are carried separately, as described below.
The treatment proportion moves with the effect, so the two arms do not share a variance. The default is the standard uncorrected score test: the variance pooled under the null carries the significance term and the unpooled variance under the alternative carries the power term. The direction of the expected effect determines p₁, and the implied treatment rate is shown beside the inputs so an impossible design is visible immediately. A legacy method that uses the baseline variance in both arms is available under Allocation & precision, for comparability with calculations made under that convention. A continuity-corrected variant, which older tools apply by default and which returns a larger sample, is not implemented.
Clustering multiplies the requirement by a design effect, where m is the number of units per cluster surviving to analysis and CV is the coefficient of variation of cluster size. Two adjustments for unequal cluster size are offered under Allocation & precision, because they correspond to different analyses. The conservative one (Eldridge/Donner) gives 1 + ((1 + CV²)·m − 1)ρ and is right when clusters are analysed as unweighted means. The efficient one (van Breukelen, Candel & Berger) applies a relative-efficiency discount, (1 + (m − 1)ρ) / (1 − λ(1 − λ)CV²) with λ = ρm/(ρm + 1 − ρ), which is right for a mixed-effects analysis and asks for less sample. The efficient form has a hard validity limit at CV = √3 ≈ 1.732, and the limit is exact: writing u = ρm/(1−ρ), monotonicity of effective sample in cluster size reduces to CV² ≤ (1+u)²/(u(2−u)), whose minimum over u is 3, at u = ½. Above it the discount collapses far enough that the design effect stops rising with cluster size — adding units per cluster can reduce power — so the tool refuses that combination rather than returning a number, and warns from CV = 1.5 as it approaches. The conservative form is monotone at any CV and keeps its full range. The attrition adjustment is applied to the enrolment target after the clustered equation is solved, not to n beforehand — the design effect depends on the analysed cluster size, so folding attrition in first counts it twice. Both cluster modes therefore agree with each other: solve for the number of clusters, feed the answer back as a fixed cluster count, and you recover the cluster size you started from.
Quantiles come from the t distribution on the degrees of freedom the design actually has — n − 2, or clusters − 2 when randomization is clustered — solved by iteration, since the quantiles depend on the answer. Power itself is the exact noncentral t, not a central t shifted by the noncentrality, which reproduces R's power.t.test to twelve decimal places. The normal approximation used by many calculators overstates power when clusters are few: in simulation, a design sized by the normal approximation with 24 clusters delivers 77% power against an 80% target, and 78% at 40 clusters. At very small samples the iteration can oscillate rather than converge, and the calculator then returns the smallest design whose power reaches the target, which is the definition the iteration is only a means to.
Two adjustments are made and kept separate: α is divided among the actual number of tests (Bonferroni, or Šidák), and the power target changes only if the claim is joint. Bonferroni's guarantee holds whatever the dependence structure, so the correction is never applied to a discounted “effective” test count; an experimental option that discounts the threshold for correlation among outcomes is available but does not carry that guarantee. "Power" can mean the chance of detecting a given comparison, at least one of them, or all of them; these differ substantially and the result states which is in force. Joint power accounts for correlation among tests: arms share a control group, correlating their comparisons at about ½, and outcomes measured on the same people move together. Treating the tests as independent overstates "at least one" substantially: at three Bonferroni-corrected arms sized for 80% joint power, the independent product gives 0.915 where the correlated model gives 0.802, an overstatement of about 11 percentage points. The correlated model itself is checked against simulation.
Rounding happens once, at the end of the iteration, never inside it. Sample sizes and cluster counts then round up to whole units, and to a whole multiple of the number of groups so the arms can actually be split evenly — two for a two-arm trial, arms + 1 with several treatment arms. An unequal allocation rounds up within each arm separately, since you cannot enrol 269.7 controls. Because rounding can land just below the target, a final pass steps the design up until its achieved power reaches the target: the answer is the smallest feasible design that hits it, which is the definition the iteration is only a means to. A solved cluster dimension is at least 2.
Published worked examples. Against SAS PROC POWER's published two-sample tables, results agree within one participant across all nine published combinations of significance level, effect and power. Power is checked directly against R: fifteen noncentral t values against pt(q, df, ncp) and six designs against power.t.test, all agreeing to better than 1e-11. That is a stronger anchor than a published table rounded to three decimals, where the last digit cannot distinguish a correct calculation from a nearly correct one.
Stata's power twomeans, cluster worked examples are computed with normal quantiles. With the small-sample t correction disabled, this calculator reproduces them exactly: 26 clusters for the target design, 17 units per cluster at a fixed 30 clusters, power 0.8560, and a minimum detectable effect of 1.0196. The production default is deliberately more conservative, because it uses t critical values on the degrees of freedom the cluster design actually has — the same example returns 28 clusters, 18 units per cluster, power 0.8308 and an MDE of 1.0565. The difference is the correction, not a disagreement about the formula.
Reference implementation. A smaller R implementation of the same core methodology was used as a benchmark during development; 23 parameter sets span both outcome types, all three design modes, take-up, attrition and the error boundaries. In a compatibility mode that restores that implementation's normal approximation and rounding convention, 16 reproduce exactly. The other 7 are clustered designs with attrition, and differ because that implementation folds attrition into n before solving the clustered equation while this one applies it to the enrolment target afterwards — the divergence described under Clustered designs, where the reasoning for this tool's order is given. The difference is a deliberate choice, not an unexplained gap.
Independent parameter sweep. An external test oracle, implemented from this tool's written specification rather than from its code, checks 948 configurations — including 258 deliberately pathological cases covering rounding cliffs, near-singular clustered designs, ICC at 0 and 1, 99% attrition, near-zero treatment contrast, extreme significance and power, binary outcome boundaries, and multiplicity up to 20 arms by 50 outcomes — plus 35 input-guardrail tests. All guardrails are enforced. The sweep found and fixed a rounding inconsistency in the iteration, two quantile-precision limitations, and several validation gaps.
Monte Carlo. Every design is also simulated end to end — cluster random effects split by the intracluster correlation, attrition dropping observations, take-up scaling the effect, covariates residualising the outcome, cluster sizes varying by their coefficient of variation — and the realised rejection rate checked against the stated power.
Binary and unequal cluster sizes. The binary calculation reproduces Stata's power twoproportions exactly on its equal-allocation output (712 total, 356 per group for 0.30 versus 0.40) and to within two participants on a 2:1 allocation, the difference being whether the allocation ratio is preserved exactly after rounding. Both unequal-cluster-size adjustments reproduce Stata's cvcluster example at 28 clusters.
Not yet externally validated. The joint-power model for several arms or outcomes, and cluster-level attrition, have been checked by simulation but not against published worked examples.
The Sensitivity tab varies one assumption across a range while holding everything else at the values on the Calculate tab, and reports the consequence in whichever direction is being solved. It is meant for the question that actually decides a design — what if the intracluster correlation is twice what we assumed, what if take-up is 65% rather than 90% — rather than for a single point estimate. The range defaults to a span around the current value and can be set directly. The list of parameters is ordered by what is usually worth varying for the quantity being solved for — the effect size and the power target lead when solving for sample size, the sample and the design lead when solving for power or MDE — and the tab opens on the first of them, though an explicit choice is kept when you switch modes.
Three optional prices — per participant, per cluster, and a fixed cost — give a total of F + cK·K + ci·N. They are inert until at least one is entered, and nothing costed appears anywhere until then; a blank price is not a zero price. The currency is unlabelled because the arithmetic is unit-free: any consistent unit works. Costs are shown to three significant figures, since a total derived from an MDE that rests on a guessed intracluster correlation does not deserve to be printed to the dollar. The budget lines are the exception and print in full, because they assert that what you spend plus what is left over is what you had.
N and K are the enrolled counts, not the analysed ones. Recruitment, enrolment and survey attempts are paid for on everyone who starts, and attrition destroys data rather than invoices. Two consequences follow, and together they are the check that the model is wired correctly: with the design stated, varying attrition moves the answer and leaves the total alone; when solving for sample size the same attrition raises the total, because reaching the target now means enrolling more. A model that priced the analysed sample would show cost falling as attrition rises, which reads as sensible and is wrong. The cluster and fixed prices apply only to clustered designs, where there is a cluster to price.
What the next five points of power cost. The design is re-solved at a higher power and the two totals differenced. Both powers quoted are the ones the designs actually achieve, not the targets: integer sample sizes overshoot, so a design solved at 80% may deliver 81.6%, and quoting the targets would print a pair that does not reproduce when the printed design is re-entered. Where a fixed number of clusters has already hit its power ceiling, that is reported rather than suppressed.
A budget, run backwards. Enter what you have and the tool reports the largest design it affords and what that design detects, in whichever direction you are already solving. The division always floors — a design quoted one cluster above what the money covers is a study that cannot be run — so the design shown always fits, and one more of whatever the budget was buying never does. The leftover is reported as a fraction of one more of that same thing: 41 clusters and 78% of another is a case for finding a little more money, and 41 and 3% of another is not. A budget the prices cannot satisfy says so, and names what would be needed.
Cost across the tabs. The sensitivity table gains a rightmost column, priced from each row's own design, and it appears in both copied forms. It is shown even where it cannot vary, because a design you stated has one bill repeated down the table, and a column that vanished whenever it happened not to move would read as the feature being broken. Both charts gain a cost series on a right-hand money axis, priced from points the chart has already solved rather than sampled again. That axis is zero-based, like the axis beside it: where a large fixed cost dwarfs the variable cost the series draws as a nearly flat line, and that is the answer — at those prices, sample size barely moves the budget.
One line of advice. If the entered cluster size is far from the one that buys the most precision per dollar, the tool says so. It minimises the design effect times the cost per analysed participant, using the same design effect the rest of the page uses, and searches rather than trusting the textbook square root — that closed form is exact when cluster sizes are equal but misses by up to 20% in variance under the efficient unequal-size adjustment. At $500 a cluster and $20 a participant with an intracluster correlation of 0.05, about 22 participants per cluster is efficient, and a design of 60 per cluster reaches an MDE about 13% larger for the same money. The line appears only outside a factor of two in either direction, because the curve is flat near its minimum, and it closes by saying that if cluster size is fixed in the field you should ignore it — which is the common case, and the sentence is a diagnostic rather than an instruction.
The model is descriptive. It prices the research design as specified, not the programme being evaluated; it does not price treatment and control differently; and it does not solve the optimal-design problem of choosing between more clusters and larger ones at a fixed budget.
Estimates are for planning. Stratification and blocking, repeated measures, subgroup and interaction power, mediation, and non-parametric outcomes are not covered; consult a statistician for those. Attrition is modelled as non-differential across arms, at both the individual and the cluster level. The core two-arm sample-size framework follows Duflo, Glennerster & Kremer (2007); the extensions for clustered designs, unequal cluster sizes, small-sample inference, covariate adjustment and multiple testing use the methods described above.
See how the answer changes as you vary different parameters. All other values are taken from the current setting in the Calculate tab.
Copy text and reproducible code for a grant, protocol, pre-analysis plan, paper or appendix. Generated from the design currently on the Calculate tab; square brackets mark what you need to fill in.