What sample size means and why it matters

Sample size is the number of individual observations or data points you collect in a study. If you survey 500 people about their shopping habits, your sample size is 500. If you test 30 water samples from a river, your sample size is 30. Getting this number right before you start collecting data is one of the most important decisions in research design—too small and your results won't be reliable; too large and you waste time and money on data you don't need.

The reason sample size matters is statistical power. A small sample can miss real patterns in your population, or it can make random noise look like a real finding. A well-sized sample gives you enough information to detect the effect you're looking for, with confidence that what you found isn't just luck.

Key Takeaways

  • Sample size depends on four things: the size of the effect you want to detect, how confident you want to be in your results, how much random variation exists in your data, and the total population size (if it's small).
  • Most studies use a confidence level of 95% and a power of 80%, meaning they accept a 5% chance of a false positive and a 20% chance of missing a real effect.
  • Online calculators and software like G*Power, R, and Python can compute sample size once you input your study parameters—you do not need to do the math by hand.
  • For surveys of large populations, sample size depends almost entirely on margin of error and confidence level, not on population size itself.
  • Pilot studies and published research in your field can help you estimate the effect size you should plan for.

The four numbers you need before you calculate

Before you can compute a sample size, you must decide on four parameters. These are not arbitrary—they come from your research question and your practical constraints.

Effect size is the smallest real difference or relationship you care about detecting. If you're testing a new drug, the effect size might be "a 10% reduction in symptoms compared to placebo." If you're comparing two teaching methods, it might be "a 5-point difference in test scores." Effect size is usually expressed as a standardized number (like Cohen's d, which ranges from 0 to 1 or higher) or as a percentage difference. The smaller the effect you want to detect, the larger your sample must be.

Confidence level (also called alpha or significance level) is the probability you're willing to accept that you'll reject a true result by accident. The standard is 95%, which means you accept a 5% chance of a false positive. This is written as alpha = 0.05. Some fields use 99% (alpha = 0.01) for higher stakes.

Statistical power is the probability that your study will detect a real effect if it exists. The standard is 80%, which means you accept a 20% chance of missing a real effect. This is written as power = 0.80. Some studies use 90% power for more sensitive detection.

Population standard deviation (or variability) measures how spread out your data is. If you're measuring height, a standard deviation of 3 inches means most people fall within 3 inches of the average. The more variation in your data, the larger your sample must be to see a signal. You can estimate this from published studies in your field, a pilot study, or historical data.

How to find or estimate effect size

Effect size is often the hardest parameter to pin down, because it depends on what you consider meaningful in your field. A 2% improvement in crop yield might be huge for agriculture; a 2% improvement in test scores might be trivial for education.

Start by reading published studies similar to yours. Look at the results section and the effect sizes they report. If five studies in your area report effect sizes between 0.3 and 0.5 (using Cohen's d), that's your ballpark. If you're designing a new study, you can use the median or the smallest effect size you found—planning for a smaller effect is more conservative and gives you a larger sample.

If no published research exists in your area, run a small pilot study first. Collect data from 20 to 50 participants, calculate the effect size from that data, and use it to plan your full study. This is standard practice and much cheaper than guessing.

Cohen's d is the most common effect size measure. A d of 0.2 is considered small, 0.5 is medium, and 0.8 is large. For proportions (like the percentage of people who prefer option A over option B), use Cohen's h instead.

Using online calculators and software

G*Power is the most widely used free software for sample size calculation. You read it to your computer, select your test type (t-test, ANOVA, correlation, chi-square, etc.), enter your four parameters, and it outputs the sample size. It runs on Windows, Mac, and Linux. The interface is not intuitive at first, but tutorials are available online, and most statistics textbooks include G*Power examples.

R is a free programming language used for statistics. The pwr package in R includes functions to calculate sample size for common tests. If you know R, this is fast; if you don't, the learning curve is steep. A straightforward command like pwr.t.test(d=0.5, sig.level=0.05, power=0.80) returns the sample size for a t-test.

Python offers the statsmodels library, which includes sample size functions. Like R, it requires coding knowledge but is powerful and free.

Online web calculators exist for specific tests—search "sample size calculator t-test" or "sample size calculator proportion" and you'll find several. These are convenient for one-off calculations but less flexible than G*Power or code-based tools.

Sample size for surveys and polls

Surveys of large populations work differently. You don't need to know the total population size (as long as it's much larger than your sample). Instead, you choose a margin of error and a confidence level, and the sample size follows from those.

A margin of error of 3% with 95% confidence means your results could be off by 3 percentage points in either direction. To achieve this, you need roughly 1,000 respondents, regardless of whether you're surveying a city of 100,000 or a country of 300 million. A margin of error of 2% requires about 2,400 respondents. A margin of error of 1% requires about 9,600 respondents.

The formula is: n = (z² × p × (1 − p)) / e², where z is the critical value for your confidence level (1.96 for 95%), p is the expected proportion (use 0.5 if you don't know), and e is the margin of error as a decimal (0.03 for 3%). Most survey software calculates this automatically.

Adjusting for small populations

If your population is small—say, you're studying all employees at a company with 200 people—the standard formulas overestimate how many you need. Use a finite population correction to adjust downward.

The correction multiplies your calculated sample size by (N − n) / (N − 1), where N is the total population and n is your calculated sample size. If your formula says you need 100 people but your population is only 150, the correction gives you roughly 60 instead. This is built into most statistical software; you just enter the population size.

Common mistakes and how to avoid them

The most common mistake is guessing at effect size. Do not assume your effect is large just because it matters to you. Read the literature, run a pilot, or ask a statistician. Underestimating effect size leads to a sample that's too small, and your study will fail to detect real findings.

Another mistake is confusing statistical significance with practical significance. A very large sample can detect tiny effects that don't matter in the real world. Plan your sample size around the smallest effect you actually care about, not the smallest effect your statistics can detect.

Do not increase your sample size after you see the data and notice your results aren't significant. This is called p-hacking and inflates your false positive rate. Decide your sample size before you collect data, and stick to it.

Finally, do not ignore dropout or non-response. If you expect 20% of your participants to drop out, increase your target sample size by 25% to account for it. This is especially important in longitudinal studies and clinical trials.

Frequently Asked Questions

What if I don't know the standard deviation of my data?

Use a pilot study of 20 to 50 participants to estimate it. Calculate the standard deviation from that small sample and plug it into your power calculation. Alternatively, look for published studies in your field that report standard deviation or variance, and use their numbers as a starting point.

Can I use the same sample size for different types of tests?

No. A t-test, ANOVA, correlation, and chi-square test all have different sample size requirements for the same effect size and power. Always calculate sample size for the specific test you plan to use. G*Power lets you switch between test types easily.

What happens if my actual sample size is smaller than planned?

Your statistical power drops, meaning you're less likely to detect a real effect. If you end up with 80 participants instead of 100, your power might fall from 80% to 70%. You can recalculate power after data collection to report this honestly in your results.

Is 80% power always the right choice?

Eighty percent is a convention, not a rule. Some fields use 90% for higher stakes (like drug trials). Some use 70% for exploratory work. Higher power requires a larger sample. Choose based on the cost of missing a real effect in your field and your practical constraints.

Do I need to account for multiple comparisons when calculating sample size?

Yes, if you're running many tests on the same data. If you plan 10 statistical tests, your false positive rate compounds. Adjust your significance level downward (using Bonferroni correction or similar) before calculating sample size, or calculate sample size for your primary outcome only and treat other tests as exploratory.