What a P-Value Tells You

A p-value is a number between 0 and 1 that tells you how likely your results are if there is no real difference or relationship in the thing you are studying. The smaller the p-value, the less likely your results happened by chance alone. Most researchers use 0.05 as the cutoff: if your p-value is 0.05 or smaller, they treat the result as statistically significant.

Think of it this way: you flip a coin 100 times and get 60 heads. A p-value would tell you how often you would see 60 or more heads in 100 flips if the coin were actually fair. If that happens rarely (small p-value), the coin might be biased. If it happens often (large p-value), the coin is probably fine.

Computing a p-value requires three things: a test statistic (a number you calculate from your data), the type of statistical test that fits your question, and the distribution that test follows. The steps differ depending on whether you are comparing two groups, testing one group against a target, or looking for a relationship between two variables.

Key Takeaways

  • A p-value is the probability of seeing your results (or more extreme results) if there is actually no real effect or difference.
  • You calculate a test statistic first—such as a t-statistic or chi-square—then use that to find the p-value from a statistical table or software.
  • The type of test you use depends on your data: t-tests for comparing averages, chi-square for categories, correlation for relationships.
  • A p-value of 0.05 or less is commonly treated as statistically significant, though this threshold is a convention, not a rule.
  • Most researchers use software like R, Python, or Excel rather than tables, because software is faster and less error-prone.

The Three Steps to Any P-Value

Every p-value calculation follows the same skeleton. First, you state a null hypothesis—the claim that there is no real effect or difference. Second, you calculate a test statistic from your data using a formula that depends on your question. Third, you convert that test statistic into a p-value using the appropriate statistical distribution.

The null hypothesis is always the boring answer. If you are testing whether a new drug works, the null hypothesis is that it does not. If you are comparing two groups, the null hypothesis is that they have the same average. The p-value then answers: "If the null hypothesis were true, how often would I see a test statistic this extreme or more extreme?"

The test statistic is a standardized number that summarizes how far your data is from what the null hypothesis predicts. A larger test statistic usually means stronger evidence against the null hypothesis. Once you have the test statistic, you look it up in a table or feed it to software, which tells you the p-value.

Computing a T-Test P-Value for Comparing Two Groups

A t-test compares the average of one group to the average of another. This is one of the most common tests. The formula for the t-statistic is:

t = (mean₁ − mean₂) / (standard error)

The standard error accounts for how spread out each group is and how many people are in each group. Once you have the t-statistic, you use a t-distribution table or software to find the p-value. The p-value depends on your t-statistic and your degrees of freedom (roughly, the number of observations minus the number of groups).

For example: you measure blood pressure in 20 people taking a new medication and 20 people taking a placebo. The medicated group has an average of 130 mmHg with a standard deviation of 10. The placebo group has an average of 135 mmHg with a standard deviation of 12. You calculate the standard error, then the t-statistic, then look up the p-value. If the p-value is 0.03, you would say the difference is statistically significant at the 0.05 level.

Computing a Chi-Square P-Value for Categories

A chi-square test compares observed counts in categories to the counts you would expect if there were no relationship. For example, you might ask whether men and women choose different political parties, or whether defect rates differ across three factories.

The formula is:

χ² = Σ [(observed − expected)² / expected]

You calculate this sum across all categories. Then you use a chi-square distribution table or software to convert your χ² value into a p-value. The degrees of freedom equal the number of categories minus 1 (or for a two-way table, the number of rows minus 1 times the number of columns minus 1).

For example: you survey 100 people and ask whether they prefer coffee or tea. You observe 60 coffee and 40 tea. If there were no real preference, you would expect 50 and 50. Your chi-square value is [(60−50)²/50] + [(40−50)²/50] = 4. Looking this up with 1 degree of freedom gives a p-value of about 0.046, which is statistically significant at the 0.05 level.

Computing a Correlation P-Value for Relationships

A correlation test asks whether two continuous variables move together. For example, do height and weight correlate? Do study hours and test scores correlate?

You first calculate the correlation coefficient (usually called r), which ranges from −1 to 1. Then you convert r into a t-statistic using:

t = r × √(n − 2) / √(1 − r²)

where n is the number of pairs. Then you look up this t-statistic in a t-distribution table with n − 2 degrees of freedom to get the p-value. Most software calculates this automatically when you run a correlation.

For example: you measure height and weight in 50 people and find a correlation of 0.75. Your t-statistic is 0.75 × √48 / √0.4375 ≈ 7.5. With 48 degrees of freedom, this gives a p-value much smaller than 0.001, so the correlation is highly statistically significant.

Using Software Instead of Tables

In practice, almost no one calculates p-values by hand anymore. Excel has built-in functions like T.TEST() and CHISQ.TEST() that do the work for you. R and Python (with libraries like SciPy) are standard in research and give you more control. Even free online calculators exist for common tests.

In Excel, if you have two columns of data, you can type =T.TEST(range1, range2, 2, 2) to get a two-tailed t-test p-value when ready. In R, you would type t.test(group1, group2) and the output includes the p-value. In Python, you would use scipy.stats.ttest_ind(group1, group2) and extract the p-value from the result.

Using software is faster, less error-prone, and lets you focus on whether your question and data are sound rather than on arithmetic. If you are learning statistics, working through one or two calculations by hand helps you understand what a p-value means. After that, use software.

Common Mistakes When Computing P-Values

One frequent error is choosing the wrong test for your data. A t-test assumes your data is roughly normally distributed; if it is not, a non-parametric test like the Mann-Whitney U test may be more appropriate. A chi-square test requires that expected counts in each category are at least 5; if they are not, Fisher's exact test is better for small samples.

Another mistake is running many tests on the same data and reporting only the ones with small p-values. If you test 20 different hypotheses, you expect about one to have a p-value below 0.05 by chance alone, even if none of them are true. This is called p-hacking or the multiple comparisons problem. If you plan to run multiple tests, adjust your threshold (for example, divide 0.05 by the number of tests) or use a method like Bonferroni correction.

A third mistake is confusing statistical significance with practical importance. A p-value of 0.01 means the result is unlikely to be due to chance, but it does not mean the effect is large or matters in the real world. Always look at the actual difference or effect size, not just the p-value.

Frequently Asked Questions

What does a p-value of 0.05 actually mean?

It means that if the null hypothesis were true (no real effect), you would see results this extreme or more extreme about 5 times in 100 repeated experiments. It does not mean there is a 5 percent chance the null hypothesis is true, and it does not mean there is a 95 percent chance your result is real. The p-value is about the data given the null hypothesis, not about the hypothesis given the data.

Can a p-value be negative or greater than 1?

No. A p-value is always between 0 and 1 because it is a probability. If you get a negative number or a number greater than 1, you made a calculation error or used the wrong formula.

Is 0.05 the only threshold that matters?

No. The 0.05 threshold is a convention, not a law. Some fields use 0.01 for stricter standards, and some use 0.10 for exploratory work. The threshold you choose should depend on the cost of being wrong in your specific situation. In drug approval, 0.05 is standard. In early-stage research, a higher threshold might be acceptable.

What is the difference between a one-tailed and two-tailed test?

A two-tailed test asks whether there is a difference in either direction (larger or smaller). A one-tailed test asks whether the difference is in one specific direction. A two-tailed test is more conservative and is usually the right choice unless you have a strong reason to predict the direction beforehand. Most software defaults to two-tailed.

Do I need to calculate the p-value myself, or can I always use software?

You can always use software. Understanding the concept and the steps helps you choose the right test and interpret the result correctly, but the arithmetic itself is software's job. Learn the concept, use software for the calculation.