Why confidence intervals and p-values get confused
In statistics, confidence intervals (CIs) and p-values often show up side by side. Both are tied to hypothesis testing, but they measure different aspects of uncertainty. Mixing them up can lead to incorrect conclusions about your data and, in some cases, misleading decisions.

What is a p-value?

A p-value is the probability of observing your data—or something more extreme—if the null hypothesis were true. It answers the question: “How surprising is this result under the assumption that nothing is going on?”

• Small p-value (e.g., < 0.05) → Evidence against the null hypothesis.
• Large p-value → Data is consistent with the null, but not proof that the null is true.

P-values give you a measure of statistical significance, but not effect size.

What is a confidence interval?

A confidence interval provides a range of plausible values for the population parameter, based on your sample. For example, a 95% CI for a mean might be (2.3, 3.7). This means that, in repeated sampling, 95% of such intervals would capture the true mean.

• CI width reflects precision—narrower intervals mean more certainty.
• If a 95% CI for a mean difference does not include zero, it corresponds to a p-value less than 0.05 in a two-tailed test.

The relationship (and key differences)

It’s true that confidence intervals and p-values are mathematically linked, but they tell you different things:

• P-value → “Is the effect statistically significant?”
• Confidence interval → “What is the likely size and direction of the effect?”

Both should be reported together: the p-value tells you whether to reject the null, and the confidence interval shows the magnitude and uncertainty of the effect.

Common mistakes to avoid

• Thinking a p-value is the probability the null hypothesis is true—it’s not.
• Interpreting a 95% confidence interval as meaning there’s a 95% chance the true value lies inside your single calculated interval—that’s not correct either.
• Using only p-values without effect sizes—you miss the practical importance of results.
• Using only confidence intervals without considering statistical significance—you may overlook whether the observed effect is distinguishable from chance.

Practical takeaway

Use p-values to test for significance, but don’t stop there. Always look at the confidence interval to understand the effect size and its uncertainty. For applied research, confidence intervals often provide more useful information than p-values alone.

Helpful resources for learning

If you’re working with data regularly, having a quick reference helps. Consider a statistics quick study guide for your desk, or a full applied statistics textbook to dive deeper. For more hands-on practice, a statistics workbook can reinforce the differences between confidence intervals and p-values in real problems.