What does a p-value actually mean

When you run a statistical test you start with a null hypothesis that represents no effect or no difference. The p‑value answers the question: if that null hypothesis were true, how likely would we see results as unusual as the ones we actually observed? A small p‑value means the observed data would be rare under the null, suggesting the null may not hold, while a large p‑value indicates the data are consistent with the null.

Formal definition

Formally, the p‑value is defined as (p = P(T $\geq$ t_{obs}\mid $H_0$)) for a one‑sided test, where (T) is the test statistic and (t_{obs}) is its observed value. For a two‑sided test the probability is doubled to capture extremes in both directions. The calculation integrates the tail(s) of the sampling distribution that assumes the null hypothesis, producing a number between 0 and 1.

Common misconceptions about p‑values

• A p‑value is the probability that the null hypothesis is true
• A p‑value tells you the size of an effect
• A p‑value is the same as the Type I error rate

How to interpret a p‑value in practice

1. 1State the null and alternative hypotheses clearly
2. 2Choose a significance level $\alpha$ before looking at the data
3. 3Compute the test statistic and its p‑value
4. 4If p $\leq$ $\alpha$, reject the null; otherwise, do not reject

Example: A researcher tests whether a new drug changes average blood pressure. The null hypothesis is that the mean change $\mu = 0$. A sample of n=25 patients yields a mean change of 5.3 mmHg with a known standard deviation of 1 mmHg. The test statistic is $$\begin{aligned}t &= \frac{\bar{x}-\mu_0}{s/\sqrt{n}} \\ &= \frac{5.3-0}{1/\sqrt{25}} \\ &= 1.5\end{aligned}$$. Looking up a t‑distribution with 24 degrees of freedom gives a two‑tailed p‑value of about 0.14. Because 0.14 > 0.05, the researcher does not reject the null and concludes there is insufficient evidence that the drug changes blood pressure.

Typical p‑value thresholds used in reporting

The p‑value is a tool, not a decision maker on its own. It must be considered alongside study design, effect size, and prior knowledge. Reporting the exact p‑value, the chosen $\alpha$, and confidence intervals gives readers a full picture of the statistical evidence.