# Statistics for Experimental Biologists

Home

Topic index

Key books

## What exactly is a p-value?

P-values have been criticised because they are widely misunderstood and don't tell scientists what they want to know. Goodman (2008) has written a nice article on the misinterpretation of p-values, and here are a few examples:

• P = 0.05 does not mean there is only a 5% chance that the null hypothesis is true.
• P = 0.05 does not mean there is a 5% chance of a Type I error (i.e. false positive).
• P = 0.05 does not mean there is a 95% chance that the results would replicate if the study were repeated.
• P > 0.05 does not mean there is no difference between groups.
• P < 0.05 does not mean you have proved your experimental hypothesis.

A p-value means only one thing (although it can be phrased in a few different ways), it is: The probability of getting the results you did (or more extreme results) given that the null hypothesis is true.

Let's look at this definition a bit closer. The null hypothesis is the hypothesis of no effect, no correlation, no association, etc., whatever the case may be. This qualitative statement needs to be converted into something more mathematical, for example, one might say that the difference between the mean of a drug group and control group is zero. This is an improvement because we now have a numeric value that we can work with (i.e. zero), but what we actually need is a distribution of possible values that one might expect to get if the drug actually has no effect. This is referred to as the null distribution, and the key thing to remember is that the null distribution is the distribution of outcomes from an experiment when there is no effect. How do you actually come up with a null distribution? This is already "built in" to the statistical test based on theory, and does not need to be specified directly. But let's make a null distribution from scratch to clarify what it is and how it can be used to make inferences. Suppose we have a fair coin (meaning that it has an equal probability of coming up heads or tails) and another coin that we are not sure about (maybe it was given to us by someone of dubious character). We know that if a fair coin is tossed 20 times, we would expect to get 10 heads. Of course we wouldn't expect to get 10 heads all the time, sometimes we would get 9, sometimes 12, etc. To get an idea of what the distribution of outcomes would look like, we could toss a fair coin 20 times and count the number of heads, and then repeat this 10000 times. An easier option is to simulate 20 tosses of a coin 10000 times, and this shown in the figure below.