What the heck is p-value?
Journal club, teaching others about science and new innovations are some of the things I love. However, most of the peer-reviewed articles are very dense and many times even a small bit of help understanding them can go a long way. This week we are looking at p-value. p-value is important to understand when reading a scientific paper, as it is used a lot. When reading the graphs in a scientific paper it is helpful to understand the terminology.
So, what the heck is p-value?
The “p” stands for probability, so p-value is the probability value.
The p-value indicates how likely it is that a result occurred by chance alone.
The “p” stands for probability, so p-value is the probability value.
The p-value indicates how likely it is that a result occurred by chance alone.
If the p-value is small, it indicates the result was unlikely to have occurred by chance. These results are considered statically significant.
So, a small p-value indicates a greater than chance alone= SOMETHING HAPPENED; TEST IS SIGNIFICANT.
If the p-value is large, this indicates that the result is within “normal” or expected limits or it could indicate a sampling error.
So, a large p-value indicates= NOTHING HAPPENED; RESULT IS NOT SIGNIFICANT.
UNDERSTANDING ALPHA (a)
When interpreting whether a p-value is significant (beyond chance) or not, we need to know the alpha (a) being used for the experiment.
The two most common levels are:
a= .05 and a= .01
Alpha should be decided on by the researcher before beginning the experiment.
If a= .05 then the following rule applies:
If p < a then the result IS SIGNIFICANT
If p > a then the result IS NOT SIGNIFICANT
Here is an example:
Test result p= .03 given that a= .05
Because .03 < .05 this result is significant
If however, we run an experiment and the result returns a p= .12
Because .12 is greater than .05 this result is NOT significant, it is not beyond sampling error.
p-value Decision Outcome of Test
.040 p< .05 Significant
.075 p> .05 NOT significant
.049 p< .05 Significant
.523 p> .05 NOT significant
.001 p< .05 Significant
OK, but what if we decide that we want a higher degree of specificity?
We decide to use Alpha (a) = .01 as our p-value
If a = .01 then the following rules apply hint: it is the same as above
If p < a the result is significant. If p > a the result is NOT significant
Here we go
If we see p = .04 We know that Alpha (a) = .01 (that is what we choose before we began the experiment).
Therefore, if p = .04 > .01 this result IS NOT Significant
If our result returns such that our p = .003 and .003 is less than .01
Since p = .003 < .01 this result IS Significant
p-value Decision Outcome
.001 p < .01 Significant
.020 p > .01 NOT Significant
.009 p < .01 Significant
.523 p > .01 NOT Significant
.012 p > .01 Significant
Please, don’t even get me started on “p-hacking” and yes, it is a thing. Maybe, I will cover it in a new post. Science is hard, I’m here to help. Sometimes it is good to be little (like a small p-value).
