Statistical power is a number between 0 and 1 which measures a study's - or experiment's - ability to detect an effect if the effect does truly exist. The closer this number is to 1, the greater the probability of making a correct decision (that is, rejecting the null hypothesis when the null hypothesis is in fact false).
Making sure your study has high power is crucial to ensure you can rely on the results you are given. This is because as your power increases, the probability of making a Type II error decreases. Having a low power means that there is a high risk of you missing important insights due to your test not being powerful enough to detect it.
Observe the following table:
| Do not reject H0 | Reject H0 | |
|---|---|---|
| H0 is actually true | Correct decision! | Type I error (false positive) |
| H0 is actually false | Type II error (false negative) | Correct decision! |
A Type I error is rejecting the null hypothesis H0 when in real life it is actually true, and a Type II error is where you do not reject the null hypothesis H0 when in real life it is actually false.
The probability of a Type I error occurring is our significance value α, so usually this is either 0.01, 0.05 or 0.1. The probability of a Type II error occurring is given by β. Power is calculated by
Power = 1 - β.
Because of this, a high power in data analysis reduces the probability of a Type II error happening.
There is no one formula to calculate power as it depends on the statistical test you are using (ANOVA, Spearman's rho, etc).
There is also no one-size-fits-all for deciding what the correct level for power is: it depends on your field. Most studies have a power of 0.8 (80%) or 0.9 (90%), however you need to consider the circumstances of your test in order to really determine what power is the right one for you.
For example, a particular healthcare study will require a power of 0.95 (or 95%) and above, whereas a particular business study can get away with a power of 0.75 (or 75%). How to determine which is right for you is to use your literature review - by looking at other papers who have done the same or similar studies that you intend to do, you can use what they used as justification for your own power level.
