Fisher's Exact Test (also known as the Fisher-Irwin Test) is a test of proportions used to determine if two categorical (or binary) variables are independent. This test is non-parametric, which means that it makes no assumption about the distribution of your data.
This test is best for small sample sizes, or when the expected count in any of the cells of a 2x2 contingency table is less than 5. Although it will work for any sample size, it becomes unwieldy.
The null hypothesis for this test is that no relationship exists between the variables. Therefore, a significant outcome means that the variables are related and therefore are not independent.
A contingency table, sometimes known as a two-way table or a crosstabulation (or crosstab), is a table which displays the frequencies of combinations of two categorical variables. The outcomes for one variable are displayed in the table's rows and the other in its columns.
Another test for independence between two categorical variables is the Chi-Squared Test of Independence. This test is more widely used when working with categorical variables because it is simpler to calculate, and it is based on the difference between observed and expected outcomes of the contingency table under the null hypothesis. However, although this test is also non-parametric, it assumes that the data does follow the chi-square distribution, which Fisher's Exact Test does not.
Whereas the Chi-Squared Test uses a test statistic and sampling distribution to calculate a p-value, Fisher's Exact Test calculates the number of all possible contingency tables with the same marginal distributions (the same row and column total) as the observed table (your own table), then calculates the p-value from the proportion of all possible tables more extreme than the observed.
The one-sided p-value for Fisher's Exact Test is calculated by dividing the sum of probabilities of tables at least as extreme as the observed by the sum of probabilities of all possible tables. Therefore, for a 2x2 contingency table, where each cell is denoted a, b, c and d, and the sample size is n, the formula to calculate the one-sided p-value is given by:
Calculating the two-sided p-value is significantly more complicated, so it is recommended you use software to compute it instead.
If the results of Fisher's Exact Test display a significant p-value, with the odds ratio and confidence interval greater than 1.0, this means that the treatment is more likely to achieve the outcome. In a similar way, if the p-value is significant and the odds ratio and confidence interval are less than 1.0, we say that the treatment group is less likely to achieve the outcome.
