When To Use Fisher's Exact Test

In the realm of statistical analysis, selecting the right tool for the job is paramount. The Fisher's exact test, a non-parametric statistical test, stands as a reliable method for analyzing categorical data, particularly when dealing with small sample sizes. Understanding when to use Fisher's exact test and how it differs from other statistical tests is crucial for drawing accurate and meaningful conclusions from your data.

Introduction to Fisher's Exact Test

Fisher's exact test is a statistical significance test used in the analysis of contingency tables. A contingency table is a table that summarizes the relationship between two or more categorical variables. Fisher's exact test is particularly useful when dealing with small sample sizes, where the assumptions of other tests, such as the chi-square test, may not be valid. The primary goal of Fisher's exact test is to determine whether there is a significant association between the two categorical variables in question.

Key Concepts

• Categorical Variables: Variables that represent categories or groups (e.g., gender, treatment type, presence or absence of a condition).
• Contingency Table: A table that displays the frequency distribution of categorical variables.
• Null Hypothesis: The assumption that there is no association between the categorical variables.
• Alternative Hypothesis: The assertion that there is an association between the categorical variables.
• P-value: The probability of observing a result as extreme as, or more extreme than, the one obtained, assuming the null hypothesis is true. A small p-value (typically ≤ 0.05) indicates strong evidence against the null hypothesis.

When to Use Fisher's Exact Test

Fisher's exact test is most appropriate under specific conditions. Recognizing these conditions will help you determine whether it is the right statistical tool for your analysis.

Small Sample Sizes

One of the primary reasons to use Fisher's exact test is when you have small sample sizes. Specifically, it is recommended when any of the expected cell counts in a contingency table are less than 5. The chi-square test, a common alternative, relies on the assumption that the sample size is large enough for the test statistic to approximate a chi-square distribution. When sample sizes are small, this assumption is violated, and the chi-square test can produce inaccurate results.

2x2 Contingency Tables

Fisher's exact test is ideally suited for analyzing 2x2 contingency tables. These tables involve two categorical variables, each with two levels or categories. For example, you might use a 2x2 table to analyze the association between a treatment (treatment vs. placebo) and an outcome (success vs. failure). While Fisher's exact test can be extended to larger contingency tables, it is most commonly used and easily applied in the 2x2 scenario.

Data in Frequencies or Counts

Fisher's exact test requires that your data be in the form of frequencies or counts. This means you need to know the number of observations that fall into each category of your contingency table. The test operates directly on these counts, calculating the probability of observing the data under the null hypothesis. If your data are in the form of proportions or percentages, you will need to convert them to counts before applying Fisher's exact test.

Independent Observations

The observations in your data must be independent of each other. This means that the outcome for one observation should not influence the outcome for another observation. For example, if you are analyzing the effect of a drug on patient outcomes, each patient's outcome should be independent of the outcomes of other patients. Violation of this assumption can lead to incorrect conclusions.

One-Tailed vs. Two-Tailed Tests

Fisher's exact test can be conducted as either a one-tailed or a two-tailed test, depending on your research question.

• One-Tailed Test: Used when you have a specific directional hypothesis. For example, you might hypothesize that a treatment will increase the likelihood of a positive outcome.
• Two-Tailed Test: Used when you are interested in detecting any association between the variables, regardless of the direction. For example, you might hypothesize that a treatment will affect the likelihood of a positive outcome, without specifying whether it will increase or decrease it.

The choice between a one-tailed and a two-tailed test should be determined before conducting the analysis, based on your research question and hypotheses.

How Fisher's Exact Test Works

Fisher's exact test calculates the exact probability of observing the given data (or more extreme data) under the null hypothesis of no association between the categorical variables. It does this by considering all possible contingency tables with the same row and column totals as the observed table.

Steps in Conducting Fisher's Exact Test

1. Create a Contingency Table: Organize your data into a 2x2 contingency table, with the rows representing one categorical variable and the columns representing the other.

2. Calculate the Probability of the Observed Table: Use the following formula to calculate the probability of the observed table:

   P = ( (a+b)! (c+d)! (a+c)! (b+d)! ) / ( n! a! b! c! d! )

   Where:

   • a, b, c, and d are the cell counts in the contingency table.
   • n is the total sample size (n = a + b + c + d).
   • ! denotes the factorial function (e.g., 5! = 5 x 4 x 3 x 2 x 1).

3. Calculate the Probabilities of More Extreme Tables: Identify all possible contingency tables with the same row and column totals as the observed table, and that are more extreme in the direction of the observed association. Calculate the probability of each of these tables using the same formula as above.

4. Sum the Probabilities: Sum the probabilities of the observed table and all more extreme tables. This sum is the p-value for Fisher's exact test.

5. Interpret the P-value: Compare the p-value to your chosen significance level (alpha, typically 0.05). If the p-value is less than or equal to alpha, you reject the null hypothesis and conclude that there is a significant association between the categorical variables.

Example Calculation

Let's consider a hypothetical example where we are investigating the association between a new drug and the outcome of a disease. We have the following data:

Here, a = 10, b = 2, c = 3, d = 8, and n = 23.

1. Calculate the Probability of the Observed Table:

   P = ( (12! 11! 13! 10!) / (23! 10! 2! 3! 8!) ) ≈ 0.0456

2. Calculate the Probabilities of More Extreme Tables:

   We need to consider tables that are "more extreme" in showing an association between the drug and success. This means increasing the value of 'a' while keeping the row and column totals constant. The only more extreme table is:

   	Outcome: Success	Outcome: Failure	Total
   Drug	11	1	12
   Placebo	2	9	11
   Total	13	10	23

   P = ( (12! 11! 13! 10!) / (23! 11! 1! 2! 9!) ) ≈ 0.0031

3. Sum the Probabilities:

   P-value = 0.0456 + 0.0031 = 0.0487

4. Interpret the P-value:

   If we use a significance level of 0.05, the p-value (0.0487) is less than 0.05. Therefore, we reject the null hypothesis and conclude that there is a significant association between the drug and the outcome of the disease.

Fisher's Exact Test vs. Chi-Square Test

The chi-square test is another commonly used statistical test for analyzing contingency tables. However, there are key differences between Fisher's exact test and the chi-square test that make Fisher's exact test more appropriate in certain situations.

Sample Size

As mentioned earlier, Fisher's exact test is particularly useful when dealing with small sample sizes. The chi-square test relies on the assumption that the sample size is large enough for the test statistic to approximate a chi-square distribution. When sample sizes are small, this assumption is violated, and the chi-square test can produce inaccurate results. In general, if any of the expected cell counts in a contingency table are less than 5, Fisher's exact test is preferred over the chi-square test.

Exact vs. Approximate

Fisher's exact test calculates the exact probability of observing the data under the null hypothesis, without relying on any approximations. The chi-square test, on the other hand, uses an approximation of the distribution of the test statistic. This approximation can be less accurate when sample sizes are small, leading to incorrect conclusions.

Assumptions

The chi-square test has several assumptions that must be met for the test results to be valid. These include:

• Independence of observations: The observations must be independent of each other.
• Expected cell counts: All expected cell counts must be greater than or equal to 1, and at least 80% of the expected cell counts must be greater than or equal to 5.
• Random sampling: The data must be obtained through random sampling.

Fisher's exact test has fewer assumptions and is therefore more robust when these assumptions are not met.

When to Use Each Test

• Use Fisher's Exact Test:
  • Small sample sizes (any expected cell count less than 5).
  • When you want an exact probability calculation.
  • When the assumptions of the chi-square test are not met.
• Use Chi-Square Test:
  • Large sample sizes (all expected cell counts greater than or equal to 5).
  • When you are willing to accept an approximate probability calculation.
  • When the assumptions of the chi-square test are met.

Practical Applications of Fisher's Exact Test

Fisher's exact test has a wide range of practical applications in various fields, including:

Medical Research

In medical research, Fisher's exact test is often used to analyze the results of clinical trials, particularly when sample sizes are small. For example, it can be used to determine whether there is a significant association between a treatment and the outcome of a disease, as illustrated in the example above.

Genetics

In genetics, Fisher's exact test can be used to analyze the association between genetic markers and disease status. For example, it can be used to determine whether a particular gene variant is more common in individuals with a certain disease than in individuals without the disease.

Ecology

In ecology, Fisher's exact test can be used to analyze the association between species distribution and environmental factors. For example, it can be used to determine whether a particular species is more likely to be found in certain types of habitats than in others.

Marketing

In marketing, Fisher's exact test can be used to analyze the association between marketing campaigns and customer behavior. For example, it can be used to determine whether a particular marketing campaign is more likely to result in a purchase than another campaign.

Social Sciences

In the social sciences, Fisher's exact test can be used to analyze the association between social factors and outcomes of interest. For example, it can be used to determine whether there is a significant association between education level and employment status.

Advantages and Limitations of Fisher's Exact Test

Like any statistical test, Fisher's exact test has its advantages and limitations. Understanding these will help you use the test effectively and interpret the results accurately.

Advantages

• Exact Probability: Fisher's exact test calculates the exact probability of observing the data under the null hypothesis, without relying on any approximations. This makes it more accurate than other tests, such as the chi-square test, particularly when sample sizes are small.
• Small Sample Sizes: Fisher's exact test is particularly useful when dealing with small sample sizes, where the assumptions of other tests may not be valid.
• Fewer Assumptions: Fisher's exact test has fewer assumptions than other tests, making it more robust when these assumptions are not met.
• Versatility: Fisher's exact test can be applied in a wide range of fields, including medical research, genetics, ecology, marketing, and the social sciences.

Limitations

• Computational Complexity: Fisher's exact test can be computationally intensive, particularly when dealing with larger contingency tables. However, with the availability of statistical software packages, this is less of a concern than it once was.
• Conservatism: Fisher's exact test can be conservative, meaning that it may fail to detect a significant association between the variables when one actually exists. This is particularly true when sample sizes are very small.
• Limited to Categorical Data: Fisher's exact test is limited to analyzing categorical data. It cannot be used to analyze continuous data.
• 2x2 Tables Primarily: While extensions exist, the test is most commonly used and readily applicable to 2x2 contingency tables. Analyzing larger tables can become complex.

Performing Fisher's Exact Test Using Statistical Software

While it is possible to calculate Fisher's exact test by hand, it is much more efficient and practical to use statistical software. Many statistical software packages, such as R, SPSS, SAS, and Python (with libraries like SciPy), provide functions for performing Fisher's exact test.

Example Using R

# Create a contingency table
data <- matrix(c(10, 2, 3, 8), nrow = 2, ncol = 2, byrow = TRUE)
colnames(data) <- c("Success", "Failure")
rownames(data) <- c("Drug", "Placebo")

# Perform Fisher's exact test
fisher.test(data)

# Output:

# Fisher's Exact Test for Count Data

# data:  data
# p-value = 0.04846
# alternative hypothesis: true odds ratio is not equal to 1
# 95 percent confidence interval:
#  0.9597757 58.4776518
# sample estimates:
# odds ratio
#  7.887047

In this example, the fisher.test() function in R is used to perform Fisher's exact test on the contingency table. The output includes the p-value, which is used to determine whether there is a significant association between the variables.

Interpretation of Results

The output from statistical software will typically include the following information:

• P-value: The probability of observing the data (or more extreme data) under the null hypothesis.
• Odds Ratio: A measure of the strength of the association between the variables. An odds ratio of 1 indicates no association, while values greater than 1 indicate a positive association and values less than 1 indicate a negative association.
• Confidence Interval: A range of values that is likely to contain the true odds ratio.

To interpret the results, compare the p-value to your chosen significance level (alpha). If the p-value is less than or equal to alpha, you reject the null hypothesis and conclude that there is a significant association between the variables. The odds ratio and confidence interval provide additional information about the strength and direction of the association.

Conclusion

Fisher's exact test is a valuable statistical tool for analyzing categorical data, particularly when dealing with small sample sizes. By understanding when to use Fisher's exact test and how it differs from other statistical tests, you can ensure that you are using the right tool for the job and drawing accurate conclusions from your data. Remember to consider the assumptions of the test, the type of data you are analyzing, and the research question you are trying to answer. With careful application and interpretation, Fisher's exact test can provide valuable insights into the relationships between categorical variables.