Statistical Test

A test used to determine the statistical significance of an observation. Two main types of error can occur:

1. A type I error occurs when a false negative result is obtained in terms of the null hypothesis by obtaining a false positive measurement.

2. A type II error occurs when a false positive result is obtained in terms of the null hypothesis by obtaining a false negative measurement.

The probability that a statistical test will be positive for a true statistic is sometimes called the test's sensitivity, and the probability that a test will be negative for a negative statistic is sometimes called the specificity. The following table summarizes the names given to the various combinations of the actual state of affairs and observed test results.

Multiple-comparison corrections to statistical tests are used when several statistical tests are being performed simultaneously. For example, let's suppose you were measuring leg length in eight different lizard species and wanted to see whether the means of any pair were different. Now, there are  pairwise comparisons possible, so even if all of the population means are equal, it is quite likely that at least one pair of sample means would differ significantly at the 5% level. An alpha value of 0.05 is therefore appropriate for each individual comparison, but not for the set of all comparisons.

In order to avoid a lot of spurious positives, the alpha value therefore needs to be lowered to account for the number of comparisons being performed. This is a correction for multiple comparisons. There are many different ways to do this. The simplest, and the most conservative, is the Bonferroni correction. In practice, more people are more willing to accept false positives (false rejection of null hypothesis) than false negatives (false acceptance of null hypothesis), so less conservative comparisons are usually used.

REFERENCES:

Kanji, G. K. 100 Statistical Tests. Thousand Oaks, CA: SAGE Publications, p. 110, 1999.