Errors in Hypothesis Testing

The choice of a rejection region for the null hypothesis is made so that we can readily understand the errors involved. At the 95% confidence level, for example, there is a 5% chance that we will reject the null hypothesis even though it is true. This could happen if an unusual result occurred that put our test statistic z or t into the rejection region. The error that results from rejecting H0 when it is true is called a type I error.

The significance level a gives the frequency of rejecting H0 when it is true. The other type of error is that we accept H₀ when it is false. This is termed a type II error. The probability of a type II error is given the symbol b. No test procedure can guarantee that we will not commit one error or the other. The error probabilities are the result of using a sample of data to make inferences about the population. At first thought, making a smaller (0.01 instead of 0.05) would appear to make sense in order to minimize the type I error rate. However, decreasing the type I error rate increases the type II error rate because they are inversely related to each other. It is important when thinking about errors in hypothesis testing to determine the consequences of making a type I or a type II error. If a type I error is much more likely to have serious consequences than a type II error, it is reasonable to choose a small value of a. On the other hand, in some situations a type II error would be quite serious, and so a larger value of a is used to keep the type II error rate under control.

As a general rule of thumb, the largest a that is tolerable for the situation should be used. This ensures the smallest type II error while keeping the type I error within acceptable limits. For many cases in analytical chemistry, an a value of 0.05 (95% confidence level) provides an acceptable compromise.