Читать книгу Applied Regression Modeling - Iain Pardoe - Страница 25

When we introduced significance levels in Section 1.6.1, we saw that the person conducting the hypothesis test gets to choose this value. We now explore this notion a little more fully.

Whenever we conduct a hypothesis test, either we reject the null hypothesis in favor of the alternative or we do not reject the null hypothesis. “Not rejecting” a null hypothesis is not quite the same as “accepting” it. All we can say in such a situation is that we do not have enough evidence to reject the null—recall the legal analogy where defendants are not found “innocent” but rather are found “not guilty.” Anyway, regardless of the precise terminology we use, we hope to reject the null when it really is false and to “fail to reject it” when it really is true. Anything else will result in a hypothesis test error. There are two types of error that can occur, as illustrated in the following table: Hypothesis test errors

		Decision
		Do not reject	Reject in favor of
Reality	true	Correct decision	Type 1 error
false	Type 2 error	Correct decision

A type 1 error can occur if we reject the null hypothesis when it is really true—the probability of this happening is precisely the significance level. If we set the significance level lower, then we lessen the chance of a type 1 error occurring. Unfortunately, lowering the significance level increases the chance of a type 2 error occurring—when we fail to reject the null hypothesis but we should have rejected it because it was false. Thus, we need to make a trade‐off and set the significance level low enough that type 1 errors have a low chance of happening, but not so low that we greatly increase the chance of a type 2 error happening. The default value of 5% tends to work reasonably well in many applications at balancing both goals. However, other factors also affect the chance of a type 2 error happening for a specific significance level. For example, the chance of a type 2 error tends to decrease the greater the sample size.