Читать книгу End-to-end Data Analytics for Product Development - Chris Jones - Страница 29

After establishing the null and alternative hypotheses and setting the significance level α, how do we decide to reject the null hypothesis?

When we conduct a hypothesis test, the results include a probability called p‐value.

We use the p‐value to determine whether we should reject or fail to reject the null hypothesis, by comparing it to the significance level α.

If the p‐value is less than α, we reject the null hypothesis in favor of the alternative hypothesis (our result is statistically significant):

 E.g. H0 (the mean performance of the new product is equal to the industry standard) is REJECTED.

If the p‐value is greater than or equal to α, we fail to reject the null hypothesis. There is not enough evidence to claim that the alternative hypothesis is true (our results are NOT statistically significant):

 E.g. H0 (the mean performance of the new product is equal to the industry standard) is NOT REJECTED.

 Example 1.5. A researcher wants to investigate whether the performance of a new product differs with respect to the industry standard.

Suppose they set the significance level α equal to 0.05 (5%). The two hypotheses are:

 H0: The mean performance of the new product is equal to the industry standard.

 H1: The mean performance of the new product is NOT equal to the industry standard.

What if the p‐value is 0.032? Would they reject or fail to reject the null hypothesis?

They should reject the null hypothesis H₀ because the p‐value is less than the significance level α and conclude that the new product has a different mean performance with respect to the industry standard. The result of the test is statistically significant at α = 5%.

And, what if the p‐value is 0.076? Would they reject or fail to reject the null hypothesis?

They should fail to reject the null hypothesis H₀ because the p‐value is greater than the significance level α and conclude that there is not enough evidence to claim that the new product has a different mean performance with respect to the industry standard. The result of the test is NOT statistically significant at α = 5%.

The p‐value indicates whether our results are statistically significant. However, just because our results are statistically significant doesn't mean that they are practically significant.

 Example 1.6. A production line manager attempts to reduce production time by modifying the process. They compare the mean production time of the old process with the mean production time of the new process using a hypothesis test.The p‐value is 0.032. Using an alpha of 0.05, would they reject or fail to reject the null hypothesis?Because the p‐value is less than alpha, they reject the null hypothesis that the production times are equal. The difference between the mean production times is statistically significant.Now, consider practical significance. If the difference between the two production times is five seconds, is that really practically significant? Is it worth the cost of implementing the process change?

Always consider the practical significance of your results and your knowledge of the process before reaching a final conclusion.