Читать книгу Design and Analysis of Experiments by Douglas Montgomery - Heath Rushing - Страница 10

3

Experiments with a Single Factor: The Analysis of Variance

Section 3.1 A One-way ANOVA Example

Section 3.4 Model Adequacy Checking

Section 3.8.1 Single Factor Experiment

Section 3.8.2 Application of a Designed Experiment

Section 3.8.3 Discovering Dispersion Effects

In this chapter, the t-test is generalized to accommodate factors with more than two levels. The method of analysis of variance (ANOVA) introduced here allows us to study the equality of the means of three or more factor levels. ANOVA partitions the total sample variance into two parts: the variance explained by the factor under study, and the remaining, unexplained variance.

The method makes several assumptions about the distribution of the random error term in the model. If the model structure represents the true structure of the process, the model residuals may be thought of as random numbers generated from the distribution of the random error term, which is typically assumed to be a normal distribution. Several diagnostics are available for the residuals. They may be plotted on a normal quantile plot to check the assumption of normality of the random error term. They may also be plotted against the predicted values: the residuals and predicted values ought to be independent, and no patterns should be present in the plot. ANOVA also assumes that the error terms are independent and identically distributed. This chapter considers two formal tests, Bartlett’s and Levene’s, for the homogeneity of residual variance across factor levels. If any of the residual diagnostics show abnormalities, a transformation of the response variable is often useful for improving the model fit.

When the ANOVA test rejects the null hypothesis that all treatment means are equal, it is often necessary to know which factor levels are significantly different from each other. Special techniques are necessary for multiple comparisons of different linear combinations of factor level means in order to control the so-called experimentwise error rate. Examples are presented for Tukey’s HSD (honestly significant difference) test and the Fisher (Student’s t) least significant difference method. If one of the factors represents a control group, Dunnett’s test may be used to compare the control group with each of the other factor levels.

Other topics covered include power analysis for ANOVA to determine a required sample size, an introduction to the random effects models that are useful when the factor levels are only a sample of a larger population, and an example of a nonparametric method. The Kruskal-Wallis test relaxes the assumption that the response distribution is normal in each factor level, though it does require that the distributions across factor levels have the same shape.

The first example will illustrate how to build an ANOVA model from data imported into JMP. This entails specifying the response column, the factor column, and ensuring that the factor column is set to the nominal modeling type. Afterward, we will show how models may be designed in JMP, and how the appropriate modeling options are saved as scripts attached to the data table. For the remainder of the text, we will assume that the data tables are created in JMP.

Section 3.1 A One-way ANOVA Example

1. Open Etch-Rate-Import.jmp.

2. Click the blue icon (triangle) next to Power in the Columns panel and select Nominal.

Even though the levels of Power are ordinal, we are not incorporating that information into the current analysis. This distinction is not critical since Power is only a factor, and not the response. Treating an ordinal factor as nominal yields the same model fit. For a response variable, a nomial modeling type prompts a multinomial logistic regression, while an ordinal modeling type prompts an ordered logistic regression.

3. Select Analyze > Fit Y by X.

4. Select Etch Rate and click Y, Response.

5. Select Power and click X, Factor.

6. Click OK.

7. To produce a box plot, click the red triangle next to One-way Analysis of Etch Rate By Power and select Quantiles.

The box plots show that the etch rate increases as power increases, and that the variability of the etch rate is roughly the same for each power setting.

8. Select Window > Close All. We will now demonstrate how this model can be created in JMP.

9. Select DOE > Full Factorial Design.

10. Under Response Name, double click Y and change the response name to Etch Rate.

11. In the Factors section, select Categorical > 4 Level.

12. Double-click the name of the new factor, X1, and change it to Power.

13. Likewise, change the Values of the new factor from L1, L2, L3, and L4 to 160, 180, 200, and 220, respectively.

14. Click Continue.

15. Leave Run Order set to Randomize. Then, the experiment should be run in the order in which the rows appear in the resulting JMP table.

Number of Runs: 4 indicates that the design requires four different runs. Once Make Table has been clicked, Number of Runs will change to 20, reflecting the runs needed for the four replicates.

16. Enter 4 for Number of Replicates. This indicates the original 4 runs will be replicated 4 times for a total of 20 runs.

17. Click Make Table.

A new data table has been created with three columns. The Pattern column indicates which combination of factor levels are being used for the current row. Since there is only one factor, Power, the Pattern column simply indicates which level of Power is being run. The Power column has automatically been set to the Nominal modeling type. Additional metadata about the columns has been included from the Full Factorial platform, as indicated by the icons in the Columns section. The dots in the Etch Rate column represent missing values. As the experiments are conducted (in the randomized order presented in the data table), these values will be filled in by the engineer.

All of the JMP platforms demonstrated in this book are capable of fitting models in the presence of missing data. That is, if it is not possible to perform the 20^th run, which is at the Power setting of 220, it would still be possible to analyze the first 19 runs. However, missing observations can affect the aliasing structure of a design, which will be discussed in later chapters. In addition, if the cause of the missing values is related to the response (missing not at random), then the resulting estimates could be biased.

18. Select Window > Close All.

19. Open Etch-Rate.jmp. This data table was created in JMP using the Full Factorial Design platform.

20. Select Analyze > Fit Model.

Etch Rate is automatically populated into the Y field, and the nominal Power factor is automatically added as a model effect. The corresponding fields in the Fit Y by X platform will not be automatically populated. However, the Fit Model platform is more general than the Fit Y by X platform and will be used much more frequently. Setting default column roles for the Fit Y by X and other platforms may be achieved via the Cols > Preselect Role menu. Please note that this can also be accomplished by selecting Run Script from the (red triangle associated with the) Model script contained in the Table Panel of the data table.

21. Select Window > Close All.

Example 3.1 The Plasma Etching Experiment

1. Open Etch-Rate.jmp.

2. Select Analyze > Fit Y by X.

3. Select Etch Rate and click Y, Response.

4. Select Power and click X, Factor.

5. Click OK.

6. Click the red triangle next to One-way Analysis of Etch Rate By Power and select Means/Anova.

The p-value for the F test of the null hypothesis of the equality of treatment means is <.0001. We conclude that the treatment means differ.

7. Leave the Etch-Rate data table open for the next exercise.

Example 3.3 Treatment Effects and Confidence Intervals.

1. Return to the Etch-Rate data table.

2. Click Analyze > Fit Model. As shown in the first example of this chapter, the modeling roles are pre-specified. Unless a different model needs to be fit than the one specified by the script attached to the data table, the screenshot of the Fit Model platform may be omitted.

3. Click Run. As noted previously, the Fit Model platform could also have been launched by clicking the red triangle next to the Model script in the Etch-Rate data table and clicking Run Script. This script was created by the Full Factorial Design platform and attached to the data table produced therein.

4. Scroll down to the Parameter Estimates report. It may be necessary to click the gray triangle next to the report title in order to expand the output window.

The coefficients for power settings of 160, 180, and 200 provided by JMP match those in the textbook. The intercept represents the grand mean of the observations.

5. To see the estimate for 220, click the red triangle next to Response Etch Rate and select Estimates > Expanded Estimates.

6. To display the confidence intervals for the parameter estimates, right-click inside the Expanded Estimates report, and select Columns > Lower 95% and Columns > Upper 95%.

Alternatively, you can click the red triangle next to Response Etch Rate and select Regression Reports > Show All Confidence Intervals.

7. In addition to viewing the Expanded Estimates, you can also click the red triangle next to Response Etch Rate and select Effect Screening > Scaled Estimates.

8. The Scaled Estimates report produces the same output as the Expanded Estimates report, in addition to a graphical representation of the magnitude of the treatment effects.

9. By clicking the red triangle next to Response Etch Rate and selecting Factor Profiling > Profiler, you obtain an interval plot of the mean responses and their confidence intervals.

The Prediction Profiler provides the estimate mean response together with a confidence interval for each power setting. We will not explore the full functionality of the Prediction Profiler here, but it may be used for optimizing parameter settings to achieve a desired response.

10. Leave the Fit Model platform open for the next exercise.

Section 3.4 Model Adequacy Checking

1. In the Fit Model platform from the previous exercise, scroll down to the Residuals by Predicted plot. This plot is discussed in Section 3.4.3 of the textbook.

The variance appears to be constant across the range of predicted etch rates, and no patterns emerge from the plot. Because only a single categorical factor, Power, is included in the model, the validity of the assumption of constant residual variance may also be checked with formal tests, as described in Example 3.4.

2. To check the normality assumption, a quantile plot is commonly used. In JMP, the first step is to generate residuals for each observation. Click the red triangle next to Response Etch Rate and select Save Columns > Residuals.

3. Return to the Etch-Rate data table (you can use a shortcut by clicking the table icon at the bottom of the report window) and notice the new column Residual Etch Rate.

4. Click Analyze > Distribution.

5. Select Residual Etch Rate for Y, Columns.

6. Click OK.

7. Click the red triangle next to Residual Etch Rate and select Continuous Fit > Normal.

8. Scroll down and click the red triangle next to Fitted Normal and select Diagnostic Plot.

The error distribution appears to be approximately normal as the points fall relatively close to a straight line. We may also perform a Shapiro-Wilk test for the hypothesis that the residuals are from a normal distribution.

9. Click the red triangle next to Fitted Normal and select Goodness of Fit.

With a p-value of 0.2152, the residuals do not display a significant number of departures from normality.

10. Section 3.4.2 of the text discusses plotting the residuals in a time sequence to look for correlations between subsequent runs, which would represent a violation of the (important) independence assumption. To generate this plot, select Analyze > Modeling > Time Series.

11. Select Residual Etch Rate for Y, Time Series.

12. Click OK.

13. Click OK for the next dialog setting the number of autocorrelation lags to 19.

The first four residuals are all greater than zero while the next seven are all less than zero. There could be a systematic cause for this behavior, such as an omitted covariate (e.g. operator or ambient temperature). Though it is beyond the scope of our discussion, the Time Series platform may be used to detect correlations between subsequent runs. Furthermore, detecting patterns in a residual by time plot is analogous to detecting out-of-control conditions on a control chart (e.g. using the Western Electric Rules). If the residual by time plot signals as out of control according to these rules, it could indicate a shift in the behavior of the process during the course of the experiment.

14. Leave Etch-Rate open for the next exercise.

Example 3.4 Test for Equal Variances

1. Select Analyze > Fit Y by X.

2. Select Etch Rate and click Y, Response.

3. Select Power and click X, Factor.

4. Click OK.

5. Click the red arrow next to One-way Analysis of Etch Rate By Power and select Unequal Variances.

As discussed in the textbook, the Levene test is robust to the assumption of normality, whereas the Bartlett test is extremely sensitive to this assumption. We saw in the previous example that the data appear to have been generated from a process that can be modeled with the normal distribution, so we may use Bartlett’s test, which has a p-value of .9332. There is no evidence that the variance of etch rate differs across the levels of the power setting. Further discussion of the tests for equal variances produced by JMP is available from the JMP help documentation.

6. Select Window > Close All.

Example 3.5 Analysis of Variance

1. Open Peak-Discharge.jmp.

2. Select Analyze > Fit Y by X.

3. Select Discharge and click Y, Response.

4. Select Method and click X, Factor.

5. Click OK.

6. Click the red triangle next to One-way Analysis of Discharge By Method and select Unequal Variances.

The Levene test rejects the hypothesis of equal variances with a p-value of 0.0032. By default, JMP produces the result of Welch’s test, which is a generalization of ANOVA with unequal population variances to factors with more than two levels. Instead, we will apply a variance-stabilizing transformation to the Discharge variable.

7. Select Analyze > Fit Model.

8. Check Keep dialog open. This will enable us to return to the model dialog to make changes to the model.

9. Click Run.

The p-value of the F test is <.0001 indicating that the treatment means are not all equal. However, the Residual by Predicted Plot shows that the assumption of constant variance (homoscedasticity) is violated: the variance of the residuals seems to grow in proportion with the level of discharge (heteroskedasticity). To remedy this, we will take an appropriate transformation, the square root transformation, of Discharge and perform an ANOVA on the transformed variable.

10. Return to the Fit Model dialog.

11. Select Discharge under Pick Role Variables, and then click the red triangle to the right of Transform. Select Sqrt.

It would also be possible to create an additional column in the data table that contains the transformed values. An advantage of using the transform option, however, is that the predicted values from the Fit Model report are automatically transformed back to the original scale for prediction.

12. Click Run.

The Residual by Predicted Plot for the ANOVA of the transformed response does not show the same increasing “funnel” of variance that appears in the plot for the analysis of the original response.

13. Select Window > Close All.

Example 3.7 Tukey Multiple Comparisons

NOTE: It is possible to perform the multiple comparisons procedures of the next three examples either using the Fit Y by X platform or the Fit Model platform. The next three examples use Fit Y by X, and the example labeled Section 3.8.2 uses Fit Model.

1. Open Etch-Rate.jmp.

2. Select Analyze > Fit Y by X.

3. Select Etch Rate and select Y, Response.

4. Click Power and select X, Factor.

5. Click OK.

6. Click the red arrow next to One-way Analysis of Etch Rate By Power and select Compare Means > All Pairs, Tukey HSD.

The LSD (least significant difference) Threshold Matrix reports positive values in components that correspond to significantly different pairs of treatment effects. This information is summarized in the Connecting Letters Report. The Ordered Differences Report lists the pairwise comparisons according to magnitude of the difference between the treatment means. The Tukey procedure indicates that all pairs of means are significantly different.

7. Leave the Fit Y by X report open for the next two examples.

Example 3.8 Fisher Multiple Comparison

1. Return to the Fit Y by X platform from the previous example.

2. Click the red arrow next to One-way Analysis of Etch Rate By Power and select Compare Means > Each Pair, Student’s t.

The interpretation of this report is the same as it was for Tukey’s test. However, this procedure does not control for the experimentwise error rate. Notice how the confidence intervals for the differences between treatment means are smaller when created using the Fisher Multiple Comparison method than the same intervals that are created when using Tukey’s method.

3. Leave the Fit Y by X report open for the next example.

Example 3.9 Dunnett’s Multiple Comparison

1. Return to the Fit Y by X platform from the previous example.