Читать книгу Applied Univariate, Bivariate, and Multivariate Statistics - Daniel J. Denis - Страница 76

The factors that influence the size of a p‐value are, of course, not only relevant to z‐ and t‐tests, but are at work in essentially every test of statistical significance we might conduct. For instance, as we will see in the following chapter, the size of the F‐ratio in traditional one‐way ANOVA is subject to the same influences. Taken as the ratio of MS between to MS error, the three determining influences for the size of p are (1) size of MS between, which is a reflection of the extent to which means are different from group to group, (2) size of MS error, which is in part a reflection of the within‐group variability, and (3) sample size (when computing MS error, we divide the sum of squares for error by degrees of freedom, in which the degrees of freedom are determined in large part by sample size). Hence, a large F‐stat does not necessarily imply that MS between is absolutely large, no more than a large t necessarily implies the size of . A small p‐value associated with a computed F could be a result of small within‐group variation and/or a large sample size. It does not necessarily mean that group‐to‐group mean differences are substantial, which was presumably the goal of the study or experiment by the investigator. That is, the goal was not to simply obtain small within‐group variation. The goal was to demonstrate mean differences between groups.

These ideas for significance tests apply in even the most advanced of modeling techniques, such as structural equation modeling (see Chapter 15). The typical measure of model fit is the chi‐square statistic, χ², which as reported by many (e.g., see Bollen, 1989; Hoelter, 1983) suffers the same interpretational problems as t and F regarding how its magnitude can be largely a function of sample size. That is, one can achieve a small or large χ² simply because one has used a small or large sample. If a researcher is not aware of this fact, he or she may decide that a model is well‐fitting or poor‐fitting based on a small or large chi‐square value, without awareness of its connection with n. This is in part why other measures, as we will see, have been proposed for interpreting the fit of SEM models (e.g., see Browne and Cudeck, 1993).