Читать книгу Applied Univariate, Bivariate, and Multivariate Statistics - Daniel J. Denis - Страница 74

The simplicity of these demonstrations is surpassed only by their profoundness. In our simple example of the one‐sample z‐test for a mean, we have demonstrated that the size of z_M is a direct function of three elements: (1) distance , (2) population standard deviation σ, and (3) sample size n. A change in any of these while holding the others constant will necessarily, through nothing more than the consequences of how the significance test is constructed and functionally defined, result in a change in the size of z_M. The implication of this is that one can make z_M as small or as large as one would like by choosing to do a study or experiment such that the combination of , σ, and n results in a z_M value that meets or exceeds a pre‐selected criteria of statistical significance.

The important point here is that a large value of z_M does not necessarily mean something of any practical or scientific significance occurred in the given study or experiment. This fact has been reiterated countless times by the best of methodologists, yet too often researchers fail to emphasize this extremely important truth when discussing findings:

A p‐value, no matter how small or large, does not necessarily equate to the success or failure of a given experiment or study.

Too often a statement of “p < 0.05” is recited to an audience with the implication that somehow this necessarily constitutes a “scientific finding” of sorts. This is entirely misleading, and the practice needs to be avoided. The solution, as we will soon discuss, is to pair the p‐value with a report of the effect size.