Читать книгу Evidence in Medicine - Iain K. Crombie - Страница 35

Small trials often give misleading estimates of treatment benefit. Several review studies, each of which examined hundreds of trials, have shown that, on average, small trials report greater effect sizes than larger ones [72–74]. Two explanations have been suggested for this finding: small studies with negative findings may be less likely to be published, and small studies may be of poorer methodological quality and more prone to bias [73]. Most likely both factors contribute to the bias.

A related phenomenon is that unusually large treatment effects are most commonly reported by small trials [75]. These often occur in the first trial of a new treatment, with subsequent trials showing much smaller effects [76–78]. A possible explanation for this is that small studies are much more likely to be influenced by the play of chance [79]. A few more events (e.g. deaths) in one treatment group, or a few less in the other, can have a large effect on small studies. When averaged across many trials, chance effects cancel out, but for an individual study it can generate large, misleading effect sizes.

The fragility index is used to identify just how susceptible statistically significant results are to the play of chance [80]. It measures how many fewer events would have to occur to change a significant treatment effect to a non‐significant one. Reviews have found that for many trials the index is one i.e. if one patient had a different outcome the finding would not be statistically significant [81, 82]. In general, the smaller the value of the index, the more fragile the study. Several reviews of trials have reported the median values of the index of 1, 2, 3 and 4 [82–84], indicating that, for half of the trials included in these reviews, a different outcome in a few patients would change the statistical significance. Other reviews have found slightly larger median fragility indices of 5 and 8 [80, 85].