Читать книгу Individual Participant Data Meta-Analysis - Группа авторов - Страница 92

Similar to conventional aggregate data reviews, assessing the reliability (quality) of included trials is also an important feature of the checking phase of IPD meta‐analysis projects. In such reviews, this is usually based on the risk of bias, a term that refers to the likelihood that included trials will generate biased results. In particular, the risk of bias assessment tool (RoB 2) can be used to evaluate potential bias in estimates of intervention effects from randomised trials.⁹¹ It includes five domains to be considered for each eligible trial: the randomisation process; deviations from intended interventions; missing outcome data; measurement of the outcome; and selection of the reported result. Within each domain, assessments are guided by multiple signalling questions (with answers: yes, probably yes, probably no, no, or no information), allowing a risk of bias classification for that domain (low, high, or some concerns). Finally, an overall risk of bias judgement can be made (low, high, or some concerns) based on all domains (Section 4.7).

In aggregate data reviews, assessment of risk of bias is usually based on the information available in trial publications and other publicly accessible documents, such as trial registration entries or published protocols, sometimes supplemented by information requested from trial investigators. In an IPD meta‐analysis project, it is common to obtain additional information from protocols, codebooks and forms, or direct from trial investigators, which can increase the clarity of risk of bias assessments compared to those based on trial reports alone.^48,105 As discussed in Sections 3.4 and 4.2.6, it may be helpful to undertake an initial risk of bias or quality assessment at the planning stages, before considering whether to obtain the IPD. However, the collection of IPD does allow a deeper and more reliable appraisal of data quality and risk of bias than is possible with aggregate data, because there is the opportunity to generate information directly from the IPD. There is also the potential to seek additional or updated trial IPD for inclusion in a meta‐analysis, in order to reduce or remove the potential for bias in particular domains. For example, participants excluded from original trial analyses may be reinstated in the meta‐analysis, or more appropriate statistical methods might be used. Therefore, an overall risk of bias assessment for each trial would be based on whether the design and conduct of the trial, and the quality of its final IPD (after correcting any data errors) are likely to lead to biased results when the IPD are analysed.

While assessing risk of bias in an IPD meta‐analysis involves using many of the same domains and items listed in the RoB 2 tool,⁹¹ some items are not relevant, because the IPD circumvents the issue. For example, the availability of IPD avoids reliance on a trial’s original analysis methods and reported results. Table 4.4 lists the domains and signalling questions in the RoB 2 tool that are particularly relevant to an IPD meta‐analysis project, and summarises how appraisal of the collected IPD, and other information harnessed through an IPD approach, can inform risk of bias judgements. These individual domains represent a useful starting point for assessing the risk of bias associated with each trial in an IPD meta‐analysis project.

Ultimately, if the trial design, management or conduct are seriously flawed (e.g. failed randomisation, poor follow‐up, or early stopping), this might lead to trials that are wholly or partially unreliable. Depending on the gravity of the issues, a trial may need to be excluded completely from the analyses or be the subject of sensitivity analysis; at the very least, the issues need to be brought to the attention of the reader in the IPD meta‐analysis publication. The IPD project’s central research team should have a plan in place for how they will deal with these sorts of occurrences. However, the objective of data checking is not to police trials or uncover fraud, and in our experience wholesale fabrication of trial data is uncommon. Also, note that this is probably less likely to occur in large, complex or multicentre trials, where collusion between multiple parties would be required. Nevertheless, sometimes trials are encountered where it is suspected that the data have been purposefully manipulated or falsified, so such trials would need to be excluded from the analyses, and the reasons for doing so described in the meta‐analysis report.

Table 4.4 Domains in the Risk of Bias 2 tool⁹¹ (RoB 2) of particular relevance to IPD meta‐analysis projects

Source: Based on Sterne JAC, Savovic J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898

RoB 2 domains and signalling questions that are relevant to IPD meta‐analysis projects	Additional information available from IPD
Domain 1: Randomisation process
1.1 Was the allocation sequence random? 1.2 Was the allocation sequence concealed until participants were enrolled and assigned to interventions? 1.3 Did baseline differences between intervention groups suggest a problem with the randomisation process?	Assessing the pattern of randomisation using trial IPD may reveal instances where randomisation has failed, or may reassure that the randomisation process appears robust (Section 4.6.1). IPD can also be used to check balance across a full range of covariates (Section 4.6.1). Though formal testing is not recommended, visually, the distribution of each covariate by treatment group can help flag any systematic or unusual differences.
Domain 2: Deviations from the intended interventions (effect of assignment to intervention)
2.1 Were participants aware of their assigned intervention during the trial? 2.2 Were carers and people delivering the interventions aware of participants' assigned intervention during the trial? 2.3 If Y/PY/NI to 2.1 or 2.2: Were there deviations from the intended intervention that arose because of the trial context? 2.4 If Y/PY to 2.3: Were these deviations likely to have affected the outcome? 2.5 If Y/PY/NI to 2.4: Were these deviations from intended intervention balanced between groups? 2.6 Was an appropriate analysis used to estimate the effect of assignment to intervention? 2.7 If N/PN/NI to 2.6: Was there potential for a substantial impact (on the result) of the failure to analyse participants in the group to which they were randomised?	Having IPD means that participants can be analysed by their allocated intervention, irrespective of how they were analysed in the original trial (Section 4.6.2). Then, provided that patients, those delivering treatments or outcome assessors are not aware of the treatments being given, and the outcomes are objective, this would mean there is no risk of bias in this domain. If the IPD do not enable analysis by the allocated intervention, there may be sufficient detail in the trial dataset to check whether participants who deviated from intended intervention did so for pre‐specified or otherwise rational reasons.
Domain 3: Missing outcome data
3.1 Were data for this outcome available for all, or nearly all, participants randomised? 3.2 If N/PN/NI to 3.1: Is there evidence that the result was not biased by missing outcome data? 3.3 If N/PN to 3.2: Could missingness in the outcome depend on its true value? 3.4 If Y/PY/NI to 3.3: Is it likely that missingness in the outcome depended on its true value?	Access to IPD provides opportunity to request data for participants completely excluded from the IPD dataset, and for any outcomes missing for particular individuals (Section 4.6.3). Also, the IPD obtained can be examined to check for any potential undisclosed exclusions (Section 4.6.3). Where all the relevant IPD are provided, any data that were not analysed in the trial can be reinstated in the meta‐analysis, as appropriate. Also, methods for dealing with missing outcome data (e.g. mixed models) can be implemented, even if not undertaken by trial investigators in the original analysis.
Domain 4: Measurement of the outcome
4.1 Was the method of measuring the outcome inappropriate? 4.2 Could measurement or ascertainment of the outcome have differed between intervention groups? 4.3 If N/PN/NI to 4.1 and 4.2: Were outcome assessors aware of the intervention received by study participants? 4.4 If Y/PY/NI to 4.3: Could assessment of the outcome have been influenced by knowledge of intervention received? 4.5 If Y/PY/NI to 4.4: Is it likely that assessment of the outcome was influenced by knowledge of intervention received?	Access to IPD offers some opportunities to redress situations where inappropriate measurement may have been used. For example, component participant‐level data can be used to construct a new composite outcome (Section 4.6.4) and this may also reduce heterogeneity across trials. Even if a range of instruments have been used in trials, a more appropriate one may be obtained via the IPD.
Domain 5: Selection of the reported result
5.1 Were the data that produced this result analysed in accordance with a pre‐specified analysis plan that was finalized before unblinded outcome data were available for analysis?	As new analyses are carried out according to the IPD meta‐analysis project protocol and SAP, this domain is not applicable.

Y: yes, PY: probably yes, N: no, PY: probably no, NI: no information. Note that in RoB 2, Domain 2 has an additional/alternative series of signalling questions for risk of bias arising from deviations from interventions in terms of adhering to interventions. The additional information that may be available from the IPD is similar to those given for the effect of assignment to the intervention in the table. Domain 5 has a sub‐list of questions that are not listed, because they are not relevant to IPD meta‐analysis projects.

Details of ways to evaluate the IPD in relation to the relevant RoB 2 domains are provided in the following sections. Findings of risk of bias and data checking can usefully be presented using a modified version of the ‘traffic light’ table used for standard risk of bias assessments (Section 4.7).