Standard randomized clinical trials designed to test positive hypotheses produced larger effect estimates than noninferiority trials evaluating similar patients, interventions, comparators, and outcomes, according to a recent study.
When blinding was high risk or unclear, effect estimates from standard randomized clinical trials (RCTs) with positive hypotheses were larger than those from noninferiority trials, with ratios reaching 1.81, whereas no meaningful differences were observed among trials with low risk of bias for blinding. Across 98 meta-analyses comprising 468 RCTs, standard trials with positive hypotheses produced effect estimates 1.47 times larger than noninferiority trials. Standard trials testing negative hypotheses produced effect estimates comparable to noninferiority trials.
To assess the influence of researchers’ hypotheses on RCT results, researchers conducted a matched cohort meta-research study reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology statement. Noninferiority RCTs published through December 2023 were identified using Web of Science, and eligible meta-analyses were selected if they included both noninferiority and standard trials evaluating similar patients, interventions, comparators, and outcomes. Standard trials were classified as testing positive or negative hypotheses based on prospectively registered protocols, retrospectively registered records, or published articles, using a structured, multireviewer classification process.
Eligible trials were required to have comparable clinical scope and to be initiated within 20 years of one another. Data abstraction included effect measures, intervention type, clinical specialty, outcome type, funding source, country income level, and risk of bias related to performance and detection blinding. Effect estimates were standardized by transforming risk ratios, hazard ratios, and mean differences into odds ratios. Within each meta-analysis, pooled effect estimates for standard and noninferiority trials were compared, and ratios were synthesized across meta-analyses using two-step random-effects models.
“The researchers’ hypotheses may bias the results of published RCTs, especially those with high or unclear risk of bias for blinding,” noted lead study author Yuanxi Jia, PhD, of the Alice Lee Center for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, and colleagues. Subgroup analyses showed no difference in effect estimates with low blinding risk, but larger estimates in standard trials with positive hypotheses when blinding risk was high or unclear.[C2]
Several limitations were also noted. The analysis relied on published meta-analyses, leaving the possibility of residual heterogeneity between standard and noninferiority trials. Hypotheses were not explicitly reported in many standard trials, potentially leading to misclassification. In addition, the search was limited to Web of Science, and publication bias could not be fully excluded.
Full disclosures can be found in the published study.
Source: Annals of Internal Medicine
