Funnel plots may show asymmetry.pdf (1.27 MB)
Download file

Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test

Download (1.27 MB)
journal contribution
posted on 27.08.2020, 10:34 by Brett Doleman, Suzanne C Freeman, Jonathan N Lund, John P Williams, Alex J Sutton
Research Synthesis Methods published by John Wiley & Sons Ltd. This study aimed to determine for continuous outcomes dependent on baseline risk, whether funnel plot asymmetry may be due to statistical artefact rather than publication bias and evaluate a novel test to resolve this. Firstly, we conducted assessment for publication bias in nine meta-analyses of postoperative analgesics (344 trials with 25 348 participants). Secondly, we attempted to resolve the observed asymmetry by considering meta-regression residuals as outcome (rather than mean difference) and (inverse) sample size as the exploratory variable (rather than SE). Since the approach resolved the asymmetry, we evaluated it, and related approaches, using a simulation study considering four scenarios comprised of every combination of baseline interactions and absolute selective publication bias being present or not (10 000 simulated meta-analyses per scenario with no residual between-study heterogeneity). The test based on meta-regression residuals and inverse sample size performed as well as conventional tests (Egger's test) when no baseline risk was present and reduced type I errors when baseline risk was present. It also had modest power to detect publication bias in the presence of baseline risk. We demonstrated that correlation between effect estimates and SEs produces funnel plot asymmetry in the presence of no publication bias for continuous outcomes dependent on baseline risk. Our novel approach of assessing funnel plot asymmetry using a modified funnel plot and test based on residuals and inverse sample size may have improved performance when carrying out publication bias assessments for unstandardized mean differences where treatment effects are dependent on baseline risk.

Funding

SCF is a member of the Complex Reviews Support Unit which is funded by the National Institute for Health Research (project number 14/178/29).

History

Citation

Doleman, B, Freeman, SC, Lund, JN, Williams, JP, Sutton, AJ. Funnel plots may show asymmetry in the absence of publication bias with continuous outcomes dependent on baseline risk: presentation of a new publication bias test. Res Syn Meth. 2020; 11: 522– 534. https://doi.org/10.1002/jrsm.1414

Version

VoR (Version of Record)

Published in

Research Synthesis Methods

Volume

11

Issue

4

Pagination

522 - 534

Publisher

Wiley for Society for Research Synthesis Methodology

issn

1759-2879

eissn

1759-2887

Acceptance date

23/04/2020

Copyright date

2020

Available date

27/08/2020

Spatial coverage

England

Language

English

Publisher version

https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1414