2 files

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE.

journal contribution
posted on 18.06.2019, 15:17 by RL Morgan, KA Thayer, N Santesso, AC Holloway, R Blain, SE Eftim, AE Goldstone, P Ross, M Ansari, EA Akl, T Filippini, A Hansell, JJ Meerpohl, RA Mustafa, J Verbeek, M Vinceti, P Whaley, HJ Schünemann, GRADE Working Group
The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach.


This research was supported by the Intramural Research Program of the National Institute of Environmental Health Sciences and the MacGRADE Centre at the McMaster University.



Environment International, 2019, 122, pp. 168-184

Author affiliation

/Organisation/COLLEGE OF LIFE SCIENCES/School of Medicine/Department of Health Sciences


VoR (Version of Record)

Published in

Environment International


Elsevier for Pergamon



Acceptance date


Copyright date


Available date


Publisher version