ROB-ME: a tool for assessing risk of bias due to missing
evidence in systematic reviews with meta-analysis

Matthew J. Page; Jonathan A.C. Sterne; Isabelle Boutron; Asbjørn Hróbjartsson; Jamie J. Kirkham; Tianjing Li; Andreas Lundh; Evan Mayo-Wilson; Joanne E. McKenzie; Lesley A. Stewart; Alex J. Sutton; Lisa Bero; Adam G. Dunn; Kerry Dwan; Roy G. Elbers; Raju Kanukula; Joerg J. Meerpohl; Erick H. Turner; Julian P.T. Higgins

doi:10.1136/bmj-2023-076754

ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis

Matthew J. Page, Jonathan A.C. Sterne, Isabelle Boutron, Asbjørn Hróbjartsson, Jamie J. Kirkham, Tianjing Li, Andreas Lundh, Evan Mayo-Wilson, Joanne E. McKenzie, Lesley A. Stewart, Alex J. Sutton, Lisa Bero, Adam G. Dunn, Kerry Dwan, Roy G. Elbers, Raju Kanukula, Joerg J. Meerpohl, Erick H. Turner, Julian P.T. Higgins

Research output: Contribution to journal › Article › peer-review

45 Citations (Scopus)

Abstract

The reporting of primary studies or results might be influenced by the P value, magnitude, or direction of the study results; this influence can lead to bias in a meta-analysis, because the available evidence (from studies or results) differs systematically from the missing evidence

Existing methods mostly help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis, which leaves users to decide their own approach for combining the risks of each type of bias into an overall judgment of risk of bias in a meta-analysis result

The ROB-ME (risk of bias due to missing evidence) tool is a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results

The tool consists of three preliminary steps: select and define which metaanalyses will be assessed, determine which studies meeting the inclusion criteria for the meta-analyses have missing results, and consider the potential for missing studies across the review; these steps inform an assessment of risk of bias due to missing evidence in a particular meta-analysis result.

The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.

Original language	English
Article number	e076754
Journal	BMJ
DOIs	https://doi.org/10.1136/bmj-2023-076754
Publication status	Published - 20 Nov 2023

Access to Document

10.1136/bmj-2023-076754

Revised ManuscriptAccepted author manuscript, 339 KB

Cite this

Page, M. J., Sterne, J. A. C., Boutron, I., Hróbjartsson, A., Kirkham, J. J., Li, T., Lundh, A., Mayo-Wilson, E., McKenzie, J. E., Stewart, L. A., Sutton, A. J., Bero, L., Dunn, A. G., Dwan, K., Elbers, R. G., Kanukula, R., Meerpohl, J. J., Turner, E. H., & Higgins, J. P. T. (2023). ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis. BMJ, Article e076754. https://doi.org/10.1136/bmj-2023-076754

@article{23847d04339447e1a943acba491da725,

title = "ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis",

abstract = "The reporting of primary studies or results might be influenced by the P value, magnitude, or direction of the study results; this influence can lead to bias in a meta-analysis, because the available evidence (from studies or results) differs systematically from the missing evidenceExisting methods mostly help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis, which leaves users to decide their own approach for combining the risks of each type of bias into an overall judgment of risk of bias in a meta-analysis resultThe ROB-ME (risk of bias due to missing evidence) tool is a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study resultsThe tool consists of three preliminary steps: select and define which metaanalyses will be assessed, determine which studies meeting the inclusion criteria for the meta-analyses have missing results, and consider the potential for missing studies across the review; these steps inform an assessment of risk of bias due to missing evidence in a particular meta-analysis result. The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.",

author = "Page, \{Matthew J.\} and Sterne, \{Jonathan A.C.\} and Isabelle Boutron and Asbj{\o}rn Hr{\'o}bjartsson and Kirkham, \{Jamie J.\} and Tianjing Li and Andreas Lundh and Evan Mayo-Wilson and McKenzie, \{Joanne E.\} and Stewart, \{Lesley A.\} and Sutton, \{Alex J.\} and Lisa Bero and Dunn, \{Adam G.\} and Kerry Dwan and Elbers, \{Roy G.\} and Raju Kanukula and Meerpohl, \{Joerg J.\} and Turner, \{Erick H.\} and Higgins, \{Julian P.T.\}",

year = "2023",

month = nov,

day = "20",

doi = "10.1136/bmj-2023-076754",

language = "English",

journal = "BMJ",

issn = "0959-8146",

publisher = "BMJ Publishing Group",

}

Page, MJ, Sterne, JAC, Boutron, I, Hróbjartsson, A, Kirkham, JJ, Li, T, Lundh, A, Mayo-Wilson, E, McKenzie, JE, Stewart, LA, Sutton, AJ, Bero, L, Dunn, AG, Dwan, K, Elbers, RG, Kanukula, R, Meerpohl, JJ, Turner, EH & Higgins, JPT 2023, 'ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis', BMJ. https://doi.org/10.1136/bmj-2023-076754

TY - JOUR

T1 - ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis

AU - Page, Matthew J.

AU - Sterne, Jonathan A.C.

AU - Boutron, Isabelle

AU - Hróbjartsson, Asbjørn

AU - Kirkham, Jamie J.

AU - Li, Tianjing

AU - Lundh, Andreas

AU - Mayo-Wilson, Evan

AU - McKenzie, Joanne E.

AU - Stewart, Lesley A.

AU - Sutton, Alex J.

AU - Bero, Lisa

AU - Dunn, Adam G.

AU - Dwan, Kerry

AU - Elbers, Roy G.

AU - Kanukula, Raju

AU - Meerpohl, Joerg J.

AU - Turner, Erick H.

AU - Higgins, Julian P.T.

PY - 2023/11/20

Y1 - 2023/11/20

N2 - The reporting of primary studies or results might be influenced by the P value, magnitude, or direction of the study results; this influence can lead to bias in a meta-analysis, because the available evidence (from studies or results) differs systematically from the missing evidenceExisting methods mostly help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis, which leaves users to decide their own approach for combining the risks of each type of bias into an overall judgment of risk of bias in a meta-analysis resultThe ROB-ME (risk of bias due to missing evidence) tool is a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study resultsThe tool consists of three preliminary steps: select and define which metaanalyses will be assessed, determine which studies meeting the inclusion criteria for the meta-analyses have missing results, and consider the potential for missing studies across the review; these steps inform an assessment of risk of bias due to missing evidence in a particular meta-analysis result. The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.

AB - The reporting of primary studies or results might be influenced by the P value, magnitude, or direction of the study results; this influence can lead to bias in a meta-analysis, because the available evidence (from studies or results) differs systematically from the missing evidenceExisting methods mostly help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis, which leaves users to decide their own approach for combining the risks of each type of bias into an overall judgment of risk of bias in a meta-analysis resultThe ROB-ME (risk of bias due to missing evidence) tool is a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study resultsThe tool consists of three preliminary steps: select and define which metaanalyses will be assessed, determine which studies meeting the inclusion criteria for the meta-analyses have missing results, and consider the potential for missing studies across the review; these steps inform an assessment of risk of bias due to missing evidence in a particular meta-analysis result. The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.

U2 - 10.1136/bmj-2023-076754

DO - 10.1136/bmj-2023-076754

M3 - Article

SN - 0959-8146

JO - BMJ

JF - BMJ

M1 - e076754

ER -

ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis

Abstract

Access to Document

Fingerprint

Cite this