ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis

The reporting of primary studies or results might be influenced by the P value, magnitude, or direction of the study results; this influence can lead to bias in a meta-analysis, because the available evidence (from studies or results) differs systematically from the missing evidence

Existing methods mostly help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis, which leaves users to decide their own approach for combining the risks of each type of bias into an overall judgment of risk of bias in a meta-analysis result

The ROB-ME (risk of bias due to missing evidence) tool is a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results

The tool consists of three preliminary steps: select and define which metaanalyses will be assessed, determine which studies meeting the inclusion criteria for the meta-analyses have missing results, and consider the potential for missing studies across the review; these steps inform an assessment of risk of bias due to missing evidence in a particular meta-analysis result.

The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.