An encouraging assessment of methods to inform priorities for updating systematic reviews

Objective: To consider the use of statistical methods that aim to prioritize the updating of a collection of systematic reviews based on preliminary literature searches.

Study Design and Setting: A new simulation-based method estimating statistical power and the ratio of the weights assigned to the predicted new and old evidence, and the existing Barrowman n approach is considered. Using only information on the numbers of subjects randomized in the "new" trials, these were applied retrospectively, by removing recent studies, to existing systematic reviews from the Cochrane Infectious Diseases Group.

Results: Twelve systematic reviews were included. When the removed studies were reinstated, inferences changed in five of them. These reviews were ranked, in order of update priority, 1, 2, 3, 4, and I I and 1, 2, 3, 4, and 12 by the Barrowman n and simulation-based power approaches, respectively. The low ranking of one significant meta-analysis by both methods was due to unexpectedly favorable results in the reinstated study.

Conclusion: This study demonstrates the feasibility of the use of analytical methods to inform update prioritization strategies. Under conditions of homogeneity, Barrowman's n and simulated power were in close agreement. We encourage further, prospective, evaluation of these methods.