Indicators for tracking programmes to strengthen health research capacity in lower- and middle-income countries: a qualitative synthesis

Background

The monitoring and evaluation of health research capacity strengthening (health RCS)

commonly involves documenting activities and outputs using indicators or metrics. We

sought to catalogue the types of indicators being used to evaluate health RCS and to assess

potential gaps in quality and coverage.

Methods

We purposively selected twelve evaluations to maximize diversity in health RCS, funders,

countries, and approaches to evaluation. We explored the quality of the indicators and

extracted them into a matrix across individual, institutional, and national/regional/network

levels, based on a matrix in the ESSENCE Planning, Monitoring and Evaluation framework.

We synthesized across potential impact pathways (activities to outputs to outcomes) and

iteratively checked our findings with key health RCS evaluation stakeholders. Results

Evaluations varied remarkably in the strengths of their evaluation designs. The validity of

indicators and potential biases were documented in a minority of reports. Indicators were

primarily of activities, outputs, or outcomes, with little on their inter-relationships. Individual

level indicators tended to be more quantitative, comparable, and attentive to equity

considerations. Institutional and national–international level indicators were extremely

diverse. Although linkage of activities through outputs to outcomes within evaluations was

limited, across the evaluations we were able to construct potential pathways of change and

assemble corresponding indicators.

Conclusions

Opportunities for improving health RCS evaluations include work on indicator measurement

properties and development of indicators which better encompass relationships with

knowledge users. Greater attention to evaluation design, prospective indicator measurement,

and systematic linkage of indicators in keeping with theories of change could provide more

robust evidence on outcomes of health RCS.