Metaevaluations

Metaevaluations, or meta-evaluations, are systematic assessments of evaluations themselves. They examine the quality, utility, and credibility of evaluation activities—how evaluations are designed, conducted, analyzed, interpreted, and reported—rather than the programs or policies being evaluated. The goal is to improve the quality of evaluations and the use of their findings for decision making. Metaevaluations can be conducted by funders, organizations that commission evaluations, or independent bodies, and they may feed back into standard-setting and capacity building.

Common purposes include judging methodological rigor, appropriateness of designs and data sources, transparency of methods, independence and ethics, clarity of reporting, and the degree to which findings inform decisions. Metaevaluations often employ rubrics and checklists to appraise elements such as relevance to stakeholders, alignment with stated objectives, rigor of data collection and analysis, treatments of limitations and uncertainties, and the integrity of conclusions. They may synthesize multiple evaluation reports to identify patterns, best practices, and recurring biases.

Challenges include heterogeneity of evaluations, varying reporting quality, access to data, subjectivity in judgments, and resource demands. Benefits include improving evaluation quality, accountability, knowledge sharing, and the demonstration of learning within programs and funders. Metaevaluation is distinct from meta-analysis of program outcomes; it evaluates evaluation quality and use, whereas meta-analysis aggregates effect estimates. Standards from professional evaluation associations often guide meta-evaluation practices.