Evaluation

Dan Williams, MEASURE Evaluation

MEASURE Evaluation’s approach to evaluation contains five guiding principles:

1. Evaluation is ideally planned for alongside program implementation.
2. It is crucial to work with country programs as well as with partners at the global level to identify needs and develop evaluation agendas.
3. Evaluation activities are carried out in partnership with local institutions to build their capacity to plan for and conduct evaluations.
4. Where appropriate, cost components are included to inform studies of cost-effectiveness or cost-benefit to help governments and donors plan for the cost of scaling up effective programs.
5. Plans for dissemination and use of study results are incorporated from the outset.

MEASURE Evaluation staff have expertise in a full range of evaluation procedures including the development of rigorous study designs, protocol development, ethical research practices (including access to U.S.-based and in-country ethics review boards), sample size estimations, questionnaire development, statistical methodologies, costing methodologies, qualitative methodologies, biomarker measurement, and monitoring and evaluation (M&E) capacity building. This expertise is applied to several types of evaluation:

Impact Evaluation

Impact evaluation uses research methods and statistical analyses that measure change in population based outcomes (usually health status) that can be attributed to a program intervention while adequately ruling out other potential factors. The data come from special studies and require strong study designs.

Outcome Evaluation

Outcome evaluation measures whether or not a program or intervention has resulted in the desired changes or has attained the desired results. Outcomes usually take less time to measure than impacts since they are more proximal to the intervention, such as changes in service quality or use, in knowledge and attitudes, or in behaviors and practices. Data for outcome evaluation usually come from a special study, and choice of study design is important. Although the study designs may be less rigorous than those used for impact evaluation, outcomes can be plausibly linked to the intervention.

Public Health Evaluation

Public health evaluation, the term used by the U.S. President's Emergency Plan for AIDS Relief, includes outcome and impact evaluation, and also operations research (OR). OR is a type of outcome evaluation.

Methodological Studies

Methodological studies expand measures and methods for evaluating programs, M&E systems, and health information systems. These studies may be diagnostic and identify measurement gaps that constrain evaluation, or they may be evaluative and test innovative measurement methods and approaches.

Population-based Surveys and Surveillance

Population-based surveys and surveillance measure the prevalence of behaviors, health status, and certain life events, and when conducted over time, they can measure changes and trends. MEASURE Evaluation also conducts secondary analysis of data of population-based surveys and surveillance to understand in more depth the factors associated with behaviors and health outcomes.

Evaluations to Improve M&E and Health Information Systems

MEASURE Evaluation is a leader in developing M&E tools and methods and evaluating health information systems (HIS). In addition to outcome and impact evaluation, staff conduct formative evaluations and pilot tests/proof of concept studies (also known as exploratory/diagnostic studies) to inform the development of M&E and HIS tools and approaches.