MEASURE Evaluation - The Polio Eradication Initiative
The Polio Eradication Initiative:
Monitoring Service Delivery during NIDs, Assessing the Local Capacity to Strengthen Disease Surveillance
Abstract
Name: The information and tools reviewed here are found in a report entitled The Polio
Eradication Initiative: Monitoring Service Delivery During National Immunization Days;
Assessing the Local Capacity to Strengthen Disease Surveillance. They evolved from
actual use during National Immunization Days (NIDs) and from field assessments in
several African countries. While this material documents and refers to polio eradication,
the concepts and tools can also be applied to routine health services.
Purpose: To monitor the performance of immunization program activities. Data can be
also be used for program planning. The reference serves as a guide for national and
district immunization program managers to assist them in their efforts to eradicate polio
and improve routine health services. Information collected is to be used to analyze and
help improve the quality of immunization services, as well as identify problem areas and
take corrective action before the next session or round of immunization activity.
Type of Design: Descriptive, cross sectional or longitudinal.
Where Used: Zambia, Kenya, Uganda, Eritrea, Ethiopia
Name: The Polio Eradication Initiative: Monitoring Service Delivery During National
Immunization Days; Assessing the Local Capacity to Strengthen Disease Surveillance.
Origin: Expanded Program on Immunizations (EPI) in Africa; Polio Eradication Initiative of
WHO, USAID, BASICS.
Source: BASICS, with involvement of EPI staff in several African countries.
Basic Description: Guide for national and district immunization program managers to assist them
in their polio eradication efforts and improvement of routine health services, with particular
emphasis on service delivery and disease surveillance.
The guide covers five areas:
Monitoring Service Delivery During NIDs
Assessing the Quality of Immunization Services and Social Mobilization During NIDs
Reporting/Documentation on NIDs
Assessing Local Capacity to Improve Disease Surveillance
Other Resources Available from BASICS on Polio Eradication
Country Applications: Checklists and guides in this document were adapted from actual
examples that were developed and used by EPI managers in Zambia, Kenya, Uganda, Eritrea and
Ethiopia. This tool has applicability throughout Africa and other regions where NIDs and to
strengthen disease surveillance are found. Although this tool was developed based on polio
eradication experience, the content could be adapted for monitoring service delivery and
assessing local capacity to strengthen surveillance for other infectious disease control initiatives.
Languages Available: English.
Technical Scope: Global, regional, national, and district levels for developing and implementing
strategies to improve the quality and sustainability of immunization.
Purpose: To monitor the performance of immunization program activities. Data can be also be
used for program planning.
Type of Methods: Quantitative and qualitative.
Design: Descriptive, cross sectional or longitudinal.
Frequency of Administration: Yearly and/or monthly, depending on NIDs and routine
immunization program needs.
Key Users of Information
Primary users: National and district level EPI managers and supervisors.
Secondary users: Local EPI staff (e.g., health facility level; health workers that monitor/provide
immunization services); international organizations involved in polio eradication and routine
immunization programs.
Objectives and Scope of Tool
Objective: To familiarize users with the "process monitoring" approach for planning and
implementing NIDs, and encourage systematic application of this experience to the routine
immunization program.
Scope: The guide recommends approaches for collecting and analyzing information to:
determine the effectiveness and quality of the NID; enable management to make timely decisions
for improving future NIDs; increase attention to priority issues during supervisory visits; and
utilize data collected during NIDs to reinforce routine program activities and determine the
priorities for future program evaluations.
Key Monitoring Needs and Evaluation Questions Tool Seeks to Address
It is suggested that the user first determine what information is needed to assess the effectiveness
and quality of the NID in areas such as:
Planning;
Monitoring the services provided;
Vaccine supply, distribution and quality;
Service delivery strategy;
Social mobilization.
The following basic questions should be answered at the national, district, and health facility
levels:
1) Was the planning adequate at all levels?
2) Are the reported data on immunization coverage reliable?
3) Were immunizations given safely and effectively?
4) Did the NIDs reach all communities, as planned?
5) Were people adequately informed about NIDs? How were people informed?
Key Indicators
Reliability of Reporting
Indicator 1: # of immunizations given and the reported coverage for children under 12 months of
age; and the reported coverage for the entire target age group (children under five years).
Indicator 2: Geographical areas reporting greater than 100% coverage.
Indicator 3: Geographical areas reporting less than 80% coverage or the level set by
management.
Indicator 4: Timeliness and completeness of reporting.
Indicator 5: Recording practices.
Vaccine Supply, Distribution, and Quality
Indicator 1: Dates OPV received at health facilities and outreach posts.
Indicator 2: OPV supply at delivery point.
Indicator 3: # of immunizations given compared with the total amount of vaccine used.
Indicator 4: Availability of ice during transport of vaccines and immunization sessions.
Indicator 5: Status and knowledge of the Vaccine Vial Monitors.
Indicator 6: Availability, knowledge, and status of cold chain monitors in health facilities with
refrigerators.
Indicator 7: Condition and capacity of refrigerators.
Immunization Delivery Strategy
Indicator 1: Attendance at immunization sessions and coverage of the target population.
Indicator 2: Immunization sites properly located.
Indicator 3: Children receiving OPV during both rounds.
Indicator 4: Mothers' travel time to the immunization site.
Indicator 5: Organization of the immunization session.
Indicator 6: Geographical coverage and coverage by catchment area.
Social Mobilization
Indicator 1: Mother's knowledge of the vaccine given, purpose of NIDs, target age group, and
dates for the next round.
Indicator 2: Mother's source of NIDs information.
Research Design
Process monitoring can provide useful qualitative and quantitative information for management,
improving decision-making capabilities. The following tools and techniques are discussed and
incorporated into this document:
Reporting, including analysis and feedback on the information collected (e.g. "Documentation
Outline: Report on NIDs" for the NID coordinating team);
Supervising, using a standard checklist with analysis and feedback on the information
collected (e.g. "Supervision Checklist - (Ia) for NIDs Immunization Sessions" for use by any
district and local health worker supervising immunization sessions);
Maintaining, reviewing, and discussing minutes from planning meetings (e.g. "Checklist:
District Assessment for Improving Disease Surveillance" for the national EPI group and/or
district health team);
Asking parents questions about NIDs during routine visits to health facilities and during the
NIDs (e.g. "Attachment II(c) - NID Mobilization Assessment" for use by district and local
health workers and/or trained volunteers involved in conducting an NID);
Keeping a diary on all discussions and meetings with health workers and all other NIDs
participants, including NGOs, government officials, the private sector, volunteers, and
religious leaders;
Keeping a diary on observations made at health facilities, outreach posts, and communities.
Implementation
The checklists provided in this document are to be used by the supervisor and health workers as
monitoring and supervision tools during micro-planning or assessment visits for NIDs, as well as
routine monthly or quarterly supervisory visits. It is therefore important that they are available at
the local or district level. Since they are to be used by program staff as part of routine activities,
the only additional cost relates to reproduction of blank checklists (and possible initial translation
into local language, if necessary) as well as some additional time required by the supervisors or
health workers to complete and assess the checklists.
Analysis
The supervision and social mobilization tools were designed to monitor service delivery or social
mobilization during NIDs. They provide an analytical approach to supervision and offer a broad
perspective on the quality of services for one to five immunization sites, ten sites, or more.
Using the suggested format, a supervisor can determine a score or rate for the various indicators
and gain an overall picture of the area to determine where improvements are needed in
immunization programs and services. The disease surveillance checklist enables district health
teams to determine the level of available capacity and resources, and plan and budget disease
surveillance and response strategies.
Use of these tools requires constructive monitoring. Supervisors and health workers must
expend time and effort on exchanging information and observations, providing realistic and
appropriate advice and feedback between levels, and conducting cooperative planning activities
based on the data collected. Checklists can be manually completed and should be stored at the
local and district levels. Computers are not required; although the checklists and data could be
entered into a computerized system like EpiInfo. The "Report on National Immunization Days"
should be clear and concise, and should be typed or, preferably, entered and printed via
computer.
Reporting
The documentation outline for reporting on NIDs is designed to guide the NID coordination team
in the preparation of an informative final report. The report will document the year's NIDs and
can also be used as a guide for planning and implementing future NIDs. In addition, this report
will fulfill donor requests for documentation on the use of funds for the Polio Eradication
Initiative. The checklists should be utilized and kept at the local and district levels, with copies
going to the national level, as required by the national immunization program. The "Report on
National Immunization Days" may need to be shared with the NID coordinating committee, the
districts, EPI and MOH staff, donor organizations, and other external contributors (such as
Rotary Clubs, businesses, local leaders, churches, and NGOs).
Manuals and Guidelines
Guidelines are contained within the document. They should be used in conjunction with NIDs
and routine immunization materials (developed by WHO, UNICEF) that are already in use in
many countries.
References
A bibliography of BASICS reports on polio eradication and disease surveillance in various
countries is attached as an appendix to this document. Related materials from WHO can be
referenced and ordered from the "WHO GPV Catalogue '98", WHO/GPV/98.02 (also available
on the Internet at: www.who.ch/gpv-documents).
Other
The spreadsheets and lists in the WHO publication, "Field Guide for Supplementary Activities
Aimed at Achieving Polio Eradication", (WHO/EPI/GEN/95.01 Rev.1) are useful tools for
monitoring the preparation for NIDs. This WHO guide should be used as a complement to the
Guide reviewed here.
Contact Person
Robert Steinglass
BASICS Project
1600 Wilson Blvd.
Arlington, VA 22209
703-312-6800
