Verbal Case Review for IMCI Clinical Practices

Abstract

• Name: Verbal Case Review (VCR)
• Purpose: VCR is a household-based survey for assessing the quality of clinical care of sick children provided by private practitioners. Private practitioners include unlicenced and traditional providers as well as government and other trained health workers serving the target population.
• Type of Design: Descriptive; cross sectional or longitudinal.
• Sample Size: A sample size of 500 to 800 sick children provides an adequate number of episodes for cross-sectional assessment in most developing country epidemiologic situations. For monitoring, three to six patients per practitioner are needed to provide specific individual feedback during one monitoring cycle.
• Where Used: The methodology has been used in rural and urban settings in Indonesia, India, Pakistan, and Bangladesh

To survey tool

Basic Information

Name: Verbal Case Review

Origin: Derived from IMCI Health Facility Assessment Exit Interview instrument, with substantial modifications and local variations

Source: Robert S. Northrup and Youssef Tawfik, BASICS Project; Sarbani Chakraborty, Johns Hopkins School of Hygiene and Public Health (Baltimore); Retna Siwi Padmawati, Clinical Epidemiology and Biostatistics Unit, Gadjah Mada University Faculty of Medicine, Yogyakarta, Indonesia; and Stephen Luby, Department of Community Health, Aga Khan University School of Medicine, Karachi, Pakistan.

Basic Description: The Verbal Case Review (VCR) is a household survey instrument and methodology used for assessing the quality of clinical care being provided to sick under-five children (under five years) by the full range of practitioners and vendors who are consulted in a target population. Households with children under five years are screened to identify those with children who have been sick with either diarrhea, fever, or cough/respiratory symptoms during the previous two weeks. The mothers of these children are then interviewed using the VCR instrument which asks them to recall from whom they purchased medicines, from whom they obtained clinical care, and the actions or behaviors of those drug sellers and practitioners consulted for the child's illness. The WHO protocol for the Integrated Management of Childhood Illness (IMCI) is the clinical standard from which the questions regarding clinical quality in the interview instrument are derived. Practitioners' behaviors are compared to these standards to determine the gap between current practices and the quality standard.

In settings where initial assessment using the VCR has led to an intervention to improve the quality of care, the VCR has been used in ongoing longitudinal monitoring of care quality. The results of such monitoring have been fed back to the practitioners in monitoring cycles of two-to-three months to stimulate further improvements. Carried out by community representatives, these monitoring activities have also generated improved interaction between the practitioners and the community. They have also indicated to practitioners that the community is concerned with the quality of care being provided and is willing to accept certain changes in care (not giving injections, not giving anti-diarrheal medicines). In some monitoring applications, the VCR has been simplified to facilitate implementation by community members.

Country Applications: Indonesia, India, Pakistan, Bangladesh

Languages Available: English, Indonesian

Technical Scope: Integrated child health services, including nutrition.

Purpose: To assess or monitor the quality of care being provided to sick children by practitioners, either private or public, from the formal, informal, or traditional sectors.

Type of Methods: Quantitative, with the possibility of adding some qualitative components.

Design: May be either cross-sectional or longitudinal, formative or summative. In any format it is descriptive. It has been used to provide an isolated assessment of health care seeking behaviors by mothers; case management of sick children by practitioners; a baseline and/or post intervention evaluation; and ongoing monitoring data to be used in feedback to practitioners as part of a quality improvement intervention strategy.

Frequency of Administration: A baseline cross-sectional assessment may be followed with a subsequent cross-sectional evaluation assessment after sufficient time has been allowed for intervention activities to take place. In India, Indonesia, and Pakistan the interval between surveys was six to nine months.

When the VCR is used by a program for ongoing monitoring and feedback to practitioners, it is carried out continuously in target communities. The VCRs completed during the monitoring cycle (two or three month intervals) are summarized, and individual as well as group reports are prepared and fed back to practitioners.

Key Users of Information

Information on the quality of care provided to sick children, particularly with regard to care being provided by private practitioners, is of immediate interest and use to program managers and health providers in government, NGOs and donor agencies. Agencies which have mounted intervention programs based on VCR results have included local NGOs (India, Pakistan), and local government units (district and subdistrict health offices in Indonesia). Central government health offices and donor agencies have found the information about the private sector particularly useful in planning child health interventions (Planning office in Indonesian MOH, World Bank, USAID, UNICEF). The data has stimulated higher level decision makers to devote additional resources to private practitioners, rather than concentrating solely on the government health system.

Objectives and Scope of Tool

To assess a) health seeking behavior and the comparative usage of the private and public sectors; and b) the quality of care provided to sick children under five years by private and public sector practitioners.

The tool at present is limited to the key child health problems represented in the WHO sponsored IMCI package. In two applications it has also included assessment of the adequacy and effectiveness of care being provided to sick children in the home, prior to seeking external care. The principle of the tool ­ a delayed exit interview aimed at making data gathering on specific types of illness and specific age groups of the population more convenient - may be readily adapted to other aspects of quality of care (e.g. maternal care, family planning, etc). The VCR has been used in one Indian setting to assess family planning and pregnancy related behaviors.

Key Monitoring Needs and Evaluation Questions Tool Seeks to Address

The VCR provides information for the monitoring and evaluation of the quality of care being provided to sick children. It also provides information on the rate of health care seeking outside the home, and comparative usage of different types of providers.

Key indicators

Access:

• % of under 5 children sick with diarrhea during a two week recall period (also fever, ARI); of those
• % seeking care outside the home
• % seeking care from government health center (also from private doctor, private midwife, unregistered doctor, traditional midwife, etc)
• % seeking care from government sector vs private sector
• % purchasing drugs from grocery store (also drug shop, pharmacy)

Quality:

• % of providers consulted for illness episode who are performing case management actions specified in the IMCI clinical protocol for diarrhea (also fever, ARI); a list of the most important follows:

Examination Practices for All Children

• Asked about duration of illness
• Asked what mother gave child at home
• Asked about immunization card
• Recommended immunization
• Weighed the child
• Recorded the weighing result

Examination of Children with Fever

• Asked whether the child has had fever
• Measured the child's temperature using thermometer
• Asked whether the child had convulsions
• Asked whether the child had measles during the previous three months
• Examined for stiff neck

Examination of Children with Cough or Respiratory Problem

• Asked whether the child has cough or breathing problem
• Measured the child's respiratory rate with watch or timer
• Lifted up the child's clothes so that the chest could be seen

Examination of Children with Diarrhea

• Asked whether the child has diarrhea
• Asked about blood in the stool
• Pinched the child's skin

Treatment and Counseling for Children with Fever

• Prescribed/dispensed anti-fever medicine
• Informed mothers about critical signs to watch for
• Informed mother about name/type and purpose of the medicines given
• Explained how to take the medicines
• Advised mothers about feeding the child
• Advised mothers about giving fluids to the child

Treatment and Counseling for Children with Cough/ARI

• Prescribed/dispensed good medicine for cough/ARI
• Informed mother about critical signs to watch for
• Informed mother about name/type and purpose of the medicines given
• Explained how to take the medicines
• Advised mother about feeding the child
• Advised mother about giving fluids

Treatment and Counseling for Children with Diarrhea

• Advised/gave Oralit ORS
• Prescribed/dispensed anti-diarrhea drug
• Informed mother about name/type and purpose of the medicines given
• Explained how to take the medicines
• Advised mother about feeding the child
• Advised mother about giving fluids
• Informed mother about critical signs to watch for

Research Design

Standard protocol:
In assessment or evaluation mode, the standard VCR protocol is a cross-sectional survey of households in the designated sample area.

Households with children under five years are screened to identify those with children who have been sick with either diarrhea, fever, or cough/respiratory symptoms during the previous two weeks. The mothers of these children are interviewed and are asked to recall the actions or behaviors of drug sellers or practitioners consulted for the child's illness. In some applications, this target has been further screened to select only those children whose parents bought medicines outside the house, or consulted a practitioner. To allow determination of rates of illness in the population, and rates of care seeking outside the home, records must be kept of the responses in each of the preliminary screening steps, either in registers or special screening forms which can be analyzed separately from the verbal case review questions.

Units of Observation and Analysis: Units of observation are individual children under five years, and individual practitioners. Analysis typically differentiates responses from drug sellers (who usually do not see the child), and responses from practitioners (who examine the child as part of the case management process). In most applications analysis provides data for practitioners in a targeted community/sample area, and differentiates among practitioners of different types ­ doctors, midwives, nurses, unlicenced practitioners (quacks), drug sellers. Where appropriate, the analysis differentiates private and public sector providers, often by institution ­ health center, private clinic, shop, etc.

In monitoring mode as part of an intervention, the units of analysis are individual practitioners or institutions (e.g. health centers, hospitals), and sample design must use the same incidence factors as those used in the cross-sectional approach. It should also use the same rate of use of individual practitioners or institutions in that population. Statistical significance has not been a priority for monitoring data, but convincing feedback to a particular practitioner must include at least three patients overall. Meaningful discourse on a particular illness (e.g. management of ARI) must include at least three patients with that type of illness, seen by the specific practitioner. In addition, consideration of how many interviews a volunteer community worker or health center staff member can be expected to carry out per month on an ongoing basis must be considered in determining the intervals (e.g. one month, three months) at which the data is collected.

Sample Size:Sample size calculations must take into account the prevalence of households with children under five years in the target area, and the expected incidence of each of the three targeted illnesses. In addition, researchers should consider: the degree to which statistical significance is needed in differentiating among different types of providers, or between private and public sector providers, or between formal and informal sector types of providers, as well as other available information (e.g. from preliminary key informant interviews or DHS surveys) regarding expected patterns of care seeking from particular types of practitioners. Seasonal factors influence the incidence of all three illnesses in many developing countries. In addition, if the VCR survey will be used as a baseline for an intervention that will be evaluated for changes in specific behaviors, calculations should consider the initial rate of performance of the particular targeted behavior (e.g. administration of ORS to only 10% of children with diarrhea seen by private sector comprehensive practitioners) to that expected to be attained after the intervention (e.g. an increase of 400% to 40% of cases seen by the same class of practitioners).

Target Population: The target population may vary depending on the purpose of the research. When it is used as a cross-sectional assessment tool with a randomly selected sub-set of households, the VCR could be applied to a large population (district or national) using a cluster sampling approach. In the majority of applications, however, the VCR has been used as part of an intervention effort, and the targeted population has been the catchment population of the institution implementing the intervention.

Lessons from experience:

• In India, significant changes in practitioner behavior in case management could be seen with a sample of 600 sick children, 300 from each of two local NGO target communities.
• In Indonesia, the low incidence of diarrhea in the target population (as determined from earlier work in the area) meant that the expected prevalence of diarrhea cases had to be used to determine the overall sample size (822 cases overall, which yielded 770 cases that consulted practitioners or bought drugs).
• In the Indian experience, overall populations of 20,000 to 40,000 yielded numbers of practitioners of around 60, or one for 1000-1500 population.

Training

Standard protocol:
Interviewers have been trained for two to three days, with detailed explanation of the reasons for the individual questions, role play in the training site, and observed interviews in the community. The extent of training has varied. In Indonesia professional interviewers were borrowed from an ongoing surveillance project and required less training. In India, village community health workers (paid) or community women were selected on the basis of educational levels and willingness to work outside the household. Thus, training was more extensive. To ensure quality, it is necessary to conduct field level reviews of completed questionnaires, and repeat interviews with 10% or 15% of respondents. Feedback to the interviewers should follow.

Lessons from experience:
It has proven particularly difficult to differentiate clearly between: a) use of drugs available at home from a previous purchase; b) purchase of drugs from a drug seller who does not see the sick child; and c) dispensing of drugs by a practitioner who has examined the child. This aspect must receive adequate attention in training.

Implementation

Standard protocol:
The number of interviewers ranged from two over a two month period, to 25-30 for monitoring VCRs during an intervention. Depending on the number of additions to the basic questions in the survey instrument, the VCR takes from 15 to 45 minutes to administer.

Transportation requirements will vary with sample size.

Basic skills in carrying out surveys, managing forms and data entry into computers, and analyzing survey data are important to enable the process to run smoothly. Community organizations and district or sub-district health offices typically need technical support from a local institution.

Lessons from experience:
External technical assistance has been useful in the development phase of this instrument, helping to establish a set of questions which are answerable based on mothers' recall, manage the survey process and data management issues, and keep the analysis focused on the critical case management issues amenable to intervention. Involvement of people with public health experience and some knowledge of IMCI is very helpful.

Analysis

Standard protocol:
Computer hardware and software requirements: 286 computer running EPI-INFO in DOS is sufficient. More sophisticated analyses are facilitated by higher power computers and use of SPSS or other statistical software package.

Analytical requirements: Simple tabulations of data, and two-way comparisons using Chi square, are the basic outputs, showing rates of performance and incidence rates comparing different subgroups in the sample.

Time and labor requirements: depend on the cleanliness of the data coming from the field. Once the data is clean, preparation of the basic tables can be accomplished in a few days.

Lessons from experience:

Reporting Issues

Collective reports provide tables comparing rates of current performance of desired behaviors with the standards, or with individual targets set as part of individual behavioral contracts made during intervention activities. A software program has also been developed to produce a monitoring report for an individual practitioner based on VCR assessment of patients he or she has seen during the previous monitoring cycle.

Standard protocol:

Lessons from experience:
The standard report from the VCR provides simple tables for the indicators and rates of performance mentioned above, and cites in the text the critical observed gaps between practitioner behaviors and the IMCI based standards of practitioner case management.

Dissemination of Results

Lessons from experience:
The data from the VCR has been presented to practitioners in an intervention target area to stimulate participation in the intervention. The data was presented showing tables which listed desired case management actions and rates (in percent) of actual performance by the group of practitioners. This led to discussion about whether the reported observations were consistent with what the practitioners were doing (they were), and ultimately to agreement to improve those behaviors.

The data has also been presented to government officials at district and subdistrict level to convince them that intervention was useful. In the Indonesian experience, presenting data showing that the private practitioners performed at a higher level of quality than those working in government facilities led the district health officer to demand that the subsequent intervention include the public sector, as well as the private practitioners.

Manuals and Guidelines

References

A paper has been submitted for publication describing the Bihar, India experience with the VCR and subsequent intervention approach

Validity

The VCR was derived initially from the instrument used for exit interviews as part of the IMCI health facility assessment process. It has been validated in India by Sarbani Chakraborty, using a comparison of directly observed actions performed by a practitioner with those reported by the mother of the child being cared for. The study compared reliability of the mother's observations at one, seven, and 15 days after the encounter. The results indicate substantial variation in reliability depending on the particular practice being recalled, some being less accurately noted by the mothers than others. Overall reliability was similar at one and seven days, and showed some decay at the 15 day recall period.

Contact Person
Beth Plowman
BASICS Project
1600 Wilson Blvd.
Arlington, VA 22209
703-312-6800