Performance of health workers after training in integrated management of childhood illness in Gondar, Ethiopia.

Performance of health workers after training in integrated management of childhood illness in Gondar, Ethiopia E.A.F. Simoes,1 T. Desta,2 T. Tessema,2 T. Gerbresellassie,2 M. Dagnew,3 & S. Gove4 The performance of six primary health workers was evaluated after following a 9-day training course on integrated management of childhood illness (IMCI). The participants were selected from three primary health centres in the Gondar District, Ethiopia, and the course was focused on assessment, classification, and treatment of sick children (aged 2 months to 5 years) and on counselling of their mothers. Immediately following this training, a 3-week study was conducted in the primary health centres to determine how well these workers performed in assessing, classifying and treating the children and in counselling the mothers. A total of 449 sick children who presented at the three primary health centres during the study period were evaluated. Most of the complaints (87%) volunteered by the mothers (fever, cough, diarrhoea, and ear problems) were covered by the IMCI charts. The assessment of commonly seen signs (tachypnoea or ear pain) or easily identifiable signs (slow return after skin pinch, wasting, or pedal oedema) was good, with sensitivities of 67-91%, whereas the assessment of uncommonly seen signs (dry mouth, corneal clouding) or less easily quantifiable signs (eyelid pallor, absence of tears) had a fair or poor sensitivity of20-45%. The classification of pneumonia, diarrhoea with signs of dehydration, and malnutrition showed sensitivities of 88%, 76%, and 85% and specificities of 87%, 98%, and 96%, respectively. However, the classification of febrile illnesses had a sensitivity of only 39% due to problems in using the draft algorithm in areas with a mixture of high, low, and no malaria risk, and due to confusion between axillary and rectal temperature thresholds. Of 39 children classified as having severe disease, 9 were misclassified, mostly by one nurse. Treatment of patients improved over the three weeks of observation, their completeness increasing from 69% to 88%. Health workers usually communicated appropriate advice to the mother. They learned to use checking questions but failed to adequately solve problems in the majority of cases. The mother's counselling card, which summarized recommendations on feeding and home fluids, and advice on when to return, was widely used to aid communication. The time taken to perform the complete management of children did not change significantly (20 to 19 minutes) during the study. Lessons from our findings have been incorporated into an improved version of the IMCI charts. Introduction This article describes the first test of the draft train- ing materials for integrated management of child- hood illness (IMCI) by first-level primary health Department of Pediatrics, Infectious Diseases Section, Univer- sity of Colorado Health Sciences Center, Denver, CO, USA. Re- quests for reprints should be sent to Dr Eric A.F. Simoes, Section of Infectious Diseases, Box B070, The Children's Hospital, 1056 E 19th Avenue, Denver, CO 80218, USA. 2 Department of Pediatrics, Gondar College of Medical Sciences, Gondar, Ethiopia. 3Division of Community Health, Gondar College of Medical Sciences, Gondar, Ethiopia. 4Division of Child Health and Development, World Health Organi- zation, Geneva, Switzerland. Reprint No. 5819 workers. Previous studies on the IMCI algorithm have assessed the performance of trained workers to use only the first case management chart, which was to assess and classify the sick child (1, 2). In Gondar District, Ethiopia, the health workers were trained in the entire process of case management (assessment, classification and treatment of the sick child and counselling of the mother). Qualitative information collected during the course provided feedback for improving the training materials. The primary objective of the study was to deter- mine how well the health workers could assess, clas- sify and treat ill children (aged 2-59 months) and counsel their mothers after receiving training using the draft version of the WHO/UNICEF course on IMCI. Our observations included assessing whether the mother was adequately taught to deliver key Bulletin of the World Health Organization, 1997, 75 (Supplement 1): 43-53 © World Health Organization 1997 43 E.A.F. Simoes et al. treatments and whether advice was effectively given. Other objectives of the study were to determine the average time, using the system, to manage one sick child and how the recording form was used after training, and to compare the adequacy of the perfor- mance of a trained health worker using the draft IMCI algorithm, with that of an expert paediatrician. The IMCI course was designed to teach inte- grated management of sick infants and young chil- dren to first-level health workers in primary care settings that have no laboratory support and only a limited number of essential drugs. The process in- volves assessment of signs and symptoms of illness and the nutritional and immunization status of the children. The illness or illnesses are classified and appropriate treatments identified for each classifica- tion. The course emphasizes making appropriate, timely decisions for referral of the seriously ill child to hospital and giving important pre-referral treat- ments (such as the first dose of an antibiotic or quinine) prior to referral. Health workers are also taught how to provide treatment in the clinic, such as oral rehydration therapy and antibiotics, and to teach the mother to give specific treatment at home. Emphasis is placed on training in communication skills to improve counselling on feeding, home treat- ments, and when to return. When a child comes for follow-up, the health workers are taught how to re- assess the problem and provide appropriate care. The pretest version of the algorithm differed from the final version since it included an exami- nation.of all children for Bitot's spots; the classifi- cation of dehydration included absence of tears, dry mouth, and dry eyes, with distinctions between "dry" and "very dry"; and the malaria algorithm for low-risk areas was based on a rectal temperature of ¢'38°C (or feels hot) and no runny nose or measles. Materials and methods Site and participants Gondar in Ethiopia was chosen as the study site for several reasons. The outpatient and inpatient clini- cal facilities in Gondar College of Medical Sciences serve a large number of patients and several paedi- atricians were available for training as course facilitators, and subsequent participation in the study. The mix of malaria-risk settings varied from low in Gondar town to high in the surrounding low- lying areas which are served by two primary health care centres near Lake Tana (Teda and Koladuba). August was chosen for the study to ensure adequate numbers of cases of malaria and diarrhoeal disease, in addition to the other common and non-seasonal paediatric problems. The three primary health cen- tres were within a 100-km radius and could be accessed every day. Six outpatient clinic nurses, who met the follow- ing criteria, were chosen to participate in the study: they could understand English (the draft training materials were written in English); the two selected from each of the three primary health centres would continue to work in their centre for a year after the training; and they could leave their workplace for 2 weeks to undergo training. These nurses had 12 years of basic education, followed by 2-3 years of clinical training. One was a 23-year-old recent gradu- ate; the others were aged 30-36 years and had practised for 10-15 years with almost no inservice training. The heads of the three primary health centres agreed to allow the 3-week study following the training course. Clearance to conduct the study was obtained from the Gondar College of Medical Sciences. Adaptation of clinical guidelines and feeding advice A few adaptations of the draft generic chart were made in preparation for the pretest. For example, a large proportion of the children in Gondar had con- junctival inflammation and blepharitis, which made the assessment of eyelid (conjunctival) pallor diffi- cult or impossible. The guidelines were therefore adapted to use either palmar pallor or conjunctival pallor to detect anaemia. While the generic sick child algorithm takes into account settings with both high and low malaria risk, the fact that in most of Gondar town there is no risk of malaria was added to the chart. Prior to the training course, a nutritional an- thropologist and a local nutritionist spent 4 weeks in Gondar and the surrounding areas conducting interviews with local mothers, health workers, and nutritionists to determine appropriate age-specific feeding recommendations and a list of common, po- tentially modifiable feeding problems. This included recommendations for energy- and nutrient-rich com- plementary foods and snacks which were available and affordable. Help was provided by the Ethiopian Nutritional Institute (ENI) to determine the calorific value and the protein and micronutrient contents of locally available foodstuffs. With the help of a local nutritionist, local paediatricians, and the ENI, recommendations were made for feeding of children aged 2 months to 5 years. These recommendations were used to produce locally a mother's counselling card, which included line drawings of the recom- WHO Bulletin OMS. Vol 75, Suppl. 1, 199744 Training in integrated management of childhood illness in Gondar, Ethiopia mended foods for mothers unable to read Amharic (the local language). Common, potentially modifi- able feeding problems were described in the adapted training course; these included too much reliance on breast milk (it was common not to give complemen- tary foods in the first year and to give them infre- quently in the second year); complementary foods that were too dilute (diluted milk or muk) or of poor variety; lack of vitamin-A-rich foods for the children; use of bottle-feeding sometimes; sudden weaning of the child when the mother becomes pregnant; and failure to give the infant or young child meat, chicken or fish when these are eaten by the family. Training Two paediatricians, a community health clinician, and an experienced facilitator from WHO were first given 5 days of preparation in the content of the course and in facilitator techniques. They then served as course director and facilitators for the six health workers from the three primary health cen- tres in Gondar District (Gondar polyclinic, Teda and Koladuba). Although the written materials were in English, the course was conducted in Amharic and English. Training was completed in 9 days using the pre- test version of the IMCI course. This draft version of the training course included modules on assessment and classification of a sick child, how to identify treatment, treat the child, counsel the mother, and follow-up; no materials were available on the young infant. After the first day of the course, each morning was spent in an outpatient clinic and on the inpatient wards. In addition to the module, training materials included three case management wall charts (Assess and classify the sick child aged 2 months up to 5 years; Treat the child; and Counsel the mother), booklets of the wall charts, recording forms for the assessment of the sick child, and a draft video and photo booklet. The facilitator techniques for the modules (including instruction in individual feed- back on written exercises, role plays, group discus- sions, and drills) and clinical practice were described in draft facilitator guides. Clinical training was carried out in the out- patient department of Gondar hospital and the Gondar town polyclinic. Severely ill children were assessed and classified in the inpatient services of the Paediatric Department of Gondar College of Medi- cal Sciences. Observational study Following the 9 days of training, the health workers returned to their primary health centres for 3 weeks of follow-up. During this period, the two health workers at each centre were exclusively as- signed to seeing all the sick children who came to that centre. The study was conducted during normal working hours from 08:30 to 16:30. Patients present- ing in the evening and at night for emergency treat- ment were not evaluated in this study. Three paediatricians (TD, TT, TG) who spoke Amharic were assigned, one to each centre while the paedia- trician (EAFS) who did not speak Amharic assisted in the physical examination and diagnosis of the children. Any sick child who presented at the primary health centre during the study hours was evaluated by one of the two trained health workers, using the three case management charts (taped to the walls of the health centre) or the chart booklet. They used a draft recording form which included only the as- sessment. For the purposes of the study, the health workers were provided with the appropriate medica- tions utilized by this course (oral sulfamethoxazole + trimethoprim (co-trimoxazole), chloroquine, paracetamol, iron, vitamin A, tetracycline eye oint- ment, oral rehydration solution, and chloramphe- nicol and quinine for intramuscular use). The paediatricians observed the health workers' performance in assessing, classifying, and treating the children and in counselling the mother for every alternate patient, and filled out detailed forms for each of these four aspects of management. A four- channel timer was used to measure how long the assessment, classification, treatment and counselling took. The assessment of counselling included an estimation of whether the mother was advised ap- propriately (using simple language and appropriate explanations), whether the nurse asked checking questions to make sure the mother understood the advice, whether the nurse avoided leading questions, and whether the nurse asked about problems and helped to solve them. After the interaction with the nurse, the paediatrician (assisted by another paedia- trician in some cases) conducted an independent as- sessment of the child using the IMCI classification scheme to make a clinical diagnosis and an expert assessment of the need for admission. No laboratory results were available to the paediatrician. In ap- proximately half the number of children, observation of the nurse's case management was not performed and only the paediatrician's independent assessment of the child was carried out. Any changes in manage- ment caused by a disagreement in classification and treatment were administered by the physician. For a sample of mothers, an exit interview was carried out by Amharic-speaking nurses trained in interviewing techniques to determine their understanding and re- call of the health worker's advice. WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 45 E.A.F. Simoes et al. Table 1: Description of patients examined in the three primary health centres No. of health workers' No. of patients examinations observed by No. aged examined the paediatrician No. of girls 2-11 months Gondar polyclinic 224 95 (42)a 119 (53) 59 (23) Teda health centre 121 57 (47) 65 (54) 59 (49) Koladuba health centre 104 47 (45) 57 (55) 43 (41) Total 449 199 (44) 241 (54) 161 (36) a Figures in parentheses are percentages. Results Patients and complaints A total of 449 sick children were seen (Table 1) during the 3 weeks of observation, almost half of whom were at the polyclinic in Gondar town. There was almost equal distribution of males and females, and a third of the patients were aged 2-11 months. In Gondar and Koladuba the patients were seen almost equally between the two health workers, but in Teda one health worker saw only a third of the patients. A total of 44% of interactions between health workers and children were directly observed by the paediatrician. The majority of presenting complaints (87%) volunteered by the mothers were fever, cough, diar- rhoea, and ear problems, all of which are covered by the IMCI charts (Fig. 1). The remaining complaints (13%), not covered by the charts, included skin le- sions, abdominal pain, and other problems. Further questioning of these patients (as directed by the charts) led to classification of most of them; the main problem and management of only 10 patients (5 with scabies and one each with ascariasis, congenital syphilis, sinusitis, penile skin infection, and rectal prolapse) were not covered by the charts. The single major condition in Gondar not covered by the charts was scabies, which was seen in 56 children (12.5%), in five of whom it was the sole disease. Assessment of the child The recognition of clinical signs and symptoms is presented in Table 2. Sensitivity is defined here as the proportion of children with a sign of disease identified by the paediatrician and also correctly identified by the health worker, while specificity is the proportion of children with no signs of disease correctly identified; the "gold standard" in these analyses was the paediatricians' assessment of the child using the definitions of clinical signs which were included in the draft IMCI course. The sensitivity and specificity of around 90% for tachypnoea is com- parable with the values in other published studies. However, the sensitivity of chest indrawing was low (62%) and mostly due to misclassification by one of the six health workers. The health workers' evaluations of patients dif- fered considerably from those of the physicians for certain signs, such as the general condition of the child, presence of tears, and dry mouth. Low sensi- tivity may have been due, in part, to the changing status of the child between examinations by the health worker and physician and the treatment ad- ministered. Other signs of dehydration, such as dry eyes and a slow or very slow return of the skin after pinching, were identified more reliably. Often the health workers either forgot to ask the duration of diarrhoea or took an inaccurate history, which re- sulted in a low sensitivity for detection of persistent diarrhoea. Perhaps because blood in the stools is culturally considered very dangerous, the sensitivity and specificity in obtaining a history of blood in the stool was the highest of all the signs and symptoms Fig. 1. Presenting complaints volunteered by patients or their mothers at the first visit. Ear problems Skin lesions Abdominal pain Eye discharge Dental problems Neck swelling Generalized swelling Anorexia Rectal prolapse Headacnes Not recorded ] 0 10 20 30 40 50 60 70 Number of children W Covered by IMCI chart (87%) _ Not covered by MCI chart (13%) WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 I I I I I I I I I 46 Training in integrated management of childhood illness in Gondar, Ethiopia Table 2: Recognition of clinical signs by the health worker, compared with that by the paediatrician Main symptom Sign No. identified Sensitivity (%) Specificity (%) Cough or difficult breathing Tachypnoea 111 91 89 (n = 254)a Chest indrawing 18 62 98 Diarrhoea (n = 217) General condition 21 20 96 Dry eyes 10 90 97 Absence of tears 9 22 99 Dry mouth 8 25 98 Drinking poorly 4 50 97 Skin pinch slow 17 71 97 Ear problem (n = 52) Ear pain 41 83 100 Ear discharge 41 66 95 Malnutrition and anaemia Visible severe wasting 6 67 99 (n = 449) Conjunctival pallor 55 45 94 Palmar pallor 57 37 91 Corneal clouding 4 25 99 Foamy patches (Bitot's spots) 4 50 99 Bipedal oedema 16 69 98 a Figures in parentheses are the numbers assessed for each main symptom. assessed in the study. Health workers obtained a history of ear pain almost as frequently as physicians, but ear discharge was missed in almost a third of patients. The recognition of fever and the classification of malaria had a poor sensitivity (39%), compared to the paediatrician's classification of fever. Most errors occurred for several reasons. Health workers used the rectal temperature cut-offs on the chart (38 °C) while taking an axillary temperature (without converting to the 37.5 °C cut-off). resulting in a significant loss in sensitivity compared to the paedia- trician's classification of fever. The malaria classifica- tion table for high and low malaria risk areas, to which were added instructions for areas with no malaria risk, were not correctly used. The presence of a significant number of children with causes of fever that were not included on the charts, including hepatitis, abscesses, and gum infections, also caused problems. All children were assessed for malnutrition and anaemia. The nutritional status of many children in Gondar is poor, kwashiorkor is common, and 38% of children have a weight-for-age Z-score of less than -2. The weight-for-age chart was used for every child to classify "some malnutrition". Two out of six children with visible severe wasting were missed by the health workers. In only two instances, when the weight fell on the borderline area close to the line, were patients misclassified. Palmar pallor was missed in two-thirds of the patients, mainly because the health workers compared the palms of the child and the mother. Often the mother also had pallor, invalidating the comparison. All four children with corneal clouding were identified by the health work- ers. but in 3 of them the clouding had occurred many years previously. Bitot's spots (foamy patches on the eyes) were identified 50% of the time. Bipedal oedema was reliably detected. Classification of illness Table 3 shows a summary of the classification of children who presented at the three health centres. While 254 children were assessed with cough or dif- ficult breathing, 112 were classified with pneumonia or severe pneumonia. Compared with the paediatri- cians, the health worker correctly identified pneumo- nia or severe pneumonia with a sensitivity of 88% and a specificity of 87%. In the 217 children with diarrhoea, dehydration and dysentery were correctly classified in the majority of children, but a history of persistent diarrhoea was often missed. The classifica- tion of the febrile child was problematic, with the correct classification occurring in only 39%. Acute ear infections characterized by ear pain were cor- rectly diagnosed more frequently than chronic ear infection characterized by pus in the ear. Overall, the recognition of some or severe malnutrition was excellent. Classification of severe disease Comparison of health worker's use of the IMCI algo- rithm with paediatrician's use of the IMCI algorithm (Table 4). There were 38 children with 41 classifica- tions of severe disease who should have been re- ferred, based on the IMCI algorithm. The health workers misclassified 14 as non-severe, of which 11 WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 47 E.A.F. Simoes et al. Table 3: Health workers' classification of sick children compared with that by the paediatrician Health workers' classification compared to the paediatrician's Classification by the No. Sensitivity Specificity Main symptom No. assessed paediatrician identified (%) (%) Cough or difficult breathing 254 Pneumonia/severe pneumonia 112 88 87 Diarrhoea 217 Some/severe dehydration 17 76 98 Persistent diarrhoea 32 56 98 Dysentery 54 94 97 Fever 248 Malariaa 39 39 99 Ear problem 52 Acute ear infection 20 65 75 Chronic ear infection 32 56 85 Malnutrition/anaemia 449 Some/severe malnutrition 171 85 96 a Based on fever only. Blood smear results were not available. Table 4: Health workers' classification of children who were classified as severe disease by the paediatrician Health workers' classification Paediatrician's IMCI Severe: treated Not severe: sent Not severe: sent home classification No. identified and referred home with treatment without treatment Severe pneumonia 18 11 6 1 Severe dehydration 2 1 1a 0 Severe persistent diarrhoea 1 1 0 0 Severe febrile disease 2 0 2a 0 Severe malnutrition/anaemia 18 14 4a 0 a In one child from each category, the health worker identified a different severe classification and consequently referred the child. would have received antimicrobial or other specific treatments at home but one would not have been treated. These 14 missed severe classifications were in 13 children, three of whom had another severe classification and would have been referred (one with severe malnutrition in whom chest indrawing and a classification of severe febrile disease were missed, one with severe malnutrition in whom severe dehydration was missed, and one with severe dehy- dration in whom severe malnutrition was missed). Failure to identify chest indrawing was respon- sible for most missed referrals; 4 out of 7 of these misclassifications were due to one health worker. In six children in whom chest indrawing (severe pneu- monia) was missed, five had fast breathing as well and received oral antibiotics. Two children with se- vere febrile disease were missed but one had another classification indicating severe disease that would have merited referral. Of the three children with severe malnutrition, all had pedal oedema and one would have been referred for severe dehydration. Three children had been referred by the health worker, but not by the paediatrician, using the IMCI algorithm, two on the grounds of chest indrawing and one with bipedal oedema. Comparison of paediatrician's use of IMCI algorithm with paediatrician's expert estimation of need for referral. Of the 449 children seen in the study, the paediatricians decided on clinical grounds that 398 could be sent home; 51 needed referral to a hospital, 32 of whom needed urgent referral for immediate management and 19 for non-urgent care (for investi- gation of tuberculosis (7), persistent diarrhoea (4), congenital heart disease (4), drainage of abscesses (2), and neurological investigation (2)). The analysis which follows includes only the children who needed urgent referral. In this analysis, the paediatrician's use of the IMCI algorithm is compared to the expert paediatri- cian's estimation of the need for admission. The IMCI algorithm would have referred 27 patients cor- rectly, failed to refer 5 patients (who were not clas- sified as severe using the algorithm) who needed admission, and referred 11 patients who did not re- quire admission (sensitivity 84%, specificity 97%, WHO Bulletin OMS. Vol 75, Suppl. 1, 199748 Training in integrated management of childhood illness in Gondar, Ethiopia Table 5: Quality of treatment given by the health worker, as observed by the paediatrician Week 1 Week 2 Week 3 Complete 69% 77% 88% Incomplete 28% 20% 12% Incorrect 3% 3% 0% positive predictive value 71%). Of the five patients not referred using the algorithm, three had bron- chiolitis (two with audible wheezing) and two had pneumonia. All had fast breathing but none had chest indrawing. Of the 11 patients who would have been referred by the IMCI algorithm but were judged by the paediatrician not to need admission, 7 had pneumonia, one had croup, two had visible severe wasting and one had febrile seizures. Comparison of health worker's use of IMCI algorithm with paediatrician's expert estimation of need for referral. Using the IMCI algorithm the health work- ers would have referred 21 patients correctly, not referred 11 patients who needed admission, and re- ferred 10 patients who did not require admission (sensitivity 66%, specificity 98%, positive predictive value 68%). Of the 11 patients not referred using the algorithm, 5 were the same ones missed by the doc- tor's use of the algorithm, 3 had chest indrawing and 3 had pedal oedema. Of the 10 patients who would have been over-referred, 7 were also over-referred by doctors using the algorithm (five with pneumonia and two with visible severe wasting); additionally, chest indrawing (in two persons) and pedal oedema (in one) were identified by the health worker using the IMCI algorithm, but these were not recom- mended for referral by the paediatrician. Treatment of children During the study period, the completeness of treat- ment of a patient improved steadily from 69% in the first week to 88% in the third week (Table 5). In Table 6: Performance of trained health workers in giving antimicrobial treatment Correct Watched treatment Dose shown to mother give selected the mother the first dose Pneumonia 98% 89% 48% Dysentery 97% 92% 64% Malaria 95% 50% 17% the first two weeks, 3% of children were treated incorrectly; in the third week no one was treated incorrectly. The antimicrobial treatment of pneumonia, dysentery and malaria was studied closely (Table 6): whether the correct dose was calculated, and whether the health worker demonstrated how to ad- minister it and actually watched the mother adminis- tering the first dose. Treatment and dosage followed the IMCI guidelines in 95-98% of children. The cor- rect dose was demonstrated to the mother, but the health worker's witnessing the administration of the first dose was more common for pneumonia and dysentery than for malaria. Of the 449 children, 178 required antimicrobial therapy, 98 for pneumonia, 56 for dysentery, 20 for malaria, and 4 for acute ear infection (Table 7). Oral antimicrobials were not administered in 12 of these children on account of severe disease and referral to hospital (6 children), or they went home with pneumonia (2), dysentery (2), malaria (1) and ear infection (1) without appro- priate antimicrobial treatment (Table 7). The recommended treatment of diarrhoea in- cluded the advice to increase fluids during the illness, to reduce milk or give yogurt, and to recommend home ORS (oral rehydration salts) therapy for chil- dren with some dehydration. Most of the children (94%) with some or no dehydration were advised to increase fluids during their illness and 82% of chil- dren with diarrhoea were recommended home use of ORS, while 73% with persistent diarrhoea were advised to reduce milk or give yogurt. Every child with some or severe dehydration received oral rehydration therapy. Table 7: Distribution of patients requiring but not given antimicrobial treatment by the health workers No. with severe disease No. sent home without No. requiring treatment No. not given treatment referred to hospital antimicrobial treatment Pneumonia 98 5 3 2 Dysentery 56 5 3 2 Malaria 20 1 - 1 Ear infection 4 1 - 1 WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 49 E.A.F. Simoes et al. Table 8: Appropriateness of advice given to mothers by the health workers Advice on: Good Fair Poor Treatment 83% 14% 3% Feeding 78% 18% 4% When to return 62% 30% 8% Counselling the mother The communication between the health worker and parent was closely monitored, and qualitative interpretations were made by the paediatricians observing them. The appropriateness of the advice given to the mother involved three important areas: treatment, feeding, and when to return. While 83% and 78% of cases were given appropriate advice for treatment and feeding, respectively, only 60% of mothers received good advice on when to return (Table 8). Problem solving was not observed 68% of the time for drug instruction, 69% for nutrition ad- vice and 810% for advice on when to return. Checking questions were not asked 35%, 35%, and 53% of the time, respectively, for these same categories. During the 3-week study period, the mother's card was almost always (93-95%) used for giving advice on feeding and on when to return. Overall, the mother appeared confused at the end of the assessment 7.2% of the time, and health workers prioritized advice correctly 72% of the time. Table 9 shows the extent of the mother's recall of the feeding advice at the exit interview. Time required for case management The total amount of time spent on each patient came down from almost 20min to 18min over the 3-week period (Fig. 2). About half of this time was spent in the assessment and classification of the child, a few minutes on treatment, and the rest on advising the mother. Table 9: Mothers' recall of nutrition advice at exit interview Give any thick, nutritious complementary food which 88% is on the list Fitfit (injera in pieces) 55% Dinnich (potato) 69% Genfo (porridge) 60% Add butter 29% Add egg 64% Add undiluted milk 52% Add beans/legumes (for child _-12 months) 35% If family eats meat, chicken, fish, give some to the child 28% Give a vitamin-A-containing food (gommen,a carrot, liver) 68% a A green leafy vegetable. The assessment recording form was initially de- signed as a teaching aid for use during the training period and in the first few weeks after training. Ini- tially (in the first week), the health workers would fill the form almost completely most of the time and only the relevant parts in <20% of cases (Fig. 3). However, by the third week, 60% filled in only the relevant parts of the form, which indicates in- creasing familiarity with the process of assessment and classification. Fig. 2. Time for assessment of children during the three weeks of the study. 25 - 20 - ci)a .S 15- E c, 0) E 5 0 _L z Advice E Treatment 3Classification Assessment - Week 1 Week 2 Week 3 Fig. 3. How the recording form was used by health workers during the three weeks of the study. 100- 80- "m 60- a) Q-I 40 0) CY) 0C)i VVeeK I W Totally filled 3 Partially filled * Relevant parts filled WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 vveeK 2 vvee?K J 50 Training in integrated management of childhood illness in Gondar, Ethiopia Discussion Overall, the health workers performed well in the clinic after IMCI training. Their use of the case man- agement guidelines resulted in the correct classifica- tion of children and appropriate antimicrobial and oral rehydration therapy, when compared to the as- sessment by the paediatrician. More problems were encountered in correctly identifying children for re- ferral. In this study, 87% of the presenting com- plaints volunteered by the mothers were covered by the charts, which were designed to include the ma- jority of childhood conditions seen in developing countries; to keep a balance between simplicity and completeness entailed leaving out some conditions. Our test of the draft training materials identified specific problems. Many changes were therefore in- corporated into the course materials which were subsequently field-tested in the United Republic of Tanzania (3). The classification of illness performed better than the detection of individual clinical signs (Table 4). Thus, the classification of pneumonia or severe pneumonia, dehydration, dysentery, and mal- nutrition was good (sensitivities in the range 76-94% and specificities 87-98%). Dependence on a history to make a classification (e.g. duration of diarrhoea for persistent diarrhoea, duration of ear drainage for ear infection) or use of a complex chart (fever) led to poorer classification of the illness (sensitivities of 39-65%). Modifications in the algorithm were made in order to reduce these errors in disease classification. Signs of dehydration often changed in the time between exmination by the health worker and by the physician, which explains differences in the assess- ment of the signs and partly accounts for the low sensitivity (20-25%). Thus a child who was quiet but rousable initially may have become irritable at the end of the physician's examination. A child with absence of tears and a dry mouth may, after drinking fluids, have developed tears and a moist mouth. Signs that changed less quickly and were more easily quantifiable such as dry eyes and skin pinch were more reliably identified (sensitivity, 71-90%). The general condition of the child is a very important sign, and many misclassifications were due to misin- terpretation of an irritable child or a drowsy child. Subsequent versions of the algorithm excluded dry mouth and absent tears, and the training materials have been strengthened to improve the clinical rec- ognition of the child's general condition. Some of the reasons for misclassification of the febrile child have been remedied by changing the axillary temperature thresholds on the IMCI charts, by improving the recognition of malaria risk, and by simplifying the malaria classification tables. The cur- rent algorithm still relies on reminders to assess and treat other causes of fever which are not included in the algorithm (e.g. periodontal disease, abscesses, hepatitis, etc.). These will continue to result in over- treatment for malaria. More research is urgently required for the accurate detection of malaria by clinical means and by simple laboratory tests such as dipsticks. Health workers readily identified severe mal- nutrition using visible severe wasting and bipedal oedema, however there was a greater lack of appre- ciation for signs of xerophthalmia. Thus, foamy patches in the eyes (Bitot's spots) were only detected 50% of the time. Of the four children with corneal clouding, one child with a new onset was correctly classified as severe malnutrition. The clouding was identified in the other three, but no importance was ascribed to this finding since they had had scarring for a number of years. This was, in fact, the right decision. Subsequent versions of the course teach that only recent corneal clouding is a sign for referral. Conjunctival pallor was poorly detected, partly because the children in Gondar often had a blepharo-conjunctivitis that obscured the pallor. Palmar pallor was defined by comparing the child's palms with the mother's in order to account for pig- mentation. This may have accounted for the poor sensitivity and specificity of this sign in our study, since both mothers and children were often pale. It may be more appropriate to compare the child's palms with those of a normal health worker. Specific training materials to support recognition of some and severe palmar pallor (such as photos and video examples) were not available for our test and have subsequently been added to the course. The complaint of ear pain was accurately as- sessed, but discharge from the ear was not detected in a third of the patients. Health workers in Gondar did not pay much attention to this common problem which even the parents often ignored. Children with severe disease who are mis- classified could potentially suffer a serious outcome. A significant number of severe classifications were missed, but some of these children were referred because of another condition. In our study most of the missed referrals were due to inaccurate observa- tion of chest indrawing by one health worker. Em- phasis needs to be placed on the accurate recognition of this sign based on sufficient clinical practice during training. Since this is one of the first signs taught in the course, the longer duration of the course can provide more practice on this sign. Local paediatricians, who are familiar with the limited hospital inpatient capacity, would not have WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 51 E.A.F. Simoes et al. referred some of the children with chest indrawing and thus disagreed with the conclusion based on the algorithm. More emphasis needs to be placed on chest indrawing during training and on further re- search to explore ways to safely limit the referral of children with chest indrawing who have no other signs of severity. Overall, correct treatment was delivered to a high proportion of children, rising from 69% in the first week to 88% in the third week (Table 5) as the health workers grew more accustomed to the charts. Antimicrobial treatment was especially good, with 95-98% of children receiving correct treatment (Table 6). No child in this study received unneces- sary antibiotics, one major advantage of the IMCI classification being the clear identification of children who do not require antibiotics. Prior sensitization from the widespread use of oral rehydration therapy in Gondar District through all three health centres probably facilitated correct treatment of diarrhoeal disease and dysentery. After training using the draft course materials, health workers were more likely to give appropriate advice on treatment and feeding than on when to return to the clinic. They usually did not try to solve problems, but in almost two-thirds of cases they used checking questions to determine whether the mother understood the advice given. Training in communi- cation, including more clinical practice, has been improved in subsequent versions of the course. The time taken with each child improved slowly from about 20 min at the start of the study to about 18 min at the end of the third week. A follow-up interview with the six participants, a year later (July 1995), revealed that it took about 3 months of con- stant working with the charts to become thoroughly familiar with them. At this time they reported that they could complete the management of one child in 5-10 min. Twenty minutes may seem to be a long time to spend on one child in some circumstances, but in our study there were an average of 10 sick children for initial visits who were seen each day (450 patients in the three centres over 15 days); other children who came to the health centre for immunization or trauma did not need this case management pro- cess applied to them. Thus, effectively each health worker saw 5-6 new cases of sick children, using the IMCI approach, from 08:30 till 13:30 without signifi- cant delays or longer clinic hours. Our youngest health worker showed steady improvement in assess- ment over the three weeks (20 min to 15min), but she took longer to counsel the mother. Good man- agement of sick children requires effective counsel- ling of the mother, for which sufficient time must be given. It is clear that health workers learned and im- proved during the 3 weeks of the study, and reduced their dependence on the recording form. From this experience we estimate that, after such training, it would take an average health worker several months to incorporate thoroughly and efficiently the IMCI process in their daily practice. Our results demon- strate that this training course can be effective in preparing health workers from first-level health facilities to take good care of sick children under 5 years of age in developing countries in an integrated fashion. Acknowledgements The authors are grateful to the following participants in the study: Gebrehwot, Zerajion, Tsehay, Bosena, Girmai and Asmara. We thank Mary Lung'Aho for assistance with nutritional counselling; Karen Mason for the statistical analysis; Fitaw and Hebretu for conducting the exit inter- views; and Georgina Armendariz and Chandrika John for assistance in preparation of the manuscript. The study was supported by the World Health Organization (Division of Child Health and Development and the Special Pro- gramme for Research and Training in Tropical Diseases), Geneva, Switzerland. Resume Evaluation des prestations des agents de sant6, apres formation A la prise en charge integr6e des maladies de l'enfant A Gondar (Ethiopie) La qualification de six agents de soins de sante primaires a ete evalu6e apres un cours de formation de 9 jours a la prise en charge integree des mala- dies de 1'enfant (IMCI). Les participants ont et6 s6lectionn6s dans les trois centres de soins de sant6 primaires du district de Gondar (Ethiopie); le cours a port6 essentiellement sur l'evaluation, la classification et le traitement d'enfants malades (de 2 mois a 5 ans), ainsi que sur le conseil maternel. DNs la fin du cours, une etude de 3 semaines a ete realisee dans les centres de soins de sante primaires, pour d6terminer la qualite des prestations fournies par ces agents, en matiere d'evaluation, de classification et de traitement des enfants, et de conseil maternel. Un total de 449 enfants malades qui se sont pr6sentes aux centres pendant la p6riode d'etude ont ete 6values. La plupart des symptomes (87%) rapportes spontanement par les meres, a savoir fievre, toux, diarrhee et affections de l'oreille, etaient inclus dans les tableaux de l'IMCI. WHO Bulletin OMS. Vol 75, Suppl. 1, 199752 Training in integrated management of childhood illness in Gondar, Ethiopia L'evaluation de signes couramment observ6s (tachypnee ou otalgie), ou facilement identifiables (signe du pli, emaciation, oedeme pretibial) etait bonne, avec une sensibilite de 67-91%, alors que l'6valuation des signes peu courants (secheresse de bouche, opacites corn6ennes) ou de signes moins facilement quantifiables (paleur des paupieres, absence de larmes) avait une sensibilite faible a moyenne, de 20-45%. La sensibilit6 de la classification des pneumopathies, de la diarrh6e avec signes de deshydratation et de la malnutrition etait respectivement de 88%, 76% et 85%, pour des specificites de 87%, 98% et 96% respectivement. La sensibilite de la classification des pathologies febriles n'etait toutefois que de 39%, en raison des difficultes d'utilisation du projet d'algorithme dans les secteurs ou le risque d'impaludation est variable (6leve, faible ou absent), et de la confusion entre les seuils de temp6rature axillaire et rectale. On a d6nombr6 9 erreurs de classification sur 39 enfants ranges dans la categorie <,maladie grave,>, presque toutes dues a la meme infirmiere. Le traitement des patients a 6te ameliore au cours de la p6riode d'observation, la completude passant de 69% a 88%. Les agents de sant6 ont en g6n6ral donn6 de bons conseils a la mere. Ils ont appris a utiliser les questions de contr6le, mais n'ont pas reussi a r6soudre les problemes dans la majorit6 des cas. La carte de conseil maternel, qui resumait les recommandations concernant l'alimentation et les liquides administres au domicile, ainsi que les conseils sur la necessite de ramener l'enfant, a largement ete utilisee pour faciliter la communi- cation. Le temps necessaire a la prise en charge complete de 1'enfant n'a pas ete significativement modifie pendant l'6tude (20/19 minutes). Les enseignements tir6s de nos r6sultats ont servi a produire une version amelior6e des tableaux de l'IMCI. References 1. Weber MW et al. Evaluation of an algorithm for the integrated management of childhood illness in an area with seasonal malaria in the Gambia. Bulletin of the World Health Organization, 1997, 75 (Suppl. 1): 113- 118. 2. Zucker JR et al. Clinical signs for the recognition of children with moderate or severe anaemia in western Kenya. Bulletin of the World Health Organization, 1997, 75 (Suppl. 1): 97-102. 3. WHO Division for Child Health and Development & WHO Regional Office for Africa. Integrated manage- ment of childhood illness: field test of the WHO/ UNICEF training course in Arusha, United Republic of Tanzania. Bulletin of the World Health Organization, 1997, 75 (Suppl. 1): 55-64. WHO Bulletin OMS. Vol 75, Suppl. 1, 1997 53