Trends in the quality of health care for children aged less than 5 years in Afghanistan, 2004–2006

940 Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Trends in the quality of health care for children aged less than 5 years in Afghanistan, 2004–2006 Anbrasi Edward,a Vikas Dwivedi,b Lais Mustafa,c Peter M Hansen,d David H Peters a & Gilbert Burnhama Objective To study trends in the quality of the health care provided to children aged less than 5 years in Afghanistan between 2004 and 2006. In particular, to determine the effect on such quality of a basic package of health services (BPHS), including Integrated Management of Childhood Illness (IMCI), introduced in 2003. Methods In each year of the study, 500–600 health facilities providing the BPHS were selected by stratified random sampling in 29 provinces of Afghanistan. We observed consultations for children aged less than 5 years, interviewed their caretakers, interviewed health-care providers and measured adherence to case management standards for assessment and counselling in a random sample. Findings The quality of the assessment and counselling provided to sick children aged less than 5 years improved significantly between 2004 and 2006. A 43.4% increase in the assessment index and a 28.7% increase in the counselling index (P < 0.001) were noted. Assessment quality improved significantly every year and was statistically associated with certain characteristics of the provider (being a doctor, having a higher knowledge score, being trained in IMCI, being part of a “contracting-in” mechanism and providing a longer consultation time) and the child (being younger and having a female caretaker). Counselling quality was also significantly associated with these characteristics, except for provider cadre and child age. The presence of clinical guidelines and the frequency of supervision were significantly associated with improved quality scores in 2006 (P < 0.05 and < 0.01, respectively). Conclusion Quality of care improved over the study period, but performance remained suboptimal in some areas. Continued investments in Afghanistan’s health system capacity are needed. Une traduction en français de ce résumé figure à la fin de l’article. Al final del artículo se facilita una traducción al español. .ةلاقلما هذهل لماكلا صنلا ةياهن في ةصلاخلا هذهل ةيبرعلا ةمجترلا a Johns Hopkins University, 615 North Wolfe Street, Baltimore, MD, 21205-2179, United States of America. b EPOS Health Consultants, Bad Homburg, Germany. c Ministry of Public Health, Kabul, Afghanistan. d GAVI Alliance, Geneva, Switzerland. Correspondence to Anbrasi Edward (e-mail: aedward@jhsph.edu). (Submitted: 13 May 2008 – Revised version received: 29 January 2009 – Accepted: 5 March 2009 – Published online: 25 August 2009 ) Introduction Afghanistan has some of the poorest health indicators in the world. According to available estimates, in 2002 Afghanistan had an infant mortality rate of 165 per 1000 live births, an under-five mortality rate of 257 per 1000 live births and a maternal mortality ratio of 1600 per 100 000 live births.1 Many different factors impede access to health services and their effective delivery; they include lack of health infra- structure and human resource capital (especially female health providers), low literacy and other cultural barriers, and poor road access and security conditions. Since 2002, the Ministry of Public Health and its donor partners have made considerable investments in Afghanistan’s health system infrastructure and human resources.2–5 The Ministry of Public Health prioritized the implementation of a basic package of health services (BPHS) to improve health service coverage and quality. This implementation has been facilitated in part through contracts with nongovernmental organizations (NGOs) and the Ministry of Public Health providers. Under an innovative “contracting-in” mechanism, state providers are sometimes contracted by the government under the same conditions and targets set for NGOs and are eligible for performance bonuses.6 Services are implemented at three health facility levels: basic health centres, comprehensive health centres and district hospitals. According to a report, in 2006 approximately 82% of the Afghan population had access to basic health services.6 Since 2004, the BPHS has been assessed through Afghani- stan’s National Health Service Performance Assessment. This assessment provides an overall measure of system performance and patient care by employing a balanced scorecard (used to manage performance in large and complex organizations) to measure six performance domains: patient and community perspectives, staff perspectives, capacity for service provision, service provision, financial systems and overall vision.7,8 The overall score for health system performance improved be- tween 2004 and 2006.8–10 However, these gains need to be sustained, and additional efforts are needed to address those performance indicators that remain suboptimal. More than 100 countries now use the Integrated Manage- ment of Childhood Illness (IMCI) strategy to address the major disease burden in children aged less than 5 years. The IMCI strategy is integrated into Afghanistan’s BPHS and is included in the country’s health and nutrition sector strategy.1,11 Despite controversies and debates on global investments in IMCI, the strategy remains a promising cost-effective way to improve quality of child health care and reduce child mortality.12–18 Afghanistan initiated IMCI training in 2003. The training is or- ganized by NGOs that provide the BPHS, with IMCI facilitators from the Ministry of Public Health and donor organizations. More than 2300 health providers (mostly physicians) and 25 941Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in AfghanistanAnbrasi Edward et al. facilitators have been trained by means of the standard 11-day course. This paper focuses on a subset of the data from the balanced scorecard to assess trends in the quality of health care for children less than 5 years in 2004–2006 and the factors associated with those changes. Methods This study, which was approved by the Institutional Review Board at Johns Hopkins University and the Ministry of Public Health Ethical Review Board in Afghanistan, presents data from the 29 provinces of Afghanistan that were included annually in the National Health Service Performance Assessment from 2004 to 2006. The methods used for designing the instruments and sampling health facilities are described elsewhere.8,9 Under the BPHS, each level of health-care facility has a specified level of human resource and infrastructure support. For each year of the study, up to 25 facilities were selected from each province through stratified random sampling of 3 district hospitals, 7 comprehensive health centres and 15 basic health centres. Trained survey teams comprising clinicians, nurses or vaccinators were selected from each province to perform the observations and interviews. Obser- vation of case management was based on a systematic sample of five children aged 2 months to 5 years. Children were selected using a random starting point and a sampling interval deter- mined by the average number of new patients seen in a day. Characteristics of patients and providers were similar across the sample of cases and provid- ers observed. Providers included clini- cians, nurses and others responsible for clinical management of sick patients in the facility. To determine facility characteristics, four providers randomly selected from a list of providers were interviewed. To determine assessment and counselling quality, 10 indicators related to patient assessment were ag- gregated to create an assessment index and 5 indicators related to counselling were aggregated to create a counselling index. Each indicator was given equal weight and scored on a percentage scale. In relation to IMCI, we considered the extent to which health-care providers adhered to case management procedures. To identify determinants of the quality of clinical and interpersonal care, we used bivariate and multivariate analysis of patient, provider and facility-related factors known to affect such quality. More than 98% of the caretakers and providers sampled responded. The same facilities, providers and patients were not compared, since the sample was selected randomly each year. We used standard procedures for quality control of data editing in the field, followed by double data entry and consistency checking. Initially, we used univariate analysis to examine the pattern of responses and extent of miss- ing values to construct the indexes. The missing values (< 5%) were omitted from the analysis and confirmed to be missing at random. We used bivariate analysis to compare differences for indicators of assessment and counsel- ling between 2004 and 2006 and to compare various groups of the selected independent variables. Multiple linear regression models were constructed us- ing ordinary least squares to compare the quality index between the groups, with clustering at the facility level ac- counted for using Huber-White esti- mates of standard error. The model fit was examined using residual plots, and multicollinearity was assessed by esti- mating the variance inflation factor.19 Results Table 1 describes the study sample. Overall, about one-third of the children observed were less than 1 year of age, and diarrhoea was the most prevalent present- ing symptom. About 80% of the cases were seen by doctors or assistant doctors in all 3 years of the study, and the rest by nurses or midwives. The proportion of female health-care providers interviewed was higher in 2006 than in 2004. Quality of care for sick child con- sultations was determined by adherence to selected indicators on taking a history, physical examination and counselling. Most assessment and counselling indica- tors improved significantly between 2004 and 2006 (Table 2). For example, the proportion of the providers observed who checked for danger signs in the child’s ability to drink or breastfeed increased from about 40% in 2004 to about 60% in 2006; those who asked about diar- rhoea, fever and cough or difficulty in breathing increased from less than 70% in 2004 to more than 80% in 2006; and those who explained the disease condition to the caretaker and provided instructions for home care of the sick child and administration of medications increased from about 55% in 2004 to about 68% in 2006. In some areas, performance remained poor in 2006, despite significant improve- ments over the study period. These areas included checking the signs for anaemia (17% in 2006), oedema (19%) or the immunization card (29%); and giving instruction on signs and symptoms re- quiring an immediate return (< 45%). Factors that did not improve significantly over the study period were providers ex- plaining adverse reactions to medications and measures to undertake when these reactions occurred. The mean scores for both the assess- ment and counselling indexes improved significantly between 2004 and 2006 (Table 2). The results of bivariate analysis of several factors (patient, provider and facility-related) that affect the quality of care are shown in Table 3 and Table 4. Provider cadre and sex: Performance of health-care providers across all cadres improved between 2004 and 2006. Doctors performed significantly better than assistant doctors and nurses in both assessment and counselling in 2005 and 2006. Sex differences were evident, with female providers showing significantly higher adherence to assessment standards than male providers in 2005 and 2006, and to counselling standards in 2006. Child sex and age: There were no significant differences for assessment and counselling quality in relation to the sex of the child, but adherence to assessment standards was higher when providers were attending children less than 2 years of age. Caretaker sex: Children accompanied by female caretakers received significantly better care at facilities in 2005 and 2006 than those who had male caretakers. Consultation time: In all the years of the study, providers who spent 10 or more minutes in a consultation provided better care, as shown by significantly better as- sessment and counselling quality. Type of health facility: Assessment quality was significantly better in com- prehensive health centres in 2005 and in district hospitals in 2006. The quality of counselling was significantly better in comprehensive health centres than in district hospitals in 2004 and 2005, but the difference was not signifi- cant in 2006. Mean scores indicate that over the years of the study care improved 942 Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in Afghanistan Anbrasi Edward et al. Table 1. Characteristics of children included in study of the quality of health care provided to children aged less than 5 years, by year, Afghanistan, 2004–2006 Characteristic Year 2004 2005 2006 No. (%) No. (%) No. (%) Age in months 2–11 546 (25.9) 1084 (38.9) 955 (33.4) 12–23 546 (25.9) 654 (23.5) 736 (25.7) 24–35 475 (22.6) 470 (16.9) 520 (18.2) 36–47 268 (12.7) 267 (9.6) 334 (11.7) 48–59 271 (12.9) 312 (11.2) 313 (10.9) Sex Male 1297 (51.3) 1496 (53.9) 1487 (52.4) Female 1232 (48.7) 1282 (46.2) 1350 (47.6) Major presenting symptoms Diarrhoea 1087 (42.6) 1326 (47.6) 1276 (44.6) Fever 560 (21.9) 521 (18.7) 575 (20.1) Cough/difficulty breathing 431 (16.9) 432 (15.5) 494 (17.3) Othera 474 (18.6) 508 (18.3) 513 (17.9) Caretaker Mother 1619 (70.5) 1795 (66.5) 1908 (66.8) Father 409 (17.8) 627 (23.2) 603 (21.1) Other female 169 (7.4) 138 (5.1) 162 (5.7) Other male 100 (4.4) 141 (5.2) 184 (6.4) Type of health facility Basic health centre 296 (52.7) 364 (58.6) 378 (61.1) Comprehensive health centre 221 (39.3) 215 (34.6) 199 (32.2) District hospital 45 (8.0) 42 (6.8) 42 (6.8) Provider interviewedb Doctor 367 (25.7) 519 (36.1) 582 (34.5) Assistant doctor 152 (10.7) 461 (32.1) 431 (25.5) Nurse 325 (22.8) 103 (7.2) 54 (3.2) Otherc 582 (40.8) 355 (24.7) 622 (36.8) Sex of provider interviewedb Male 1047 (75.2) 924 (65.7) 1058 (64.3) Female 346 (24.8) 483 (34.3) 587 (35.7) Provider observedd Doctor 1673 (65.6) 1907 (68.4) 2189 (76.6) Assistant doctor 510 (20.0) 316 (11.3) 135 (4.7) Nurse 295 (11.6) 467 (16.8) 445 (15.6) Otherc 74 (2.9) 97 (3.5) 89 (3.1) Sex of provider observedd Male 2345 (94.2) 2560 (92.0) 2646 (92.7) Female 144 (5.8) 223 (8.0) 209 (7.3) a Skin infection, pus, injury, earache, jaundice, etc. b Data from interviews of health-care providers in each health facility. The sex of the provider was missing for some of the data. c Midwife, auxiliary midwife, pharmacist, technologist and vaccinator. d Data from case-management observations of health-care providers and patients. in all types of facilities, without any ob- servable trend between types of facilities. Provider knowledge, satisfaction, refresher training and IMCI training: The proportion of providers who received refresher training was more than 13% greater in 2006 than in 2004. The in- crease in the proportion trained in IMCI was modest; more providers reported IMCI training in basic health centres and comprehensive health centres than in district hospitals. Knowledge scores were significantly higher in 2006 than in 2004. Provider knowledge, job satisfac- tion, refresher training and IMCI training were associated with better performance in assessment and counselling. Supervision, availability of clinical guidelines and shura-e-sehie (village health committees): Facilities that received six or more supervisory visits in the previous 6 months and those with case manage- ment guidelines provided significantly better quality of care in 2006. The presence of active shura-e-sehie also significantly improved the quality of care in 2006. Type of contracting: Although there was an overall improvement in perfor- mance in all contracting mechanisms between 2004 and 2006, providers in facilities with a contracting-in mecha- nism had higher mean scores for assess- ment and counselling across the 3 years of assessment than those in facilities with other contracting mechanisms. Table 5 shows the results of multiple linear regression analysis using selected patient, provider and facility character- istics. The quality of care was higher in 2005 and 2006 than in 2004, upon hold- ing other variables constant. Other factors that were significantly associated with quality of care were high provider knowl- edge, presence of IMCI-trained providers, support of facilities by the contracting-in mechanism, the provider being a doctor, a consultation time of at least 10 minutes, having a female caretaker and the child being aged less than 24 months. Other factors that were significant in the bivari- ate analysis were not significant predictors of quality in the multivariate analysis. The model explained 18% of the variance in assessment quality and 10% of the vari- ance in counselling quality. Discussion Efforts to restructure the health system in Afghanistan have improved over- all health-care provider training and satisfaction, the availability of drugs and equipment, service provision, the availability of clinical guidelines, in- frastructure and the use of the health management information system. This is illustrated by the balanced scorecard results for 2004 to 2006.9 This study demonstrates that spe- cific improvements are also being made in the quality of the clinical care pro- vided to children aged less than 5 years, specifically in relation to IMCI. The re- sults indicate significant improvements between 2004 and 2006, particularly in the assessment of danger signs and presenting complaints. Nonetheless, the mean assessment and counselling index in 2006 yielded an average score 943Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in AfghanistanAnbrasi Edward et al. Table 2. Provider performance with respect to patient assessment and counselling in study of the quality of health care provided to children aged less than 5 years, by year, Afghanistan, 2004–2006 Assessment and counselling indicator 2004 2005 2006 Change from 2004 to 2006 (%) (n = 2552) (n = 2787) (n = 2858) No. % No. % No. % Asked about or checked if Drinking or breastfeedinga 1030 41.8 1472 54.8 1742 63.0 +21.1* Vomiting everythinga 946 38.6 1545 57.6 1838 66.4 +27.8* Lethargic or unconsciousnessa 423 17.2 748 27.9 754 27.2 +10.1* Having convulsionsa 402 16.3 330 12.3 555 20.1 +3.8* Presenting complaint Diarrhoeaa 2012 81.5 2360 87.7 2454 88.7 +7.2* If diarrhoea present Duration 993 90.0 1484 97.1 1450 96.3 Blood in stool 808 73.2 1143 74.8 1113 73.9 Skin turgor 510 46.2 662 43.3 832 55.3 Cough/difficulty breathinga 1818 73.6 2064 76.7 2252 81.4 +7.8* If cough present Duration 556 79.7 822 91.0 847 90.9 Stridor/wheezing 202 28.9 284 31.5 266 28.5 Respiratory rate 226 32.4 280 31.0 415 44.5 Lifted shirt 445 63.7 635 70.3 643 69.0 Listened with stethoscope 447 64.0 599 66.3 654 70.3 Fever in previous 24 hoursa 1174 47.5 1815 67.6 2143 77.4 +29.9* Checked Child’s palm for anaemiaa 141 5.7 289 10.7 467 16.9 +11.2* Child’s feet/ankles for oedemaa 178 7.3 296 11.0 533 19.3 +11.9* Child’s immunization carda 299 12.2 556 20.7 790 28.6 +16.4* Assessment index 2359 34.1 2663 42.6 2746 48.9 +14.8* Counselled caretaker on Disease, causes and courseb 738 29.9 966 36.0 1320 47.9 +17.9* Home careb 1487 60.7 1925 71.7 2101 75.9 +15.2* How to administer medicationsb 1836 75.2 2074 77.2 2242 81.1 +5.9* Adverse reactionsb 343 14.1 391 14.6 217 7.8 –6.3* Signs for immediate return to health facilityb 472 19.3 815 30.3 1188 42.9 +23.6* Counselling index 2396 39.7 2666 45.9 2752 51.1 +11.4* * P < 0.001. a Assessment index. b Counselling index. of only about 50 points out of a pos- sible 100. These findings strengthen the case for continued investments to support provider adherence to stan- dards of care. Failure to check immu- nization status and provide appropriate counselling about the administration of medications and the conditions that signify a need to return to the health facility will have a detrimental effect on overall patient outcome and conti- nuity of care. Given that Afghanistan has exceptionally high rates of infant and child mortality, providers must be motivated to provide better counselling and health education to those caring for young children. One question is “What type of health worker provides better care?” In a study from Morocco, adherence to IMCI tasks was significantly higher for nurses (compared to doctors), female providers, providers who did not iden- tify supervision as a constraint, younger children, children accompanied by their mothers, and those presenting with multiple complaints.20 In other studies, nurses and female providers have also been found to deliver better care than doctors and male providers, respectively.21,22 In our study, female health-care providers gave better care to younger children and showed better performance than their male counter- parts in 2006, but doctors performed better than other providers. Supervision and training are clearly important factors; for example, provid- ing six or more supervisory visits and IMCI training of assistant doctors and nurses were associated with better qual- ity of care in 2004.23 Similarly, in a study from Uganda, the quality index was 44% higher in facilities with at least one provider trained in IMCI,24 a finding that further strengthens the case for greater investments in IMCI. We found that the proportion of providers receiving refresher training increased by 16% between 2004 and 2006, but the proportion of IMCI-trained providers 944 Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in Afghanistan Anbrasi Edward et al. Table 3. Results of bivariate analysis of quality of care for children aged less than 5 years based on provider and patient characteristics, by year, Afghanistan, 2004–2006 2004 2005 2006 Changed (%) No.a Meanb P-valuec No.a Meanb P-valuec No.a Meanb P-valuec Assessment quality Health-care provider Doctor 1591 34.4 1886 43.7 2173 50.9 +16.5** Assistant doctor 485 34.2 0.82 316 41.4 0.06 134 45.1 < 0.001 +10.9** Nurse 283 32.4 0.13 461 39.3 < 0.001 439 40.4 < 0.001 +8.0* Provider sex Male 2198 34.0 2502 42.4 2588 48.6 +14.5** Female 107 34.3 0.89 158 45.7 0.05 156 54.4 < 0.001 +20.1** Child sex Male 1196 33.6 1440 42.2 1429 48.2 +14.6** Female 1143 34.4 0.36 1215 43.2 0.21 1299 49.7 0.08 +15.3** Child age in months < 24 1427 36.4 1661 44.3 1623 51.8 +15.3** ³ 24 932 33.5 < 0.001 1002 39.9 < 0.001 1123 44.7 < 0.001 +11.2** Caretaker sex Female 1642 34.6 1837 44.2 1982 51.6 +16.9** Male 478 32.6 0.05 747 38.5 < 0.001 764 42.0 < 0.001 +9.4** Consultation time in minutes < 10 1861 32.0 2355 41.6 2375 46.4 +14.5** ³ 10 438 42.7 < 0.001 307 50.6 < 0.001 368 64.6 < 0.001 +22.0** Facility type Basic health centre 1200 33.7 1496 41.5 1607 48.2 +14.5** Comprehensive health centre 959 34.2 0.15 973 44.7 < 0.001 931 49.3 0.21 +15.1** District hospital 200 36.1 0.12 194 41.6 0.89 208 52.9 < 0.001 +16.8** Counselling quality Health-care provider Doctor 1612 38.8 1891 45.4 2117 52.4 +13.6** Assistant doctor 498 43.1 < 0.001 313 53.0 < 0.001 133 50.1 0.30 +6.9* Nurse 286 39.1 0.87 462 42.9 0.07 442 45.0 < 0.001 +5.9* Provider sex Male 2232 39.6 2503 45.9 2597 50.6 +11.0** Female 109 43.3 0.16 159 44.5 0.51 153 60.9 < 0.001 +17.6** Child sex Male 1210 38.7 1440 46.4 1431 50.7 +12.0** Female 1164 40.6 0.09 1218 45.4 0.34 1302 51.7 0.32 +11.1** Child age in months < 24 1453 41.6 1661 46.2 1630 51.6 +10.0** ³ 24 943 41.0 0.59 1005 45.4 0.47 1122 50.4 0.24 +9.5** Caretaker sex Female 1674 40.5 1839 46.9 1984 53.1 +12.6** Male 480 38.1 0.08 746 42.9 < 0.001 767 45.9 < 0.001 +7.8** Consultation time in minutes < 10 1893 37.1 2359 44.7 2376 49.3 +12.2** ³ 10 446 49.8 < 0.001 306 54.8 < 0.001 371 63.2 < 0.001 +13.5** Facility type Basic health centre 1212 41.7 1501 44.9 1608 51.0 +9.3** Comprehensive health centre 974 37.6 < 0.001 973 48.5 < 0.001 937 51.1 0.91 +13.5** District hospital 210 38.0 0.06 192 40.5 0.03 207 52.2 0.54 +14.2** *P < 0.01; **P < 0.001. a Total in each category. b Mean performance score. c P-value for difference between groups with the reference category indicated in italics. d Change in the mean performance index for each category variable between 2004 and 2006. 945Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in AfghanistanAnbrasi Edward et al. Table 4. Results of bivariate analysis of quality of care for children aged less than 5 years based on facility characteristics, by year, Afghanistan, 2004–2006 2004 2005 2006 No.a Meanb P-valuec No.a Meanb P-valuec No.a Meanb P-valuec Assessment quality Provider knowledged,e Low 92 29.1 6 28.7 7 41.6 Medium 291 34.7 0.01 303 41.0 0.07 319 44.8 0.64 High 87 37.4 < 0.001 273 44.7 0.02 256 53.5 0.09 Provider satisfactiond,f Low 64 30.2 40 41.1 2 7 42.2 Medium 303 33.9 0.09 427 41.9 0.76 377 46.4 0.26 High 71 33.7 0.22 113 46.0 0.10 173 54.4 < 0.001 Refresher training No staff 175 33.3 82 37.4 119 46.6 Some staff 108 34.7 0.49 140 42.0 0.05 145 48.3 0.46 All staff 195 34.0 0.67 366 44.1 < 0.001 323 49.4 0.16 Training in IMCI No staff 239 32.8 204 40.7 251 45.1 Some staff 99 35.9 0.12 173 43.8 0.06 168 51.1 < 0.001 All staff 140 34.4 0.35 211 43.5 0.08 168 51.3 < 0.001 Supervision (prior 6 months) < 6 visits 214 34.0 134 41.4 126 44.7 ³ 6 visits 302 34.4 0.81 443 42.9 0.36 468 49.5 0.01 Clinical guidelines None 63 33.1 33 37.8 15 39.0 Some 352 33.6 0.84 541 43.0 0.08 564 48.7 0.05 Shura-e-sehie g None or not active 303 34.6 259 40.0 165 45.4 Active 209 33.5 0.50 334 44.6 0.00 421 49.6 0.02 Contracting Other 153 35.1 125 41.7 111 42.3 Contracted in 58 42.0 0.01 61 49.1 0.00 59 67.4 < 0.001 Contracted out 323 31.9 0.05 409 41.9 0.89 424 47.5 0.01 Counselling quality Provider knowledged,e Low 92 32.9 6 44.7 7 46.3 Medium 290 40.5 0.01 303 43.1 0.87 319 45.3 0.90 High 87 43.6 < 0.001 273 49.8 0.56 256 57.6 0.16 Provider satisfactiond,f Low 64 34.3 40 54.2 27 39.2 Medium 302 39.5 0.09 427 45.1 0.01 377 48.3 0.03 High 71 42.1 0.04 113 48.0 0.13 173 57.4 < 0.001 Refresher training No staff 175 40.0 82 43.3 119 47.6 Some staff 108 38.0 0.47 140 43.8 0.89 145 49.3 0.53 All staff 194 39.4 0.80 366 48.0 0.08 323 52.4 0.04 Training in IMCI No staff 239 38.8 204 45.2 251 46.9 Some staff 99 39.1 0.91 173 45.4 0.84 168 52.5 0.01 All staff 139 40.4 0.52 211 48.3 0.15 168 54.4 < 0.001 Supervision (prior 6 months) < 6 visits 213 38.9 134 47.8 126 44.8 ³ 6 visits 302 41.0 0.29 443 45.9 0.38 468 52.0 < 0.001 Clinical guidelines None 63 39.6 33 47.4 15 36.8 Some 351 39.0 0.86 541 46.6 0.84 564 50.8 0.01 946 Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in Afghanistan Anbrasi Edward et al. 2004 2005 2006 No.a Meanb P-valuec No.a Meanb P-valuec No.a Meanb P-valuec Shura-e-sehie g None or not active 303 40.8 259 45.0 165 46.3 Active 208 38.6 0.30 334 47.3 0.22 421 52.3 < 0.001 Contracting Other 153 38.5 125 50.6 111 47.3 Contracting in 57 62.3 < 0.001 61 54.2 0.30 59 65.9 < 0.001 Contracting out 323 36.3 0.30 409 43.8 0.01 424 49.2 0.42 IMCI, Integrated Management of Childhood Illness. a Total in each category. b Mean performance score. c P-value for difference between groups with the reference category indicated in italics. d Low = (below mean score for 2004) – (1 standard deviation); medium = (between mean score for 2004) – (1 standard deviation) and (mean score for 2004) + (1 standard deviation); high = (above mean score for 2004) + (1 standard deviation). Note: for subsequent years, the cut-offs created for 2004 were used to classify as low, medium and high. e Provider knowledge: included knowledge of IMCI, Expanded Programme on Immunization and reproductive health. f Provider satisfaction: comprised 19 indicators related to the working relationship with the staff and health system, satisfaction with the job, supervisor support, opportunities for training and professional development, availability of medications and equipment, security, salary, etc. g Village health committees. (Table 4, cont.) increased by only 6% over that period. Our results also support findings from other studies in which provider perfor- mance and adherence to standards im- proved with more frequent and better supervision and with the presence of case management guidelines.25 This study used quality-of-care indicators based on clinical standards and structural indicators, but did not use indicators that focus on patient or community perceptions of quality. Effective health service delivery in Af- ghanistan faces formidable challenges due to scarce resources, cultural bar- riers and political and security risks. The challenge for improving and sustaining the gains achieved by the Afghan Ministry of Public Health and international donor community will require innovative mechanisms and the support of local communities. In view of the low numbers and potential turnover of providers and the costs of IMCI training, policy-makers need to consider other creative mechanisms to ensure equitable coverage of basic health services through the training and deployment of community health-care providers. In high-mortality settings, IMCI training has to be accompanied by efforts to strengthen district health systems and community IMCI.14 Ac- cording to recent estimates, more than 19 000 community health workers have been trained and deployed in health posts in Afghanistan, and a recent evaluation endorsed their value and demand by the community.26 Com- munity providers, particularly women, could compliment efforts at the facility level to achieve equitable coverage of health services. Other strategies for improving health-care quality can be participatory assessments by users of the health system, team-based problem- solving, and creative engagement of health providers through national qual- ity improvement initiatives.25,27–29 As in all assessments that observe provider performance, the presence of observers may have led the providers in our study to alter their usual manner of working. This is illustrated by a study in Benin.30 Other limitations of this study are the lack of re-examination of the sick child by an independent health-care provider to establish a comparison of diagnosis and treatment, lack of risk adjustment for illness sever- ity and the inability to link observed providers to those interviewed. Pro- vider client load has also been associ- ated with quality of care.22 However, we found that the observed quality continued to improve even though pa- tient consultations more than doubled between 2004 and 2006. The finding that the type of contracting mechanism can affect the quality of the care given by providers raises opportunities for benchmarking best practices in health service delivery.6 Further empirical evidence is needed to assess how improvements in clinical care affect health outcomes or other aspects of the health system. The inclusion of data on community-based efforts for prevention, improvement of health-care seeking behaviours and reduced disease burden would provide more realistic measures of health sys- tem performance. The Afghan Ministry of Public Health has demonstrated its commitment to improving quality by instituting a National Quality Assur- ance Committee and by periodically reviewing evidence through the high- level Consultative Group on Health and Nutrition. Improving the quality and coverage of the BPHS must be a high priority if Afghanistan is to effec- tively address health needs and achieve those Millennium Development Goals that call for reduced child mortality and improved maternal health.31 Conclusion Afghanistan needs to increase support for effective implementation of IMCI at the facility and community level if it is to sustain and build on gains in health-care quality. The concerted plan to improve basic health services across Afghanistan is clearly improv- ing the quality of essential services for child health. However, to reduce child mortality, equitable coverage and access to services are needed. This will require continued investments in health infra- structure; a focus on improving health- worker performance through training, material support and supervision; greater accountability; and increased community capabilities, particularly in 947Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in AfghanistanAnbrasi Edward et al. Table 5. Multiple linear regression coefficients for assessment and counselling quality in a study of the quality of health care provided to children aged less than 5 years, by year, Afghanistan, 2004–2006 Characteristic Assessment qualitya Counselling qualityb Coefficient Standard error P-value Coefficient Standard error P-value Facility-level Year 6.62 1.14 < 0.001 4.71 1.57 < 0.001 2004 12.19 1.17 < 0.001 10.05 1.48 < 0.001 2005 2006 Facility type 2.55 1.79 0.15 0.56 2.49 0.82 Basic health centre or comprehensive health centre District hospital Provider knowledge 3.38 0.91 < 0.001 6.07 1.23 < 0.001 Poor/medium High Refresher training –2.65 1.31 0.04 –1.99 1.84 0.28 No staff Some or all staff Training in IMCI 2.56 1.06 0.02 2.93 1.44 0.04 No staff Some or all staff Supervision (visits in prior 6 months) 0.85 1.12 0.45 1.50 1.49 0.31 < 6 ³ 6 Clinical guidelines 1.49 1.65 0.37 –2.85 2.90 0.33 None Some Shura-e-sehie c 0.73 1.00 0.47 0.49 1.31 0.71 None or not active Active Contracting Contracting in 11.78 2.03 < 0.001 11.38 2.43 < 0.001 Contracting out 0.10 1.16 0.93 –2.75 1.52 0.07 Patient- and provider-level Provider cadre 5.30 1.01 < 0.001 0.42 1.35 0.76 Assistant doctor/nurse Doctor Provider sex 1.92 1.73 0.27 3.21 2.35 0.17 Male Female Child sex 0.76 0.50 0.13 0.02 0.61 0.97 Male Female Caretaker sex 4.62 0.71 < 0.001 4.00 0.97 < 0.001 Male Female Child age in months 4.68 0.58 < 0.001 0.55 0.74 0.46 ³ 24 < 24 Time of consultation in minutes 10.64 1.00 < 0.001 9.33 1.26 < 0.001 < 10 ³ 10 IMCI, Integrated Management of Childhood Illness. a R ² = 17.65. The model explains 17.65% of the variation in quality of care measured by the mean score. b R ² = 10.22. The model explains 10.22% of the variation in quality of care measured by the mean score. c Village health committees. 948 Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in Afghanistan Anbrasi Edward et al. under-served rural areas. Notwithstand- ing the difficulties faced by children and health providers in Afghanistan, health care for children is moving in the right direction and deserves greater support. ■ Acknowledgements The article represents the investments of Afghan and international staff who sup- ported the design, implementation and analysis of the National Health Service Performance Assessment. We are also grateful to the supervisors, health-care providers, mothers and children who participated in the assessments. The authors would like to thank Dr Jennifer Bryce and Dr Cesar Victora for their insights on the results. We appreciate the comments and recommendations provided by the editorial committee and the anonymous reviewers. Funding: Funding was provided through a Third Party Evaluation Con- tract between Afghanistan’s Ministry of Public Health and the Johns Hopkins Bloomberg School of Public Health, in collaboration with the Indian Institute of Health Management Research. Competing interests: None declared. Résumé Tendances de la qualité des soins de santé apportés aux enfants de moins de 5 ans en Afghanistan sur la période 2004-2006 Objectif Étudier les tendances de la qualité des soins de santé délivrés aux enfants de moins de 5 ans en Afghanistan entre 2004 et 2006. Déterminer en particulier les effets sur cette qualité d’un ensemble de services de santé essentiels (BPHS), incluant la Prise en charge intégrée des maladies de l’enfant (PCIME), introduite en 2003. Méthodes Pour chaque année d’étude, 500 à 600 établissements de soins délivrant le BPHS ont été sélectionnés par sondage aléatoire stratifié dans 29 provinces d’Afghanistan. Nous avons observé les consultations dont ont bénéficié les enfants de moins de 5 ans, interrogé les personnes s’occupant de ces enfants et les prestateurs de soins et mesuré sur un échantillon aléatoire l’observance des normes de prise en charge des cas pour ce qui concerne l’évaluation et les conseils. Résultats La qualité de l’évaluation et des conseils délivrés aux enfants malades de moins de 5 ans s’est améliorée significativement entre 2004 et 2006. On a ainsi noté une augmentation de 43,4 % de l’indice d’évaluation de l’évaluation médicale et de 28,7 % de l’indice d’évaluation des conseils (p < 0,001). La qualité de l’évaluation a progressé significativement chaque année et était statistiquement associée à certaines caractéristiques du prestateur (être médecin, obtenir un score de connaissances plus élevé, être formée à la PCIME, travailler sous contrat pour le gouvernement et assurer une consultation plus longue) et de l’enfant (jeune âge et prise en charge par une femme dans la vie courante). La qualité des conseils présentait également une association statistique avec ces caractéristiques, excepté le niveau de qualification du prestateur et l’âge de l’enfant. L’existence de directives cliniques et la fréquence de la supervision étaient significativement associées à une amélioration des scores de qualité en 2006 (p < 0,05 et p < 0,01, respectivement). Conclusion La qualité des soins s’est améliorée sur la période étudiée, mais les performances sont restées sous-optimales dans certaines zones. Il faut continuer à investir dans le renforcement des capacités du système de santé afghan. Resumen Tendencias de la calidad de la atención sanitaria dispensada a los menores de 5 años en Afganistán, 2004–2006 Objetivo Estudiar las tendencias de la calidad de la atención sanitaria proporcionada a los menores de cinco años en Afganistán entre 2004 y 2006. En particular, determinar el efecto de la calidad de un paquete básico de servicios de salud (PBSS) que incluía la atención Integrada a las Enfermedades Prevalentes de la Infancia (AIEPI), introducida en 2003. Métodos Mediante muestreo aleatorio estratificado, cada año del estudio se seleccionaron 500–600 centros de salud que empleaban el PBSS en 29 provincias de Afganistán. Observamos las consultas de atención a menores de cinco años, entrevistamos a sus cuidadores y a los profesionales que los atendieron, y medimos su cumplimiento de las normas de manejo de casos para evaluación y asesoramiento en una muestra aleatoria. Resultados La calidad de la evaluación y de los consejos proporcionados para los niños enfermos de menos de 5 años de edad mejoró significativamente entre 2004 y 2006. Se observó un aumento del 43,4% del índice de evaluación y un aumento del 28,7% del índice de asesoramiento (p < 0,001). La calidad de la evaluación mejoró considerablemente cada año y demostró estar relacionada de forma significativa con determinadas características del proveedor (condición de médico, mayor puntuación de sus conocimientos, capacitación en la AIEPI, participación en un mecanismo de contratación de la Administración, y mayor duración de la consulta) y del niño (una menor edad, y el hecho de tener a una mujer como cuidadora). Se observó que la calidad del asesoramiento también estaba asociada significativamente a esos factores, exceptuando la profesión del trabajador sanitario y la edad del niño. La existencia de directrices clínicas y la frecuencia de la supervisión se asociaron significativamente a mejores puntuaciones de la calidad en 2006 (p < 0,05 y p < 0,01, respectivamente). Conclusión La calidad de la atención mejoró a lo largo del periodo de estudio, pero el desempeño seguía siendo subóptimo en algunas zonas. Es preciso seguir invirtiendo de forma continuada en la capacidad del sistema de salud de Afganistán. 949Bull World Health Organ 2009;87:940–949 | doi:10.2471/BLT.08.054858 Research Quality of health care for children in AfghanistanAnbrasi Edward et al. References 1. Afghanistan, Ministry of Health. A basic package of health services for Afghanistan. Kabul: Islamic Traditional Government of Afghanistan; 2003. 2. Reilley B, Frank T, Prochnow T, Puertas G, Van Der Meer J. Provision of health care in rural Afghanistan: needs and challenges. Am J Public Health 2004;94:1686-8. PMID:15451732 doi:10.2105/AJPH.94.10.1686 3. Bartlett LA, Mawji S, Whitehead S, Crouse C, Dalil S, Ionete D, et al. Where giving birth is a forecast of death: maternal mortality in four districts of Afghanistan, 1999-2002. Lancet 2005;365:864-70. PMID:15752530 4. United Nations Children’s Fund. Best estimates of social indicators for children in Afghanistan 1990-2005. New York, NY: UNICEF; 2006. 5. Waldman R, Strong L, Wali A. Afghanistan’s health system since 2001: condition improved, prognosis cautiously optimistic (Unit Briefing Paper Series). Kabul: Afghanistan Research and Evaluation; 2006. 6. Sabri B, Siddiqi S, Ahmed AM, Kakar FK, Perrot J. Towards sustainable delivery of health services in Afghanistan: options for the future. Bull World Health Organ 2007;85:712-8. PMID:18026628 doi:10.2471/BLT.06.036939 7. Kaplan RS, Norton DS. The balanced scorecard. Boston, MA: Harvard Business School Press; 1996. 8. Peters DH, Noor AA, Singh LP, Kakar FK, Hansen P, Burnham G. A balanced scorecard for health services in Afghanistan. Bull World Health Organ 2007;85:146-51. PMID:17308736 doi:10.2471/BLT.06.033746 9. Hansen PM, Peters DH, Niayesh H, Singh LP, Dwivedi V, Burnham G. Measuring and managing progress in the establishment of basic health services: the Afghanistan health sector balanced scorecard. Int J Health Plann Manage 2008;23: . PMID:18435428 doi:10.1002/hpm.931 10. The Johns Hopkins University and Indian Institute of Health Management. Afghanistan health sector balanced scorecard national and provincial results round 3 (2006). Kabul: Ministry of Public Health and Planning Division; 2006. 11. The world heath report 2005: making every mother and child count. Geneva: World Health Organization; 2005. Available from: http://www.who.int/ whr/2005/en/index.html [accessed on 24 October 2008]. 12. Armstrong Schellenberg J, Bryce J, de Savigny D, Lambrechts T, Mbuya C, Mgalula L. The effect of Integrated Management of Childhood Illness on observed quality of care of under-fives in rural Tanzania. Health Policy Plan 2004;19:1-10. PMID:14679280 doi:10.1093/heapol/czh001 13. Chopra M, Patel S, Cloete K, Sanders D, Peterson S. Effect of an IMCI intervention on quality of care across four districts in Cape Town, South Africa. Arch Dis Child 2005;90:397-401. doi:10.1136/adc.2004.059147 14. Adam T, Amorim DG, Edwards SJ, Amaral J, Evans DB. Capacity constraints to the adoption of new interventions: consultation time and the Integrated Management of Childhood Illness in Brazil. Health Policy Plan 2005;20 Suppl 1;i49-57. PMID:16306069 doi:10.1093/heapol/czi057 15. Amaral J, Leite AJ, Cunha AJ, Victora CG. Impact of IMCI health worker training on routinely collected child health indicators in Northeast Brazil. Health Policy Plan 2005;20 Suppl 1;i42-8. PMID:16306068 doi:10.1093/heapol/czi058 16. Arifeen SE, Bryce J, Gouws E, Baqui AH, Black RE, Hoque DM. Quality of care for under-fives in first-level health facilities in one district of Bangladesh. Bull World Health Organ 2005;83:260-7. PMID:15868016 17. Gouws E, Bryce J, Habicht JP, Amaral J, Pariyo G, Schellenberg JA, et al. Improving antimicrobial use among health workers in first-level facilities: results from the multi-country evaluation of the Integrated Management of Childhood Illness strategy. Bull World Health Organ 2004;82:509-15. 18. Schellenberg JR, Adam T, Mshinda H, Masanja H, Kabadi G, Mukasa O. Effectiveness and cost of facility-based Integrated Management of Childhood Illness (IMCI) in Tanzania. Lancet 2004;364:1583-94. PMID:15519628 19. Chatterjee S, Hadi AS, Price B. Regression analysis by example. 3rd ed. New York, NY: John Wiley and Sons; 2000. 20. Naimoli JF, Rowe AK, Lyaghfouri A, Larbi R, Lamrani LA. Effect of the Integrated Management of Childhood Illness strategy on health care quality in Morocco. Int J Qual Health Care 2006;18:134-44. PMID:16423842 21. Amaral J, Gouws E, Bryce J, Leite AJ, Cunha AL, Victora CG. Effect of Integrated Management of Childhood Illness (IMCI) on health worker performance in Northeast-Brazil. Cad Saude Publica 2004;20 Suppl 2; S209-19. 22. Adam T, Amorim DG, Edwards SJ, Amaral J, Evans DB. Capacity constraints to the adoption of new interventions: consultation time and the Integrated Management of Childhood Illness in Brazil. Health Policy Plan 2005;20(Suppl 1);i49-57. 23. Hansen PM, Peters DP, Edward A, Gupta S, Arur A, Niayesh H, et al. Determinants of primary care service quality in Afghanistan. Int J Qual Health Care 2008;20:375-83. PMID:18799469 doi:10.1093/intqhc/mzn039 24. Bishai D, Mirchandani G, Pariyo G, Burnham G, Black R. The cost of quality improvements due to integrated management of childhood illness (IMCI) in Uganda. Health Econ 2008;17:5-19. PMID:17387710 doi:10.1002/hec.1231 25. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health technology assessment (Winchester, England) 2004;8:1-72. 26. The Johns Hopkins University and Indian Institute of Health Management. Operations research on community health worker performance, 2006. Kabul: Ministry of Public Health and Planning Division; 2006. 27. Bradley J, Igras S. Improving the quality of child health services: participatory action by providers. Int J Qual Health Care 2005;17:391-9. PMID:15951311 28. Marquez L, Madubuike C. Country experience in organizing for quality: Zambia. QA Brief 1999;8:16-9. PMID:12322022 29. Ovretveit J. Formulating a health quality improvement strategy for a developing country. Int J Health Care Qual Assur Inc Leadersh Health Serv 2004;17:368-76. 30. Rowe AK, Lama M, Onikpo F, Deming MS. Health workers’ perception of how an observer influences their practices during consultations with ill children, Benin. Trop Doct 2002;32:166-7. PMID:12139161 31. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA. Overcoming health- systems constraints to achieve the Millennium Development Goals. Lancet 2004;364:900-6. PMID:15351199 doi:10.1016/S0140-6736(04)16987-0 صخلم 2006-2004 ،ناتسناغفأ في ةسماخلا رمع نود لافطلأل ةيحصلا ةياعرلا ةدوج تاهاجتا رمع نود لافطلأل ةمدقلما ةيحصلا ةياعرلا ةدوج تاهاجتا ةسارد :ضرغلا يرثأت لىع فرعتلا ماَّيسلاو ،2006-2004 ماوعلأا في ناتسناغفأ في ةسماخلا لماكتلما يربدتلا لمشت ةيحصلا تامدخلا نم ةيساسأ ةمزح لىع ةدوجلا هذه .2003 ماع تلخدأ يتلاو لافطلأا ضارملأ ةنس لك في ،قّبطلما ئياوشعلا نايتعلاا مادختساب ،نوثحابلا راتخا :ةقيرطلا 29 في ةيحصلا ةمدخلا مّدقي يحص قفرم 600 – 500 ةساردلا تاونس نم ،ةسماخلا رمع نود لافطلأل تاراشتسلاا نوثحابلا ظحلاو .ناتسناغفأ في ةيلاو ةيحصلا ةياعرلا نومدقي نمو ،لافطلأا ءلاؤهب نونتعي نم عم تلاباقم اورجأو في ةروشلما ءادسلإو ميـيقتلل تلااحلا يربدت يرـياعبم مهمازتلا ىدم اوساقو ،مهل .ةيئاوشعلا تانّيعلا لافطلأل ةمّدقلما ةروشلما ءادسإو ميـيقتلا ةدوج تنسحت دقل :تادوجولما داز دقف ،2006 - 2004 ةترفلا في ًاظوحلم ًانسحت ةسماخلا رمع نود ضىرلما %28.7 رادقبم ةروشلما ءادسإ بسنم داز ماك ،%43.4رادقبم ميـيقتلا بسنم لك ًاظوحلم ًانسحت ميـيقتلا ةدوج تنّسحتو .)0.001 نم لقأ P لماتحلاا ةوق( نم اوناك ًءاوس( ةياعرلا يمدقم ىدل صئاصخلا ضعب عم ًايئاصحإ تقفارتو ماع يجلاعلا يربدتلا لىع ينبردتلما نم مأ ،مهيدل فراعلما زارحأ تداز نيذلا ءابطلأا لوطأ ةترف نوضقيو دقاعتلا ةيلآ نم ًاءزج اوناك مأ ،لافطلأا ضارملأ لماكتلما دقف( لافطلأا ىدل صئاصخلا ضعب كانه ناك ماك )ةروشلما ءادسإ في مهيرغ نم ءادسإ ةدوج تقفارت ماك .)مهتياعرب نمقي ثانإ كانه ناكو ،ًارمع رغصأ اوناك ةياعرلا ميدقت مقاط ءانثتساب كلذو ،صئاصخلا هذه عم ظوحلم لكشب ةروشلما لكشب اقفارت دق فاشرلإا راركتو ةيريسر ةيداشرإ لئلاد دوجو نإ .لفطلا رمعو 0.05 نم لقأ لماتحلاا ةوق( 2006 ماع في ةدوجلا زارحأ نسحت عم ظوحلم .)فاشرلإا راركتل 0.01 نم لقأو ةيداشرلإا لئلادلا دوجول لازي لا ءادلأا نكلو ،ةساردلا ةترف مدقت عم ةياعرلا ةدوج تنّسحت :جاتنتسلاا ماظنلا تاردق في رماثتسلاا لصاوت دعُيو ،تلااجلما ضعب في لثملأا هاوتسم نود .ًايروضر نياغفلأا يحصلا