Cost and effectiveness analysis of immunization service delivery support in Andhra Pradesh, India

221Bulletin of the World Health Organization | March 2008, 86 (3) Introduction In November 2003, an immunization ser- vice delivery support (ISDS) model was launched in Andhra Pradesh, India, with the aim of strengthening immunization services through supportive supervision. Traditionally, supervision of immuniza- tion practices in Andhra Pradesh is non- existent or conducted minimally in an un- structured manner. Recognizing attitude problems as well as lack of coordination and transportation as common barriers to effective supervision, the Andhra Pradesh state government decided to introduce a three-year collaborative initiative with Program for Appropriate Technology in Health (PATH) in all 23 districts covering a population of 76 million.1 Supportive supervision is one of five essential components of the WHO’s Reaching Every District strategy, an operational approach to reach 90% DPT3 (three doses of vaccine against Cost and effectiveness analysis of immunization service delivery support in Andhra Pradesh, India Chutima Suraratdecha,a CBS Venkata Ramana,b Satish Kaipilyawar,c JVG Krishnamurthy,c Srilatha Sivalenka,c Naveena Ambatipudi,c Sanjay Gandhi,c K Umashankar c & James Cheyne d Abstract The immunization service delivery support (ISDS) model was initiated in Andhra Pradesh, India, in November 2003 with the aim of strengthening immunization services through supportive supervision. The ISDS model involves a well-established supervision system built upon the existing health infrastructure. The objectives of this approach are to: (1) identify areas of high performance and those that need improvement, (2) assist staff in identifying and correcting wrong practices, (3) improve staff skills, (4) motivate staff, and (5) initiate corrective actions at appropriate levels through information sharing. An evaluation of cost and effectiveness of ISDS in 16 districts that participated in the programme found that the incremental cost associated with three rounds of supportive supervision visits was approximately US$ 110 630 (US$ 36 877 per round). The performance of health centre and immunization sessions was evaluated using 43- and 28-point checklists, respectively, and demonstrated significant improvement during and following the two- year implementation of ISDS. The average percentage change in health centre performance scores from baseline to the fourth round of evaluation was approximately 36%, and immunization session performance scores increased by an average of 9%. The incremental costs per additional per cent increase in average health centre performance score and per additional per cent increase in average immunization session performance score over the evaluation period were estimated to be US$ 3091 and US$ 12 760, respectively. The incremental cost-effectiveness ratios are relatively sensitive to personnel and travel costs. Integration of ISDS into the Andhra Pradesh immunization system is projected to result in a 39% potential cost savings per round of supervision visit. Bulletin of the World Health Organization 2008;86:221–228. Une traduction en français de ce résumé figure à la fin de l’article. Al final del artículo se facilita una traducción al español. .ةلاقلما هذهل لماكلا صنلا ةياهن في ةصلاخلا هذهل ةيبرعلا ةمجترلا a PATH, 1455 NW Leary Way, Seattle, WA 98107, United States of America. b Family Welfare, Government of Andhra Pradesh, Hyderabad, India. c PATH, Hyderabad, India. d PATH, Ferney Voltaire, France. Correspondence to Chutima Suraratdecha (e-mail: csuraratdecha@path.org). doi:10.2471/BLT.06.039495 (Submitted: 6 December 2006 – Revised version received: 19 June 2007 – Accepted: 21 June 2007 – Published online: 22 January 2008 ) Diptheria, Pertussis and Tetanus) cover- age nationally and at least 80% DPT3 coverage in every district by the year 2005, the “90/80 goal” adopted by the GAVI Alliance.2 Supportive supervision involves on-site training by supervisors, which promotes quality at all levels of the health system by strengthening relationships within the system, focuses on the identification and resolution of problems, and helps to optimize the al- location of resources. The approach also promotes continuous improvements to the quality of care by providing leader- ship and support and by promoting high standards, teamwork and better two-way communication.3 Different approaches to supportive supervision to strengthen health worker performance and health services have been developed and documented. For instance, an integrated health package to guide essential health service delivery was developed by the United Republic of Tanzania Ministry of Health fol- lowing health sector reform in 1999. This included a plan to bring a team of supervisors to district health facili- ties to evaluate how services are being delivered, to provide feedback and to conduct on-site training.4 Since implementation, supervisory visits occur more frequently and have become an opportunity for health workers to solve problems and learn additional skills. In 1999, EngenderHealth adapted a Client-Oriented, Provider-Efficient Services (COPE) model, a participatory supervision approach that includes self- assessment and performance plans, for use in child health services in selected study sites in Guinea and Kenya.5 Results show that supervision was better than before, supervisors were practic- ing new skills and following up on identified problems, and clients were Policy and practice Analysis of immunization service delivery support 222 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) very satisfied with the overall service received. The Honduran nongovern- mental organization ASHONPLAFA introduced a new supervisory system and strengthened supervisory skills while reducing operating costs to meet sustainability goals. Strengthening su- pervision resulted in improved service quality and increased client satisfac- tion without increasing cost.3,4 A study conducted in rural Mexico concluded that supportive supervision and self- assessment of physicians significantly improved doctor performance and doctor-patient communication.6 The Andhra Pradesh model The Andhra Pradesh ISDS model is a well-established immunization supervi- sion system and builds upon the exist- ing health infrastructure. The objectives of implementing this model are to: (1) identify areas of high performance and those that need improvement, (2) assist staff in identifying and correcting wrong practices, (3) improve staff skills, (4) motivate staff, and (5) initiate corrective actions at appropriate levels through information sharing. With the total combined number of urban health centres and primary health centres ranging from 70 to 80 per district with only one district im- munization officer (DIO) in each district, a supervision visit can accom- modate only five to ten health centres (HCs). To establish an effective sup- portive supervision system, the ISDS model initially used both external and government supervisors to monitor immunization practices and to ensure quality immunization service delivery. The model is a combined interven- tion of the external consultant mode and medical college mode (Fig. 1). Twenty-one districts were included in the programme during the first two years of implementation. The external consultant mode was coordinated by designated persons at PATH and used a team of external consultants and re- gional coordinating officers (RCOs), retired state or district level health of- ficers, to implement the model in 14 districts. In each supervision round, a team consisting of six to seven mem- bers, including PATH staff, external consultants and RCOs, performed four- to five-day site visits to all urban health centres and primary health centres in each district and to regular fixed and outreach immunization sessions. In the other seven districts, supportive super- vision was provided by faculty and post- graduate students of medical colleges in coordination with RCOs and DIOs. In both interventions, the supervision team was accompanied by the DIO and other district programme managers or DIO-appointed supervisors to ensure capacity strengthening and sustainabil- ity. PATH provided on-the-job train- ing, appropriate tools, and financial and technical support to the teams. Over the course of the programme, management responsibilities were gradually handed over to trained RCOs. As assistance from external consultants and PATH staff phased out, seven RCOs were as- signed responsibilities for conducting supportive supervision in 14 districts and coordinating with the medical colleges in nine districts and all DIOs (Fig. 1). Objectives The overall objective of this paper is to assess the cost and effectiveness of the Andhra Pradesh ISDS model over a two- year period. Information on cost and effectiveness of the intervention is criti- cal for decision-making and planning for sustainable integration of support- ive supervision into the immunization system and replication in other states of India. Specific objectives include assess- ing cost and effectiveness of the model Fig. 1. The supervision support framework Consultant mode: RCO, PATH staff, external consultants and DIO or appointed supervisors or Medical college mode: medical college faculty/postgraduate student, RCO and DIO or appointed supervisors Visit primary and urban health centres 1. On-site correction and training 2. Feedback to medical officer 3. Record data in a computer daily Visit immunization sessions 1. Feedback to DIO 2. Submit report to state and district offices DIO, district immunization officer; RCO, regional coordinating officer. during the study period and estimating financial requirements for integration and expansion. Methods Two rounds of supervision visits per district were conducted within a 12- month period. The period between rounds provided an opportunity for divisional/district level monthly meet- ings to identify areas that need to be improved. During each round of su- pervision, information on activity costs and key indicators of immunization performance were collected. This paper bases its analysis on outcomes at base- line and after three rounds of supervi- sion and costs incurred during three supportive supervision visits over an 18- month period. Baseline outcome data at all HCs in all districts were collected at the same time as the first introduction of supportive supervision activities. Measurement of cost All costs related to ISDS activities were based on the provider’s perspec- tive, interviews with programme staff, and labour and financial reports from November 2003 (the start of the pro- gramme) to May 2005 (the end of the third round of supervision visits). All financial support from PATH, the as- sociated costs at PATH’s Hyderabad office and of the Andhra Pradesh state government and field costs during the Policy and practice Analysis of immunization service delivery support 223 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) study period to implement the model were included. Daily activity time logs and activity reports kept during pro- gramme implementation allowed for tracking of intervention costs based on the actual staff time spent on supportive supervision activities. Total costs were presented in US dollars at 2000 prices by using a gross domestic product deflator and ex- change rate (45.55 rupees = US$ 1) and included wages and benefits of the project personnel involved with sup- portive supervision activities, printing of checklists, meals and incidental expense allowances, accommodation, transportation and administration. The estimates represent incremental costs of delivering the programme which are additional costs to the government that would not have been incurred other- wise. Salaries and benefits of health staff at district and HC levels and costs of recruiting external consultants were excluded. Fixed costs (such as house- keeping, capital costs and utilities) were not included because they were jointly incurred by other projects, and supervision activities were conducted at HCs. Costs associated with perfor- mance evaluations were minimal and not incremental to the programme, so therefore were not included in interven- tion costs. Measurement of effectiveness Performance data were collected us- ing checklists. User-friendly checklists, which include 43 indicators on pro- gramme management, immunization sessions, immunization supplies, cold chain maintenance, immunization bio- waste disposal and adverse events fol- lowing immunization, were used in evaluating HC performance. Fixed and Table 1. Costs for ISDS programme per supervision round, 16 districts, November 2003–May 2005 Item Round 1 (US$)a,b Round 2 (US$)a,b Round 3 (US$)a,b Total (US$)a,c Average per round (US$)a Wages, benefits and per diem 7 962.07 (36.00%) 30 134.63 (60.39%) 18 095.53 (46.87%) 56 192.23 (50.79%) 18 730.34 Transportation, vehicle and accommodation 12 768.43 (57.73%) 18 823.61 (37.72%) 18 957.92 (49.10%) 50 549.95 (45.69%) 16 849.98 Expenses associated with training and retraining of programme staff 1 319.30 (5.96%) 879.60 (1.76%) 1 203.70 (3.12%) 3 402.60 (3.07%) 1 134.20 Printing 24.9 (0.11%) 23.7 (0.05%) 24.6 (0.06%) 73.20 (0.07%) 24.40 Administration 44.66 (0.20%) 36.89 (0.07%) 330.24 (0.86%) 411.80 (0.37%) 137.27 Total 22 119.35 49 898.43 38 611.99 110 629.77 36 876.59 ISDS, immunization service delivery support. a US$ values are year 2000 prices. b Values presented in parentheses are percentages of the total for each round. c Values presented in parentheses are percentages of the total. Table 2. Average HC performance scores and percentage change between rounds of performance evaluation District No. of HCs Average HC performance scores Percentage change between rounds Baseline (a) Round 2 (b) Round 3 (c) Round 4 (d) (a) to (b) (b) to (c) (c) to (d) (a) to (d) Anantapur 65 32.18 55.06 59.95 65.97 71.08* 8.88* 10.03* 104.97* Chittoor 91 48.27 56.94 59.93 53.01 17.96* 5.25** –11.55* 9.81* Cuddapa 73 49.67 59.01 62.08 64.20 18.81* 5.20** 3.42*** 29.26* Guntur 77 48.61 60.01 58.90 64.51 23.46* –5.19** 13.37* 32.70* Karimnagar 63 41.93 50.16 60.44 62.58 19.60* 20.51* 3.54*** 49.24* Khamman 61 44.54 47.83 57.08 64.29 7.40 19.33* 12.64* 44.35* Krishna 70 53.01 62.13 53.74 62.63 17.19* –13.50* 16.53* 18.13* Kurnool 74 44.20 57.34 54.20 54.93 29.71* –5.47*** 1.35 24.27* Medak 71 42.98 60.60 65.17 66.53 40.99* 7.53* 2.09*** 54.78* Nalgonda 70 44.04 47.18 58.77 62.91 7.13*** 24.55* 7.05** 42.85* Nellore 74 56.08 55.70 65.54 66.28 –0.67 17.66* 1.13 18.19* Prakasam 81 56.65 64.80 64.25 69.49 14.38* –0.86 8.17* 22.66* Srikakulam 63 52.22 55.92 65.22 68.09 7.08** 16.63* 4.40* 30.39* Vizianagaram 55 53.36 57.00 61.11 66.11 6.81** 7.21** 8.18* 23.88* Warrangal 45 17.58 16.02 56.47 55.64 –8.85 252.43* –1.46 216.56* West Godavari 83 48.99 55.84 59.58 68.00 13.99* 6.69* 14.13* 38.81* All 1 116 46.73 54.97 60.10 63.45 17.64* 9.33* 5.57* 35.79* HCs, health centres; * statistically significant at 0.01 level; ** statistically significant at 0.05 level; *** statistically significant at 0.1 level. Policy and practice Analysis of immunization service delivery support 224 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) outreach immunization sessions were randomly observed and evaluated using a 28-point checklist to assess practices related to vaccine logistics, immuniza- tion practices and community mobili- zation activities (checklists are available upon request from the authors). Following each session of perfor- mance evaluation, gaps were identified and discussed immediately. HC staff were observed and trained on-site, with feedback given to HC medical officers during the visit. Each day, supervisors consolidated forms and recorded all data from checklists into a computer database. At the end of each visit, feed- back and HC performance measure- ments were provided to district officials, and district-wise aggregate data were then made available to state and district offices. Each point on the checklist was scored and weighted according to its importance (weights are available upon request from the authors), with a total possible score of 75 points. During the two-year period, supportive supervi- sion activities occurred in 1724 HCs and 436 immunization sessions in 21 districts. Cost-effectiveness analysis Very few publications address the cost- effectiveness of peer training and su- pervision to enhance immunization services. One study evaluated an im- munization training programme in Maluku, Indonesia, and concluded that it significantly improved the practice of vaccination and the immunization coverage rate.7,8 The average costs were about US$ 0.05 per additional reported dose and US$ 0.50 per additional fully immunized child. Applying guiding principles including monitoring and supervision to the management of pri- mary health care services in Papua New Guinea resulted in increased immuniza- tion coverage from a range of 57% to 67% between 1980 and 1982, to 89% to 94% between 1983 and 1984.9 A systematic review of literature on cost and effect of a range of interventions for expanding the coverage of immuniza- tion services suggests that peer training is one of the least costly interventions when vaccine costs are not included.10 In this paper, cost effectiveness is defined as the ratio of incremental cost to the incremental effectiveness as mea- sured by the percentage improvement in performance scores at HCs and immu- nization sessions. The ratio is expressed as US dollars per percentage increase in performance scores. Between each round of supervision, additional HCs were identified and included in the pro- gramme, and some HCs were integrated with others. To allow for comparison of HC performance scores measured pre- and post-intervention, the analysis is limited to cost and outcomes associated with HCs and districts that had partici- pated in the programme from the start until the fourth round of performance evaluation. One-way sensitivity analy- ses were performed to assess the robust- ness of cost-effectiveness ratios. Results Cost Since supportive supervision activities were conducted in a phased manner, there were 1116 HCs in 16 districts (Adilabad, East Godavari, Mahabub- nagar, Nizamabad and Visakhapatnam were excluded from the analysis) with complete data on performance scores available during the study period. Total associated costs of supportive supervi- sion were approximately US$ 110 630. Table 1 presents the cost of supportive supervision by item. Travel and personnel expenses were cost drivers of activities and added up to 96% of the total costs. Approximately 46% (US$ 50 550) of the total costs were spent on transportation, vehicles and accommodations. Personnel costs Fig. 2. Average performance scores and percentage change in scores during four rounds of evaluation in 16 districts Round 2 Pe rc en ta ge c ha ng e (% ) 0 20 Percentage change Average scores Baseline 18 16 14 12 10 8 6 4 2 Round 3 Round 4 70 60 50 40 30 20 10 0 Av er ag e pe rf or m an ce s co re s Fig. 3. Average performance scores and percentage change in scores during four rounds of evaluation of immunization sessions Round 2 (245 sessions) Pe rc en ta ge c ha ng e (% ) 0 6 Percentage change Average scores Baseline (280 sessions) Round 3 (270 sessions) Round 4 (270 sessions) 65 56 Av er ag e pe rf or m an ce s co re s 64 63 62 61 60 59 58 57 5 4 3 2 1 Policy and practice Analysis of immunization service delivery support 225 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) (US$ 56 192) including per diem pay- ments accounted for about half of the total cost. The training and retraining sessions conducted by PATH for ex- ternal consultants, RCOs and medical college staff amounted to an average cost of US$ 1134.20 per round. The design and printing of 20 000 copies of checklists cost US$ 0.07 per set of checklists. The total cost per round of supervision was on average US$ 36 877 and ranged from US$ 22 119 to US$ 49 898. The average programme cost was US$ 6914 per district (US $99 per HC) or US$ 2305 per district per round of supervision (US$ 33 per HC per round of supervision). Additional external consultants and RCOs were recruited to the su- pervision team over the course of the programme. As a result, wages and benefits increased dramatically in the second round of supervision. In round three, four districts (Chittoor, Krishna, Kurnool and Guntur) were transferred to the medical colleges mode, resulting in lower costs. Effectiveness Of the 1116 participating HCs in 16 districts, the average district-level HC performance score (maximum 75 points) was 46.73 (mode 57; median 48.5) at baseline, 54.97 (mode 63; me- dian 58) after the first supervision visit, 60.10 (mode 70; median 62) after the second supervision visit and 63.45 (mode 70; median 62) after the third supervision visit (Table 2). The Pair- wise t-test shows that the intervention produced statistically significant im- provements in HC performance scores during the study period at the 99% confidence level. The majority of dis- tricts show significant increases in per- formance scores over time. The average percentage change in performance scores between baseline and the fourth round of performance evaluation for all 16 districts was 36% (range: 9.81–216.56) and statistically significant at 0.01 level. Fig. 2 depicts average scores of each round of evalu- ation and percentage change in scores between rounds. The evaluation of immunization sessions was conducted at random dur- ing each round and included only a small number of sessions. The number of sessions visited in each district during each round was therefore inadequate Table 3. Incremental cost-effectiveness ratios (ICERs) of base case and one-way sensitivity analyses (baseline to round 4) Item ICER (US$)a,b Per 1% change in HC mean score Wages, benefits and per diem  high (90 403.90) 4 047  base (56 192.23) 3 091  low (23 886.20) 2 188 Transportation, vehicle and accommodation  high (56 873.35) 3 268  base (50 549.95) 3 091  low (38 305.28) 2 749 Training  high (3 957.90) 3 106  base (3 402.60) 3 091  low (2 638.80) 3 070 Discount rate  costs discounted at 5% 3 040  base at 0% 3 091  costs discounted at 3% 3 060 Per 1% change in immunization session mean score Wages, benefits and per diem  high (90 403.90) 16 706  base (56 192.23) 12 760  low (23 886.20) 9 034 Transportation, vehicle and accommodation  high (56 873.35) 13 489  base (50 549.95) 12 760  low (38 305.28) 11 348 Training  high (3 957.90) 12 824  base (3 402.60) 12 760  low (2 638.80) 12 672 Discount rate  costs discounted at 5% 12 548  base at 0% 12 760  costs discounted at 3% 12 630 HC, health centre. a US$ values are year 2000 prices. b Cost-effectiveness ratios between each round are available from authors upon request. to perform a statistical comparison. At the aggregate level, there was a statisti- cally significant increase in the average performance score of immunization sessions from 58.85 points at baseline to 59.71 at the second round of evalua- tion (1.46%; not statistically significant) and to 63.96 points at the fourth round of evaluation (8.68%; P < 0.0001), from 59.71 at the second round to 62.57 at the third round (4.79%; P = 0.0005), and from 62.57 in the third round to 63.96 in the fourth round (2.22%; P = 0.08). Fig. 3 shows the overall per- centage change in performance scores and average scores. Incremental cost-effectiveness It costs US$ 110 630 to significantly increase the mean performance score of HCs and immunization sessions by 36% and 9%, respectively – from 47 and 59 points at baseline to 63 and 64 points in the fourth evaluation. This re- sults in a programme cost of US$ 3091 (an average of US$ 193 per district) to increase mean HC performance scores by 1%. Similarly, it costs US$ 12 760 (an average of US$ 797 per district) to increase mean immunization session performance scores by 1% during the same period of time. Table 3 calculates incremental cost-effectiveness ratios and Policy and practice Analysis of immunization service delivery support 226 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) reports sensitivity of cost-effectiveness ratios to changes in cost components and discount rates. For each cost item (wages and benefits, transport and ac- commodation, training) and percent- age change in performance scores, the maximum and minimum values from the cost data were selected to proxy the magnitude of potential variations. Costs for wages, benefits, per diem, transporta- tion, vehicles and accommodation had a large impact on cost-effectiveness. A reduction in compensation and travel- ling costs could significantly improve the incremental cost-effectiveness ratio, however it is relatively insensitive to costs associated with training. Financing immunization service delivery support Table 4 describes the implementation costs associated with service delivery support if the ISDS model were inte- grated into the immunization system and covered all 23 districts of Andhra Pradesh. The integrated ISDS model would comprise seven RCOs and nine medical institutions. During each super- vision round, seven RCOs would form a team of supervisors to visit all HCs in 14 districts twice a year. In the other nine districts, ISDS activities would be performed by medical colleges. The RCOs, medical colleges and DIOs would coordinate and be jointly respon- sible for implementing ISDS activities upon approval of an action plan by the state office. Continuing and integrating the ISDS programme into the state im- munization system in 23 districts would cost US$ 34 896 (US$ 31 370 in 2000 US$) per round of supervision. Travel expenses are the highest proportion of cost associated with ISDS. Overall, the integration represents a potential sav- ings of 38.71%, which results primarily from replacing wages of PATH staff and consultants and PATH-expense allow- ances for meals and incidental expenses with government wages and allowance rates for meals and incidental expenses. Discussion and conclusions This paper offers the first attempt to quantitatively evaluate the supportive supervision framework. Total costs of activities in 16 districts over the evalu- ation period were US$ 110 630 (US$ 36 877 per round). The statistical signifi- cance of results on effectiveness suggests that performance at HCs and immu- nization sessions have been greatly im- proved since the ISDS programme was launched. Travel and personnel expenses, specifically wages, meals and inciden- tal expense allowances for appointed PATH staff, appeared to be the major cost drivers. The involvement of PATH staff in training and other contributions in the early stage of the programme suggests that replication in other states or other settings is probable, but would require a plan for technical assistance. A phase-out of PATH support and suc- cessful integration of ISDS activities into the immunization system, how- ever, would result in potential savings of 38.71%, which may improve cost- effectiveness provided that performance scores are maintained or improved. The analysis between each round of supervision visits provides valuable insight into the impact of the Andhra Pradesh ISDS model. Fig. 2 and Fig. 3 illustrate that the model is significantly effective in improving performance at HCs and immunization sessions. The Table 4. Implementation costs associated with the integrated ISDS model per supervision round, 23 districts Item Integrated ISDSa Original ISDSb Cost per round (current price, US$) Cost per round (2000 price, US$) (2000 price, US$) Wages, benefits and per diem 15 113.06 (43.3%) 13 586.00 (43.3%) 26 925.44 Transportation, vehicle and accommodation 19 743.69 (56.6%) 17 748.73 (56.6%) 24 221.85 Printing 39.01c (0.1%) 35.07 (0.1%) 35.07 Total 34 895.76 31 369.80 51 182.37 ISDS, immunization service delivery support. a Assuming no additional training is required and RCOs will be responsible for administration of the programme following the integration. b Costs incurred in 16 districts, excluding training and administration costs, were extrapolated to cover 23 districts. c Printing cost was estimated from an average printing cost in 16 districts (Table 1). average performance score of HCs consistently improved, and good per- formance was maintained or improved, as measured at subsequent supervision visits. Statistically significant regressive performance found in Chittoor and Kurnool perhaps resulted from the change of supervisors during the transi- tion to medical college. Although Fig. 3 demonstrates that it is more difficult to improve immunization session scores, the improvement recorded after the second supervision visit is significant. The relatively slow improvement might be due to the use of indicators that were more closely related to the practice of immunization and the behaviour of immunization officers, and the perfor- mance score was good at the time the model was launched (average scores at baseline were at 58.85). Our findings underscore that continuation of sup- portive supervision is critical in strength- ening and enhancing performance at HCs and immunization sessions. The limitations of this study are those attributed to the limited data and scope of the programme and our analysis. The relatively small number of immunization sessions observed may not be representative. Immunization ses- sions are usually conducted two days per week with fixed timing. Each supervi- sion visit could incorporate observation of a few sessions. Although checklists were introduced in the programme to minimize subjectivity of supervisors in rating performance, this issue was not assessed. Additionally, the analysis does not consider ongoing training, which may be required to improve knowledge and strengthen supervisors’ capacity. As there are no comparators, the implication of cost-effectiveness results Policy and practice Analysis of immunization service delivery support 227 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) is limited. Our findings thus provide a basis for a comparison to be made across settings and programmes aiming to improve performance of HCs and im- munization sessions. Such comparisons will generate information on how cost- effective the intervention is and how it can be further improved. Similar ap- proaches should be further explored and applied to other health services, such as primary care and maternal and child health, where supportive supervision can play a role in improving knowledge and transferring new skills to multiple and remote sites. Appropriate models should be identified and assessed to be applicable to the local context. The analysis on cost and effectiveness can be a useful tool in planning for sustainable intervention. ■ Acknowledgements The authors would like to acknowledge the comments provided by the anony- mous reviewers. We are grateful to the staff of the government of Andhra Pradesh for their support. Special thanks go to Jodi Udd and Deborah Phillips for editing and manuscript preparation. Competing interests: None declared. Résumé Analyse du coût et de l’efficacité du soutien à la prestation de services de vaccination dans l’Etat de l’Andhra Pradesh, en Inde Le modèle de soutien à la prestation de services de vaccination (ISDS) a été mis en place dans l’Andra Pradesh (Inde) en novembre 2003, dans l’objectif de renforcer les services de vaccination par une supervision de soutien. Le modèle ISDS suppose l’existence d’un système de supervision bien établi au-dessus de l’infrastructure sanitaire existante. Cette démarche a pour objectifs : (1) d’identifier les domaines où les performances sont très satisfaisantes et ceux exigeant des améliorations, (2) d’aider le personnel à identifier et à corriger les pratiques inadéquates, (3) d’améliorer les compétences du personnel, (4) de le motiver, et (5) de lancer des actions correctives à des niveaux appropriés grâce au partage d’informations. Une évaluation du coût et de l’efficacité du modèle ISDS dans 16 districts ayant participé au programme a établi que le coût différentiel pour trois tournées de visites de supervision de soutien se situait autour d’US $ 110 630 (US $ 36 877 par tournée). Les performances du centre de santé et des sessions de vaccination ont été évaluées à l’aide d’une liste de contrôle en 43 et 28 points respectivement et des améliorations notables ont été relevées au cours et à la suite des deux années de fonctionnement de l’ISDS. En moyenne, la variation en pourcentage des scores de performances entre la période de référence et la quatrième tournée d’évaluation était d’environ 36 % pour le centre de santé et d’environ 9 % pour les sessions de vaccination. Les coûts différentiels par pourcentage supplémentaire d’augmentation du score de performances moyen du centre de santé et d’une session de vaccination ont été estimés respectivement à US $ 3091 et US $ 12 760 pour la période d’évaluation. Les ratios différentiels coût/efficacité sont relativement sensibles aux coûts de main-d’œuvre et de déplacement. D’après les projections, l’intégration de l’ISDS dans le système de vaccination de l’Andhra Pradesh pourrait entraîner une économie de 39 % sur les coûts d’une tournée de visites de supervision. Resumen Análisis de costo-eficacia del apoyo a la prestación de servicios de inmunización en Andhra Pradesh, India En noviembre de 2003 se puso en marcha en Andhra Pradesh, India, un modelo de apoyo a la prestación de servicios de inmunización (APSI) con la finalidad de fortalecer ese tipo de actividades con medidas de supervisión. El modelo de APSI integra un sistema de supervisión consagrado creado a partir de la infraestructura sanitaria existente. Este sistema tiene por finalidad: (1) identificar las áreas de alta eficacia y las que necesitan mejoras, (2) ayudar al personal a identificar y corregir las prácticas incorrectas, (3) mejorar las aptitudes del personal, (4) motivar al personal, y (5) poner en marcha medidas correctivas en los niveles pertinentes mediante el intercambio de información. La evaluación del costo y la eficacia del APSI en 16 distritos que participaron en el programa reveló que el costo adicional asociado a las tres rondas de visitas de supervisión de apoyo fue de aproximadamente US$ 110 630 (US$ 36 877 por ronda). La eficacia de los centros de salud y de las sesiones de inmunización se determinó mediante listas de verificación de 43 y 28 puntos, respectivamente, que pusieron de relieve mejoras considerables durante y después de la aplicación del APSI durante dos años. El cambio porcentual medio de la eficacia de los centros de salud entre la situación basal y la cuarta ronda de evaluación fue de aproximadamente un 36%, y la eficacia de las sesiones de inmunización aumentó como promedio un 9%. El costo marginal por punto porcentual adicional de la eficacia media de los centros de salud y por punto porcentual adicional de la eficacia media de las sesiones de inmunización durante el periodo de evaluación se estimó en US$ 3091 y US$ 12 760, respectivamente. La relación entre costo marginal y eficacia es relativamente sensible a los gastos de personal y de viaje. Se calcula que la integración del APSI en el sistema de inmunización de Pradesh Andhra puede traducirse en un ahorro del 39% por ronda de visitas de supervisión. Policy and practice Analysis of immunization service delivery support 228 Chutima Suraratdecha et al. Bulletin of the World Health Organization | March 2008, 86 (3) References Andhra Pradesh: 1. building a model immunization system. PATH; 2004. Available from: http://www.childrensvaccine.org/files/CVP_AP.pdf Supportive supervision [presentation]. 2. Meeting of the GAVI Implementation Task Force on Supportive Supervision, Geneva, 15–17 October 2002. WHO. Marquez L, Kean L. 3. Making supervision supportive and sustainable: new approaches to old problems, Maximizing Access and Quality (MAQ) Initiative paper No. 4, 2002. Washington, DC. Guidelines for implementing supportive supervision: a step-by-step guide with 4. tools to support immunization. Seattle, WA: PATH; 2003. Bradley J, Igras S, Shire A, Diallo M, Matwale E, Fofana F, et al. 5. COPE for child health in Kenya and Guinea: an analysis of service quality. New York: EngenderHealth; 2002. Kim YM, Figueroa ME, Martin A, Silva R, Acosta SF, Hurtado M, et al. Impact 6. of supervision and self-assessment on doctor-patient communication in rural Mexico. Int J Qual Health Care 2002;14:359-67. Robinson JS, Burkhalter BR, Rasmussen B, Sugiono R. Low-cost on-the-job 7. peer training of nurses improved immunization coverage in Indonesia. Bull World Health Organ 2001;79:150-8. PMID:11242822 Robinson JS, Burkhalter BR, Rasmussen B, Sugiono R. 8. Evaluation of immunizer-training-immunizer program in Maluku, Indonesia: An on-the-job peer training approach to improving the performance of health workers. BASICS II; 2001. Van Zwanenberg TD, Hull C. Improving immunization: Coverage in a province 9. in Papua New Guinea. BMJ 1988;296:1654-6. PMID:3135060 Pegurri E, Fox-Rushby JA, Walker D. The effects and costs of expanding the 10. coverage of immunization services in developing countries: A systematic literature review. Vaccine 2005;23:1624-35. صخلم دنهلا في ارهدنأ في عينمتلا تامدخ ءاتيإ معدل ةيلا َّعفلاو فيلاكتلا ليلحت في دنهلا في شيدارب ارهدنأ في عينمتلا تامدخ ءاتيإ معد جذونم ئشنأ دقل عينمتلا تامدخ زيزعت هنم فدهلا ناكو ،2003 برمفون/لولأا نيشرت رهش ماظن جذومنلا اذه ن َّمضتيو ،اهل معدلا لفكي يذلا فاشرلإا للاخ نم ،ةدوجولما ةيحصلا ةيساسلأا تامدخلا لىع زكتري ميمصتلا ديج ٍفاشرإ تاذ تلااجلما لىع فُّرعتلا .1 :بولسلأا اذه ءارو ةنماكلا يمارلما تناكو فُّرعتلا في ينلماعلا ةدعاسم .2 ؛روطتلل جاتحت يتلا تلااجلماو عيفرلا ءادلأا زيفحت .4 ؛ينلماعلا تاراهم ينسحت .3 ؛اهحيحصتو ةئطاخلا تاسرمالما لىع كراشتلا للاخ نم ةئملام تايوتسم في ةييرضحت ةطشنأب عوشرلا .5 ؛ينلماعلا في تكرـتشا ةظفاحم 16 في يرجأ يذلا ميـيقتلا حضوأ دقو .تامولعلماب ةيمكارـتلا فيلاكتلا نأ جذومنلا اذهل ةيلا َّعفلاو فيلاكتلا لوح جمانبرلا اذه ًايكيرمأ ًارلاود 110630 تغلب ةمعاد ةيفاشرإ تارايز ثلاث عم تقفارـت يتلا تارودو ةيحصلا زكارلما ءادأ ميـيقت امأ .)ةرايز لكل ًايكيرمأ ًارلاود 36877( كانه نأ ترهظأ دقف ،ةطقن 43و 28 ن َّمضتـت ققحت حئاول للاخ نم عينمتلا لدعم غلب مايف ،جمانبرلا اذه ذيفنت نم ْنينَتنس ضيم دعب ًاظوحلم ًانسحت ةميقلا نم ًاقلاطنا ةيوئلما ةبسنلا زارحأب ًاردقم يحصلا زكرلما ءادأ في يرغتلا تداز ماك ،%36 تغلب دقف ميـيقتلل ةعبارلا ةرودلا لىإ ًلاوصوو ةيدعاقلا ةبوسحلما ةيمكارـتلا فيلاكتلا امأ ،%9 لدعبم عينمتلا تارودل ءادلأا زارحأ 3091 رادقبم تداز دقف ةيحصلا زكارلما ءادأ زارحأ في ةدايزلا في ةيوئم ةبسنك نم عينمتلا تارود في ءادلأا زارحلأ ةيوئلما ةبسنلا تداز مايف ،ًايكيرمأ ًارلاود اهيف مت يتلا ةرـتفلا رادم لىع ًايكيرمأ ًارلاود 12760 رادقبم فيلاكتلا ثيح لكل ةيبسن ةيساسح تاذ فيلاكتلا لىإ ةيلا َّعفلا تلادعم تناك دقو .ميـيقتلا تامدخ ءاتيإ معد جذونم جاردإ نإ .ينلماعلا فيلاكتو رفسلا فيلاكت نم نم %39 ـب ر َّدقي يرفوت لىإ ىدأ عينمتلل شيدارب ارهدنأ ماظن في عينمتلا .ةيفاشرإ ةرايز لكل فيلاكتلا