Madagascar - Health Sector Improvement Project

 ICRR 10764 Report Number : ICRR10764 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 08/22/2000 PROJ ID : P001520 Appraisal Actual Project Name : Health Sector Improvement Project Costs 42.5 34.36 Project US$M ) (US$M) Country : Madagascar Loan/ Loan US$M ) 31 /Credit (US$M) 30.22 Sector (s): Specific Diseases, Cofinancing including Malaria, TB, US$M ) (US$M) Others L/C Number : C2251 Board Approval 91 FY ) (FY) Partners involved : Closing Date 06/30/1997 12/31/2000 Prepared by : Reviewed by : Group Manager : Group : 2. Project Objectives and Components a. Objectives The project was designed to support the implementation of the government's Health Sector Program (HSP) over the period 1992-96, with the following (original) objectives: (i) to reduce mortality and morbidity; (ii) to moderate fertility levels; and (iii) to improve the efficiency and sustainability of public expenditures for health, the sector policy framework and institutional coordination mechanisms. The revised project objectives (approved by the Board in May 1996) were to: (i) reduce mortality and morbidity by strengthening priority disease control programs and epidemiological surveillance; (ii) increase availability and affordability of essential drugs; (iii) improve the quality of district health services and promote community-managed cost recovery; and (iv) strengthen management and planning capacity of MOH. b. Components The revised project supported the following components: (A) communicable disease programs (US$ 18.6 m); (B) improvement of drug availability and affordability (US$ 2.0 m); (C) support to district health services and community-managed cost recovery (US$ 10.8 m); and (D) institutional development (US$ 5.0 m). c. Comments on Project Cost, Financing and Dates The project was approved on May 1991, but, due to political turmoil, was not made effective until April 1992. At appraisal, the total project cost was estimated at US$42.5 m, with an estimated government contribution of US$11.5 m. The latest estimate of total project costs of US$34.4 m included a government contribution of only US$4.1 m. 3. Achievement of Relevant Objectives: The project achieved modest success in meeting the revised objectives. The malaria and tuberculosis control programs were strengthened, concurrent with decreases in related morbidity and mortality, though it is not clear whether the decreases are attributable to the project. Project support also helped to lower the incidence of schistosomiasis, but had little effect on the Plague lethality rate. A central drug procurement and distribution agency, SALAMA, became operational in 1998, which is procuring generic drugs at reduced prices through ICB. It is not yet involved in direct drug sales to health facilities, however, and no data are presented regarding the project's impact on drug availability. With respect to support for district health services, the project did not carry out most of its proposed activities and achieved few of its objectives (see below). Institutional development objectives were only partially achieved, due in part to the extensive, but not always successful, role of foreign consultants and organizations. Project activities strengthened in-service training and IEC service capacity. In the realms of health policy development and personnel management, project results were mixed and fell below targets. The project also contributed to the establishment of a cost recovery system involving local communities and a HMIS. No data are presented, however, on the coverage of the system, cost recovery rates, the extent of actual community participation, the impact on service quality, or access by the poor. 4. Significant Outcomes/Impacts: After the comprehensive mid-term review, carried out in collaboration with the MOH and other donors and international organizations, project performance and donor coordination improved significantly. The project also contributed to the development of the government's 1996-2000 health sector policy, which is providing a framework for government and donor programs. The establishment of an independent drug agency is an important accomplishment, although the impact at the facility level remains to be determined. The project contributed to wider availability of anti-malarial and TB drugs, which may have contributed to the control of malaria and tuberculosis, albeit below the ambitious targets set at appraisal. The achievements highlighted in the ICR were in areas in which other donors and international organizations were actively involved, however, making it difficult to ascertain the project's relative contribution. 5. Significant Shortcomings (including non-compliance with safeguard policies): As the ICR points out, poor project performance during the first four years of the project were largely due to deficiencies in the original project design, which underestimated risks and did not include: (i) concise project objectives; (ii) a provision for the establishment of a project implementation or management unit; (iii) a monitoring and evaluation system; or (iv) donor coordination mechanisms. Other domestic factors impeding implementation were political turmoil and the lack of clearly formulated health sector policies or strategies to support project implementation and management. Project restructuring addressed many of these problems, and was responsive to the government's 1996-2000 health sector policy, though there seems to have been an over-reliance on external actors to achieve project objectives. There were also two notable issues of disagreement between the Bank and the borrower, which impeded the implementation process: (1) the government's 1997 hiring of a National Coordinator, of whom the Bank disapproved, which led to a temporary suspension of disbursements and (2) the scope and execution of component (C), which was ultimately restricted to supporting central and provincial-level structures, the benefits of which only trickled down to the district level in the form of equipment and selective rehabilitation projects. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory Although project performance improved after restructuring, the project only partially achieved its objectives, and the ICR suggests a number of concerns with regard to efficiency and accountability in the use of project funds. Institutional Dev .: Modest Modest Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Quality at entry and initial supervision were unsatisfactory, but improved after the mid-term review. Borrower Perf .: Satisfactory Satisfactory MOH commitment and performance were unsatisfactory in the early years of the project. Performance improved subsequently, but counterpart funding was inadequate and various systemic problems continued to affect implementation. Quality of ICR : Unsatisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: The ICR/project highlights the following lessons: (i) project objectives, implementation and management responsibilities, and sustainability issues must be clearly and realistically defined and agreed on by all parties involved, taking into account sector conditions and institutional capacity levels; (ii) in countries and sectors in which there is extensive donor/international organization involvement, the government should develop a comprehensive strategy that forms the basis of donor interventions and includes coordination mechanisms; (iii) the MOH or other relevant government agency should be actively involved in every stage of the project. 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR provides a good description of process issues, but the data and analysis presented in the ICR are not sufficient to assess the project's development effectiveness. Some outcome indicators are provided, but the information and analysis in the ICR are insufficient to assess whether project activities might have plausibly contributed to the changes. The borrower's ICR provides a good description of project outputs, however, which provided sufficient evidence to support a "moderately satisfactory" rating.