Colombia - Municipal Health Services

 ICRR 11120 Report Number : ICRR11120 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 02/14/2002 PROJ ID : P006854 Appraisal Actual Project Name : Co- Municipal Health Project Costs 83.2 32.5 Services US$M ) (US$M) Country : Colombia Loan/ US$M ) 50 Loan /Credit (US$M) 21.2 Sector (s): Board: HE - Health (100%) Cofinancing US$M ) (US$M) L/C Number : L3615 Board Approval 93 FY ) (FY) Partners involved : Closing Date 12/31/2001 12/31/2000 Prepared by : Reviewed by : Group Manager : Group : Timothy A. Johnston Alice C. Galenson Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The project was envisioned as the first stage in a 10-12 year program to reorient the public health system in Colombia. The specific objectives were : Enhance municipal capacity to plan and manage provision of primary care, in nine "departments" ; Improve the quality, efficiency and coverage of primary health services in participating municipalities; Strengthen design, execution, and evaluation of key policy measures to support strengthening of primary care . b. Components Components included: Institutional development (11 % of costs), including (a) introduction of municipal health management information systems, and (b) training and technical support for administrative authorities, health administrators, and community representatives. Service Development (80 % of costs), (a) municipal subprojects supporting the introduction of a basic package of primary health services, including support for supplies and infrastructure improvements, (b) training and technical support for health workers . Policy Development and Evaluation and Project Management (9 % of costs), including support for policy design and implementation (pricing, cost recovery, quality control ), and to undertake an evaluation of project impact . c. Comments on Project Cost, Financing and Dates A new government was elected in 1998, which did not fully support the program . In January 1999, $22.7 million was reallocated from the project to respond to post -earthquake rehabilitation needs . The government subsequently appointed a "liquidation team" to close down remaining project activities, and the project was closed a year early . $2.2 million of the remaining loan was cancelled . 3. Achievement of Relevant Objectives: The project provided useful technical support to central government and municipalities during a major sector reform, but did not achieve its primary objective of reorienting and strengthening the public health system in Colombia . Several months after project approval, the government passed legislation decentralizing responsibility for both operation and financing of health facilities to local government . The project sought to provide flexible support for the reform process, but both the project and the government failed to maintain a focus on essential public health function during reforms. Following the mid-term review in 1996, project objectives were changed substantially -- for example, focussing on supporting municipal management rather than primary health care -- and increasing coverage to 16 'departments'. But the project was never formally restructured . The Bank attempted to refocus the project again in 1998, but the government decided shortly thereafter (with Bank agreement) to reallocate remaining funds to earthquake relief. During the life of the project, immunization coverage -- one of the original key project indicators -- declined both nationally and in project areas (from 86% to 80% for DPT in target areas). (The ICR presents data indicating that declines were less in project localities, but the borrower's comments suggest that these findings are not robust). The project financed 106 infrastructure projects, which were appreciated by local governments, but the Ministry of Health indicated that not all were finished . 4. Significant Outcomes/Impacts: The project supported a range of studies, training, guidelines, information systems, and other activities at national and local level to support the reform process, and to strengthen the ability of municipalities to adapt to their new roles. Overall, 19 municipalities received a full package of technical assistance, and 270 municipalities were given critical elements of the TA package . The ICR reports that these were perceived by municipal health staff as valuable in adjusting to the new reforms, and that the manuals produced with project funds are being used in non -project municipalities. (The impact of these capacity building and reform activities is not well documented, however ). 5. Significant Shortcomings (including non-compliance with safeguard policies): The project's major shortcoming was that it lost its original focus on strengthening primary health care, but was never adequately restructured or refocused to support the reform process . The original primary health care outcome indicators were abandoned at MTR, and replaced by input /output indicators for training, studies, etc . Project design envisioned development of a comprehensive package of primary care services, supported by extensive clinical training, but this was not implemented . Project infrastructure investments disproportionately benefitted better -off communities, despite the project's stated pro -poor focus. Despite an initial allocation of US$1 million for monitoring and evaluation, the project failed to establish any sort of evaluation system . As such, when the new government came to power, "the project failed to fully justify its benefits ." The project also allocated US$5 million for computer equipment in support of Y2K compliance in secondary and tertiary hospitals, which was neither in project design nor consistent with project objectives . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately The project made some useful Unsatisfactory contributions to capacity building, but its original objectives with respect to strengthening primary health care were not met. Project objectives were not formally revised despite changes in the sectoral context, and a significant reorientation in project activities . Institutional Dev .: Modest Modest Sustainability : Likely Unlikely Project activities were abruptly discontinued, with no arrangements put in place to ensure sustainability . Bank Performance : Satisfactory Unsatisfactory The Bank provided flexible support and good technical inputs, but neither adequately maintained focus on original objectives, nor adequately restructured the project to adapt to sector reforms . Reform legislation was well advanced at project appraisal, and could have been better anticipated. Borrower Perf .: Satisfactory Unsatisfactory The borrower shares responsibility for not retaining sufficient priority or focus on this project. Borrower comments on the ICR state that both project outcome and borrower performance were unsatisfactory. Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: Maintaining a focus on essential public health functions during a structural /financial reform process is essential . While some degree of flexibility is appropriate, projects should either maintain a focus on original objectives, or formally restructure to develop new objectives and outcome (not just process) indicators. Incorporating indicators and funding for monitoring and evaluation into project design is not enough; both Bank and borrower need to sustain attention and priority during implementation . Financing sub-projects based on proposals from local governments may have negative equity consequences in the absence of technical support for weaker local administrations (which tend to have more poor people ). 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR provides a good description of the project's design and implementation experience, both strengths and shortcomings. But the ICR's narrative would suggest less positive project ratings . Borrower comments include a number of criticisms of the project, which are not adequately reflected in the main ICR text . More information on the sector context, particularly the specific content of reforms, would have been useful .