Argentina - Provincial Health Sector Development Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No: 24325 IMPLEMENTATION COMPLETION REPORT (CPL-39310; SCL-3931A; SCPD-393IS; PPFB-P2270; PPFB-P2271) ONA LOAN IN THE AMOUNT OF US$101.4 MILLION TO THE REPUBLIC OF ARGENTINA FOR A PROVINCIAL HEALTH SECTOR DEVELOPMENT PROJECT 06/30/2002 Country Management Unit for Argentina, Chile and Uruguay Human Development Sector Management Unit Latin America and the Caribbean Regional Office This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENTS (Exchange Rate Effective ) Currency Unit = Argentine Peso AR$3.7 = US$ 1.00 US$ 1.00 = AR$3.7 FISCAL YEAR January I through December 31 ABBREVIATIONS AND ACRONYMS CAS: Country Assistance Strategy GoA: Government of Argentina HPAs: Hospital Piublico Autogestionados (Autonomous Public Hospitals) IBRD: International Bank for Reconstruction and Development ICR: Implementation Completion Report MIS: Management Infornation System MoE: Ministerio de Economia y Obras y Servicios Puiblicos (Ministry of Economy and Public Works) MoH: Ministry of Health MSP: Ministerio de Salud Provincial (Provincial Ministry of Health and MCBA) O&M: Organizational and Management OSs: Obras Sociales (Health Insurance Fund) PBA: Province of Buenos Aires PCU: Project Coordination Unit PRESSAL: Proyecto de Reforma del Sector Salud (Provincial Health Sector Development Project) PRESSS: Programa de Reconversi6n del Sistema de Seguro Social (Restructuring Program of Health Insurance System) UEC: Unidad Ejecutiva Central del Proyecto de Desarrollo del Sector Salud en las Provincias UEP: Unidad Ejecutora Provincial UNDP: United Nations Development Program Vice President: David de Ferranti Country Director: Myma Alexander Sector Director: Ana-Maria Arriagada Task Team Leader: Ruth Levine  ARGENTINA PROVINCIAL HEALTH SECTOR DEVELOPMENT CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 6 5. Major Factors Affecting Implementation and Outcome 9 6. Sustainability 11 7. Bank and Borrower Performance 11 8. Lessons Learned 13 9. Partner Comments 14 10. Additional Information 14 Annex 1. Key Performance Indicators/Log Frame Matrix 15 Annex 2. Project Costs and Financing 17 Annex 3. Economic Costs and Benefits 20 Annex 4. Bank Inputs 21 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 23 Annex 6. Ratings of Bank and Borrower Performance 24 Annex 7. List of Supporting Documents 25   Project ID: P006030 Project Name: AR-Prov. Health Sector Development Team Leader: Maria Lucy Giraldo TL Unit: LCSHD ICR Type: Core ICR Report Date: June 25, 2002 1. Project Data Name: AR-Prov. Health Sector Development L/C/TF Number: CPL-39310; SCL-393 1A; SCPD-393 iS; PPFB-P2270; PPFB-P2271 Countrv/Department: ARGENTINA Region: Latin America and Caribbean Region Sector/subsector: HR - Reform and Financing KEY DATES Original Revised/Actual PCD: 08/24/1993 Effective: 03/01/1996 03/01/1996 Appraisal: 03/14/1995 MTR: 12/02/1998 12/02/1998 Approval: 08/03/1995 Closing: 06/30/2001 12/31/2001 Borrower/lImplementing Agency: REPUBLIC OF ARGENTINA/MIN OF HEALTH Other Partners: STAFF Current At Appraisal Vice President: David De Ferranti Shahid Javed Burki Country Manager: Myrna L. Alexander Gobind T. Nankani Sector Manager: Evangeline Xavier Alain Colliou Team Leader at ICR: Maria L. Giraldo Marie Odile H. Waty ICR Primary Author. Luis Perez; Ruth E. Levine; Sati Achath 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: UN Institutional Development Impact: M Bank Performance: U Borrower Performance: U QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: Yes  3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The objective of the US$144.72 million investment project was to support the Government of Argentina's goal of rationalizing health sector spending. To achieve this goal, the project took a two-pronged approach: (i) strengthening the policy-making capacity of the central Ministry of Health (MOH) and the provincial health ministries, and assisting the Government in the implementation of specific changes in the incentives faced by public hospitals; and (ii) improving the capacity to deliver services by pilot testing the implementation of autonomous public hospitals (Hospitales Publicos Autogestionados, or HPAs) through altemative managerial models. The objectives were consistent with the Government's sector policy in several ways. First, the Government had taken steps to reform the health insurance system. Second, the operational and financial decentralization of services to provinces and municipalities was well underway, and the legal and financial bases were set for the establishment of autonomous public hospitals. Third, new regulations had opened the way for competition in the pharmaceutical market. Fourth, the Government, with support from the Bank, was committed to improvement of govemment-provided matemal and child health services. Fifth, reform of public hospitals and rationalization of health services in urban areas were considered priorities. The project objectives complemented the objectives of other Bank-financed projects, were supported by sector work, and were fully consistent with the Bank's Country Assistance Strategy (CAS), which emphasized the Government's ambitious reform program. In particular, the objectives were relevant to help the Government: (a) restore and maintain fiscal balance by improving the efficiency of public expenditures; and (b) support investments in human resources, especially with improved education, nutrition, and health. The Bank's heavy involvement in Argentina's health sector through a number of other activities - Maternal and Child Health and Nutrition Project (6025-AR), Social Protection Project (3957-AR), Health Sector Financing Study (Argentina: Facing the Challenge of Health Insurance Reform, Report No. 16402-AR), and Health Sector Insurance Reform Project (4002/3-AR) - placed the Bank in a strong position as a partner to the Government which it sought to address sector efficiency. The project envisioned several significant benefits. First, the project was expected to set the stage and provide the technical tools for an in-depth transformation of the health sector by strengthening policy-making through the definition of specific short- and medium-term strategies and actions aimed at furthering sector reform. Second, the project aimed to establish a new and more efficient delivery model, for future nationwide replication, by strengthening the organization and management of 15 public hospitals and guiding their transformation into HPAs. The project was also expected to increase cost-recovery, allow for more flexibility in the utilization of resources by hospital managers, reduce unnecessary utilization of hospital beds, and increase hospital productivity. The change of managerial model for public hospitals was expected to lead to progressive public-private integration in the provision of health services. The project took into account main risk factors to successful project implementation. The first important risk was the lack of future government support to reform in the health sector. While it was not possible to guarantee support to the project by all stakeholders, risks related to the sustainability of the project were minimized by: (a) the attention given during preparation to ensure support of key stakeholders, particularly of the jurisdictional governments, the Ministry of Economy (MOE), and some professional associations; and (b) a project design that was not overly ambitious, had a pilot and demonstration features, as well as specific and operational studies, focused on gradually building the sector's knowledge base for reform. The second important risk was that project implementation demands would outstrip the managerial - 2 -  capacity of the MOH, the provincial health ministries and the Municipality of Buenos Aires Health Secretariat. This risk was expected to be minimized by: (a) full-time project units, National Project Coordination Unit (Unidad Ejecutiva Central, or UEC) and Provincial Project Coordination Units ( Unidades Ejecutivas Provinciales, or UEPs), which had successfully managed preparation and had become familiar with Bank guidelines during the administration of an advance from the Project Preparation Facility (PPF); and (b) annual and mid-term reviews to assess implementation progress and adjust its pace and scope as needed. The project was technically demanding for the Borrower. Because the concept of autonomous public hospitals was novel in Latin America, the Government was initially not aware of its full implications. A wide variety of technical and political challenges and constraints, which were not fully known at the project's inception, emerged during implementation. 3.2 Revised Objective: Responding to a request from the Government, the Bank amended the Loan Agreement in July 1999 to use the savings of US$4.2 million to finance technical assistance activities aimed at contributing to the Government's health insurance reform program; this component was called Programa de Reconversion del Sistema del Seguro de Salud (PRESSS). Consequently, new development objectives were introduced into the project, including: (i) improving the institutional capacity of the funds of the health insurance system, to design restructuring plans for health insurers, known as Obras Sociales; (ii) improving the technical capacity of the health insurance superintendency (Superintendencia de Seguros de Salud, SSS), and implementing effective monitoring, control and regulatory tools for OSs to assure the provision of adequate services to beneficiaries; (iii) improving the institutional capacity of the MOH to improve the quality and efficiency of the information system; (iv) improving institutional capacity of the Social Development Secretariat to develop monitoring systems to finance health insurance for retired persons; and (v) designing on a pilot basis health insurance for the uninsured poor persons in selected provinces. In December 1999, PRESSS objectives and activities were modified, leaving those referred to as "SSS Strengthening" and "Strengthening of the MOH and Social Development Secretariat," as well as the tasks related to strengthening the provincial OSs. The remaining activities included those related to diagnostic studies and drafting of conversion plans for provincial OSs, and designing (as a pilot) health insurance for the poor and uninsured in selected provinces (Salta, C6rdoba and Rio Negro). It was expected that the health insurance pilots would be financed with a new World Bank loan - US$4.9 million from the Bank, US$ 1.01 million from the federal government, and US$1.16 million from provincial funds, under the auspices of a "Learning and Innovation Loan" (LIL). After several delays and discussions over a two-year period about the approach of the project, the LIL was cancelled in June 2002. 3.3 Original Components: The project consisted of three components: Component I. Formulation of Policy for Reform. Under this component, the project financed a limited number of studies and system designs to improve the incentive environment in which public hospitals operate. The aim of the studies was to develop action-oriented recommendations and designs in key areas, which would be implemented during the second half of the project. These areas were identified on the basis of the Government's priorities for action, their technical relevance to further reform, and their implementation feasibility. The studies included: (a) upgrading the national information system on sector resources, epidemiological information and productivity of service delivery; (b) defining priority health services financed by - 3 -  government subsidies (burden of disease and priority services packages); (c) formulating a proposal for a new financing model, which would include: designing a public hospital reimbursement system; drafting a performance contracting model; creating a strategy to separate the financing from the service delivery functions of the public sector at the provincial level; and creating a system to identify beneficiaries of the public insurance; (d) designing a regulatory framework for private health insurance and health care providers; (e) defining a quality guarantee system; (f) designing a strategy to reform provincial social insurance programs, in an effort to improve their financial sustainability and managerial efficiency; (g) reviewing the relevant legislation and institutional framework to improve the regulations on hospital waste disposal; (h) developing investment and project proposals for HIV/AIDS control; (i) nationwide expansion of the HPA initiative; and (j) an impact evaluation of the project. Component II. Pilot Implementation of Autonomous Hospitals. Under this component, the project sought to establish the basis for a broader reform of public hospitals. To that end, the project was to support pilot implementation of HPAs in three eligible jurisdictions identified by the Government, namely, the Province of Mendoza (PM), City of Buenos Aires (CBA), and the Province of Buenos Aires (PBA). While PM was chosen because of its achievements in health sector reform, CBA and PBA were chosen because the Government considered these major urban centers critical for health sector reform in Argentina. The HPA model was based on the separation of health financing and delivery functions of the public sector, which implied the gradual development of a purchasing agency role for provincial ministries of health, and the transformation of public hospitals into self-governing trusts. HPA hospitals would be autonomous in the management of all resources, and would establish contractual arrangements with public and private purchasing agencies. The project supported the process by introducing the most urgently required organizational, management, and financial tools into hospitals, and by pilot testing the HPA model during the second half of project implementation. The institutional development intervention of hospitals were to be carried out through organization and management (O&M) consulting firms, which would concentrate their efforts in: general organization and management; accounting and finance; patient administration; pharmacy materials and supplies; hotel services; diagnostic procedures; infrastructure and equipment maintenance; control of hospital infections; medical auditing; and management of medical waste. To facilitate the implementation of institutional development, the project was also to finance the design of a hospital management information system (MIS). To support and complement the institutional development activities, the component financed investment in hospital physical plant and equipment. These infrastructure improvements were intended to: solve bottlenecks to increase productivity; reduce the risk for patients and hospital staff; and increase productive efficiency. Investments were limited to the rehabilitation of existing infrastructure and replacement of basic equipment, and were to be completed in two phases. Component HI. Dissemination of Reform Initiatives. This component supported national dissemination of the findings and experiences of the first two components, as well as strengthening of the technical and management capacity of sector professionals. To this end, the project supported: (a) technical assistance, training, and workshops aimed at increasing the awareness and knowledge of the activities to reform the health sector, and disseminating the findings and lessons learned from the other two components to provinces which did not participate in Component II; and (b) 20 international scholarships in the areas of hospital management, public health and health economics. 3.4 Revised Components- Component I. Based on the 1999 amendment mentioned earlier, the following activities were financed under the project but administered under PRESSS: -4 -  (a) technical assistance to: (i) provincial OSs to design pilot restructuring plans and strengthen their information systems; and (ii) national OSs of less than 10,000 beneficiaries, to design pilot plans and operational mechanisms for their merger or acquisition. (b) institutional strengthening of the SSS: (i) monitoring and control systems, and (ii) design and implementation of norms for crisis resolution and prevention of failure of social health insurance funds. (c) institutional strengthening of the MOH in managing registration and filing systems for health providers and insurers, and of the Secretariat for Social Development in developing monitoring systems for, the financing of medical coverage for pensioners without health insurance coverage (non-contributing pensioners); (d) design of a pilot health insurance plan for poor, uninsured populations in selected provinces. Component H. A January 21, 1998, amendment to the Loan Agreement, which was retroactive to March 1996, established that the infrastructure and equipment investments would be completed in a single stage, rather than the two stages originally planned. This change was made to simplify the approval and execution of individual projects, thereby accelerating the overall program of works. Component III. The technical assistance activities for ministries of health, public hospitals and POSs in non-eligible provinces were not foreseen in the original Loan Agreement; they were incorporated in a September 15, 1999, amendment. This revision was necessary because of the strong demand for technical assistance in provinces that were not included in the pilot and because PRESSAL assumed a commitment to support provinces benefiting from Provincial Reform Loans, Rio Negro (4218-AR), Salta (4219-AR), Tucuman (4221-AR), San Juan (4220-AR) and Catamarca (4578-AR). 3.5 Quality at Entry: The project design predates the existence of the Quality Assurance Group (QAG), thus the project's quality at entry was not formally assessed. For the purposes of this report the quality at entry is judged to be marginally satisfactory. As mentioned earlier, project objectives were consistent with the CAS and the Government priorities, and the operation constituted a fundamental part of the reform of public hospitals and health sector in Argentina. Project components were reasonably well related to achieving the project objectives. Sufficient attention was given to strengthening hospital administration and management, as well as to investment in infrastructure and hospital equipment; these could reasonably be expected to lead to improvement in hospital productivity and capacity. However, several design flaws ultimately contributed to the project's overall poor performance. First, with respect to hospital autonomy, the project failed to take into account several external factors that are essential for strengthening autonomy. For example, no investment was made to strengthen provincial ministries, including their administrative capacity, training, information systems, performance evaluation, and capacity to monitor cost and quality indicators. Second, for several of the risks identified during project preparation, the remedial measures built into the design proved to be insufficient. For example, the risk of possible lack of future govermment support at the national and provincial levels was taken into account. However, in the course of the implementation, this risk became a reality and hindered the project's achievements. Letters of intent, which were signed by - 5-  provincial executive bodies, proved difficult to translate into legislative agreements, which required consensus. In the case of CBA, for example, the legislative consensus took a long time, and the political accord was disrupted with a change in administration. As a result, the concept of self-management of hospitals was very slow to take hold. Similarly, the risk that project implementation demands would outstrip the managerial capacity of the MOH, the provincial health ministries and the Buenos Aires Health Secretariat was considered, and preventive measures were adopted into the design. This risk also became a reality when the administrative weakness of these agencies hampered smooth project implementation. Several other flaws in project design were also manifested during implementation: (i) the design did not sufficiently take into account issues of govemance and institutional capacity, including corruption, which significantly hampered implementation; (ii) combining civil works and equipment investments with the planned policy, organizational and management reforms did not lead to positive results, in part because the design assumed that the institutional reform and hospital rehabilitation could proceed in a coordinated fashion; this did not happen; (iii) the design did not include actions to disseminate the reform to other sectors of the provincial governments, restricting them to health sector authorities, although many of the measures required for successful implementation (e.g., human resources, budget decentralization, and others) concemed other parts of the govemment. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The overall achievement of the project is judged to be unsatisfactory. The desired-for results - stronger policy-making and regulation, autonomous hospitals, and a generalization of the hospital pilot experiences - did not result from project interventions and investments. Component I. Strengthening of the Policy-making and Regulatory Capacity of the Central Ministry of Health (MOH) and the Provincial Health Ministries. Significant strengthening of policy-making capacity in the national and provincial MOH was achieved, fostering reforms and new programs in the health sector. Studies carried out supported the definition of a mandatory health benefits package (PMO), and the design and development of the quality guarantee national program, which furthered the policy dialogue regarding financing and insurance reforms. It also initiated national programs in epidemiological surveillance, which eventually led to the identification of the Public Health Surveillance and Disease Control Project (VIGI-A), Loan 4516-AR and HIV/AIDS prevention, which led to the identification of the AIDS and Sexually Transmitted Disease Control Project (LUSIDA), Loan 4168-AR. Component II. Pilot Implementation of Autonomous Hospitals. The project did not succeed in transforming even a single hospital into an autonomous entity with full juridical personality; thus, a major objective of the project was not attained. Strong opposition and resistance by many stakeholders - including ministers, trade unions and even some provincial hospitals - greatly hindered achievement of full-fledged autonomy. With the change of government administration in November 1999, the idea of autonomy itself was discredited because it was considered "anti-poor"; leaders in the health sector thought that autonomy would promote discriminatory behaviors against uninsured patients. Specific improvements were achieved in some of the pilot hospitals in terms of quality of service and efficiency. Some of these hospitals showed increased client orientation, increased productivity, efficient use of resources and effectiveness. With the same number of staff and smaller number of beds, these hospitals increased their admissions, ambulatory and emergency room consultations, laboratory tests and imagery - 6 -  examinations. Increased productivity was also accompanied by improved lengths of stay and higher occupancy rates. However, overall only about 40 percent of the performance indicators in the participating hospitals improved. All pilot hospitals that conducted organizational development exercises subsequently included new processes for hazardous waste separation, collection, disposal and treatment. Strangely, in some hospitals, civil works completed along with the managerial training and technical assistance appear to have had a negative impact: some quality and efficiency indicators worsened. Component M. Dissemination of Reform Initiatives. The project played an important role in the reform of the health sector at the provincial level. It assisted provinces not originally included in the project to design their reform of the health sector - one of the key elements supported by the Bank through the PRLs. The UEC successfully carried out a plan for disseminating and transferring technology and know-how (instruments and knowledge) to all provinces. This was done through short-term technical assistance to provincial ministries of health, public hospitals and provincial OS. Component IV. PRESSS. This component played an important role in disseminating the concept of public health insurance. Under the project, seminars and workshops were conducted to interest provincial leaders in the design and implementation of insurance programs, and the provinces of Santa Fe, Rio Negro and Chubut were supported in their efforts to work on this issue. 4.2 Outputs by components: Component I. Policy Reform Studies. Moderately satisfactory. The contribution of this component to the policy-making capacity of the MOH and the provincial Ministries of Health has been partially positive. Main expected outputs and end-of-project status are described below: (a) The inventory of sector resources and services was partially developed, although never completed and not structured for permanent updating. It succeeded in providing new information on human resources, physical resources and production in the public and private sectors. (b) The definition of health priorities/burden of disease study was not conducted. (c) Work on health sector financing included: definition of a system for the payment of integrated services packages to hospitals; study of the separation of health financing and delivery functions; creation of a management contracts model for PM and PBA, and support for experiences in La Pampa, Salta, Rio Negro and Tucuman provinces; study of beneficiary identification, which was the basis for the creation of a national OSs beneficiaries padr6n, coordinated among national agencies (SSS, ANSES and DGI). (The development of the PMO and the national OSs beneficiary padr6n were transferred to PROS (4004-AR).) (d) Design of a system to assure service quality both on the public and private sector was developed. Technical assistance was provided for the development of the Medical Attention Quality Guarantee National Program. Component II. Pilot Implementation of Autonomous Hospitals. Unsatisfactory. This component had several problems. First, execution of the component started at a different time in each one of the pilot provinces because the legislative bodies in PBA and CBA experienced delays in approval of - 7-  the subsidiary agreements for loans. The national government decided to maintain those jurisdictions rather than relocating the resources to interested provinces, which led to a major change in project design: to execute in a shorter period of time, the sequencing of activities - first institutional strengthening within the hospitals, and then investment in infrastructure and equipment - was altered. Second, commitment of the relevant authorities varied greatly across the 15 eligible hospitals with respect to the organizational and managerial improvements. In general, changes related to hospital management were accepted faster than those requiring policy changes from the jurisdictions. However, the commitment of the provincial MOH for implementation of some of the management tools was weak. As described later, the hospital information system was not developed, hindering the implementation of some key management tools, such as the record of production and costs. Third, only a small to moderate number of improvements were seen in the outcomes related to autonomous hospitals in the six major areas (organizational, financing, planning and budgeting, human resources, quality of services and knowledge transfer). According to a major end-of-project impact evaluation, indicators in CBA, PBA, PM improved by 14, 20 and 43 percent, respectively (PRESSAL Assessment Report, Clapp & Mayne, 2001). In general, training activities showed the greatest improvement (75, 80 and 83, respectively), which suggests that in the future improvements also will be seen in the other managerial areas. Fourth, activities under this component resulted in improvements in the physical plant for several hospitals (12 out of the original 15 expected - five in PBA, four in CBA and in PM), but results did not match the level of investment. Although facilities were improved, most did not fulfill expectations, particularly with respect to the quality of the finished work and the execution period. Virtually all the works started later than they were supposed to - between two and three years from the start, and two years before the closure. (The first started in September 1997, four started in 1998, three in 1999, and four in 2000.) Delays also occurred in the execution of works because of changes introduced during the work process, undefined terms in bidding documents and/or requests from hospital authorities. Contractors' financial problems worsened the already bad situation. Fifth, delays in the signing of subsidiary agreements with the provinces and delays in infrastructure activities caused delays in bidding processes, reception and delivery of medical equipment to respective hospitals. The procurement process started in late 1997 for PM, while the rest of the provinces started this process during 1998, with the first delivery in early 1999. Despite this problem, provision of equipment achieved reasonably good results. Component II. Dissemination of Reform Initiatives. Satisfactory. The major achievement of this component was to sensitize decision makers to the reform, and to increase awareness of the advances in the jurisdictions. The project provided technical assistance, training, national and international seminars, and workshops aimed at increasing the awareness and knowledge of the reform-related activities. In addition, some of the technical tools developed under Components II and III were disseminated to 14 provinces. The graduate study grant program was carried out satisfactorily (22 scholarship holders went to international institutions, and 63 went to national institutions). However, no information is available about the positions currently held by the recipients. Hospital administrators from provinces (152 participants) were given training in both national and international universities in the areas of hospital management, public health and health economics. -8 -  Technical assistance activities conducted under this component were related to: institutional strengthening of hospitals; support to the opening and moving of new hospitals; restructuring ,projects for health ministries/secretariats; health sector legal reform; support for implementation of management contracts; improvement of the skills of human resources staff; support for development of the provincial health insurance system, and separation of health financing and delivery functions; direct support for complying with the conditions of the PRLs. Component IV. PRESSS Activities under PRESSS were conducted in a satisfactory manner, and included: (a) A diagnosis report on the overall situation of provincial OSs, including Obras Sociales in the provinces of Catamarca, Chaco, Chubut, C6rdoba, Corrientes, Entre Rios, Formosa, Jujuy, Mendoza, Misiones, Neuquen, Rio Negro and San Juan, and conversion plans for OSs in the provinces of Rio Negro and Misiones. (b) Preparation of a framework for public health insurances for the poor, uninsured in Santa Fe, Chubut and Rio Negro. This included drafting the conceptual developments needed for the pilot implementation that were to be financed under the LIL. 4.3 Net Present Value/Economic rate of return: Not able to estimate with the available data. 4.4 Financial rate of return: N/A. 4.5 Institutional development impact: The institutional development impact of the project was modest. Reasons for limited impact included: (i) problems with the implementation agency (lack of continuity in management and professional staff), which resulted in an inadequate transfer of knowledge to fornal ministerial structures; (ii) a low capacity in pilot provinces to effectively use human and financial resources; and (iii) relatively little involvement in implementation from the national ministry, except when difficult problems arose. The major positive impact was achieved in many of the jurisdictions that were beneficiaries of technical assistance under Component III. The project led to greater interaction among the technical staff from the provincial Ministries. As mentioned earlier, the overall improvement in the performance indicators of participating hospitals was not as positive as expected. In many hospitals, the combination of delays in civil works and difficulties in procurement reduced the expected benefits from the installation of new equipment. Under PRESSS, the impact of the work on public insurance for the poor and uninsured was minimal because the pilot projects had a small number of beneficiaries. In the province of Santa Fe the impact may have been more significant because the strategy was a condition of the PRL. 5. Major Factors Affecting Implementation and Outcome 5 1 Factors outside the control of government or implementing agency: The economic recession experienced by Argentina from 1998 until project closure had a significant -9-  negative impact on construction firms, which in turn affected project implementation. Financial problems faced by some of the construction firms resulted in delays and the need to renegotiate contracts; some contracts were closed, and new bidding processes had to be started (e.g., hospitals in San Roque (PBA) and Durand (CBA). 5.2 Factors generally subject to government control: Two major factors within government control affected implementation of the project. First, the lag time in finalizing the Loan Subsidiary Agreement of PBA (January 1998) and CBA (November 1997) delayed initiation of the project. Second, fluctuations in government policy regarding autonomy and the government's weak capacity to advance the policy agenda affected project implementation. While the earlier administration was committed to public hospital autonomy, the new administration (December 1999 to December 2001) was very much opposed to granting autonomy to public hospitals, believing that strategy to be regressive and "anti-poor." 5.3 Factors generally subject to implementing agency control: Multiple factors that were under the control of the implementation agency greatly hampered the effective and timely execution of the project. These included: (i). Procurement problems. The project experienced a large number of procurement and contracting problems resulting in frequent changes in project management, delay in disbursement, and a negative reputation for the project within Argentina. In PM, major problems were observed in procurement in the construction of hospitals and supply of oxygen plants; in the CBA, procurement issues arose in the construction of Hospital Fernandez and Durand; and in the PBA, procurement of medical equipment was troublesome. Out of seven allegations investigated by the World Bank's Anti-Corruption Unit, mis-procurement was declared in two cases. On the Borrower's side, the federal anti-corruption office referred two cases to the judicial court for criminal investigation (bribe solicitation and manipulation of bids). (ii). Architectural problems. In some hospitals - and particularly in CBA - flawed architectural designs led to incomplete or inaccurate blueprints and technical specifications at a late stage in the bidding process. For instance, in the case of Hospital Fernandez, an audit of both the original technical specifications and the blueprints, and a contract performance audit on the actual construction showed that there were a large number of significant omissions and miscalculations in the technical specifications (e.g., elevators that could not accommodate beds to transport patients, lack of fire alarms, lack of clinical gas installations, etc.). In the contract itself, an amount of nearly US$900,000 worth of additional payments had been approved to remedy these shortcomings. Additional work resulting from such changes led to increased expenditure and delays in implementation. (iii). Operational problems. The PRESSAL project lacked coordination among its components, which operated in isolation, missing opportunities for the synergy anticipated in the original design. 5.4 Costs andfinancing: The total cost of the project was US$124.72 million, compared with the SAR estimate of US$144.7 million. Actual expenditure was lower than expected because: (i) the number of studies was reduced; (ii) some of the activities under institutional strengthening were not completed; and (iii) US$4.2 million was released to PRESSS for the creation of health insurance schemes for the uninsured persons and also for the reform of provincial OSs. The Bank financed US$84.84 million (68.0 percent), and the Goverrnent - 10 -  contributed US$39.88 million equivalent in local costs (32.0 percent). 6. Sustainability 6.1 Rationale for sustainabdlity rating: Unlikely. Despite some important accomplishments in advancing policy reforms, much of the project's work is unlikely to be sustained. As mentioned earlier, the project did not succeed in achieving autonomy for participating provincial hospitals, and thus it did not attain the project's major objective. This situation will persist as long as the incentive structure does not change. For instance, if the management of a provincial hospital does not have sufficient autonomy and flexibility on key matters - control of the goveming board; stability in management team; authority to control revenues, define strategic planning, and service portfolio; and control on staffing to achieve its performance targets - the concept of autonomous public hospitals will never materialize. At the close of the project, construction and equipping of three hospitals remained incomplete. Unless the governments in CBA and PM finance their completion - which is unlikely in an era of severe fiscal restrictions - they will remain unfinished. In general, sustainabiity of this component will largely depend on the government's support and commitment. In addition, it does not appear that the MOH or the provincial health ministries have significantly strengthened their structure and capacity in a sustainable way. Despite the generally gloomy picture, some of the improvements made by some hospitals in performance indicators are likely to be sustainable. With the support of the project, for example, PM was using performance agreements in its six major hospitals (five of which were included in PRESSAL) and expanded this reformn to all its public hospitals by 2002. PBA is using these agreements in 12 hospitals (two of them associated with PRESSAL) after a highly successful initial experience on 2000; the inclusion of all other PRESSAL hospitals will follow the completion of on-going civil works. CBA, although less advanced when compared to the other two regions, has approved a new law that mandates further decentralization of hospital management. The benefits of some of the studies financed by the project under Component 1 will be sustained; they have resulted in the design of additional projects (Public Health Surveillance and Disease Control Project, and AIDS and Sexually Transmitted Disease Control Project). The provincial OS studies conducted under this component are being used by the provinces to reduce financial deficits and provincial subsidies. 6.2 Transition arrangement to regular operations: The Bank will finance expenditures related to disseminating the experiences of the 15 pilot hospitals to other provinces. At the national level, the UEC will also share the reports from consulting firms with the relevant departments on the MOH, which have the potential to improve their functioning. At the provincial level, as advised by the Bank, provincial health ministries of CBA, PBA and PM have agreed to include the expenditure for maintaining hospital equipment into their annual budget. Nationwide, health policies reflect substantive aspects of the sector transformation, including Decree 939/2000 (public hospital with decentralized management). Transfer of outcomes and knowledge related to planning, control and regulation to the relevant MOH subsecretariat has been foreseen. 7. Bank and Borrower Performance Bank 7.1 Lending:  The Bank's perfomiance in the identification of the project was marginally satisfactory. Bank documents indicate that project preparation was well organized and highly satisfactory. The Bank, which had a harmonious team with a good skill mix, included intemational specialists in hospital management, hospital architecture and health management. During the project appraisal, the Bank team took into account the technical complexity of the project and assessed the project's risks and benefits. The Bank had a consistently good working relationship with the Borrower during preparation and appraisal. The positive evaluation of the preparation process, however, must be weighed against the design flaws that were made manifest during implementation - namely, the underestimation of the severity of the risks implied by the hospital reform. 7.2 Supervision The Bank's performance during the implementation of the project was unsatisfactory, largely because the Bank appears to have missed the "big picture" during PRESSAL's implementation. Over the five-year implementation period, there were 12 supervision missions, with an average of 2.4 missions per year, and in general the Bank's client relationship was cordial and productive. However, Bank supervision did not manage the considerable challenges of the PRESSAL project in a fully satisfactory manner. In particular, at various times during implementation, Bank task managers and management may have missed signals that the project was seriously off-track. Specific disbursement delays, evidence of misprocurement and lack of substantive progress on the development of autonomous hospitals were all identified in a timely manner and dealt with. For example, the Bank addressed the procurement and corruption problems appropriately, alerting the corresponding authorities and conducting intemal investigation by the Investigations Unit under the Department of Institutional Integrity. However, the Bank consistently chose to maintain the original project objectives, making only marginal changes in a somewhat patchwork fashion. Full-scale reformulation was not pursued, despite evidence - quite clear in retrospect - that the original development objectives were not realistic, given the health sector environment in Argentina. When the gravity of the problems was finally recognized it was already too late to redirect the project and supervision concentrated on damage control. 7 3 Overall Bank perfornmance: Overall, the Bank performance was unsatisfactory during project preparation, appraisal and implementation. Borrower 7.4 Preparation: The Borrower's performance in the preparation of the project was satisfactory. The Borrower embraced a strategic approach to health sector reform that was gradual, participatory, and based on the implementation of pilot and demonstration experiences. During the preparation stage, the Borrower displayed strong willingness and support to the objectives of the project. Commitment to change and technical capacity was identified at the level of individuals in the MOH, provincial health ministries, and hospitals and they worked closely with the Bank's project team on a continual basis, with full cooperation and enthusiasm. 7.5 Government implementation performance: The Government's performance during implementation was unsatisfactory. Although the project did not suffer from any counterpart funding problems, the Government failed in tackling both the political risk and the managerial capacity risk, which were identified at appraisal. The project implementation demonstrated the Government's weak commitment and lack of institutional capacity. The project was also plagued by perpetual problems in procurement, which became so serious that implementation was suspended at one point. - 12 -  Hospital information system software was not acquired in a timely fashion because the MOH stopped the process, arguing that a different software system, which the Ministry already owned, could be used. That software was later assessed by an international expert to be obsolete, but by that point it was too late in project implementation to initiate a new round of bidding. The Government did not respond effectively to the Bank's proposal to deal with the delays in the Subsidiary Loan agreements, and did not agree to the option of expanding PRESSAL's coverage to interested provinces. 7.6 Implementing Agency: Performance of the MOH was unsatisfactory. Because of the Ministry's internal bureaucracy, implementation progress was slow, especially during the first half of the project. For instance, cumulative disbursement in December 1997 - 20 months after the project was declared effective - was only US$7.9 million, compared to the SAR estimate of US$40.0 million by that point; the disbursement lag was greater than 80 percent. The December 1998 mid-term review estimated a disbursement lag of 74 percent, with US$18.4 million in cumulative disbursement compared to the anticipated US$68.0 million. Although this situation later improved slightly, the overall disbursement lag was relatively high during entire implementation period. Performance of the UEC in project management was generally unsatisfactory. The unit failed to address complaints from bidders and consultants and did not take measures to strengthen and separate functions in the area of procurement as advised by the Bank. During the five-year implementation period, the Project Coordinator was changed six times because of various allegations, and flaws in procurement and administrative concems in the MOH. This reflected serious lack of stability on the part of the UEC. There were numerous complaints from bidders and consultants regarding poorly conducted procurement procedures, inadequate budgeting, conflicts of interest, and lack of attention to the financial health of firms awarded large contracts for works and consultancy contracts. The MOH was ineffective in dealing with these allegations and procurement problems, both at national and provincial levels. This delayed the implementation and resulted in the public discrediting of the project, starting in 2000. The UEC's performance in monitoring and evaluation was satisfactory. There were both intemal and extemal evaluations of the performance of the pilot hospitals. Its performance in financial management was also satisfactory. The project had an adequate accounting system, financial reporting system, and separation of responsibilities and tasks. These provided timely and reliable information on project financial flows. 7.7 Overall Borrower performance. The overall perforrnance of the Borrower was unsatisfactory. 8. Lessons Learned 1. The Bank should encourage the Borrower to request a reformulation of a project if and when it becomes clear that the development objectives are no longer feasible. PRESSAL limped along during most of its lifetime, and consistently was a troubled and troublesome project, from the perspectives of both Bank and client. Large amounts of managerial and technical energy were devoted to marginal "fixes" to problems that in fact required a full-scale re-thinking of the project. In retrospect, it can be argued that cancellation or reformulation of the project would have led to better outcomes. - 13-  2. Operations requiring provincial commitments should obtain those commitments before project approval. Required legal instruments - such as the legislative authorization required when a province becomes indebted - should be signed as a condition for negotiation, to minimize post-effectiveness delays. 3. A reform is a political process first, and only later a technical process; it requires on both a creation of the political viability and development of technical feasibility. For both of these aspects, participation outside of the health sector and commnunication with civil society is essential. Having a broader base of support for the concept of autonomous hospitals might have led to less variation in politicians' views about PRESSAL with changes in government administrations. 4. In a reform-oriented project, investment in civil works should be made contingent on adoption of specific changes in the sectoral "software" (e.g., management, human resources, or the other subjects of the reform). In PRESSAL, the link between the reform and the civil works was inconsistent, and the civil works ended up being implemented largely in isolation from management improvements. Thus, many opportunities for synergy and sustainability in project benefits were lost. 5. Intense supervision is crucial for smooth project implementation, particularly In the face of political turbulence. In the early part of the project, the task team was provided with sufficient budget for close supervision, and the Bank was able to keep track of all aspect of the project and take necessary remedial measures. Towards the end of the project, severe budgetary constraints prevented the task team from close supervision of civil works and technical aspects of the project. 9. Partner Comments (a) Borrower/implementing agency: The Borrower has carried out its own evaluation of the Project and has reported that it agrees with the content of the Implementation Completion Report. (b) Cofinanciers: N/A. (c) Other partners (NGOs/private sector): N/A. 10. Additional Information ICR Team Ruth Levine (ICR Task Team Leader) Luis Perez (Consultant) Sati Achath (Consultant) Juan P. Uribe (Co-Task Team Leader) Natalia Moncada (Program Assistant) Coments Received from: Ariel Fiszbein (Sector Leader) Evangeline Javier (Sector Manager) Alexandre Abrantes (Sector Manager, AFTH2) - 14 -  Annex 1. Key Performance Indicators/Log Frame Matrix Impact Indicators Project Components Indicator Projected in Current / Last evaluation SAR ___________________ 1. Political Framework Studies were initiated, completed as Dec. 1996 Dec 1998 Mid Term Mission, all studies Reforms well as the initiation of Pilot test on were completed. recommendations made on the studies Dec. 1997 2. Progress Reports on The Implementation of Pilot Hospitals * Infrastructure Civil works initiated and in Dec. 1996 Delay in the initiation of implementation 5 in the Province of Buenos mplementation at 40 % in all of civil works: Aires, urisdictions (i) in Sep. of 1997 1 is 4 in the City of Buenos Aires nitiated, and Concluded phase of 40 % and Dec.1997 (ii) in 1998 4 are 3 in the Province of Mendoza implementation on the remaining 60 % initiated n all jurisdictions (iii) in 1999, 3 and Total: 15 forecasted (iv) in year 2000, another 4 works are initiated. Total: 12. * Information Technology National Strategic Plan Dec. 1996 Dec. 1997 Initiation of the bidding process for the first two hospitals Dec 1996 First two hospitals are completed and fully operational in each jurisdiction. Dec 1998 Initiation of the bidding process for the three last hospitals in each jurisdiction Dec 1999: Terms of Dec 1998 Reference for the Implementation process of the last International Public three hospitals Bidding. In August 2000. Suspended All hospitals are fully operational Dec. 1999 due to lack of time for the fulfillment on the terms Dec. 2000 Institutional Development Initiated in the two first hospitals of Dec. 1996 each junsdiction Completion in the two first hospitals and initiated in the last ones of each Dec 1998 Fully executed. Objectives on global urisdiction hange are reached at 40 % In process in the last three hospitals of eachjurisdiction Dec. 1999 Finalized and in full operation in all targeted hospitals Dec. 2000 - 15 -  3. Dissemination of Reform 1. Technical Assistance to jurisdictions In December 1998 TA was made available to 14 provinces not Initiatives and other entities no beneficiaries of the transfer of included under component II the pilot experience. recommendations made under 2. Human Resources Development and component I of the Tining roject should have been 2. 2.1: Biddings results: 2.1 Training scholarships Plan nitiated. I International: 7 II International: 15 2.2 National and Intemational III National: 42 Seminars Amendment IV National: 20 September 15 1999, added "the I st. International Seminar about Financing 2.3 Human Resources technical nefit of TA to of the Health Sector Bariloche (June 1997) development System: the Ministries of Regional Seminar on Management (i) Hospital Management and Public Health of Contracts La Pampa. Date: April 1998. dministration the Provinces Exchange on Management Experiences (ii) Health Economy. (MSP), Public Bariloche. Date: November 1999 (iii) Public Health (Epidemiology Hospitals and "Management Experiences of the Health and statistics) Social Works on Services" Buenos Aires. Date: November the Provinces". 1999. 2. 2.3: Results: 152 trained hospital administrators within he provinces. 3.) Web Page of PRESSAL as of October 1998. 4.) December 2000 3.-) information dissemination and Workshops in the provinces of social communication Santa Fe, Cordoba, Catamarca; Misiones 256 attending officials 4.) Dissemination Program and _Technological Transfer - 16 -  Annex 2. Project Costs-and Financing Annex 2a (1) Project Cost by Component Component Budget Accumulative Reimbursements Projected Projected % of s/ SAR Execution Jan -April '02 Reimbursements Accumulative Advance unffl (sue 87) (1) May- Execution tfil 31/1212001 Jun '02 (2) 30/06/02 1. Formulation of policy reforms I Separation design between loans and financing in the provinces 0.04 0.04 0.00 0.00 0.04 100.0% 2. Relevamiento of health services 0.82 0 82 0.00 0.00 0.82 100.0% 3. Sanitary pnorities in Argentina 0.09 0.09 0.00 0.00 0.09 100.0% 4. Payment systems design in hospitals 5. Design of systems to identify the 0.07 0.07 0.00 0.00 0.07 100.0% beneficiaries of the diverse coverage's 0.11 0 11 0.00 0.00 0.11 100.0% 6. Regulatory framework for insurers and pnvate loaners 0.00 0.00 0 00 0.00 0.00 0.0Yo 7. Quality Control 8 Social Provincial projects Reform 0.00 0.00 0.00 0.00 0.00 0.0% 9. AIDS 1.85 1.85 0.00 0.00 1.85 100.0% 10. Study of Pre-inversion 0 32 0.32 0.00 0.00 0.32 100.0% 0.03 0 03 0.00 0.00 0.03 100.0% Total 3.33 3.33 0.00 0.00 3.33 100.0% 11. Pilot program for Independent Hospitals 1. Infrastructure improvement 2. Insatutional development 80.36 73.06 0.21 0.25 73.52 91.5% 14.49 13.49 0.00 0.00 13.49 93.1% Total 94.85 86.55 0.21 0.25 87.01 91.7% 111. Reforms disemination 1. Technical Assistance 2. Training programs 5.36 5.36 0.00 0 06 5.42 101.5% 3.03 2.99 0.02 0.00 3.01 99.3% Total 8.39 8.35 0.02 0.06 8.43 100.5% IV: Management 1. Management 8.73 9.21 0.00 0.00 9.21 105.5% Total 8.73 9.21 0.00 0.00 9.21 105.5% PRESSS 2.86 1.96 0.00 0.00 1.96 68.5% P.P.F. 1.19 1.19 0.00 0.00 1.19 100.0%/0 LOAN TOTAL 119.35 110.59 0.23 0.31 111.13 93.1% Non Eligible Expenses Administatve Cost PNUD 2.16 0.00 0.02 2.18 Auditing Cost 0.23 0.00 0.01 0.24 Honoranes UEC y UEPs 0.00 0 03 0.06 0.09 Banking Costs UEPs 0.02 0.00 0.00 0.02 Non-eligible consulting conracts 0 13 0.00 0.00 0.13 Non- eligible project contacts- 0.75 0.00 0.00 0.75 Non-eligible Equipment Contracts 0.23 0.00 0.00 0.23 Expenses UEP Pcia. of Buenos Aires 0.25 0.00 0.00 0.25 ( Honoranes, Others, Equip. Mant.) Non Eligible expenses subtotal 0.00 3.77 0.03 0.09 3.89 0.0% PROJECT TOTAL 119.35 114.36 0.26 0.40 115.02 96.4%h - 17 -  Annex 2b: Project Costs by Procurement Arrangements dgets/SARN ' Aumulatve'ProJdcteilExecutl6ntto, -9 t~, , ___ __ _______ n tr w +s ^Ns fl 30/06/2002 Categoryq IUPI LPN: 'OTHERS ITOTL i _,,_ LPI ;" "LPL4,?1 >OTHERS -NFBS : '-.T.O.TAL 1. Community Service 48.97 3. 10 52.07 31.21 19.40 0.34 0.75 51.70 2. Teams 9.60 9 36 2.90 21.86 21.04 0.36 0.07 0 23 21.70 3. Consultoria 32.64 32.86 26 35 0.13 26.48 4. P.P.F. 1 19 1.19 1.19 1.19 5. Not Assigned 0.00 0.00 0.00 6. Administration 8 73 8.73 9.21 2.78 11.99 PRESSAIL Total 9.60 , 58.33 ,*4856 4116.49 52.255 19.'7,6 437r16 '.3.89 ',,2 113.06 PRESSS 2.86 2.86 1.96 - 1.96 Generob~Total 9.60 ' 58.33 A 51.4 119.35 52.25^., 19.76 - 39`2, 3.89,, !- tI115.02 "Paitiipation 8.0% 48.9% 43.1% 100.0% 45.4% 17.6% 34.0% 3.4% 100.0% , ., . _____________ 56.9% 43.1% 100.0% 62 6% 34.0% 3.4% 100.0% Notes: Detamls of the concepts included in others: Details of the concepts included in NBF Selection based in cost and quality 20.61 Administrative Cost 2.18 Individual Consultants 9.7 Auditing Cost 0.24 Direct Contracting 2.62 Honoraries UEC & UEPs 0.09 Contest and Minor expenses 3 04 Banking costs UEPs 0.02 P.P.F. 1.19 Non Elegible Consulting Contracts 0.13 PRESSS 1.96 Non Elegible Work Contracts 0.75 Other Totals 39.12 Non Elegible Equipment Contracts 0.23 UEP Pcia. De Buenos aires Expenses 0.25 Total of non eligible concepts 3.89 - 18 -  Annex 2c: Project Financing by Component Budget s / SAR Accumulative % de Advance Execudon until 31/12/2001 Component BIRF LOCAL BIRF LOCAL BIRF 70% LOCAL 70% 30% 70% 30% 30% 1. Pollcitcs Formuladon for the reforms 1. Separation design between loans and financing in the provtnces 0.3 0.3 0.03 0.01 100.0% 100.0% 2. Relevamiento of health services 3. Sanitary pnorities in Argentina 0.57 0.57 0.57 0.25 100.0W 100.0% 4. Payment systems design in hospitals 0.06 0.06 0.06 0.03 100.0% 100.0% 5. Design of systems to identify the beneficiaries of the 0.05 0.05 0.05 0.02 100 0% 100.0% diverse 0.08 0.08 0.08 0 03 100.0% 100.0% coverage's 6. Regulatory fiamework for insurers and pnvate 0.00 0.00 0.00 0.00 0.00/0 0.0