Argentina - Road Maintenance and Rehabilitation Sector Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 22479 PERFORMANCE AUDIT REPORT ARGENTINA ROAD MAINTENANCE AND REHABILITATION SECTOR PROJECT (Loan 3611-AR) June 27, 2001 Sector and Thematic Evaluation Group Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents Currency Unit = Peso (A$) A$1.00 = US$ 1.00 Weights and Measures m = meter (3.281 feet) km = kilometer (0.621 mile) Abbreviations and Acronyms CAS Country Assistance Strategy COT Construir, Operar, Transferir (Build, Operate, Transfer) CREMA Contrato de Recuperaci6n y Mantenimiento (Contract for Rehabilitation And Maintenance) DNV Direccion Nacional de Vialidad (National Highway Directorate) DPV Direcci6n Provincial de Vialidad (Provincial Highway Directorate) GNP Gross national product HDM Highway Design Model ICR Implementation Completion Report IMF International Monetary Fund IRI International Roughness Index Km kilometer OED Operations Evaluation Department PAD Project Appraisal Document PMS Pavement management system SAR Staff Appraisal Report Fiscal Year January 1 - December 31 Director-General, Operations Evaluation Mr. Robert Picciotto Director, Operations Evaluation Department Mr. Gregory K. Ingram Manager, Sector and Thematic Evaluation Mr. Alain Barbu Task Manager Mr. Heman Levy FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. Office of the Director-General Operations Evaluation June 27, 2001 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Argentina Road Maintenance and Rehabilitation Sector Project (Loan 3611-AR) Attached is the Performance Audit Report prepared by the Operations Evaluation Department on the above project, which was approved for a loan of $340 million in FY93 and closed on schedule on June 30, 2000. An undisbursed amount of $11 million was canceled. The project objectives focused on the maintenance and rehabilitation of the national road network and on assisting central and provincial highway agencies to reform in line with government policies initiated in the early 1990s to decentralize, to privatize, and to reduce the role of the state. By the time of project preparation, about 9,000 kilometers, or one-quarter of the national road system had been concessioned to private operators, and the project aimed to help the national highway agency (DNV) to further outsource management of the road system. The project also aimed to help DNV introduce environmental assessments for its road investments. The project financed a four-year tranche (1993-97) of DNV's maintenance and rehabilitation program (including emergency works to repair damages from a flood in 1992), technical assistance covering a wide range of areas (including modem highway management systems, assistance to the provincial highway departments, monitoring the project, and specific studies), and equipment. The project was successfully implemented despite shortages of counterpart funds, and delays in launching some of the works, especially during 1995, when Argentina's economy was being severely affected by the Tequila effect (the Mexican fiscal crisis). A new outsourcing instrument created by DNV during project execution, a performance-based maintenance contract (labeled CREMA), covering a mix of rehabilitation and routine maintenance for a subnetwork of roads (200-400 kilometers per subnetwork), proved highly successful. It allowed the agency to catch up with implementation delays, surpass appraisal targets, and the high competition for the contracts led to prices below appraisal estimates. Budgetary allocations appeared to have given priority to payments under CREMA contracts. The Bank cost-sharing for the CREMAs was 75 percent, which was higher than for the other project components. This high level of Bank financing proved essential to attract contractors, supported by their banks, to bid for this new type of contract, since the CREMAs (i) were launched when Argentina was still in a major economic and fiscal crisis, and (ii) as designed This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 by the DNV, required contractors to pre-finance part of the works, and therefore take significant financial risks. Under the follow-on highway project, the cost-sharing has been reduced to 60 percent and satisfactory competitive contracting has continued. The project yielded important outcomes. First, the percentage of the non-concessioned national road network rated poor dropped from 35 percent at appraisal to about 5-10 percent at project completion. Second, with the introduction of the CREMAs and delegation to the provinces, the DNV now has six different outsourcing instruments, each one for a specific type of work, traffic level, and unit cost of the works. Third, delegation to the provincial highway departments for the maintenance of national roads proved successful. Following the experience with the two pilot provinces (La Pampa and Santa Cruz), DNV entered into performance-based contracts with an additional 12 provinces. Fourth, the project substantially strengthened DNV, since it helped the agency further extend and diversify outsourcing, establish a pavement management system, develop and enforce strict environmental safeguards, become leaner, more effective (DNV's staff was reduced by a third between project appraisal and completion), prepare a reformulation of the national road network, and prepare plans for installing modem traffic information systems and updated axle-load control systems. The economic rate of return was 44 percent (30 percent at appraisal) for the rehabilitation component and 60 percent for the CREMA contracts. The audit rates the project outcome highly satisfactory, institutional development high, and sustainability likely. Bank performance is rated satisfactory, and borrower performance highly satisfactory. These ratings are consistent with those in the ICR, although the audit upgraded borrower performance, because DNV created a new outsourcing instrument that allowed it to turn around the project and exceed the targets, and because it made significant progress in delegating work to the provinces. Experience with this project confirms a number of OED lessons: (i) Delegation of maintenance works to the provincial governments under performance-based contracts helps strengthen the provincial agencies, while shifting the role of the national agency away from execution and towards management; (ii) Because of their preferential treatment in the budget, CREMA-type contracts may be a way to secure more reliable funding for highway maintenance. (iii) A high level of cost sharing by the Bank in projects in middle-income countries helps bring the private sector to new activities in times of fiscal crisis, but for such activities to be sustainable they should be designed to reduce and eventually eliminate dependence on external finance. (iv) In leading reform countries, the Bank role should be one of supporting reforms, of helping to devise mechanisms to monitor the benefits of reforms, and of learning and disseminating the experience to other member countries. Attachment i Contents Principal Ratings ........................................................................................................................... iii K ey Staff Responsible ................................................................................................................... iii Preface ............................................................................................................................................. v 1. Background .............................................................................................................................. 1 2. O bjectives, Preparation, and Design ..................................................................................... 1 3. Im plem entation ........................................................................................................................ 4 4. O utcom e ................................................................................................................................... 9 Road Condition................................................................................................................... 9 M ultiple Outsourcing M ethods ........................................................................................... 9 CREM AS: Innovative Performance-Based Contracts ...................................................... 10 Delegating Work to the Provinces and Strengthening Their Highway Agencies ............. 11 A Strengthened Administration of the National Road System........................................... 12 Study of the Buenos Aires - Colonia (Uruguay) Bridge................................................... 13 5. Ratings .................................................................................................................................... 13 Outcome, Institutional D evelopment, and Sustainability.................................................. 13 Bank Performance ............................................................................................................ 14 Borrower Performance ..................................................................................................... 14 6. Lesson .................................................................................................................................... 14 Annex A. Basic DataSheet .......................................................................................................... 17 This report was prepared by Heman Levy (Task Manager) and reviewed by Antti Talvitie. William B. Hurlbut edited the report. Romayne Pereira provided administrative support. iii Principal Ratings ICR ES PAR Outcome Highly Satisfactory Highly Satisfactory Highly Satisfactory Sustainability Likely Likely Likely Institutional Development Substantial Substantial High Bank Performance Satisfactory Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory Highly Satisfactory Key Staff Responsible Task Manager Division Chief Country Director Appraisal Raul Auzmendi Alfonso Sanchez Ping-Cheung Loh Completion Gerard Liautaud Danny Leipziger Myrna Alexander V Preface This is a Performance Audit Report (PAR) on the Argentina Road Maintenance and Rehabilitation Sector Project (Loan 361 1-AR), for which the World Bank approved a loan for $340 million on June 3, 1993. The loan closed on schedule on June 30, 2000. An undisbursed balance of $11 million was canceled. This report is based on reviews of the Implementation Completion Report (ICR), the Staff Appraisal Report (SAR), loan documents, project files, transcripts of Board proceedings, and other Bank documents, and on discussions with Bank staff. An OED mission visited Argentina in March 2001, inspected the project-financed works, and reviewed the project with officials of the National Highway Directorate. Their kind assistance is gratefully acknowledged. The PAR confirms that the project was a highly successful operation and focuses on the outcome and the lessons that can be derived from the project. In particular, the PAR expands over the ICR on the following areas: (i) policies relating to the financing of recurrent expenditures such as the routine maintenance of roads; (ii) the sustainability of the road maintenance contracts comprising a mix of investment and recurrent costs, and (iii) the project's cost-sharing arrangements. Following standard procedures, copies of the PAR were sent to the Argentinean authorities for comments. No comments were received. 1 1. Background 1.1 Since the launch in 1991 of the Convertibility Plan, the Argentine economy has been undergoing a massive transformation. Sweeping reforms altered the monetary system, improved fiscal and tax policies, liberalized trade, and reformed the public sector, including a rapid privatization program. The reforms were intended to reverse a pattern of poor economic management and volatility the country had suffered for the previous 25 years. 1.2 In the transport sector, Argentina initiated massive privatization of its national railway system, suburban rail in Buenos Aires, and the country's ports. At the same time, it started a program to involve the private sector in the management of the national road system through a dual system of concessions, one based on toll revenues and another based on budget resources. 1.3 Argentina suffered a serious external shock when the project under review was mid-way through implementation. The shock, known as the "Tequila effect," came in late 1994/95 after the devaluation of the Mexican peso. As a result, the economy shrank by 2.8 percent in 1995. A second shock came in 1998/99, toward the end of the project, as a result of the combined effect of the downturn in East Asia, the devaluation of the Russian ruble and the Brazilian real, and the deterioration of commodity prices. 1.4 Decentralization and improved provincial governance has long been key government objectives. In the highway sector, however, the Bank had not addressed these objectives in any significant measure. Until preparation of the Road Maintenance and Rehabilitation Sector Project started in 1991, the Bank had made five loans for highway projects in Argentina. It was only with the Highway Sector Project (Loan 2296, January 1984, $91 million after cancellation), which had closed in 1990, that the Bank had started to focus on the relationship between the federal highway department (DNV) and the provincial highway departments (DPVs). Attention to decentralization also became a goal of the project under review. The Provincial Roads Project (Loan 4093-AR), currently underway, supplemented those efforts by focusing on the strengthening of the DPVs and improving the provincial highway networks. 1.5 Current highway operations. In addition to the provincial highway project, the Bank currently has underway the National Highways Rehabilitation and Maintenance Project (Loan 4295 -AR), approved in FY98. This project essentially continues the support to DNV and its multiple outsourcing methods started under the Road Maintenance and Rehabilitation Sector Project. 2. Objectives, Preparation, and Design 2.1 The Road Maintenance and Rehabilitation Sector Project focused on maintenance and rehabilitation of the national road network and on assisting central and provincial highway agencies to modernize and reform in line with government policies launched in the early 1990s to decentralize and streamline the state sector. 1. This chapter draws from the Bank's Country Assistance Strategy (CAS) for Argentina, dated September 8, 2000. 2 Table 2.1. Project Objectives and Components Objectives * Ensure adequate maintenance and rehabilitation of the National Road Network * Assist with the quality and financial supervision of maintenance executed by concessionaires * Support institutional reform of the national and provincial highway agencies to respond to government's decentralization policies * Assist with achieving an optimal balance between delegation to provinces and other ways to execute maintenance * Support the creation within the highway agency of an environmental unit, to prepare and monitor compliance with Environmental Assessment Components * A four-year tranche (mid 1993- mid1997) of DNV's road maintenance and rehabilitation program for the network not concessioned to private operators * Emergency works to repair damage to roads and bridges caused by flooding in 1992 * Consulting services for engineering and supervision of project-financed works * A program of technical assistance and training to help DNV monitor the project, introduce modern management and information systems, provide assistance to the DPVs, and carry out specific studies * Office, communication and special-purpose field equipment (traffic counting, vehicle weight controls, road condition rating) 2.2 The objectives were well selected and highly relevant to government aims, and the components adequately supported the objectives. Underfunding for road maintenance in the late 1980s, had caused some 55-60 percent the network to be in fair or bad condition. As a result, and in line with economy-wide privatization policies prevailing at the time, the government in 1991 decided to privatize operations of key national roads through concessions to private operators collecting tolls from road users. By the time of project preparation, about 9,000 kilometers or one quarter of the national road network had been concessioned. The concessions (i) covered practically all the high-traffic roads where operating costs could be recovered through tolls, and (ii) had to meet strict maintenance standards. Three quarters of the network remained under the direct responsibility of the DNV. Thus, the project's focus on maintenance and on reforms appeared appropriate for the needs of the highway system and in consonance with macroeconomic directions. 2.3 Integrated view of maintenance. The project covered periodic maintenance, road rehabilitation, and routine maintenance. The latter amounted to a third of total project cost. Since poor routine maintenance can cause significant road damage, and lead eventually to the need for rehabilitation or reconstruction, the project was right in taking a comprehensive view of the road system's maintenance requirements and in financing routine maintenance. This was the first highway project in Argentina that financed routine maintenance. Government highway officials that were involved with the project from its early stages interviewed during preparation of this audit claim that persuading the Bank to include routine maintenance in the project required much discussion until an agreement was reached. 2.4 Financing of routine maintenance. Project files do not contain evidence to support, or contradict, the claim that the Bank initially resisted the inclusion of routine maintenance under the project. The files do show the Bank's approach to cost sharing for this component, which was reflected in the loan agreement. The loan states that Bank funds would be applied to routine maintenance on a declining cost-sharing formula, starting at 60 percent until $45 million are disbursed, and gradually decreasing to 20 percent until $73 million, and 0 thereafter. This 3 principle of declining cost sharing for recurrent expenditures is embedded in Bank policies (OMS1.21)2 The assumption is that countries should gradually be able to take over with budget funds the financing of such expenditures. 2.5 The audit finds the declining cost-sharing for routine maintenance under the loan to be questionable. While the Bank generally demands that incremental, recurrent expenditures be financed on a declining basis (OMS 1.21), the requirement is flexible. The auditor questions whether a declining cost-sharing for a highway project, applied within the timeframe of a single project, is sensible. First, in the case of highways, inadequate funding and poor quality of recurrent maintenance eventually results in the need for high-cost road reconstruction. Second, to assume that a government will be able to increase its budgetary resources over the life of a project would be highly optimistic in most cases. In the project under review, the creation during project implementation of a new approach for outsourcing maintenance, consisting of contracts including routine maintenance and rehabilitation, cut across the division between routine and other kinds of maintenance which, as discussed later, in practice sidestepped the issue of declining cost-sharing. 2.6 Good project preparation. Preparation for the provincial decentralization program, as well as for the civil works and the rest of the project, was very thorough. The DNV prepared' a detailed program for the transfer of responsibilities, starting with the pilot provinces, to the provincial highway departments, including a specific list of actions required and their sequence, model legal and contractual documentation, resources required and criteria for pricing of works. The DNV identified tasks, such rating of pavement condition, management of federal funds, and periodic control of bridge and other structures that could not be decentralized and would need to remain under DNV's responsibility. 2.7 Preparation of the project, especially of its institutional objectives, was helped by the DNV's creation of a special department (Gerencia de Reconversion Vial) prior to appraisal that aimed to manage the key reforms already launched or envisaged by the DNV: concession of 9,000 kilometers of roadway to private operators (a scheme fitting with national economic policies toward privatization in all sectors), transfer to the provinces of operational responsibility for part of the national road network, and preparation of a new structure for the DNV. 2.8 Performance Indicators. During preparation and appraisal, project performance indicators were identified. The key outcome indicator was road condition. A number of implementation indicators were also identified, such as actual and planned values for costs, disbursements, physical works, and amounts committed for the technical assistance program. Argentina's long-standing tradition of conducting reliable annual surveys of road condition made it possible to use this indicator, which was substantially more sophisticated than road condition indicators used elsewhere in Latin America.' When the project started, the road condition 2. The following are relevant sections of the Operational Manual Statement 1.21..."it [the Bank] finances projects intended to overcome a backlog of deferred maintenance.. .A maintenance project is essentially a project to overcome a backlog of deferred maintenance so that it can be regarded as a kind of capital expenditure, i.e., a one-time investment to bring the infrastructure back to full productivity.. .For all these reasons, Bank financing of maintenance projects is not regarded as financing recurrent costs in the same sense as is described above.. desirability of disbursing for incremental recurrent expenditures on a declining percentage basis... the proportion of incremental recurrent expenditures covered by loan proceeds should not normally exceed 25% in the final year." 3. Letter Administrador General, DNV to the Bank, 20 Oct. 92. 4. The most used indicator of road condition is surface roughness, measured by the International Roughness Indicator (IRI). The Argentina index uses the IRI, but adds other parameters such as deflection, patches, rutting, and cracks. DNV's central office in Buenos Aires has the equipment to measure the IRI, while the rest of the parameters are measured by DNV's regional offices. 4 indicator was already was being used as a key performance target in the toll road concessions. Determining indicators for the institutional development program proved more challenging, and less progress was achieved. 2.9 The project's aim to assist with the supervision of the concessioned network is questionable. The concessions were a new mode of road operations in Argentina, and involved significant financial, economic and technical issues. However, the quality of supervision was just one of the issues. At the same time, DNV's unit in charge of concessions was well financed, including funding from the concessions themselves, and therefore it had the resources necessary to review the issues in detail. 2.10 Sector Loan Approach. The design of the project as a sector loan was appropriate as it provided flexibility during execution. At the same time, Argentina had enough experience with Bank projects, and endowed with qualified staff to carry out technical and economic analysis of subprojects, while capacity for conducting environmental assessment was being developed up front and was expected to be fully operational by early project implementation. 2.11 A Large Technical Assistance Component. The project included a fairly large component for technical assistance and training. In what can be described as an unusual situation, the large program was requested by the government and initially resisted by the Bank as being too large. In view of the wide-ranging institutional objectives, a large technical assistance program was relevant. 3. Implementation 3.1 The project was implemented within the original schedule and with costs 19 percent lower than the original estimate. The project was amended three times during implementation, although the project objectives were not changed. The first two amendments mainly redistributed loan funds among disbursement categories, and the third one introduced a new maintenance contract concept (comprising a mix of rehabilitation and routine maintenance) that required a new disbursement category. 3.2 Costs were below appraisal estimates, mainly thanks to successful competitive contracting that led to reasonable prices, even though the number of bids under the new category (para. 2.5) was large and the bids were tendered within a relatively short period. This probably stretched the capacity of contractors and may have pushed prices upward. 3.3 The project was successfully implemented despite shortages of counterpart funding during periods of macroeconomic crisis, especially during 1995, when Argentina's economy was seriously affected by the Tequila crisis, and which caused delays at various times during implementation. 3.4 A Project Coordination Unit (PCU) was created early during project preparation, an uncommon feature since in most projects such units are created close to or after loan approval. The PCU proved helpful during preparation as well as during implementation. It was especially effective in preparing for implementation by having bid documentation early and being able to call for bids soon after project effectiveness. By September 1993, 4 months after project approval and before loan effectiveness (declared in December 1993) 80 out of the 120 bids comprising the 1993 program for road rehabilitation and periodic maintenance had already been invited, and the 5 remaining 40 were ready for invitation. As a result, 20 contracts were already under execution, including about 80 percent of the emergency works.' The fast pace continued: during January 1994, the Bank approved more than 50 contracts for a total of $70 million. 3.5 Also helpful was the project's retroactive financing, which made it possible to contract PCU staff and start flood emergency works and other activities. The launching of routine maintenance works was slower. Reaching agreement between DNV and the provinces proved more difficult than anticipated 3.6 There were significant delays in launching the institutional development program. Two years after loan approval, only one of the major studies, the feasibility study of the Buenos Aires- Colonia bridge, promoted by a Bi-National Commission of Uruguay and Argentina, was underway. However, later in project implementation the PCU managed, under sustained prodding by the Bank, to launch the institutional development program essentially as originally conceived. 3.7 Environmental Manual and Management. A notable accomplishment of the project was the rapid preparation and implementation of an Environmental Manual. The SAR included a tight schedule for DNV to prepare, discuss, obtain government agreement and implement the manual. Completion of the manual, for which an outline had been agreed at appraisal, was the only project-specific condition for declaring the project effective.' The manual was completed on time. DNV started to enforce its use even in contracts already signed, for which contract amendments were made.' Project-financed attendance of DNV personnel to environment seminars helped to further strengthen DNV's capabilities in this area. The manual also was made compulsory for the road concessions operating parts of the federal road network. Use of the manual has gone beyond the federal system as some of the provinces are adopting it for their own road development programs. 3.8 At the same time, DNV upgraded the importance of environmental work. An environmental division, under the Planning Department, was created and substituted for the environmental unit under the Feasibility Studies division. The importance of the environment was reflected in DNV's organization structure. In 1999, a Working Group was created to coordinate environmental actions at the regional level in Argentina's six regions. DNV reports that contractors under the ongoing CREMA2 program are required to comply with all sections of the manual. 3.9 Delegation to the Provinces. At appraisal, La Pampa and Santa Cruz provinces had been selected as pilot provinces to start testing delegation of routine maintenance of the national road network to the provinces. These provinces had been selected based on their interest in participating in the pilots. Yet, the negotiation process between DNV and these provinces was protracted, suggesting that the difficulties in the negotiation process between DNV and the provinces had been underestimated. Still, by end 1993, DNV had signed and the Bank had approved, under DNV's Highway Management "Reconversion" Program, agreements with the two pilot provinces for 2,783 kilometers, or more than half the original total target. By a year later, thanks to the experience gained in negotiating and managing the pilot contracts, and despite 5. Report of the first Supervision Mission, dated Sept 16, 1993. 6. The other effectiveness condition was a standard requirement for the Borrower to sign a Subsidiary agreement with the implementing agency satisfactory to the Bank. 7. Letter by Project Coordinator to the Bank in December 1993. Project files also show that Bank environment staff were instrumental in the speedy preparation of the manual, by making available to the DNV relevant environmental manuals from other Spanish-speaking countries, and guiding DNV staff in the process thus saving time and avoiding the need to recruit a consultant for this task. 6 the problems posed by wide-ranging capacities among the provinces, routine maintenance contracts were underway in 14 provinces. 3.10 Several factors influenced the selection of these 14 provinces, notably: * The provinces' interest in the program. * The capacity of the DPVs for maintenance management. * The availability of DNV personnel in the provinces-as a result of DNV's retrenchment process, DNV had lost personnel located in the provinces. Preference for selection was given to the provinces where DNV's personnel losses had been greatest. * Existence of earth and gravel roads in the national network located in the province, since DNV intended to launch the delegation process with these lower-class roads, while retaining paved roads under its direct control. 3.11 The CREMAs. The most important milestone during project implementation was the introduction by DNV of a new type performance-based maintenance contract, the CREMA,' comprising both rehabilitation and routine maintenance. The ICR claims that the CREMAs were triggered or inspired by the Tequila effect. The audit could not verify this assertion. On the other hand, it is highly probable that the Tequila effect influenced contractors' and their banks' reaction to the CREMA concept, as discussed below. More likely, it may be that the DNV devised the CREMA concept in reaction to: (i) cumulative delays in project implementation, stemming from problems with counterpart funds that became critical with the onset of the macroeconomic crisis in 1995, and which affected all Bank projects in Argentina, and (ii) DNV's conclusion by that network was in poor shape and could not be improved fast enough without infusion of fresh funds. 3.12 Experience accumulated by DNV with performance-based contracts likely was instrumental in the conception of the CREMAs. The experience with "kilometer-month" contracts was particularly relevant. These were performance-based contracts, covering routine maintenance exclusively, for a duration of four years, and with equal monthly payments to the contractors (except that penalties for non-compliance standards are deducted). 3.13 Thus, DNV conceived the CREMAs on the basis of three key assumptions: * Highway maintenance contractors and their banks would need to make financial commitments beyond what is required in conventional maintenance contracts, in line with the government drive for a stronger role of the private sector in the management and finance of infrastructure. * The World Bank would be asked to support the CREMAs at a higher rate than stipulated in the loan agreement for the various project components. A letter from the government' argued that the CREMAs would allow better use of project funds and requested the Bank (i) to restructure and create a new category in the loan to allow disbursements for the CREMA contracts (the loan documents had set routine maintenance) and (ii) to allow a higher Bank cost-sharing percentage for this category. * CREMA contracts would be performance-based. 8. An acronym that happens to work both in Spanish (Contrato de Recuperacion y Mantenimiento) and English (Contract for Rehabilitation and Maintenance). 9. Letter from Secretary of Public Works dated March 16, 1995. 7 3.14 Loan Amendment. The Bank promptly agreed to amend the loan. An internal Bank memoo supported a disbursement percentage of 75 percent for the CREMAs. This change would bring overall Bank disbursement to 54 percent of total cost, compared to 45 at appraisal. In supporting this change, the memo noted that this would be the first project in which the Bank would be increasing its cost sharing in response to the current "stringencies," that is, the fiscal crisis facing Argentina at the time. The memo also said that the increased disbursement percentage for this project would be consistent with existing cost sharing guidelines for the country as a whole. The audit finds that under current Bank policy (which was also in effect during project implementation), the cost-sharing values for the project overall as well as for the CREMA component are legitimate." 3.15 The amendment also increased the proportion of civil works to 96 percent (from 89 percent in the SAR) and reduced the technical assistance component consulting and studies to 3 percent (from 1 0)-while leaving equipment at 1 percent. The reduction in the technical assistance component had less impact than might have been expected because the CREMA contracts included detailed engineering and supervision, which represented about half of the project funds for technical assistance (see para 4.4 and its footnote). 3.16 The CREMA program launched under the project consisted of 61 subnetworks. Of these, 1 contract was eliminated and 1 was canceled after signature, leaving 59 contracts actually implemented. Fifty of these contracts will expire between April 2002 and 2003. An example of a CREMA subnetwork is in Box 3.1. 3.17 While the CREMA have proved successful (and the CREMA model was soon adopted by several other Latin American countries), experience shows that compliance with performance standards could be improved by ensuring that key materials are available when needed. The CREMA1 (under loan 3611-AR) was based exclusively on performance. The next series of CREMAs will, in addition, include input requirements such as availability of a small asphalt plant to avoid the recurrence of problems with availability of asphalt. 3.18 High cost-sharing ratio. The new cost-sharing ratio appeared unusually high, especially for a middle-income country.12 Two related reasons justify the Bank's decision: * The CREMA was an original concept that, as designed by DNV, required contractors to prefinance a significant part of the contract. To attract contractors to bid under this financially risky scheme (considering traditional mistrust of treasuries' reliability), it was necessary to provide a strong assurance that funds would be available for payments to be made on time. * This assurance was especially important at the time the CREMAS were launched, as Argentina's economy was being affected by the Tequila effect. Banks financing the contractors were concerned about their clients' ability to repay." 10. Memo Faiz to Nankani, January 16, 1996. 11. The Bank's Operational Policy OP 6.30 (September 1993) says that "Bank cost-sharing limits are applicable not to individual projects but to the Bank's overall lending program for a country... .the country limit is applied to a rolling three-year investment lending program.... for individual projects, the Bank usually expects the borrower to demonstrate commitment to the project by making a 10 percent minimum contribution to project cost." 12. Under Bank rules, the level of cost-sharing decreases with the country's GDP per capita. 13. The April 1997 CAS notes that the Argentine economy contracted by 4.6 of GDP in 1995 and that in the first four months of that year there was a 17.6 percent loss in bank deposits. 8 3.19 The audit finds that, under the circumstances, the high disbursement ratio was justified. However, the Bank would do well to review its cost-sharing policies and their application, especially to ensure that (i) similar policies, though not necessarily similar cost-sharing ratios, are applied consistently across sectors and countries, (ii) the high cost sharing ratio is not financing a high amount of local costs, thereby imposing an unnecessary increase in the borrower's foreign debt. 3.20 CREMAs and Budget Allocation for Road Maintenance. Experience during implementation showed that even at time of fiscal constraints, the budget process gave priority (but not absolute assurances) to CREMA contracts in the allocation of funds Possible reasons for this may be that the government wanted to show that it respects contracts, give confidence to the private sector, and prevent an early failure of the CREMA approach. In practice, it means that such contracts may become a useful mechanism to ensure some minimum levels of maintenance funding. This can be of significant benefit for the upkeep of the road network, since from 1990, all road-related taxes (which included 10 percent of fuel taxes, various percentages taxes on tires, lubricants, vehicle registration, fees, and sales) assigned to the DNV have gradually been eliminated. 3.21 Reduction in Funding for Technical Assistance. Project amendments successively reduced project funding for studies, consultants, and training from $29.7 million at appraisal to $12.3 million. The cutting of the TA component was largely justified because of the large reduction in engineering and supervision requirements (amounting to more than half of the component) resulting from the CREMAs. Box 3.1 Example of a CREMA - A Subnetwork in the Entre Rios Province National Network and CREMAs Entre Rios Province * Maintenance ofNational Road Network in Entre Rios o Total length 1400 km o CREMAs (3 contracts) 550 o Delegated to Province 100 o Force Account 350 o Concession (Toll) 400 * CREMAS o Routine maintenance component done by local subcontractors o Average value of contract (4 years): $10 million/contract o Requirements * CREMAl: Performance targets * CREMA2: Performance targets and key equipment * CREMA Subnetwork 510 (Entre Rios Province) o Total distance 172 km o Rehabilitation 90 km o Routine Maintenance 82 km 9 4. Outcome Road Condition 4.1 Road condition is the most important single indicator of outcome for the project's physical investments. As shown in Table 4.1, the condition of the national road network (both concessioned and non-concessioned) improved dramatically over the project period. The audit agrees with the ICR that the project target of 7.0 for the condition index was too demanding, and probably unjustified for all roads except for the major, toll-concession, expressways. The outcome is even more valuable considering that Argentina's Road Condition Index is more rigorous than the widely used International Roughness Index (IRI). The routine maintenance subcomponent was instrumental in achieving these results. Table 4.1. Changes in the Condition of the National Road Network Non-Concessioned Network Toll-Concession Network Percentage of Road in Average Condition Index Percentage of Roads in Poor Condition (SAR target.7.0) Poor Condition (Based on Roughness)' At Appraisal 35 5.5 25 At Project Completion 5-10 6.6 Close to 0 a. International Roughness Index (IRI) greater than 5. Multiple Outsourcing Methods 4.2 By delegating responsibilities to the provinces and the CREMAs, DNV now has 6 different instruments to manage the whole national network, with use depending on the nature and value of the works to be done, and on traffic levels (Table 4.2) This menu of options is practical and allows DNV to cater to specific conditions. Of the instruments, five are by contract and one is force account. During the project period, works by force account were reduced from about 29,000 kilometers at the start of the project to about 6,000 kilometers (mostly gravel and earth roads) at completion The most important, highly trafficked expressways, the toll- concessions, although part of the national road network, are regulated by an autonomous agency recently created by the merging of two agencies: one dealing with the Buenos Aires toll concessions and another dealing with the toll concessions elsewhere in the country. This multiple outsourcing system, which has evolved over the past 10 years (5 for the CREMAs), has proved an effective way to manage the national road network. 14. Argentina's Road Condition Index includes the IRI in addition to other indicators such as deflection, potholes and cracks. 10 Table 4.2. Maintenance of National Road Network: Management Instrument, Unit Costs, and Traffic Unit Cost Routine Highway Management Instrument (US$/Kmlyear) Maintenance Distribution by Management instrument '000 Percent Traffic, annual average (vehicles per day) km 0-300vpd To 1,500vpd To 2,500vpd Over 3,000 vpd Over $25,000 Concessioned TOLL CONCESSION Network Contract: all maintenance & rehabilitation Up to $25,000 1.0 3.4 NON-TOLL CONCESSION Contract:: All maintenance & rehabilitation Up to $15,000 11.7 40.3 CREMA Contract Routine maintenance& rehabilitation Up to $3,600 ROUTINE MAINTENANCE ONLY -Contract (km- 3.8 13.1 month) -Provinces 6.2 21.4 -DNV Force 6.3 1 21.7 Account CREMAS: Innovative Performance-Based Contracts 4.3 The innovative CREMA contracts and their successful implementation is a significant project outcome, one that had not been anticipated at project appraisal. While Argentina had from the beginning of the 1990s launched performance-based contracts for highway management (such as the contracts for the concessioned network and those for the kilometer-month regime), the CREMA concept was completely new. Further, as shown in Table 4.2, the CREMAs filled a niche with specific traffic levels and unit costs. 4.4 Did the CREMAs generate cost savings? The ICR states that rehabilitation costs under CREMA are $66,000 per kilometer versus $59,000 per kilometer under conventional contracting. This appears to be a surprisingly negative result. However, the comparison is more complicated and must take into account a number of factors, which likely will reverse the ICR's assertion: * CREMA costs include detailed engineering and supervision" costs, which under conventional contracting are borne by the highway agency. * CREMA costs include financial charges by the contractor because the contractor is only paid 50 percent of total contract value by the end of the first year, when the expensive 15.The CREMA contractor carries out the expensive laboratory works, and reports all findings to DNV. Therefore, DNV supervision costs are reduced to a minimum. 11 rehabilitation works are completed. The balance is paid in equal monthly installments over the remaining four years of the contract. This payment structure means that the CREMA contractor pre-finances part of the works. Such financial costs would be borne by the highway agency under conventional contracting. * CREMA contracts, because of high penalties for non-compliance with performance targets (which include the Road Condition Index as well as individual targets for other variables such as rutting and cracks) practically guarantee that by the end of the contract, all the roads in the CREMA subnetwork would be in good shape, therefore reducing the future needs of road rehabilitation compared to works done under conventional contracting." 4.5 CREMAs so far have demonstrated a strong resilience in cycles of budgetary crisis, and the budgetary process has given these contracts a higher priority than other expenditures. This makes it attractive for DNV to extend the part of the national road network maintained with CREMAs. However, claims (in the ICR as well as by officials interviewed for this audit) that CREMA contracts are immune to budget problems are exaggerated: during the past year, CREMA contractors working under the ongoing project have seen payments for their work delayed by 12 to 18 months. 4.6 The CREMA contract, with its mix of capital-intensive, high-value rehabilitation and labor-intensive, low-cost routine maintenance, has encouraged large contractors to subcontract routine maintenance. This approach is promoting a healthy development of small, local subcontractors, while providing DNV an adequate guarantee of performance, since the contractor retains overall responsibility for the whole CREMA road subnetwork. Delegating Work to the Provinces and Strengthening Their Highway Agencies 4.7 The DNV contracts with 14 provinces for the routine maintenance of about 6,000 kilometers of the national network represent strong progress in the process of delegating work to the provinces. DNV reports that the performance of the two pilot provinces has been excellent, while that of the other provinces has been satisfactory. The participating provinces have clear comparative advantages-they are taking over maintenance of mostly earth and gravel roads, for which the DPVs have qualified personnel and expertise, and which DNV has lost as a result of its staff retrenchment. In provinces where the national network does not have such type of roads, for example, in Corrientes, DNV has delegated the maintenance of paved roads. 4.8 The delegated roads, mostly earth and gravel roads, are those likely to carry mainly local traffic, and to serve low-income populations in relatively isolated areas. In principle, provincial governments should be attuned to the needs of those populations, and therefore there should be a built-in incentive for the provincial highway departments to maintain the roads in good condition. However, international experience shows that local governments are not necessarily more sensitive to vulnerable populations than is the central government. Thus, it is important for the DNV to monitor closely the performance of the DPVs. 4.9 Unlike past attempts to delegate, which failed because transfer of maintenance responsibility was not accompanied by a corresponding transfer of funds, the process under the project has proved successful because budgetary resources were transferred in accordance with the agreements signed between DNV and the provinces. In these transfers, the budget allocation 16. The CREMA contracts also include a requirement for the rehabilitated roads at the end of the contract to have a minimum, pre-determined pavement thickness. This ensures that the road's bearing capacity on completion of the contract is adequate and that roughness targets are not met by milling the road surface 12 is routed via the DNV, with the provinces acting as agents for DNV. In the future, it may be possible to make the allocations directly to the provinces. This, because the fiduciary aspects of the federal-provincial delegation of works and transfer of funds has been improved as several provinces have submitted legislation to implement at the provincial level financial management and personnel control systems in place at the federal level." 4.10 DPVs have reduced personnel in recent years" but with a few exceptions they probably remain overstaffed. In addition, they own highway equipment. For some time to come, therefore, provinces are likely to carry out the delegated work for the maintenance of national roads through force account rather than by contract. 4.11 Is there a risk that delegation of part of the network could preempt an expansion of the road concessioning program? The risk appears to be small because the delegated roads generally carry low traffic, and therefore are not candidates for a toll collection operation that would make them attractive for private operators. The provinces are likely to be interested in concessioning the delegated roads once traffic levels reach sufficiently high levels. Several provinces have privatized other revenue-collection infrastructure operations. For example, under the Bank's Water Supply II Project (Loan 3281-AR), five out seven participating provinces concessioned water and sanitation services to private operators.19 4.12 Experience under the project demonstrated that some functions should not be delegated. Some tasks initially conceived as delegable to the provinces appear to be better managed centrally, such as traffic counts, axle weight controls, and measuring road condition. A Strengthened Administration of the National Road System 4.13 The project had a significant impact on strengthening DNV.20 The project promoted or directly supported a number of improvements, including: * Extension of outsourcing, comprising both a greater coverage of the national road network and a diversification of instruments * Establishment of a comprehensive pavement management system (including use of the HDM model under budget constraints, calibrated for individual regions and adjusted to road deterioration conditions in Argentina) * Enabling DNVs to enforce compliance with environmental safeguards, through establishment of an environmental division, training of its staff, and preparation of an Environmental Assessment Manual * Supporting DNV restructuring to become a more effective, leaner agency (the agency staff was reduced by one-third between appraisal and completion) * Preparing a reformulation of the national road network that will be essential for the extension of the road decentralization program * Preparing plans for installing a modem traffic information and updated axle-load control systems. 17. OED ICR-Review, Evaluation Summary, Public Sector Management Adjustment Loan (L3836). 18. For example, the OED Evaluation Summary of the Public Sector Management Adjustment Loan (Loan 3836) reports that during the project period the participating provinces had elliminated 55,000 redundant personnel, well in excess of the 20,000 targeted. 19. OED Evaluation Summary. 20. The ICR has an excellent description that is summarized here. 13 Study of the Buenos Aires - Colonia (Uruguay) Bridge 4.14 The study of the international bridge, financed jointly by the loan under review and a parallel loan in Uruguay (Loan 3021 -UR), produced a report that covers technical, economic, financial, environmental, and BOT documentation and is considered to be an excellent base for the eventual launch of a BOT process for private finance, construction and operation of the bridge. As noted in the audit of the Uruguay project, the Uruguayan parliament has given the government the green light to proceed to launch the process once Argentina has secured the same approvals. Discussions are underway in Argentina, but the degree of support for the bridge is not as strong as in Uruguay, and it is unlikely that parliamentary approval would happen soon. 5. Ratings Outcome, Institutional Development, and Sustainability" 5.1 Outcome. Judged against the standard dimensions of outcome used by OED, the project performed very well. Project objectives were highly relevant at project appraisal, and remained so during implementation; efficacy was high, since the project's physical targets were exceeded (Table 5.1), the overly ambitious road condition targets were practically achieved (Table 4.1), and all the institutional objectives were achieved; and, efficiency was high, since economic rate of return for the project-financed investments is estimated at 44 percent (30 percent at appraisal) for the rehabilitation component, and at 60 percent for the CREMA contracts. Overall, outcome is rated highly satisfactory. Table 5.1. Physical Targets Achieved (km) Targets Actual Achievement of target Pavement and rehabilitation 7,700a 9,440 123% (km) Routine maintenance na 3,845 Na Contract 6,186 Provinces (14) 11,667 CREMAs 21,698 a. This target is not explicit in the SAR but is included in the 1993-1997 plan. 5.2 Institutional development impact. As noted in the preceding chapter, the project promoted or directly supported many institutional improvements. These ranged from expansion in the coverage and diversification of methods for the outsourcing for execution of maintenance operations; to a successful launch of delegation of works to the provinces; to the creation of the innovative and successful CREMA contracts; to a significant restructuring and strengthening of DNV, especially its conversion from executing maintenance to managing outsourcing and its capacity to institute and enforce compliance with strict environmental regulations for road works. Institutional development impact is rated high. 5.3 Sustainability. The sustainability of the investments is practically guaranteed, since the CREMA contracts are for 5 years, not much shorter than the economic life of the rehabilitation investment. At the same time, institutional development improvements are unlikely to be 21. Comparison with ICR ratings is done only when the audit rating is different. 14 reversed. Over the longer term, however, sustainability of the CREMA concept, is uncertain. Given Argentina's cycles of fiscal crises, will contractors be interested in bidding in the future, when the Bank's share of financing for the contracts is reduced, and eventually terminated? The ongoing Bank highway project (loan 4093-AR) is financing a new series of CREMA contracts with a disbursement ratio of 60 percent (rather than 75 percent under Loan 3611). Bidding under the ongoing project has proceeded satisfactorily. However, only a few contracts have so far been let, and it is too early to draw any conclusion. On balance, sustainability is rated likely (rather than highly likely, OED's highest possible rating). Bank Performance 5.4 The Bank did an excellent job during project preparation, identification, and appraisal. It set strict requirements to ensure that DNV had the capacity and the programs in place to support the project's physical and institutional objectives. Assistance with the preparation of the environmental manual was excellent, and allowed DNV to comply with the related effectiveness condition. Bank performance during supervision was more erratic, however. Three different Bank teams were involved with the project at various times. DNV's view is that communications with one of the teams was unnecessarily difficult. Supervision reports, while generally good, failed to disclose the cancellation of two studies related to supervision of the concessioned network. On balance, Bank performance is rated satisfactory. Borrower Performance 5.5 Borrower performance was, with minor exceptions, good throughout. During preparation, it promptly prepared and submitted to the Bank programs of action, list of measures taken and to be taken, and managed to launch the bid process early enough to allow an intensive program of works during the project's initial year. DNV undertook difficult negotiations with 14 provinces, and carried these out effectively. DNV promptly implemented the environmental manual. At completion, the PCU prepared a detailed final report. DNV's most noteworthy action was, however, the design and implementation of the innovative CREMA contracts, that allowed the project to catch up with implementation delays, mitigate if not fully eliminate problems with counterpart funding, and eventually surpass SAR targets. DNV's performance was not always fully effective, particularly as it delayed launching the technical assistance program and on occasion it did not provide support contract modifications sent to the Bank for approval with necessary data and analysis. Yet, the borrower's competence was instrumental in the excellent project outcome and its timely completion, and its performance is rated highly satisfactory (versus satisfactory in the ICR). 6. Lessons 6.1 The following lessons emerge from this project: * Delegating maintenance responsibility from the federal to the provincial governments under well-conceived, performance-based contracts has some clear benefits: it helps strengthen the provincial agencies, shifts the federal role toward management and away from execution, and brings service delivery closer to road users. But long-term success of the delegation system is not assured in all provinces, and the federal agency should closely monitor the process. Argentina has succeeded in achieving the benefits of delegation, but it is too early to judge the long-term sustainability of delegation. Risks to those benefits, such as non- compliance with agreed maintenance standards on the delegated roads, and the entrenchment 15 of force account operations which preempts an eventual privatization of such operations, should be monitored. Experience may show that some of the provinces have the capacity to carry out the works at the standards and costs agreed, while others do not. * Contracts between the government and private operators which involve significant financial risks for the operators appear to be given priority in budgetary allocations designed to seek a broader role for the private sector. This may be a way to secure an adequate and reliable funding mechanisms for highway maintenance. Argentina's CREMA contracts, which require contractor pre-financing and therefore involve significant financial risks, get a higher priority in budgetary allocations than works executed under traditional contracts or by force account. Therefore, the CREMA program appears to bring a higher degree of reliability of budget finance for highway maintenance than other methods for executing maintenance. * A high level of cost-sharing by the Bank in projects in middle-income countries may be necessary to help bring the private sector to new activities in times offiscal crisis. For such activities to be sustainable, however, it is essential to design the intervention in a way that reduces and eventually eliminates dependence on externalfinance. The Bank's financing of 75 percent of the CREMA contracts provided strong assurances to commercial banks and contractors that contract payments would be made by the highway agency and was instrumental in getting contractors to bid for the works. The follow-on project reduced the cost-sharing to 60 percent, with no visible decrease in contractors' interest. In the future, Bank cost-sharing should continue to decline. It appears that when external financing is low or no longer exists contractors need to be enticed to continue bidding for these works, either through a special funding mechanism with local resources or by changing the pre-financing requirement. * In leading reform countries, the Bank's role should be to support reforms, to help devise mechanisms to monitor the short- and long-term benefits ofsuch reforms (and assess the need for adjustments), and to disseminate learning from these experiences to other member countries. Argentina's innovative reforms in the management of the highway system (as well as in other sectors) have proved mostly successful over a period of years. Those reforms need continued monitored to assess their long-term efficiency (and equity). Several countries in the region, most with Bank assistance, have adapted and replicated those reforms, which makes continued monitoring of the outcome essential to the future of transportation in the region. 17 Annex A. Basic Data Sheet ROAD MAINTENANCE AND REHABILITATION SECTOR PROJECT (LOAN 3611-AR) Key Project Data (amounts in US$ million) Appraisal Actual or estimate current estimate Total project costs 756.0 614.6 Loan amount 340 329 Cofinancing - Cancellation - - Date physical components completed 06/30/2000 06/30/2000 Economic rate of return 30% 40% Project Timetable Stage of project cycle Actual/Latest Estimate No. Staff weeks US$(,000) Identification /Preparation 47.7 142 Appraisal/Negotiation 89.4 232.5 Supervision 48.9 143.7 ICR 4.4 12.7 Total 631 530.9 Mission Data Stage of project cycle No of Persons and Speciality (e.g. 2 Performance rating Month! Economists, I FMS, etc) Implementation Development year Status Objectives Comments Count Speciality Identification/Preparation October April 1990 2 ECN, EGR June 1990 2 ECN, EGR August 1990 2 ECN, EGR October 1991 2 ECN, EGR 3 ECN, EGR, INST Appraisal/Negotiation ECN, EGR, ENS December 1991 3 ECN February 1992 1 ECN, EGR July 1992 2 ECN, 2EGR ENS, P October 1992 5 Supervision November 1992 1 EGR March 1993 2 ECN, EGR August 1993 2 ECN,EGR December 1993 4 2ECN, EGR, PRO HS HS March 1994 2 ECN, EGR S HS September 1994 3 ECN, EGR, PRO S HS April 1995 2 ECN, EGR S S November 1995 1 ECN U S June 1996 2 2EGR U S December 1996 2 ECN, EGR S S June 1997 2 EGR, PRO S S April 1998 2 EGR, PRO S S June 1998 2 EGR, INST S S December 1998 1 EGR S S March 1999 1 EGR S S November 1999 1 EGR HS HS March 2000 2 EGR, PRO HS HS ICR ne 2000 1 EGR HS HS