Colombia - Highway and Rural Roads Projects

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 14806 PERFORMANCE AUDIT REPORT COLOMBIA SEVENTH HIGHWAY PROJECT (LOAN 1471-CO) RURAL ROADS PROJECT (LOAN 1966-CO) AND HIGHWAY SECTOR PROJECT (LOAN 2121-CO) JUNE 30, 1995 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents Metric System Abbreviations and Acronyms CPM Central Projects Memorandum CPN Central Projects Note ED Executive Director ERR Economic Rate of Return FINDETER Financiera de Desarrollo Territorial S.A. FNCV National Feeder Roads Fund HD Highway Directorate (in MOPT) HDM Highway Design Model H VII Seventh Highway Project HS I Highway Sector Project HS II Second National Highways Sector Project HS III Third National Roads Sector Project MOPT Ministry of Public Works and Transport MOT Ministry of Transport NHA National Highways Agency OED Operations Evaluation Department OMS Operational Manual Statement PCR Project Completion Report RR I Rural Roads Project RR II Rural Transport Sector Project RR III Second Rural Roads Sector Project SAR Staff Appraisal Report SPW Secretariat of Public Works (in Department) TA Technical Assistance Fiscai Year Government: January 1 - December 31 FOR OFFICIAL USE ONLY THE WORLD BANK Washington, D.C. 20433 U.S.A. Office of Director-General Operations Evaluation June 30, 1995 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Colombia; Seventh Highway Project (Loan 1471-CO); Rural Roads Project (Loan 1966-CO) and Highway Sector Project (Loan 2121-CO) Attached is the Performance Audit Report (PAR) on the Colombia: Seventh Highway Project (Loan 14721-CO, approved in FY77), Rural Roads Project (Loan 1966-CO, approved in FY82) and Highway Sector Project (Loan 2121-CO, approved in FY83), prepared by the Operations Evaluation Department. The objectives of the Seventh Highway Project (H VII) are all related to maintenance improvements: rehabilitation of trunk highways, elimination of truck overloading, improvements in maintenance works efficiency, and upgrading of transport planning. In the Rural Roads Project (RR I), the focus was on rural road construction and rehabilitation with minor attention being paid to maintenance and institution building. The objectives of the Highway Sector Loan (HS I) were comprehensive: strengthening of highway management capacity, including planning, design and execution of civil works, road safety, personnel training, axle load control, and road maintenance; and improvement of sector policies. Eighty percent of physical targets were achieved for H VII, 90 percent for RR I, and 25 percent for HS I. Only the RR I complied satisfactorily with the covenants. Both H VII and HS I had mixed results in improving management and institutions, although the training component was completed satisfactorily. An indigenous innovation in routine maintenance contracting, the creation of "micro- empresas", small labor-based cooperative work organizations, can, however, be counted as one success story of H VII's institutional development. The RR I was successful in constructing and improving rural roads. The Audit rates the projects' outcome as marginally satisfactory except for that of HS I, which is unsatisfactory. Institutional development is rated as negligible and sustainability as uncertain for all projects. Two major lessons were learned from the unsatisfactory experience of HS I: (i) the financing of investment program time-slices, which leaves most of the planning and execution to the Borrower, should only be done when the implementing agency has a proven record in these areas, and (ii) changes in Government and local staff increase the risk that institutional components may not be implemented. In view of the unsatisfactory experience in institutional and policy development in Colombia's road sector, the audit recommends that a comprehensive assessment be undertaken - focusing on identification of highway sector issues, policy and institutional measures and their sequencing, and the role of the Bank, - to help shape the future Bank strategy in the sector. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  FOR OFFICIAL USE ONLY I Contents Preface..... .....3 Basic Data Sheets 5...................................................5 Evaluation Summary .. ............................................... 13 1. Introduction ................................................... 17 2. Project Objectives and Design ...................................... 19 Overview ... ................................................... 19 Seventh Highway Project ........................................... 20 Highway Sector Project ............................................ 20 Rural Roads Project ............................................... 21 3. Project Implementation ........................................... 23 General ...... ................................................. 23 Seventh Highway Project ........................................... 23 Highway Sector Project ............................................ 24 Rural Roads Project ............................................... 25 4. Project Outcomes ............................................... 27 National Roads Sector ............................................. 27 Physical Results . ................................................. 27 Institutional Development ........................................... 28 Transport Policies ................................................ 30 Rural Roads Sector ............................................... 30 5. Determinants of Project Outcomes ................................... 33 6. Project Ratings ................................................. 37 7. Conclusions .................................................... 39 This report was prepared by Mr. Antti Talvitie and Mr. Albert Weckerle (Task Managers) who audited the projects in May 1994. Ms. Maryvonne Mauprivez provided administrative support. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed wihout World Bank authorization.  3 Preface This is a Performance Audit Report (PAR) of three road projects in Colombia: Project Name Loan No. Loan Amount (US$ Million) Approved Disbursed Canceled Seventh Highway Project 1471-CO 90.0 87.8 2.2 Rural Roads Project 1966-CO 33.0 33.0 0.0 Highway Sector Project 2121-CO 152.3 149.6 2.7 Project Completion Reports (PCRs) were published as follows: for Loan 1471-CO, on November 15, 1988 (Report No. 7494); for Loan 1966-CO, on April 28, 1989 (Report No. 7740); and for Loan 212 1-CO, on February 19, 1991 (Report No. 9355). The PCR for Loan 2121-CO was a superbly crafted and incisive document of unusually high quality. The Operations Evaluation Department (OED) prepared this report. To prepare it, OED staff reviewed the President's Reports, Staff Appraisal Reports (SARs), PCRs, legal documents, transcripts of Board proceedings, project correspondence files, Bank documents on other transport projects and other Bank material. In May 1994, an OED mission travelled to the country where it held discussions with officials and representatives of the Ministry of Transport (MOT), Instituto Nacional de Vias (National Highways Agency - NHA), National Planning Department, National Feeder Roads Fund (FNCV), Secretariats of Public Works (SPWs) in Departments, Financiera de Desarrollo Territorial S.A. (FINDETER), road maintenance contractors, and farming communities. OED also reviewed the projects with Bank staff. The PAR had four objectives: to verify, update and expand PCR findings on project outcomes; put the three projects into a larger context, by presenting them as a group and linking them with earlier lending; elaborate on determinants of project outcomes; raise issues about the rationale for continuing Bank support to the national roads sector; and draw lessons for the future. Following standard procedures, copies of the PAR were sent to the Borrower for comments; however, no comments were received.  5 Basic Data Sheet SEVENTH HIGHWAY PROJECT (LOAN 1471-CO) Key Project Data Original Actual Total Project Cost (US$ million) 153.75 187.6 Cost Overrun (%) 22.1 IBRD Loan Amount 90.0 Disbursed 87.8 Cancelled 2.2 Estimated Economic Rate of Return 90.0 27.0 Physical Components Completed 12/30/81 06/30/86 Cumulative Estimated and Actual Disbursements FY78 FY79 FY80 FY81 FY82 FY83 FY84 FY85 FY86 Appraisal 15 43 69 84 90 - - - - Estimate Actual - - 11 42 64 75 85 87 88 Actual or % - - 16 50 71 83 95 97 98 of Estimate Project Dates Original Revision Identification 08/13/71 Negotiations 05/27/77 Board Approval - 06/29/77 Loan Agreement 07/05/77 Effectiveness 10/14/77 11/28/77 Closing Date 10/31/82 12/31/83 Borrower: The Republic of Colombia Executing Agency: The Ministry of Public Works and Transport Follow-on Project: Highway Sector Project (Loan 2121-CO) 6 Staff Inputs (staff weeks) FY72 FY73 FY74 FY75 FY76 FY77 FY78 FY79 FY80 Preappraisal 1.9 1.5 21.6 31.0 7.6 11.2 - - - Appraisal - - - - - 64.7 - - - Negotiations - - - - - 7.3 - - - Supervision - .3 - - - - 21.9 15.5 14.3 Other - - 2.4 .4 2.7 10.4 - - - Total 1.9 1.7 24.1 31.4 10.4 93.7 21.9 15.5 14.3 Staff Input (continued) FY81 FY82 FY83 FY84 FY85 FY86 FY87 FY88 Total Preappraisal - - - - - - - - 74.9 Appraisal - - - - - - - - 64.7 Negotiations - - - - - - - - 7.3 Supervision 4.0 13.9 19.0 12.1 2.3 .8 3.8 2.8 110.7 Other - - - - - - - - 16.0 Total 4.0 13.9 19.0 12.1 2.3 .8 3.8 2.8 273.6 7 Mission Data Month/ No. of No. of Staff Date of Year Persons Weeks Weeks Report Preparation 10/71 1 1 1 12/18/71 Preparation 03/72 4 0.5 2 03/31/72 Preparation 06/72 1 0.5 0.5 07/17/72 Preparation 09/72 1 0.5 0.5 10/11/72 Preparation 02/73 2 1 2 03/30/72 Preparation 05/73 1 1 1 07/03/73 Preparation 06/73 1 1 1 07/27/13 Preparation 07/73 2 0.5 1 09/14/73 Preparation 11/73 3 1 3 01/30/74 Preparation 02/74 1 1 1 03/28/74 Preparation 05/74 3 0.5 1.5 05/20/74 Preappraisal 02/75 2 3 6 04/03/75 Preappraisal 04/75 3 2 6 06/05/75 Preappraisal 09/75 2 3 6 10/07/75 Preappraisal 04/76 1 2 2 05/20/76 Preappraisal 07/76 2 2.5 5 07/26/76 Preappraisal 10/76 1 1 1 11/11/76 Appraisal 10/76 2 3 6 11/19/76 Post-appraisal 02/77 1 1 3 03/11/77 Supervision 10/77 3 1 3 11/02/77 Supervision 04/78 2 3 6 05/04/78 Supervision 10/78 1 0.5 0.5 12/01/78 Supervision 02/79 4 2.5 10 05/11/79 Supervision 05/79 1 0.5 0.5 07/10/79 Supervision 08/79 1 2 2 09/14/79 Supervision 11/79 2 1.5 3 01/16/80 Supervision 12/80 2 0.5 1 01/28/81 Supervision 08/81 2 0.5 1 10/07/81 Supervision 01/82 4 0.5 2 02/19/82 Supervision 06/82 2 0.5 1 07/14/82 Supervision 11/82 4 1.5 6 11/24/82 Supervision 05/83 3 1 3 06/22/83 Supervision 11/83 3 3 9 02/10/84 Supervision 04/84 1 0.5 0.5 04/23/84 Supervision 06/84 3 0.5 1.5 07/12/84 Supervision 11/84 1 0.5 0.5 01/23/85 Supervision 06/85 3 0.6 1.5 06/28/85 Supervision 11/85 2 1 2.0 12/13/85 Supervision 04/86 3 0.5 1.5 04/30/86 Supervision 09/86 1 1 1 09/12/86 8 Basic Data Sheet RURAL ROADS PROJECT (LOAN 1966-CO) Key Project Data Original Actual Total Project Cost (US$ million) 63.0 52.0 Cost Underrun (%) 17.0 Financing (US$ million): IBRD Loan Amount 33.0 33.0 Disbursed 33.0 33.0 Physical Component Completed 12/31/84 6/30/87 Time Overrun (percent) 76.0 Proportion Actually Completed by Original Date 100% 81 Percent Economic Rate of Return 37.0 29.3 Cumulative Estimated and Actual Disbursements FY82 FY83 FY84 FY85 FY86 FY87 Appraisal Estimate 7.7 18.2 27.4 33.0 33.0 33.0 Actual 1.3 6.0 16.0 21.0 30.5 33.0 Actual as Percent of Estimate 16.0 32.0 58.4 63.6 92.4 100.00 Other Project Data Original Actual Identification - 03/02/78 Appraisal - 09/20/80 Negotiations - 02/19/81 Board Approval - 09/21/81 Loan Agreement Date - 09/24/81 Date of Effectiveness 12/23/81 04/21/82 Closing Date 06/30/85 06/30/87 Borrower: Republic of Colombia Executing Agency: Fondo Nacional de Caminos Vecinales Follow-on Project: Loan 2668-CO - Rural Transport Sector 9 Staff Input (staff weeks) FY78 FY80 FY81 FY82 FY83 FY84 FY85 FY86 FY87 FY88 FY89 Total Pre- appraisal - 37.9 10.8 - - - 48.7 Appraisal - - 37.4 - - - - - 37.4 Negotiation - - 1.2 - - 1.2 Supervision - - .5 8.2 10.0 5.2 7.1 4.6 3.9 3.9 2.6 46.0 Other .1 .2 13.7 - - - - - - - - 14 Total .1 38.0 63.6 8.2 10.0 5.2 7.1 4.6 3.9 3.9 2.6 147.3 Mission Data Month! No. of No. of Staff Date of Year Days Persons in Weeks Report Field Preparation 05/80 3 4 12 07/14/80 Appraisal 09/80 2 4 8 10/15/80 Supervision 01/82 1 3 3 03/18/82 Supervision 04/82 1 1 1 04/29/82 Supervision 06/82 2 2 4 06/23/82 Supervision 11/82 4 2 8 12/03/82 Supervision 05/83 4 3 12 05/27/83 Supervision 11/83 3 3 9 01/13/84 Supervision 06/84 4 3 12 07/19/84 Supervision 11/84 2 1 2 01/07/85 Supervision 06/85 2 2 4 06/21/85 Supervision 11/85 1 2 2 01/17/86 Supervision 06/86 2 2 4 06/26/86 Supervision 12/86 1 1 1 12/22/87 10 Basic Data Sheet HIGHWAY SECTOR PROJECT (LOAN 2121-CO) Key Project Data Original Actual Total Project Cost (US$ million) 152.3 Cost Overrun (percent) IBRD Loan Amount Disbursed 149.6 Cancelled 2.2 Estimated Economic Rate of Return Physical Component Completed 12/31/86 12/31/88 Cumulative Estimated and Actual Disbursements FY83 FY84 FY85 FY86 FY87 FY88 FY89 FY90 Appraisal 28.3 70.3 122.3 152.3 152.3 152.3 152.3 152.3 Estimate Actual 2.3 29.1 56.4 93.0 127.8 137.0 147.7 149.6 Actual as Percent of Estimates 8.1 41.1 46.1 61.2 83.9 90.0 97.0 98.2 Project Dates Original Actual Identification 08/26/80 Negotiations 03/02/82 Board Approval 04/01/82 Effectiveness - 09/22/82 Closing Date 12 /31/86 12/31/88 11 Staff Input (Staff weeks) FY80 FY81 FY83 FY84 FY85 FY86 FY87 FY88 FY89 FY90 TOTAL Through Appraisal 1.5 81.2 4.6 873 Appraisal Through 95.6 95.6 Board Approval Board Approval Through Effective- ness 5.4 5.4 Supervision 24.1 14.8 13.5 6.5 9.2 10.0 11.0 5.4 94.5 TOTAL 1.5 81.2 105.6 24.1 14.8 13.5 6.5 9.2 10.0 11.0 5.4 282.8 12 Mission Data Stage of Project Cyce Mo~t/ No. øf Days in speciaia P~rormace Type of Year Persons Fid y Repr~ 7iei' Rating/Staus Y Prablems. Through Oppraisal 09/80 1 12 En 12/180 2 8 Ec, En 03/81 4 15 2Ec, 2En 07/81 10 15 4Ec, 3EN, 1TrS,IFA, ITrE Appraisal through Board 01/82 4 10 lEc, 2En, IFA Approval Board Approval Through 06/12 3 6 Ec, En, TrS 1 effectivene"s Supervision 10/82 3 5 Ec, E, FA i 11/182 1 3 Tra 02/83 1 4 TrE 04/83 3 10 Ec, En, FA 2 F,M 05/83 1 5 En 07/83 1 3 En 09/83 2 5 Ec, TrS 11/83 3 6 Ec, Ea, F.A 2 P,M 01184 1 3 En 04/84 1 2 En 2 F.M 06/84 3 6 Ec, Ea, FA 2 F,M 11/84 1 6 En F,M 01/85 1 5 En 05/85 1 5 TrE 07/85 3 6 Ec, En, FA 2 F,M 11185 2 6 Ec, En 2 F,M 04/86 3 6 Ec, En, FA 3 FM 09/86 1 4 En 2 F,M 03/87 1 5 En 2 F,M 10/87 3 4 IEc, 2En 2 FM 04/88 2 6 Ec, En 2 F,M 03/89 2 4 Ec, En 3 F,M "Moet missions were combined with supervision of other projects in the Transport Sector Z Ec = Economist; En = Engineer; FA - Financial Analyst; TrS - Tranpor Specialist; TrE = Training Export " 1 Minor Problems; 2 - Moderat Problems; 3 - Major Problem& " F =Financial; M Managerial 13 Evaluation Summary Introduction 1. The Bank-supported the Colombian roads sector through thirteen projects delivered over a 43-year period (1951-1994). Eleven projects have been completed, and two are under implementation. Total approved lending was US$822 million nominally, which in current terms would be well over US$ 1 billion. In addition, the Bank lent for a string of railway and ports projects. Ten road loans aimed at national highways and three aimed at rural roads. Another (national) roads project is under consideration. The first six projects, that is all those preceding this audit's projects, dealt primarily with major civil works - road construction, rehabilitation and improvement. Subsequent projects gave increased attention to road maintenance, institution building, transport services and traffic management. 2. The first six projects performed unevenly due in great part to weak institutions. The sixth project suffered a huge time overrun (about eight years), and it had a shortfall in physical works completed. A major factor was limited institutional strength, which the appraisal had underestimated. Thus, there was ample justification for taking a hard look at institutional capacity in the design of follow-up projects. 3. Road transport is the dominant mode. It carries some 91 percent of inter-urban domestic freight, excluding pipelines, and some 90 percent of inter-urban domestic passengers. Traffic on many inter-city roads reaches thousands of vehicles. Traffic movement is often chaotic, and the accident rate is high. The inter-urban road network consists of some 88,000 of public roads, of which 26,500 km (30 percent) are classified as national, 43,800 km (49 percent) as departmental and 18,500 km (21 percent) as feeder roads. Departmental and feeder roads serve both local needs and feed into the national roads system.' 4. Administration of the inter-urban road network is divided into three parts. Until late 1993, the Ministry of Public Works and Transport (MOPT) was in charge of national roads, which it administered through its Highway Directorate; thereafter, the National Highways Agency (NHA), an autonomous government organization created out of the former Highway Directorate, took over MOPT's national roads functions. Secretariats of Public Works (SPWs), within the Departmental Governments, are in charge of departmental roads, and the National Feeder Roads Fund (FNCV) is in charge of the feeder roads. Starting 1988 the FNCV was required to cofinance all projects with local governments. MOPT/NHA are the most developed, best-endowed and most Bank-supported entities in road administration. FNCV I. Since the audit the road classification has changed. Currently, the inter-urban road network consists of some 118,700 km of public roads, of which 12,400 km are now classified as national. The remainder of the former national roads, 14200 km, will be devolved to the departmental roads during in 1995-96. 14 comes in a distant second place. SPWs are more underprivileged than the national agencies, with a large scope of works and little financial and institutional means.2 Objectives and Design 5. The Seventh Highway Project (H VII) was classical Bank lending, with components fully defined and prepared at appraisal. The Rural Roads Project (RR I) was a variation of such a project - the major component (civil works) was only partly defined and prepared at project start. The Highway Sector Project (HS I) became the first Colombian highway "sector" loan, a lending instrument defined in specific Bank operational instructions. Adequate implementation capacities were essential for such a project. 6. The goals of H VII, all related to road maintenance improvements, were comprehensive: rehabilitation of main trunk highways, elimination of truck overloading, improvements in maintenance efficiency and upgrading of transport planning. Project components were programs for road rehabilitation, vehicle weight control, highway maintenance and transport sector management. 7. The goals of HS I were comprehensive. They included strengthening of MOPT's highway management capacity; adjustment and improvement of sector policies; and strengthen- ing of the construction industry. The areas addressed by institutional strengthening were: planning, design and execution of civil works; road safety; personnel training; vehicle weight control; and road maintenance. Policies to be improved concerned sector planning, road user charges and trucking industry regulation. The SAR certified that MOPT's "administrative capacity ... and organizational framework [were] ... adequate to manage a Highway Sector Project". Little analysis was offered to make such a case, and the appraisal seemed oblivious to the implementation difficulties which H VII was experiencing. The risk assessment was perfunctory. 8. RR I had no explicit goals. It dealt only with FNCV, and it contained no provision to support any SPW directly. Focus was on road construction and rehabilitation; road maintenance and institution building received little attention. Maintenance played a small role despite large maintenance deficits both at FNCV and SPW levels. Implementation 9. H VII and HS I implementation was arduous. The Bank's long association with national roads, and MOPT's institutional capacity were plainly insufficient for good project delivery. Of the audited projects RR I was the best, if only because it was unpretentious (though not necessarily well focused) and modest in objectives and scope. 2. Substantial institutional changes have taken place recently, and more institutional changes are underway. For example, the FNCV was slated for liquidation in 1992 and will be phased out by the end of 1995. The 1993 Transport Law provides that the responsibility for financing the feeder roads goes to the departments (not to local governments) and it also will have the jurisdiction over the former FNVC roads as well as the departmental network. The departments will coordinate access to the Road Cofinancing Fund. In addition the GOC has decided that IDB should concentrate its transport operations at the subnational level, while the Bank will continue supporting roads development at the national level. There also is a Bank supported effort underway to transform the Instituto Nacional de Vias into a more politically independent agency. 15 delivery. Of the audited projects RR I was the best, if only because it was unpretentious (though not necessarily well focused) and modest in objectives and scope. 10. H VII ran into trouble right from the start. No component - programs for road rehabilitation, highway maintenance, vehicle weight control and transport sector management - proceeded well. None reached fully their physical target, and some program items were not implemented at all. The achievement rate for civil works was 80 percent. Project completion came to an end with a delay of four and one half years and a cost overrun of about 20 percent. Compliance with loan covenants was mixed. 11. Implementation of HS I was even more labored, and achievement rates were again low. Civil works items reached only 25 percent or even less of appraisal targets. Some items were not implemented. Project implementation came to an end with a delay of two years. Total costs were about 85 percent of appraisal estimates. Compliance with loan and guarantee covenants was spotty. 12. The Bank followed the stop-and-go HS I implementation with apprehension and gave plenty of advice. It had at times misconceptions about project performance, rating it more often than not as "2" in supervision. Bank staff input for the project was a high 260 staff- weeks, which meant that one expected benefit of sector lending, low Bank staff costs, did not materialize. 13. RR I had start-up problems, but it ultimately delivered between 82 and 95 percent (kn) of planned civil works. Delivery of the study program was satisfactory; however, its implementation did not take place. The implementation of the physical components of RR I lasted two and one-half years longer than planned. Actual total costs were 17 percent below appraisal estimates. Compliance with covenants was good. Results 14. H VII and HS I contributed to the physical amelioration of national roads. They completed civil works on about 2,000 km of roads, which was about 8 percent of the national roads length at that time. Actual accomplishments fell far short of rehabilitation needs. Some works were of mediocre quality. The PCRs suggested that the investments had good to excellent ex-post economic rates of return (ERRs). However, little information was given about key assumptions and methods of calculation, and thus the returns eluded close examina- tion and verification. 15. Neither SARs nor PCRs for H VII and HS I provided statistics about road network conditions. Without this basic information, verification of maintenance results "on the ground" is impossible. It is likely, that the projects improved maintenance capacities (as distinct from delivery of maintenance works). But it is also likely that these results were of limited scope, given the numerous maintenance failings which prevailed during and upon project completion. 16. H VII and HS I had mixed results in improving road infrastructure management. Improvements in MOPT's internal communication, office technologies, coordination and economic planning were satisfactory. MOPT also pioneered routine maintenance contracts with "micro-empresas", which are small labor-based cooperative works organizations. The 16 areas of institutional import, where H VII and HS I did not satisfactorily deliver, were MOPT staff training, road safety, vehicle weight control, road periodic maintenance and MOPT budgeting, and construction industry development. Much unfinished business shifted to a new time-table and new lending. Since then, advances in these areas were made reportedly to varying degrees. But in at least two of them, road safety and vehicle weight control, no real progress has been made. 17. RR I was successful in constructing and improving rural roads and exposing FNCV to the rigors and demands of an international lending institution. However, the project contributed little to the resolution of the two most pressing issues in the sector - road maintenance and sector organization. Most institutional energies were absorbed by construction. The project had been designed without a sound framework of sector organization. In the meantime, the government declared its intention to consolidate all rural roads at departmental levels, to decentralize government. This could mean the elimination of the RR I-supported FNCV, or at least a severe curtailment or major change of functions. 18. The limited project achievements overall were influenced by four sets of constrain- ing factors: the country's political and social environment; the technical capacity of MOPT; project design; and the limited knowledge that existed at the time about the processes of institutional change. * Throughout much of the eighties, the country was in the grips of destabilizing forces which limited the orderly functioning of Government and the political process. This made the purposeful marshalling of national energies and resources for development projects difficult at best. * The technical capacity of MOPT, and indeed of other branches of government, were unsuitable for good implementation. In particular, Government did not have the institutional strength to pull off a demanding project such as HS I. * Both H VII and HS I were overambitiously designed and project "quality at entry" was not addressed in sufficient depth. The institution's goal was to lend, and while the importance of institutions was known, there was a lack of appreciation of how difficult and time consuming institutional change processes are. * A contributing factor also was the lack of know-how in the Bank -- and elsewhere -- on institutional development. In the mid-eighties it was thought that changes in organizations could be brought about mechanically and that these changes themselves would bring about necessary broad-based redesign of core processes. Several projects, and not only HS I, in both developed and developing countries, and in other types of institutions than the road administration, have in the past few years developed approaches to implement institutional change. Substantial knowledge has been accumulated, but much remains to be learnt. It would be unfair to single out the HS I as a singular failure. What was attempted was into the right direction, what needs to be done is to correct or redo things that went wrong or failed. 17 Project Ratings and Conclusions 19. H VII had a lackluster and HS I had a marginal performance. The sustainability prospects are unclear. The national roads sector is in recovery, but over 40 years of mixed sector performance are cause for concern. RR I had a reasonably good score on construc- tion, and a limited score on institutional impact and road maintenance. Sustainability is also a question, in particular because of the unsettled issue of sector organization. 20. The PAR rates the projects as follows: H VII overall outcome: marginally satisfactory; institutional impact: negligible; and, sustainability: uncertain. In the PCR the overall outcome was rated as satisfactory, the other aspects were not evaluated. The PAR ratings for the HS I are the following: overall outcome unsatisfactory; institutional development negligible; and sustainability uncertain. The corresponding ratings in the PCR were: satisfactory, partial, and uncertain. For RR I the PAR overall outcome rating is marginally satisfactory, institutional development negligible and sustainability uncertain. The corresponding PCR ratings were satisfactory, partial, and likely. 21. The uncertain PAR outcome ratings for sustainability reflect the unsettled state of Colombian roads administration. The lending experience so far has been mixed and, in the area of institutional development, the progress has been slow. There can be do doubt, however, that the Bank's presence has been, and continues to be, a positive factor in shaping institutional initiatives, in addition to bringing benefits derived from the physical improvements in the road network. 22. The Colombian projects are in a traditional Bank sector which has received long- lasting Bank support and where the achievements are slow in coming and the outlook is uncertain. The projects audited mark a shift in the Bank's highway sector agenda -- H VII (approved 1977) made national roads maintenance the main project issue; HS I (approved 1982), while emphasizing maintenance, focused on policy and institutional issues; and RR I (approved 1981) was the first Bank transport operation which dealt with the needs of non-national roads. They demonstrate the difficulties of using traditional Bank lending assistance for accomplishing institutional sustainabililty and the ultimate goal of making a sector autonomous. 23. The three audited projects, in combination with the earlier six projects, also raise questions about the strategy and method of application of Bank interventions in the sector, especially with regard to institutional development. After implementation of ongoing lending, the Bank will have completed thirteen road sector projects in Colombia, of which ten were exclusively directed at national roads; and it will have been associated with Colombian roads for 46 years - with very slow progress. At this time, it is suggested that, in the context of preparing future Bank strategy in Colombia's highway sector, a comprehensive assessment be undertaken, aimed at identifying priority highway sector issues, possible institutional and policy reform measures and their sequencing, and the role of the Bank in the sector. The assessment would need to take into account relevant international experience in institutional development in the sector, in both developed and developing countries.  19 1. Introduction 1.1 The Bank supported the Colombian roads sector through thirteen projects delivered over a 43-year period (1951-1994). Eleven projects were completed, and two are under implementation. Total approved lending was US$ 822 million nominally, which in current terms would be well over US$ I billion. In addition, the Bank lent for a string of railway and ports projects. Ten road sector loans, including the first seven, were made for the benefit of national highways; three were made for rural roads. Another project (national roads) is under consideration. The first six projects, that is those preceding the three operations analyzed by this audit, dealt primarily with road construction, rehabilitation and improvement. Subsequent projects paid increased attention to road maintenance, institution building, transport services and traffic management. 1.2 The first six projects performed unevenly in terms of implementation and results. Among the issues encountered were geophysical problems, inadequate engineering, shortage of local funds, slow payments to contractors and weak institutions. The first four projects did not finish all planned works. 1.3 The fifth project (approved 1968) managed to deliver all planned works, though with a moderate cost and a large time overrun. Lack of government commitment impeded institution building, which was attempted at a limited scale and directed at the Ministry of Public Works and Transport (MOPT), the national highways ministry. The planned institutional measures, primarily reorganization of the ministry based on the findings of a management study, were ultimately implemented years after project completion. Progress on road maintenance, also a minor project item, was slow. Road maintenance has been a deeply entrenched problem since the Bank started lending to the sector. 1.4 The sixth project (approved 1970) suffered a substantial cost overrun (about 40 percent), and an even larger time overrun (about 230 percent for a total of an additional eight years), and it had a shortfall in physical works completed. A major factor in the performance was the limited institutional strength of MOPT, which the appraisal had not properly assessed. Thus, there was ample justification for taking a hard look at MOPT's institutional capacity in the design of the follow-up projects. 1.5 Road transport is the dominant mode. It carries some 90 percent of inter-urban domestic freight and inter-urban domestic passengers. The percentages gradually etched up in the last 30 years and there is still much growth potential with the expansion of the economy and the road networks. The railway and river, coastal shipping and air transport services carry the balance of transport. Traffic on many inter-city roads reaches thousands of vehicles. Traffic movement is often chaotic, and the accident rate is high. The road transport industry is dynamic in a complex regulatory system. 1.6 The inter-municipal or inter-urban roads network consists of some 88,800 km of public roads, of which 26,500 km (30 percent) are classified as national, 43,800 km (49 percent) as departmental and 18,500 km (21 percent) as feeder roads. Little distinguishes the functions of departmental and feeder roads, as both serve local and provincial communications 20 needs and feed into the national roads system. However, feeder roads generally connect larger settlements and carry higher traffic. The inter-urban road network has steadily grown. Since 1980, for example, the total stock of roads (in km) has increased by 15 percent. At the same time, road standards have steadily improved. Of the national roads, some 40 percent are paved, and most of the remaining 60 percent are graveled. Of the stock of departmental roads, paved roads are the exception, with some 3 percent of the total length; 75 percent have gravel surfaces, and 22 percent are earth roads. The percentages for feeder roads are almost identical to those of departmental roads.' 1.7 Administration of the (inter-municipal) roads network is divided into three parts. Until late 1993, the Ministry of Public Works and Transport (MOPT) was in charge of national roads, which it administered through its Highway Directorate; thereafter, the National Highways Agency (NHA) - Instituto Nacional de Vias - an autonomous government organization created out of the former Highway Directorate, took over MOPT's national roads functions, and MOPT changed into the Ministry of Transport (MOT). The Secretariats of Public Works (SPWs) in the Departmental Governments handle departmental roads; and the National Feeder Roads Fund (FNCV) is in charge of the feeder roads. FNCV was created in the early sixties as a national government agency to give feeder roads a boost and support local road administrations. Charter tasks of FNCV are road construction, improvement and maintenance. MOPT/NHA are the most developed, best-endowed and most Bank-supported entities in road administration. FNCV comes in a distant second place. SPWs are more underprivileged than the national agencies, with a large scope of works and little financial and institutional means. 3. Since the audit the road classification has changed. Currently, the inter-urban road network consists of some 118,700 km of public roads, of which 12,400 km are now classified as national. The remainder of the former national roads, 14200 kin, will be devolved to the departmental roads during in 1995-96. 4. Substantial institutional changes have taken place recently, and more institutional changes are in the offing. For example, the FNCV was slated for liquidation in 1992 and will be phased out by the end of 1995. 'Te 1993 Transport Law provides that the responsibility for financing the feeder roads goes to the departments (not to local governments) and it also will have the jurisdiction over the former FNVC roads as well as the departmental network. The departments will coordinate access to the Road Cofinancing Fund. In addition the GOC has decided that IDB should concentrate its transport operations at the subnational level, while the Bank will continue supporting roads development at the national level. Reorganization of the NHA - Instituto Nacional de Vias - is also being considered. 21 2. Project Objectives and Design Overview 2.1 The three projects audited - Seventh Highway Project (H VII), Road Sector Project (HS I) and Rural Roads Project (RR I) - marked a shift in the Bank's highway sector agenda. H VII made national roads maintenance the main project issue. About 25 years of Colombian roads experience, which consistently had highlighted maintenance problems, came now to fruition. HS I emphasized again maintenance, and it focused additionally on policy and institutional improvements, all in the national roads sector. RR I became the first Bank transport operation which dealt with the needs of non-national roads. The three projects were approved within a span of five years - H VII in 1977, RR I in 1981 and HS I in 1982. 2.2 H VII was a classical Bank project, with components fully defined and prepared at appraisal. RR I was a variation of such a project, with the distinction that the major compo- nent (civil works) was only partly defined and prepared for implementation at project start. Two more rural roads projects followed - Rural Transport Sector Project, Loan 2668-CO (RR II) of 1986 and Second Rural Roads Sector Project, Loan 3157-CO (RR III) of 1990. HS I became the first Colombian highway "sector" loan, which in objectives and design had special features. Another two such projects, Second National Highways Sector Project, Loan 2829-CO (HS II) and Third National Roads Sector Project, Loan 3453-CO (HS III), were approved in 1987 and 1992. 2.3 Guidelines and rules for sector lending, or specifically highway sector lending, were set out in these Bank operational instructions: Central Projects Memorandum (CPM) No. 8.2 (1974); CPM No. 8.3 (1975); Operational Manual Statement (OMS) No. 1.19, Annex A; and Central Projects Note (CPN) No. 10.03 (1982, just prior to approval of Colombia HS I). Of these, CPN 10.03 was the most important for transport. 2.4 CPN 10.03 on Highway Sector Lending stated that sector loans were appropriate in countries in which highway development programs were well designed and institutions could satisfactorily prepare and implement such programs. The Bank would, under a sector loan, finance a time slice of a multi-year investment (or expenditure) program; and the appraisal would focus on the adequacy of institutional planning and the program content, criteria used for subproject appraisal and preparation, and on institutional capacities (including implementa- tion capacities) in the entire sector. CPN 10.03 was very explicit on adequate implementation capacities, which had to be confirmed through careful appraisal analysis. 2.5 The sector lending instructions listed advantages to both borrowers and lender from such operations. Among the advantages for the borrowers were: assured financing for a significant slice of the investment program; flexibility in the selection of subprojects (during implementation); discipline in the selection of subprojects, achieved through application of pre- defined and -agreed appraisal criteria; and forceful attack on any remaining institutional weaknesses. Among the advantages for the Bank were: broadened impact on the full spectrum of sector issues, including sector policies; ability to commit larger loans and accelerate loan disbursements; and lower project management costs, particularly for Bank staff. 22 Seventh Highway Project 2.6 The specific goals of H VII, all related to road maintenance improvements, were: rehabilitation of main truck highways, elimination of truck overloading, efficiency improve- ments in maintenance operations, and upgrading of transport planning. Truck overloading was a major cause of road deterioration, as in many other countries. Past remedial efforts had not gone far. Project components were programs for road rehabilitation, vehicle weight control, highway maintenance and transport sector management. The vehicle weight control program entailed, in laboriously laid out detail, definition of a plan of action for "coordinated govern- ment actions concerning the regulatory and legal framework, institutional arrangements and development of public awareness". It further included installation of some twenty weighing stations on main trunk roads.' The road maintenance program had two major components, procurement of equipment and technical assistance (TA). A large set of covenants, most dealing with road maintenance and vehicle weight control, contributed to the definition of project tasks. 2.7 Total estimated project costs were US$154 million, of which over 70 percent were earmarked for road rehabilitation and about 20 percent for maintenance equipment. The approved Bank loan was US$90 million. Implementation was scheduled over five years, 1977 to 1981. The Staff Appraisal Report (SAR) judged project risks as small. The rehabilitation program carried no "special risks", it said. Regarding the maintenance program, it said, "risks had been sufficiently reduced to allow hope of success". For the vehicle weight control program, the SAR made no explicit prognosis. It hinted, however, at large obstacles.6 2.8 At Board presentation, an Executive Director (ED) noted that the project was rehabilitating roads which the Bank had financed before and that this was a Bank defeat in Colombian roads maintenance. The ED expressed hope that no further lending was needed in another ten years to fix maintenance again. Bank management explained that the maintenance issue, which the Bank was facing, was difficult to resolve. Highway Sector Project 2.9 The goals of HS I, derived from the goals of the underlying highway sector program and tucked away in various sections of the SAR, were sweeping. They included, as interpreted by the PCR: strengthening of MOPT's highway management capacity; adjustment and amelioration of sector policies; and strengthening of the construction industry.' The areas addressed by institutional strengthening were: planning, design and execution of civil works; road safety; personnel training; vehicle weight control; and road maintenance, through country- wide implementation of a newly designed Pavement and Equipment Management Systems. Sector policies to be improved concerned sector planning, road user charges and regulation of 5. SAR, Seventh Highway Project, 1977, paras. 4.08-4.10. 6. On project risks, refer to: SAR, Seventh Highway Project, 1977, paras. 4.37-4.39. 7. Transcript of Board Proceedings, June 28, 1977. 8. PCR, Highway Sector Project (Loan 2121-CO), 1991, para. 3.01. 23 the trucking industry. Most tasks were defined in much detail in the voluminous SAR, which an appraisal team of no less than nine sector specialists did compose. 2.10 The project covered MOPT's 1982-86 highway program. Total costs were about US$2,365 million, of which road maintenance (exclusive of rehabilitation) claimed a large part - almost 30 percent. In this sense, maintenance was a credibly large item. The Bank loan was smallish -- US$152.3 million -- given that so much was attempted and this amount would cover only 6 percent of the entire program. Some 85 percent of the loan was reserved for major works - road construction, rehabilitation and paving. A long list of covenants on a broad range of issues (implementation, policies and capacity building) complemented the definition of project components. 2.11 The SAR certified that MOPT's "administrative capacity ... and organizational framework [were] ... adequate to manage a Highway Sector Project"." Little analysis was offered to make such a case. In fact, the appraisal seemed oblivious to the implementation difficulties, linked to weak institutional performance, which H VII was experiencing. At HS I approval, H VII was in its fifth year of implementation. The SAR went as far as stating, in contradiction of actual developments later documented in the PCR, that H VII was "proceeding well" after an initial delay. The SAR's risk assessment, while consistent with the certification of adequate implementation capacity, was perfunctory. However, it conceded uncertainty about Bank ability to influence sector policies. 2.12 At Board presentation, an Executive Director noted the SAR's complimentary assess- ment of MOPT's institutional capacity and inquired whether more institution building was really needed. Senior Bank management gave a lengthy discourse on what constituted a sector loan and what institution qualified for it. It explained that such a loan was appropriate when the institution could function entirely on its own, without needing institutional support from the Bank, or would soon meet such a condition. Another Executive Director took the sector format (wrongly) as proof of Bank success in past Colombian capacity building." Rural Roads Project 2.13 RR I had no explicitly stated objectives, which was an undesirable feature of project design, but presumably was due to the RR I being the first loan to non-national highways in Colombia. In what came closest to a statement of objectives, the SAR said that the project would provide "a framework" to address the sector needs." It implied, but did not demonstrate, that this would best be accomplished through exclusive support to FNCV, so the agency could position itself to provide competent planning, programming and financing support to departmental SPWs. Accordingly, the project dealt only with FNCV, and it contained no provision to support any departmental SPW directly. 9. SAR, Highway Sector Project, 1982, para. 3.10. 10. SAR, Highway Sector Project. For its account of H VII progress, see para. 1.26; for HS I assessment, see paras. 5.36 and 5.37. I H. Transcript of Board Proceedings, April 1, 1982. 12. SAR, Rural Roads Project, 1981, para. 5.01. 24 2.14 The project dealt primarily with road construction and rehabilitation, and it gave limited attention to road maintenance and institution building. About 85 percent of expendi- tures were reserved for construction and rehabilitation, and about 15 percent for road mainte- nance equipment. A small amount of financing was reserved for TA, recruited to facilitate some institution building. Maintenance received secondary treatment despite the large maintenance deficits at FNCV and especially at SPW levels, which the SAR clearly recog- nized. The "limited" TA, as the SAR noted, was to help FNCV streamline administrative procedures, particularly for managing technical information. 2.15 The project carried the usual implementation covenants. Another covenant obligated the borrower "to balance construction, rehabilitation and maintenance." Further covenants dealt with institutional improvements of FNCV - the upgrading of the management information system, decentralization and staff development. Project implementation was scheduled over three years, 1982-1984. The SAR's risk assessment was meager and signalled no appreciable risks." 13. SAR, Rural Roads Project, paras. 5.29-5.30. 25 3. Project Implementation General 3.1 Implementation was arduous for H VII and HS I. The Bank's long association with national roads and MOPT's institutional capacity were insufficient for smooth implementation. The Bank's appraisal of the borrower's implementation capacity and sector conditions was proven over-optimistic, even erroneous. The best in the performance of the tri-project group audited was RR I, if only because it was unpretentious (though not necessarily well focused) and modest in scope. Seventh Highway Project 3.2 H VII, the platform from which the Bank was to launch the first Colombian highway sector lending, ran into trouble from the start. No component - programs for road rehabilitation, highway maintenance, vehicle weight control and transport sector management - proceeded well. * Road rehabilitation suffered from inefficient bidding procedures, fragmentation and complexity of administrative steps in managing this component, poor contractor performance and shortage of local funds. The physical scope of works needed reduc- tion by some 20 percent. * Procurement of highway maintenance equipment took place. However, overall maintenance performance fell short of expectations. Maintenance budgets rose and fell again, ultimately not meeting agreed targets. Efficiency of maintenance operations did not develop as expected. * The vehicle weight control program, which was to eliminate a major cause of premature road deterioration, was stopped quickly. A plan of action was developed after difficult negotiations. But, no implementation took place. The borrower showed little drive in moving ahead. * The program of improving transport sector management produced mixed results. Among the items not implemented was management training of local contractors. 3.3 The Bank tried its best to support implementation once the project got underway. In its interactions with the borrower, it emphasized measures for institutional strengthening, though with limited results. Confronted with numerous implementation obstacles, it opted for the "rolling over" of project obligations.4 3.4 Project completion came to an end with a delay of four and one half years and a cost overrun of about 20 percent. The overrun on rehabilitation, partly caused by faulty engineering and inflation, was about 45 percent. Compliance with key covenants matched the overall implementation performance. 14. PCR, Seventh Highway Project (Loan 1471-CO), 1988, para. 7.01. 26 3.5 The PCR had straightforward -- and correct -- recommendations on how to better future implementation performance. It said, improvements were needed in: preparation of engineering studies; procurement and contracting procedures; administration of contracts; budgeting and financial controls; road maintenance administration; pavement and equipment management systems; information systems; and MOPT personnel training. All these were issues critical for the design and implementation of the follow-up project. Highway Sector Project 3.6 Implementation of about all HS I components and undertakings was an even more labored exercise than H VII, but the results were unsatisfactory. * Civil works, which were delayed right from the start, were adversely affected by change in Government, lack of counterpart funds, poor contractor performance and poor supervision. Actual physical achievements of sub-components were only 25 percent or even less of appraisal targets. * Strengthening of highway management capacity was even more profoundly affected by the change in Government. Almost nothing proceeded as planned. The road safety program was only partially implemented. Personnel training had a 60 percent achievement rate. The program to change MOPT's administrative and financial procedures took off well but then tapered off. There was no progress in controlling overweight vehicles; some portable scales were procured, but that was all. Resolution of the issue shifted again to the next project. The envisaged steps to improve road maintenance were only partially taken. In particular, MOPT did not extend the Pavement and Equipment Management System beyond the four pilot districts to all 23 maintenance districts. * The studies to launch sector policy improvements were only partly completed. Support to the domestic contracting industry did not materialize. 3.7 The project came to a close two years after the originally planned date. Total costs were about 85 percent of appraisal estimates, largely a function of two factors which partially compensated each other - a severe cut - back in physical works and a rise in unit costs due to inflation. Compliance with loan and guarantee covenants was spotty. Audited financial accounts were prepared late and not fully satisfactorily. Expenditures on road maintenance personnel and administration continued at undesirable high levels, limiting budgets available for actual works. The borrower did reach targets for the volume of contracted routine maintenance, but did not do so for periodic maintenance. 3.8 The Bank followed the troubled implementation with much apprehension and gave plenty of advice. However, it had at times misconceptions about project performance, rating it more often than not as "2" in supervision. Such a rating is usually reserved for projects with minor problems which are being cured. On average, the Bank mounted almost three and one- half supervision missions annually, and it spent about 95 staff-weeks on supervision (mission and headquarter time combined), which is above averages for highway projects. Total Bank staff input for the project was a high 260 staff-weeks, which meant that one objective for which the Bank introduced sector lending, the lowering of Bank staff costs, did not material- ize. Typically, the Bank spends 80-150 staff-weeks on highway projects. 27 3.9 The PCR noted implementation issues which were readily identifiable at HS I appraisal but did not enter project design. It concluded that MOPT was institutionally too weak to implement a sector project as demanding as HS I, and that HS I should have been defined in much simpler terms. It further determined that HS I should have been postponed, pending execution of the delayed H VII, and that the appraisal did not realistically assess the risks of MOPT's absorptive capacity. The PCR appropriately identified a long list of issues which the implementation had brought to light. Among them were: lack of sustained government commitment; frequent changes in MOPT's high-and mid-level staff; defective engineering and cost estimates; cumbersome bidding and procurement procedures; inconsisten- cies between budgetary and physical programming; weak contractor performance; and failure to establish adequate maintenance and cost accounting systems. Actions on many fronts were needed to provide remedies, it said." If there is a positive side to these many negatives it is this: the Bank perceived and admitted the true state of affairs in the management of Colombian highways. This realization should augur well for the future projects. Rural Roads Project 3.10 RR I also had its start-up problems, due in part to shortage of local funds and unavailability of engineering studies. The project ultimately delivered about 95 percent of planned construction (in kin) and about 82 percent of planned rehabilitation (km). The PCR cited cost underestimates, poor works programming and supervision, poor contractor perfor- mance and bad weather as responsible factors for the problems. The project procured all required road maintenance equipment. Delivery of the study program (studies of FNCV organization, statistical and information needs, and archiving needs) was satisfactory. However, no implementation took place due, in the view of the PCR, to lack of government commitment and funding shortages." The argument of funding constraints appears to be an excuse for a more fundamental reason as the amounts were small in comparison to the project's road construction works. 3.11 RR I implementation lasted two and one-half years longer than planned. Actual total costs were about 17 percent below appraisal estimates, which was largely the result of local currency devaluations. Compliance with covenants was good. 15. PCR, Highway Sector Project, paras. 9.01-9.04. 16. PCR, Rural Roads Project (Loan 1966-CO), 1989, para. 3.08.  29 4. Project Outcomes National Roads Sector Physical Results 4.1 H VII and HS I, who's combined lives spanned over more than ten years (1977-1988), contributed to the physical amelioration of national roads. They completed civil works on about 2,000 km of roads, which is about 8 percent of the national road network (of 26,500 km) at the time. Actual accomplishments fell far short of rehabilitation needs. About 90 percent of the improvements concerned rehabilitation, and 10 percent paving. Some works were of mediocre quality because of faulty engineering or construction. 4.2 The PCRs suggested that the investments had good to excellent economic rates of return (ERRs) ex-post. However, they offered limited information about key assumptions and methods of calculation, and thus the returns eluded close examination and verification. PCRs were also written shortly after project completion, when the improved roads were still in relatively good conditions and did not yet show the effects of wear and tear (by overloaded trucks) and maintenance neglect." The ERR analyses require, as those in many other PCRs, an act if faith and can serve only as crude indicators of investment performance. The PCR for H VII calculated an ex-post ERR of 23 percent for major civil works, against the appraisal's ex-ante ERR of 36 percent. It noted that the ERRs were not comparable because the analysis of principal benefits (savings in vehicle operating costs) was not comparable either. Projected traffic on these roads was high, between 1,000 and 6,000 vehicles per day, which is a condition favorable but not sufficient for satisfactory ex-post ERRs. * The PCR for HS I suggested an ex-post ERR of 57 percent for major civil works, with ERRs for individual sections of 5-85 percent. However, the number of roads analyzed was a small fraction of all roads which the project actually completed. The ex-ante ERR, calculated for a sample of roads only, was 75 percent. Again, the ex-post and ex-ante ERRs were not comparable, this time because the identity of road sections differed. Actual post-implementation traffic was between 350 and 5,300 vehicles per day and seems to corroborate the wide range of individual ex-post ERRs. 4.3 Neither the SARs nor PCRs for H VII and HS I provided statistics about road network conditions. In the absence of this basic and essential information, the ultimate test of road maintenance success, verification of results "on the ground", is impossible. 4.4 Regarding the creation of maintenance capacities, as distinct from delivery of actual maintenance works, it is likely that the projects achieved positive net results, and that they advanced the debate about the maintenance issue and the formulation and implementation of 17. PCR, Seventh Highway project, paras. 6.02-6.05; and PCR, Highway Sector Project, para. 6.02 and Part III, Statistical Information, 6. Project Results. 30 remedies. But it is also likely that results were of limited scope, given the numerous failings of maintenance which prevailed during and upon project completion. Among them were low operational efficiency, weak planning, diversion of funds to other uses, low labor and equipment productivity, high unit costs (highest in Latin America), and slow progress in contracting. 4.5 Recent statistics (of mid-1992) about network conditions indicate what still lies ahead: of the high-trafficked paved roads, 21 percent were in good, 45 percent in fair and 34 percent in poor condition; and of the unpaved roads, 46 percent were in good, 31 percent in fair and 23 percent in poor condition."' It is to be noted that road condition is an economic variable, it is not economical to maintain all roads in good condition. It is nor known how much the 1992 distribution of the road condition states differs form the optimal. It is noted that the capability to address this question was developed, but not yet taken into managerial use dealing with road condition surveys, as part of the institutional development (see para 4.8). Institutional Development 4.6 Both H VII and especially HS I had an extensive institutional capacity building program and both made progress toward the goal of a self-sufficient road organization. The shortfalls identified in the PCR derive from four sources: the change in Government just after HS I was approved, the MOPT's absorption capacity, the inability of the Bank approved consultants to train the MOPT professionals in a satisfactory manner, and the unworkable design of methods employed to implement the institutional component of the loan. 4.7 HS I had thus an important handicap from the very start: the loan was negotiated under an outgoing administration and the new incoming administration apparently did not have commitment to the institutional goals of HS I. In retrospect, the loan should have been renegotiated with the new administration to gain commitment and "contract" for the loan's objectives. 4.8 Satisfactory development took place in upgrading MOPT's internal communication, coordination and economic planning. Modern office technologies for financial management, personnel management and other functions found their way into the ministry. MOPT improved planning for strategic road infrastructure interventions (maintenance, rehabilitation, etc.) with the help of the Bank-sponsored Highway Design Model (HDM). The model is useful in rationalizing and substantiating necessary interventions, in particular for road maintenance. 4.9 MOPT also pioneered routine maintenance contracts with "micro-empresas", which are small labor-based cooperative works organizations located in the vicinity of roads. About 400 micro-empresas exist today, with a total labor force of 5,000 people. They cover 85 18. Data furnished by NHA during audit mission. 19. For an example of a detailed HDM analysis of strategic network interventions, refer to : MOPT, General Technical Secretariat, Highway Directorate, Highway Maintenance Unit, Evaluation Technico-Economica de la Red Vial Nacional, Informa Final, December 1993. 31 percent of the national road length and provide services for over US$ 10 million annually. The micro-empresas concept, an indigenous product, is a true success story.2 4.10 The areas of institutional import, where H VII and HS I did not satisfactorily deliver, were MOPT staff training, road safety, vehicle weight control, road (periodic) maintenance and MOPT budget administration, and construction industry development. Thus, much unfinished business shifted to a new time-table and new lending. Since then, advances in these areas were made reportedly to varying degrees. But in at least two of them, road safety and vehicle weight control, real progress has yet to be made. 4.11 In retrospect it is to be expected that these areas would lag behind because experience has proven that progress in them is slow and difficult. * Staff training and improved budget administration can hardly be done before there has been a broadly based redesign of core processes in planning (including economic planning), management, contracting procedures, and financial arrangements. * By international standards, traffic safety in Colombia is alarming due to traffic flow hazards (linked to driving attitudes and behavior, lack of adequate vehicle inspections, etc.) and highway robberies, assaults and theft. Driving attitudes change slowly and there must be a longstanding traffic safety program starting with school children. This traffic safety program must be a joint product of many institutions. To put such a program in place takes a decade. Some aspects of road safety squarely fall outside the jurisdiction of road authorities. Highway robberies and thefts fall into this category. An expectation that road safety should be much improved during the first loan period is patently unrealistic. * Vehicle weight control is another element which is moving forward only slowly. A master plan is under preparation. The installation and operation of a comprehensive system of stationary weigh bridges is not yet advanced far, though progress is being made under current Bank financing. Assuring adherence to regulations on the books constitutes a major challenge.' This is because experience in the developed countries shows that vehicle weight control cannot be accomplished by weigh bridges. A far more comprehensive approach, also enlisting the cooperation of the truckers, insurance companies, and shippers is necessary. Experience also shows that, because of its complexity, weigh bridge data collection is normally the last to be included in a road organization's information system. The available documents do not describe the sophistication of the MOPT/NHA information system, an element in the TA program, but one may surmise from the available material that it is one of the causes for the lack of progress in vehicle weight control. 20. For further details, refer to: MOPT, Mantenimiento y Conservacion de Vias a Traves de Microempresas Asociativas, May 1993. 21. For a recent analysis of outstanding issues, refer to: MOPT, Instituto Nacional de Vias, Subdireccion de Ingenieria, Pesaje y Control de Vehiculos de Carga en Vias Nacionales, undated. 32 4.12 The apparent slow progress in capacity building in important areas in MOPT/NHA in general may be attributable to several causes. The first and most important was the lack of "contract" with the Bank and the incoming administration. The second was the lack of appreciation of the slowness of institutional change by the Bank that initially may have led them to overestimate the absorption capacity of MOPT\NHA. The third was the reductionist way in which institutional change was assumed to take place - there really was no process plan geared to proceed from the initial conditions of MOPT\NHA - and which probably led the TA consultants to import methods or procedures which were alien to MOPT\NHA and which were abandoned right after the consultants departed. 4.13 On the positive side, there also was real, albeit slow, progress that was carried over from H VII to HS I and to the subsequent loans. Transport Policies 4.14 HS I dealt primarily with policy formulation, rather than implementation, and in this it was successful on balance. The project led to the design and updating of the National Transport Plan, a study of road user charges, and a study of trucking regulations, services and administration. Some implementation reportedly took place. The unfinished tasks shifted to new lending. The often stated argument that policy development should take place earlier in the lending sequence, particularly in Colombia where the issues in question had existed for some time, is an important matter for evaluation. The timing of transport policy issues in the lending sequence should be examined in the context of the proposed (para 23) comprehensive assessment of institutional development for the benefit of developing future Bank strategy in the sector. Rural Roads Sector 4.15 Within the frame-work of its limited objectives, RR I was successful. It led to the construction and improvement of a sizeable length of rural roads, and it exposed FNCV to the rigors and demands of an international lending agency. An accomplishment was completion of a road inventory, which provided a basis for rationalizing future road programs. The audit, however, does not fully share the PCR's view that the project contributed to "significant institutional and operational improvements" of FNCV.22 The PCR's assessment is overly generous for a project which had little institutional content. Bank experience also suggests that significant institution building does not occur as a by-product of implementing major physical works. 4.16 RR I focus on major civil works, rather than routine and periodic maintenance, probably limited investment returns. It is widely believed that maintenance expenditures carry higher economic returns than construction and improvement expenditures, and that network improvement and expansion are rarely viable unless accompanied by adequate provisions for maintenance follow-up. The priority given to construction, inconsistent with the Bank's own operational recommendations for good project design, should have been based on a thorough 18. PCR, Rural Roads Project, para. 6.01. 33 examination of "optimal" network size. It is possible that rural road network is very limited and that its expansion was an economic necessity. 4.17 The PCR estimated the ex-post ERR at 29 percent overall. Again, it gave little information about data and methodology used, and it based the calculation on a sample of project roads only. The ex-ante ERR was 37 percent; but the roads originally planned were largely different from those actually financed. This result suggests three conclusions: caution is warranted in citing and comparing project ERRs; that rural road expansion in Colombia apparently has rather high ERR, suggestion too small a network; and, because of the shortage of funds and returns expected, the selection of project loans has had strong political undercurrents. 4.18 The sector re-organization was not carried out under RR I, which apparently also served as a "fact-finding" exercise in this respect. The Bank came around to this task under RR II. There, it commissioned a study of departmental roads maintenance. In the meantime, the government moved ahead on sector organization on its own, when it declared its intention to consolidate all rural roads at departmental levels, to promote national government decentralization. This means major change of the functions of FNCV, and the build-up of competent SPWs in earnest." Implementation of the new organizational plan is imminent. 23. The situation regarding FNCV is fluid. Among the ideas under consideration is to retain FNCV as a technical assistance agency which would find a place in the rural roads development program proposed by Financiera de Desarrollo Territorial S.A. (FINDETER). Refer to FINDETER, Fondo de Cofinanciacion de vias, Fondo de Cofnanciacion para ia Infraestructura Urbana, January 1994. See also foot,ote 4.  35 5. Determinants of Project Outcomes 5.1 The project achievements were influenced by five sets of interrelated factors. Even though these are discussed separately below, the interrelationships of these factors must be kept in mind. * the country's political and social environment * project design * performance of the Bank * technical capacity of MOP'llNHA; and * the process employed to effect institutional change 5.2 Throughout much of the eighties, the country was in the grips of destabilizing forces which limited the orderly functioning of government and the political process. In such an environment, the purposeful marshalling of national energies and resources for development projects is difficult at best; and a government's preoccupation tends to be to survive, to maintain public law and order and to assure the supply of the most basic government services first. Many Bank project components, the greater part of policy and institutional components, required high-level government coordination of agencies and ministries, policy debate and, of course, decisions. The diffusion of project tasks and the nature of agreed undertakings made it infeasible to relegate the projects to the folds of MOPT only. The ministry itself faced internal coordination issues.2" Among the policy issues which would need high-level govern- ment attention were liberalization of transport policies, streamlining of governmental procure- ment procedures, changes in road sector financing (inter alia through road user taxes) and increased shift from force account to contract works (with the obvious implications for public sector employment). 5.3 Both H VII and HS I were overambitiously designed and project "quality at entry" was not addressed in sufficient depth. The institution's goal was to lend, and while the importance of institutions was known, there was a lack of appreciation of how difficult and timecon- suming institutional change processes are. The "initial conditions" in MOPT\NHA for imple- menting H VII and HS I were properly assayed during loan preparation. However, institutional processes, interlocked with physical implementation of the project, that move the road organization from its "initial conditions" to a truly autonomous road agency, to one which at the conclusion of the project had learnt and internalized important procedures, were not envisaged as part of the loan implementation. 5.4 The Borrower's need for capacity building was correctly assessed and a project whose goals were good was put together. This was especially true for HS I that learned and inherited the institutional shortcomings of MOPT1NHA from H VII. However, the project design 24. PCR, Highway Sector Project, para. 10.02. 36 focused too much on outcomes and ignored both resistance to change, which is always present, and motivation to change which either endorsed the status quo or would have brought about a desired outcome. These are crucial oversights that came to favor such consultant efforts that force-fed an outcome directed program without removing the underlying causes. The project design lacked an incremental process for change. 5.5 The Bank may have overestimated MOPTlNHA's absorptive capacity to carry out these ambitiously designed projects. The Bank underestimated the impact of the change in Government for obtaining "contract" for executing the project programs. This is particularly so when, during H VII, the deficiencies of MOPT\NHA implementation capacities were acute and visible. The Bank lacked an insightful technique how to implement changes. 5.6 The technical capacity of MOPT, and indeed of other branches of government, was unsuitable for project implementation in a straightforward manner; that is, without a resource- ful process. This was certainly true for a demanding project such as HS I. Such a project with its highly ambitious sector policy and capacity building components would be difficult even in much more organized countries. Earlier Colombia projects envisaged limited institution building, and some may even have been implemented well for good short- or medium-term gains. But in long-term sustainability, they failed, creating a situation in which H VII and HS I had little to build on. Thus, the two projects were confronted with MOPT's institutional weaknesses which stop-gap measures and crisis management could not easily and sufficiently rectify. MOPT's weak technical capacity reinforced the government's inherent indisposition to keep H VII and HS I on track and target. 5.7 The PCR for HS I suggests that implementation would have proceeded better if the Bank had shown more weight than it did. In its view, the threat of disbursement suspension, which was never carried out, would have brought the government into better compliance." This is doubtful. Threats and covenants, even when complied, rarely induce creative, sustained solutions to difficult problems. Rather, because the long list of "lessons of experi- ence" identified in the PCR for HS 126 were apparent during the implementation of the project, they could have been addressed without delay. Admittedly, some of these may have been be so difficult and persistent that they could not have been accomplished during HS I and had to be continued during the next loan. Apparently this follow-up regarding several of the "lessons of experience" was indeed done in the Second Road Sector Project. Recent Developments 5.8 After HS I, soul searching within MOPT/NHA, the government and the Bank has generated a new sense of urgency in shaping up and redirecting the sector. Neither govern- ment nor political circles misunderstand the importance of a well-functioning national roads system. This new or renewed "awareness", reinforced by private-sector interests in supplying roads infrastructure services, has produced or is producing action on several fronts: in roads maintenance, remuneration of NHA engineers and road concessions, and organizational 25. PCR, Highway Sector Project, paras 8.02 and 10.01 26. PCR, Highway Sector Project, paras 9.03 and 9.04. 37 changes. The thrust of the new initiatives is to "downsize government" (through reduction of direct government employment) and - while keeping and strengthening core functions in public sector agencies - to give the private sector a larger role in infrastructure management. This is line with current trends of making government "lean and efficient". At this point, it is too early to determine whether, in the Colombian context, the initiatives are paying off in terms of efficiency, timeliness and quality of road infrastructure supply. 5.9 MOPT/NHA has accepted the need to manage road maintenance better and to afford it high priority. They have granted maintenance organizational status, which it did not have before, though an even higher place in the organizational chart may be warranted. NHA is also studying an enlarged private sector role in contract maintenance, particularly periodic maintenance. To make progress on privatization (while also advancing government decentral- ization), it is dismantling its maintenance districts and seeking to turn their functions over to private contractors. One concept under review is to let companies, which would be formed by previous MOPT/NHA district staff supplied with previously MOPT/NHA-owned equipment, step into the vacuum. This arrangement would ensure that existing maintenance expertise and skills would be preserved and the pain of reducing direct government employment would be reduced. 5.10 Another post-HS I development, which augurs well for national roads, was introduc- tion of special and improved NHA pay scales for highway engineers. Without these scales, NHA would face great difficulties in attracting and retaining engineering expertise and delivering on its road infrastructure programs. 5.11 A further promising development was design of blueprints for road concessions to private sector entrepreneurs. Concessionaires would finance the road investments, maintain the facilities and charge tolls. Many countries are studying and some are implementing such schemes to ease their burden of investment financing and maintenance management. High- traffic roads are particularly suited for such schemes. Private sector interest in Colombia is high. The National Planning Department estimated the medium-term potential for concessions at 3,000 km of national trunk roads, many in the vicinity of urban centers. Such a length would be significant in terms of the relief to the national budget.17 5.12 Nonetheless, the sustainability of benefits for H VII and HS I is uncertain. Similarly, the sustainability for RR I is also an open question in light of the still unsettled issues of sector organization and also because some ingredients may still be missing. On balance, however, the Colombian national road sector does show progress in institutions, policy and finance over the 40 years of mixed performance, and there are causes for cautious optimism. 5.13 Recent developments in making road organizations function satisfactorily assert the centrality of institutions: organization, management, and financing." It is argued that once 27. National Planning Department, Avances del Programa de Concession de Carreteras, Document CONPES-2686 MINTRANS- PORT-DNP: UINF, January 1994. 28. Ian Heggie: Management and Financing of Roads: An Agenda for Reform, SSATP Working Paper No. 8, The World Bank 1994. 38 these institutional concerns have been adequately addressed, the performance of the agency is set on the right course and desirable outcomes will follow. 5.14 The organizational component in this thinking focuses on three issues: (i) defining the role and mission of the road organization(s); (ii) restructuring relationship between the government and the customers; and (iii) developing an appropriate policy framework by (re-)- classifying roads both functionally and administratively. The second of these three, organiza- tional restructuring (aiming at decentralization of the national government) is being implement- ed in Colombia. This restructuring also addresses the concern of accountability of the road organization to its customers by means of commercializing at least the NHA. The third issue, reclassification of the road network, is also done. These developments, carried out by the government on its own, are one set of reasons for cautious optimism. Even if the first phases of restructuring fall short in some respects, remedial, corrective, or augmentative actions can be undertaken later. 5.15 The managerial concern in the new thinking also focuses on three things: (i) improving management by improving decision-support systems; (ii) creating data information systems; and (iii) clarifying and requiring managerial accountability and responsibility. Regarding this issue, decision-support systems and data information systems and manpower training have been central Technical Assistance themes. These systems and associated management practices evolve slowly, are related to organization and ownership of roads, and, in spite of some concerns - such as lack of communication between departments, frequent executive staff changes, some managerial failings, and uncertainty of Government commitment - the work commenced in improving management also suggests that cautious optimism is warranted. 5.16 The final leg of an autonomous and responsive road organization is a stable funding source. Although the PCR for the HS I goes on record by calling the Road Fund experience in Colombia "disappointing",29 such characterization is hardly warranted. According to the PCR approximately 72 percent of the financing was to come from earmarked fuel tax revenue. In actuality this percentage (1982-1986) was 66 percent, and the funding shortfall in absolute terms was 20 percenteo. The amount collected was sufficient for rehabilitation, and for periodic and routine maintenance. Given Colombia's political situation in the eighties, the underdeveloped nature of the road network, and the governments inherent right to subsidize certain road works for social reasons, this contribution from the fuel tax revenue was amazingly good. It is unfortunate that the well performing Road Fund has apparently been discontinued. The motivation for such a development calls forth examination. There may be important reasons for restoring the Road Fund, in fact the reestablishment of the road fund is currently under consideration. 5.17 In sum, there are several reasons for cautious optimism. Technical Assistance in the current loans should focus on the three key institutional issues - organization, management, and financing - and develop incremental learning processes to achieve the objectives implied by them. 29. PCR, Highway Sector Project, para 6.01. 30. The planned income from the fuel tax revenue for the period 1982-1986 was US$1694.8 million. The actual fuel tax revenue for the period was US$1337.1 million creating a shortfall of US$357.7 million or 20 percent. 39 6. Project Ratings 6.1 H VII, HS I and RR I were a mixed project assortment by their outcomes. The national highway projects, which came on the heels of extended Bank lending to the same sector, did worse than RR I. 6.2 H VII had a lackluster performance, though it did squarely better than HS I. The sector project performed poorly on many scores, including road construction which other borrowers often deliver successfully even when country and sector conditions are grim. Performance ratings, however, strongly hinge on original project objectives, and thus a more modest formulation of goals would have put actual HS I accomplishments into a better light. The sustainability prospects for both projects are unclear. While the national roads sector shows progress in institutions, policies and finance, there is ambivalence -- concern and cautious optimism -- about progress after 40 years of lending. The audit's project ratings, together with the PCR-based ratings, are: Audit PCR Overall Outcome H VII marginally satisfactory satisfactory HS I unsatisfactory satisfactory Institutional Impact H VII negligible (not evaluated) HS I negligible partial Sustainability H VII uncertain (not evaluated) HS I uncertain uncertain 6.3 RR I had a good score on construction, and - with modest institution building objectives - a limited score on institutional impact. In road maintenance, the score was low, but not much was attempted either. Sustainability is an open question in light of the unsettled issue of sector organization. Even with a continuation of the status quo, project sustainability, especially for maintenance, would not necessarily be assured. The audit's project ratings, together with the PCR-based ratings, are: Audit PCR Overall Outcome marginally satisfactory satisfactory Institutional Impact negligible partial Sustainability uncertain likely  41 7. Conclusions 7.1 The Colombian projects are in a traditional Bank sector which has received long- lasting Bank support and where the achievements have been slow in coming and the outlook is uncertain. The projects audited mark a shift in the Bank's highway sector agenda -- H VII (approved 1977) made national roads maintenance the main project issue; HS I (approved 1982), while emphasizing maintenance, focused on policy and institutional issues; and RR I (approved 1981) was the first Bank transport operation in Colombia which dealt with the needs of non-national roads. The projects demonstrate the difficulties of using traditional Bank lending assistance for accomplishing institutional sustainability and the ultimate goal of making the sector autonomous. 7.2 The three audited projects, in combination with the earlier six projects, raise questions about objectives, methods and efficacy of past Bank interventions and about future Bank strategy in the sector. After implementation of ongoing lending, the Bank will have completed thirteen road sector projects, of which ten were exclusively directed at national roads; and it will have been associated with Colombian roads for 46 years - with mixed success. At this time, the audit suggests that, as part of the Bank dialogue with Colombia on highway sector issues, and in the context of preparing lending strategies and programs for the sector, an assessment be made of the three following questions: a) what are the highway sector management issues that require the most urgent attention, and which should be considered for the longer-term; b) what measures - including policy as well as institutional reforms - and implemented in what sequence, are necessary to address the priority issues, and c) what should be the Bank's role in supporting improvements to the managment of the highway system." 7.3 The following would provide guidance for defining the focus of the proposed assessment: * Highway sector issues: based on the analysis of HS1 and RR1, priority areas for consideration in the proposed assessment would be maintenance budgeting; highway safety; vehicle weight controls, and construction industry development. Other areas addressed in on-going highway projects, could be added if current efforts are not being successful. A program of staff training would be prepared to support the implementation of policy and institutional reform measures. * Measures and their sequencing: the assessment should take into account Colombia's experience as documented in ex-post evaluations (PCRs and PARs) and international experience both from developed and developing countries. Much successful experimentation has taken place over the last decade with reforms in highway management (eg. New Zealand, Scandinavia, Chile) which is likely to be relevant in Colombia. In particular, it could provide clues on rules for identifying 31. The Latin American and the Caribbean Region has noted that a Transport Strategy Paper, currently under preparation, will deal with policy, institutional and financial issues, and that the Region is folowing closely the establishment of the Instituto Nacional de Vias (National Highway Institute), with a view to exploring whether its transformation into a public corporation could help mitigate political interference, strengthen management capacity, and increase financial autonomy and accountability. 42 and establishing an effective sequencing of policy and institutional reforms. * Role of the Bank- the role that the Bank should play in Colombia in the highway sector should originate in the Bank's Country Assistance Strategy, and should take into account the accumulated knowledge of the highway sector in the Bank and the quality of the current dialogue. The Bank should clearly identify sectoral objectives to be achieved, should establish indicators to measure achievement of the objectives, and should define criteria under which the Bank should exit the highway sector in Colombia.  么＇ 潺＿, ，州 r1 。L。 ，。叩 ‘』一內 方口． 7二， 一磧以 ·‘他二訕 《亡必細 蕪中〉 r州」磚“一