Colombia - Second Rural Roads Sector Project and Third National Highway Sector Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No.: 20680 PERFORMANCE AUDIT REPORT COLOMBIA SECOND RURAL ROADS SECTOR PROJECT (LOAN 3157-CO) THIRD NATIONAL HIGHWAY SECTOR PROJECT (LOAN 3453-CO) June 28, 2000 Sector and Thematic Evaluation Group Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents (annual averages) Currency Unit = Colombian Peso (Col. $) 1990 US$1.00 Col. $ 497 1992 US$1.00 Col. $ 677 1995 US$1.00 Col. $ 875 1997 US$1.00 Col. $ 1,263 1999 US$1.00 Col. $ 2,014 Abbreviations and Acronyms DNP National Planning Commission FNCV National Rural Road Fund ICR Implementation Completion Report INDERENA National Institute of Renewable Resources, Ministry of Agriculture INVIAS National Institute of Highways LAC Latin American and Caribbean Region, World Bank MOPT Ministry of Public Works and Transport (up to 1993) MOT Ministry of Transport OED Operations Evaluation Department, World Bank SAR Staff Appraisal Report Government Fiscal Year January 1 - December 31 Director-General, Operations Evaluation Mr. Robert Picciotto Director, Operations Evaluation Department Mr. Gregory Ingram Manager, Sector and Thematic Evaluation Mr. Ridley Nelson (Acting) Task Manager Mr. Hernan Levy FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A Office of the Director-General Operations Evaluation June 28, 2000 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Colombia Second Rural Roads Project (Loan 3157-CO) Third National Highway Sector Project (Loan 3453-CO) Attached is the Performance Audit Report prepared by the Operations Evaluation Department on two Colombia projects: Second Rural Roads Project (Loan 3157, approved in FY90) and Third National Sector Highway Project (Loan 3453-CO, approved in FY92). The rural project closed on schedule and the national project closed six months late. The two loans totaled about $321 million, which was nearly fully disbursed at closing. The objectives of the projects were consistent with the Bank's assistance strategy for the transport sector: to help strengthen the management of the sector, to assist with the transfer of parts of the road system to the autonomous regional governments and departments, to improve environmental management and to support trade by reducing transport costs. The rural project focused its physical investments on the improvement and construction of rural roads throughout Colombia, while the national project financed part of a five-year highway rehabilitation, paving, and construction program. Following project restructuring, two components of the national project were canceled: (i) acquisition of road maintenance equipment was replaced by privately- executed road maintenance and (ii) improving ports in the Magdalena river was taken up by government's own funding. Both projects largely achieved their objectives, although implementation was affected by major structural reforms undertaken by the government in the transport sector: midway through implementation in the case of the rural project and soon after Board approval in the case of the national project. The reforms, which were remarkable in their scope and speed, essentially comprised the abolition (later put on hold) of the implementing agency of the rural project, the National Rural Roads Fund (FNCV), the decentralization of its core functions to the regional and local authorities, the transformation of the Ministry of Public Works and Transport into a Ministry of Transport, the creation of a new, autonomous road agency, INVIAS, that was put in charge of the national roads and became the implementing agency for the national project, and the devolution of former national roads and secondary roads to regional governments. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. -2- The rural project helped finance the rural roads network: construction of about 775 km of rural roads (97 percent of target) and improvement of 1,050 km (81 percent of target). The return on these investments was lower than estimated at appraisal, but still over 12 percent. The national project financed a large number of subprojects that were ongoing under a previous operation. Roads in the national network rated as good more than doubled during the project period to 76 percent. The economic return on the project-financed investments, based on a sample of civil works, was estimated at 36 percent (compared to a 38 percent appraisal estimate). Both projects intended to make a significant contribution to institutional development. The national project was, however, more successful, as it helped substantially in the process of launching INVIAS in its early years of operations, notably improve planning capacity and establishment of strong environmental unit. The project also helped set the rules for concessioning roads to private operators. The rural project's institutional goal were hindered by the erratic process of restructuring the management of rural roads, notably by government's decision, in part encouraged by the project, to freeze the decentralization process and keep in place a weakened central government agency. The outcome of the two projects is rated satisfactory. Institutional development impact is rated modest in the rural project and substantial in the national project. Sustainability is rated uncertain in both projects, mainly because the reforms are still not completed, the financial and technical capacity of the regional governments to take over remains an issue, and there has been some deterioration in road maintenance since the projects' completion. The audit acknowledges, however, that the creation of INVIAS, and the way its scope and functions are defined is a notable achievement that has the potential to bring much improvement in efficiency in the highway sector over the long term. Bank performance is rated satisfactory in both projects. These ratings are broadly consistent with those of the implementation completion reports. The following lessons emerge from these projects: Major highway management reforms are more likely to succeed under a good economic climate. Streamlining the management of the highway sector, making it more efficient and more commercially oriented normally requires downsizing, laying-off significant numbers of personnel. This is best done under a growing economy with good employment potential for laid- off personnel, and when complementary initiatives are put in place, such as support for creation of microenterprises to take over some maintenance activities while providing employment. The implementation risks posed by a large number of subprojects and contracts are difficult if not impossible to mitigate. Even when subprojects are underway or in advanced stages of preparation at the time of project appraisal, a large number of such subprojects scattered geographically taxes any highway administration. Measures to improve monitoring of procurement and execution will help but are likely only to minimally mitigate the risks involved. A concentration of subprojects on a few geographical areas, as the government and the Bank are now doing in the preparation of new projects focusing on a small number of corridors, appears to be a better approach. The devolution of responsibility for roads to local governments requires a detailed assessment of the ownership and financial and technical capacities by the receiving government. Failure by decentralization programs to carefully assess such ownership and capacities is certain to result in rejection of the new responsibilities by the receiving governments, leading to a reversal of the program or to a sustained deterioration of the devolved assets. -3- Benefits of organizational structure need to be preserved through performance benchmarks and a stronger civil service. The experience with the reorganization of Colombia's highway sector management confirms that the benefits of a good sectoral organization can be eroded when political changes bring about frequent changes in management. The use of objective, systematic and publicized measures of agency performance, coupled with a stronger civil service that leaves only a minimal number of managerial positions subject to political appointments would go a long way in preserving the benefits of organizational reform. Attachment  1 Table of Contents Principal Ratings ............................................................................................................................ ii Key Staff Responsible .............................................................................................................ii Preface ........................................................................................................................................... iii 1. Background .............................................................................................................................. 1 2. O bjectives ................................................................................................................................. 2 Project Preparation .................................................. 4 3. Im plem entation ........................................................................................................................ 5 Second Rural Roads Project ............................... ....................5 Third Highway Project....................6........... ................6 4. O utcom e.................................................................................................................................... 8 Physical Components ....................................... .......... 8 Institutional Components ....................8....... ..................8 5. Sectoral Restructuring and Decentralization ................................................................ 12 National and Secondary Roads ............................. .............. 12 Rural Roads....................................................... 13 Decentralization and Restructuring Issues.................. .................. 14 The Projects and the Reforms .......................................... 15 A Tentative Conclusion ............................................... 15 6. Ratings .................................................................................................................................... 15 Outcome, Institutional Development Impact, and Sustainability . ................. 15 Bank Performance................................................... 16 Borrower Performance ...................................... ......... 17 7. Lessons .................................................................................................................................... 17 Annexes: A. Basic Data Sheet .................................. .............19 B. Government comments .........................................23 This report was prepared by Mr. Heman Levy (Task Manager), who audited the projects during February-April 2000. Mr. William Hurlbut edited the report. Ms. Romayne Pereira provided administrative support. 11 Principal Ratings Second Rural Road Project (Loan 3157-CO) ICR EVM/ES* PAR Outcome Satisfactory Satisfactory, Satisfactory Sustainability Uncertain Uncertain Uncertain Institutional Development Partial Substantial Modest Bank Performance Satisfactory Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory Satisfactory * OED's Evaluative Memorandum or Evaluation Summary of the ICR. Third National Highway Sector Project (Loan 3453-CO) ICR EVMIES PAR Outcome Satisfactory Satisfactory Satisfactory Sustainability Uncertain Uncertain Uncertain Institutional Development Partial Substantial Substantial Bank Performance Satisfactory Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory Satisfactory Key Staff Responsible Second Rural Road Project (Loan 3157-CO) Task Manager Division Chief Country Director Appraisal A.Paz G.Smith Y.Abe Completion J.L.Jrigoyen P.Ludwig A. Solimano Third National Highway Project (Loan 3453-CO) Task Manager Division Chief Country Director Appraisal J.L.Irigoyen G.Smith Y.Abe Completion J.L.Irigoyen P.Ludwig A. Solimano 111 Preface This is a Performance Audit Report (PAR) for the Second Rural Roads Sector project (Loan 3157-CO) and the Third National Highway Sector project (Loan 3453-CO). The two projects totaled US$321 million and were nearly fully disbursed. The Operations Evaluation Department (OED) prepared this report based upon a review of the President's Reports, Staff Appraisal Reports (SARs), Implementation Completion Reports (ICRs), transcripts of Board proceedings, project correspondence files, Bank documents on other transport projects and other Bank material. OED also consulted Bank staff in charge of the projects. During March-April 2000, OED sought the views mostly through telephone interviews of borrower officials and other stakeholders. OED is grateful to the authorities involved for their invaluable assistance in the preparation of this report. The PAR gives special attention to the process and outcome of decentralization and restructuring in the highway sector, and.to updating ICR information with a view to ascertaining the projects' sustainability. Following standard procedures, copies of the PAR were sent to the Colombian authorities for comments. Feedback received is attached as Annex B.  1 1. Background 1.1 Social and economic conditions in Colombia have substantially improved over the past four decades thanks to strong economic growth - on average, 4.5 percent a year -made possible by prudent and stable macroeconomic management.1 Poverty declined steadily from 50 percent in the mid-1960s to 20 percent in the mid-1990s. Most other social indicators also improved. In the last years of the 1990s, however, socioeconomic progress stagnated. 1.2 Colombia's rugged topography, with three massive ranges of the Andes mountains running through two thirds of the country, presents a serious obstacle to communications and makes the cost of land transport high. This affects long-distance movements as well as access to services in rural areas. While the first hinders domestic and international trade, the latter is critical to the country's social objectives because extreme poverty in Colombia is mainly a rural problem, as 70 percent of the people with income below subsistence levels live in the rural areas. 1.3 Colombia's land transport system consists of (in round numbers) 110,000 km of roads (comprising the mostly paved national network of 26,000 kin, and the rest largely unpaved departmental roads and private and public feeder roads), 2,500 km of single-track railway, and 1,500 km of navigable waterways. This system was for many years managed by the central government with a traditional public sector mentality. The focus was mainly new construction, with less interest in maintenance, cost-effective operations, or meeting the specific needs of transport users. The key sectoral institutions were the Ministry of Public Works and Transport (MOPT), directly responsible for the planning, construction and maintenance of the country's national highway network, and the National Fund for Rural Roads (FNCV), an autonomous national agency responsible for rural road construction, improvement, and maintenance. 1.4 The 1991 constitution introduced significant new directions with major implications for the organization and management of public services. It has promoted a culture of results-oriented public sector management and has given the community greater freedom to choose the provider of services (state, private sector, or community organizations). The constitution strengthened the moves toward decentralization that had been launched in the previous decade. 1.5 In order to continue to fulfill the country's development potential, the government, in its 1994-98 development plan, focused on four critical areas of action: social development, competitiveness of the economy, environment, and decentralization and institutional development. Government policies in the transport sector in the 1990s were very much influenced by these national directives. These directives also influenced ongoing Bank operations, and set the framework for new lending. In its 1993 Country Assistance Strategy (CAS), the Bank established three main strategic areas: private sector development, poverty alleviation, and environmental management. Based on the experience with this strategy, the Bank, in its 1997 CAS, identified two main objectives, comprising six areas of intervention: * Poverty reduction and social development - comprising promotion of peace and development, promotion of rural development, and developing human capital * Sustainable growth - comprising attaining public sector responsiveness and efficiency, improving infrastructure services, and ensuring sustainable development. 1. This chapter draws largely from the 1997 CAS, the 1999 CAS Progress Report, and the SAR of the Third National Highway project. 2 1.6 The CAS Progress Report of November 1, 1999, notes that a number of factors contributed to recent deterioration of the macroeconomic outlook, leading to an increase in the budget deficit to about 6 percent of GDP in 1999. To tackle this situation, congress approved in mid-1999 legal amendments aimed to improve the fiscal situation, and others were well advanced for passage during 1999. Among these measures are several that are especially relevant to the transport sector and to decentralization objectives and policies: reform of the Budget Law to reduce earmarking and improve the allocation of resources; rationalization of public finances and curbs on their expansion; measures to curtail the current expenditures of local governments; and a constitutional amendment that would freeze, in real terms, future transfers to local governments. 2. Objectives 2,1 Both projects continued past Bank transport lending intended to strengthen Colombia's capacity to manage its road network and to improve the road system by helping finance the country's road development plans. The objectives of the projects (Box 1) were relevant to those plans, and the projects' components were generally well articulated to support the objectives. 2.2 The attention both projects gave to the decentralization of management of the road system is especially significant. That the rural road project, which was appraised three years ahead of the 1991 constitution, also contained this objective is remarkable. It signals a good understanding during project preparation by both the borrower and the Bank that decentralization was a worthy and realistic objective. At the same time, the absence of a strong constitutional mandate towards decentralization could have cast doubts on the feasibility of this objective. 2.3 It is also noteworthy that the rural roads project promoted balanced investment in new construction and maintenance. In contrast, the Third Highway project sought to reduce transport costs, a relevant objective in itself, but one that devoted much of the project funds to the construction or improvement of roads, consigning the funding of road maintenance to a covenant of the loan.' Project files provide no clue about why the two projects followed a different approach regarding the funding of road maintenance. The Third Highway SAR stated that "the overall allocation for maintenance has been reasonable." This may well be the reason that this project did not finance maintenance. Would the different approach to project finance of road maintenance make a difference in the outcome? This audit discusses this in Section 6 (under Bank performance). 2. As noted in the SAR, among the agreements reached at negotiations, "MOPT will implement the 1991-95 Road Maintenance Plan and will allocate the required amounts for road maintenance." (para 2.19) 3 Box 1. Project Objectives and Components Second Rural Roads Project Objectives. Help expand markets and reduce poverty in rural areas, specifically by helping: * Define road management functions and responsibilities of national, regional and local authorities * Ensure satisfactory implementation of key policies in the road sector, including decentralization of decision-making to local governments * Consolidate ongoing institution-building * Finance a balanced 4-year investment and maintenance program Components * Carry out road maintenance (including resurfacing of 4,400 km), improvements (1.650 km of dry-weather roads plus bridges), and new construction (950 km) * Spare parts for road maintenance equipment * Studies, training and technical assistance Third National Highways Project Objectives * Reduce transport cost, especially for the long-distance, export-import traffic * Improve the quality of project preparation and execution in the sector * Improve road maintenance planning and practices * Assist in the process of decentralization of the national road system * Improve the capacity for environmental management of road projects Components * Completion of about 2,000 km of roads (started under previous Bank project or MOPT's own funds) * Construct 12 new roads for about 400 km * A bridge rehabilitation program * A river training and dredging program * Purchase and rehabilitation of road maintenance equipment. * Road safety works * Technical assistance, studies and training (organization, management, engineering, supervision and environment) 2.4 To its credit, the Third National Project had as a specific objective the improvement of capacity for environmental management of road projects. This may have been in part the result of experience gained with the Second Rural project (which had been underway for some three years when the Third Highway project was appraised) where environmental issues posed many problems during implementation. In the preparation of the rural project, a call of attention early on during preparation: "this project could be among the environmentally problematic in Colombia's lending program",3 led to a substantial strengthening of the project's environmental dimension. Such strengthening included, as noted in the project's SAR, an agreement regarding use of "... procedures that integrated environmental and economic considerations, from initial screening stage to final selection and inclusion in the project." In practice, as noted in Section 3 of this audit, compliance with this agreement was weak. 2.5 It is also important to note that while the project had no objective relating to road user charges, it contained covenants requiring the government to adjust gasoline prices and to maintain an appropriate level of road user charges.' These covenants had the potential for significantly improving economic efficiency in the use of the road system. 3. Internal memorandum from the LAC's environmental division to LAC's infrastructure division dated August 8, 1989. 4. On the first, the requirement stated that the government would adjust retail prices of gasoline so that, on average, it would be equal to the border price level. On the second, the requirement was that the government put into effect and maintain user charges at a level that would gradually increase the coverage of the public road system (excluding new construction) until reaching 100 percent during 1994. No mechanisms existed to ensure that such user charges would be funneled back for road maintenance. 4 Project Preparation 2.6 As already noted, both projects were identified and prepared under preceding operations: the rural project was initially identified during the supervision of the Rural Transport Project (Loan 2668), which helped finance investment and maintenance program of the FNCV. The national project was a follow-up to the Second National Road project (Loan 2829). 2.6 While both projects aimed to continue the institutional strengthening launched by the preceding projects, the national project also aimed to provide financing to complete a significant number or road works initiated but not completed under the preceding project. Therefore, a substantial part of the project's investments were carry-over of works already under contract or in advanced stages of bidding preparation. 2.7 Environmental aspects were singled out from the beginning as potential issues in both projects, but especially in the rural project. LAC's environmental unit noted that a post-appraisal should be carried out to identify those rural roads that posed significant risks of increased deforestation (because, for example, they would pass through or near forest reserves or other protected natural areas) and that roads so identified should either be deleted or would need a special environmental protection or mitigation component. In the national project, it was noted that this was the first time a Colombia project (of any sector) required project-financed investments to be classified according to environmental categories. 2.8 During processing of the national project, before and after appraisal, there was significant discussion about the low retail price of gasoline and need to require the government to eliminate the implicit subsidy. In mid-1991, the price of gasoline and diesel at the consumer level was about US$0.59 per gallon, compared to the border price of about US$0.67. This was the only outstanding issue where the preappraisal mission (later reclassified as the appraisal) sought management guidance. The decision was that government should bring gasoline prices to the level of international border prices before Board presentation, at that government would adjust prices of gasoline thereafter twice a year to keep retail prices in line with border prices. 2.9 A special feature of the national project was the requirement that the MOPT sign with the government a Performance Plan contract with specific targets and deadlines, covering investments as well as institutional and managerial measures. This was the first time a Colombia transport project loan required such a plan. The audit finds that this approach was generally consistent with Colombia's approach to public sector management, since an entity depending from the Ministry of Finance, the National Council of Economic and Fiscal Policy (CONFIS) was specifically charged with the preparation, negotiations, and monitoring performance of performance plans with public sector agencies. However, as noted in the ICR for the Public Sector Reform Loan, CONFIS found this activity too absorbing and transferred this function to the line ministry, except in the case of entities that had a special fiscal or policy significance. Roads represented a special situation, since there was no road agency at the time of project preparation and road management was directly under the MOPT. In this case, the government decided that the contract would be signed by MOPT with CONFIS and the National Planning Department (DNP). As noted, CONFIS soon abandoned its role on this. 5 3. Implementation 3.1 The major economic. and administrative reforms of the early 1990s affected both projects significantly. Key sectoral institutions were dismantled or severely reduced, while new ones with different mandates and philosophies were created. These reforms affected a large number of projects in the Bank's Colombia portfolio, requiring them to change the borrower or implementing agency, appraise the capacity of a new implementing agency, or restructure the projects.' 3.2 During implementation of both audited projects, the Bank had to decide whether to restructure the projects to fully accommodate the new government directives on decentralization and sector restructuring. In the rural roads project, the Bank decided against restructuring, on the basis that it would have hampered execution of ongoing works and that most institutional development activities had been completed. Further, the project already comprised objectives and components aimed to support decentralization. The national project, which was just being launched when the new measures took place, was restructured just after effectiveness. 3.3 The early years of implementation of the two projects happened just after the government launched its Public Sector Reform Program. The Bank supported this program with a Public Sector Reform Loan. 'This government program sought to improve productivity in the use of public sector assets and to improve efficiency in resource allocation in sectors traditionally dominated by the public sector. While this program covered transport activities in ports, railways and shipping, it did not cover road transport infrastructure. The ICR for this project (July 24, 1995) showed that significant gains were achieved in public sector management, notably in improved planning and accountability in the main public sector enterprises. The reforms in ports and shipping eliminated highly inefficient monopolies and generated new opportunities for private sector participation. It is likely that these reforms helped created an environment for further reforms in the transport sector at a time when both the rural and the national road projects were being implemented. Second Rural Roads Project 3.4 Implementation of the Second Rural project was severely hindered by the government decision in 1993 to abolish the project's implementing agency, the FNCV, and to transfer its functions to the regional governments. Thus, during the later years of implementation, the FNCV was in a transitional liquidation status. In 1995, the government suspended the liquidation of FNCV. 3.5 One difficult hurdle during implementation was ensuring compliance with environmental regulations. This task involved diverse entities. On the government side, the FNCV had to seek approval from the government's environmental agency at the time (INDERENA, in the Ministry of Agricultare, prior to the creation of the Ministry of the Environment in 1995) and with the regional governments submitting the individual road projects. On the Bank side, both the operations and the environmental divisions were involved. During project implementation, a working relationship gradually evolved that helped compensate for the initial delays while at the same time ensuring compliance with environmental regulations. (Box 2 in Section 4 illustrates this.) 5. As noted in the ICR of the Second Rural Roads project. 6. Loan 3278, approved December 11, 1990. 6 Third Highway Project 3.6 Initially, it appeared that implementation of this project would proceed smoothly. This was expected since a large number of the project-financed civil works (totaling US$148 million) were for works either underway - launched with financing under Loan 2829 or with MOPT's own financing - or were in advanced stages of preparation for procurement. To prevent the sort of delays and weak monitoring that happened in earlier transport operations, the project included a Performance Plan, to be signed by the government and the MOPT, that contained a broad set of sectoral performance targets for both civil works and institutional objectives. 3.7 Before the project was declared effective,7 however, a government decree in December 1992 restructured the transport sector. The decree converted the MOPT into a Ministry of Transport (MOT) and provided for the elimination of the MOPT regional districts within three years. More significantly from the project's standpoint, the decree eliminated the National Highway Fund, which was the borrower for this project, and replaced it with the newly created National Institute of Highways (INVIAS). These changes required the Bank to decide whether to accept INVIAS as the new borrower and executing agency or to cancel the project. 3.8 The consequences of the sector restructuring for project implementation were that the loan documentation had to be amended, a new Project Agreement had to be signed with INVIAS, the capacity of INVIAS for executing the project had to be appraised, and some changes had to be made in the project. Notably, the decrees set new, market-oriented policies that affected road maintenance policies. It eliminated road maintenance by force account, thereby making senseless the project's targets in this area, and it eliminated the MOPT districts' maintenance functions, making the planned purchases of maintenance equipment under the project irrelevant., 3.9 The Bank decided that since the transition from the existing to the new sectoral structure was planned to be gradual, with the National Highway Fund not expected to disappear until the end of 1993, the project should proceed while INVIAS was being appraised. In retrospect, this audit validates that decision. During implementation, Bank missions appraised INVIAS several times, as initially the Bank found that the agency was not properly staffed and that it did not have adequate financing controls over ongoing contracts. When these problems were corrected, legal documents were changed to make EIVIAS the executing agency. 3.10 Despite the difficult sectoral framework, project implementation proceeded and the fact that many contracts were underway at the time of appraisal was instrumental in facilitating execution of the works. However, the atomization of the works into many different contracts made very it difficult for the road agency to supervise these works.'o 3.11 There were significant problems in the early part of project implementation, when works were still under MOPT's responsibility. The MOPT's own internal control unit highlighted the following problems": 7. Effectiveness was declared on April 13, 1993, when the two conditions of effectiveness, establishment of a project implementation unit and establishment of an environmental unit within the MOPT, were met. 8. Decree 2171 of December 30, 1991. 9. An excellent analysis of the impact of sector restructuring on project implementation is found in the internal memo from Irigoyen to files of January 25, 1993. 10. As reported in INVIAS' evaluation of the project included in the ICR. 11. Letter by the Chief of MOPT's Internal Control to the Bank dated December 7, 1993. 7 * "No rigorous analysis is done of the proposals submitted by contractors that allow to know the real financial and technical capacity of the firms, as well as their provision of construction equipment. This has led to delays in implementation, poor quality of work, and cost overruns. * Poor controls and information systems have led to some 700 ongoing contracts under termination procedures that have serious financial and accounting problems, such as advance payments that are not fully legalized. * [Often] advance payments are made for up to 100 percent of the value of the contract, which is against [Colombia's] procurement rules, signals the lack of liquidity of the contractors, and suggests lack of care by the authorities in evaluating the proposals and the financial capacity of the selected contractors." 3.12 INVIAS inherited these problems, which made its startup period very difficult. In particular, the cost overrun problem haunted the project for a long time. As a Bank mission reported, it was surprising that there were significant overruns in subprojects whose contracts had been initiated under the previous project (Loan 2829) and had been under execution for more than 10 years. The overruns, which were mostly attributed to the addition of new works and changes in design standards, had reached 27 percent, and the Bank had not received technical justification for such additions nor had it received copies of the amended contracts. Further, several contracts (started under the previous loan) had reached their approved disbursement ceilings. The Bank at this point threatened to suspend disbursements for these works until the situation was corrected." As a result, for the first time since the start of project implementation, the Bank's supervision mission of May-June 1994 rated implementation of the project as unsatisfactory. 3.13 The restructuring of the road sector necessitated considerable expansion of the institutional development program during execution. New studies were added to strengthen support to the decentralization process, to follow up work on road user charges, to evaluate the condition of the national road network, and to strengthen the economic and financial analysis of the road concessioning program. 3.14 Another major change in the project was the elimination of road maintenance equipment, since INVIAS was not to perform direct maintenance operations. While rehabilitation of some equipment required to carry on during the gradual disbanding of the MOPT regional districts was retained under the loan, the method of execution was changed from direct financing of spare parts to contracting out. This was an interesting turn towards expanding privatization to all areas of maintenance. 3.15 The purchase of weigh scales was delayed because of the need for substantial clarification about the purpose and types of equipment to be provided. For example, a choice had to be made between fixed, static scales; portable, static scales; and dynamic scales, and the roles and locations of each type had to be specified before the procurement could be finalized.13 12. Supervision Report dated August 11, 1994. 13. A Bank letter to MOPT dated July 1, 1993, provided detailed explanations about the use of scales. 8 4. Outcome Physical Components 4.1 Civil Works. Both projects largely achieved their physical objectives. The rural project came close to achieving its stated goals, as it reached 97 percent of the original targets (775 km built) for construction of rural roads, and 81 percent of the targets for road improvement (1,050 km improved). The shortfall can largely be attributed to the delays introduced by the FNCV liquidation program and the more stringent environmental regulations introduced by Regional Environmental Corporations, which stalled execution of several projects and delayed others. 4.2 The national project achieved 92 percent of planned targets, as 2,200 km of roads were constructed, rehabilitated, or paved. More significantly, the percentage of paved national roads in good condition increased from 37 percent in 1994 to 76 percent at the end of the project in 1997. (The percentage of roads in good condition declined to 69 percent in 1999, still significantly better than at project start.) This outcome is especially important as this project did not include funding for maintenance, which was to be fully funded from the national budget and committed through a project covenant. 4.3 Economic performance. The economic rate of return also was satisfactory for both projects, 36 percent for the national project (slightly less than at appraisal) and somewhat over 12 percent for the rural project (lower than at appraisal but still satisfactory). 4.4 Weigh scales. The program to supply, install, and operate weigh scales to collect data and protect road pavement through enforcing truck loading limits achieved much less than expected. While this was a small component of the nation project (about US$1.0 million) it could have had a significant impact on reducing pavement deterioration. However, during implementation it became evident that there was little clarity regarding the purposes to be attained with the purchase of weigh scales. While the obvious ultimate objective was to control truck overloading (with its benefits for pavement life), this requires not only equipment but also an established institutional system to use the equipment and enforce the existing regulations. While enforcing weight limits is a universal problem, and few countries have been successful in this endeavor, there is little evidence that highway authorities were aware of this problem or that they knew how to deal with it." At the operational level, only a small part of the original target for controlling road freight was achieved: 29 percent in 1998, and 6 percent in 1999. Institutional Components 4.5 Sector Restructuring and Decentralization. The reforms in both these areas were significant, particularly regarding the management of national and secondary roads. The reforms regarding the national highway network probably were implemented too fast, considering the time it takes to put in place a new institutional set up and to train managers and staff to operate under the new conditions. The reforms, which were supported by but not triggered by the projects, and the process they underwent, are described in the next section of this audit. In essence, their outcome was: 14. According to project files, the first mention of the problem with enforcing weight control is a statement by the Minister of Transport to a Bank mission in 1993, about two years after project appraisal. 9 * Transformation of Ministry of Public Works and Transport into a streamlined Ministry of Transport * Elimination of the National Highway Fund and transfer of responsibilities for the national highway network to a newly created agency, INVIAS, which contracts out all works and operates with little staff * Transfer to the regional governments of a part of the trunk roads previously managed by the central government (MOPT) * Transfer to the regional governments of a part of the rural roads previously managed by the central government (FNCV) 4.6 The first two reforms took place and, while they had some initial problems, they remain solidly in place. The second two have been made, but they are not completed mainly due to the regional governments' claim that they lack the resources to take over responsibility for these roads. A fifth element of the reform package, creation of a large number of microenterprises to take care of the maintenance of the rural road network, has been put on hold as FNCV's resources have been diverted to direct employment generation by the municipal governments. The difficult fiscal situation of the country in the past three years, and the pervasive problems in conflict areas have probably been instrumental in hampering further progress in the reforms. 4.7 In the rural project, project-financed studies and technical assistance helped establish a new road classification system that was useful to define responsibilities among the various levels of governments prior to the reforms. However, the usefulness of these works was hampered because they failed to address the financial issues associated with the transfer of responsibilities. 4.8 In the national project, due to the addition of studies and technical assistance not originally contemplated, the institutional strengthening part of the project, which was already substantial at appraisal, ended up costing more than twice as much as foreseen in the appraisal report (US$11.2 million versus US$4.9 million expected). The audit finds that the expanded program was effective and had significant impacts. Of particular note are two added studies that helped the road concessioning program: one helped improve the economic and financial analysis of the road concessioning program, which helped review new concessions and launch a second phase of the concession program; the other derived realistic construction costs for the Tobiagrande-Puerto Salgar Road Concession Project,"s which appears to have been a critical factor in the successful award of the concession. Another important study evaluated the condition of the national road network using international standards on road surface." 4.9 Several other studies were completed in the national project but had less or little impact. In particular, a study on decentralization, which identified major areas for the institutional strengthening the highway management capacities of departmental governments, was completed but not used, at least initially, because the transfer of national roads to the departments went ahead before the end of the study. 4.10 Performance Plan. While the Performance Plan attained the objectives to provide targets and a way to monitor progress, its success as an instrument with the broader mandate to establish a mechanism for the periodic exchange of views among the key sectoral agencies was marred by erratic updating of the plan and little initiative to use this document for sectoral dialogue. Several 15. This road was the most expensive section ($440 million out of a total project cost of $570 million, according to SAR estimates) of the road concessioned under the Toll Road Concession Project. 16. The study measured the IRI, or International Roughness Index. 10 of the persons interviewed for this audit stated that they were unaware of the existence of the Performance Plan. 4.11 Financing. The two projects were mutually complementary as both included studies. In particular, a study on road user charges launched under the rural project, which recommended changes in road user charges structure and proposed reviewing funding sources at the subnational level, was followed up by a second phase under the national project, which focused on financing mechanisms. This study recommended the creation of a national gas surcharge and other sources to be distributed to the regional governments through a new road fund, to be created and administered by the private sector. These studies were instrumental in developing an understanding of road user charges, and for the decision of the government to institute the recommended gasoline surcharge. However, current economic policies to address the tight fiscal situation of the country have precluded the setting up of a road fund. 4.12 Environment. Both projects made a significant contribution in this area. This project required, as a condition of effectiveness, the creation and staffing of an environmental unit. Such a unit was created at MOPT, and currently the subdirectorate for the environment in INVIAS comprises a multidisciplinary team of some 50 professionals. INVIAS now requires the conduct of environmental as well as social assessment studies at the project planning stage and has developed appropriate documentation as guidance for such studies. The Second Rural Roads project, although it did not contain any specific objective or component focused on environment - because most of the physical investments were short rural roads - ended up being very helpful in strengthening Colombia's capacity for environmental assessment and action plans. This strengthening happened mostly through the Bank-government (FNCV and other agencies) dialogue during the project, as exemplified in Box 2, and the project's legal covenant on environmental issues. 11 Box 2. Bank-Government and Internal Bank Dialogue on Environmental Issue: The Case of the Mitu-Monfort Road This box chronicles the exchanges between Bank and government agencies and within Bank units regarding the Mitu-Monfort road and illustrates how such exchanges conveyed knowledge and led to the deletion of the road from the project. August 8, 1989, internal memo from the environmental division (during preparation): The project could be among the environmentally problematic ones in the Bank's Colombia lending program. November 19, 1990, internal memo from infrastructure division (during preparation): The Bank is developing software that will facilitate control... including of environmental studies. (Project files are not clear whether such software was actually developed). November 19, 1990, internal memo from the environmental division (during preparation): provides list of projects identified as risky by LATEN. The Mitu-Monfort road is one of the risky projects. October 29, 1991, letter from FNCV to the Bank: INDERENA will only approve construction of the [Mitu- Monfort] road's first 56 km. The remainder, about 50 km, will be excluded from the program because the ecosystem in that part in very vulnerable. October 30, 1991, letter from Bank/infra to FNV: In view [that the Mitu-Monfort] road will traverse a federal forest reserve and that road will pass through indigenous communities, Bank requires to see a copy of the environmental assessment for Bank review. June 5, 1992, internal Bank memo from environmental to infrastructure division: This detailed memo explained the potential problems, clearly specifying Bank requirements and critiquing the environmental assessment study. The memo recommends that the Bank withhold 'no objection' to this road. Other highlights of the memorandum: * The entire right-of-way proposed lies within existing forest reserve * The Environmental Assessment fails to quantify the probably amount of colonization in terms of (i) probably number of new colonists, and (ii) the amount of forested land to be cleared, (iii) definition of compensatory forest conservation component for the expected amount of deforestation (as estimated by INDERENA) * The EA quantifies the amount of indigenous people (about 97% of areas rural population), but gives no indication whether people along road were consulted and/or support the project, or whether an Indigenous People Development Plan has been developed * The EA provides no indication how the measures suggested in the Environmental Mitigation Plan will be funded, or a chronogram for implementing the measures July 23, 1992, Supervision Report, Aide-Memoire: The Bank mission issues its objection to the Mitu- Monfort road and it is excluded from the program. 4.13 Training. The rural project had a meager training program, as the budget for training was severely curtailed and limited to US$0.2 million, or about one fifth of the SAR estimates. The curtailment was due to the reorganization and staff retrenchment of FNCV during the process of liquidation, and to obvious changes in the institutional priorities for FNCV's management. The sectoral project financed assistance of several staff of MOT and INVIAS to PROVIAL seminars in Latin America. Officials reported to the evaluator that these seminars were instrumental in creating a "maintenance culture" among engineers who traditionally assigned high value to construction and little to maintenance. Exchanges with other colleagues from Latin America, including from countries that were carrying out important institutional reforms, made possible by these seminars, were given high marks by government officials. 12 4.14 Gasoline subsidy. Elimination of the subsidy on gasoline prices was also a direct achievement of the national project. This was initially an agreement at negotiations that stated government's commitment to adjust retail prices of gasoline price throughout the year so that on average they would be at least equal to the border price level. This important measure was complied with throughout the project. Further surcharges to the price of gasoline were added in 1998 as part of the macroeconomic adjustment launched by the new government. 5. Sectoral Restructuring and Decentralization 5.1 A major restructuring of the highway sector, together with devolution of functions from the central government to the regional governments occurred during the implementation of the projects. The two processes are intertwined and required parallel planning and measures. These radical changes were of major interest to the Bank and are therefore worth detailed analysis. 5.2 The reforms were initiated by the 1991 constitution, which provided further stimulus to decentralization programs that commenced in the earlier decade. The constitution granted the executive branch of government transitory but sweeping powers to reform government via the use of decrees that bypassed the need for congressional approval. The constitution also changed the culture of public service provision towards more efficiency and commercial orientation. 5.3 The main elements of the reforms were: * The dismantling of the Ministry of Public Works and Transport * The creation of a new highway management agency, the National Highway Institute, INVIAS, that would take over the MOPT's responsibilities for the highway sector * The elimination of all maintenance works done by force account, and replacement by contracting out * The intended elimination of the government agency responsible for rural roads, the National Fund for Rural Roads, FNCV * The intended transfer of all secondary roads and all rural roads to the regional governments 5.4 The status of decentralization is summarized in Table 1. Table 1. Status of Decentralization Road networks Government agency Situation at project Situation as Situation in March under central responsible appraisal (national, reported in the ICRs 2000 government 1992; rural, 1989) National Roads MOPT (at appraisal) 26,300 km 12,900 kma 16,500 km INVIAS (from 1993) Rural Roads FNCV 24,300 km 16,000 km 23,000-34,000 kmb a. The ICR only reports that 13,400 km had been transferred in 3 years. The figure in this table is the difference. b. Law 188 of 1995 halts the transfer of roads from FNCV to the regional authorities. As reported by FNCV, there is discrepancy in the size of the rural road network currently under FNCV. DNP estimates it at 23,000 km, while FNCV's data shows 34,000 km. National and Secondary Roads 5.5 Decentralization of the network traditionally managed by the MOPT followed the mandate established in the decrees dissolving the ministry and creating INVIAS. The latter decree specified that the new agency would retain only some of the road network originally under the central government. 13 5.6 Of the some 25,000 km of the national road network, about 10,000 kin, or close to 40 percent, was transferred to the departments, with INVIAS retaining some 15,000 km that became the new, downsized and reclassified national road network. 5.7 The changes in the management philosophy of the trunk roads - the complete elimination of force account works and the contracting out of all road works - were radical. Few countries can make these kinds of radical changes. They appear to have been possible in Colombia because the country had a well-developed contracting industry and because the reforms stimulated existing micro-enterprises and the creation of new ones for the execution of routine maintenance works. In recent audits, OED has found that in many countries where such restructuring has taken place, for example, in the former socialist economies or in African countries, and where construction industry was relatively little developed, a gradual 5-10 year program needed to be put in place. Rural Roads 5.8 The restructuring of the highway sector and the decentralization policies also affected the central government's rural road organization, the FNCV. " The FNCV in 1993 was officially requested to liquidate. Two years later, however, apparently as a result of strong congressional support for FNCV during parliamentary discussion of the National Development Plan, which involved a review of rural roads policies and the role of the government, the FNCV liquidation was suspended. By that time, FNCV had already transferred about 8,000 km of rural roads, or one third of FNCV's road network, to the regional governments. Thereafter, FNCV restructuring continued, with further downsizing of its staff and reduction of its regional offices. By early 2000, FNCV's staff was 200, compared with 2,300 before the restructuring process started. 5.9 As Table 2 shows, the FNCV has not been able to achieve the goals it had set for itself and described in the ICR for that project. Table 2. Status of FNCV Liquidation Target current Status (April2000) Reorganization of FNCV FNCV was further reorganized in 1997. Its staffing was further reduced to 200 staff (versus 2300 at start of process in 1993). Distribution of responsibilities Not achieved. The departments are still not accepting to take over more length of between FNCV, RCF, departments, rural road network on oasis that the central government failed to transfer the and local governments required resources. Creation of microenterprises for Not achieved. The goal to create some 900 new microenteprises through 1998 rural road network was not achieved, as less than 20 were actually created. FNCV following national policies has redirecteo efforts towards creation of employment through direct recruitment by the municipal governments (Programa de Generacion de Empleo Comunitario Temporal) 5.10 The current situation of FNCV is judged by most to be unsustainable - some officials referred to the present FNCV as being in 'agonic' state. There appear to be two broad possible courses of action: Complete the liquidation of FNCV, transferring all its roads to the regional governments, with government retaining some monitoring, advice and financing responsibilities to be vested in the MOT or in CINVIAS 17. The ICR for the Second Rural Roads contains an excellent summary of Colombia's decentralization and cofinancing policies as they affected the rural road system. 14 Bring FNCV back to a condition where it can effectively discharge its responsibilities. 5.11 The auditor's discussions suggest that the first course of action would be the most effective. Yet, this may be a case of missed opportunity (not uncommon in policy reform), at least temporarily. While liquidation of FNCV and complete devolution of FNCV's road network to the regions appeared as politically feasible when first attempted, and could have been achieved if implemented within the originally-proposed timeframe, changed political and economic conditions probably make it difficult today to complete the decentralization process. Decentralization and Restructuring Issues 5.12 The decentralization process faced major obstacles regarding the roads devolved to the regional governments. First, there was lack of technical capacity in many of the departmental highway agencies for managing the maintenance of the secondary roads transferred to their jurisdiction. This suggests that this process probably should have included a strong program of technical assistance to the departments. While some assistance was provided, it did not prove effective. 5.13 Another major problem, still unresolved, is the funding required by the departments to maintain the new roads. The original transfer to the departments of various public sector responsibilities (including roads) was accompanied practically pari passu with the transfer of central government budgetary resources to the departments. Initially, the central government arranged for the transfer of block grants corresponding broadly to the funds it allocated to each department, but without the obligation for the departments to earmark such funds for the maintenance of the road system. It now appears that most departments allocated the new resources to different uses, not necessarily to take care of the new responsibilities. In particular, the roads are receiving significantly less funding overall than when they were a central government responsibility. On the other hand, the central government, in addressing the fiscal crisis, is looking to get back from the departments some of the resources it had transferred to them. 5.14 The central government, through a law passed in 1998, attempted to correct this problem by allowing the departments to establish surcharges on gasoline prices and earmark such revenues for road maintenance. However, as reported by officials interviewed for this audit, the Supreme Court has recently ruled such earmarking illegal. This appears to be in stark contrast with what happened in other sectors, notably in education and health, where the decentralization process involved transfer to the departmental budgets of central government budgetary revenues that were specifically earmarked for such sectors. 5.15 One significant result of the financing and capacity problems mentioned above is that there is now a process of "transfer back" of secondary roads from the departments to INVIAS. These roads, not originally intended to be part of INVIAS mandate, are being labeled "the expanded network." It is estimated that some 3,600 km of roads are in the process of being returned to central government administration. Most of the returned roads are non-paved and come from diverse geographical locations and departments, suggesting the overwhelming problem is financial, as clearly some departments are endowed with better technical skills that presumably could have absorbed the devolved roads had the financing been available. INVIAS reports that the majority of the roads being returned are in poor condition, having received little or no maintenance. 15 5.16 In the case of rural roads, ownership was the key problem with the devolution of rural roads to the regional governments. This process was hindered because most of the roads transferred were small ones that mostly provided services inside municipalities, rather than served the needs of the departments. The departments have no constituencies for such roads. Thus, these roads should have been entrusted to the municipal rather than to the departmental governments. The SAR of the rural project had clearly specified that roads to be transferred were municipal roads, and suggested that these roads would be entrusted to the corresponding administrative authority at the local level, that is, the municipal levels. The Projects and the Reforms 5.17 Objectives. Assisting with the decentralization of the road system was a specific objective of both projects. The national project aimed to decentralize the national road network (consisting of about 26,000 km at the time of appraisal) and the rural project aimed to decentralize the rural road network under the FNCV (consisting of some 25,000 km at appraisal). The project financed a decentralization study, but, as noted in the ICR, transfer of the national roads to the regional governments went ahead without major planning and was defined before the end of the study. 5.18 Anticipated Risks and Suggested Mitigation Measures. The rural roads projects had anticipated as a project risk "the ability and readiness of local governments to organize, develop technical capability and effectively manage the local road network." The SAR proposed to reduce this risk by drawing up, implementing, and monitoring action programs intended to achieve full-fledged transfer of road maintenance responsibilities, and by linking FNCV cofinancing to performance in implementing the action programs. Surprisingly, the third project did not identify any specific risk linked to the decentralization program. This may have been due in part to the fact that this project limited its assistance with decentralization to the preparation of a study to plan the decentralization process, and included no specific targets, regarding either decentralization or sector restructuring. A Tentative Conclusion 5.19 This audit concludes that despite the problems, the sectoral restructuring and decentralization program was valuable and, compared to most other countries, can be considered successful. Some key features of the reform, notably the creation of an autonomous agency that proved highly effective (at least in its early years of operation) and the contracting out of all central government-managed highway activities, remain in place and are unlikely to be reversed. 6. Ratings Outcome, Institutional Development Impact, and Sustainability 6.1 The audit agrees with the ICR in rating of outcome of both projects as satisfactory and in rating institutional development impact as substantial in the national project. The audit rates institutional development impact as modest in the rural project, which is consistent with the ICR "partial" rating. However, this rating downgrades OED's previous evaluation (EVM) because the reforms suffered major setbacks, the process is still not completed, and the contribution of the project to this process was minor. The vision emerging for the management of the FNCV- 16 developed rural road system is one more decentralized to the regional governments than foreseen by the project and the activities it supported. 6.2 The audit agrees with the ICR in rating sustainability of the rural roads project as uncertain. Information gathered by the audit reveals that the institutional reforms are still in a state of flux. In particular, the transfer of responsibilities to the regions has not proceeded as planned, apparently because of lack of resources on the municipal and departmental levels. At the same time, FNCV's own situation remains unclear. Further, creation of microenterprises for maintenance, a pillar of the decentralization and privatization program, has not progressed as efforts have been redirected to finance employment generation programs contracted directly by the municipal governments. 6.3 The audit also confirms the ICR's rating of the national project as uncertain. This is a difficult assessment because INVIAS as an institution appears to be solidly established. But two important indicators support retaining the uncertain rating: first, the condition of the network has deteriorated since project completion, and second, there have been several changes in the management team of INVIAS in recent years, creating instability that is affecting the performance of the agency. Regarding the decentralization of the secondary roads, while it has taken hold, the departments did not receive adequate preparation for this process, nor do they have a sustainable funding system to finance the maintenance of their road networks. Bank Performance 6.4 Bank performance during identification and preparation was satisfactory, although in the national project the Bank apparently underestimated the complexities of a project with a very large number of subprojects located throughout the country. The improvement in the condition of the road network despite direct project financing to it vindicates the approach taken at appraisal to rely on government funding for road maintenance. 6.5 During implementation, the Bank closely monitored progress of the projects and the institutional reforms and the impact of project execution and development objectives. Bank missions in both projects established close and credible working relationships with government officials and provided sound advice. Audit discussions with government officials (past and present) confirm the claim of the ICR for the second project that, in part thanks to Bank missions' advice... "the government in 1995 acknowledging the difficulties of implementing the decentralization in the continuous absence of local technical, financial and administrative capacity, suspended the liquidation of the FNCV, cancelled transfer of rural roads, and established a financing systems that would take into account the capacity of the departments." 6.6 This audit finds that freezing the dismantling of FNCV may have helped implement the project and even provide for a more orderly decentralization process over a longer period of time. While it appears clear that the Bank fully supported the decentralization policies, it also appears that putting on hold the demise of FNCV may have resulted in a loss of momentum for the decentralization process. 6.7 The Bank pondered whether to cancel the national project in light of the drastic sectoral restructuring taking place just after loan signing. In deciding to proceed with the project, however, the Bank judged that the reforms merited support and that the project, with appropriate restructuring, could be useful in supporting the government reforms. It is undisputed that the reforms, notably the creation of INVIAS was a major achievement and that the Bank was right in remaining with the project and lending assistance to INVIAS. 17 6.8 The Bank prepared generally excellent ICRs for both projects, covering practically all areas candidly and thoroughly. Their only significant weakness is the complete lack of reference to environmental aspects in the ICR for the rural project. Environmental issues became a major problem during implementation of the project, as they caused some subprojects to be deleted from the list submitted by the implementing agency. At the same time, the Bank during supervision prompted FNCV to take measures (including preparation of a map with environmentally risky areas) to allow it to better identify environmentally risky investments. A recent internal OED paper has noted that ICRs of rural road projects often fail to discuss environmental issues, and that rarely assess the environmental impacts of such investments." 6.9 On the whole, the audit confirms the ICRs' satisfactory rating of Bank performance in both projects. Borrower Performance 6.10 The audit rates borrower performance as satisfactory in both projects, confirming the ICR ratings. The borrower carried out major sectoral reforms not anticipated at appraisal, which, while creating problems in the implementation of the projects, are likely to generate substantial improvements in the sector. In particular, the newly created national agency, INVIAS, was ably managed and rapidly put itself in a position to efficiently fulfill its role as implementing agency for the national project. 7. Lessons 7.1 The following four key lessons emerge from these projects: * Major highway management reforms are more likely to succeed under a good economic climate. Streamlining the management of the highway sector, making it more efficient and more commercially oriented normally requires downsizing, laying-off significant numbers of personnel. It also requires the adoption of new management philosophies and strategies. The experience in Colombia, where reforms were done by a new government under a relatively good economic climate, shows that economic growth, good employment potential for laid-off personnel by, for example, encouraging the creation of microenterprises to carry out small routine maintenance works, and high political capital are essential ingredients for the success of such reforms. * The implementation risks posed by a large number of subprojects and contracts are difficult if not impossible to mitigate. Even when subprojects are underway or in advanced stages of preparation at the time of project appraisal, a large number of such subprojects scattered geographically taxes any highway administration. Measures to improve monitoring of procurement and execution will help, but likely only to minimally mitigate the risks involved. A concentration of subprojects on a few geographical areas, such as the government and the Bank are now doing in the preparation of new projects focusing on a small number of corridors, appears to be a better approach. * The devolution of responsibility for roads to local governments requires a detailed assessment of the ownership and financial and technical capacity by the receiving 18. Rural Roads and Environmental Sustainability. A synthesis of Bank performance. S. Lawaetz, Operations Department, World Bank. Draft. November 22, 1999. 18 government. Failure by decentralization programs to carefully assess such ownership and capacities is certain to result in rejection of the new responsibilities by the receiving governments, leading to a reversal of the program or to a sustained deterioration of the devolved assets. Benefits of organizational structure need to be preserved through performance benchmarks and a stronger civil service. The experience with the reorganization of Colombia's highway sector management confirms that the benefits of a good sectoral organization can be eroded when political changes bring about frequent changes in management. The use of objective, systematic, and publicized measures of agency performance, coupled with a stronger civil service that leaves only a minimal number of managerial positions subject to political appointments would go a long way in preserving the benefits of organizational reform. 19 Annex A Basic Data Sheet SECOND RURAL ROADS SECTOR PROJECT (LOAN 3157-CO) Key Project Data (amounts in US$ million) Appraisal Actual or estimate Current estimate Total project costs 94.4 93.8 Loan amount 55.0 55.0 Cofinancing None Cancellation None Economic rate of return Over 12% Over 12% Cumulative Estimated and Actual Disbursements FY90 FY91 FY92 FY93 FY94 FY95 FY96 Appraisal estimate 9,1 16,6 24,3 32,0 39.7 47,4 55,0 Appraisal Revised Estimate 9,1 16,6 24,3 32,00 39.8 54,0 55,0 Actual 0 6,5 13,8 25,8 41,8 50,6 55,0 Actual as % of Estimate 0 39,2 56,8 80,6 105,3 106,8 100,00 Actual as % of Revised 0 39,2 56,8 80,6 105,0 93,7 100,00 Date of final disbursement: May 9, 1996 * *US$13,865.71 remaining undisbursed were canceled on that date Project Dates Steps in project cycle Date planned Date actual Identification January 1988 January 1988 Appraisal November 1988 February 1989 Negotiations April 1989 October 1989 Board presentation June 1989 January 16, 1990 Signing March 7, 1990 Effectiveness January 1990 June 6, 1990 Project completion December 31, 1994 December 31, 1995 Staff Inputs (staff weeks) Stage of project cycle Actual Weeks US$ Through appraisal 11.0 23,17 Appraisal-Board 20.7 46,250 Board-Effectiveness 0.4 1,017 Supervision 58.6 174,443 Completion 4.6 6.887 Total 95.37 251,767 20 Annex A Mission Data Stage of project No. of Staff days in Specialized staff skills Performance Ratinq Types of cycle Month/year persons field represented Implementation Development problems Status impact Through appraisal Jan-88 2 5 TE,HE Jun-88 2 12 TE, HE Feb-89 2 12 TE, HE Appraisal through Apr-89 2 7 TE,HE Board approval Jul-89 2 7 TE,HE Board approval thru' effectiveness Supervision Nov-90 2 10 HE,ES 1 1 ENV Sep-91 1 7 HE 1 1 AF Jul-92 1 11 FA 1 1 STD Jan-93 1 5 HE 1 1 CLC Mar-93 1 11 HE,TE 2 1 STD Aug-93 2 16 HE,FA 2 1 CLC,PROC,STD Feb-94 2 11 HE,FA 2 2 CLC,PM Sep-94 2 10 HE S S CLC,PM Mar-95 2 13 HETE S S PM Nov-95 2 7 HE,FA U U CLC,PM,AF,ENV Completion Nov-95 2 7 HE,FA U U CLC,PM,AF,ENV Mar-96 3 2 HE,TE,FA S S PM Staff Skills Type of problems TE Transport Economist AF = Availability of funds HE Highway Engineer PM = Project Management Performance FA Financial Analyst CLC = Compliance with Legal Covenants (financial covenants) ES Engironment Specialist STD = Study delayed ENV = Environmental control PROC = Procurement delayed 21 Annex A Basic Data Sheet THIRD NATIONAL HIGHWAY SECTOR PROJECT (LOAN 3453-CO) Key Project Data (amounts in US$ million) Appraisal Actual or estimate Current estimate Total project costs 414.0 425.4 Loan amount 266.00 266.0 Cofinancing None Cancellation None Economic rate of return 38% 36% Institutional performance Cumulative Estimated and Actual Disbursements FY93 FY94 FY95 FY96 FY97 FY98 Appraisal estimate 72.2 156.4 219.0 251.3 266.0 266.0 Appraisal Revised Estimate N/A N/A N/A N/A N/A N/A Actual 45.9 70.1 154.0 205.5 240.1 265.9 Actual as % of Estimate 63.8 44.8 70.3 81.8 90.3 99.9 Actual as % of Revised N/A N/A N/A N/A N/A N/A Date of final disbursement: 15 May 1998 1. The loan will closed with an unused balance of 1,381.24, which will be canceled. Project Dates Steps in project cycle Date planned Date actual Identification 05/90 Preparation Appraisal 05/91 Negotiations 01/92 Board presentation 03/92 Signing 12/16/92 Effectiveness 04/13/93 Project completion 12/31/96 06/30/97 Credit closing 06/30/97 12/31/97 Staff Inputs (staff weeks) Stage of project cycle Actual Weeks US$ Through appraisal 33.1 84.1 Appraisal-Board 5.3 12.7 Board-Effectiveness Supervision 122.1 293.6 Completion 6.0 8.4 Total 166.5 398.8 22 Annex A Mission Data Stage of project No. of Staff days in Specialized staff skills Performance Ratinq Types of cycle Month/year persons field represented Implementation Development problems Status impact Through appraisal 07/90 2 12 FA, HE N/A N/A N/A 11/90 3 30 FA,HE N/A N/A N/A 02/91 3 30 FA,HE N/A N/A N/A 05/91 5 30 FA,HE,ES,C N/A N/A N/A Appraisal Through Board Approval N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Board Approval Through Effectiveness N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Supervision 12/92 2 TE,HE 1 1 05/93 3 HE, TE, ES 1 1 09/93 6 TE,HE,FA,ES,C 2 1 CC,PP,TP,FP 06/94 4 TE,HE,C U S CCTAP 03/95 5 TE,HE,FA, U S CC,TAP TS,C 03/96 6 TE,HE,ES,TS,C U U CC,PMP,PP,TP, TAP,SP,FP 02/97 3 33 TE,HE,TS S 09/97 3 27 HE,ES,TS S Completion 09/97 HE,ES,TS,HE,C 04/98 HE,C Staff Skills: ES = Environmental Speciaoist; TE = Transport Economist; TS = Transport Specialist; C= Consultant; HE = Highway Engineer; FA = Financial Analyst Problems: CLC: Compliance with covenants; FP: Financial Performance; TAP: Technical Assistance Progress; PMP: Project Mgt. Progress PP: Procurement Progress; SP: Studies Progress 23 Annex B GOVERNMENT COMMENTS op. 014794 Santa ISe de tiogotiá Docor RADlEY NELBON J* Interno Grupo de Evaluaciones Sectoriales y Tmátio* Departamento de E-valuación de Operaciones BANCO MUNDIAL WASHIGTON,D.C 20433 U.S.A. REF: SU OFICIO SIN NUMERO DEL 30 DE MAYO DEL 2000 Respetado doctor Nelson: En amición al oficio de la referoncia, informo a usted que la Evaluación Ex-Post dol Teraor loroyouto Seutoríal do Carreteras (Prestamt3453-CO), no presnta ningún comentario do diclha versión pot pátU da ea* Rntidad, Cordial saludo, ARIA EUGiA ORRERO RESTR PO Jefe Oflicnt Atesora do Planeación Copiás:; Oficina Ascsora de PImcación DG 3596 Unidad de AnMbais Econóinico y Pinancienaino Diocó1 Gnm1 AlI3/Liliansá St