Madagascar - Antananarivo Plain Development Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No.: 22414 PERFORMANCE AUDIT REPORT MADAGASCAR THE ANTANANARIVO PLAIN DEVELOPMENT PROJECT (CREDIT 2117-MAG) June 20, 2001 Sector and Thematic Evaluation Group Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents Effective December 15, 2000 Currency Unit: Malagasy Franc (FMG) FMG1 = US$.00016 US$1 = 6260 FMG Abbreviations and Acronyms AFD Agence Frangaise de D6v6loppement APIPA Autorit6 pour la Protection contre les Inondations de la Plaine d'Antananarivo CAS Country Assistance Strategy FMG Madagascar francs GDP Gross domestic product GNP Gross national product GR G6nie Rural ICR Implementation Completion Report IDA International Development Association JIRAMA National Water and Electricity Company OED Operations Evaluation Department OP Operational Policy (of the World Bank) PRSP Poverty Reduction Strategy Paper (of the World Bank) SAMVA Service Autonome de Maintenance de la Ville d'Antananarivo SSP Sector Strategy Paper (of the World Bank) Fiscal Year Government of Madagascar: January 1 - December 31 Director-General, Operations Evaluation : Mr. Robert Picciotto Director, Operations Evaluation Department Mr. Gregory K. Ingram Manager, Sector and Thematic Evaluation : Mr. Alain Barbu Task Manager : Mr. Klas Ringskog FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. Office of the Director-General Operations Evaluation June 20, 2001 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Madagascar The Antananarivo Plain Development Project (Credit 2117-MAG) The Antananarivo Plain Development Project was approved on March 29, 1990, and closed on September 30, 1999, two years and. three months after the original closing date of June 30, 1997. Project implementation suffered from macroeconomic and politic instability in Madagascar. In addition, the project's co-financier, Agence Frangaise de Devloppement (AFD), suspended disbursement for the 1993-97 period due to government debt service arrears, which caused the IDA-financed project additional implementation problems and delays. OED rates the outcome of the project "moderately unsatisfactory" with its relevance "high," but its efficacy and efficiency "modest." Its sustainability is rated "unlikely" because of the absence of sufficient financial and political support of the institutions in charge of operating and maintaining the facilities built under the project. The institutional development impact is rated "negligible." The Bank performance and borrower performance are both rated "unsatisfactory." The OED ratings agree with those in the ICR except for those of outcome and borrower performance, both of which the ICR rated "satisfactory". The audit confirms three lessons, all of which well-known from past projects and which should have been incorporated in the project. The first lesson is that a project has little chance of orderly implementation and no chance of sustainability unless proper institutional arrangements have been appraised and put in place. For the Antananarivo Plain Development Project, institutions and private companies to manage and maintain the flood control works, drainage and sewerage works, and solid waste collection installations, respectively, were only notionally identified in the project appraisal documents The credit agreement stipulated that the key agency to manage the flood control works was to be established not until almost three years after Board presentation. In the event, the agency was not created until five years after Board approval and was given insufficient financial and political support. It would have been better to have decided all institutional issues before negotiations which would have forced a proper preparation of the institutional arrangements and would have revealed the degree of political support that actually existed in favor of the institutional arrangements. The second lesson is that the relatively short period of implementation of a Bank project is often insufficient to underpin and cement the reforms that projects often attempt. The Bank and its client countries require long, stable collaboration to change institutions and of policies. This speaks in favor of a series of Bank projects, matched to the growing capability of the institutions created under the first operation. In the case of the Antananarivo Metropolitan Area, This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 it would have been preferable to use the concept of an Adaptable Program Loan where the successful completion of one component would trigger the financing of subsequent project components. Such a project design would have reduced the risks of failed project implementation and operations due to weak institutions, the risks of which were underestimated by the Bank. The third lesson is that projects must carefully match ambitions and complexity with the administrative and financial capacity of the country. The audited project actually combined four quite disparate sub-projects: (1) one to control flood damage; (2) one to expand drainage and sanitary sewerage; (3) one to boost rice production through increased irrigation; and (4) one to boost real estate tax collections in the Antananarivo municipality. Each of these projects was ambitious in its own right in the light of the country's previous stagnation and history of economic and political challenges. Again, an Adaptable Program Loan would likely have been a more efficient instrument to reduce risks and encourage orderly project implementation through carefully gauging and confirming the political support for each project component. In addition to the two general lessons, the audit makes a number of detailed recommendations of follow-up financial and technical assistance. In particular, sustainable drainage, sanitary removal of sewage, and solid waste collection remain an unfulfilled aspiration although the project was supposed to provide these essential services. The solution to these pressing environmental problems is urgent given the incidence of disease affecting the low- income population residing in the project area. Attachment Contents Principal R atings........................................................................................................................... iii K ey Staff Responsible ........................................................................................................... im P reface.............................................................................................................................................v 1. Country and Sector Background ...................................................................................... 1 2. Project Background .......................................................................................................... 1 3. Project Objectives and Components.................................................................................. 2 General Project Objectives ....................2.... ...........2 Physical Objectives ....................3...... ..............3 Project Components ................................. ........ 3 4. Project O utcom e ...................................................................................................................... 4 Relevance .................................................. 5 Efficacy.................................................5 Efficiency.....................................................6 Project Outcome...........................................7 Institutional Development Impact........................... ............7 Sustainability.............................................8 Bank Performance.......................................... 9 Borrower Performance....................................... 9 5. Lessons and Recommendations ......................................................................................... 9 Annex A. Basic Data Sheet ....................................................................................................... 13 Annex B. OED Project Evaluation Criteria .......................................................................... 15 This report was prepared by Klas Ringskog, who audited the project in December 2000. William Hurlbut edited, and Helen Phillip provided administrative support.  111 Principal Ratings Madagascar The Antananarivo Plain Outcome Sustainability Institutional Bank Borrower Development Project (Credit 2117) Development Performance Performance ICR Satisfactory Unlikely Negligible Unsatisfactory Satisfactory Audit Moderately Unlikely Negligible Unsatisfactory Unsatisfactory Unsatisfactory Permitted Ratings Outcome Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory Sustainability Highly Likely, Likely, Unlikely, Highly Unlikely, Not Evaluable Institutional Development Impact High, Substantial, Modest, Negligible Bank Performance Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory Borrower Performance Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory Key Staff Responsible Task Manager Division Chief Country Director Appraisal Gerard Tenaille Marc Blanc Paul Isenman Completion James Wright Jeffrey S. Racki Michael N. Sarris  V Preface This is the Project Performance Audit Report (PAR) of the Antananarivo Plain Development Project (Credit 2117-MAG). The credit in an amount of US$30.5 million equivalent was approved on March 29, 1990, became effective on January 31, 1991, and closed on September 30, 1999, two years and three months after the original closing date. At the time of closing the Bank canceled an unused balance of US$7.0 million. The PAR is based on the Staff Appraisal Report (Report 8152-MAG, February 23, 1990), the Implementation Completion Report (Report no. 20108), credit agreements, project documents, and discussions with Bank staff. In addition, an Operations Evaluation Department mission visited Antananarivo in December 2000 for discussions with borrowers, project executing agencies, and direct beneficiaries of the loan. Following customary procedures, copies of the draft PAR were sent to the relevant government officials and agencies for their review and comments.  1 1. Country and Sector Background 1. Since gaining independence from France in 1960 Madagascar has suffered economic stagnation. Madagascar is large - about 590,000 square kilometers- and has considerable economic development potential and an unique but fragile ecology. In spite of the possibilities, economic mismanagement and political instability have led to an almost unmitigated decline where per capita GDP decreased about 1.5 percent per annum over the 1980-99 period. Since 1997 an opening of the economy has produced total annual GDP growth rates of 5 percent, but this export-led growth in free-trade zones is now facing constraints of insufficient trained manpower and inadequate urban infrastructure. Income is poorly distributed and in 1993 about 89 percent of the population was reported to be subsisting on less than US$2 per day - one of the highest shares in the world. Economic and social indicators are weak: gross domestic investment has dropped to 12 percent of GDP, resulting in an aged and decrepit infrastructure; the under-five mortality rate is high at 146 per thousand live births; and adult illiteracy is on the order of 35 percent. 2. The urban population accounts for about 28 percent of the total population of 15 million. The Antananarivo Metropolitan Area population has reached 1.5 million, or 10 percent of Madagascar's population and 40 percent of the country's urban population. Over the 1990s the share of the urban population with water supply house connections has stagnated at 35 percent, of whom half reside in Antananarivo. The parastatal JIRAMA, under the Ministry of Mines and Energy, is responsible for the distribution of electricity and potable water in two-thirds of the country's urban centers. JIRAMA operates existing systems credibly, but lack of funding has prevented rehabilitation and network extensions. Municipalities are in charge of waste water collection and disposal but lack the necessary financial and human resources to operate and maintain existing rudimentary systems, most of which predate independence. Sewerage is mostly combined with the drainage of rainwater, and there is no treatment of the sewage collected. The consequences are what could be expected: sanitation-related diseases are endemic with at least 2,000 cases of cholera in 1999, and recipient waters suffer from the lack of wastewater treatment. The "brown" urban environmental problems can be expected to deteriorate significantly with migration from rural to urban areas adding to urban growth. 2. Project Background 3. The multipurpose Antananarivo Plain Development Project was designed to combine and realize four separate project ideas: (1) a flood control project to mitigate the periodic flooding of the plain; (2) a drainage and sewerage project to collect and remove the rainwater and sanitary wastewater from part of the plain; (3) an agricultural project to increase rice production in the irrigated areas on the plain; and (4) a fiscal reform project to increase real estate tax revenue in the Antananarivo municipality. 4. The flood control component was needed because of the difficult hydrography of the plain. The city of Antananarivo is located on hills descending to a plain where the Ikopa River and its tributaries drain the high seasonal rainfall. (See Map.) A 15-kilometer hard-rock threshold where the river descends from the high-lying plain restricts the river flow and causes periodic flooding when the river swells after heavy rains. Successive French colonial administrations made efforts in the early 20th century to blast a channel through the rock in order to augment the river flow and remove the flooding risk. The magnitude of the task was such, however, that the effort was abandoned. Subsequently, the environmental wisdom of permanently changing the 2 water regime of the plain was questioned. Instead, the idea arose to create a polder that would be protected within raised riverbanks. A disastrous flood in 1959 lent urgency to the project to protect the polder areas against future floods. 5. The drainage and sewerage components were required because of the extremely low hydraulic gradient of the plain, which caused flooding, environmental degradation, and a health hazard. The slope of the plain is only 1 per 10,000, which causes rainwater to drain very slowly, if at all. The precarious drainage has been worsened by an inoperative solid waste collection system. As a result, garbage has been dumped into surface drains which has further obstructed drainage. Because of the low slope and the blockage by garbage, the existing main drain, the Andriantany Canal, silts up rapidly and must be dredged often. Much of the city's incomplete sewerage system collects both rainwater and untreated sewage. When it rains heavily the silted up Andriantany Canal quickly becomes a serious public health hazard because sanitary sewage will flood populated areas and spread disease. Cholera is endemic in Antananarivo and flares up with each rainy season. Plague is also endemic and is spread by rodents who thrive in the garbage. 6. The agricultural component aimed at raising the productivity of the rice growers on the plain by guaranteeing adequate irrigation through a dedicated irrigation system. Before gaining independence in 1960 Madagascar had been the second-largest rice exporter worldwide, but decades of economic mismanagement had turned the country into a rice importer by the late 1980s. A new canal, G6nie Rural (GR), was built to meet the irrigation needs only and remove the conflict between irrigation and drainage needs. The conflicting needs had actually caused flooding in 1982 when the irrigation department did not release water from reservoirs to retain water for irrigation needs. Continued rains then caused the reservoirs to overflow their banks and flood inhabited areas. 7. The fourth project idea focused on updating the Antananarivo land taxation system, which dated back to 1936, in order to raise revenue. The generation of tax revenue was low, inequitable, and unreliable and the project was designed to change all this. 3. Project Objectives and Components General Project Objectives 8. The general project objectives were set against the backdrop of economic stagnation and a delicate political situation in the country. The economic decline is illustrated by the fact that GNP per capita was only US$250 in 1999, that private consumption in Madagascar had fallen an average 2.2 percent per annum over the 1980-98 period. Accordingly, at the time of appraisal, project objectives were set to try to reverse the economic miasma and renew the decayed physical and institutional infrastructure. In particular, the general objective in the appraisal was "to improve the living conditions of the low-income population living in the low-lying areas of the city (ofAntananarivo) and in the Plain." 9. Specific project objectives were set to accomplish the general objectives: * To create an enabling environment for the resumption of private investment and economic growth in the Antananarivo metropolitan area * To consolidate the financial and managerial development of the regional municipalities, already undertaken under an earlier First Urban Project (Credit 1497-MAG) 3 * To set up an institutional framework for efficient operation and maintenance of the structures and facilities provided by the project * To make possible an integrated urban and agricultural development on the Antananarivo Plain. Physical Objectives 10. To help bring about the four separate project ideas that had been prepared in parallel over a number of years and were now folded into the Antananarivo Plain Development Project, physical project objectives were defined to: * Provide 100-year flood protection to the suburbs of Antananarivo (on the plain), and provide an efficient flood warning system * Improve, rehabilitate, and expand the drainage and sewerage facilities to evacuate surface runoff and sewage from the plain under all conditions, developing these facilities consistently with, and in a manner fully compatible with other urban, agricultural, and flood protection developments * Rehabilitate and expand the irrigation system of the Ikopa River's Right Bank. Project Components 11. To accomplish its general and specific objectives the project had 12 components: To control flooding: * Raising and reinforcing dikes of the Ikopa and confluent rivers in order to protect the right bank of the Ikopa * Implementing a flood warning system with hydrometric stations, radio links and tele- metric systems To drain away rainwater from the polder area on the plain: * Dredging and enlarging the existing Andriantany Canal, construction of two retention ponds; and constructing a new drainage canal, C3 * Constructing the Ambodimita pumping station with a capacity of 9 cubic meters/second To provide sanitary sewerage from limited areas of the polder: * Rehabilitating and expanding sanitary sewerage and rehabilitating and adding to the capacity to pump sanitary sewage away from inhabited areas on the plain To improve and extend solid waste collection and removal from areas of the plain: * Improving and adding to the capacity to collect, transport and compost solid waste To increase rice production: * Constructing a new irrigation canal, GR, to irrigate rice fields on the plain and enable the Andriantany Canal to become an exclusive surface water drainage canal 4 * Rehabilitating existing irrigation dikes and drainage canals on the right bank of the Ikopa river To increase tax collections in the Antananarivo Municipality: * Updating the land registry, cadaster, of the Antananarivo Region * Strengthening the Land Tax Department of the Antananarivo Municipality * Creating institutions to manage and maintain the hydraulic facilities of the plain; to operate and maintain the sewerage and drainage systems; to manage solid waste collection in the project areas; and to create irrigation user associations Consultancy Services: * A number of technical studies to prepare the above investments (but no preparatory institutional studies). 4. Project Outcome 12. A project with 12 components is complex and the project ran into severe difficulties during implementation. The IDA credit and co-financing from the Agence Frangaise de D6v6loppement (AFD) were approved in 1990 but got off to a slow start due to political problems in the country and, particularly due to problems with the acquisition of land necessary for the drainage works and with the resettlement of people affected by the project. Then, in January 1994, AFD suspended disbursement to the project following Madagascar's failure to service previous debt. The suspension lasted more than three years, until October 1997, which further delayed project implementation. The result was a excessively compressed implementation period of between two and three years. The IDA credit closed on September 30, 1999, and the AFD financing closed on December 31, 2000, and the bulk of the flood control and drainage works had to be implemented in only two years, 1998 and 1999. 13. The short implementation period and the reduced financing forced several objectives to be scaled back. New sanitary sewerage was not provided but about 45 km of existing sanitary sewers were rehabilitated. Only two of a planned total of five rainwater retention ponds were built. The solid waste management component was scaled back and although the cadastre and the improvement of the land tax department was completed, political opposition from wealthy and influential property owners thwarted its effective implementation. The table below provides an overview of the completion of the project components. The reasons for any shortfalls in completion and the achievement of the project objectives are analyzed more fully in the rating of the five OED evaluation categories (see Annex B for a full description of this evaluation framework). 51 Table 1: Completion and Sustainability of the Project Components Component State of Completion Likelihood of Sustainability Ikopa River Dikes Completed Unlikely Early Warning System Completed Unlikely Dredging Andriantany Completed Unlikely Ambodimita Pumping Station Completed Likely Sanitary sewerage rehabilitation Partially completed Unlikely Solid waste collection Not completed Unlikely Building irrigation canal GR Completed Likely Irrigation canal rehabilitation Completed Likely Updating land registry cadaster Completed but not implemented Unlikely Strengthening Tana land tax dept Not completed Unlikely Creating institutions for O&M Partially completed Unlikely Technical studies Completed Not Applicable Relevance 14. Relevance, is defined as "the extent to which the project's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in PRSPs, CASs, SSPs, OPs). " 15. The general objective "to improve the living conditions of the low-income population living in the low-lying areas of the city (ofAntananarivo) and in the Plain" was relevant and fully consistent with the Bank's current sectoral assistance strategy and corporate goals and policies. Consequently, the project relevance is rated high. The specific project objectives were also consistent with the country CAS and for this reason as well the relevance is rated high. The project set out to improve the living conditions by resolving the problem of regional flooding which arguably affected the low-income population of Antananarivo disproportionately since they constituted the majority of residents in the flood-prone areas. The project also attempted to provide drainage of local rainwater and the collection of sanitary wastewater which remained a serious public health hazard among the low-income population who suffered disproportionately from the endemic cholera in Madagascar. The third objective of raising agricultural productivity was also of high priority to stimulate local economic development and raise income levels of the farmers. Finally, the fourth objective of increasing fiscal revenue, was highly relevant since in its absence the sustainability of the project would riot exist. Efficacy 16. 'Efficacy is defined as: "the extent to which the project's objectives were achieved, or expected to be achieved, taking into account their relative importance." The efficacy of the project is rated modest since major objectives were met, or expected to be met, but with significant shortcomings. At project closing the principal project objective to improve the living conditions of the low-income population had only been achieved partially as it related to the flood control. Two of the remaining specific project objectives to improve living conditions through improved rainwater drainage and sanitary sewerage and extension and improvement of irrigation works were achieved albeit with significant shortcomings. The right-hand bank of the Ikopa River now is enjoying improved flood control through the raised river embankment, but the left-hand bank is more prone to flooding since the embankment on that side was not raised. Similarly, the rainwater drainage has been temporarily improved for a portion of the originally intended area but 6 the rapid silting-up of the surface canals decreases the capacity of the canals to drain the rainwater. On a positive note, the new irrigation canal GR is operational, allowing the Andriantany Canal to be used exclusively for drainage. This is a considerable improvement over the situation without the project where the Andriantany Canal had to meet the sometimes conflicting needs of irrigation and rainwater drainage. 17. In contrast, other specific project objectives were not achieved such as the rehabilitation of the sanitary sewerage system, the upgrading of the solid waste management system, and the effective updating of the cadastre. The sanitary sewerage is largely non-operational since sewage cannot drain into the canals because silting has raised the canal bed above the level of the sewers. The problem has been compounded by the breakdown of solid waste collection, which has forced the residents to use the drainage canals as garbage collection sites. As a result, the canals have clogged up and their drainage capacity has been further reduced. The cadastre was updated as planned, but strong political opposition from property owners prevented its implementation. Efficiency 18. Efficiency is defined as: "the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives." The project efficiency is rated modest because the project fails to meet sector/industry standards for cost effectiveness. The efficiency is only relevant for those objectives that were achieved and excludes those of providing reliable sanitary sewerage, of providing sanitary solid waste removal, and increasing real estate tax collections. 19. The flood control component is the only one that lends itself readily to economic analysis. One common way of quantifying the higher productivity of the land and lower damages to existing infrastructure is to quantify the rise in land value with the project as compared to the situation (in comparable areas) without project investments. The Implementation Completion Report has used this method to estimate the economic rate of return of the flood control works at 17 percent, assuming that half of the reported rise in land values is attributable to the flood control provided by the project. But this calculation is likely to exaggerate the economic gains of the flood control component for three reasons. First, only about 170 hectares of flood-prone land has been permanently protected by the construction of two retention ponds, while the ICR uses an area of 390 hectares for its economic cost benefit calculation. The remaining three planned retention ponds could not be constructed due to the closing of the Bank loan on September 30, 1999. As a consequence, about 220 hectares will likely experience lower rises in land value since the flood protection cannot be guaranteed to the same degree as for the 170 ha. Second, the rise in land values attributable to the flood control project is real but imprecise. There is anecdotal evidence of tenfold increases in some areas but the effect from the flood control project is difficult to disassociate from those resulting from a reactivation of economic activity in the plain, brought about by mostly foreign investors building industrial plants. However, such anecdotal data are imprecise and suffer from the influence of speculation in land values. Third, the rise in land value in the protected areas of the plain is partially offset by the rise in economic damage to areas on the left-hand side of the Ikopa River where flood control damage has risen after the right-hand bank has been protected from flooding. The reported damage is again anecdotal but potentially affects about 10 of the 18 municipalities in the Antananarivo Metropolitan Area which represent a population of some 260,000 inhabitants. The affected populations are now demanding equal flood protection to the right-hand bank. '7 Project Outcome 20. The rating of the project outcome combines the sub-ratings of the relevance, efficacy, and efficiency of the project. Outcome is defined as "the extent to which the project's major relevant objectives were achieved, or are expected to be achieved, efficiently." Under this definition the outcome of this project is rated moderately unsatisfactory since the project is expected to achieve only some of its major relevant objectives of improving the living conditions of the low- income population in the low-lying areas of Antananarivo. The positive rate of return calculated in the ICR exaggerates the benefits but likely remains positive. Institutional Development Impact 21. 'Institutional development impact is defined as "the extent to which a project improves the ability of a country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources through: (a) better definition, stability, transparency, enforceability, and predictability of institutional arrangements and/or (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements." 22. The project institutional development impact is rated negligible because the project is expected to make little or no contribution to the region's ability to effectively use human, financial, and natural resources. (See Annex B for exact definition of rating.) The rating of the institutional development impact hardly surprises since the two institutions established under the project, Autorit6 pour la Protection contre les Inondations de la Plaine d'Antananarivo (APIPA) and the Service Autonome de Maintenance de la Ville d'Antananarivo (SAMVA) are severely hampered by their lack of financial resources, political support, and adequate staffing. No proper institutional assessment was made at the time of appraisal and APIPA and SAMVA were only created some four years into the project, far too late to stand a good chance of operating and maintaining the facilities. At present the funding of the two institutions is running about one-third of needs. As a result, APIPA cannot cope with the maintenance requirements and even less with the future investment needs. It is responsible for operating, maintaining, and extending the hydraulic infrastructure that protects the polder area of the plain, but the task is beyond its capacity. Its administrative and financial weaknesses have prevented APIPA from protecting existing infrastructure and the result has been destruction of the embankment in certain sections by the theft of structural iron and protective rocks. 23. SAMVA is largely non-operational. Its double mandate to collect and dispose of solid waste and operate and maintain the rainwater drainage and sanitary wastewater exceeds SAMVA's modest financial and human capacity. Its ability to sub-contract solid waste management with a private operator has failed because of insufficient reliable financing. Similarly, lack of financing has prevented it from taking over more than the operation and maintenance of 32 km of rainwater drains and 15 km of sanitary sewers, as compared to another 128 km of combined sewers that continue to be under the nominal responsibility of the Antananarivo Municipality. In practice, neither the Antananarivo Municipality nor SAMVA are up to the task of maintaining and extending the infrastructure entrusted to them. 24. In summary, the two institutions belatedly created under the project were not based on a comprehensive institutional analysis and are unable to discharge their limited duties for lack of political support, which has translated into insufficient resources. The problem of insufficient financial resources will remain until APIPA and SAMVA are given a clear mandate to bill and collect sufficient user charges. 8 Sustainability 25. Sustainability is defined as "the resilience to risk of net benefits flows over time. Assessments of sustainability take into account nine factors, including technical, financial, and economic resilience. " The sustainability of the project is rated unlikely since the project's net benefits are unlikely to be sustained in the face of the insufficient administrative and political support and insufficient financial and human resources provided the institutions responsible for operating, maintaining, and extending the systems. Table I on page 5 shows that 8 out of 11 project components are rated "unlikely" to provide sustainable net benefits. 26. The key determinant of likely sustainability is the level of financial resources provided. Table 2 compares the approved funding levels with the required levels. The required annual funding levels are in excess of 14 billion Madagascar francs (FMG) but only 4.3 billion FMG have been budgeted for fiscal year 2000. Including an allowance for the necessary maintenance of flood control works upstream of the project polder area, and of irrigation structures, domestic funding of operations and maintenance is likely to be in the order of 20 percent of required funding. Over time, maintenance will become more costly as facilities age and the imbalance between requirements and needs will worsen unless corrective measures are implemented. Table 2 - Annual Domestic Funding Levels Related to Key Institutions (Billions of Madagascar francs, FMG) Responsible Agency O&M Responsibility Required Funding Approved Funding APIPA Ikopa River Dikes Dredging Andriantany Ambodimita Pumping Sub-total APIPA 7.5 2.4 G6nie Rural Dikes upstream polder Undetermined 0 Meteorology Department Early warning system 2.3 0 SAMVA Sanitary Sewerage Solid waste collection Sub-total SAMVA 4.0 1.9 Irrigation user associations Irrigation canals Undetermined 0 27. In the case of APIPA, the approved revenue sources comprise a surcharge on the initial investment costs of landfills and an annual fee thereafter. These charges are costly to bill and collect and are subject to considerable political influence. APIPA's difficulties are exacerbated by the fact that the Antananarivo Municipal Council determines the assessment percentages each year. The Municipal Council is not a strong supporter of local taxation and is less so now that it is in direct political opposition to the national government with which APIPA is identified. 28. The sophisticated early warning system that was recently completed and is fundamental to effective and timely flood control has up till now been operated by staff from El6ctricit6 de France under French grant financing. Madagascar cannot continue relying on foreign grants to fund current expenditure and the four year cut-off in funding from Agence Francaise de D6veloppement from 1993-97 is proof of the risks of continued reliance on outside financing. 29. SAMVA relies for its funding on a 10 percent surcharge on the Antananarivo water user fees, billed and collected by JIRAMA. It has been decided to increase the surcharge to the 15 to 25 percent range. However, JIRAMA has little incentive to raise the surcharge because they receive only a small fixed remuneration per bill collected. This is poor compensation for the risks that JIRAMA's consumers will resist the payment of the entire water bill. 9 Bank Performance 30. Bank performance is defined as "the extent to which services provided by the Bank ensured quality at entry and supported implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of the project)." Here the Bank performance is rated unsatisfactory. 31. The identification of the project is rated unsatisfactory. The project was excessively complex in comparison to local implementing and financial capacity and the ownership was doubtful. There was also a poor match between the general objectives and the components of the project. The unsatisfactory quality of the identification was compounded by preparation that is also rated unsatisfactory. The institutional feasibility had not been analyzed and resolved at the time of appraisal and second-best institutional solutions were not implemented until five years into the project implementation period. 32. In contrast, the quality of Bank supervision was satisfactory but could not compensate for the serious flaws in identification and preparation. Finally, the follow-up of the project is rated unsatisfactory because no follow-up to the project is planned that might correct the sub-optimal project outcome, lack of sustainability, and lack of an urban strategy for the Antananarivo Metropolitan Area. Borrower Performance 33. Borrower performance is defined as "the extent to which the borrower assumed ownership and responsibility to ensure quality ofpreparation and implementation, and complied with covenants and agreements, towards the achievement of development objectives and sustainability. " The overall borrower performance is rated unsatisfactory. The breakdown of the borrower performance by component is as follows: * Flood control component: - Satisfactory for implementation of civil works - Unsatisfactory for financial support of APIPA - Unsatisfactory for political support of APIPA * Rainwater drainage component: - Unsatisfactory for implementation of civil works - Unsatisfactory for implementing financial policies to provide SANVA with a dependable source of revenue * Sanitary sewage drainage: - Unsatisfactory for implementing financial policies to provide SANVA with a dependable source of revenue * Updating of cadastre: - Unsatisfactory for implementing the higher property values in the updated cadastre. 5. Lessons and Recommendations 34. The first, and obvious, lesson of the project is that projects are not sustainable without viable institutions and supportive policies. The project provided no institution to plan, operate, maintain, rehabilitate, and extend the flood control, rainwater drainage, sanitary sewerage, and 10 solid waste management systems. Without a follow-up urban infrastructure project, the civil works financed under the project will likely fall into disuse and generate no benefits. 35. The second lesson is that it is unrealistic to expect far-reaching reforms to take root in the limited period of project implementation of externally funded projects. The two key institutions, APIPA and SAMVA, were only created five years after the loan had been approved and four years before the Bank loan closed. In many ways their creation was perceived by certain local political interests as an imposition and was therefore resisted. It is unlikely that either one of these institutions will prosper in the absence of follow-up support from within the country or from external financing and technical assistance. 36. The third lesson is that project design must carefully match project ambitions and design to the administrative and financial capacity of the country. The audited project actually combined four quite disparate sub-projects: (1) one to control flood damage; (2) one to expand drainage and sanitary sewerage; (3) one to boost rice production through increased irrigation; and (4) one to boost real estate tax collections in the Antananarivo municipality. Each of these projects was ambitious in its own right in the light of the country's previous stagnation and history of economic and political challenges. Again, an Adjustable Program Lending would likely have been a more efficient instrument to reduce risks and encourage orderly project implementation through carefully gauging and confirming the political support for each project component. 37. The project-specific recommendations are for follow-up financial and technical assistance to provide: * Flood management works on the left bank of the Ikopa River to avoid loss of property and life in the future, the likelihood of which has increased as a result of the flood control measures undertaken under the project for the right bank of the Ikopa River * Solid waste management to remove a grave health risk in densely populated areas where cholera is recurring * Sanitary sewerage and rainwater drainage rehabilitation and extensions to remove the worst foci of cholera. 38. It is further recommended that autonomous agencies be created or strengthened to: * Manage the hydraulic structures to protect against periodic flooding on both the right and left banks of the Ikopa River upstream of the Bevomanga threshold. The needed agency would likely be an Ikopa River Basin Authority where all stakeholders in the Ikopa River would coordinate their investments and operating decisions to ensure optimal use of the river. * Manage solid waste collection and disposal for the entire Antananarivo Metropolitan Area. It is suggested that SAMVA could take on this role that it has dejure at present but that it fails to fulfill defacto. 39. It is further recommended that in order to provide the autonomous agencies with a dependable source of operating revenue that: * APIPA and its successor broaden its revenue base by gaining access to the local real estate tax, possibly to national budget support and possibly to a new kind of tax that would capture, before and during the investment, a portion of the rise in land value that would result from the flood control investments 11 * SAMVA and its successor gain jurisdiction for solid waste collection and disposal in the entire Antananarivo Metropolitan Area and that it gain the authority and capacity to levy a separate solid waste management fee.  13 Annex A Annex A. Basic Data Sheet MADAGASCAR THE ANTANANARIVO PLAIN DEVELOPMENT PROJECT (CREDIT 2117) Key Project Data (Amounts in US$ million) Appraisal Actual or Actual as % of Estimate current estimate Appraisal estimate Total project costs 68.60 63.28 92 Project Dates Original Actual Appraisal 06/23/89 06/23/89 Effectiveness 01/31/91 01/31/91 Credit Closing 6/30/97 9/30/99 Staff Inputs (staff weeks) Actual Weeks Actual US$000 Appraisal/Negotiation 57.20 131.2 Supervision 213.40 757.7 Total 270.60 888.9 Mission Data Performance rating' Date No. of Specialization (month/year) persons represented Implementation Development Status objectives Identification/ 2/86 1 Eng N.A. N.A. Preparation Identification/ 12/86 1 Eng N.A. N.A. Preparation Identification/ 2/88 2 Eng, FMS N.A. N.A. Preparation Identification/ 10/88 4 Eng, UrbPlan, Econ. N.A. N.A. Preparation FMS Identification/ 2/89 6 2Eng. FMS, Urb Plan, N.A. N.A. Preparation Econ, Ag Appraisall 7/89 6 2Eng, Econ, 2 FMS, Ag N.A. N.A. Negotiation Appraisal/ 12/89 3 2Eng, FMS N.A. N.A. Negotiation Supervision 1 2/91 3 2Eng, FMS N.A. N.A. Supervision 2 2/92 3 2Eng, FMS N.A. N.A. Supervision 3 7/92 2 Eng. FMS N.A. N.A. i. 1: Highly satisfactory; 2: Satisfactory; 3: Unsatisfactory; N/R; Not Rates. 14 Annex A Performance rating' Date No. of Specialization (month/year) persons represented Implementation Development Status objectives Supervision 4 2/93 2 2 Soc N.A. N.A. Supervision 5 3/93 2 Eng, FMS, Soc. N.A. N.A. Supervision 6 7/93 3 Eng, FMS, Soc, Research N.A. N.A. Supervision 7 12/93 4 Eng, FMS, Res N.A. N.A. Supervision 8 6/94 3 Eng, 2FMS S S Supervision 9 11/94 4 Eng, 2FMS, Research S U Supervision 10 5/95 3 Eng, 2FMS, Research U U Supervision 11 10/95 4 2Eng, 2FMS U S Supervision 12 6/96 4 Eng, Econ, FMS, U S Research Supervision 13 10/96 4 Eng, 2FMS, Econ, U S Research Supervision 14 3/97 5 U S ICR 12/97 3 Egn, 2FMS U S ICR 4/97 3 Planner, Eng, Soc U S ICR 10/97 3 Planner, Eng, FMS S S ICR 3/99 3 Planner, FMS, Research S U ICR 10/99 3 Planner, Econ, FMS S U 15 Annex B Annex B. OED Project Evaluation Criteria Definitions of Eight Key Criteria 1. Relevance of Objectives Definition: the extent to which the project's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in PRSPs, CASs, SSPs, OPs) Ratings (i) Each type of objective will be rated as follows: High Project objectives of this type play a key role in the country's current development priorities and the Bank's current country assistance strategy, and are fully consistent with the Bank's current sectoral assistance strategy and corporate goals and policies Substantial Project objectives of this type are mostly consistent (minor shortcomings only) with the country's current development priorities, the Bank's current country and sectoral assistance strategies, and the Bank's current corporate goals and policies Modest Project objectives of this type have one or more significant inconsistencies with the country's current development priorities, the Bank's current pountry and sectoral assistance strategies, or the Bank's current corporate goals and policies Negligible Project objectives of this type are mostly inconsistent with, and possibly counterproductive to, the country's current development priorities, the Bank's current country and sectoral assistance strategies, or the Bank's current corporate goals and policies (ii) Overall relevance will be rated as follows: High Most of the major objectives were highly relevant Substantial Most of the major objectives were at least substantially relevant Modest Most of the major objectives were not highly or substantially relevant Negligible Most of the major objectives were irrelevant or negligibly relevant 2. Efficacy Definition: the extent to which the project's objectives were achieved, or expected to be achieved, taking into account their relative importance Ratings (i) Each type of objective will be rated as follows: High Objectives of this type were fully met, or expected to be fully met, with no shortcomings Substantial Objectives of this type generally were met, or expected to be met, with only minor shortcomings 16 Annex B Modest Objectives of this type were met, or expected to be met, but with significant shortcomings Negligible Objectives of this type were not met, or expected not to be met, due to major shortcomings (ii) Overall efficacy will be rated as follows: High Major objectives were fully met, or expected to be fully met, with no shortcomings Substantial Major objectives were met, or expected to be met, with only minor shortcomings Modest Major objectives were met, or expected to be met, but with significant shortcomings Negligible Most objectives were not met, or expected not to be met, due to major shortcomings 3. Efficiency Definition: the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives Ratings High Project represents sector/industry best practice in terms of cost effectiveness, and economic returns (if estimates are available) greatly exceed the opportunity cost of capital Substantial Project meets sector/industry standards in terms of cost effectiveness, and economic returns (if estimates are available) exceed the opportunity cost of capital Modest Project fails to meet sector/industry standards in terms of cost effectiveness, and economic returns (if estimates are available) are near the opportunity cost of capital Negligible Project is well below sector/industry standards in terms of cost effectiveness, and economic returns (if estimates are available) are significantly below the opportunity cost of capital 4. Sustainability Definition: the resilience to risk ofnet benefits flows over time. Assessments ofsustainability take into account nine factors, including technical, financial, and economic: * Technical resilience * Financial resilience (including policies on cost recovery) * Economic resilience * Social support (including conditions subject to Safeguard Policies) * Environmental resilience * Government ownership (including by central governments and agencies, and availability of O&M funds) * Other stakeholder ownership (including local participation, beneficiary incentives, civil society/NGOs, private sector) * Institutional support (including supportive legal/regulatory framework, and organizational and management effectiveness) * Resilience to exogenous influences (including terms of trade, economic shocks, regional political and security situations) 17 Annex B Ratings (i) Each factor will be rated as follows: High The factor clearly is currently met and highly likely to continue, contributing to sustainability of net benefits Substantial The factor is currently met and likely to continue, contributing to sustainability of net benefits Modest The factor may currently be met, but it seems unlikely to continue, to the detriment of sustainability of net benefits Negligible The factor is currently not met and is highly unlikely to be met, to the detriment of sustainability of net benefits (ii) Overall sustainability will be rated as follows: Highly Likely Project net benefits flow meets most of the relevant factors determining overall resilience at the "high level", with all others rated at the "substantial" level Likely Project net benefits flow meets all relevant factors determining overall resilience at the "substantial"' level Unlikely Project net benefits flow meets some but not all relevant factors determining overall resilience at the "substantial" level Highly Unlikely Project net benefits flow meets few of the relevant factors determining overall resilience at the "substantial" level Not Evaluable Insufficient information available to make a judgment 5. Institutional Development Impact (IDI) Definition: the extent to which a project improves the ability ofa country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources through: (a) better definition, stability, transparency, enforceability, and predictability of institutional arrangements and/or (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements. IDI includes both intended and unintended effects of a project. Ratings (i) Each objective type will be rated as follows: High Project objectives of this type made, or are expected to make, a critical contribution to the country's/region's ability to effectively use human, financial, and natural resources Substantial Project objectives of this type made, or are expected to make, a significant contribution to the country's/region's ability to effectively use human, financial, and natural resources Modest Project objectives of this type increased, or are expected to increase, to a limited extent the country's/region's ability to effectively use human, financial, and natural resources 18 Annex B Negligible Project objectives of this type made, or are expected to make, little or no contribution to the country's/region's ability to effectively use human, financial, and natural resources (ii) Overall institutional development impact will be rated as follows: High Project as a whole made, or is expected to make, a critical contribution to the country's/region's ability to effectively use human, financial, and natural resources, either through the achievement of the project's stated ID objectives or through unintended effects Substantial Project as a whole made, or is expected to make, a significant contribution to the country's/region's ability to effectively use human, financial, and natural resources, either through the achievement of the project's stated ID objectives or through unintended effects Modest Project as a whole increased, or is expected to increase, to a limited extent the country's/region's ability to effectively use human, financial, and natural resources, either through the achievement of the project's stated ID objectives or through unintended effects Negligible Project as a whole made, or is expected to make, little or no contribution to the country's/region's ability to effectively use human, financial, and natural resources, either through the achievement of the project's stated ID objectives or through unintended effects 6. Outcome Definition: the extent to which the project's major relevant objectives were achieved, or are expected to be achieved, efficiently. The outcome criterion takes into account relevance at the time of the evaluation: whether the operation's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals; efficacy: whether the operation is expected to achieve its stated goals; and efficiency: the relation of results to costs. Ratings Highly Satisfactory Project achieved or exceeded, or is expected to achieve or exceed, all its major relevant objectives efficiently without major shortcomings Satisfactory Project achieved, or is expected to achieve, most of its major relevant objectives efficiently with only minor shortcomings Moderately Satisfactory Project achieved, or is expected to achieve, most of its major relevant objectives efficiently but with either significant shortcomings or modest overall relevance Moderately Unsatisfactory Project is expected to achieve its major relevant objectives with major shortcomings or is expected to achieve only some of its major relevant objectives, yet achieve positive efficiency Unsatisfactory Project has failed to achieve, and is not expected to achieve, most of its major relevant objectives with only minor development benefits Highly Unsatisfactory Project has failed to achieve, and is not expected to achieve, any of its major relevant objectives with no worthwhile development benefits 7. Bank Performance 19 Annex B Definition: The extent to which services provided by the bank ensured quality at entry and supported implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of the project) Ratings Highly Satisfactory Bank performance was rated as Highly Satisfactory on both quality at entry. and supervision, or Highly Satisfactory on the one dimension with significantly higher impact on project performance and at least Satisfactory on the other Satisfactory Bank performance was rated at least Satisfactory on both quality at entry and supervision, or Satisfactory on the one dimension with significantly higher impact on project performance and no less than Unsatisfactory on the other Unsatisfactory Bank performance was not rated at least Satisfactory on both quality at entry and supervision, or Unsatisfactory on the one dimension with significantly higher impact on project performance and no higher than Satisfactory on the other Highly Unsatisfactory Bank performance was rated as Highly Unsatisfactory on both quality at entry and supervision, or Highly Unsatisfactory on the one dimension with significantly higher impact on project performance and no higher than Unsatisfactory on the other 8. Borrower Performance Definition: the extent to which the borrower assumed ownership and responsibility to ensure quality ofpreparation and implementation, and complied with covenants and agreements, towards the achievement of development objectives and sustainability Ratings Highly Satisfactory Borrower performance was rated Highly Satisfactory on at least two of the three performance factors Satisfactory Borrower performance was rated at least Satisfactory on two of the three factors Unsatisfactory Borrower performance was not rated at least Satisfactory on two of the three factors Highly Unsatisfactory Borrower performance was rated Highly Unsatisfactory on at least two of the three factors