Madagascar - First Environment Program Project

Document of THE WORLD BANK FOR OFFICIAL USE ONLY Report No: 20667 PERFORMANCE AUDIT REPORT MADAGASCAR FIRST ENVIRONMENT PROGRAM (CREDIT 2125--MAG) August 2, 2000 Operations Evaluation Department Sector and Thematic Evaluation Group This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents (annual averages) Currency Unit = Malagasy Franc (FMG) 1989 US$1.00 FMG 1590 1990 US$1.00 FMG 1530 1991 US$1.00 FMG 1650 1992 US$1.00 FMG 2000 1993 US$1.00 FMG 1850 1994 US$1.00 FMG 1900 1995 US$1.00 FMG 4000 1996 US$1.00 FMG 4200 1997 US$1.00 FMG 4800 Abbreviations and Acronyms ANAE Association Nationale pour les Actions Environnementales (National Association for Environmental Actions) ANGAP Association Nationale pour la Gestion des Aires Proteg6es (National Association for the Management of Protected Areas) CAS Country Assistance Strategy CFSIGE Centre de Formation pour les Sciences de I'Information G6ographique et Environnerentale (Environment Information Training Center) EIA Environmental impact assessment EP1 First Environment Program (1991-96) FORAGE Fonds R6gional d'Appul A la Gestion de I'Environnement (Regional Fund for Environment Management) GDP Gross domestic product ICDP Integrated Conservation and Development Project IDA International Development Association, of the World Bank Group MDS Multi-Donor Secretariat M&E Monitoring and evaluation MOE Minist&re de I'Environnement (Environment Ministry) NEAP National Environmental Action Plan OED Operations Evaluation Department ONE Office National de l'Environnement (National Environmental Office) SAC Structural Adjustment Credit Fiscal Year Government: January 1 - December 31 Director-General, Operations Evaluation : Mr. Robert Picciotto Director, Operations Evaluation Dept: : Mr. Gregory Ingram Manager, Sector and Thematic Evaluation Group : Mr. Alain Barbu Task Manager : Mr. Fernando Manibog FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. Office of the Director-General Operations Evaluation June 29, 2000 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Madagascar First Environment Program (Credit 2125-MAG) Attached is the Performance Audit Report prepared by the Operations Evaluation Department (OED) on the above project, for which IDA approved a credit of SDR 19.8 million (US$26.0 million equivalent at appraisal) on April 17, 1990, and made effective on February 6, 1991. The borrower was the Republic of Madagascar. The credit closed on June 30, 1997, and the credit was fully disbursed. The objectives of the project were to: (i) conserve and manage Madagascar's biodiversity; (ii) promote sustainable development and natural resources management; (iii) improve living standards; and (iv) develop human resources and institutional capacity. The project outcome is rated as marginally unsatisfactory. The over-ambitious and complex project did not put in place the long-term foundation for biodiversity conservation and management, including sustainable institutions, regulatory frameworks, databases, policy studies, education and training programs, and the monitoring and evaluation systems critical to assessing biodiversity conservation impacts. Moreover, the physical objectives were only barely executed, or not attempted at all. The overall institutional development impact of what turned out to be an enclave project, is rated as modest to negligible, because the agencies created under the project lack sustainable financing sources and continue to have a limited ability to achieve results to a degree that would start to have measurable impacts on reducing the country's very serious environmental problems. Sustainability of project benefits is rated as unlikely because environmental stewardship has not been mainstreamed and local benefits have not been adequately demonstrated, due to the absence of an effective monitoring and evaluation system. The key lessons from this project are: * The Bank's reluctance to take realistic account of risks leads to faulty project designs and serious implementation problems, particularly when local capacities are weak. * Creating national environmental institutions cannot be accomplished within the constraints of a five-year project timetable. * Lack of continuity in project and country management in the Bank results in frequent re-ordering of priorities, thus hindering the prospects of successful project implementation. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  1 Contents Principal Ratings ...........................................................................................................................iii Key Staff Responsible .......................................................................................................... iii Preface ............................................................................................................................................. V 1. Background .............................................................................................................................. 1 National Environmental Action Plan (NEAP) ............................. 1 2. Project Objectives, Components, and Implementation...................................................2 Project Objectives ......................................... ....... 2 Project Components ..................................... .............. 2 Project Cost and Financing ............................... .......... 3 Project Implementation ................................................ 3 Implementing Agencies ...........................................3 Project Execution ........................................ ........ 4 Protection and Management of Biodiversity............ ..............4 Soil Conservation, Agroforestry, Reforestation, and Other Rural Development Activities....................5........... ............5 Mapping, Remote Sensing, and Geographic Information ......... ...........5 Cadastral Surveys and Pilot Land Titling .......... ...............6 Environmental Training, Education, and Communication... .....................6 Environmental Research ............................. .........7 Support Activities ................................. ......... 7 3. Results and Sustainability .................................................................................................. 8 Achievement of Project Objectives.................8........ ...........8 Institutional Development Impact........... ................. ......... 9 Project Sustainability ....... .......... ................................ 10 4. Borrower and Bank Performance .................................................................................... 11 Borrower Performance ............................................ 11 Bank Performance..... .......... ................... ........... 11 5. Overall Assessment and Lessons Learned......................................................................12 Overall Assessment and Summary of Ratings .............................12 Lessons Learned................ ............................... 13 This report was prepared by Fernando Manibog (Task Manager) and Kevin Warr (Consultant). Mr. Manibog visited Madagascar in August, 1999. William Hurlbut edited the report and Marie Daramy provided administrative support. ii Annex A: Basic Data Sheet ..................................................................................................... 15 Annex B: Borrower's Comments .......................................................................................... 17 Annex C: Borrower's Comments (English Translation) ................................................... 25 Annex D: OED Response to Borrower's Comments........................................................... 31 111 Principal Ratings ICR Audit Outcome Satisfactory Marginally Unsatisfactory Sustainability Unlikely Unlikely Institutional Development Impact Partial Modest Borrower Performance Satisfactory Unsatisfactory Bank Performance Satisfactory Unsatisfactory Key Staff Responsible Year Task Manager Sector Manager Country Director Vice President 1988-90 Francois Falloux Adrian Otten Paul Isenman Paul Isenman, Acting 1991 Bruno Ribon 1992-94 Luciano Mosele Mr. Serageldin Armand Rioust de Mr. Jaycox Nils 0. Tcheyan Largentaye (Acting) Callisto Madavo 1994-95 Michel Sim6on and Cynthia Cook 1995 Mary Barton-Dock 1996-97 Michel Simbon Michael Sarris  V Preface This is the Performance Audit Report (PAR) for Madagascar's First Environment Program (Credit 2125-MAG) for which IDA approved a credit of SDR 19.8 million (US$26.0 million equivalent at appraisal) on April 17, 1990 and made effective on February 6, 1991. The borrower was the Republic of Madagascar and the beneficiaries and executing agencies were the Office National de l'Environnement (ONE), the Association Nationale d'Actions Environnementales (ANAE), and the Association Nationale pour la Gestion des Aires Prot6gies (ANGAP). The credit closed on June 30, 1997 and was fully disbursed. Cofinancing was provided by the governments of Norway (NORAD), Switzerland, France (FAC), Germany (KfW), and United States (USAID) and by UNDP. This PAR is based on a review of the Implementation Completion Report (ICR) prepared by the Africa Region, the Staff Appraisal Report (SAR), credit documents, project files, and discussions with IDA/Bank staff, Malagasy government officials, and officials and staff of the implementing agencies. An OED audit mission visited Madagascar in August 1999 to discuss EPi with the government and implementing agencies, donor representatives, the Multi-Donor Secretariat (MDS), nongovernmental organizations and private entities. The assistance and cooperation of all stakeholders is greatly appreciated. Following standard OED procedures, the draft PAR was sent to the Malagasy authorities for comments. Comments received have been incorporated in the final version as annex B.  1 1. Background 1. Since independence in 1960, Madagascar's agrarian economy has not provided an adequate engine for wealth creation and sustained growth remains elusive. The country's per capita gross domestic product (GDP) plummeted almost 50 percent between 1971 and 1994, mostly as the result of statist economic mismanagement. During the 1990s, per capita GDP has stagnated at approximately US$230. However, political changes since 1996 provide hope for increased growth. The fiscal deficit fell to about 5 percent of GDP in 1996 and inflation is now in the single digits. The country has also reduced its dependence on agricultural commodity exports (mainly coffee, vanilla, and cloves) from 70 percent of total exports in 1984 to 25 percent in 1996. In quality and quantity, Madagascar's exports remain uncompetitive, and earning hard foreign currency is difficult. Domestic capital markets are similarly weak and (although slowly expanding) private investment is held back by an uncertain political climate, a fragile legal system, the lack of infrastructure, and high interest rates. 2. Social indicators reflect the difficult economic conditions for the average Malagasy. About 70 percent of the population are poor and live in rural areas where the quality of life is extremely low. Madagascar is one of the few countries experiencing declines in education, nutrition, and life expectancy, with indicators at or below the highly unsatisfactory averages for sub-Saharan Africa.' Madagascar's difficulty in modernizing its economy and improving its social conditions contributes significantly to the rapid deterioration of its globally unique environment. 3. Madagascar is one of the most biologically diverse areas in the world. The variety and concentration of plants and animals in such a relatively small area is unrivalled. About 80 percent of Madagascar's plant species are endemic and it is the exclusive home of the rare lemur. Twenty-five percent of Africa's plant species reside there. In 1950, primary forest covered roughly 25 percent of the country. Today, only about 15 percent remains forested. Agricultural activity and a rapidly growing population continue to encroach upon the little forest that remains. Exhausted plots are replaced by cutting down and burning forested land. Soil erosion, estimated at up to 200 tons per hectare per year, and loss of irreplaceable wildlife habitat through deforestation have become Madagascar's principal environmental concerns. Various sources estimate the current rate of deforestation at 0.8 to 2 percent per year of remaining forested areas, which represents a tremendous loss of natural habitat. A hectare of forest lost in Madagascar has a greater negative impact on global biodiversity than a hectare of forest lost virtually anywhere else. 4. The Bank's strategy in Madagascar is focused on alleviating poverty and its attendant ills, such as environmental degradation, through broad-based economic growth, which places strong emphasis on sound natural resource management. To achieve the Bank's objective, the National Environmental Action Plan (NEAP) was drawn up in 1987 by the Government of Madagascar (GOM), the Bank, and a consortium of donors.' National Environmental Action Plan (NEAP) 5. In its day, Madagascar's NEAP was the most ambitious and comprehensive environmental program to date in Africa. The plan, launched in 1989, was designed to transform the Malagasy 1. Poverty Assessment, June 1996 2. The consortium included UNDP, UNESCO, USAID, World Wildlife Fund (WWF), Conservation International (CI), and French, German, Norwegian, and Swiss aid. 2 approach to natural resource management. NEAP had four objectives: (i) to develop human resources; (ii) to promote sustainable development in the management of natural resources; (iii) to conserve and manage the country's legacy of biodiversity; and (iv) to improve the quality of life for rural and urban populations. The NEAP was conceived in three five7year stages as a means of dramatically reversing widespread environmental degradation while assisting the Malagasy people in poverty reduction. The First Environment Program (Credit 2125-MAG) represents the first five-year stage of the NEAP. The guiding principle of the program is the understanding that policies, institutions, and incentives must be altered to enable resource users to have the authority and responsibility for managing their own resources in a sustainable manner. While its links to economic analysis appear weak or non-existent at the time of preparation, central to the NEAP's hypothesis is the belief that the battle to protect Madagascar's biodiversity will be won or lost on agricultural land away from the forest. Thus, a principal focus of the First Environment Program was agricultural land use reform. 2. Project Objectives, Components, and Implementation Project Objectives 6. The First Environment Program (EPI) took a broad, multi-component approach to Madagascar's interlocking environmental and social problems: (i) erosion and siltation; (ii) diminishing biodiversity; (iii) lack of environmental institutional framework, including agencies and policies; (iv) lack of environmental assessment procedures; (v) poor land tenure security; (vi) lack of consciousness among the population over the severity of problems; and (vii) lack of environmental data and research efforts. 7. To address these difficulties, EPI encompassed two main objectives, according to the Staff Appraisal Report and the Implementation Completion Report. The first objective was to support operations of an urgent nature, namely (i) protecting the heritage of biodiversity in the parks, reserves and gazetted forests, in conjunction with the development of the surrounding communities; and (ii) fighting deforestation and erosion in priority watersheds where the negative economic impact was the highest. The second objective was to establish the foundations for environmental management through institution building, studies, and human resource development. Project Components 8. EPI had seven components, all partly funded by IDA: (i) protect and manage biodiversity at the level of 44 Protected Areas and a number of gazetted forests'; (ii) develop community- based soil conservation and watershed management mini-projects; (iii) develop land management tools through mapping and remote sensing; (iv) improve land security through cadastral operations; (v) promote environmental awareness, education, and training; (vi) conduct environmental research programs on land, coastal, and marine ecosystems; and (vii) develop a support program including institution building, reinforcement of the environmental database, marine and environmental research, monitoring and evaluation. 3. The number of Protected Areas in the SAR is 51, but some areas have been consolidated. 3 Project Cost and Financing 9. The SAR's project cost estimate was US$85.5 million. The actual cost was US$94.1 million. IBRD/IDA contributed US$26 million, which was fully disbursed. Cofinancers (the governments of Norway (NORAD), Switzerland, France (FAC), Germany (KfW), the United States (USAID) together with UNDP) originally committed US$42.3 million and actually contributed US$64.66 million. The Government of Madagascar contributed US$3.42 million. Nearly half of EPl's funding (about US$39 million) was allocated to biodiversity protection, assistance to the Water and Forests Directorate (DEF), and the development of buffer zones. Support and other institution-building activities constituted the next largest claim on EPI funds, estimated at US$34 million. Project Implementation Implementing Agencies 10. The government created three new environmental agencies to implement EPI: the Office National de l'Environnement (ONE), the Association Nationale d'Actions Environnementales (ANAE), and the Association Nationale pour la Gestion des Aires Protigies (ANGAP). In principle, the agencies were organized in such a way that ONE (presently affiliated with the Ministry of the Environment) was the coordinator of the NEAP, with a number of responsibilities. First, ONE was to implement national environmental policies and develop public relations activities, training, and policy formulation. Second, it was also meant to improve the country's environmental regulatory framework as well as to set environmental impact assessment (EIA) regulations and procedures and establish environmental units in other sector ministries. Third, ONE was to consolidate annual financial statements of all participating agencies, prepare annual and semiannual reports on program components, produce comprehensive reports on the progress of EP1, manage environmental data and coordinate environmental studies. 11. ANAE was created, under the management of ONE, to have oversight responsibility for environmental mini-projects. It also acts as a funding and contracting channel for demand-driven soil conservation/erosion control activities. It designs and develops coordination linkages between villages and provides grants to finance part of the cost of approved mini-projects. 12. Similarly, ANGAP was established to coordinate the biodiversity component, especially in the area of policy development. The government and the donors envisioned that, under the Second Environment Program, ANGAP would also manage protected areas. ANGAP administers funds for protected areas management, trains staff in biodiversity management, and launches communications campaigns. It is also responsible for monitoring and evaluation for biodiversity management, including data system management. 13. In practice, however, these three new environmental agencies have not functioned as planned. The interaction between ONE, ANAE, and ANGAP was problematic from the start and lacked a systemized and formalized approach. All of the agencies had widely distributed and generally unclear responsibilities. Their duties were often ill-defined and/or overlapped and coordination between the three agencies was virtually non-existent. Feuds and overlapping responsibilities developed between the agencies during EP1, thus hindering their effectiveness. Moreover, they lacked the capacity to implement established policies, which contributed to EPI's mixed results. 4 Project Execution 14. Project execution was delayed by nearly a year due to the government's difficulties in meeting effectiveness conditions, particularly those related to institutional development. However, the borrower fully complied with the 14 conditions in the Credit Agreement. The start- up of implementation of most components was further delayed because of social and political instability that subsided in late 1991. After the project became effective, there was also a significant disbursement lag because of the newly established agencies' unfamiliarity with the Bank's procurement procedures. Moreover, the withdrawal of the African Development Bank (because GOM defaulted on repayment of outstanding arrears) seriously affected the training, education, and communication component. KfW and NORAD financing was stalled in the finalization stages due to disagreements over levels of counterpart funding and lack of appropriate signatures and delays in the allocation and disbursement of counterpart funds. Implementation startup was further hindered by the fact that a financing plan for the cofinancers had not been finalized at appraisal. By the midterm review in December 1993, only US$4.8 million had been disbursed. However, from mid-1994 onward, disbursements increased to over US$400,000 per month because of several amendments in the credit agreement and improved institutional performance. 15. EPI's achievements under each component are discussed below, denominated mostly in terms of output rather than outcome or impact indicators, due to the absence of monitoring and evaluation systems under EPI. Protection and Management of Biodiversity 16. This component had three parts: (i) establish and equip 44 Protected Areas (PAs); (ii) provide assistance to approximately 70,000 families living in the proximity of the PAs to develop sustainable economic activities and improve their living conditions; (iii) improve the protection and sustainable management of 0.3 million hectares of natural forests near the PAs. ANGAP was created and took responsibility for the establishment, coordination, and management of the country's PAs. Forty-four PAs were established, but at EPI's close, they had not been legally recognized, although the process of formalization was in place. Seven of the PAs were managed by ANGAP and 13 by contractors. This sub-component was partially achieved. 17. Local development activities were supported by Integrated Conservation and Development Projects (ICDP) in 19 of the 44 Protected Areas (43 percent of the total). 13,482 families (19 percent of appraisal estimate) have directly benefited from these activities. Although ANGAP maintains that 65,000 families have either directly or indirectly benefited from ICDP activities." ICDPs were eventually judged to be too small-scale and difficult to replicate, and were replaced by Landscape Development Interventions under the Second Environment Program (EP2). This ICDP sub-component was barely achieved. 4. The Government clarified that the ICDPs are primarily projects for the conservation of Protected Areas. Their focus is on developing alternative activities to those causing pressures in the surrounding zones, and on conserving the local biodiversity at the same time. This was the formula adopted for the conservation of the PAs during EPI and certain donors are continuing to finance them as part of EP2. As a result, the population benefits from both development activities (village granaries, agricultural techniques, production infrastructure, etc.) and social infrastructure (schools, hospitals, etc.), but this makes it difficult to be accurate in calculating the number of beneficiary families (e.g. entire villages benefit from social infrastructure, individual families and farmer associations from production infrastructure). 5 18. The forest sub-component consisted of protection and sustainable management of 0.3 million hectares of natural forests near the Protected Areas. A new forestry law was finalized and, at the time of EPI's completion, was awaiting approval from the country's Council of Ministers. Also, a national forestry plan has been completed and was used as the framework for new policies on national forest management. The forestry plan has resulted in a major overhaul of forestry policy and legislation and has spawned the development and application of training modules followed by local and international training. One forest management plan was developed for a gazetted forest and guidelines prepared for the development of additional plans for three other forest reserves. The forestry inventory, however, was only partially completed at EP1's close and thematic maps had yet to be produced. This component was partially achieved. Soil Conservation, Agroforestry, Reforestation, and Other Rural Development Activities 19. This component was designed for rural communities not covered under EP1's other components and which were particularly susceptible to soil erosion. The component included: (i) protection of watersheds; (ii) adoption of soil conservation measures; (iii) development of productive capacity; and (iv) improvement of living conditions. Approximately 30,000 families and 150,000 hectares were expected to benefit from about 150 mini-projects under this component. To implement this component, the Association pour les Actions Environnementales (ANAE) was established. 20. During the program period, ANAE executed 1,177 mini-projects involving 45,074 families, which far exceeded the appraisal estimate. However, the area covered by the projects was only 16,832 hectares, which fell significantly short of the appraisal goal, and little work was done in the peripheries of the protected areas. Field surveys indicate that farmers who adopted improved production and protection technologies considerably augmented their agricultural income by 20 to 85 percent. Also, the frequency of bush fires has been reduced in some areas and soil losses have been reduced appreciably (0.5 tons per hectare per year, as compared to the country's average figure of 200 tons per hectare per year). Given the importance of wide area coverage (versus area intensity) to this component's objectives, and the fact that only 11 percent of the area was covered as compared to the targets, the goal of protecting watersheds was barely achieved.' Mapping, Remote Sensing, and Geographic Information 21. The goal of this component was to support the development of a national capacity to provide services necessary for the planning, management, monitoring, and evaluation of the country's Protected Areas and their surrounding peripheral areas. This component had the 5. The Government is of the view that this component was concerned with both the intensity of project interventions and the area covered, demonstrated by the fact that these two dimensions were the subject of monitoring and impact assessments. The component was designed to lessen erosion in watersheds (Madagascar's relief is such that 80% of the island is made up of watersheds) and to improve the living environment (for production and use of natural resources) in the localities targeted. The Government explains that the shortfall in spatial coverage (11% of the target area) should be regarded as offset by the higher incomes earned by the beneficiary population, from 20% to 85% higher, which is rare in developing countries and very substantial for Madagascar, whose economic growth rate is of the order of 4%. Moreover, the Government considers the fact that soil losses have been reduced from 200t/ha/year to a figure of less than It/ha/year, as an exemplary success, whatever the scale of the operation. The Government adds that the techniques used are those most readily accessible to the population and very innovative (zero labor, among other things). 6 following elements: (i) establishment of a geodetic network and cartographic coverage of about 6 million hectares, (ii) production of orthophotos for approximately 2 million hectares, and (iii) establishment of a national remote sensing laboratory. 22. Only 2.4 million hectares (40 percent of appraisal estimate) of the proposed geodetic network were established because of changes in the needs and/or budgetary constraints of the institutions participating in other project components. For the same reasons, and because of technical problems and delays in contracting, orthophotos were produced for only 0.94 million hectares (47 percent of appraisal estimate). The national remote sensing laboratory was established and is functioning. This component was partially achieved. Cadastral Surveys and Pilot Land Titling 23. This component was aimed at defining the boundaries between public and private lands for project-supported Protected Areas and gazetted forests. It had two parts: (i) survey and title approximately two million hectares of public lands and (ii) title about 200,000 hectares of private land in peripheral areas, thus increasing land security and reducing pressure on the Protected Areas. 24. Only 838,000 hectares of land (38 percent of the appraisal estimate) was surveyed and titled. The underperformance of this component was due to a host of factors including unreliable maps, disagreements on the boundaries of some Protected Areas, the complexity of the titling process, the social and political controversy associated with the titling activities, technical difficulties related to the mapping component. Moreover, this component was meant to be a pilot whose continuation would depend on the results of an evaluation completed in time for the Bank's midterm review. However, the component started too late for the midterm evaluation and no conclusive results are available. This component was barely achieved. Environmental Training, Education, and Communication 25. This component had four parts: (i) provide training in support of the program's other component institutions; (ii) build human resources to meet the staff needs of new institutions; (iii) introduce environmental curricula to the primary education system; and (iv) increase decision- makers' awareness of environmental problems. ONE was to coordinate this component in association with the kcole Superieure des Sciences Agronomiques and the Ministre de l'Education de Base. 26. Training occurred in various project executing agencies, but results have not been aggregated. Public awareness campaign activity was also limited and focused on regional seminars, radio dissemination programs and public information exhibitions. To date, there is no agency responsible for upgrading the environmental content of education curricula, and there has been intense infighting among staff of the Ministry of Environment, the Ministry of Education, 7 and the Environmental Information Training Center as to who has the responsibility and what actions to take.' This component was barely achieved. Environmental Research 27. This component consisted of two environmental research projects: (i) five studies in coastal research and (ii) four land-based studies. Three coastal projects were completed and appear to be in conformity with the original research priorities. Only one of the four land-based studies appears to be in line with the main concerns of the program. The research component suffered from a lack of focus and a deficit of coordination with the other project components. Research on biodiversity has been conducted in Protected Areas in partnership with foreign institutions and Malagasy universities. The results have been published in collaboration with ANGAP. This component was partially achieved (60 percent for coastal studies and 25 percent for land-based studies). Support Activities 28. This component had two goals: (i) ensure coordination of the other project components, and (ii) develop national environmental policies and establish the tools and procedures for its implementation. 29. ONE was responsible for coordination of EP1 components, as follows: annual planning and fund allocation, monitoring and evaluation (M&E), studies, training, education, consciousness raising, consolidated accounting and procurement, and liaison with funding agencies. However, ONE was able to carry out only some of these responsibilities and, on the whole, ONE's performance was seriously weak. It approved EP1 annual work programs, served as the secretariat to the monitoring committee and participated as a representative on the boards of ANGAP and ANAE. ONE's weak under-performance is the result of a host of problems including: confusion over the proper role of a coordinating body, staff constraints, alliances between direct donors and the other implementing agencies that bypassed ONE, donor-induced shifts in the definition of ONE's main tasks, ONE's uncertainty of its own role within the larger context of the Malagasy government, and infighting between ONE and the other implementing agencies as well as between ONE and other government ministries. Moreover, ONE's responsibility for M&E was not implemented, and as a result, EPI as a whole has suffered. Given the lack of a fully operational M&E system, linked to environmental policy formation, which 6. The Government clarified that, "CFSIGE has been responsible for environmental education only during EP2 and there was never any infighting between it and the Ministry of Environment over responsibilities for this activity. CFSIGE is an implementing agency, while the Ministry supervises the technical side of the Program. Between the Ministry of Education and the Ministry of Environment, a leadership conflict was observed in connection with formulation and design of an environmental education policy, in the context of EP2; the draft of this policy is under discussion within the Executive Branch." 8 persists to date, there is no reliable information whether the core EPI goal of biodiversity conservation is being attained.' This component was not achieved. 30. (ii) The environmental policy and procedures component included the development of national environmental policies, legislation, and regulations, environmental impact assessment (EIA) procedures, a monitoring and evaluation program, an environmental data management system, and representing the country in international environmental fora. While an environmental framework law (Charte de l'Environnement) and a national environmental impact assessment decree (MECIE) were adopted and policy studies for several sectors were completed, it remains unclear whether the government has assumed effective stewardship of the existing process or ownership of the sector policy recommendations. To date, environmental capacity within sectoral agencies remains weak or are non-existent. This component was barely achieved. 3. Results and Sustainability Achievement of Project Objectives 31. While none of EPI's seven components have been fully implemented, and its biodiversity conservation impact is unknown, EPI did have many achievements. It led to the development of a Forestry Policy and the incorporation of good techniques for managing protected areas and forests. EPI had a strong human resources development content for ANGAP. This technical strengthening, coupled with the USAID SAVEM Program's financing of major Integrated Conservation and Development Projects (ICDPs) and basic infrastructure, were key factors in the decentralization of ANGAP's operations. International conservation and aid organizations managed 19 ICDPs in protected areas, under ANGAP's supervision through an Administrative Council composed of donors and the international NGO operators. This joint effort fostered technical exchange and helped increase global awareness of Madagascar's importance in global biodiversity. Through partnerships with global NGOs, forestry training at University of Tana has markedly increased its social and environmental orientation. This new orientation is more relevant to Madagascar's conditions than the previous, mainly technical, orientation. EP1 did modestly achieve some institutional and environmental management goals, improve local understanding of biodiversity loss and its consequences, and introduce participatory techniques into project implementation. 32. Despite these successes, EPI had fallen short in a number of areas. It adopted an "enclave" approach that did not effectively address the economic and social pressures on biodiversity resources. Moreover, EP1's strategic objectives were ill-defined (e.g., which kind of biodiversity should be conserved as a priority? Which biodiversity uses should be promoted?). Coupled with an overly ambitious set of objectives and project design, EPI demanded implementation capacities that the new institutions could not deliver. Consequently, the activities implemented under EP1 did not demonstrate an understanding of the methods for dealing effectively with 7. The Government is of the view that, "While it is true that the M&E system for EP1 did not reach the desired level, especially as regards global indicators of large-scale social, economic and ecological impacts, the ICR acknowledges the existence of M&E systems at implementation agency level at the end of EPl. The assertion that no M&E system has existed so far does not accord with the facts: the indicators for EP2 were negotiated at the same time as its financing, with the result that data are now systematically gathered for each component, and then consolidated by ONE. In implementing agency intervention zones, there is intensive monitoring of the abovementioned indicators, so that the system is already well-established and running smoothly." 9 economic and social pressures in perimeter zones, where the local people earn their livelihood through forest exploitation. EPI also did not show a practical understanding of the need and means for integrating conservation within an overall natural resource management strategy. In the end, EP1 became trapped in a conservation mentality that perhaps unintentionally categorized the local community as encroachers to be kept at bay, thus forfeiting community support for biodiversity goals. A high number of key informants characterized EPI as "too much conservation, too little development".! 33. Despite the fact that EP1 was the first of three phases and therefore not intended to solve all of Madagascar's environmental problems, the project must be evaluated on the basis of its overly ambitious objectives as stated in the Staff Appraisal Report. EP1's marginally unsatisfactory outcome is based on the fact that its the physical components were only barely executed, or not at all.' There is also no adequate information whether EPI made any progress in achieving its goals of biodiversity conservation and improvements in people's living standards. EPI met with only limited success, with 8 out of 14 indicators under-performing significantly. The effectiveness of EPI was undermined by its excessive scope and ambition. Its most important accomplishment was establishing three new environmental agencies. However, the inherent complexity involved in coordinating at least six separate agencies, many without previous experience, stymied effective actions. Institutional Development Impact 34. Many of the efforts under EPI were directed at institution building. When the NEAP was launched in 1989, Madagascar lacked national environmental institutions. EPI set out to build environmental thinking into all levels of Malagasy society, where previously it was largely nonexistent. Indeed, EPI did achieve some institutional and environmental management goals, improve local understanding of biodiversity loss and its consequences, and introduce participatory techniques into project implementation. However, the success of EPI must be measured by the degree to which sound natural resource management and environmental sustainability actually have been incorporated into both private and public actions. The project documents point out that "institution building takes a long time to translate into material and quantifiable benefits." Nevertheless, beyond the relatively simple task of establishing an institution on paper and in a physical location, indicators must be identified and used to determine whether an institution is fulfilling its stated mandate and there is little evidence of the effects of these efforts. The overall institutional development impact of EP1 is modest. In 1995, a Beneficiary Assessment to evaluate the impacts of the first five years of the NEAP indicated the 8. The Government clarified that the aim of certain actions taken during EPI was to restrict access to Protected Areas, which were not always well understood. However, PAs throughout the world are subject to controlled access, and this approach is just a start toward application of the law in force in Madagascar (enacted well before EPI) and that the Malagasy State is always being criticized for not applying the law. Moreover, the Government is of the view that there is a bias in implying that EPI emphasized conservation too much and development not enough. The Program led the way in a good number of "development" fields, and in many regions unreached by other development players. 9. According to the Region, "EPI was explicitly designed as a first of three phases-the one to put in place the institutional framework. Given the long term view that was taken from the onset, what matters is that the institutional setup is stable and sustainable at the end of EP3. It was never foreseen that ONE, ANGAP, or ANAE would be fully sustainable at the end of EPI, and not even at the end of EP2. The ICR rated the project as satisfactory because it paved the way to the next phase, and the lessons learned have been incorporated into the design of EP2. This has been recognized by all the donors that decided to contribute to EP2, including the Bank itself." 10 lack of public confidence in the institutions charged with steering the NEAP toward its core objectives."o 35. Generally, the population strongly mistrusts the agencies, outsiders, and expatriates involved in EPI. Many local groups viewed EPI's implementation "champions" as intrusive in their lives and insensitive to their needs. They felt that environmental protection has been placed above their inherent right to work the land that provides their livelihood. Criticism has been especially sharp regarding EP l's park and protected areas component. Actions by NGOs and ANGAP to block access to forests without providing adequate compensation or alternatives have produced conflicts. Promises have also not been kept, further heightening the mistrust between beneficiaries and EPI workers. For example, specific development components, such as building schools, have not been executed. Animosity between agents and clients has made the institutions far less effective in carrying out their work, thus impeding EP l's goal of participatory development. A persistent problem has been the poor outreach and communication, partly due to a large number of components, subcomponents, and activities that are difficult to understand. 36. The program failed to put in place the key elements of a long-term foundation for environmental conservation and management, including sustainable institutions, regulatory frameworks, databases, M&E, training, and sector policy studies. Consequently, important institutional handicaps persist under the Second Environment Program. Project Sustainability 37. The Bank's interventions in Madagascar's environment through the EP1 and the ongoing EP2 have not yet yielded significant and measurable results." An important criterion for evaluating the Bank's development effectiveness is the likelihood for long-run sustainability of its interventions. Environmental sustainability is unlikely given the lack of evidence on the impacts of the programs. The lack of information on results based on outcome and impact indicators is a central issue. Without an M&E system, it is impossible to determine the extent to which EPI has met its objectives of developing human resources, promoting sustainable development in the management of natural resources, conserving and managing the country's legacy of biodiversity, and improving the quality of life for rural and urban populations. Until the M&E systems being set up under EP2 become fully operational, yield evidence of results, and generate biodiversity values, establishing net benefits and sustainability of the environment programs will remain problematic. 38. The institutions created by EPI require further strengthening and sustainable financing sources. The environment has not been mainstreamed within other sectoral agencies or within society as a whole. Given EPl's limited achievements and EP2's lack of concrete evidence on 10. Office national de l'environnement, Banque mondial, Miara-Mita, 1995, Evaluation par les bieficiaires du programme environnemental I (PEl), Antananarivo, Madagascar, (November). I1. See, "An Evaluation of the World Bank's Assistance for Madagascar's Environment Programs: A Country Case Study (CCS) for the Report on 'Promoting Environmental Sustainability in Development: A Review of the World Bank's Performance,"' Operations Evaluation Department, The World Bank, Washington, DC (March 13, 2000). This report was sent to the Government of Madagascar in April 2000 as input to the multi-donors' self-evaluation and midterm review. 11 local benefits and their positive impacts on poverty alleviation, based on today' s evidence, the environment program's sustainability is unlikely.12 4. Borrower and Bank Performance Borrower Performance 39. The government that designed and committed to the NEAP (1987-90) is very different from the governments that implemented the ensuing environment programs (1991 to present). Madagascar's recent history has been characterized by political factionalism, hesitant economic management, and a leadership that is unsteady in its partnership with the Bank and the IMF. A continuation of this on-and-off reform record does not bode well for a large, complex, and ambitious environment program that remains to be mainstreamed across several economic sectors and requires an integrated and sustained policy reform framework. While the borrower's performance improved from the end of 1992 onwards, on the whole it was unsatisfactory. The borrower performed basic tasks of complying with covenants and providing a fraction (20 percent) of counterpart funds. However, it was unable to carry out EPI's very challenging components related to institutional development and the generation of local benefits in order to take people away from deforestation, thus conserving biodiversity. EPI did not foster, as expected, a constructive collaboration between the new environmental institutions and the borrower's other agencies who, to date, have not bought into the environmental agenda and the leadership role claimed by the environmental entities." Bank Performance 40. The Bank's performance at entry was flawed. The program was overly complex and excessively ambitious for a country with no environmental institutional framework. Moreover, the intricacy of environmental protection and tremendous amount of effort required to achieve sustainability, especially given the capacity in the country, appear to have been overlooked. While institutional risks were identified, little was done to mitigate them. New agencies were established without clearly defined goals and objectives, as the ICR recognizes. The project files yield numerous back-to-office reports, memoranda and reports during the first two years of EPI that strongly criticized project design. The donors, in particular, were especially concerned about 12. The Government is of the view that it is at the local level that the impact of EP2 is being felt most, in the form of conservation activities (ICDPs still in existence supported by bilateral donors, national parks bringing the benefits of eco-tourism, etc.), soil conservation activities, actions to improve rural living standards, settlement following land titling, etc. The problem lies, rather, in making the transition of from one scale to another: from success at the local level, to success on a broader scale (community, region). 13. The Government is of the view that, "The mainstreaning of environmental concerns by other sector agencies and the population at large is much more extensive than indicated here. In point of fact, almost all sectoral policies now incorporate the environmental dimension, and the practice of drawing up environmental impact assessment reports on investment projects is becoming increasingly widespread. The duty to protect the environment is covered in the Malagasy Constitution, and that the National Council for the Environment (established during EP2) is consulted by the Government on major sectoral decisions whenever protection of the environment is significantly or tangentially involved (e.g., in connection with road program, petroleum sector, mining sector, industrial sector, tourism, etc., etc.). At the regional or local level, stakeholders and authorities are becoming increasingly involved in the diagnosis and resolution of environmental problems." 12 the ambitiousness of EPI and the attendant high risks. Numerous reviewers and agencies repeatedly raised issues related to EPI's scale, stringent technical requirements, the lack of local expertise, the political situation, and the lack of community participation. Several memoranda reflected an insistence on the Bank's part for a comprehensive program to carry out all the components and to create ONE, while a group of donors opposed costly and complex components such as the one related to mapping, cadastre, and remote sensing being integrated into EPI, which was eventually barely achieved.14 The records show the repeated blaming of the government for ONE's severe weaknesses during the latter years and at the closing of EPI, without much reference to the early and persistent warnings by reviewers and donors about the risks of creating a new environmental agency with a sweeping mandate when local capacities were: very low. 41. Also, the supervision missions did not adequately meet the borrower's needs. The 14 supervision missions, while relatively frequent, were mostly led by agricultural specialists and did not benefit from the participation of institutional development and environmental (particularly biodiversity) specialists. The borrower, while expressing overall satisfaction with the Bank's performance, also articulated its consternation at the fact that there were four task managers in the span of five years. Each of these task managers had a different vision and agreements that had been reached with one task manager often had to be renegotiated with the new one. On the whole, the analytical underpinning for the design of EPI was weak when matched against (i) the technical complexity of conserving biodiversity by creating sustainable livelihoods that reduce poverty and take rural people away from deforestation activities, and (ii) the absence of monitoring and evaluation systems, which are only becoming operational midway through EP2. The fundamental drivers that create incentives for environmental degradation were not well identified at the design stage, and those that were identified were not amenable to drastic reform through the NEAP process. The amount of time required to lay institutional foundations and change behavior were seriously underestimated. Given these factors, OED rates the Bank's performance as unsatisfactory. 5. Overall Assessment and Lessons Learned Overall Assessment and Summary of Ratings 42. Overall, OED rates the outcome of the First Environment Program as marginally unsatisfactory because the components were only barely or partially achieved, and with the absence of M&E systems, it is unclear whether the project's objectives have been fully met. This differs from the rating of the Implementation Completion Report (ICR), which considers EPI's outcome as satisfactory despite the well-documented shortcomings discussed above, and the fact that the ICR itself rates EPI's core biodiversity components as unlikely to be sustainable. The institutional development impact of the project is rated as modest, bordering on negligible, because EP1 failed to put in place the key elements of a long-term foundation for biodiversity conservation and environmental stewardship, including (financially) sustainable institutions, regulatory frameworks, databases including those for biodiversity valuation as well as monitoring and evaluation, environmental education and training, and environmental mainstreaming in other official and private agencies. The institutions continue to lack coordination and their financial 14. The Government clarified that these elements were included in the project because at the time cadastral and mapping data in Madagascar left much to be desired. This information was necessary for the success of the project (or at least the biodiversity/Protected Areas component and the rural miniprojects component). 13 viability, once the ongoing EP2 closes and/or donor assistance decreases, is highly questionable. The audit agrees with the ICR rating for sustainability as unlikely. 43. Both Bank performance and borrower performance are unsatisfactory. The Bank's key shortcomings were in EP1's quality at entry, particularly with respect to the unrealistic scope of the project, and the quality of the supervision missions. The borrower's main deficiency was its inability to implement components that yield adequate local benefits, as well as its demonstrated lack of commitment to environmental stewardship and mainstreaming. These ratings differ from the ICR's satisfactory ratings for both the Bank and the borrower. Lessons Learned 44. The absence of evidence on that EPI achieved its goals of biodiversity conservation, coupled with improvements in people's living standards to take them out of forest exploitation, casts strong doubt on whether large environment programs are worth pursuing and whether they can be sustainable in very poor countries whose economies stagnate or have slow growth prospects. 45. The Bank's propensity to package activities into projects, projects into programs, and programs into action plans can lead to an intellectual construct so rigorous that (i) the Bank is unwilling to accommodate reality checks from independent reviewers and advisers, and (ii) the authors become imbued with an unfounded sense that implementation success can be assured. This rigidity and sense of pressure is highest when the Bank team perceives that it is complying with a policy mandate, such as the preparation of NEAPs for IDA countries. The risks, however, are very high when the programs are so demanding yet local capacity is low or nonexistent. 46. New national institutions require sufficient time to evolve in order to become effective in managing the environment, in such a way that it is mainstreamed across sectors and legitimized by broad-based acceptance. OED's review of NEAPs recommended that they should be adapted more to the needs, capacity, and planning horizons of the client countries, and that the environmental priorities identified in the NEAPs should be better integrated in country assistance strategies as well as economic and sector work. "1 Forcing institutional foundations to be laid, behavior to change, and the agencies to invent themselves within five-year project limits and a time-bound action plan (e.g., the NEAP) is artificial and highly risky. After over a decade, the experience with NEAPS shows that, while they have heightened environmental awareness, they have not resulted in any measurable impact on resolving the environmental predicament of the client countries. " 47. Lack of continuity in project and country management in the Bank results in frequent re- ordering of priorities that seriously diminish prospects of successful project implementation. 15. OED. Effectiveness of Environmental Assessments and National Environmental Action Plans: A Process Study. World Bank, June 28, 1996. Report No. 15835. 16. Environment Department. "National Environmental Strategies: Learning From Experience". World Bank, 1995.  15 Annex A Annex A: Basic Data Sheet FIRST ENVIRONMENT PROGRAM (CREDIT 2125--MAG) Key Project Data (amounts in US$ million Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 85.53 91.30 107% Loan amount 26.00 26.00 100% Cofinancing 42.30 64.66 153% Cumulative Estimated and Actual Disbursements FY91 FY92 FY93 FY94 FY95 FY96 FY97 Appraisal estimate (US$M) 3.4 10.2 15.6 19.9 24.0 26.0 26.0 Actual (US$M) 2.0 2.4 3.8 6.9 12.1 17.6 26.0 Actual as % of appraisal 58.8 23.5 26.4 34.7 50.4 67.7 100.0 Date of final disbursement: December 9, 1997 Project Dates Original Actual Identification November, 1988 Preparation December, 1988 Appraisal July, 1989 June, 1989 Negotiations March, 1990 Board approval April, 1990 April, 1990 Signing May, 1990 Effectiveness January, 1991 February, 1991 Closing date June, 1996 June, 1997 16 Annex A Staff Inputs (staff weeks) Planned Actual Weeks US$ ('000) Weeks US$ ('000) Preappraisal 99.2 205.6 Appraisal 97.6 189.5 Negotiations 11.8 29.6 Supervision 233.2 625.6 Other 7.9 29.1 Total 456.6 1098.9 449.7 1079.4 Mission Data Date No. of Staff days SfeIalizations Performance Performance Types of problems3 (monthlyear) persons In field represented rating' rating' Implementation Development Status Objectives Identification/ 12/88 7 19 Preparation Appraisal 6/89 12 FA, EC, E, J, C, ISS, EC, Ec, TR Post Appraisal 12/89 3 FA, EC, Ec, C, LT Supervision I 1. 7/90 1 FA 2. 12/90 3 12 FA,E,DS 1 1 3. 4/91 2 22 FA, DS 1 1 4. 12/91 2 F, DS 2 2 F 5. 6/92 3 18 AE, F, DS 3 3 F, M, PR 6. 12/92 2 17 AE, DS 2 3 F, M, PR 7. 4/93 1 9 AE 2 2 F 8. 10/93 1 18 AE 2 2 F 9. 12/93 17 AE, E, DS 2 2 (Mid-Term Review) 10. 4/94 2 17 AE, OA (partial) 11. 6/94 3 29 AE, YP, OP 2 2 12. 4/95 3 1? AE, EC, OA S S 13. 4/96 5 ? AE, OE, ISS, S S EC 14. 11/96 2 ? AE, EC S S Completion 5/97 2 20 E, AE I 1. AE=Agricultural Economist; C=Cartographer; DS=Donor's Secretariat; E=Environmentalist; EC=Economist, Ec=Ecologist; F=Forester; FA=Financial Analyst; ISS=Information System Specialist; J=Jurist; LT=Land Tenure Specialist; OA-Operations Analyst; TR=Training Specialist; YP=Young Professional. 2. 1 =No or minor problems; 2=Moderate problems; 3=Major problems; HS=Highly Satisfactory; S=Satisfactory. 3. F=Financing; M=Management; l=Implementation delays; IN=Institutional prblems; PR=procurement delays. 17 Annex B Annex B: Borrower's Comments To: Ridley Nelson Fax: (202) 522-3123 From: ALPHONSE Ministre de Date: 07/07/00 l'Environnement de MADAGASCAR Re: MADAGASCAR- Premier Pages: 1 Programme Environnement tCrédit 2125- MAO) CC: SEM Le Premier Ministre, Chef du Gouvernement, SEM Le Directeur de la Banque Mondiale à Madagascar Ul For Review O Please COmment C Please Reply 0 Pleatie Recycle Mals Rf V/L en date du21 juin 2000 Le Ministère de l'Environnement a pris connaissance du rapport d'évaluation rétrospective de la Banque Mondiale relatif au premier Programme Environnemental, objet de votre correspondance sus référencié, et prend acte de dos constats et perspectives y relatés. . Les conclusions dudit rapport, bien que en contradiction avec celles du rapport d'achèvement, ont été longuement analysées. Des leçons ont été tirées, et des mesures &cidées pour des orientations futures don't nous partageons tous la pertinence. Par ailleurs, j'esp-%re que les observations jointes à la présente permettraient à tous d'avoir une perception assez équilit.rée de ce Programme Environnemental et de ses acquis. Je cous prie d'agréer, Monsieur, l'assurance de ma très respectueuse considération, Siené: ALPHONSE Ministre de l'Environnemei .................. .. . .... .......... . .. .. .. ... . . 18 COMMENTAIRES SUR L'EVALUATION RETROSPECTIVE Les commentaires suivantes portent sur les conclusions de l'évaluation rétrospective sur l'exécution du projet et sur sa réalisation. Le Programme environnement, est sensible aux conclusions et propositions y contenues, et a déjà considéré cette évaluation lors de la récente revue à mi-parcours. Exécution du projet page 4, 5 13: « Pendant le PE 1, les dissensions entre eux (ONE, ANA E et ANGAP) se sont envenimées, chacun cherchant à empiéter sur le territoire de l'autre.. En outre, ils n'étaient pas aptes à mettre en oeuvre les mesures décidées », page 7, # 29 et page 8 : « rivalités entre l'ONE et les autres agences d'exécution, ainsi qu'entre l'ONE et d'autres services ministériels » L'évaluation a, dans cette assertion, versé dans une description négative des relations de travail qui existaient entre les AGEX (dont l'ONE) d'une part, et entre l'ONE et les autres services ministeriels d'autre part Si de telles situations existaient, les divers outils développés avec les divers ministères sectoriels pendant le PE-I rencontreraient une fin de non recevoir, ce qui n'est pas le cas. Par ailleurs, les mandats et les zones d'intervention des Agences d'exécution sont clairement définies, et des dispositions additives ont été adoptées lors du PE-Il pour une meilleure coordination. Réalisation du projet page 5, § 16: « Quarante quatre AP ont été bien établies mais à lafin du PE-I, elles n'avaient pas encore d'existence juridique » A la fin du PE-I, le changement de statuts pour certaines Aires protégées était en cours: des réserves naturelles intégrales étaient proposées pour être des parcs nationaux, et dans une perspective de génération de revenus aussi bien pour la population que pour l'ANGAP. Mais toutes ont eu une existence juridique bien assise, les limites sont bien déterminées quoique pas toujours matérialisés sur terrain. Des études étaient mêmes faites pendant le PE-I pour établir de nouvelles Aires protégées, dont les décrets de création étaient sorties au début du PE-II. page 5, f 17 : « Les activités de développement local ont été appuyés par des PCDI (19 des 44 aires protégées, soit 43% du total). Le nombre des familles bénéficiaires a été estiméà 13 482 même si 'ANGAP affirme que 65 000 familles ont directement ou indirectement bénéficié de ces activités. » et page 8, 8 31 : « Des organisations internationales d'aide et de conservation ont géré 28 PCDI (64 19 % du total ?) dans des aires protégées sous la supervision de l'ANGAP ...Cet effort concerté a encouragé les échanges techniques et aidé à sensibiliser l'opinion publique à l'importance que revêt Madagascar pour la biodiversité mondiale ». Il y a contradiction sur les chiffres, et beaucoup de confusion dans la compréhension : les PCDI sont avant tout des projets de conservation des Aires protégées, qui, dans leur approche, développent des activités alternatives aux pressions au niveau des zones périphériques tout en conservant la biodiversité. Les PCDI étaient la formule consacrée pour la conservation des aires protégées durant le PE-1 (et certains bailleurs de fonds continuent à financer des PCDI dans le PE-II). De facto, la population bénéficie à la fois des activités de développement (greniers villageois, techniques agricoles, infrastructures de production,..) et des infrastructures sociales (écoles, hopitaux..), ce qui engendre une ambiguïté sur la décompte des familles bénéficiaires (des villages entiers pour les infrastructures sociales, des familles et groupements paysans pour les infrastructures de production). L'ANGAP a su surmonter ce problème dans l'évaluation ex-post, ce qui n'a pas été le cas pour les évaluateurs. Par ailleurs, selon qu'on parle de l'ANGAP (une agence nationale) ou des « organisations internationales d'aide et de conservation » , les PCDI (les mêmes) sont perçus comme des échecs ou des succès (alors que l'ANGAP n'a jamais géré des PCDI). page 5, § 20 : « Compte tenu de l'importance qu'accordait cette composante (Conservation des sols, agroforesterie, reboisement et autres activités de développement rural - ANAE) du projet à la couverture d'une vaste superficie (plutôt qu'à l'intensité des activités) et du fait que seuls 11% de la superficie prévue a été couverte, l'objectif de protectié Z:des bassins versants n'a été atteint que dans une faible mesure » La composante en question se souciait à la fois de lrintensité des interventions que de la zone couverte. La preuve en est que ces deux dimensions faisaient l'objet de suivi et d'évaluations d'impact. Elle est appelée à la fois à atténuer l'érosion au niveau des bassins versants (le relief de Madagascar fait que 80% du territoire est constitué de bassins versants) et à améliorer le cadre de vie (de production et d'utilisation des ressources naturelles) au niveau des localités d'intervention. De la sorte, le déficit en couverture spatiale (11% de la superficie prévue) devrait être compensée par le gain à la fois en revenus pour la population touchée par les activités (de 20% à 85%, ce qui est rare dans les pays en développement, et très substantiel à Madagascar dont le taux de croissance économique est de l'ordre de 4 /), et que les pertes en terre (pour un pays essentiellement agricole comme Madagascar) est essentiellement réduit (de 200t/ha/an à moins de lt/ha/an; 1 e:valuation a retenu le chiffre de 0,5 t/ha/an), ce qui constitue une réussite exemplaire, quelle que soit son échelle. Si, de plus, les techniques utilisées sont des plus accessibles pour la population (dimension ignorée par l'évaluation) et très novatrice (le zéro labour entre autres), force est de reconnaître que l'évaluation a été un peu légère. Par ailleurs, a à travers ce volet, le programme, le mérite de ne pas aborder le développement rural de manière sectorielle, et de s'occuper des autres paramètres d'amélioration de cadre de vie en milieu rural, qui ne se projettent pas en termes de superficie couverte, et qui sont presque ignorées par l'évaluation rétrospective. Malgré tout cela, on reproche (cf page 9, § 32) au PAE de faire passer la conservation avant le développement: a trop sur la protection , pas assez sur le développment», optique qui semble être adoptée par les évaluateurs. page 7, f 26 : « d'intenses rivalités ont éclaté entre les personnels du ministère de l'environnement, du ministère de l'éducation et du Centre de formation des sciences de l'information géographique et environnementale (CFSIGE) au sujet des responsabilités et des mesures d prendre » Le CFSIGE n'a été chargé de l'éducation environnementale que durant le PE-II, et il ny avait jamais eu de rivalité entre le ministère de l'environnement et le CFSIGE sur les attributions en matière d'éducation environnementale, le CFSIGE étant une agence d'exécution, et le Ministère assurant la tutelle technique du Programme. Entre le Ministère de l'éducation et le Ministère de l'environnement, un conflit de leadership (et non une intense rivalité) a été observée pour l'élaboration et la définition de la politique d'éducation relative à l'environnement, dans le cadre du PE-II; le texte définissant la politique en matière d'éducation relative à l'environnement est actuellement au niveau du gouvernement. page 8, § 29 : « absence d'un système opérationnel de suivi et évaluation, absence qui persiste à cejour » Certes, le système de suivi-éval -ation du PE-I n'a pas atteint le niveau souhaité, surtout pour les indicateurs globaux portant sur les impacts sociaux, économiques et écologiques à grande échelle. Pourtant, le rapport de fin d'exécution reconnaît l'existence de systèmes de suivi au niveau des Agences d'exécution à la fin du PE-I. Quant à avancer que le système de suivi n'existe pas, même à ce jour, relève d'une méconnaissance de la réalité : les indicateurs du PE-II ont été négociés avec le financement et font l'objet de collecte systématique au niveau de chaque composante, et d'une consolidation au niveau de l'ONE. Au niveau des zones d'intervention des Agences d'exécution, des suivis intensifs comprenant les indicateurs cités supra existent, et le système en place est actuellement bien rodé et fonctionnel. page 9, § 32: « le PE 1 insistait trop sur la protection, pas assez sur le développement » Certes, certaines mesures prises lors du PE-I consistaient à limiter l'accès aux Aires protégées. Elles n'ont pas toujours été bien comprises, ni par la population, ni par certains évaluateurs. Pourtant, partout dans le monde, les aires protégées sont à accès contrôlés. Ce qui n'est pas mentionné par le rapport d'évaluation rétrospective, c'est que cette démarche n'est qu'un début 21 de l'application de la loi en vigueur à Madagascar (adoptée bien avant le PE-1), et qu'on reproche toujours à l'Etat malgache de ne pas appliquer la loi. Par ailleurs, insinuer que le PE-I insistait trop sur la protection, et pas assez sur le développement est partial. Le programme a été précurseur dans bien de domaines « de développement* et dans beaucoup de régions où d'autres acteurs de développement n'arrivent pas à toucher. Une des reproches faites PE-1 en général est la tendance à se substituer aux projets de développement (ou même à certains services étatiques de vulgarisation) au niveau local et régional et aux PCDI en particulier; à cela s'ajoute, semble-t-il le coût des activités de développement qu'ils sont obligés de mettre en œuvre. page 10, § 36 : « Le Programme n'a pas réussi à jeter les bases d'une action de gestion de l'environnement, y compris des institutions durables, des cadres réglementaires, des données de base, des systèmes de suivi et d'évaluation, des programmes de formation et des études sectorielles. En conséquence, de graves handicaps institutionnels persistent dans le cadre de la deuxième phase du Programme environnement ». * Le PEl a développé le réseau des Airee protégées - c'est la stratégie adoptée par le PE 1 pour préserver la bit liversie malgache face à rurgence devant la spirale de dégradation des ressources naturelles dans at pays. # Le PE1 a développé des nouvelles politiques de gestion des ressources naturelles renouvelables pour renverser un siècle de domanialité et de centralisme de gestion des ressources naturelles. Il a ainsi développé la nouvelle politique forestière et la politique de transfert de gestion des ressources naturelles aux communautés de base - ces politiques sont consacrées par des lois actuellement + Le PE 1 a développé la législation pour la mise en compatibilité des investissements avec l'environnement définissant les procédures d'évaluation de l'impact environnemental. * Le PE 1 a développé l'approche et les techniques pour les mini-projets de conservation de sols et d'amélioration de cadre de vie des populations rurales. Malgré les superficies encore minimes touchées par ces mini- projets, les impacts sont palpables, l'évaluation rétrospective les reconnaît d'ailleurs. La question à poser est le passage à plus grande échelle ? + Le PE 1 a réalisé l'inventaire écologique - forestier national et l'atelier de mise en commun de toutes les connaissances scientifiques sur la biodiversité malgache pour définir les priorités en matière de conservation. Les résultats de ces travaux ont en grande partie orienté le PE2 et ont élargi la stratégie de conservation de la biodiversité en dehors des Aires protégées. I2 # 11, § 38 : « Etant donné que le PE 1 n'a eu que des résultats minimes et qu'à ce jour, le PE 2 n'a pas fourni de preuves que le Programme a eu des avantages locaux et un impact positif sur la réduction de la pauvreté, il est peu probable que le Programme Environnement soit durable. » Au contraire, c'est au niveau local que les impacts du PE-II se matérialisent le plus, tant pour les activités de conservation (PCDI qui existent toujours, supporté par des bailleurs de fonds bilatéraux, parcs nationaux générant des bénéfices écotouristiques, ..), que pour les activités de conservation des sols et d'amélioration du cadre de vie rural, sédentarisation conséquente à la sécurisation foncière, ... Et les indicateurs sont suffisamment explicites sur cet impact (même le RER reconnaît que les résultats obtenus en conservation du sol, sont, au niveau local, très substantiels : baisse de la perte en terre, amélioration des revenus, ..). Le problème réside, plutôt, dans le passage à l'échelle: d'une réussite au niveau local, à une réussite au niveau plus large (communal, régional). page 11, § 38 et 39: « l'environnement n'a pas été intégré aux préoccupations d'autres organismes sectoriels ou de la population dans son ensemble » et « à ce jour, les seconds (les autres organismes de l'emprunteur) n'ont toujours pas adhéré à l'idée de protéger l'environnement ni suivi l'orientation donnée par les premières (les nouvelles institutions chargées de l'environnement) » La réalité sur l'intégration de l'environnement aux préoccupations d'autres organismes sectoriels et de la population dépasse ce qui est mentionné ici. En effet, presque toutes les politiques sectorielles intègrent maintenant la dimension environnementale, et la pratique de rétude d'impact environnemental pour les investissements est de plus en plus manifeste. Quant à l'adhésion de la population, l'assertion s'est fondée sans doute sur des observations marginales . Point n'est besoin de mentionner que le devoir de protection de renvironnement est mentionné dans la constitution malgache actuelle et que l'avis du Conseil national pour l'Environnement (mis en place lors du PE-I) est entendu par le gouvernement pour les grandes décisions sectorielles touchant de près ou de loin la protection de l'environnement. Les exemples abondent pour cette dernière remarque (programme routier, secteur pétrolier, secteur minier, secteur industriel, tourisme,...). Au niveau régional ou local, de plus en plus d'acteurs et de responsables s'impliquent pour le diagnostic ou la résolution des problèmes environnementaux. Page 12, § 40: la Banque a insisté sur la réalisation d'un Programme complet et sur la création de l'ONE alors qu'un groupe de bailleurs de fonds s'opposait à l'intégration au projet de composantes coûteuses et complexes comme celle de la cartographie, du cadastre et de la télédétection: L'intégration de ces dernières composantes au projet est conséquente au déficit existant à Madagascar à l'époque, en matière d'informations cadastrales et cartographiques. Ces informations étaient nécessaires à la réussite du projet 23 (au moins dans les volets biodiversité/aires protégées et mini-projets en milieu rural . Conclusions La complexité et lexhaustivité du PAE est un fait. L'effort fourni aussi bien par Madagascar que par les Agences d'exécution du Programme pour relever le défi en est un autre. Certes, les objectifs sont ambitieux (très ambitieux, trop ambitieux?) mais ils sont à la mesure de la spirale de dégradation de renvironnement qui est séculaie. Beaucoup reste à faire, et l'horizon de changement de comportement et de renversement de tendance dépasse les phases classiques des projets (dimension reconnue par 'évaluation rétrospective) ; vouloir tout faire et s'attendre à avoir des impacts en cinq ans relève de rirréalisme, qui dépasse l'ambition. La Charte de l'environnement malgache mentionne, en tout cas, que la mise en ouvre des dispositions pour enrayer la spirale de dégradation nécessite au moins un plan de quinze ans.  25 Annex C Annex C: Borrower's Comments (English Translation) COMMENTS ON THE PERFORMANCE AUDIT REPORT The following comments relate to the conclusions of the Performance Audit Report on the implementation and outcomes of the project. The Environment Program is sensitive to the conclusions and proposals contained in it and has already considered this PAR during the recent mid-term review. Project implementation page 4,* para. 13: "Feuds and overlapping responsibilities developed between the agencies during EPI, thus hindering their effectiveness. Moreover, they lacked the capacity to implement established policies"; page 7, para. 29 and page 8: "infighting between ONE and the other implementing agencies as well as between ONE and other government ministries. With this assertion, the PAR moves into a negative description of the working relations between the implementing agencies (of which ONE was one) on the one hand, and between ONE and the other government ministries on the other. If such situations had existed, the various tools developed with the different sector ministries during EPI would have picked the matter up and required action on it, which was not the case. Moreover, the mandates and spheres of action of the implementing agencies are clearly defined, and further steps have been taken during EP2 to ensure better coordination. Project outcomes page 5, para. 16: "Forty-four PAs were established, but at closure of EP1 they had not been legally recognized" At closure of EPI, the change of status for certain Protected Areas was under way: integral natural reserves were proposed as national parks, with a view to generating income for both the population and ANGAP. However, they were all definitely legally recognized, with boundaries that had been firmly decided upon although not always marked in the field. Studies were also made during EPI as a basis on which to establish new PAs, the enabling decrees for which were published at the beginning of EP2. page 5, para. 17: "Local development activities were supported by Integrated Conservation and Development Projects (ICDPs) in 19 of the 44 Protected Areas (43% of the total). An estimated total of 13,482 families have benefited directly from these activities, although ANGAP maintains that 65,000 families have benefited either directly * Page nos. refer to the French version (Tr.) 26 Annex C or indirectly from ICDP activities." and page 8, para. 31: "International conservation and aid organizations managed 28 ICDPs (64% of the total?) in Protected Areas under ANGAP supervision.... This joint effort fostered technical exchange and helped sensitize public opinion to Madagascar 's importance in global biodiversity. " The figures given are contradictory, and there is evidence of considerable misunderstanding. The ICDPs are primarily projects for the conservation of Protected Areas. Their focus is on developing alternative activities to those causing pressures in the surrounding zones, and on conserving the local biodiversity at the same time. They were the formula adopted for the conservation of the PAs during EP1 (and certain donors are continuing to finance them as part of EP2). As a result, the population benefits from both development activities (village granaries, agricultural techniques, production infrastructure, etc.) and social infrastructure (schools, hospitals, etc.), but this makes it difficult to be accurate in calculating the number of beneficiary families (e.g. entire villages benefit from social infrastructure, individual families and farmer associations from production infrastructure). Although, according to the PAR, ANGAP succeeded in overcoming this problem, this was not true of the evaluation team itself. Furthermore, depending on whether one is referring to ANGAP (a national agency) or "international conservation and aid organizations", the ICDPs (the same) are perceived either as failures or as successes (although ANGAP has never managed any ICDPs). page 5, para. 20: "Given the importance of wide area coverage (versus area intensity) to success with the objectives of this component (namely, soil conservation, agroforestry, reforestation and other rural development activities - by ANAE), and the fact that ultimately only 11% of the target area was covered, the goal of protecting watersheds was barely achieved " This component was concerned with both the intensity of project interventions and the area covered. This is demonstrated by the fact that these two dimensions were the subject of monitoring and impact assessments. The component was designed to lessen erosion in watersheds (Madagascar's relief is such that 80% of the island is made up of watersheds) and to improve the living environment (for production and use of natural resources) in the localities targeted. The shortfall in spatial coverage (11% of the target area) should be regarded as offset by: (i) the higher incomes earned by the beneficiary population (from 20% to 85% higher, which is rare in developing countries and very substantial for Madagascar, whose economic growth rate is of the order of 4%); and (ii) by the fact that soil losses (important for an essentially agricultural country like Madagascar) have been reduced (from 200t/ha/year to less than It/ha/year), which is an exemplary success, whatever the scale of the operation. If, moreover, the techniques used are those most readily accessible to the population (a dimension the report fails to mention) and very innovative (zero labor, among other things), it has to be recognized that the PAR team's evaluation was a little on the light side. Furthermore, it is because of this component that the program can claim the merit of not approaching rural development on a sector basis but concerning itself instead with other aspects of the improvement of living standards in rural areas, aspects which do not 27 Annex C register in terms of area covered and are virtually ignored by the PAR. In spite of this, the PAR (cf. page 9, para 32) speaks critically of the emphasis the Environmental Action Plan puts on conservation over development: "Too much conservation, too little development", a view that seems to have been adopted by the audit team. page 7, para. 26: "......intense infighting among staff of the Ministry of Environment, the Ministry of Education, and the Geographic and Environmental Information Training Center (CFSIGE) as to spheres of responsibility and measures to be taken. " CFSIGE has been responsible for environmental education only during EP2 and there was never any infighting between it and the Ministry of Environment over responsibilities for this activity; CFSIGE is an implementing agency, while the Ministry supervises the technical side of the Program. Between the Ministry of Education and the Ministry of Environment, a leadership conflict (and not intense infighting) was observed in connection with formulation and design of an environmental education policy, in the context of EP2; the draft of this policy is currently under discussion within the Executive Branch. page 8, para. 29: "......lack of a fully operational monitoring and evaluation system, which persists to date ". While it is true that the M&E system for EP1 did not reach the desired level, especially as regards global indicators of large-scale social, economic and ecological impacts, the ICR acknowledges the existence of M&E systems at implementation agency level at the end of EPI. The assertion that no M&E system has existed so far does not accord with the facts: the indicators for EP2 were negotiated at the same time as its financing, with the result that data are now systematically gathered for each component, and then consolidated by ONE. In implementing agency intervention zones, there is intensive monitoring of the abovementioned indicators, so that the system is already well- established and running smoothly. page 9, para. 32: .......EP1 focused too much on conservation and too little on development ". It is a fact that the aim of certain actions taken during EP 1 was to restrict access to Protected Areas. These actions were not always well understood, either by the population or by some of the evaluation team. However, PAs throughout the world are subject to controlled access. What the PAR fails to mention is that this approach is just a start toward application of the law in force in Madagascar (enacted well before EP l) and that the Malagasy State is always being criticized for not applying the law. Moreover, to imply that EPI emphasized conservation too much and development not enough is biased. The Program led the way in a good number of "development" fields, and in many regions unreached by other development players. One of the criticisms of EP 1 in general has been its tendency to take the place of development projects (and even some government extension services) at the local and regional levels, 28 Annex C and of ICDPs in particular; it would appear that the cost of the development activities they are obliged to carry out should be added to this. page 10, para. 36: "The Program failed to put in place the key elements of a long-term foundation for environmental conservation and management - for instance, sustainable institutions, regulatory frameworks, data bases, M&E, training, and sector policy studies. Consequently, important institutional handicaps persist under the Second Environment Program. " * EP 1 developed the network of Protected Areas - the strategy adopted by EP I for preserving Madagascar's biodiversity, given the urgency created by the downward spiral of natural resource degradation in the country. * EP1 developed new renewable natural resource management policies to reverse a century of state control and centralized management of natural resources - for instance, the new forest policy and the policy of transferring natural resource management to grassroots communities, both now incorporated in current legislation. * EP1 developed the legislation requiring that investments be compatible with the environment, and defined the associated environmental impact assessment procedures. * EPI developed the approach and techniques for miniprojects focused on soil conservation and improvement of rural community living standards. Although the areas benefited by these miniprojects are still very small, their impacts are obvious, something the PAR acknowledges. The question now is how best to replicate these actions on a larger scale? * Under EP1, a national ecological and forest inventory was conducted, and a workshop on pooling all available scientific knowledge on Madagascar's biodiversity was organized as a first step toward formulation of national conservation priorities. The results of this work provided much of the foundation for EP2 and helped broaden the country's biodiversity strategy to encompass more than its Protected Areas. page 11, para. 38: "Given that the achievements of EP1 were limited and that EP2 has not so far produced clear evidence that the Environment Program has generated local benefits and had a positive impact on poverty alleviation, it is unlikely the Program will prove sustainable. " On the contrary, it is at the local level that the impact of EP2 is being felt most, in the form of conservation activities (ICDPs still in existence, supported by bilateral donors, national parks bringing the benefits of eco-tourism, etc.), soil conservation activities, actions to improve rural living standards, settlement following land titling, etc. 29 Annex C And the indicators are sufficiently explicit on this impact (even the PAR acknowledges that the results obtained in soil conservation are, at local level, very substantial: reduced soil loss, improved incomes, etc.). The problem lies, rather, in making the transition of from one scale to another: from success at the local level, to success on a broader scale (community, region). page 11, paras. 38 and 39: "Environmental concerns have not been mainstreamed within other sector agencies or within society as a whole," and "the Borrower's other agencies have still not bought into the environmental agenda or adopted the policies advocated by the new institutions vested with the responsibility for environmental matters. " The mainstreaming of environmental concerns by other sector agencies and the population at large is much more extensive than indicated here. In point of fact, almost all sectoral policies now incorporate the environmental dimension, and the practice of drawing up environmental impact assessment reports on investment projects is becoming increasingly widespread. As regards awareness and support on the part of the population, the assertion made is doubtless based on faulty observations. It need hardly be said that the duty to protect the environment is covered in the Malagasy Constitution, and that the National Council for the Environment (established during EP2) is consulted by the Government on major sectoral decisions whenever protection of the environment is significantly or tangentially involved (e.g., in connection with road program, petroleum sector, mining sector, industrial sector, tourism, etc., etc.). At the regional or local level, stakeholders and authorities are becoming increasingly involved in the diagnosis and resolution of environmental problems. page 12, para. 40: "The Bank insisted that the Program be comprehensive and that ONE be created, while a group of donors opposed incorporation in the project of such costly and complex undertakings as mapping, cadastre development, and remote sensing." These elements were included in the project because at the time cadastral and mapping data in Madagascar left much to be desired. This information was necessary for the success of the project (or at least the biodiversity/Protected Areas component and the rural miniprojects component). Conclusions The complexity and exhaustiveness of the EAP is a fact. The effort put forward by both Madagascar and the Program implementing agencies to meet the challenge is another. While the objectives in view are ambitious (very ambitious, too ambitious?), they are consistent in scale with the downward spiral of environmental degradation, which has been going on for a century. Much remains to be done, and the horizon for changing behaviors and reversing trends extends beyond the conventional project timeframe (a fact acknowledged in the PAR); wanting to do everything and expecting to see impacts in five years smacks of a lack of realism that goes beyond ambition. In any 30 Annex C event, Madagascar's Environmental Charter states that implementation of the provisions for halting the degradation spiral will require at least a fifteen-year plan. 31 Annex D Annex D: OED Response to Borrower's Comments The World Bank 1818 H Street N.W. (202) 477-1234 INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT Washington, D.C. 20433 Cable Address: INTBAFRAD INTERNATIONAL DEVELOPMENT ASSOCIATION U.S.A. Cable Address: INDEVAS August 2, 2000 Mr. Alphonse Minister of the Environment Republic of Madagascar Ampandrianomby Antananarivo 101, Madagascar Monsieur le Ministre: Performance Audit Report: Madagascar First Environment Program (Credit 2125--MAG) Thank you for your comments dated July 7, 2000 on the Performance Audit Report for the Madagascar First Environment Program. As noted in our letter of July 18, 2000, we have incorporated your comments in the report, and have replicated them in their entirety as an annex. We will be providing you shortly with a copy of the revised report. We note that in the set of comments several references have been made to the Performance Audit Report arriving at evaluative conclusions without sufficient documented evidence, or adequate time in the field. We would like to note that many of the evaluation conclusions were based on information and analysis derived from the Implementation Completion Report itself (World Bank Report No. 17297, dated January 15, 1998). The Implementation Completion Report, as you know, is the Bank regional staff's own self- evaluation upon project completion. For instance, the references to inter-agency feuds, the legal status of the Protected Areas, the number of families that benefited from the ICDPs, and the lack of established Monitoring and Evaluation systems are all confirmed by the Implementation Completion Report, which served as one of the key sources for the OED evaluation. We highly appreciate and thank you for the time and effort that went into putting together those substantive set of comments. Sincerely, Andres Liebenthal Acting Manager Sector & Thematic Evaluation Group Operations Evaluation Department