Mozambique - Coastal And Marine Biodiversity Management Project and P035919

 ICRR 12864 Report Number : ICRR12864 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/27/2008 PROJ ID : P070305 Appraisal Actual Project Name : Coastal And Marine US$M ): Project Costs (US$M): US$ 9.7 M US$ 10.16 Biodiversity Management Project and P035919 Country : Mozambique Loan /Credit (US$M): Loan/ US$M ): US$ 5.6 M US$ 5.98 M Sector Board : ENV Cofinancing (US$M ): US$M): US$ 4.1 M U$$ 4.18 M Sector (s): Sub-national government administration (47%) Other domestic and international trade (33%) General education sector (10%) Other social services (8%) General agriculture fishing and forestry sector (2%) Theme (s): Environmental policies and institutions (25% - P) Biodiversity (25% - P) Water resource management (24% - P) Participation and civic engagement (13% - S) Export development and competitiveness (13% - S) L/C Number : C3366 Board Approval Date : 06/04/2000 Partners involved : GEF Closing Date : 06/14/2005 06/29/2007 Evaluator : Panel Reviewer : Group Manager : Group : John Redwood Fernando Manibog Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: Development objective (DO): to carry out on a pilot basis an integrated approach to sustainable development in the project areas, including; (i) strategic integration of conservation measures with regional economic development; (ii) establishment and protection of conservation areas; (iii) initiation of conservation-oriented community activities; (iv) building the capacity of stakeholders responsible in the project areas for biodiversity protection; (v) raising public awareness of the importance of biodiversity conservation issues and measures; and (vi) establishing best practice for environmentally and biodiversity friendly development . GEO - to promote sustainable development in selected coastal districts of Mozambique's northern provinces and to protect coastal and marine resources in a network of protected areas . Both the DO (to test and refine an approach to achieve sustainable economic development of coastal zone resource ) and the GEO (to ensure the effective protection of globally significant coastal and marine habitats and species ), however, were stated in more general terms in the PAD . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): 1. Spatial development planning (appraisal: US$ 1.7M; actual: US$ 1.22M) 2. Biodiversity conservation and sustainable community development (appraisal: US$ 3.6M; actual: US$ 5.65M) 3. Sustainable private sector development (appraisal: US$ 1.3M; actual: US$ 0.07M) 4. Training and public awareness (appraisal: US$ 0.7M; actual - see comments below) 5. Project management and monitoring and evaluation (appraisal: US$ 2.4 M; actual: - see comments below) According to the ICR, the DO and GEO indicators were "adjusted" at the Mid-Term Review (September 2003) "to reflect changes in components " but elsewhere it states that these objectives were not "formally revised" even though the outcomne targets were. The table presented in the ICR only shows some minor changes in four project indicators (basically referring to changes in the timing of concessions, conservation area management plans, training programs, and NGO involvement) while one indictor was dropped altogether (again referring to concessions and private sector development). However, a major new investment not foreseen as such (although two much smaller "field stations" were) in the original project documents, the Pemba Research Center, was undertaken at the Borrower's request and approved by senior management during implementation under component 2 above. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The ICR does not differentiate among the costs of components 4 and 5 on completion, whose combined actual costs were reported to have been US$ 3.22M (as compared to a combined appraisal cost estimate of US$ 3.1M). However, while there was a significant underrun of the costs for combined components 4 and 5 charged to the IDA credit (US$ 1.72M actual, compared with the US$ 2.7M appraisal estimate), exactly the opposite occurred with respect to the GEF grant, where the combined costs of components 4 and 5 increased more than threefold from the US$ 0.4M estimated at appraisal to US$1.5M actual). The ICR clarifies that components 4 and 5 were "merged to streamline all administrative functions " within the project coordination unit, presumably at the time of the Mid -term Review (MTR). The team indicated further that due to decentralization of management to the provinces there were savings in the project management category (from 1.9 M SDRs estimated at appraisal to 795,000 SDR), part of which were reallocated to construction of the research center at Pemba . On the other hand, operating costs related to the GEF grant increased by US$ 127,000 in order to cover implementation of the community microprojects, which increased from the 5 estimated at appraisal to 52. Significant resources were also shifted from component 3 (private sector development) to component 2 (biodiversity conservation and commuity development ). Component 3 was dropped altogether at the time of the MTR reportedly due to delays in developing the strategic development plans under component 1, although an unexplained "influx of private sector in the districts before the completion of the SDPs " and differing "visions on priorities for the districts" by different government departments were mentioned as contributing factors . The substantial increase in the cost of component 2 seems to be exclusively due to the construction of a single large biological research center at Pemba at the Government's request during implementation (as opposed to the two small "research stations" foreseen at the time of appraisal ), whose construction ultimately cost US$ 3.9 million and was not included as such in the original project design (i.e., there is no mention of the Center in the PAD in which civil works assocated with the project were only estimated to be on the order of US$ 430,000). Component 1 was also "restructured to include policy and localized integrated development planning functions, to ensure consistency with: i) the district planning process supported by the Government and IDA at local level [although no further clarification is given ]; and ii) a programmatic approach and integration with coastal management program supported by DANIDA in Gaza and Inhambane provinces [not included under the project, whose pilot activities took place in Cabo Delgado and Nampula Provinces ]," but no further explanation is provided as to what this meant in practice and how it differed from the original project design . According to the cost table in the ICR, the borrower's contribution to project costs was US$ 950,000 for the IDA credit (as compared with the US$ 800,000 estimated at appraisal, but zero for the GEF grant for which the IDA credit must have served as counterpart resources . Finally, there was a two year extension of the closing date resulting from the aforementioned delays in implementing Component 1 and the additional time required to construct the large Pemba Research Center which was introduced into the project during implementation . 3. Relevance of Objectives & Design: Relevance of the project's objectives and design were - and are -- substantial although this does not come through as strongly as it might in the ICR. According to the ICR, project objectives "are still relevant to Mozambique's development priorities as defined by the Poverty Reduction Strategy Paper (PARPA II) and the Country Partnership Strategy (CPS 2008-11)." However, the argument presented to support this statement -- "the CPS recognizes that economic growth is closely related to significant investments to unleash the potential of the economy, coupled with sound management of the natural resource base and capacity to respond to disasters " -- is too vague and general to be convincing as an ex-post justification for this operation . Nor does the ICR refer to the original rationale for the project as stated in the PAD ("while the biodiversity and natural beauty of the pristine coast combine for a high potential for supporting sustainable coastal development, this potential is being undermined by inadequate capacity to manage the coastal resources of the country . Although development pressure in Mozambique is increasing rapidly, there is an opportunity to evolve institutional structures, technical capability, and a philosophy of natural resource management that will eventually lead to environmentally and socially sustainable development "). The original objectives and design of the project were -- and presumably still are -- relevant in view of the increasing pressures on the coastal zone (although, unlike the PAD which drew attention to them, the ICR is silent on this ) following resolution of the political conflict in the 1980s and '90s that resulted in effective protection of the coastal zone (and other rural areas) in Mozambique and, thus, of the rich and unique marine /coastal biodiversity (including five species of endangered sea turtles ) that depends on these areas . Project design, involving a pilot approach combining strategic planning with protection of high biodiversity coastal zones through establishment and proper management of protected areas in two pilot regions in northern Mozambique, support for private sector and local community involvement, including development of alternative livelihoods, and public awareness and training, was also relevant. 4. Achievement of Objectives (Efficacy): Achievement of project objectives as stated in the legal documents and ICR was modest . With respect to the GEO, the ICR affirms that "although the Project failed to realize the explicit protection of key coastal habitats, it did put some systems and plans in place that make realization of such benefits in the future more likely ." According to the ICR additionally, spatial development plans "incorporating the biodiversity dimension " were completed in all four pilot districts and were under implementation in two districts . But the ICR does not indicate the relative importance from a biodiversity standpoint of the two areas where implementation is taking place compared with that of the two where it is not. Similarly, two conservation areas were demarcated and management plans prepared and approved, but it is not clear from the ICR what the implementation status of these management plans is . However, the subsection on "potential future benefits of effective coastal protection " in the section on "efficiency" suggests that it has not commenced. On the more positive side. a research center was constructed (although not contemplated in the original project design) and biological monitoring of marine ecosystems, including small -scale fishing activities, had started, more than 50 community development "micro-projects" (but whose specific nature and results are not clearly identified) had been "successfully implemented," and the environmental management capacity of communities and local government "for sustainable management of natural resources " had been strengthened (also without elaboration). The strategic planning component of the project reportedly also supported preparation of a coastal zone management strategy and regulations for the prevention and protection against marine and coastal pollution, but the ICR does not specify the implementation status of these measures . Finally, the ICR does not rate performance of the planned private sector development component, which was dropped during project implementation, but it was clearly unsatisfactory. 5. Efficiency (not applicable to DPLs): Project efficiency was also modest . An incremental cost analysis is included as an annex in the PAD for the GEF-financed parts of the project . With respect to "efficiency," however, the ICR notes the following, "the economic analysis is limited to analyses of readily identified economic benefits; the lack of baseline information and absence of a results framework wtihin project monitoring prevents a reliable estimate of avoided costs from being made . Also, the cancellation of the private sector component prevented realization of any substantial employment and efficiency benefits that were identified at project appraisal ." A "collective financial IRR" (23.7 %) was estimated on completion for the small community development subprojects ("financed at an aggregate level of about US$ 700,000 over the project life plus an additional inkind contribution from the beneficiaries valued at US$ 70,000"), but the ICR points to serious sustainability concerns and suggests that the associated "overhead costs" were comparatively high (29% of delivered funds). In any case, there is -- and apparently was -- no economic or financial analysis of the project's single largest investment, the Biodiversity and Marine Research Center at Pemba, whose annual operating costs are expected "to exceed US$ 350,000 and are not yet entrenched in the government budget, " according to the ICR, which also describes it as "a potential direct cash liability ." In addition, the project took two years longer to implement than initially expected. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Based on a rating of substantial for relevance but modest for both efficacy and efficiency, IEG rates the overall outcome of this project as moderately unsatisfactory . The ICR itself affirms that "the project failed to realize the explicit protection of coastal habitats, " which was its declared global environmental objective, and indicates that it also largely failed as a "learning project" due to "an inadequate monitoring and evaluation system ." While the ICR argues that the project met most of its DO and GEO "indicators," it does not evaluate project outcomes in relation to project objectives and, thus, fails to provide a sufficient basis to demonstrate that it met many of these objectives as stated in the legal documents . In addition, one of the project's three major components failed altogether and results for the other two were only partially satisfactory at best and of doubtful sustainability (see the next section). a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: As noted in the ICR, even though spatial development plans (SDPs) were eventually finalized for 4 pilot districts and under implementation in two of them, it is "not clear what legal basis these plans will be given "the lack of adequate legal status within the planning framework puts their future use in doubt, " noting further that "the project could have benefited from policy and institutional reform to provide legal basis and recognition of the SDPs as a legal entity in the district planning process ." Elsewhere the ICR states that this is important because giving the SDPs legal standing "confers some certainty over tenure and resource use (by communities and potential private sector investors)". The ICR also indicates there "are some issues with replication of the [SDP] model in the country" due to "limited human and financial resources " and" weak technical and institutional capacity to meet future demand, " among other factors. The ICR likewise mentions uncertainties surrounding the gazettement of the "two conservation areas of recognized global importance " that were identified and demarcated under the project because "none of the existing categories under the current legal framework for conservation areas is considered appropriate for the areas proposed under the project, mainly because of the communities living inside the areas, " which, as a result, "might present some challenges for declaring these areas as conservation areas after the project closes ." Similarly, the sustainability of the small community development subprojects is questioned in the ICR ("some micro-projects in industry, fishing and tourism were implemented towards the end of the project, and it is likely that many of them will fail without further project support . Extension services are necessary to sustain the support on micro -project investment. However [the project] will close with no ability to provide a few years of critical monitoring and operating support for the investments.") The longer-term financial and institutional sustainability of the Pemba Biodiversity Center is also in question and is likely to depend heavily on (as yet uncertain) future generosity of external donors . Considering all of these factors, the ICR's judgment that there were "significant" risks to develoment outcome appears to be an understatement of the actual risks, especially as no specific follow -on project is clearly envisaged (according to the task team, a sector study will be carried out to identify a future Bank assistance program for natural resource management in the country which could well focus on coastal and marine resources and areas, especially given the extent of these areas and their vulnerability of the effects of climate change, but this will depend on the results of the study). a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: While both project and Bank performance, due to restructuring and in part to field -based supervision, appear to have improved after the mid-term review, this nonetheless seems to have been a case of "too little, too late." As the ICR affirms, "basic design issues and low level of readiness negatively affected implementation of the project." More specifically, project design emphasized the importance of strategic spatial planning, which, once significantly delayed, made the private sector development component inviable and affected the progress of other components as well. Even though the overall risk rating in the PAD was "substantial," country ownership and capacity risks -- and perhaps also the strength of opposing coastal "development" pressures (especially as compared with real government commitment to coastal /marine conservation goals) -- appear to have been insufficiently identified, weighed, and addressed during project preparation and early implementation . There was also an unclear relation between stated project objectives, components, and associated realistically achievable results, especially in view of the proejct's initial complex and similarly unrealistic institutional arrangements . Insufficient attention was given by the Bank to the borrower /implementing agency's lack of familiarity with Bank procurement and financial management procedures during preparation and appraisal . Bank supervision appears to have eventually given greater attention to procedural aspects (procurement, disbursements, financial management, etc.), although mainly after the mid-term review, but to have been less focused on -- and successful regarding -- more strategic and substantive aspects of the project . And, according to the ICR, the Bank also gave inadequate attention to the monitoring and evaluation system and project results framework . Finally, Bank supervision during the first two years of project implementation also appears to have been inadequate judging from the very low number of staff weeks and assocated costs identified in Annex 4, although it is not clear that this table is very accurate or complete (see section on ICR quality below ). a. Ensuring Quality -at-at -Entry :Unsatisfactory b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: The ICR highlights serious shortcomings in the Government and implementing agency performance, stemming both from poor ownership of the project's objectives and planned activities and weak coordination and institutional capacity: (i) inadequate provision of counterpart funds ("a problem from the beginning"); (ii) delays in hiring consultants for the SDP component, resulting in its delayed implementation (and cancellation of the private sector development component ); (iii) changes in Ministry staff; (iv) implementation agency withdrawal from the project very early on; and (v) two planned oversight bodies which never actually met . The ICR attributes this to "lack of ownership of the project " by some of the agencies involved, together with "lack of management experience" on the part of the main implementing agency, which the Bank seems to have done little at first to help correct. On balance, therefore, neither Government nor implementing agency performance, although the latter reportedly improved over time, appears to have been even moderately satisfactory . a. Government Performance :Unsatisfactory b. Implementing Agency Performance :Moderately Unsatisfactory c. Overall Borrower Performance :Unsatisfactory 10. M&E Design, Implementation, & Utilization: As the ICR states, "the project was designed as a learning project, with periodic reporting and review meetings for dissemination of the lessons learned but an inadequate monitoring and evaluation system prevented capture of most of the learning." In short, M & E was poorly designed, poorly implemented, and apparently not effectively utilized . a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): This is not entirely clear from the ICR, which does not provide much information on these aspects of project performance, except to note that there were "substantial weaknesses in financial management " early on but this situation improved over time as the result of "supervision inputs from the Bank . At one point, however, it hints that there may have been resettlement concerns in one or more of the project -supported conservation areas, but does not elaborate. This is one of the weakest sections of the ICR . 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately The project did not achieve its global  Satisfactory Unsatisfactory environmental objective and a critical component (private sector development) was dropped altogether following delays in implementation of another component (strategic planning). Nor is it clear that the project achieved its overall development objective. An "integrated approach to sustainable development" although ultimately quite different from that originally designed (e.g., the project's single largest investment was not mentioned in the PAD and only introduced after implementation had begun) may have been partially attempted, but the results have been modest at best and there are significant sustainability concerns. Risk to Development Significant High Based on the text of the ICR, risks Outcome : seem very high, especially in the absence of any follow-on interventions with Bank and/or GEF support. Bank Performance : Moderately Moderately Poor quality at entry and initial Satisfactory Unsatisfactory supervision could not be overcome even with improved supervision (which still seems to have remained deficient in a number of ways) following the mid-term review. Borrower Performance : Moderately Unsatisfactory Delayed and incomplete project Satisfactory implementation, lack of counterpart funds, oversight bodies that never met, and cancellation of the private sector component, reflected a combination of poor borrower ownership and weak capacity. Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: In addition to the lessons correctly highlighted in the ICR (e.g., M & E system should be an integral part of project design; small pilot projects should have modest objectives and indicators; and multiple implementing agencies make implementation difficult), all of which are generic ones, experience with this project also clearly illustrates the importance of two other generic preconditions for project success that : (i) strong borrower commitment to project objectives and planned interventions is essential; and (ii) sufficient institutional capacity, including adequate familiarity with Bank and GEF procedures and requirements, needs to be in place prior to project implementation . A third lesson which is more specific to this particular case is the need to adequately assess and monitor risks to project implementation and outcomes posed by increased tourism and other economic activities in the coastal zone. This may well have been one of the most important factors adversely affectingg both project and borrower performance in this particular case, but the ICR gives virtually no attention to it (see comments on quality of ICR below). 14. Assessment Recommended? Yes No Why? The ICR leaves many basic questions unanswered concerning the legal status of conservation areas, the implementation of conservation area management plans, and the nature and sustainability of small community development projects and a major biodiversity research center, which was not part of the original project design . Furthermore, it is essentially silent on one of the main factors that motivated the project in the first place, rapidly increasing pressures on the coastal zone and marine biodiversity by rising tourism and other economic activities . An assessment would seek to answer all of these pending questions and help to identify needed additional actions to effectively meet the project's original development and global environmental objectives . 15. Comments on Quality of ICR: The ICR does not base its assessment of project outcome on actual project objectives as stated in the PAD . It is also internally inconsistent, stating that project outcome was "moderately satisfactory" on the basis of the achievement of "most of PDO and GEO indicators" (rather than objectives), thus seemingly overlooking its own statement that "the Project failed to realize the explicit protection of coastal habitats, " which was precisely its GEO. The ICR does not provide specific information on the community development subprojects it financed or adequately explain the Bank's decision to agree to finance a costly biodiversity center at Pemba (or indicate on what analysis this decision was based) that was not part of the original project design . Nor does it sufficiently address project performance in relation to fiduciary and, especially, safeguard aspects even though it suggests there may have been some concerns around resettlement. In addition, the project cost table (Annex 1) appears to be incomplete and is unclear (e.g., no mention is made of the borrower's and beneficiaries' contributions to total project costs, although considerable attention is given to counterpart funding delays ). The Staff time and cost table is likewise very incomplete (e.g., only one staff member is identified as having taken part in project preparation and appraisal -- as compared with the 24 identified in the PAD -- and the associated budgetary costs of this input are similarly grossly understated ). Finally, although they are implicit in its discussion of project implementation experience and results, the ICR missed two of the most important generic lessons (see the section on lessons learned above ) stemming from this operation and gives no attention to an important contextual factor, (presumably) increasing "development" pressures in the coastal zone, which is identified in the PAD as one of the most important reasons for undertaking the project in the first place . a.Quality of ICR Rating : Unsatisfactory