Turkey - Eastern Anatolia Watershed Rehabilitation Project

Document of The World Bank Report No. 28274 PROJECT PERFORl'klANCE ASSESSMENT REPORT TURKEY EASTERN ANATOLIA WATERSHED REHABILITATION PROJECT (LOAN 3567-TR) March 19, 2004 Sector and Thematic Evaluation Operations Evaluation Department  Currency Equivalents (annual averages) Currency Unit = Turkish Lira (TL) 1994 US$l .OO =TL29,818 1998 US$l.OO =TL261,604 1995 US$l.OO =TL45,845 1999 US$l.OO = TL421,139 1996 US$l.OO =TL81,591 2000 US$1.OO = TL625,208 1997 US$l.OO =TL152,438 2001 US$l.OO = TL1,228,367 Abbreviations and Acronyms AGM Department of Reforestation and Erosion Control (MOF) AKRSP Aga Khan Rural CAE Country Assistance Evaluation CDD Community Driven Development EAWRP Eastern Anatolia Watershed Rehabilitation Project FCPCPS Farmer Centered Problem Census Problem Solving FD Forestry Department GEF Global Environment Fund GET Global Environment Trust ICR Implementation Completion Report KHGM General Directorate of Rural Services MARA Ministry of Agriculture and Rural Affairs MC Microcatchment MOF Ministry of Forestry MUHTAR Village Leader OED Operations Evaluation Department OGM General Directorate of Forestry ORKOY Forest Village Development Fund PCSU Project Coordination and Support Unit PPAR Project Performance Assessment Report QAG Quality Assurance Group TAGEM General Directorate for Agricultural Research TUGEM General Directorate of Production and Development Fiscal Year Government: January 1to December 31 Director-General, Operations Evaluation : Mr. Gregory K. Ingram Director, Operations Evaluation Department : Mr. Ajay Chhibber Manager, Sector and Thematic Evaluation : Mr. Alain Barbu Task Manager : Mr. Ridley Nelson  1 OED Mission: Enhancingdevelopment effectivenessthrough excellenceand independencein evaluation. About this Report The Operations EvaluationDepartmentassesses the programs and activities of the World Bank for two purposes:first, to ensurethe integrityof the Bank`s self-evaluationprocess and to verify that the Bank`s work is producingthe expected results,and second,to help develop improveddirections, policies,and proceduresthrough the disseminationof lessons drawn from experience.As part of this work, OED annuallyassesses about 25 percentof the Bank's lendingoperations. Inselecting operationsfor assessment, preferenceis given to those that are innovative,large,or complex; those that are relevantto upcomingstudies or countryevaluations;those for which ExecutiveDirectors or Bank managementhave requestedassessments; and those that are likelyto generate important lessons.The projects,topics, and analyticalapproachesselected for assessmentsupport larger evaluation studies. A Project Performance Assessment Report (PPAR) is based on a review of the Implementation Completion Report (a self-evaluation by the responsible Bank department) and fieldwork conducted by OED. To prepare PPARs, OED staff examine project files and other documents, interview operational staff, and in most cases visit the borrowing country for onsite discussions with project staff and beneficiaries. The PPAR thereby seeks to validate and augment the information provided in the ICR, as well as examine issues of special interest to broader OED studies. Each PPAR is subject to a peer review process and OED management approval. Once cleared internally, the PPAR is reviewed by the responsible Bank department and amended as necessary. The completed PPAR is then sent to the borrower for review; the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the OED Rating System The time-tested evaluation methods used by OED are suited to the broad range of the World Bank's work. The methods offer both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. OED evaluators all apply the same basic methodto arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (more information is available on the OED website: http://worldbank.org/oed/eta-mainpage.html). Relevance of Objectives: The extent to which the project's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Possible ratings: High, Substantial, Modest, Negligible. Efficacy: The extent to which the project`s objectives were achieved, or expected to be achieved, taking into account their relative importance. Possible ratings: High, Substantial, Modest, Negligible. Efficiency: The extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. Possible ratings: High, Substantial, Modest, Negligible. This rating is not generally applied to adjustment operations. Sustainabilify: The resilience to risk of net benefits flows over time. Possible ratings: Highly Likely, Likely, Unlikely, Highly Unlikely, Not Evaluable. lnsfifufionalDevelopment Impact: The extent to which a project improves the ability of a country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources through: (a) better definition, stability, transparency, enforceability, and predictability of institutional arrangements and/or (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements. Institutional Development Impact includes both intended and unintended effects of a project. Possible ratings: High, Substantial, Modest, NegIigible. Outcome: The extent to which the project's major relevant objectives were achieved, or are expected to be achieved, efficiently. Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Bank Performance:The extent to which services provided by the Bank ensured quality at entry and supported implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of the project). Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower assumed ownership and responsibility to ensure quality of preparation and implementation, and complied with covenants and agreements, towards the achievement of development objectives and sustainability. Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory.   ... 111 Contents Principal Ratings ................................................................................................................ v Key Staff Responsible ........................................................................................................ v Preface .............................................................................................................................. vii Summary ........................................................................................................................... ix Background ........................................................................................................................ 1 Findings ............................................................................................................................... 3 Lessons ................................................................................................................................ 4 Future Directions ............................................................................................................... 5 Analysis ............................................................................................................................... 6 Outcome................................................................................................................... 6 Relevance ................................................................................................................. 6 Efficacy .................................................................................................................... 7 How Effective Were Community Processes?.............................................. 8 How Effective Was the Traditional Catchment Approach Overlaid on the CommunityApproach?....................................................................... -10 How Effectively Were Skills Transferred?................................................ 11 Project Readiness....................................................................................... 11 Efficacy of the Community Approach in Forestry .................................... 11 Efficacy of Technical Work....................................................................... 11 Efficacy of the GEF Component................................................................ 12 The Efficacy of Monitoring and Evaluation .............................................. 12 Efficiency................................................................................................................ 12 The Economic Analysis ............................................................................. 13 Cost Recovery............................................................................................ 15 Institutional Development..................................................................................... -1.5 Did Women or the Poor Get Much Say in Prioritizing Investments?........17 NGO Involvement..................................................................................... -17 Bank Performance ................................................................................................. 17 Did the Bank do Enough on the Policy Front? .......................................... 18 Borrower Performance .......................................................................................... 19 This report was prepared by Ridley Nelson, who assessed the project in February 2003. The report was edited by William Hurlbut. and Helen Phillip provided administrative support.  iv Sustainability.......................................................................................................... 19 GEF Project................................................................................................ 21 Annex A Basic Data Sheet . .............................................................................................. 23 Annex B.Summarized Extracts from Village Focus Group Discussions ...................25 Annex C.Borrower Comments ...................................................................................... 31  V Principal Ratings ICR* ES* PPAR _I _I__ Outcome Satisfactory Satisfactory Satisfactory Sustainability Likely Likely Non-evaluable Institutional Substantial Substantial Substantial Development Impact Bank Performance Highly Satisfactory Highly Satisfactory Satisfactory Borrower Satisfactory Satisfactory Satisfactory Performance * The Implementation Completion Report (ICR) is a self-evaluation by the responsible operational division of the Bank. The Evaluation Summary (ES) is an intermediate OED product that seeks to independently verify the findings of the ICR. Key Staff Responsible Project Task Manager/Leader Division Chief/ Country Director Sector Director Appraisal Marjory-Anne Bromhead James Goering Michael Wiehen Midterm Tekola Dejene Sushma Ganguly Completion Nedret Durutan Marjory-Anne Bromhead Ajay Chhibber   v11 Preface This is a Project Performance Assessment Report (PPAR) for the Turkey Eastern Anatolia Watershed Rehabilitation Project (Ln. 3567-TR) for which a loan of US$77 million was approved on March 11, 1993. The project closed on October 30,2001, one year behind schedule. The final total disbursed was US$47.97 million equivalent to 62 percent of the original amount. Cofinancing was provided by the Global Environment Trust (GET). An Implementation Completion Report (ICR) was submitted on March 31, 2002 (Report no. 24181). A separate ICR for the GEF project for In-situ Conservation of Genetic Resources was submitted April 30, 1999.No separate PPAR is being done for that project. The PPAR was prepared by the Operations Evaluation Department (OED) based on the ICR, the Staff Appraisal Report, the Development Credit Agreement, review of Bank files, the GEF project ICR, mission interviews and focus groups by a consultant'. The project was discussed with Bank staff, beneficiaries, and government staff at the central, provincial, and district levels, and with NGOs'. The mission was in the field for about 10days and undertook field visits to Malatya and Kharamanmaras Provinces, visiting 16 communities selected in pairs at random initially but then adjusted in some cases for travel efficiency. Generally, farm households were interviewed in focus groups with men and women separately. The mission was free to select and talk to any households it chose in any micro-catchment it chose and many of the groups were interviewed without the presence of government officials or Bank operational staff. The cooperation and assistance of all stakeholders and government officials is gratefully acknowledged, as is the support of the staff of the World Bank Country Office in Turkey. The ICR is clear, informative, and well presented, although it leaves room for further drawing of lessons. The main reason for selecting this project for a performance assessment was to evaluate a project that was reported to have performed well in the area of Community Driven Development (CDD). The project was given the Bank's Award for Excellence in 1999.A second reason was that OED has a study underway on the CDD issue. The focus group sessions were designed to answer questions related mainly to the CDD evaluation. A third reason was that Turkey is a country on which OED is planning a Country Assistance Evaluation (CAE). Following standard OED procedures, the draft PPAR was sent to the borrower for comments before being finalized. 1. Ms. Sibel Astarcioglu. Annex B gives extracts of selected focus group comments. There were 14 focus groups done by the consultant following a guidance format and about another 7 partially following the same format by the PPAR task manager. Attendance ranged from 4 to 20. 2. Ms. Nedret Duratan (consultant) assisted the mission with both arrangements and substantive contribution.   ix Summary The impact of the various forms of Community Driven Development (CDD) interventions is of particular interest to the Bank given the large number and range of projects adopting this approach. The attached Project Performance Assessment of the Turkey experience in a rural project offers some important institutional lessons for the early evolutionary stages of this type of intervention. The project was nominated for a Bank Award for Excellence in 1999. A loan of US$77 million was approved for the Turkey Eastern Anatolia Watershed Rehabilitation Project on March 11, 1993. The project closed on October 30, 2001, one year behind schedule. The total amount disbursed was US$47.97 million, equivalent to 62 percent of the original amount. There was substantial devaluation over the project period providing a larger amount of local currency than originally projected. Partly due to this, but with the objective of spreading experience and skills, the project was spread to 11Provinces from the original 3, giving broader spread but lower density coverage. The main objectives of the project were to "help to restore sustainable range, forest and farming activities in the upper watersheds of the three project provinces, reducing soil degradation, erosion and sedimentation in reservoirs as well as increasing productivity and incomes in this impoverished region of Turkey." These objectives were to be pursued through efforts to improve productivity of range and forestland, promote production of fuelwood, fodder, and more sustainable use of marginal lands, facilitate the adoption of treatments for range and forestland to yield quick benefits, and to ensure increased involvement of local communities. A key underlying objective was environmental rehabilitation of degraded land. There was also a component for genetic resources conservation of indigenous species. The project was to be implemented through an interactive planning process whereby local implementing agencies would work together with villagers to prepare and implement a plan across a micro-catchment, defining interventions for improved range management, reforestation and improved soil moisture cultivation methods. There were five main components: (a) Watershed Rehabilitation (US$58.7 million base costs), including technical farming packages, improved range management on communal rangeland, increased productivity of government owned forestland widely used by villagers, and strengthening of the Forestry, Agriculture and Rural Services Directorates; (b) Income Supporting Activities (US$22.8 million) including beekeeping, horticulture, upgrading livestock, terracing, and small-scale irrigation. (c) Planning and Management (US$5.9 million); (d) Applied Research (US$0.8 million); and, (e) In Situ Gene Conservation (US$4.8 million) including conservation of wild crop relatives. The Global Environment Trust was to fund this. OED rates outcome satisfactory, in agreement with the ICR, sustainability non- evaluable compared with likely for the ICR, and institutional development substantial in agreement with the ICR. Bank performance is rated as satisfactory compared to highly satisfactory and Borrower performance is rated satisfactory in agreement with the ICR.  X On the issue of sustainability, the non-evaluable rating is because of uncertainties about the sustainability of parts of the forest planting due to grazing pressure, but against a long-term decline in grazing pressures nationally, and uncertainties about what elements of the interactive community and government processes will survive. A Bank performance of highly satisfactory in the ICR is assessed as too favorable although there were a number of aspects of strong Bank performance, including the introduction of a participatory approach in a very traditional society and the achievement of exceptional institutional coordination. But there were some weaknesses in strategy, community forest management and land policy issues, assessment of treatable rangeland, identifying and targeting the poorer households and women, and early attention to M&E, especially the measurement of soil and water impact. The main project achievements included the establishment of community based participation processes mainly involving consultation and using a problem solving community interaction technique with villages mapped to micro-watershed catchments. It did not reach as far as achieving what is understood today as community driven development including control over financial resources and often communal land assets too. It achieved the planting of nearly 100,000ha of trees and other treatments on forest department land -about 50 percent higher than the target, and substantial funding of agriculture supporting activities, although somewhat below target, including irrigation aimed partly at compensating for the loss of benefits on closed forestland areas. These achievements should reduce soil loss and flooding but with the largest impact in the longer-term and not yet measurable. The main weaknesses include the failure to achieve any significant portion of the substantial planned rangeland rehabilitation due partly to issues related to land ownership; the limited progress on establishing sustained processes that effectively raise the voice of women and the poorer households; and, concerns about project sustainability both at community level and government level in the absence of a follow-on project or some other government supported means to fully consolidate these new processes. The five main lessons are: (i) Pre-existing administrative or community processes, with risks of elite capture, often need to be challenged to accommodate the needs of women and the poorer households. (ii) Generally it takes more than the span of one project to develop and sustain new processes and skills to support community-driven development. (iii) Policies related to community forest management rights and responsibilities need careful analysis and possibly enabling legislative action in advance of a natural resources management project. (iv) In a project with substantial environmental objectives and often complex treatment trade-offs it is important to measure at least local environmental impacts. (v) In watershed treatments there are important issues of depth versus coverage, with potential trade-offs between high cost/high impact treatments on smaller land areas and low-cost low impact treatments on larger land areas. Gregory K. Ingram Director-General Operations Evaluation  1 Background 1. The government of Turkey has been attaching increasing priority to sustainable environmental management and natural resource conservation. Soil erosion is one of the most serious problems affecting the sustainability of agriculture. Approximately 16 million hectares are affected, over 70 percent of the cultivated or grazed land area. It is a particularly severe problem in the three provinces initially selected under the project which are in the upper watershed of the Euphrates River Elazig, Malatya, and - Adiyaman. Over one-third of the land in Turkey has slopes of more than 20 percent. Turkey has 21 million hectares of cultivated land about 3.6 million hectares of which are irrigated. The country is largely self-sufficient in food and has substantial agricultural exports. Agricultural GDP growth rate averaged 1.4percent over the period 1982to 1992, then 1.1percent over the period 1992 to 2002, but with wide fluctuationsrecently with 2001 at -6% and 2002 at +7.6%.About four million households are engaged in agriculture.Agriculture is 13percent of GDP. Crops contribute about a half of agriculture GDP, animal products about a third and forestry and fisheries the remainder. 2. Cultivated land is dominated by cereal production accounting for nearly 50 percent of the cultivated area. Rangeland, accounting for about 16percent of area, have been reduced substantially in area over the recent decades as fertility declined and as sloping land has been brought under cultivation. But livestock numbers have fallen too. Farmland is largely privately owned. Average farm size is about 6.5 hectares. Fragmentation of holdings is an increasing issue making soil and moisture conservation more difficult. Poor management of rangeland has increased pressure on forestland. Only about 40 percent of forestland is classified as productive. The mountainous areas of Eastern Anatolia are among the least developed areas of Turkey. The main agricultural sector objectives of the Government of Turkey are to: (a) modernize production techniques, to raise productivity, yields, and farmers incomes and to reduce dependence on the weather; (b) maintain the food requirements of the population; and (c) promote agricultural exports. 3. The Eastern Anatolia Watershed Rehabilitation Project (EAWRP) was a natural resource management project in selected micro-catchments of the upper watersheds of the Euphrates River. Treatment in the initial three provinces of Elazig, Malatya, and Adiyaman was planned to reach about 250,000 hectares on a total three province area of 2.9 million hectares - a coverage of about 9 percent of total Provincial land area. In the event the total actual area achieved was about 160,000hectares, but in 11provinces, much reducing the percentage coverage but the intention was to spread the pilot participatory process experience more widely. The areas selected, being in critical sub- watersheds, had a disproportionate contribution to erosion. The shift Rural human population has been falling in Elazig and Malatya but was still rising somewhat in Adiyaman. It was the first project of its type in Turkey to be built around a community participation approach to enable a negotiated balancing of longer-term watershed management treatments such as closure and afforestation of degraded hillsides with shorter term income generating investment such as improved forage crops, irrigation, and horticulture. The objectives of the project were stated as: to "helpto restore sustainable range,forest andfarming activities in the upper watersheds of the threeproject provinces, reducing soil degradation, erosion and sedimentation in reservoirs as well as  2 increasingproductivity and incomes in this impoverished region of Turkey." These objectives were to be pursued by efforts to improve productivity of range and forestland, promote production of fuelwood, fodder, and more sustainable use of marginal lands, facilitate the adoption of treatments for range and forestland to yield quick benefits, and to ensure increased involvement of local communities. A key underlying objective was environmental rehabilitation of degraded land. There was also a component for genetic resources conservation of indigenous species. 4. Components. There were five main components:(a) Watershed Rehabilitation (US$58.7 million base costs), including technical farming packages, improved range and management on communal rangeland, increased productivity of government owned forestland widely used by villagers, and strengthening of the Forestry, Agriculture and Rural Services Directorates; (b) Income Supporting Activities (US$22.8 million) including beekeeping, horticulture, upgrading livestock, terracing, and small-scale irrigation. (c) Planning and Management (US$5.9 million); (d) Applied Research (US$0.8 million); and, (e) In Situ Gene Conservation (US$4.8 million) including conservation of wild crop relatives. The Global Environment Fund (GEF) was to fund this. The latter was not evaluated in depth as a part of this PPAR. That would have called for a specialist mission. But the documentation of performance was reviewed. 5. The heart of the project was the community participation processes. While this process took some time to become fully established, essentially the approach involved: P initial visits by technical specialists to communities in the potential selected micro-catchments (there are a number of communities in each micro-catchment); P meetings with the muktar (the elected village leader -a form of village mayor); > one or more open village meetings with the community; P a so-called Farmer-Centered, Problem-Census, Problem Solving (FCPCPS) Exercise in which, with the help of facilitators, technical staff, and the muktar, each family writes down lists of problems, consolidates these in small groups, then the small groups offer findings to a plenary to develop a final village list; P this list is then prioritized and technical staff offer ideas about how the project could help. Options for closing grazed areas are discussed at this point. The community, at any point, is given the option to reject project assistance; 9 individual households arethen selected aspotential beneficiaries andeventually have to sign a Memorandum of Understanding. 6. With slow disbursement and depreciation of the currency, a substantial expenditure shortfall resulted in a decision in 1997 and then again in 1999to expand the number of micro-catchments (MC) tackled from the initial 54 in 3 provinces to a final 87 in 11provinces. The aim was to spread the experience more widely. There were some delays with these and some MCs were unable to complete investments before project closure. Final project cost after a one year extension was US$78.3 million against US$109.8 million appraisal estimate.  3 Findings 7 . The project largely delivered what it intended with respect to inputs (as opposed to outcomes or outputs) in the selected micro-catchments on forest land and arable land - but there was failure on rangeland. In other words, trees were planted, terraces were constructed, and agriculture was improved, especially forage, but there was limited change in range areas. Whether the inputs put in place will translate into the intended longer-term impacts such as reduced flooding and sedimentation in dams it is too early to say. At costs of around $400 up to $750 per hectare, there are questions about the efficiency of resource allocation and about the national capacity to scale up. There were weaknesses in directing benefits towards women and the poorer households and in M&LE.~ With respect to processes, important experience was gained by the public sector with community based participation. But this was not really a full Community Driven Development project as understood today. Communities were consulted but had limited control over decisions and none over the management of financial resources. Whether these processes can be sustained and taken further is uncertain. There was very good - unprecedented in Turkey -coordination between ministries and departments The lessons have been quite well accommodated in the follow-on project design. However, it now appears uncertain, perhaps even unlikely, that this project will proceed due to lack of commitment in some government ministries. Whether this reflects insufficient commitment on the community approach itself or whether it is simply a temporary budget expediency is still unclear. 8. Findings of particular importance include the following: e The project achieved a nationally important learning experience through a series of first steps with rural community participatory approaches (para 14). although it was not a full CDD approach. e useful national rural coordination experience (para 31)'. Inter-agency coordination was exceptionally strong and has provided a very a There were impressive technical achievements in afforestation of degraded hillsides with shorter-term impacts on local flooding and possibly local soil loss and much longer-term impacts on basin sedimentation (para 13).But, unless expanded greatly, the latter will be modest in scale because it is such a small percentage of the total basin. e A total actual project cost per household of about US$2000 per household is high relative to many comparable projects (para 26). At this cost, and with treatment 3. Annex C gives selected quotes from Focus Group meetings and individual interviews. 4. The Bank Region notes rightly that good cooperation is always easier at provincial level where line agency staff know one another and are more likely to mix socially but is more difficult at central level in capital cities, OED agrees that in this case the central level coordination was greatly facilitated by task management from Ankara with the task manager ensuring that staff from different ministries met every two weeks. The Region notes that somehow such coordination at the center needs to be institutionalized better, not only in this sector.  4 costs reaching as high as US$750 per hectare, there are questions about the extent to which it can be scaled up. To get coverage there is a need to vigorously explore lower cost treatments, perhaps simply closure. 0 The Impact Study claims a more than doubling of incomes over three years but the methodology is problematic and, in fact, real incomes have fallen (para 29). However, it is difficult to separate project impacts from recent economy-wide changes. 0 Nearly half the project costs were for plantations on government land. There are some questions about the extent to which different types of household were really interested in this or simply went along with it to get the shorter-term agriculture benefits (para 19).Pressures to open grazing can be expected to continue. 0 Four effective project years is rarely enough to reach sustainabilityof community processes, particularly since in this case those processes had only reached an early stage of process development (para 44) With the project now closed, it is .5 doubtful whether sustainableprocesses have really been built to last. There is little evidence of similar decision-making in other community endeavors, except in one or two cases of project-created Water Users Associations. Possibly the recently proposed government decentralization will help in due course. Lessons 9. There are five main generic lessons (more specific lessons for Turkey are in the Future Directions section): > As has been found in other cases, preexisting administrative or community processes, with risks of elite capture, need to be challenged to accommodate the needs of women and the poorer households. This will be so even if the main elements of the traditional structure are found to be essentially workable. It is especially so in cases such as this where elected officials may face conflicting incentives as both people's representatives and answerable to a government department. (paras 33 and 44) > This project, and wider experiences, suggest that generally it takes more than the span of one project to develop and sustain new processes and skills to support community-driven development. But experience suggests that with longer support agreed criteria for phased exit at community level need to be developed early in the process (paras 14and 44). > Policies related to community forest management rights and responsibilities need careful analysis and possibly enabling legislative action in advance of a natural resources management project. In this case there was insufficient attention to incentives for community forest management (para 39). 5. While the project lasted seven years, community processes were really only fully functioning for four years.  5 > In a project with substantial environmental objectives, and often complex efficacy and efficiency treatment trade-offs, it is important to measure at least local environmental impacts (para 22). This data may later be modelled into broader environmental impacts as treatment coverage spreads. > In watershed treatments there are depth vs. coverage trade-offs between high codhigh impact/lower coverage treatments and low-cost/low impact/higher coverage treatments. These need careful exploration at appraisal alongside "without project" scenarios to approach optimal overall impact (para 24. Future Directions 10. This paragraph offers specifically Turkey-oriented lessons with the focus mostly on the sector. There is strong learning from the first project evident in the design of the follow-on project - learning that has come from both the borrower and the Bank. Unfortunately, it appears that the follow-on project is now uncertain. Whether this is evidence of insufficient commitment to community-driven development approachesper se or a more temporary prioritization concern within a tightening budget is not clear. However, a number of issues warrant attention for the future. 0 Continued focus on policy, particularly related to the Constitutional constraints to increased community role in forest management and the incentives for marginal areas cultivation provided by the flat-rate direct subsidy (para 39). 0 Greater attention to challenging traditional community decision processes, in particular to demonstrate additional processes of wornens' consultation and poverty targeting including ensuring the inclusion of semi-nomadic livestock herders (paras 33 and 44)6. 0 A program to rapidly generate technical data for assessment of least cost impact in different soils and slopes (para 0 Greater attention to off-farm income support (para 29). Out-migration will be the future for many households, projects may be able to assist this process. 0 The setting up-front of community process-related targets with trigger points for phased graduation from intensive support (para 44) .' 6. The Bank Region argues that communities (and governments) get rather tired of process changes and want to see investments on the ground. OED agrees but notes that the report is not suggesting process change alone. 7. For example, even after many years of project treatment it is still not clear how full forest planting compares to partial, compares to deep-ripping, compares to open grazing, compares to closure, compares to alternative grazing management systems, etc. with respect to soil retention and flood reduction or useable forage production. 8. Difficulty in developing graduation criteria would be symptomatic of an as yet incomplete establishment of participatory community processes in the first place.  6 Analysis OUTCOME 11. Outcome is rated satisfactory on balance, mainly due to the substantially environmental focus of the objectives and the good achievements in planting of forest lands and intensifying agriculture. But there are a number of important reservations. The project was relevant, although the environment/welfare balance was probably not what communities (with incomplete environmental knowledge) would have voted for. Efficacy was ,on balance, substantial. Efficiency is also rated substantial,but with some concerns about low forestry returns, the high costs per household and per hectare, and the possibility of better cost effectiveness for long-term impacts of foresthange treatments by spreading investments more broadly but more thinly. Moreover, it is difficult to entirely divorce concerns about sustainability-rated non-evaluable - from the outcome rating given the sustainability implications in the environmental objective. RELEVANCE 12. The relevance of the project is assessed, on balance, as substantial.However, there is a question - the relevance to Appraisal could have whom? If households had been given a free vote on funding done a better job in allocation it is very doubtful that they would have chosen so diagnosis of the much forest land treatment. However, it is also clear that degradation problem in households do not have the information or the incentive to relation to changing address such larger issues as reservoir sedimentation. Moreover livestock pressures over wants are not the same as needs. The objectives - more time and the efficiency of environmental than growth related, although both are mentioned treatment impacts. - were clearly consistent with both the Bank's and borrower's strategy, both at the time of appraisal and now. As a first attempt at a rural community based participatory intervention it is questionable whether the objectives should have been stated largely in physical terms rather than in process and social capital terms'. The substantial environment focus seems to have been pushed even further in implementation. There appears to be more intent now to pursue 100percent protection in the Forestry Department (FD) plantings than is implied in the appraisal report economic models. The project was characterized by both project and Bank staff as being 60:40 in favor of environment. 9. For example, to get 50% of communities to a point of: having a workable community informationlcommunication system; having a regularly functioning women's group; having an agreed process for identification of the disadvantaged; having a certain level of skills in financial management; having a rangeland management system operating, or at least being tested; having an agreed set of community development targets and a project support exit strategy; having a women's savings and loan group; collecting maintenance fees to a certain minimum level, etc..  7 Box 1: Did Appraisal Correctly Diagnose the Degradation Problem? The preparation reports did, largely, identify the key issues and parameters, but these was not translated very clearly at appraisal into the project strategy. With some benefit from hindsight, the elements of the strategic argument appear to be the following. While data is limited, soil loss and flooding is due to three broad causes: (i) steep and still rising elevations (mountain building) contributing to some unavoidable geological erosion and flooding (which might be temporarily slowed by treatments but which cannot be prevented); (ii) loss of vegetation due to excessive grazing or fuelwood cutting many hundreds, if not thousands, of years ago, amenable to some treatment depending on slopes but leaving now severely eroded hillsides with very low potential; (iii) loss of vegetation cover due to more recent pressures, more amenable to treatment depending on migration takes place, as alternative fuels become available, and as real wages rise and herding is abandoned . location. Overlaid on these scenarios is thefalling human and animal pressures in most areas as urban In many of the forestry models, by maturity (many 60 to 100years), much of these areas will be recreational. Overlaid again on this scenario should be the technical evidence relating the impact of alternative treatments across a wide cost range on soil loss and hydrology. The grazing relationship is a critical aspect of this. The relationship between livestock pressure and erosion and water in such dryland areas is usually extremely complex. Unfortunately neither preparation nor the project was able to generate sufficient data. Just how degraded these areas really are was not clear at the outset and is not clear now, nor even whether closure is best for erosion. While the decline in livestock pressure may result over time in vegetation cover improvements, it is also possible that, at least for an interim period, lack of livestock pressure may be a cause of either slower recovery or further deterioration. (This would be due to lack of sometimes beneficial soil disturbance by animal hooves and due to accumulation of smothering uneaten dry material.) While touched on in disparate ways in the preparation reports, the appraisal report missed an opportunity to fully collate and analyze these issues in the light of the very long timeframes of the past degradation and the future forestry models and to assemble them into a convincing treatment strategy matrix. This was done somewhat better at the micro-catchment level but project level strategic coherence could have been much better articulated. EFFICACY 13. Overall, efficacy - the extent to which the project objectives were achieved taking into account their relative importance -is rated substantial, but with three Targets were more than important qualifications. First, impact data is weak, achieved on forestland, although one would not expect to see ultimate basin not quite fully achieved on arable land, and impacts quickly. The main objective of the project was: "to barely touched on restore sustainable range, forest, and fanning activities in the upper watersheds of the three Provinces ....." yet data rangeland. But significant incremental to assess the actual erosion and sedimentation impacts of environmental impact the project in the Euphrates watershed, either at the lower in the basin on any regional or even local level, are not available. One would scale with respect to such not expect to measure attributable impacts far down in a impacts as sedimentation basin over a project timeframe but one might measure will not be measurable for more localized changes. At the community level, there was many years. some anecdotal evidence of impact on sedimentation of small dams close to the treatment areas and of reduced village flooding in large rainfall events. But for these impacts to be felt in the major reservoirs lower in the system it can be expected to take between ten and a hundred years or more given the typical residence time for the movement of particles down basins. Second, while tree planting covered a  8 substantial area (over 70,000 ha), improved management of range areas barely started partly because of unclear land ownership. Third,the attributable project incremental impact on the environment is difficult to assess since livestock pressures are falling in most areas even without the project and since a significant percentage of erosion is probably geological in nature. The number of micro-catchments treated was 87 against the appraisal target of 54. The table below indicates the main physical achievements. Efficacy, in this case, has been assessed against the background of the very tradtional initial social situation and the lack of any previous experience with participation. Relatively, substantial strides were made in a new direction for Turkey. Area Achievements Against Planned Indicators output Projected SAR Actual (ha) (ha) " -"-I Forest land (MOF) 62,370 95,316 Rangeland (MARA) 116,312 2,382 Arable Land (MARA) 38,167 29,810 Agriculture Supporting Activities 22,334 19,558 (mostly horticulture and irrigation) (MARA) Irrigation Supporting Activities plus rainfed terraces (KHGM) 10,530 12,368 How Effective Were Community Processes? 14. The project was a valuable first learning experience in community participation. However, it was not a Community Driven Development (CDD) project as is understood today. It was a community-based participatory intervention representing a first step on ' the CDD ladder. Moreover, there were limited sustained changes in community decision- making processes. However, there was at least a temporary grafting on of additional processes and substantial skill transfer in participatory processes and a great deal was learned in a quite conservative social environment. Project and borrower staff claim that community members have learned to be more demanding of performance from public agencies and to be given more say in project selection.  9 Box 2: How Effective and Efficient was the Project in Reaching the Poor? The main aim of the project was avowedly environmental, even more so in implementation. It was apparently made clear at community meetings that there would be gainers and losers but the aim was to offer alternative income sources where possible. Moreover nearly all were expected to benefit in some way from the environmental gains. There is little doubt from field visits that there have been some short-term losers, in particular landless livestock owners. However, generally it appeared that most muktars had attempted to allocate compensatory benefits, especially those not requiring land (e.g.,apiculture), to the disadvantaged. However, some benefit packages such as apiculture presented problems being demanding of management. Overall, the poverty impact evidence is somewhat ambiguous. The majority of livestock are owned by the wealthier households, although they may employ the poor to herd. In fact, in some communities the larger livestock owners tried to stop the project to avoid closing grazing areas. However, there are also many poor who own smaller numbers of livestock but for whom these are a bigger share of income. With few other assets these owners have almost certainly lost. Some were reported to have left the village. However, the Impact Study (1998 baseline and 2001 follow-up survey), suggests some progressive impact -farm incomes at the bottom end of the range rose more than at the top end or fell less. But with many of the project interventions technically linked to land or water source ownership there were technical reasons for some unavoidable bias towards the less poor. Loss of grazing has been significant although closure has never covered all village land. In theforestland component, grazing closure has been from 5 to 7 years and in many cases permanent. Thinnings forfuelwood are mostly available by year 10.The agreed benefit sharing of 20 percent of timber output to the community at 40 percent of the market price -about 8 percent of total value -would be received in not less than about 60 years, so far into the future as to be irrelevant. (Such areas will be largely recreational by then.) In the rangeland components, closure was for five years with cut-and-carry or grazing after that depending on circumstances. After the initial willingness to agree to grazing closure in return for the immediate high-value compensatory benefits such as irrigation, forage crops, apiculture, horticulture treatments etc., and with those benefits now no longer holding leverage, incentives to open up closed areas for grazing are increasing in a number of villages and can be expected to escalate further in the future.  10 15. There were strengths and weaknesses in the process changes. Strengths included community meetings with government technical staff and community leaders and the discussion of investment options and prioritization within a problem-solving protocol. Technical choices - in the past often made almost entirely by government -were discussed openly with the community; individual households had a say in the selection of investment or training options. There was generally increased information for those who wished to seek it out, but many did not or could not." Weaknesses included the fact that, based on mission discussions, there appear to have been many people who did not attend the opening meetings and many, probably most, women seem to have known little about the project. In the process of collective decision-making, if the majority agreed to an action it was considered by the project accepted by the community, raising some questions about the extent to which the disadvantaged were truly heard. There was no significant movement towards shifting forest management responsibility to communities, although there is the intention to do this. Finally, the traditional structure and processes with the muktar - as both an elected people's representative and a paid government representative - remains essentially unchanged with some evident conflicts of interest. How Effective Was the Traditional Catchment Approach Overlaid on the Community Approach? 16. The project attempted to blend a technical watershed approach, in which investments were substantially determined by location of land and water resources, with a bottom-up participatory approach in which communities and individuals were being asked to make choices. The dilemma presented by this was raised in a letter from FA0 during project preparation. It was resolved on the ground to some extent through the problem-solving procedure itself but, as in other natural resource management projects, inevitably with tensions. The appropriate stance to take in such projects on this dilemma is by no means obvious. Development professionals with social skills tend to put people's wants paramount. Development professionals with technical skills tend to put soil and water exigencies paramount. In the end, the technical realities that water does not flow uphill and plants will not grow without water dictates much of what can be done in agriculture. Yet these realities should not preclude a starting point of people's needs and a focus on incentives to reward the achievement of objectives. In this project, there was more of a technical starting point than most -partly arising from the limited soil and water related investment options determined prior to the community prioritization exercises. In this case, since water sources were invariably springs, those who were able to benefit from associated irrigation structures had to be those who had access to the springs. With a limited community budget, such investments had opportunity costs. Some, but by no means all of the potential losers, were accommodated with other investments. There was also the often encountered dilemma that catchments do not coincide with administrative or community boundaries resulting in vehement complaints in at least one community visited that one hamlet over the ridge had been left out. 10.For example, of 11 in a focus group discussion in Elmali, Malatya, none had attended the early planning meetings. In another village 2 of 12present had attended.  11 How Effectively Were Skills Transferred? 17. Quite well. Government staff and community skills developed under the project in the FCPCPS technique, and the marrying of that with technical project design, should remain valuable for later agricultural extension work and, more broadly, for community work with other agencies, but only if such newly found skills are continually exercised. Also, for example, under the project selected farmers were taken to see successful micro- catchments to transfer knowledge and experience. Project Readiness 18. In any project which takes 8 years to closing and still only disburses 75 percent- and most of that in the last four years- there must be questions about readiness and whether the timeframe was realistic. While the QAG review praised Quality at Entry, the very slow start is a negative factor in rating efficacy with also a negative impact on efficiency. However, the rapid catching up done in later years is a counterbalancing factor. Moreover, OED has generally found that community driven development projects should expect, and plan for, a slow start up as participatory processes are established and community confidences gained. In many respects it is not an uncertain process. Slow build up can be predicted. Efficacy of the Community Approach in Forestry 19. It is unusual for an environment-focused Bank-funded project to have no handover of forest management responsibility to communities. The project could probably have done more in this direction. The FD argues that the Constitution does not allow this. But they have begun to experiment. The problem for increased community forestry involvement in this very dry environment is that harvestable forest benefits are mostly very far in the future, in some cases over a hundred years. However, there are also some shorter term benefits and there are surely enough sites where more community management at least could have been experimented with. Turkey lags far behind most other countries on this issue. There are questions about the extent to which the forestry plantations, the largest single component, were really in tune with community wishes. These were on Forest Department (FD) land but, following the project strategy, now placed there with some agreement with the community. Prior to the project such works were undertaken compulsorily, resulting in unhappy relationships. Selected communities generally went along with the plantations but it would be surprising if they did not given the incentive of immediate high value compensatory agriculture benefits. Efficacy of Technical Work 20. Generally technical work during preparation, appraisal and implementation was sound. There were some technical weaknesses during preparation and appraisal, partly due to consultant recommendations, for example the recommendation to test aerial seeding in range situations. Aerial seeding has rarely been successful or economic in dry zones and, in any case, aerial seeding experiments can be replicated usually at much lower costs by hand broadcasting using low cost labor (which also allows guidance  12 through training on the seeding of promising localized niches, e.g.,depressions, gully heads to carry seed set downhill, etc.). However, in this case, both government and the Bank were quick to recognize the weakness here and abandon aerial seeding. Efficacy of the GEF Component 21. With the associated GEF Grant covering the In Situ Conservation of Genetic Resources Diversity Project, there was support for the in situ conservation of genetic resources, an important issue since the area is important for a number of forage legumes, wild wheats, Lilliaceae and Compositae. This project performed well. It identified a number of Gene Management Zones where wild relatives of wheat, legumes, and forest species could be conserved. It surveyed for species and did initial stakeholder workshops and social studies in 22 villages. It developed a National Plan for In Situ Conservation.. There was, again, good cooperation between MARA and MOF. Public awareness material was developed. A GIS center was established. The National Plan gives hope for sustainability. However, sustainability is by no means yet assured. Commitment is strong but funding limitations are a concern since such under budget stress such programs tend to be seen as luxuries, notwithstanding the generally proven high value of diverse genetic material. The Efficacy of Monitoring and Evaluation 22. As is so often the case, monitoring and evaluation was weak. First, the baseline survey was not done until 1998, only 3 years before the project closed. Second, for a substantially environment-focused project there has not been enough measurement of soil and water impact of different treatments." Now 10years later, there are far too many land treatment impact unknowns. Valuable time has been lost. The State Planning Organization, in their comments (Annex C) notes that the responsibilities of all parties concerning follow-up and evaluation could have been more clearly defined and a sustainable structure established. EFFICIENCY 23. Efficiency is rated substantial on balance - the ERR is likely to be somewhat above 10percent. The aggregate masks a substantial share of investments with an ERR well below 10percent counterbalanced by quite high returns from high value irrigated horticulture, vegetables, and forage12.There are two problems in assessing efficiency. First, analysis is difficult in such a project with a substantial environmental focus where many environmental impacts are expected to become evident over a very long period. Second, there are methodological questions related to the extent to which an aggregate ERR should be permitted to mask wide ranges in component ERRSi.e economic cross- 11.For example, on different slopes and soils the comparison of costs and impacts of full afforestation, simple closure, deep ripping, a range of partial lower cost treatments, alternative seeding approaches, different grazing regimes, etc. 12. The new forage introductions show impressive performance and represent a significant technology gain.  13 subsidies. Detailed re-analysis of the economic rate of return is not possible given the resources available and the complexity of environmental benefits,. However, some conclusions can be drawn from some sensitivity adjustments and from the economic analysis for the proposed follow-on project. The Economic Analysis 24. The ICR found an ERR of 16%.The analytical methodology appears sound although the horticulture and irrigation benefits appear somewhat ~ptimistic'~and the soil conservation and afforestation benefit stream appears quite optimistic in the light of the level of degradation and the apparent intent to not permit harvesting on substantial areas of "protection forestry". There are also doubts about sustainability as pressures from villagers buildI4. But given the dominance of the substantial horticulture and irrigation benefit streams, even with a 50 percent reduction in forestry benefits sensitivity analysis done for this PPAR shows that the aggregate ERR does not fall below 15percent. However, at a 50 percent reduction in forestry benefits the ERR of the forestry component itself drops to 3 percent. Thus, while in aggregate, the project appears likely to give an ERR of significantly over 10percent, within that there is a substantial forestry component that appears uneconomic unless uncounted flood and sedimentation benefits reach quite high levels fairly quickly Although it is in different areas and of a somewhat , different scale, the economic analysis for the proposed new project suggests that: (i) soil erosion reduction with positive productivity impacts within the immediate selected catchments could possibly reach a scale that would significantlyimprove the afforestation ERR; (ii) annual value of reduced flood losses is unlikely to reach a scale that would significantly improve the ERR: (iii) sedimentation benefits lower down in the basin would be much too far in the future to have any significant impact on the ERR even if the small percentage of land area treated were to eventually have a measurable impact. With respect to Trees were expensive "labels". allocative efficiency, there is little doubt that other Agriculture gave high ERRs, combinations of investment could have considerably forestry low. Some "cross- raised the ERR, for example more investment on the subsidy" may be justified, but, quicker return forage crops on agricultural land and less with high costs of forestry and declining livestock numbers, on high cost afforestation on steep very dry stony greater attention to lower cost hi1l~ides.l~However, the objective of the project was not options using community simplyproductivity. management enabling wider coverage could probably have increased total impact. 13.For example, in the significant small-scale irrigation model, while the projected yields themselves are not unachievable in absolute terms, there is no assumption of rising yields in the "without project" situation. In fact, there is no assumption of rising "without project" yields anywhere in the analysis. The State Planning Organization notes correctly (Annex C) that the optimistic profitability ratios in the ERR analysis did not have sufficient explanation. 14.The forestry assumption of a 5 cubic meter average annual wood increment in the ICR analysis by Year 30 appears very optimistic for such degraded steep hillsides even if it were to be harvested -which seems not to be the intention with substantial areas. The SAR, also quite optimistic, assumed 3.7 cubic meters by Year 45. 15.Maturity and harvest are in some cases over 100years, making adequate ERRs very unlikely without huge supplementary environmental benefits.  14 25. Without large and very quick environmentalbenefits, forestry treatment costs averaging about US$400 per hectare, and as high as of US$750 per hectare, are almost impossible to recoup in such dry areas. The need for high cost tree planting treatments can be attributed partly to the fact that trees have become a form of "signpost" telling herders to keep animals out. This is a high cost way of signposting land use unless there are also significant economic benefits. 26. This was also a relatively high cost project in terms of total costs per household. At an actual Total Project Cost per household of $1,958 over 7 years (US$78.33 million divided by 40,000households) it is well above, for example, the IFAD average of about $420 over about an average 6 year project period and also above high cost NGO programs such as the Aga Khan Rural Support Program in Pakistan. (see Table C1 in OED 2002 AKRSP Evaluation for some comparators) 27. The ICR analysis shows the quicker and higher return agriculture and horticulture components to have an ERR range of about 18percent to 30 percent and the much slower and inevitably lower return forest land treatments to have an ERR around 10percent. In both the SAR and ICR the economic rates of return for the afforestation appear quite optimistic against the predominant current view by the Forestry Department that much of the planting must remain unharvested as "protection forestry" in order to achieve the environmental benefits projected. However, on balance, allowing for uncounted benefits in the ICR that are, at least anecdotally, beginning to become evident at the micro- catchment level, such as flood damage reduction (reported in two or three communities visited), small dam sedimentation reduction (reported in one community visited), and biodiversity benefits, and accepting some element of aggregation of higher return and lower return components (i.e. economic cross-subsidy), it is concluded that the aggregate economic rate of return may still be marginally above the opportunity cost of capital, although probably well below the 17percent claimed.16'" 28. With respect to allocative efficiency across the different components, there are clearly uncertainties about the economics of the longer-term forestry investments The methodology estimating unless considerably greater utilization levels can be income increases is incorrect. expected than are contemplated now, or unless Real incomes actually fell rather environmental benefits are unusually rapid. It is likely that than rose. However, there are no community pressures for such earlier forest land comparators and this is probably utilization will build (and, if acceded to, probably with more an economic downturn impact. Agriculture investments only modest impact on erosion rates). More broadly, were observed to be quite within the responsible ministries, the project failed to profitable at field level. 16. It is not clear from the economic analysis that all public institution overhead costs have been included. The project cost stream does not appear to include all the substantial costs of government staff involved. Only those deemed within the project costs tables as incremental where included. This issue of overhead costs was also a concern of the State Planning Office. 17. While there is limited selling of land in these areas, land price increases over recent years reported in project villages for both dryland and irrigated land are not inconsistent with a satisfactory ERR in terms of capitalized land values. But there is insufficient comparative data on "with" and "without project" villages to be conclusive.  15 achieve improved institutional efficiency through reduction in the generally acknowledged over-staffing. '* 29. Income Increases. A more than doubling of incomes over three years from the baseline survey in 1998to the follow-on survey in 2001 was found by the Impact Study. However, there is a problem with the use of the same US$/Turkish Lira exchange rate for both years. Average annual On-Farm Income in the base year was TL430 million and three years later was TL1017 million. At official exchange rates" this would actually give not a doubling of incomes but a fall from US$1,646 to US$828 per household per annum. However, this probably also does not adequately reflect real attributable project impact for four reasons. First, the 100percent loss in value in one year makes assessment of the real purchasing power impact at farm gate level difficult. Second, field discussions suggest that many of those households who did receive project support have gained real income from those particular investments without obvious losses elsewhere within the farm system, e.g., due to diverted labor etc. Third, there is anecdotal evidence of reduced out-migration but whether this is a push or pull factor between sectors is unclear. Fourth, there are the usual counterfactual problems - what would have happened without the project? Regardless of average income data, there were still many households who did not receive direct project benefits and some who have lost grazing from forest or range closure. Aggregate estimates suggest that on average about 25 percent of households benefited directly. Sample estimates at community-level during field visits for the percentage who did not benefit ranged from about 40 percent to 60 percent. For example, in Yaygim village in Malatya the attending group estimated that about 60 percent had benefited and about 40 percent had lost in some way. With a transitioning economy and sector increasing attention will be needed in future to off-farm income opportunities. Cost Recovery 30. For some of the private investments cost recovery was relatively low - for example, in irrigation with generally high returns, it was only 3 percent to 5 percent. Considering the relative asset wealth in the community of those with irrigable land and exploitable water resources in the form of springs, and considering that project resources were unable to assist often as many as half the households, such low contributions raise efficiency questions related to resource allocation and equity. Larger contributions, at least by identified less poor households, would have allowed greater spread of benefits. INSTITUTIONALDEVELOPMENT 31. Institutional development is rated, on balance, as substantial, but it was variable across different elements and there are questions related to sustainability. Bringing together the three main agencies involved was an impressive feat that was widely acknowledged at local level, at provincial and central level, and by NGOs. It has set an example that hopefully will be followed in other interventions in future, even outside the sector. It appears to have been due to strong ownership at the start, particularly good 18.The Ministry of Forestry has about 35,000 staff and Ministry of Agriculture 80,000. 19. 1998:US$l= TL261,604; 2001: US$l = TL1,228,367  16 project leadership, and close supervision by the Bank, particularly from the local office. Also, as often is the case, the timing was right. The agencies involved were ready and receptive for a new more participatory approach. Good training was provided and the Bank brought skills in this area both directly through Bank missions and indirectly through consultants. But with the project now closed, and no follow-on project in the program to date, there are now questions about whether this government organizational cooperation can be sustained. 32. The extent to which institutional processes really changed at community level is less clear. The project used the existing system with the village muktar as the leader. While elected by the village and supported by elected (unpaid) elders, the muktar is paid a salary as a government servant. There are therefore inevitable loyalty tensions. He (it is invariably a man, The project could have although there have been a few women muktars) is very "challenged the system'' much a local politician. Many vote for him because he more in the areas of seems the most likely to pull in public funds. Indeed, in womens' and poorer several community meetings with the mission it was clear households' that government funding support, whether it be through a involvement. Bank project or from other sources, was seen by rural households as a right and the muktar was expected to deliver on such entitled central support. The project itself, however, had a relatively narrow menu of investment options linked mainly to improved natural resource management. Moreover, management of contractors remained in the hands of the technical departments and focus group meetings found some community concerns about quality with limited redress. 33. Under the project, using this existing administrative system and simply adding the community planning processes and consultation to it was clearly the easiest route to take. Was it the most effective? Totally changing a long established administrative system would have been unrealistic for a single rural project. However, this assessment concludes that greater challenging ofthat system in the areas of women's involvement including the development of informal women leaders, in targeting, and in participation and in special identification and consultation with the poorer households - something done in many other community-based projects - would have helped the growth and equity elements of the objective and in due course enhanced sustainability''. While the project FCPCPS communityproblem solving procedures appear to have addressed "elite capture" to some extent, there were still clearly a number of "losers". 20. The Region in the Bank argues that in assessing the extent to which established power structures were sufficiently challenged one needs to remember that the project was designed after the first Iraq war and when Kurdish conflict was at its height and there were virtually no NGOs in that part of Turkey. Challenging the established structure would have been unrealistic. OED accepts this to some extent but notes that the report is suggesting not a fundamental overturning but significant adjustments in processes towards accommodating the needs of women and the poor. The Region also argues that targeting the poorest within already selected poor areas might have been an instance of the best being the enemy of the good given the often-faced dilemma of growth versus poverty. While acknowledging such trade-offs, OED would argue that there were still some untaken opportunities in this direction.  17 Did Women or the Poor Get Much Say in Prioritizing Investments? 34. The focus group findings (see selected extracts in Annex B) suggested that women and the poor had only a limited voice in the prioritizing of investments, particularly women. In particular, the project was unable to significantly increase the role of women in community leadership, although greater consultation with women was achieved. However, allowance must be made for the enormous social constraints to greater wornens' involvement in this very traditional Islamic society. Men at meetings noted that their women had not attended planning meetings but that, "they always discussed these things at home." There were meetings for women during the planning process but only a few women interviewed in focus groups had attended these. The poorer households had the opportunity to participate and received some benefits but, as noted earlier, lack of land limited their investment options. It appeared that substantial numbers of people did not attend the initial planning meetings. (For example, in Elmali village in Malatya none of the 11men present at the focus group meeting recalled attending the initial project planning meetings.) Reasons range from not knowing about them, to not being interested, to not being in the village at the time. The semi-nomadic pastoralists almost certainly had no say", but in most communities there are relatively few. NGO Involvement 35. There was an intention to involve NGOs but this did not eventuate. Earlier NGO involvement during preparation would probably have helped. Negotiations were held between government and one NGO but could not be concluded to the satisfaction of both parties. There appear to have been some concerns from the NGO side about the extent of freedom they would have and about the scale of resources. The limited number of NGOs with rural natural resource management skills in Turkey makes a significant NGO role in such a project difficult and not one that can be created artificially. The intent under the proposed follow-on project was to increase NGO involvement to the extent possible. BANKPERFORMANCE 36. Bank performance was, on balance, satisfactory, but there were some areas of weakness in strategy and policy and the Bank could have pushed further, even at mid- term, towards challenging the community decision processes in the direction of identifying and targeting the poor and involving women. Legal rangeland ownership issues and associated data were clearly not adequately understood at appraisal. The project started with a target of 116,000ha, the largest of all the land sub-component targets, and ended with a little over 2000 ha, the smallest of all the achievements. However, with respect to knowledge transfer, at central, provincial, and community level, there was quite widespread acknowledgment that the Bank had brought new ideas related to community participation processes, although one or two project antecedents were also 21. They come and go with seasonal change and often are renting pasture through a local intermediary who, being local, has a better chance of renting the grazing rights from either government or communities, and who then sells on this right -at a substantial premium.  18 noted. There was a slow and fmstrating start for the first three years partly related to over-optimism in planning but both Quality at Entry and Quality of Supervision was assessed as Best Practice by a Quality Assurance Group (QAG) review in 19982223. 37. The pattern of many of the initial strengths and weaknesses continued through the project, supporting the frequent OED finding that many project problems emanate from the design stage and are difficult to correct later. As noted, notwithstanding some strong preparation and appraisal work, appraisal did not quite pull it all together into a coherent strategy (see Box 1). 38. In supervision, Bank performance was generally strong - after a slow start. Project Supervision reports were valuable.24The 1995Mid-Term Review pointed to some new directions. However, M&E remained weak which was clearly a supervision responsibility. The decision to extend to new areas so late in the project could be challenged since it has left some community investments uncompleted, but, on balance, this decision to use savings substantially created by exchange rate changes to spread the experience more widely was probably the right one, although it somewhat loses its appeal in hindsight in the absence of a follow-on project. Did the Bank do Enough on the Policy Front? 39. Probably not, although policy changes with national implications may often not be achievable in association with a project for a limited number of provinces. There were three policies with significant implications for project achievement. First, the constitutional and legislative constraints to community forest management. There was insufficient focus on creating the incentives for community forest management, although FD is now addressing this issue more directly. Second, the direct farm subsidy of about US$50 per hectare regardless of land quality (which encourages the cultivation of otherwise unprofitable marginal lands). The Bank should probably have made more effort to at least initiate analytical work and dialogue on both these issues25.The third important 22. The following strengths were noted in that review: the strong community participation in a country previously highly centralized; the coordination among ministries; the increased focus in participating ministry services; the contribution to national policy on rangelands; the combination of good external expertise and good local knowledge during preparation; the project flexibility; the teamwork, persistence, and quality of Bank input; and the "learning by doing" nature of the project. Areas needing improvement included: the lack of quantitative and qualitative project impact data and lack of serious monitoring and evaluation as a management tool; the need for improvement in some of the technologies; the need for improved staff incentives; the need for a greater role by the Ministry of Agriculture in agriculture research; the potential for decentralization of funding allocation; and the need for clarification of maintenance responsibilities. 23. The State Planning Organization points out correctly (see Annex C) that initial cost estimates were unrealistically high. 24. For example, frequently missions drew the borrower's attention to the lack of research on rangeland treatment and sedimentation rates, an issue raised by this report and, for example, in 1998, a supervision mission raised the important issue of nomadic livestock producers in the development of micro-catchment plans in the new Toros Mountain areas. 25. The Bank Region argues with respect to feasibility of upfront legislative changes that project implementation took place at a time when there was difficulty with moving ahead with the broader  19 policy area was the Rangeland Law enabling greater community management of rangelands. This was passed during the project after, apparently, a 37 year delay, but, by 1998,this was really too late for much project impact given the uncertainties of range ownership found during early project implementation26. BORROWER PERFORMANCE 40. Borrower performance is rated satisfactory but there were some weaknesses. The borrower was committed to the project and provided very strong leadership in achieving the inter-departmental coordination between the Department of Reforestation and Erosion Control (AGM) within MOF, the General Directorate of Production and Development (TUGEM), the General Directorate for Agricultural Research (TAGEM), and the General Directorate of Rural Services (KHGM) within the Ministry of Agriculture and Rural Affairs (MARA) . Technical skills and support at field level was observed to be generally strong, albeit with weaknesses in focus on treatment impact. However, the project did get off to a very slow start with implementation mainly because of lack of readiness and over-optimism about the establishment of communityprocesses and the development of staff skills. But it is rare to find a CDD project that is not over-optimistic in these areas. Financial procedures and budget constraints reduced expenditure capacity. KHGM performance was particularly impacted by budget constraints. SUSTAIN ABILITY 41. On balance, sustainability is rated non-evaluable, based on a weighting of several elements. The objectives of the project as stated were, essentially, ". .. to restore sustainable (land There are large uncertainties management) in the upper watersheds". There are about both Physical and a number of concerns about sustainabilitybut also, sustainability*The given the very long-term impact of these objectives scale of future pressure on forest and the lack of early impact data, unusually large land is uncertain. It is not yet uncertainties. clear that either improved community processes or 42. The sustainability elements of the project government processes are really can be divided into: sustainabilityof watershed consolidated enough to be investments; sustainability of government sustained in future government support without a follow-on processes; and sustainability of community project in those provinces and processes. With respect to investments, field communities. observation suggests that most agricultural investments on private land, representing 39% of the actual total treatment area, are likely to be sustained because of private profitability. agricultural policy reform agenda. Therefore the strategy was to focus more narrowly on technical, poverty reduction and pilot interventions to maintain a foot in the door. OED appreciates the difficulty here but notes that what is proposed is more the initiation of analytical work and dialogue than conditionality or hasty completion. 26. While such issues can be complex, the extent to which the rangeland ownership issue was misjudged by both Bank and borrower at appraisal suggests insufficient consultation or excessive optimism.  20 However, forestry investments on Forest Department land face a threat. As noted earlier, it has been almost inevitable that, when offered the immediate high value benefits on agriculture land e.g. irrigation, forage, terracing, etc., households would accept these against a quidpro quo of closing lower value selected forest land to grazing. However, even for those who did receive the agriculture benefits, having received them, and with the memory of that benefit fading, there remains an incentive now to utilize the forest land for grazing. Whether the community commitments on this can be sustained for the unusually long maturity periods of these planting is questionable. However, this issue is complicated by two facts. First, grazing pressure is declining nationally as wage increases, employment alternatives, and competition from more intensive livestock production make shepherding, either by paid employees or family members, less attractive. Second, and related to the first, heavy pressure is more likely on the nearby grazing areas due to the convenience of proximity. It is therefore anticipated that some percentage, at a guess say one third, may face damage sufficient to significantly reduce impact. Thus, about 20% of total actual project treatment area might be 43. With respect to governmentprocesses, which were one of the means to the physical and welfare objectives, while there was a substantial gain in community interaction skills which should be of benefit to government staff for some years, the lack of a continued vehicle to utilize those skills, and the project specificity of the improved interactive processes adopted under the project, raises questions about the sustainability of the incipient skills of public servants. However, the Forest Department at least seems to be attempting to maintain the consultative processes tested under the project. Whether this can be sustained given the higher cost of such approaches, the now much tighter budget and the low current morale with recent institutional changes is very uncertain. 44. With respect to community groups which were another means to the physical and welfare ends, it has been argued above that the traditional processes were not challenged by the project enough with respect to women and the poorer households. The project utilized the traditional structure and the traditional representational system obviously will be sustained as it has been in the past. However, since no further project investments are anticipated in these micro-catchments it is somewhat less clear that the introduced participatory processes themselves initiated under the project will remain active, or at a sufficient state of readiness to be re-activated for any set of community investments. Moreover it is very uncertain that these only recently introduced processes will become the established processes for normal government services. What is needed is for projects to stay with the community processes long enough to reach an agreed set of benchmarks with respect to decision processes, money management, and processes to help the disadvantaged, and to agree at the outset a phased exit strategy for each community with trigger points in those areas. What one would hope to see if improved community processes have really been put in place is that a community would have shifted from being a largely passive recipient of proffered public resources to a pro-active seeker of resources from all sources for largely independently developed and community-planned ideas. Such a permanent and sustained shift which would signal true sustainability still 27. The data is insufficient to show whether, even if vegetation is substantially lost, the physical treatments such as terracing and deep ripping would, in any case, achieve significant soil and water benefits at least for some years.  21 seems some way off. But it is noteworthy that in the focus group and key informant discussions there seemed to be little change evident in community infrastructure and services except for continued quite strong school operation support and perhaps bus services, and nothing attributable to the new consultative processes outside the immediate project investments except perhaps in the case of the few Associations formed. The latter seems to be a community threshold step to be aimed at for sustainability. GEF Project 45. The mission did not have the specialist resources to focus sufficiently on this parallel GEF project. It appears to have been generally well implemented. However, there are still concerns about the future sustainability of this In situ Conservation of Genetic Diversity Project mainly in terms of future budgetary commitment. This is notwithstanding the probably high value for plant breeding of conserving in situ a wide range of unique genetic material and the escalating returns as one approaches the point of species extinction. Beneficiaries would be both Turkey itself and the broader global development community. Increased community management of communal forest and range land resources would complement this activity.   23 Annex A Annex A. Basic Data Sheet TURKEYEASTERNANATOLIAWATERSHED REHABILITATION PROJECT Key Project Data (Amounts in US$ million) Appraisal Actual or Actual as percent of Estimate current estimate Appraisal estimate Total project costs 109.79 78.33 71 Loan amount 76.9 48.0 62 Project Dates Original Actual ~ ~ - ~ x - ~ - l _ l _ _ _ _ ~ - - " Initiating memorandum 0311990 0311990 Board Approval 0311993 0311993 Effectiveness 0711993 0711993 Closing date 1012000 1012001 Staff Inputs (staff weeks) Actual Weeks Actual US$OOO Preappraisal Na na AppraisalINegotiations Na 243.3 Supervision* Na 581.O Completion** Na 36.0 Total 860.3  24 Annex A Mission Data Status objectives Identification/Preparation 03-04/1991 6 E, A, RN, WM, EC, F identification/Preparation 11/1991 5 E,A,NA, NA, NA Appraisal/Negotiation 01/02/1992 6 E, A, NA, NA, NA, NA AppraisaVNegotiation 06-07/1992 7 E, A Appraisal/Negotiation 11/1992 6 E, A AppraisaVNegotiation 01/25-28/1993 Supervision 1 05/1993 3 E, A, F S 5 Supervision 2 11-12/1993 3 E, A, IE S 5 Supervision 3 05-06/1994 2 IE, A U 5 Supervision 4 10/11/1994 4 A, IE, E, BS U 5 Supervision 5 05/1995 4 S 5 Supervision 6 10-11/1995(mtr) 4 F, A, IE, BS A, E,F, BS S 5 Supervision 7 05/1996 4 TM, A, BS, OP S 5 Supervision 8 10/1996 4 TM, A, IE, OP S 5 Supervision 9 05-06/1997 5 A, TM, IE, F, OP S 5 Supervision 10 07/1997 2 A, IE, F,OP, E A, OP S 5 Supervision 11 11-12/1997 5 S 5 Supervision 12 04P998 4 S 5 Supervision 13 07/1998 2 A, IE, F,OP A, OP S 5 Supervision 14 1111998 3 S 5 Supervision 15 031999 5 A, OP, IE, F,L A, IE, F S 5 Supervision 16 10/1999 4 A, OP, IE, F S 5 Supervision 17 05/2000 S 5 Supervision 18 11/2000 S 5 Supervision 19 04/2001 2 A, IE S 5 Supervision 20 09/2001 5 A, IE, IE, F, OP S 5 Completion 12/2001 2 E, NR S S  25 Annex B Annex B. Summarized Extracts from Village Focus Group Discussions2* "Women do not take part in decision-making mechanisms nor are there any women leaders in the village." (Comments of this type were almost universal, although the understanding of what is a "decision-making mechanism" may have varied somewhat.) "30 officials came in 1998to introduce the project to the village. The village government called all villagers together at the school and 70 males of 19 households participated in the first meeting. Only three or five people intentionally did not participate because they did not want the project implemented in our village. These people were owners of large livestock herds who were using the rangeland and did not want the rangeland closed for rehabilitation. The villagers wrote down their demands and these demands were then graded according to the majority's choice. The following year some measurements were taken from water, soil, etc. and the project started to be implemented." (This was a fairly typical description of the process.) ". .. but the conditions of all villagers do not change .... There have not been any changes for us." "Our village will benefit today and also in the future from the project activities." .". all of the villagers received something from the project and it was of great benefit to villagers." "The only project activity that was bothering some of the villagers was the land closure for rangeland rehabilitation..... There are not any costs to the poor households, on the contrary they all benefited from the project.." "All the villagers except women participate in decision-making." "Most of the villagers were satisfied with the project." "One meeting was held with women before the project implementation." "After the project, we lost the rangeland and we cannot graze our livestock because of closure." "People wishing to get hives gave their names to the village governor and the ones who received hives were chosen by drawing lots." (done in other villages too.) "The project has been of greater benefit for those who have their own land." ". ,.Women do not take part in decision-making mechanisms.." "But in time (villagers) became interested and accepted the project because they saw that this project was not like the others." "I believe that since we owned the project the activities have increased in our village." "Purchasing power of the people is weakening..." 28. An attempt has been made to make these both informative and representative. In some cases there were contradictory statements from the same group. In some cases these are direct quotes from the focus group or individual informants interviewed separately or from within the group, in other cases they are the interviewers summarized interpretation of what was being said.  26 Annex B 0 "They brought apiculture which the villagers were unfamiliar with and we could not cope with it." 0 "Four meetings were held for the selection and implementation of the project in the village." 0 "All of the villagers benefit from the project, including the 10poorer families." 0 "Eighty percent of the youth goes to Kahramanmaras to work or migrate." 0 "The project has been a source of income for many villagers during the implementation process. It has also helped the co-operative to strengthen. Flooding and erosion has stopped over a period of five years. Nearly the whole land has been closed and livestock farming has stopped." 0 "The villagers are generally satisfied with the project." 0 "No contribution from women. They do not have any information about the land being afforested. .... They know that three ponds were built but they do not have any contribution to the project activities or the decision mechanism." 0 "Some women said that they had never seen any help provided to the village." 0 "Marketing conditions have improved since 1966but the income of the villagers stays constant." 0 "The only successful project activity was afforestation." 0 "The villagers did not have trust in the project at the beginning and they thought nothing could be done. In time, with the implementation, the consultation with the villagers, and the participatory approach, the villagers had a positive feeling about the project." 0 "We cannot ask Rural Services to help in the maintenance of the ponds because no one listens to us. When ponds are shared by four or five households the officials do not asked for monetary contribution for cement but if it is an individual pond the villager pays for cement." 0 "One of the hamlets of the village had no benefit from the project." 0 "Since the ponds were not built in a healthy way and less cement (was used) the walls are peeling off." 0 "Formerly they used to have large livestock flocks in the village and children used to work as shepherds. Nowadays, the young families either migrate to Istanbul or work in Malatya. Only a few households deal with livestock flocks. The rest have sold their sheep and goats and bought one or two cows.'' (This was a common response about the shift out of small grazing livestock.) 0 "About 30 percent of the planted seedlings have died" 0 "If they could get back to 1996they would not have let the project be implemented.Not all that was promised by the project was realized and those that were realized were not done in a "healthy way."" 0 "Women of the village do not have any information about the project." 0 "The first meeting for the project was held in 1998. Almost all of the households participated in the meeting and 99 percent of them accepted the project." 0"Afforestation has not been completed."  27 Annex B e "Market conditions have improved, nowadays you can find anything you want to purchase. But the economic condition of villagers weakens, although we earn much money we cannot afford (things) as we used to " "The villagers were very positive about the project." "About 80 percent of the villagers were interested in the project activities and they welcomed the officials. There was only one villager who has 40 sheep that objected to the project." " ..they had some disputes with a neighboring village about the project activities." "The villagers were told that the project would provide seedlings, but they have not received any seedlings from the project." "Ahundred beehives were given to five households who had given up livestock farming with the project .. some courses were given .... some of , (them) could not cope with bees." e "The villagers say that they got poorer compared to the past. (But) ... in three to five years time they are going to start to get yields from almonds that the project planted..... But the project affected livestock owners in the village negatively." "They were satisfied with most of the project activities." "Before the project implementation, the meeting was held with this village's women, 4 out of 20 women (in the focus group) knew about the project. .... According to women there have not been any changes in the village since 1998. " "Women do not take part in decision-making..." "They should plant trefoil or something else in the lands that were closed to grazing otherwise the land should be opened back up to grazing livestock." "In general villagers were not satisfied with the project and they were complaining about incomplete activities." e "The villagers had had positive feelings about the project because of the rotational closure to grazing (the rangeland selected was to be separated into two or three parts and these lands were to be used for range activities in turn.) But this activity was not realized." "Thirty men, representing all of the village tribes participated in the meeting. ,..There were a few villagers that did not want the project......Thesepeople were livestock owners." e"There are 10poorer households who do not have land in the village. The project did not affect these households either good or bad. Nothing has changed for these households with project implementation but they will benefit from afforestation in the long run." "Villagers are not dealing with livestock as they used to any more.... Because of the improvement in living standards of the villagers the youth prefer to migrate to the cities and the rest dislike dealing with livestock. With the project, the income of villagers has not changed but it may possibly increase in the future."  28 Annex B e "In general villagers were not satisfied with the project because not all of the activities that were proposed were implemented. ....They were complaining about the contractors and weak supervision by Rural Services." e "If the objective of the project is preventing erosion why do they not let us graze in rangeland?" 0 "There were no women leaders and no female participation in decision- making mechanisms." 0 "They (women) did not have any idea about the project. The women did not have any participation in the activities or meetings about the project and they did not have any demands." e "Market conditions improve but the income of villagers stays constant." e "Before and after the project water (drinking and irrigation) is still the most important problem of the village." (This is a common response in relation to questions about main problems.) e "(villagers) used to take decisions collectively in the past but with the developments in media such as TV or radios in homes unity and cooperation has weakened in the village." 0 "They especially mentioned that they would have left the village if the project was not implemented." e "But the previous village governor and some other villagers did not believe that the officials would be helpful and therefore few of the villagers participated (in the opening meetings). Only 20 to 30 percent of the villagers participated in the first meeting held in 1994..... None of the villagers objected but they thought that the project would not be beneficial. " e "The villagers did not take part or contribute in labor." e "People could not cope with apiculture." 0 "Irrigation conditions have improved with the project.. However the poor are still poor and nothing has changed for the villagers who do not have any land or livestock. .... 10to 15 households did not benefit from the project, they were offered beehives but did not want them." e "There were no women leaders and no participation of women in decision- making. They did not even hear about the project.. .. They had really big problems about water and their workload was too much." 0 "The project provided water to some of the households but it would be better if it had provided something that would serve all of the villagers." 0 "The project was not very active in this village since it was not the project's main target due to its settlement being on land that is suitable for irrigation which was not affected by erosion or flooding directly." e "There was one pond built but it was banned because it was on rangeland." e "Since agricultural pesticides were being used in the village the bees died." e "They mentioned that they used to take decisions collectively in the past but in time cooperation and unity have weakened." e "Participation in the meetings (opening meetings) was low...... nobody objected to the project but everybody had hesitation that it would not be implemented." e"Villagers did not work in any of the activities implemented by the project."  29 Annex B 0 "In agriculture, there has not been any change with the project...." 0 "Women do not have any information about the project... (But) they seem to know about irrigation ponds, apiculture, and seeds distributed." 0 "No meeting was held with women for the project"  30 Annex C  31 Annex C Annex C. Borrower Comments REPUBLIC OF TURKEY PRIME MINISTRY THE UNDERSECRETARIAT OF TREASURY Ref: B.02.1.HM.O.DE~.01.02.177/ J-! i: LI Ankara, Mr. Alain Barbu Manager Sector and Thematic Evaluation Group Operations Evaluation Department 27.02.04* 10037 The World Bank Dear Mr. Barbu, With reference to your message about the draft Project Performance Assesment Report of Eastern Anatolia Watershed Rehabilitation Project (Loan 3567-TR), please find enclosed the comments of Ministry of Forestry and Environment (General Directorate of Afforestration and Erosion Control) and, General Directorate of Rural Affairs. Please also take note that the comments of State Planning Organization will be submitted to you by the end of next week at the latest. Taking this opportunity, I would like to thank you and your staff for the efforts on finalization of this comprehensive report. Best regards. Seven YUCEL Acting Director General of Foreign Economic Relations Encl. Hazine Mustegarligi, Dig Ekonomik lligkiler Gene1Mudurlugu lnonu Bulvari No: 36,06510 BalgaVANKARA Tel: (0312) 212 88 55 Faks: (0312) 212 85 50  32 Annex C o It is said in this report that; there have beef?some dozrbts on developing the natural vegetations sufficiently, even IJIthere would h i e been some risks on - sustainability because of closing down the area against grazing after soil protection afforestation arid it has been claimed that the project did not produce enough&tu on the basis of mstainability. o This is the factthat, in Turkey, most ofthe land degradatione.g. deforestationhave been resulted from over and uncontrolled grazing. So, we are not agreeing with this opinion. Additionally, this subject had been examined with the World Bank expert in the oflice and the field. It has been cleared that closing the area against grazing has a positive effect on regeneration of existing and degraded forest and the area could improve by itself. o Besides this, with afforestation activities on bared land, new forests have been established by seeding and planting methods. Finally; there has been improved on water, soil and vegetation sources in watershed and decreasing the erosion. With these precautionsthe flood and torrent have been prevented. At the end of the "Performance and Evaluation Report", it has been said that; o The project has highly plentihl effects on capacity building, o The performances of the World Bank and the other organizations have been evaluated satisfactory, o There is someuncertaintyof sustainability ofthe project, o As a result, there are no deviations to be indicated according to beginning and implementing of the project, o In 1999,the project has gained "The World Bank PerfectionReward". With this regard, The Ministry of Emironment and Foreshy has the same opinionas the report. As a result, from the point of Ministry of Environment and Forest, the "Project Performance Evaluation Report" is regarded suitable and there are no more objections on it.  33 Annex C COMMENTS OF GNERAL DIRECTORATE OF RURAL AFFAIRS (KHGM) Eastern Anatolia Watershed Rehabilitation Project has been implemented as an erosion control and watershed rehabilitation project. The project, which has been coordinated by General Directorate of Afforestration and Erosion Control(AGM) was jointly implemented by AGM, Ministry of Agriculture and Rural Affairs (MARA), General Directorate of Rural Affairs (KHGM) and General Directorate of Forest and Village Relations (ORKOY). Although some problems have aroused during the implementation phase, the project was successfully implemented in general. Firstly, the coordination between government institutions was sufficent. Secondly, the coordination between government institutions and local people was well organized during the project preparation, implementation and protection steps. Participation of local people in the project, in terms of support for supervising and contribution, was very high compared to the other projects implemented in Turkey before. Thirdly, technical and environmental experts rated the project very successful1from technical and environmental aspects. During the project implementation, KHGM faced some problems because of the insufficient local budget funds and some of the villagers demand could not be covered by the project. Finally, in the project areas, erosion control activities and rural development activities were completed in coordination with government institutions and local people.  54 Annex C State Planning Organization's(SPO) Opinionon Eastern Anatolia Watershed RehabilitationProject "Project PerformanceADpraisal Report" The project, which was launched under the loan agreement concluded between the World project cost was US$ 110 million, including US$ 77 million to be provided from the proceeds Bank and our Government on March 25, 1993, was completed at the end of 2001. Total of a foreign loan. Project activities were planned for micro-catchmentsand launched in three micro-catchments in three provinces (Elazig, Malatyaand Adiyaman) in 1993. Project activities were carried out by the General Directorate for Rural Services (KHGM) and the General Directorate for Agricultural Production and Development (TUGEM) and coordinated by the General Directoratefor Afforestation and Erosion Control (AGM). The project's objective included rehabilitation of damaged forests, pastures and farming areas; restoration of sustainable pasturage, forestry and farming activities in upper catchment areas, reduction of sedimentation, erosion and soil deterioration in reservoirs; training and awareness-raising on proper use of natural resources; increasing efficiency and income of poor population living in the catchments; and encouraging participationat planning and application phases. In line with those objectives, the project has three components, namely rehabilitation activities, income-generating activities, and catchment management support activities. Those activities, which were initially planned to be limited to three provinces and 54 catchments, were subsequently expanded to 11 provinces and 88 catchments until the end of 2001. AGM took part in project-related works in 85 of those catchments while TUGEM and KHGM were active in 86 and 51 catchments, respectively. Until the end of 2001, AGM carried out soil conservation afforestation on 71,991 hectares in the 85 microcatchments where it took part in project-related activities. In addition, it rehabilitated 19,282 hectares of pastures, 2,240 hectares of oak and 1,687 hectares of cedar in addition to gallery afforestation on 81 hectares. To carry out those works, AGM spent a total of US$ 43,027,000, including US$ 27,039,000 earmarked from the proceeds of the foreign loan during the period between the inception of the project and the end of 2001. TUGEM's activities in connection with the project included agronomic package applications on 7886 hectares; fallow reduction on 9813 hectares; demonstrations on 1124 hectares; grafting of 1224000 pistachio trees; planting of 238049 seedlings along borders of fields; growing fodder plants on 1903 hectares of irrigated land; establishment of vineyards and gardens on 8864 hectares, as well as distribution of 34560 beehives. To carry out those works, TUGEM spent a total of US$ 6,047,000, including US$ 5,224,000 earmarked from the proceeds of the foreign loan. KHGM constructed irrigation channels with a total length of 1,069,498, 1260 irrigation ponds and 2643 hectares of farming terraces under the project and thus opened 12,368 hectares of land to irrigated farming. To carry out those works, KHGM spent a total of US$ 29,258,000, including US$ 15,711,000 earmarked from the proceeds of the foreign loan. All the organizations concerned spent a total amount of US$ 78,333,000, including US$ 47,975,000 funded by the foreign loan during the nine-year project implementation period. C \DOCUME-l\wb05764\L0CALS-l\Temp\k notes data\DBAN4CIB doc 1 S T  35 Annex C The physical targets remained unchanged although the number of provinces and catchments included in the project had been increased. AGM and KHGM's physical implementation ratios were 152 and 85 percent, respectively as compared with the initial targets. TUGEM's physical implementation ratio was much lower. If the physicaltargets are adjusted in line with the increase in the number of provinces and catchments, it would be seen that realization ratios were actually lower. Therefore, the realization ratios calculated are misleading, because the targets outlined in SAR (Staff Appraisal Report) relate to 54 catchments in three provinces. The project was, however, implemented in 11 provinces and 88 micro- catchments. Meanwhile, cash spending under the project corresponded to 71.2 percent of the initial cost. Credit utilization was 62.3 percent. The initial cost was not adjusted in line with the expansion in physical scope (the increase in the number of provinces and catchments). In spite of this fact, the spending ratio in the project remained low. Had the project cost been increased in line with that change, the realization ratio would have been even lower, which indicates that (a) the initial project cost was considerably high, and (b) problems that hindered activities on pastuTescould not be foreseen. Both factors have resulted from designing and planning errors during project development and preparation phases. This shortcoming caused by the planning phase has resulted from the failure to analyze the project adequately and insufficient involvement of the parties concerned (public agencies such as the SPO and the Treasury, target groups, etc.) and their analyses. Thus, the report states that "participation" was of crucial importance to development, preparation and implementation of the project, adding that it constituted the underlying philosophy of the project. In fact, "summarized excerpts from interviews with village target groups", which is attached to the report in question, indicates that participation of target groups was not adequately efficient. Had all the stakeholders been involved in project planning and implementation processes and adequately analyzed, AGM's lack of experience in working with local people, TUGEM's lack of experience in working upper catchments and the problems stemming from KHGM's regulations could have been identified and thus necessary measures could have been taken and the project could have been designed more efficiently. The report also notes that there were some shortcomings in follow-up on and evaluation of the project. Similarly, responsibilities and duties of all the parties concerning follow-up and evaluation could have been defined clearly and a feasible and sustainable structure could have been established. It is known that SPO establishes connections between development projects and plans and programs, analyses feasibility studies and prioritizes projects before allocation of funds. The next step will involve a decision on funding sources for the priority projects included in the investment program. Therefore, such problems could be avoided if the SPO and the parties concerned efficiently design the project before the financing negotiations related to projects proposed to be funded by foreign loans. The Implementation Completion Report, drawn up by the Bank, and the Project Performance Appraisal Report state that performance of the Bank, which assumed a key role in all phases (project development, preparation, analysis, follow-up of implementation and post- implementation assessment) of the project, which is said to have been developed and implemented based on a participatory approach, was satisfactory. The findings and opinions expressed in sections "Lessons Learned" and "Future Orientations" of the report and "Excerpts from Interviews With Village Target Group", which is attached as an annex, substantially overshadows the project's success. As a consequence, the project's physical and cash realization ratios remained low. As a natural outcome of that fact, the cost of the C \DOCUME-l\wb05764\LOCALS-l\Temp\k nates data\DBAN4CI 8 doc S T 2  36 Annex C paid for undisbursed amounts (commitment fee). loan, which was utilized at a rate of 62 YO,to our country further increased due to amounts Although the amount representing interest payments arising from utilization of the said loan should have been included in the project cost and economic profitability calculations should have been made accordingly, the appraisal reports make no mention of that amount. On the other hand, high figures are given in the evaluations pertaining to economic profitability ratios, but contain no satisfactory explanation as to how those ratios were calculated. A total of 27 missions consisting of the Bank's personnel were organized during project implementation and a total of US$ 860,300 was spent, which corresponds to 1.8 percent of the loan borrowed to fund the project and is therefore considerably high. C \DOCUME-l\wwb05764\LOCALS-1\Temp\k S T notes data\DBAN4C18 doc 3