Nepal - Agricultural Research and Extension Project

Document of The World Bank Report No: 25624 IMPLEMENTATION COMPLETION REPORT (IDA-29770) ONA CREDrr IN THE AMOUNT OF SDR 17.5 MILLION (US$ 24.3 MILLION EQUIVALENT) TO THE KINGDOM OF NEPAL FOR AN AGRICULTURAL RESEARCH AND EXTENSION PROJECT March 24, 2003 Rural Development Sector Unit South Asia Region  CURRENCY EQUIVALENTS (Exchange Rate Effective March 20, 2003) Currency Unit = Nepalese Rupee NRs I = US$ 0.0132 US$ 1 = NRs 75.77 FISCAL YEAR July 16 July 15 ABBREVIATIONS AND ACRONYMS APP Agricultural Prospective Plan BUP Bottom-up-Planning CATC Central Agricultural Training Center DAO District Agricultural Office DADO District Agricultural Development Officer DFID Department for International Development (UK) DOA Department of Agriculture ED Executive Director (NARC) ERR Economic Rate of Return HMGN His Majesty's Government of Nepal ICR lmplementation Completion Repoot MOAC Ministry of Agriculture and Cooperatives MTR Mid-term Review M&E Monitoring and Evaluation NARC Nepal Agricultural Research Council NARDF National Agricultural Research Development Fund NGO Non Govermment Organization PAD Project Appraisal Document PCU Project Coordination Unit RATC Regional Agricultural Training Center RARS Regional Agricultural Research Station RTWG Regional Technical Working Group SDR Special Drawing Rights SOE Statement of Expenditure USAID United States Agency for Intemational Development Vice President: Mieko Nishimizu Country Director Kenichi Ohashi Sector Director Constance Bemard Task Team Leader Robert Epworth  NEPAL AGRICULTURAL RESEARCH AND EXTENSION PROJECT CONTENTS Page No. 1. Project Data 1 2. Pnncipal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 4 5. Major Factors Affecting Implementation and Outcome 8 6. Sustainability 9 7. Bank and Borrower Performance 10 8 Lessons Leamed 11 9. Partner Comments 11 10. Additional Information 12 Annex I Key Performance Indicators/Log Frame Matrix 13 Annex 2. Project Costs and Financing 14 Annex 3. Economic Costs and Benefits 16 Annex 4. Bank Inputs 17 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 19 Annex 6. Ratings of Bank and Borrower Performance 20 Annex 7. List of Supporting Documents 21 Annex 8. Borrower's Completion Report 23   | Project ID: P048026 Project Name AGRI RES & EXTENSION Team Leader. Robert S. Epworth TL Unit* SASRD ICR Type: Core ICR Report Date. March 24, 2003 1. Project Data Name: AGRI RES & EXTENSION L/C/TFNumber: IDA-29770 Country/Department: NEPAL Region: South Asia Regional Office Sector/subsector: Agricultural extension and research (100%) KEY DATES Original Revised/Actual PCD: 10/22/1996 Effective 11/25/1997 11/25/1997 Appraisal: 02/13/1997 MTR: 12/31/1999 02/07/2000 Approval: 08/26/1997 Closing. 09/30/2002 09/30/2002 Borrower/lImplementing Agency Kingdom of Nepal/Ministry of Agriculture & Cooperatives (Dept of Agriculture); Kingdom of Nepal/Nepal Agricultural Research Council Other Partners STAFF Current At Appraisal Vice Piesident. Mieko Nishimizu Mieko Nishimizu Country Manager Kenichi Ohashi Joe Manickavasagam Sector Manager Constance A. Bernard Shawki Barghouti Team Leader at ICR: Robert Epworth Gallus Mukami ICR Primary Author: Charles Bevan (FAO/CP) 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: UN Institutional Development Impact M BankPerformance U Borrower Performance: U QAG (if available) ICR Quality at Entry: S U Project at Risk at Any Time: Yes  3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The objective of the project was to assist His Majesty's Government of Nepal (HMGN) improve the management and capacity of agricultural research and extension services by (a) developing location-specific agncultural technology on the basis of close consultation with farmers; and (b) improving the technology delivery system. The objective addressed a recognized need to introduce substantial institutional reform in both services, although it implied that better management would result from improving specific technology development and dissemination activities, whereas logically better performance in these areas should flow from better management. As they were written, the objectives tended to be more output than outcome based, focusing on technology generation and dissemination as ends rather than as means; whereas aiming to increases productivity and improve farm incomes might have been more appropriate goals. Nevertheless, the intended thrust of the project objective was clear and conformed broadly with the Country Assistance Strategy (CAS Report No 15508-NEP), HMGN's 9th Five Year Plan and the then new Agricultural Perspective Plan (APP), all of which identified agriculture as the lead sector in alleviating rural poverty - with the generation and dissemination of agricultural technology playing an integral role in raising agricultural productivity. 3.2 Revised Objective. The mid-term review (MTR) in February 2000 considered that the project objective was still relevant, and provided that immediate remedial project management action was taken, improving the timeliness and effectiveness technology transfer could be substantially achieved in the project districts in the limited remaining project period; although it was recognized that extrapolation to non-project districts would take more time and additional external financial support. Although development of substantial location specific technology of practical significance to farmers was not expected to be fully achieved, because of the limited time available, it was anticipated that at least the research system and operating procedures to do this should be in place and functional by the end of the project. A partial Credit cancellation of $5 3m was made at that time, based on the revised implementation plan. 3.3 Original Components: The project consisted of two components: agricultural research and extension service. The research component (US$16.2m) was to (a) strengthen research institutions by improving the management information systems, priority setting, monitoring and evaluation, (b) support human resource development, through training and the introduction of a reward system based on results and performance, (c) expand on-farm adaptive research responsive to farmers' needs, and (d) provide facilities, equipment and working capital. The extension component (US$14.3m) would strengthen the extension service by (a) intensifying decentralization to grassroots level and strengthening farmer self-help groups, (b) developing human capital by training and implementing a reward system based on results and performance (c) provision of facilities, equipment and working capital. The components and sub-components were, in principle, well related to achieving the project objective. This project was a successor to two earlier IDA funded agricultural support services projects (Agricultural Extension and Research Project Cr. 1100 and Agricultural Extension Project Cr. 1570) and the USAID funded Agricultural Technology Support Project and DFID funded Hill Agricultural Research Project. However, experience from these projects, especially with regard to the financial, personnel, and general management capacities of the implementing agencies, was not taken into account adequately during project preparation. Components activities were broad ranging, but were not well defined and did not specify clear implementation milestones by which performance could be judged; and in the Nepal context, required many prerequisite institutional and procedural reforms in a relatively short period of time. 3.4 Revised Components: At the MTR, the project was not re-structured, but rather greater clarity of design and focus was given to the existing components/activities. This redefinition of activities aimed to provide a clearer understanding of the institutional and procedural reforms needed, together with achievement targets, and to give a practical guide to implementation procedures. A clear statement of outcome, activity, and output indicators was incorporated into a - 2 -  new logical framework - designed through a series of stakeholder workshops led by the two implementing agencies, the Department of Agriculture (DOA) and the Nepal Agricultural Research Council (NARC). These were closely related to the project objective, and were considered to be within the implementation capacity of each agency, provided ownership and commitment of the senior mangers expressed at the time of the MTR was maintained. The Agricultural Research Component was structured into the following clearly identifiable sub-components. * Modernising and reforming NARC: (a) Clarification of the NARC structure; (b) Human resource development; (c) Consolidation of resources around high priorities, (d) Monitonng and evaluation of research; (e) Information technology and management information systems; and (f) Enhanced collaboration with other R&D providers. * Strengthening Management and Systems Research at regional and farm level: (a) Relocation of staff to Regional Stations; (b) New outreach policy and guidelines for on-farm research; (c) More systematic discussion and pnontization of the research and extension agenda in Regional Technical Working Groups (RTWG); (d) Improved research-extension linkages at the national and regional levels; and (e) Adoption of impact and feedback studies of technologies generated by NARC. * Beyond NARC toward a National Agricultural Research Policy and Fund- (a) Formulation of a national agricultural research policy, (b) Decision on where and how to establish a National Agncultural Research and Development Fund (NARDF). The Extension Services Component was structured into the following sub-components: * Modernization of Public Extension Services (a) Improvement of district infrastructure (agricultural service centers and sub-centres), communications; and projectization of district programs (i e. description of discrete extension interventions based on clear objectives and related actions); (b) Strengthening support programs (training, mass media, M&E, technical support);(c) Modermization of Regional and Central Agncultural Training Center (R/CATC) curricula; (d) Formulation of an up-dated National Extension Strategy; and (e) Improving financial management systems and the introduction of an MIS system. * Promotion of Partnerships: (a) Strengthening process of group formation; (b) Training of private extension providers; (c) Piloting of a program of joint financing with private extension providers. * Decentralizing Extension Services: (a) Development of more effective 'Bottom-up' planning and prionty setting; (b) Piloting decentralization under Decentralization Act. * Human Resources Development: (a) Orientation of all staff in new Extension Strategy; (b) Provision of degree and in-service training; (c) Development of distnct based databases; and (d) Study of human resource needs. This sub-component also included some provision for the Institute of Agricultural and Animal Sciences (IAAS) to upgrade post-graduate computer facilities; and scholarships for eight post-graduate students. 3.5 Quality at Entry: At the time the project was appraised, the new Project Appraisal Document (PAD) format had just been introduced The PAD covered the essential features and expenditure of the project, with an Implementation Volume presented as an annex. However, in this case, the latter was more a technical review of research and extension in developing economies and proposed future areas of research focus in Nepal. Implementation arrangements and detailed activities for each component were to be detennined by DOA and NARC once the project had commenced, with technical assistance coming from consultants appointed under the project. To oversee project implementation there was to be an inter-ministerial Steering Committee, assisted by a Technical Advisory sub-Committee. Within the MOAC a Project Coordination Unit (PCU), headed by a Project Coordinator was to be established. -3 -  The project design incorporated past experience, especially from extension projects, by emphasizing decentralization; strengthening of research/extension linkages; and adoption of a participatory approach by involving farmer groups. It also provided flexibility in design by not being overly prescriptive. But in doing so, it made implementation success overly dependent on the early provision of substantive amounts of international and national technical assistance - which in the end did not eventuate for some time. The design process did not fully involve all stakeholders, nor take into consideration some of the three earlier preparation studies undertaken by FAO/CP. It also underestimated the nsks associated with project implementation, especially the delay in TA recruitment and the likelihood of frequent senior staff changes in implementing agencies - although the extreme situation at NARC could hardly have been anticipated. Little was leamt also from the experiences of earlier projects that had performed poorly because of a failure of steering committees to meet regularly and function effectively. After more than seven years in the making, final preparation and appraisal appears to have been rushed, moving ahead of the borrower's ownership of new concepts such as decentralized planning and implementation, and public-private partnerships in service delivery. This ownership came eventually, but only later in the implementation period. Overall the Quality at Entry is therefore rated as unsatisfactory, because of the weak borrower ownership of some of the key concepts of the project at that time; over-dependence on timely provision of substantial techmcal assistance; and inadequate higher-level coordination and monitoring arrangements. This resulted in a slow project start-up, as implementing agencies were initially unsure of precisely what they were supposed to be doing. In this vacuum, by default, the PCU itself started implementing the project, rather than providing administrative and coordination support to the two implementing agencies, DOA and NARC, as it was set up to do. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Despite the implementation difficulties, especially in the first two years of the project, some progress has been made towards achieving a number of important outcomes and outputs, following a notable improvement in implementation pace in the latter stages of the project. During this time, a number of achievements were made: (a) in research, the NARC 2020 Vision was finalized and approved, and management of the research program at the regional level started improve, including better priontization and liaison with extension and farmers; (b) in extension, the introduction of bottom-up planning, expansion of partnerships and contracting out of some extension activities, and initiating the projectization of district work programs; and (c) in both components, substantial higher level training of scientific and technical staff. However, the greatly improved implementation pace after the MTR faltered towards the end of the project, due partly to senior staff changes (particularly at NARC), and partly to the worsening security situation in the final ten months, which greatly constrained the ability to do fieldwork. Significantly, in the last year of implementation, the Project Management Committee established at the MTR hardly met, and there have continued to be changes in NARC leadership (six Executive Directors during the project period, with many holding the post in an acting capacity for long periods). Compared to what was anticipated at the time of the MTR, NARC has been unable to fully institutionalize the necessary organizational and procedural reforms needed to meet the ultimate goal of developing new location specific agricultural technology; while the activities of the extension component have not yet led to the widespread adoption of improved service delivery mechanisms. The project has therefore only partially met the reform targets which were fundamental to achieving the objective of improving the management and capacity of agricultural research and extension services, and as a result, the project outcome is rated as unsatisfactory. To a considerable extent these shortfalls in achievement can be also attributed to the fact that institutional reform within the public sector always takes time, especially if it is expected that this will be generated to some extent from within the organization. In the end there was just not enough time left in the second half of the project to institutionalize many of the new initiatives coming from the MTR. There is every reason to believe, however, that if funds had been available and the field security situation had improved to make an extension of the project possible, then the overall achievement of project objective might eventually have been satisfactory. - 4 -  4.2 Outputs by components Research. A review of the baseline output indicators agreed at the MTR indicates that, although a number of important new initiatives were started and made early progress, at the end of the project, only a limited number of the key performance indicator have been achieved in a substantive and sustainable way. Overall the research component is therefore rated as unsatisfactory. * Modernization and NARC Reform. There has been important reform of the NARC structure, with policy development and research coordination now formally separated from research implementation. The number of research projects being undertaken has been rationalized and the program consolidated, a successful proactive search for new funding sources for high priority topics has been undertaken; and there has been an increase in collaborative R&D activities with NGOs and the private sector. A monitoring and evaluation division has been established, although the systems by which it will operate is still to be developed and institutionalized. A computerized accounting system has been introduced, but is not yet operative at all NARC stations. Databases for a National Budget Management System and Personnel Management Information System, have recently been established at the headquarters, but are not yet available or accessible elsewhere. However, most of the institutional strengthening has been only partially implemented, and has therefore not been fully tested, and modified as necessary. A significant amount of short term local training within NARC has been undertaken, and postgraduate training was provided to a substantial number of scientists (67 MSc and 13 PhD). Most have completed their studies, but a significant number remain uncompleted at project end because of late approval of scholarships. Although the process of staff promotions was restarted in the final months of the project, this was not completed and a result, there has been no promotions of scientists for more than six years. There has also been very limited (39) reduction in the numbers of support staff. * Strengthening Regional Agricultural Research Management and Linkages. There has been some improvement in an integrated multi-disciplinary approach to outreach programs at Regional Agncultural Research Stations (RARS) There has been significant improvement in the systematic discussion and prioritization of research.in the regions, with evidence that some of these new programs are being driven by the farmers' agenda. The operation of Regional Technology Working Groups (RTWG) has improved significantly and now provides an effective forum where staff from all agencies involved in technology development and dissemination can exchange ideas. There has also been some improvement to RARS facilities in the Terai (notably renovation of accommodation and the construction of new office buildings). None of the NARC vacant positions has been filled, so as to strengthen the core group of scientists and technical staff in place at Regional Agricultural Research Stations; and although some initial transfers were made, in the end, there has been no net redeployment of scientists from Kathmandu (Khumultar) to priority programs based at the regional research stations. As a result no RARS have reached the targets of scientific staff set at the MTR. Inability to introduce incentives in a public sector environment to encourage scientists to live and work in the regions suggests that these targets will not be reached, at least not in short run. * Beyond NARC: National Agricultural Research Policy and National Agricultural Research Development Fund (NARDF). The NARC Vision 2021 has been produced and approved, although it still requires final endorsement by the Cabinet. Plans for implementing the Vision, which defines NARC's future role in the wider national research system vis a vis other players such as universities and the pnvate sector (and is widely considered to be a good document) are under preparation, but much more time is needed to complete the work In a parallel but interlinked activity funded by DFID, a decision in principle to establish a NARD Fund has been taken by the government, but the final modus operandi is still to be worked out. Extension Services. Many new initiatives have been started since the MTR, but these are far from being sustainably institutionalized. The review (Appendix 3) of Farmer Groups undertaken by the ICR mission confirms - 5 -  that anticipated extension service reforms still have some way to go before they can be said to be fully incorporated into regular activities throughout the country. In particular most farmers at this point in time continue to see the DOA extension service as a possible source of subsidized inputs, rather than as a source of knowledge. Often input suppliers and traders are seen as the most credible source of technical knowledge. Overall, the extension component is therefore rated as unsatisfactory. o Modernization of the Public Extension Service: The project has resulted in improvement of DOA infrastructure at the district level, with numerous agricultural service centers and subcenters rehabilitated and two new Distnct HQ buildings constructed. The concept of projectization introduced at the MTR (development of district extension programs in discrete projects designed to address specific local problems, and with performance indicators built-in) was to be started in five districts on a pilot basis. But MOA quickly saw this as a more transparent management tool which tied financial inputs with Gutputs and impact, and before the pilot could be tested fully, enthusiastically decided to adopt the approach first in all 23 project districts, then expanded it to all 75 districts of Nepal. This was too fast, and as might be expected in the start-up year, in most distncts the process has been used to simply disaggregate the on-going district program into smaller discrete activities. This means that most of the projectization continues to be applied to the Pocket Package Programs, which focus on providing mini-kits and other subsidized inputs to crops with high potential in limited geographical areas. The initial prioritization proformas and procedures are considered by most Distnct staff to be too complicated, and require refinement to enable them to be used effectively throughout the system. With an initiative such as this, it was always expected that quality would improve greatly as more annual planning and implementation cycles were completed. Given the high level interest in this initiative, there is good reason to believe that had the project been extended, that this projecttzation process would have gone on to become an effective, practical extension management tool. o The distnbution of extension materials (calendars, monthly agricultural bulletin etc) has expanded greatly, particularly in the last year of the project. Preliminary attempts have been made to reform and expand the RATC/CATC curricula to cater for the new training needs of extension staff, rather than providing basic technical training (much of it directly to farmers) as it did in the past. A number of policy related documents and guidelines (e.g. project formulation, forming partnerships, M&E procedures, projectization and contracting-out) have been prepared, but have yet to be consolidated in to a new National Extension Strategy - although a stakeholder workshop was held in Kathmandu in September 2002 to discuss the strategy . Financial management of the project, especially at field level, has been a chronic problem throughout, although it was exacerbated late in the project by the wholesale transfer of accounting staff by HMGN and the deteriorating secunty problem in the field, which obstructed planned training programs to address the accounting skills problem. Development of both an M& E system and MIS has been very slow, although a start was made on these in the final month of the project, with the support of some effective technical assistance. o Promotion of Partnerships. There has been some increase in the number of groups formed by DADOs, but the survival rate beyond the first year of establishment has been a problem. There has been some development of partnerships with groups formed by other organizations, although it seems that pnvate organizations are reluctant to enter into formal partnership agreements with District Agricultural Offices (DAOs), especially those involving joint financing. An interesting initiative has been taken up with the agro-vet shop operators, where farmers get ad hoc advise when agncultural chemical and vetennary products are purchased. This involved workshops on environmental health issues, which later evolved towards more general extension advice. o Decentralizing Extension Services. A start has been made in training field staff in Bottom-Up-Planning (BUP) processes, using problem census/problem solving techniques, although many field staff are not yet fully conversant with the use of this technique. Although not an original project activity, in response to HMGN's announced intention to decentralize agncultural extension, as one of the first five government departments to start, a limited decentralization pilot was introduced at the MTR. However, in the end, - 6 -  delays by the government in finalizing the arrangements for decentralization senously inhibited the piloting of this activity. It was only in April 2002 the Government finally confirmed that agncultural extension would be decentralized, and although planning had started, by then it was too late under the project for MOAC to work out detailed arrangements and pilot these with District Development Committees. * Human Resources Development. Most DOA staff have received some training in the key aspects of the new extension strategy (e.g. projectization, BUP, and partnerships). In addition, over 6000 short in-service courses have been provided and 115 short courses overseas were completed. Significant higher level training has also been done. Three DOA staff have undertaken PhD degrees, 53 have either completed or are completing M.Sc degrees, and 28 have completed B.Sc degrees, either regionally or in Nepal Eight members of staff of IAAS also received post-graduate training. 4.3 Net Present Value/Economic rate of return Not applicable At appraisal, ex-ante calculations projected a project ERR of 51% and 70%, depending on partial or full implementation of the APP In view of the delays in project implementation and subsequent inability to fully achieve many of the basic project objectives, it is not possible to attribute any change in agncultural productivity at this early stage to project interventions. Under the circumstances, it was considered that no purpose would be served by attempting to recalculate the ERR. It is interesting to note, however, that agricultural production and productivity in Nepal has increased during the project period. From a situation of being a net importer for most of the 1990s, Nepal had a surplus of just under 70,000 MT of rice in 1999/2000 According to DOA figures, in the project districts the productivity of paddy increased from about 3.35 mt/ha to 3.7 mt/ha during the project life; while the average yield of wheat increased from 2.35 mt/ha to 2.67 mt/ha, maize from 2.35 mt/ha to 2.82 mt/ha, vegetables from 10.9 mt/ha to 12.87 mt/ha and potatoes from 10.6 to 14.99 mt/ha. These gains are claimed by DOA to be mainly due to wider adoption of new technologies. Allied reasons for this could also be that this is the result of the liberalization of the fertilizer market, and the fact that many farmers appear to be responding to opportunities presented by freer input and output markets. 4 4 Financial rate of return: No financial rate of return was calculated at Appraisal 4.5 Institutional development impact: Perhaps the major impact of the project has been the significant change in the mindset of management and senior staff of both DOA and NARC as to how their operations can be conducted more efficiently and effectively; and to the importance of developing partnerships with other interested parties in the research and extension field. From this, the foundations for reform have been laid in a number of important institutional and operational areas. Within the DOA, a good start has been made on changing the way business is done (e.g. projectization; bottom up planning; involvement of farmers groups). Similarly within NARC there have been important policy and management changes (e g. finalization of the Vision document; fewer, better prepared research projects; partnerships with other researchers; improved effectiveness of RTWGs) There is also much better linkage between the two agencies and a genuine recognition of the synergistic benefits from cooperation. Both agencies now also recognize the important role that can be played by the private sector and NGOs in both extension and research. In some cases, such as in the Nawal Parasi district, close cooperation between DOA and NARC is attributed to a considerable extent to support from the Distnct Development Committee - an example of decentralization starting to have a positive effect on service delivery. But many of the reforms are at an early stage, and others have yet to really begun in earnest (e.g. reform of CentraURegional Agncultural Training Centers training programs, or change in RARS research programs). And what is not clear is whether the reforms will be sustained after the project resources are no longer available. Overall the achievement of the institutional reform aspect of the project is therefore considered, at this stage, to have made a positive but limited contnbution to improving the ability of implementing agencies to make more effective use of their human and financial resources in providing a better service to farmers Institutional development impact is therefore rated as moderate. - 7 -  5. Major Factors Affecting Implementation and Outcome S. I Factors outside the control of government or implementing agency: The principal factor outside the control of either the government or the implementing agencies was the detenorating secunty situation in much of the country. This has severely restricted access and freedom of movement in many districts. It has also had an indirect effect through the curtailment of resources available for regular government programs. A second important factor was the fluid political situation in Nepal during the project penod, with ten changes of government over the past seven years, including three since the MTR in early 2000. 5.2 Factors generally subject to government control: Frequent changes of leadership of implementing agencies, particularly NARC, had a detrimental effect on the project; as did the inability of government to allow and encourage redundancies and staff transfer dictated by basic business needs of implementing agencies. Delays also arose at times from slow release of counterpartlHMGN funds. Indecision on TA by MOAC was in part responsible for delaying recruitment of the initial consultants in a properly scheduled way, which resulted in serious delays in project start-up activities. 5 3 Factors generally subject to implementing agency control: A major issue affecting implementing was the failure of both DOA and NARC to take ownership of their respective components in the initial years of the project. It was not until late 1999 that both became proactive, contributing fully to the MTR process in early 2000 and beyond. But by that stage, effectively two years of implementation had been lost. Frequent changes/redeployment of staff in both implementing agencies also resulted in serious delays to implementation and even in some cases (e.g. staff transfers to regional stations and then later reversing the decision and posting them back to Khumultar) to reform decisions being reversed. 5.4 Costs andfinancing: The total project cost at appraisal was US$30.5 million, of which US$6.2 million equivalent was to be financed by HMGN and US$24.3 million equivalent (SDR 17.5 million) would come from the IDA Credit. The bulk of the Credit (US$11.1 million) was to be used for consultancies, implementation support and training for capacity building The other main project expenditure categones in order of magnitude were goods (equipment and vehicles), incremental operating costs and civil works. Disbursement in the early years of the project was much lower than anticipated, and therefore at the MTR, the Credit was revised to SDR 13.5 million, with US$5.3 million equivalent (SDR 4.0 million) being cancelled, based on the revised implementation schedule and reduced estimated project costs of US$ 24.6 million. Some reallocations between disbursement categories was also done at this time, with a decrease in the allocation for consultancies and training, and a small increase in expenditure on civil works and incremental operating costs. At Credit closing total project costs were estimated at US$ 23.8 million, or 96% of the MTR cost projection. Final Credit disbursements stand at US$ 17.59 million (SDR 13.39 million), equivalent to more than 99% of the revised Credit of US$ 17.75 million - leaving only US$ 0.15 million to be cancelled. In terms of actual expenditure, consultancies and training exceed MTR estimates by 50 %, whilst expenditure on works, and goods were 80% of the MTR estimates and operational expenditure was 94%. Cost over-runs in the former category relate mainly to training, rather than consultancies: in-country training of staff of DOA and NARC along with NGOs and farmers workshops; and also additional un-costed higher level training, especially for staff of the Institute of Agriculture and Animal Sciences (IAAS). This was a pre-LACI project and hence no formal financial assessment was conducted. A project specific Accounts Manual in local language was produced in the first year, and this manual was disseminated to all cost centers and training was also provided But transfer of accounts staff is a common, generic problem in Nepal, and the necessary follow-up dissemination and training by the PCU did not take place effectively. In most of the Bank implementation reviews, financial management advice was provided to address the obvious weaknesses, but the - 8 -  response of the PCU was not adequate and financial management remained unsatisfactory throughout implementation. Issues that contnbuted to poor performance were: (a) frequent change of accounting staff and lack of sufficient skills to manage project accounts; (b) insufficient coordination between the PCU and NARC on matters related to financial information flow, (c) absence of an effective system to monitor field-based cost centers and insufficient supervision visits to field cost centers by PCU accounting staff, (d) lack of a functioning information system to enable retrieval of management information including a proper SOE tracking system, and (e) lack of training of distnct accounting staff. Towards the end of the project, the PCU staff made special efforts to collect information from cost centers and submit all outstanding claims, which led to the smooth closure of the Credit: Many of the problems identified then could have been resolved earlier if the project management had shown a strong commitment to improved financial management; if implementation had not affected by frequent changes of staff; and if the PCU had played a more dominant role in monitoring, training the staff of cost centers and in maintaining the system as envisaged in the Accounts Manual. Audit'reports, in general, were received on time with only a few delays. Auditors had noted weak internal controls which included lack of maintaining certain ledgers and an asset register. Most audit opinions were unqualified, but the management letters had several suggestions to improve the financial management system which were not adequately addressed by the PCU. 6. Sustainability 6.1 Rationale for sustainability rating: Despite the good efforts of project agencies over the last two years, as explained in earlier sections, the institutional development impact has only been modest, and only some of the changes introduced under the project are expected to really take hold. In a post project situation, it is probable that the DOA will continue to use the new projectizatton procedures, as they provide for better management flexibility, a key consideration at the time of budget cuts; and to a lesser extent bottom-up-planning and partnerships in service delivery. Similarly, NARC regional stations will probably continue to use the procedures adopted under the project for problem identification with farners and extension agents and for research project formulation. But the continuation of a number of operational and institutional reforms started under the project is questionable. Reform of research and extension systems is by its very nature a long term undertaking and requires political stability, adequate budget resources and sustained consistent higher level commitment to change. Given the current circumstances in Nepal, it seems that the sustainability of many of the reforms initiated under the project is probably unlikely. 6.1.1 The prime reason for the rating is simply that there has not been enough time since the MTR to properly institutionalize many of the institutional reforms and new operational procedures. This situation is exacerbated by the ongoing civil unrest associated with the Maoists which meant that many project districts were affected for the last ten months of implementation. With the increased risk to staff and the probability of vehicle abductions, the scope for researchers or extension agents to work with farmers was greatly constrained. There was also an associated effect through budget cutbacks, resulting in substantially reduced the funding for research and extension activities in the last year of the project. Finally, there is some doubt over higher-level commitment (with the inevitable knock-on effect on staff of DOA and NARC) to maintain the major project reforms. Since the middle period of the project when strong leadership within the MOAC led to the notable improvement in project performance, there appears to have been some erosion of commitment to change in the latter months. 6.2 Transition arrangement to regular operations: 6 2 1 It was envisaged at the time of approval that the PCU would provide only coordination and administrative support, and that the DOA and NARC would implement the project with existing staff and organizational structures. In this way at project closure there would be a seamless transition to regular operations. This has happened to a large extent in the case of NARC, which always operated fairly independently of the PCU. Although the PCU was always more closely connected administratively to DOA, transition to regular extension operations should still be relative seamless, as post MTR, most project initiatives have been implemented primarily -9-  through the existing district extension apparatus. 7. Bank and Borrower Performance Bank 7.1 Lending: Although the appraisal team made some use of the earlier preparation studies, it designed a project using concepts (e.g. public/private partnerships) and implementation arrangements (heavy TA inputs, not grant funded) which at that stage, were not yet fully accepted by the borrower -albeit they became accepted later, during implementation. This led to poor quality at entry, as explained in para 3.5. The overall rating for Bank lending performance is therefore rated as unsatisfactory. 7.2 Supervision: Supervision in the first year or so of the project, although regular, failed to properly address the then apparent project design faults and improve performance. To some extent this is attributed to a lack of commitment to the project by the implementing agencies and MOAC during that time; but also to the fact that the project had four task leaders before it was 18 months into implementation. From late 1999, just prior to the MTR, supervision performance improved substantially, and can be considered as fully satisfactory from there on. As stated by the borrower, review teams developed a strong working partnership with both implementing agencies and through this constructive interaction, were helpful in addressing and resolving numerous implementation issues as they arose, with good support coming from the Kathmandu office between the six monthly supervisions. Overall, supervision performance is therefore rated as satisfactory. 7.3 Overall Bankperformance: Despite good supervision performance in the last three years of the project, the performance of the Bank in the crucial preparation/appraisal phase and early implementation was unsatisfactory. In balance therefore, overall Bank performance is rated as unsatisfactory. Borrower 7.4 Preparation: Although NARC and DOA senior staff participated in project preparation, because of frequent staff changes, continuity of involvement was a problem. In the end, it appears that the borrower entered into a project not beng- entirely convinced of some of the basis tenets of project concept (role of private sector and NGOs) and not entirely comfortable with the amount of TA being funded from the Credit. As both adversely effected early implementation, borrower performance durng preparation and appraisal is assessed as unsatisfactory. 7.5 Government implementation performance: The proposed Steering Committee was supposed to be the main mechanism for project direction. However it was never effective identifying and rectifying the lack of commitment and interest in the project by implementing agencies in the early stages of implementation. Things improved greatly for a period of about 18 months at the time of the MTR, when the then Minister took an active interest in the project, but slipped again towards the end. Procrastination over appointment of TA; inordinate numbers of staff changes (particularly the leadership of NARC); and sometimes inadequate provision of timely counterpart funds also adversely effected the project. As these were under the direct control of the government, their performance overall is therefore rated as unsatisfactory. 7.6 Implementing Agency: The November 1999 implementation review concluded that the project was having little impact. This was attributed partly to problems in project design, but also to lack of interest and commitment on the part of the two implementing agencies. From then, and through the MTR and beyond, things improved considerably, following increased attention to the project by then Minister of Agriculture and the appointment for four years of a full time Executive Director (ED) of NARC. Following the MTR a new Project Coordinator was also appointed and for much of the next two years the performance of both implementing agencies improved. However, towards the end of the project the performance of both agencies again weakened, prompted to some extent by the deteriorating security - 10-  situation in the field, but also at NARC, by yet another change of Executive Director and a number of other senior staff posts. Both these developments were beyond the control of the executing agencies, and therefore, based on the very solid performance leading up to the MTR and for most of the period after, the performance of DOA and NARC is rated satisfactory, but only marginally so. 7.7 Overall Borrower performance Based on the above arguments, the overall performance of the borrower is rated as unsatisfactory 8. Lessons Learned 8.1 The design and preparation of projects like AREP must be based on broad-based stakeholder participation, especially the project beneficiaries, and must reflect national pnorities, to help establish full borrower ownership. In addition, project preparation should include a thorough institutional and organizational analysis of potential project agencies, to ensure appropriate project implementation arrangements. 8 2 Major institutional reform and the adoption of new management systems require long-term support and are only possible when there is broad commitment from the borrower. Where a project calls for extensive institutional reforms the design must realistically reflect the ability, and the commitment of the borrowing country to adopt new systems. Institutional reform is a slow process, and might take ten years or more to complete and sustain. The Bank should only initiate and support institutional reform projects if it is prepared to commit itself to the full period needed for the adoption of reforms (provided borrower commitment remains strong). This is especially so when the Bank is promoting a system change which varies from what it promoted earlier, as has happened in Nepal, in the post-TV extension era. 8.3 The need for a Project Coordination Unit should be carefully assessed on a project by project basis to ensure line agencies take full responsibility for project implementation, and on project closure that there is no institutional sustainability. Where issues such as procurement or large training programs are perceived to be administrative problematic without a PCU arrangement, outsourcing to specialized agencies should be investigated. 8.4 Credit funds should be cancelled judiciously in institutional reform projects such as this, and only after it is certain they will not be needed. In the case of AREP, cancellation of funds at the MTR effectively precluded the extension of the project, just when it was beginning to show that additional resources might have helped consolidate many of the changes introduced by the project. 8 5 The need for technical assistance, especially long term consultancies, must be critically assessed, and where it is needed, it must be fielded on time, and preferably be grant financed. If Technical Assistance is considered essential, procurement should be sufficiently advanced prior to the Board date, to avoid start-up delays. The use of IDA Credit for TA is often unpopular with borrowers, and results in procedural difficulties and delays; and should only be used in a last resort where there is no grant funding available. This implies greater effort by the borrower and the Bank to identify and mobilize TA on a grant basis, from other interested donors 8.6 Higher level training should be undertaken as early as possible in the life of a project. This means planning for the first batch of trainees needs to be well advanced before the Board date. It should also be phased in such a way as to minimize disruption caused by long staff absences and to allow initial trainees to be involved in implementation in the later stages of the project.. 9. Partner Comments (a) Borrower/implementing agency In general, the borrower agrees with the general thrust of the ICR: that project design and early implementation were deficient in crucial areas; commitment to the project concept and the performance of both the borrower and the Bank improved greatly from the MTR onwards; but then project performance waned at the end, partly due to - 11 -  the civil unrest in Nepal, especially in the rural areas. The borrower is however more positive about sustainabihty of project reforms, and considers that the ICR ratings for Project Objective, Outcomes, and performance of implementing agencies should therefore be more positive - suggesting that "slightly " satisfactory is a more realistic rating for each of these than the unsatisfactory rating given. The borrower has prepared a completion report which is presented in Annex 8. (b) Cofinanciers: (not applicable) (c) Other partners (NGOs/private sector): Not applicable 10. Additional Information - 12 -  Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: .' .