Sri Lanka - Second Agricultural Extension Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 20674 PERFORMANCE AUDIT REPORT SRI LANKA SECOND AGRICULTURAL EXTENSION PROJECT (Credit 2380-CE) June 29, 2000 Sector and Thematic Evaluation Group Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents (annual averages) Currency Unit = Sri Lanka Rupees (Rs) Fiscal Year Government: January 1- December 31 Abbreviations and Acronyms ASC Agrarian Service Center DAPH Department of Animal Production and Health DEA Department of Export Agriculture DOA Department of Agriculture EDU Extension Development Unit ES Evaluation Summary GOSL Government of Sri Lanka ICR Implementation Completion Report MAL Ministry of Agriculture and Lands MTR Mid-term Review PAR Performance Audit Report PC/PS Problem Census/Problem Solving PC/PA/PS Problem Census/Problem Analysis/Problem Solving PMU Project Management Unit QAG Quality Assurance Group SAEP Second Agricultural Extension Project Director-General, Operations Evaluation Mr. Robert Picciotto Director, Operations Evaluation Department Mr. Gregory K. Ingram Manager, Sector and Thematic Evaluation Group Mr. Ridley Nelson (Acting) Task Manager Mr. Madhur Gautam FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. ROBERT PICCIOTTO Director-General Operations Evaluation June 29, 2000 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Sri Lanka Second Agricultural Extension Project (Credit 2380-CE)) Attached is a Performance Audit Report for the Second Agricultural Extension Project (SAEP), which was supported by Credit 2380-CE for US$14.34 million, approved in August 1992 and made effective in February 1993. The total cost at appraisal was estimated to be US$18.32 million. The project was closed one year ahead of schedule, on June 30, 1998, after having been declared a "problem" project in March 1996. Final project cost was US$10.05 million, of which IDA contributed US$8.02 million. The remainder of the credit was cancelled. The objective of the project was to increase farmers' incomes and agricultural production in the non-plantation sector by improving farm productivity, inter alia, through the promotion of innovative technologies. The project had two main components: a multifaceted program to strengthen extension services, and the development and implementation of a national seed policy. The former included creating an integrated extension delivery system; establishing an innovative and cost-effective farmer-centered extension approach; improved use of mass media techniques; strengthening the plant protection and pesticide control divisions of the Department of Agriculture (DOA); staff training; and technical assistance for an innovative pilot private extension program. The project failed to achieve its objective of increased farmer incomes or productivity. The innovative participatory extension approach it promoted did not work as anticipated and could not be institutionalized. The mass media program produced a variety of materials after considerable delays, but their dissemination has been limited. The pilot private sector extension component was not implemented. With significant delays, the Cabinet approved a new seed policy but it has yet to be implemented. Several problems contributed to the poor performance of the project. The complexity of the institutional arrangements across several agencies was complicated by the decentralization of some functions of the government services. Weak links between the provincial and central departments, and a virtually total lack of commitment and ownership of the project objectives at the top levels of management made the integration of line agency extension services a serious challenge. The reported lack of consultations in project design alienated the provincial officers, who viewed the project was an external imposition. A major factor affecting implementation was the tension between SAEP and the new Integrated Participatory Process for Agricultural Development, popularly known as the AMA program, of the government that took office in 1994. The senior management of the Ministry of Agriculture and Lands actively promoted the AMA program, often in direct competition for resources and staff time with SAEP. The resulting lack of clarity and confusion among field staff significantly affected the implementation of SAEP. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. -2- There were problems in the design of the project as well, as may be expected in any institutional development project, especially one with a radically different way of doing business. Although the project appropriately started out as a pilot, the Bank's pre-MTR supervision missions erred strategically in endorsing the rapid expansion of the project from a pilot to full national coverage. The MTR recognized the poor performance of the project, declaring it a "problem" project. Restructuring the project was clearly needed to get it back on track. In what ensued, however, the Bank's performance was highly unsatisfactory. The sector unit, based at headquarters, wanted to restructure the project, consistent with the advice from a QAG review and the region's own operations advisor. The Country Director's office, however, decided to close the project. The debate on the fate of the project was entirely between the sector unit and the Country Director's office. There was no communication with the government for more than six months, by which time the Bank had unilaterally decided to close the project. There was no attempt to engage the government in negotiating possible options. Whether or not this would have resulted in any better outcome is moot, but the process raises important issues for the Bank as it becomes increasingly demand-driven and decentralized: the need to engage the borrower on important decisions regarding investments, and the decision making within the Bank on important project related matters, with implications for outcomes and accountability. The audit confirms the ICR's ratings for outcome as unsatisfactory, sustainability as unlikely and borrower performance as unsatisfactory. The audit disagrees with the ICR rating for institutional development as partial (modest), and rates it negligible. It also disagrees with the ICR's rating for Bank performance as satisfactory, and instead rates it unsatisfactory. The project experience offers four lessons: * The need for sound analytical foundations for project design, either through economic and sector work or in-dept project preparation. * The need for partnership with, and building ownership by, increasingly decentralized implementing agencies. * The need for appropriate incentive structures that hold service providers accountable to their clients for a truly demand-driven service delivery mechanism. * With increasing decentralization of its own work, the Bank needs to clearly resolve where the accountability for project outcomes lie. As a development partner, the Bank needs to involve the borrower in important decisions affecting its investments, and to engage the borrower in the choice of restructuring and closure options. Attachment 1 Contents Principal Ratings...........................................................................................................................iii Preface.............................................................................................................................................v 1. Introduction ............................................................................................................................. 1 2. M ain Findings.......................................................................................................................... 2 3. Im plem entation ........................................................................................................................ 4 4. Issues.........................................................................................................................................5 5. Principal Ratings....................................................................................................................10 6. Lessons .................................................................................................................................... 10 Annex A: Basic Data Sheets ..................................................................................................... 12 Annex B: Borrower Com m ents .............................................................................................. 15 Annex C: OED Response to Borrower's Comments............................................................ 16 This report was prepared by Madhur Gautam (Task Manager). William B. Hurlbut edited the report. Marcia Bailey provided administrative support.  111 Principal Ratings ICR Audit Outcome Unsatisfactory Unsatisfactory Sustainability Unlikely Unlikely Institutional Development Partial Negligible Bank Performance Satisfactory Unsatisfactory Borrower Performance Deficient Unsatisfactory Key Staff Responsible Task Manager Sector Manager Country Director Appraisal 1. Serejski H. Hellman P. Isenman Midterm 1. Serejski R. All R. Bendterjodt Completion N. Fernando R. Ali R. Bendterjodt  V Preface This is a Performance Audit Report (PAR) for the Second Agricultural Extension Project (SAEP) in Sri Lanka. Credit 2380-CE in the amount of US$14.34 was approved in August 1992 and made effective in February 1993. The total project cost was estimated at appraisal to be US$18.32 million. The difference between the project costs and IDA credit was to be financed by the Government of Sri Lanka (US$3.79 million) and farmers (US$0.19 million). The project closed one year ahead of schedule, on June 30, 1998. Final projects costs were US$10.05 million (70% of appraisal estimate), of which IDA contributed US$8.02 million. The remainder of the credit, about US$6.0 million, was cancelled in May 1998 and the undisbursed amount of US$0.54 million was cancelled at the time of the last disbursement in September 1998. This PAR is based on a review of the Staff Appraisal Report, the Credit Guarantee Agreement, Bank Supervision reports, correspondence files, project and related background reports and documents, internal Bank memoranda, discussions with Bank staff, and field work by an OED mission in December 1999. OED is grateful to the Government of Sri Lanka, the Ministry of Agriculture and Lands, the Coconut Cultivation Board, and other agencies associated with project implementation for the cooperation and assistance they extended to the OED mission during preparation of this PAR. Following standard OED practice, a draft of this report was sent to the borrower for review and comments. Comments received from the Ministry of Finance and Planning are attached as Annex B.  1 1. Introduction The Second Agricultural Extension Project (SAEP) adopted an innovative project design, incorporating key lessons learned from an earlier extension project. These lessons were similar to those emerging from extension projects around the world at the time. The project made a conscious attempt to depart from the training and visit (T&V) approach adopted by the previous project.' SAEP introduced a participatory, farmer-centered methodology to make extension more demand-driven, and emphasized an integrated approach to extension service delivery, including the use of mass media, the use of groups rather than individual contact farmers, and private sector participation to make the system more cost-effective. This was in sharp contrast to the traditional top-down approach of using agricultural extension to deliver technical messages from research. The Implementation Completion Report (ICR) provides an accurate account of the implementation experience of SAEP and its outcomes. As the details are not repeated in this Performance'Audit Report (PAR), this report should be read in conjunction with the ICR. The discussion here highlights the key findings and discusses implementation and outcome details to the extent necessary to address the main issues. The OED Evaluation Summary for the project agreed with all but one ICR rating (the one on institutional development impact). Project ratings were thus not a major issue for this audit. The principal motivation was to understand the reasons that SAEP failed to achieve results. This is especially important since the innovative design of the project reflected many of the views that were current at the time of SAEP's preparation, and are also currently considered good practice. The main questions for this audit, therefore, are: Was there something inherently wrong with the project concept or the project design? If not, what lessons can be drawn from SAEP's experience to avoid similar outcomes in the future? 1. Thus, although it was "second" in a chronological sense, it was a completely new design and was made effective more than six years after the "first" project closed. 2 Box 1: Project Objectives, Components, and Design Objectives: To increase farmers' income and agricultural production in the non-plantation sector by improving farm productivity, inter alia, through promotion of innovative technology. Components: (a) A multifaceted program to strengthen extension services by (i) creating an integrated extension system with a smaller cadre of better-trained staff from the existing extension services spread over four departments in two ministries; (ii) establishing a cost-effective, farmer-centered farming system extension approach (a holistic approach using farmer reference groups and the problem census-problem solving methodology); (iii) improved use of mass media techniques, with an increasing role of the private sector in mass media production; (iv) strengthening plant protection and pesticide control divisions of the Department of Agriculture; (v) formal and in-service training for extension staff and (vi) providing technical assistance and support for piloting an innovative private sector extension program. (b) Development and implementation of a national seed policy to improve the production, import and distribution of seed and planting material by recognizing the appropriate roles of public and private sectors in the seed production, and promoting an environment conducive to private sector participation in seed production and distribution. Evaluation of Project Objectives and Design: The SAEP objectives were consistent with the Government of Sri Lanka (GOSL) objective of promoting agricultural diversification and the Bank's assistance strategy at the time of project preparation. However, GOSL did not have an articulated agricultural policy or a strategy for agricultural development. Nor did it have an agricultural extension policy to provide an overall framework within which the different approaches promoted by the project could be placed. While there was a need to re-establish a functioning extension service, which was all but disbanded after the first extension project, and there is a potential role for extension services to help farmers on certain agro-economic issues, the project design did not fully appreciate the critical constraints, many policy related, which inhibited agricultural growth and diversification. It also did not fully analyze the institutional issues and constraints to "integrating" disparate extension services operating in different departments and ministries. As the PAR analysis shows, these were among the main problems SAEP faced in achieving its main objective. 2. Main Findings The project failed to contribute towards its primary objective of increased farmer incomes or agricultural productivity, either through increased efficiency in the use of existing technologies or through the spread of new or innovative technologies. Accomplishments by specific components are discussed in detail in the ICR and are briefly summarized here. On the positive side: * A new seed policy was approved by the Cabinet of the Government of Sri Lanka (GOSL), although almost three years later than expected at appraisal. The policy was yet to be implemented at the time of the audit mission. * The project succeeded to some extent, albeit belatedly, in re-establishing the field-level presence of the Department of Agriculture (DOA) extension service by getting back qualified officers who had been transferred to administrative and other jobs after the first extension project ended in 1986. * The project achieved its targets, again with delays, for strengthening the capacities of the relevant agencies for plant protection and pesticide control. * The project strengthened the operational capacity of participating agencies for extension services by providing equipment, vehicles, training, and some civil works. 3 However, little progress was made on the central components of the project design: * The project failed to establish an integrated extension service. Different line agencies continue to operate in parallel. There is little desire on the part of the agencies to even collaborate, let alone integrate the provision of their services to farmers. They have their own priorities and work programs (including delivery of subsidized inputs) focused on supporting the expansion of specific activities. * In general, the extension services fell into disarray after the project closed. With the exception of some technical advice rendered by the officers of the Department of Export Crops (DEA) and the Coconut Cultivation Board as part of the package of inputs they deliver, general extension services are functioning very poorly. * The participatory farmer-centered farming system approach did not work as anticipated, and could not be institutionalized. The problem census/problem solving process (PC/PS), later modified to problem census/problem analysis/problem solving (PC/PA/PS), using farmer reference groups, generated initial enthusiasm among farmers and extension officers. However, with some exceptions, the process failed to fulfill farmers' expectations, as most problems identified could not be addressed by extension officers. Nor were extension officers able to influence other agencies to address the non-technical problems. * Research-extension linkages were not improved and, on the contrary, may have deteriorated over time (although not because of the project). As a result, there was neither any help from research in resolving farmers' problems, nor any feedback about farmer constraints to help in setting research priorities. * The project did not contribute to any significant increase in farm productivity or farmer incomes. There were either few innovative technologies to promote, or the adoption of available technologies was constrained by other factors. The Bank's appraisal report anticipated that the concurrent Agricultural Research Project would yield sufficient new technologies to be extended. This turned out not to be the case. However, the role of extension goes beyond simply a transfer of technology. In this regard, the extension service failed to help farmers improve the management of existing technologies, or to help them diversify into other crops, particularly higher-valued ones. * Although the DOA produced a variety of mass media materials (bulletins, leaflets, posters, and radio and television programs), their use was limited for lack of interest and by bureaucratic procedures required at the provincial levels to fund and distribute printed materials. The DOA has also been reluctant to use the private sector for production of communication materials. * The pilot private sector extension component was not undertaken. The government was not interested in pursuing this option, and the funds for technical assistance to design and support the pilot program were never used. After repeated appeals by Bank supervision missions, and warnings of remedial action for violating a dated covenant for this component, the cabinet approved a fee for service/cost sharing private sector extension program. The government also took the next step and advertised an invitation for private sector proposals, but no further action was taken. 4 3. Implementation Many aspects of SAEP's design and implementation affected project performance and raise important issues that contributed to its poor outcome. To put the issues in context, the implementation experience is briefly recapitulated. After becoming effective in February 1993, the project apparently proceeded relatively smoothly until the Midterm Review (MTR) in March 1996. Until that time, Bank supervision missions rated the project "satisfactory" in its progress towards achieving its development objectives as well as for implementation. The Bank was generally supportive of the government's actions, and endorsed the GOSL's plans to expand the program to attain full coverage by end of 1995 as a "sound strategy."2 The Bank noted the failure to comply with dated covenants, but until the MTR, these violations were not raised as major issues even though many of them were central to the project's innovative design and to achieving its objectives. The earlier supervision missions also failed to note, or at least report, the emergence of significant tensions between SAEP processes and the Integrated Participatory Process for Agricultural Development, popularly known as the AMA program, of the Ministry of Agriculture and Lands (MAL)? The AMA program incorporated the National Policy Framework of the new government (which came into power in 1994), and was conceived by the senior management of MAL at that time. The MAL also viewed AMA as a response to the problems and weaknesses it saw in the design of SAEP. These were primarily the lack of a proper institutional framework for project activities. Among these, the aspects subject to particular criticism were the exclusion of the Department of Agrarian Services representatives in the field extension teams, and the establishment of farmer reference groups outside of existing farmer organizations. It also considered the project too narrowly focused ("extension for extension's sake"), ignoring many key aspects such as marketing of output, post-harvest activities, and provision of inputs. AMA was intended to be a development program. The Bank considered AMA a politically driven, unsustainable, and inefficient program involving significant government intervention in the delivery of agricultural inputs and services. It also considered the program too centralized and too bureaucratic and contrary to the principles of SAEP. The statements and directives issued by MAL in support of AMA created a number of implementation problems for SAEP, creating competition for resources (AMA was not properly funded and competed with SAEP for staff time) and confusion among field staff regarding their roles and responsibilities. It also led to a duplication of many activities (e.g., participatory rural appraisal in addition to PC/PS). These factors added to the already poor commitment and ownership by participating agencies and provincial officials to project objectives. The participating agencies already viewed SAEP activities as "additional work" to their normal (subsidy delivery) work programs, while provincial departments of agriculture viewed SAEP as being externally imposed. The MTR was the first time the Bank conducted a proper analysis of the problems affecting project implementation. It rated SAEP a "problem" project, and the project never recovered. The MTR identified a list of actions that the GOSL needed to fulfill urgently to get the project back on track. The subsequent supervision mission (in October 1997) acknowledged progress in some regards, but continued to note problems in many key areas. It identified a list of 10 remedial actions the GOSL needed to take within six months to prevent the Bank from resorting to 2. Follow-up letter to the government from the Bank, dated May 31, 1995, following the April/May 1995 supervision mission. 3. The ministry unveiled the program in the districts of Anuradhapura and Matale (hence the name AMA, combining the initials of the two districts). In Sinhala, AMA connotes an 'immortal' activity. 5 remedies available to it under the Development Credit Agreement. The government responded - at least on paper by issuing statements or initiating orders at the central level - to many of the actions. In the meantime, based on the recommendations of a desk study of projects "at risk" by the Bank's Quality Assurance Group (QAG), the Bank proposed to the government to restructure the project, including scaling down the project to a few pilot areas, reducing the scope of the PC/PS process, and addressing the issue of the commitment of the central government and provincial governments to the project's objectives and principles. To understand the events that followed, it is important to note that the project was being managed from the Bank's Sector Unit at headquarters in Washington, while the Country Director's office was based in the Bank's Sri Lanka Country Office. Following an exchange of correspondence (between January and March 1997) with MAL, outlining a more streamlined plan of action and a proposal for restructuring the project, the Sector Unit sent a letter to the Country Office (dated March 21, 1997) to be delivered to GOSL. The letter proposed a mission to discuss and implement a restructuring along the lines suggested by QAG. The letter also acknowledged the actions the government had taken following the October 1996 supervision mission report. The Country Director, however, decided not to forward the letter to MAL, on the grounds that it did not reflect the reality on the ground and was misleadingly positive.' According to the Country Office, the project design was fundamentally flawed - it focused on processes of PC/PS, without linking these processes to the resolution of problems through provision of adequate resources as well as the mechanisms for delivering the required resources. Also, despite the reported progress by the implementing agency, there were no concrete results on the ground. The entire discussion of the fate of the project after March 1997 was between the Country Office and headquarters. The Bank did not attempt to negotiate with the government on concrete next steps; it did not even officially communicate with GOSL until discussions surrounding the closure of the project. There was some informal contact between Country Office staff and MAL officials, in the context of an internal study initiated by MAL on the problems afflicting SAEP.' Ironically, the Country Office also proposed restructuring the project. It is unfortunate that while both the Country Office and headquarters were talking about restructuring, they were in fact talking past one another. A subsequent memo (dated August 21, 1997) indicates that the "restructuring" that the Country Office had in mind was in fact an entirely new and unrelated project. It is apparent that the Country Office had already made the decision to close this "problem" project. The project was formally closed in June 1998. 4. Issues The implementation experience with SAEP raises some important issues, and points to some important lessons for the Bank, especially as it moves increasingly towards a more decentralized work environment, as well as towards more decentralized and participatory service delivery mechanisms, as in the case of agricultural extension services. There are particularly important lessons for Bank and borrower performance, and the nature and quality of their interaction. Project complexity. The first striking feature of SAEP is the complexity of its institutional arrangements. The project sought to integrate extension services spread over four departments in two different ministries. The departments of agriculture (DOA), animal production and health (DAPH) and export agriculture (DEA) were in the Ministry of Agriculture and Lands, while the 4. Office memorandum dated March 31, 1997, from Sri Lanka Country Office to headquarters Sector Unit. 5. The study was initiated after a change in the Secretary of MAL in 1997. 6 coconut extension services were with the Coconut Cultivation Board in the Ministry of Public Administration, Home Affairs, and Plantation Industries. In addition, the 1987 constitutional amendment on decentralization made agriculture and livestock extension services (i.e., for DOA and DAPH) the responsibility of Provincial Councils, while DEA continued to be a central department. The responsibility for inter-provincial areas continued with MAL. Coconut extension services also continued to be a central mandate handled by the Coconut Cultivation Board. DOA and DAPH extension services were administratively under the Provincial Councils, but technical support was to be provided by MAL. Finally, while extension services for agriculture and livestock have been devolved, research is still with the central government. These arrangements adversely affected the operational efficiency of the project. To assist in coordination and smooth implementation, the project established a Project Coordination Committee (with representatives from all departments, provincial councils, Ministry of Finance, etc.), a Project Management Unit (PMU) and later an Extension Development Unit (EDU). The performance of all three was less than satisfactory. The Project Coordination Committee met irregularly and was not very effective. The PMU's role was generally relegated to routine administrative matters and project management, with the MAL exercising substantial influence on project activities. The ineffectiveness of PMU and lack of coordination among different agencies led to the establishment of the EDU after the MTR, but the effectiveness of the EDU was also limited. Neither the PMU nor the EDU succeeded in establishing a proper monitoring and evaluation unit to track the project's progress. Integration. The project sought to develop an integrated extension service, in an effort to develop a cost-effective and holistic farming systems approach. Integrating the service delivery function of four line agencies, however, proved significantly more difficult than anticipated by the project design. Project appraisal failed to appreciate the institutional incentives and priorities of individual agencies that eventually prevented them from effectively integrating their extension services. While some cooperation among officers at the lowest level (through the mechanism of Field Extension Teams or FETs) was achieved, even coordination was limited in higher level mechanisms, including the Guide Extension Teams (at the district level), the Provincial Extension Teams (at the provincial levels), and at the national level. Each line agency has its own work program and priorities. Among these, information delivery and advice, which is generally considered to be a pure extension activity, is top priority only for DOA, while the main activity of other departments is to deliver inputs and supplies, usually with a significant subsidy element. For the officers of the other departments, thus, SAEP activities were generally regarded as additional work, and given their priorities, often as discretionary. This modus operandi was not conducive to the concept of SAEP and significantly affected implementation. On the practical (physical) aspects of integration, the largest number of extension staff are attached to the DOA, with one officer attached to one Agrarian Service Center (ASC). For other departments, one officer often covers more than one ASC and hence could not effectively participate in all the Field Extension Teams in his area of charge. This added to the time requirements for Field Extension Team activities of some agents more than others, making for an uneven implementation of the project activities through farmer reference groups. While these issues became progressively worse, at least initially there was enthusiasm with the new approach of interacting with farmers. At the Field Extension Team level, the increased cooperation led to some efficiency gains where previously different agencies were targeting the same households within their overlapping geographical areas. However, the services were never really integrated. More importantly, the senior management of individual agencies were not even interested in cooperation. Almost every official met during the audit mission, in every department, and at all levels (central, provincial, and district), as well as project management staff noted the lack 7 of commitment to SAEP concepts at the top management levels of the agencies. It is clear that there was no interest in integrating their services in any substantive manner. The cooperation was limited, as noted by many officials, to fulfilling project requirements, with the objective of accessing project resources (vehicles, equipment, and training). Incentives, accountability, and ownership. Integral to the issue of integration, and the commitment to project objectives at all levels, is the critical issue of incentives. The institutional and staff incentives were not conducive to the effective implementation of project concepts. There was no accountability of the service providers to the farmers. The line agencies inherited an entrenched top- down, supply-driven approach to extension and applied the same mind set to the new approach promoted in the project of PC/PA/PS, which was specifically meant to make the system more demand-driven. The project made provisions for piloting a private sector extension program, but this was resisted by the government, for various political reasons, and was never implemented. Institutionally, the project was the responsibility of MAL or the central government. However, extension agents are answerable to provincial councils, who have their own priorities. Among these, extension activities were apparently low on the priority list. Further, when non-technical constraints identified by the farmer reference groups in the problem census sessions were brought to the attention of the divisional committees and provincial councils, they received little or no response. Without explicit benefits to the concerned agencies, in the majority of cases their cooperation was not forthcoming. In general, while provincial officials appreciated the project concept and objectives, they viewed the project and its activities as externally imposed. This view was held as well by many provincial directors of agriculture and animal production and health, who consider extension activities their mandate and do not like taking directives from the central MAL. Many complained about the lack of participation in the design of the project, which perpetuated the feeling of external imposition and lack of ownership of the project's activities. Technology transfer, extension, and staff capacity. Prior to SAEP, the role of extension was widely perceived by extension staff, and senior policymakers, as the transfer of technology from research to farmers. In the case of coconut and export crop development, delivery of subsidized inputs to promote production, which also entails advice on production methods to meet established targets, was the primary task of extension officers. SAEP sought to change the perception of extension as a mechanistic delivery of technological messages to a more holistic demand-driven service to assist farmers in identifying and resolving problems at the farm level. While new technology and technical advice are integral to the extension function, farm productivity and incomes can be improved through improved farm management, even in the absence of new technology. Implicit in the project design was the confidence that the extension agents were equipped with the skills to mobilize farmers into "reference" groups, i.e., groups of farmers with relatively homogenous socio-economic and agro-ecological status. In practice, the extension service continued to be driven by a need to search for technical solutions to farmers' problems. The PC/PS process was applied mechanistically, with little attention to economic aspects of farm management. The farmer reference groups helped enumerate many problems, including many that could not be addressed by the extension agent. These were referred to other agencies, and in the few places where other agencies cooperated, farmer problems were addressed. In most cases, however, the problems remained unsolved, seriously reducing the credibility of extension agents and the PC/PS process. Until the MTR, little attention was paid to analyzing the problems identified by farmer reference groups. Extension staff continued to perceive their primary role as that of dissemination of technological messages and continued to seek technical fixes to identified problems. But, with few new technological messages coming from research, such advice soon became ineffective. After the 8 MTR, with additional technical assistance to review the PC/PS process, a consultant introduced the notion of problem analysis between the processes of PC and PS. The immediate and serious implications of making this seemingly obvious link an explicit process step is the need for significant training to improve the analytical skills of extension staff, including the senior staff responsible for supervising and backstopping the frontline staff In this sense, the lack of capacity of frontline staff appears to have been an important impediment to an effective implementation of SAEP. It is also an issue that goes back to the design of the project and the failure at appraisal to properly identify this important constraint (in addition to poor research-extension linkages and low productivity of research). Project preparation and design. Another problem with the design of the project, as well as with Bank supervision of the project, is highlighted by another recommendation of the consultant: the scope of the PC/PS process. With complex inter-agency relationships, lack of effective institutional mechanisms or incentives for inter-agency cooperation, the PC/PS process adopted an unrealistic approach to identify and solve all types of problems faced by farmers. It generated unrealistic expectations that could not be met. Local administrations had their own priorities and the project had no way to leverage results from non-participating agencies. The Bank accepted the recommendation to restrict the focus of the PC/PS process to issues that could be readily addressed by extension agents.' While the broad scope of the PC/PS approach adopted at appraisal indicates that the project anticipated that other factors are likely to be important, an implicit assumption appears to have been that the need for technical advice would emerge as a major constraint to enhancing agricultural productivity and, hence, incomes.' This was an important assumption in the design of the project, since its absence would have had significant implications for the skills of extension agents to help farmers address their problems. An analysis of all the constraints affecting farm production is no doubt important, but this step should have been undertaken through economic and sector work prior to the project, or during the preparation of the project to properly identify the role of extension and put it in perspective. The lack of appreciation of the situation on the ground or the priority given to extension services by farmers and local govemments proved to be an important factor in project performance. A subsequent Bank report identified key problems in the development and diversification of agriculture as being policies that biased incentives in favor of rice (even though rice production was not very efficient), labor market realities, and significant imperfections in land markets. The lack of optimism about the potential for agricultural research and extension to break the cycle of stagnation in agriculture was also noted by a MAL report on "Technology Transfer in the Non-Plantation Sector," in 1994. Bank performance. Several facets of the Bank's performance raise important issues. As noted above, during preparation the impact of agricultural policies on the effectiveness of extension services and institutional issues were not adequately analyzed. Beyond the setting of broad objectives, the government did not have a strategy for the agricultural sector, nor did it have a vision for the development of extension services. An important shortcoming in project preparation, particularly in light of the devolution of authority to provincial councils, was the lack of consultation and participation of provincial officials in the design of the project.! 6. Of course, since the Bank never got around to restructuring the project, this recommendation could never be applied. 7. This assumption is also evident from the statements about the availability of existing or soon to be made available innovative technologies from research. 8. While MAL maintains that consultations were held, provincial staff note that the workshops were not really consultations but more like project initiation workshops to familiarize staff with the project design and concepts. These workshops were reportedly held after the project had been formally accepted. 9 Until the MTR, the optimistic and positive assessment of the Bank's supervision missions was not warranted, and may have contributed to some strategic errors, as in the case of the rapid expansion of the project from a pilot phase in three districts to the national level. The MTR and later supervision missions identified this, among others, as an important factor affecting the performance of the project, although as noted above, the Bank had been supportive of such a move just before the MTR. Besides uneven quality, supervision missions also wavered in their views on the government's AMA program and its relationship to SAEP. The government considered the Bank too inflexible in sticking to the design outlined in the appraisal documents, and the Bank considered the government's program to have a significant adverse impact on project activities. Although these issues had a bearing on project performance, and were clearly disturbing, they were eventually identified and were being addressed. As in all institutional development projects, especially one promoting an innovative approach, several design and implementation issues are bound to come up. It is also well known that significant institutional change, and in particular the attitudinal change from a top-down to a demand-responsive mode of operation, takes time to resolve. An important move in this context was the need to be proactive in addressing the issues and the need to restructure the project in light of the emerging issues and developments. In this context, Bank performance was highly unsatisfactory, and raises an important process issue. As noted earlier, there was a significant difference of opinion on how to deal with the "problem" status of the project. The debate on the fate of the project remained between the Country Office and headquarters, without engaging the borrower. There was no communication with the borrower for almost six months, by which time the Bank had essentially reached the conclusion, unilaterally, to close the project.' The primary argument of the Country Office was that the project was fundamentally flawed in its design. However, the restructuring proposed by headquarters was intended to address this very issue, that is, the scope of the PC/PS process. It should also be noted that the region's project advisor also recommended restructuring the project, as did QAG. Whether the decision to "cut the losses" was appropriate will never be known. It is clear that as implemented the project was not achieving its development objectives. It is also possible that for various reasons, including continued lack of commitment by the borrowing agencies, a restructuring would also not have produced any better results. However, considering that the project was two- thirds into its implementation period, with significant investment already made, the Bank should have at least engaged the government in a serious discussion with clear-cut options on the future course of action. This is important since the government had committed itself to implementing the recommendations of the Bank's supervision missions. What is particularly troubling is the Bank's internal process in which the decision on the fate of the project was made without any attempt to negotiate options with the borrower, and ignoring expert technical opinion from within the Bank. Borrower performance. Except at the very initial stages, the performance of participating agencies was unsatisfactory. With the exception of procurement of equipment and vehicles, and to some extent training, most components were implemented with considerable delays. The issues of lack of commitment and ownership were among the most critical factors affecting project implementation. At the level of the central government, the change in government in 1994, and the adoption of the AMA program was a major factor affecting the implementation of SAEP. The AMA program was pursued by the government, often in direct competition with and at the expense of SAEP. At the provincial and individual agency levels, commitment to the project objectives and processes were a 9. As noted by the Sri Lanka Country Department, in its comments on an earlier draft of this PAR, the formal decision to close the project was mutually agreed between the Government and the Bank. While this is true, since decisions to close any project have to be formally mutually agreed upon, the issue here is that the Bank had already decided not to continue with the project by the time it met with the government in October, 1997. Nor did it present any concrete proposals to salvage the operation. 10 major problem. These problems reflect a more fundamental problem, that is a lack of a vision and a strategy for the development of extension services, and indeed for the development of the agricultural sector as a whole. 5. Principal Ratings The audit confirms the ICR and ES ratings for outcome as unsatisfactory, sustainability as unlikely, and borrower performance as unsatisfactory. The audit's findings also support the ES rating for institutional development as negligible, which differs from theICR's assessment as partial. The audit disagrees with both the ICR and the ES in the rating for Bank performance, and rates it unsatisfactory. The current state of extension services, which is the same as in the pre-project situation, and the lack of institutionalization of either of the two key institutional features promoted by the project, the integrated extension approach or the PC/PA/PS process, indicates negligible institutional development. With respect to Bank performance, shortcomings at the time of project preparation, variable quality of supervision, and most importantly the process by which the Bank decided to close the project lead the audit to assess Bank performance as unsatisfactory. 6. Lessons Analytical foundations ofproject design: An important lesson emerging from this project is the need for a proper assessment, either through economic and sector work or in-depth project preparation, of (a) the policy and other constraints to agricultural development; (b) the institutions and institutional framework for project implementation, including monitoring systems, staff capacity, and coordination mechanisms. Participation and ownership: The lack of commitment of the various agencies suggests a much greater need for the participation of the agencies in project design and formal agreements among agencies to pursue a common objective. The importance of borrower ownership has been known for some time, but the experience of SAEP, implemented in a decentralized setting, demonstrates the importance of getting not only the central ministries, with which the Bank has traditionally had contacts, on board, but also to nurture ownership among the decentralized implementing agencies. Incentives and accountability: Another old lesson is strongly reiterated by SAEP design. Even with an innovative, participatory, and farmer-centered methodological approach, to make the service delivery system truly demand-driven, it is critical to have incentive structures that hold service providers accountable to their clients, the farmers. The traditional bureaucratic setup, where performance is not linked to rewards or sanctions and accountability is primarily to superiors, will not yield an effective demand-driven system. Bank process: As the Bank becomes increasingly decentralized, accountability and decision- making processes will become increasingly important. The Bank needs to resolve where the accountability for project outcomes will lie (with the task manager or country director). This has implications for decision making on important issues such as pre-mature closing of projects, and restructuring. As a development partner, the Bank also needs to be particularly aware of substantively involving the borrower in important decisions affecting investments, and to directly engage the borrower in negotiating options such as restructuring and closure. 11 Annex A Annex A: Basic Data Sheets Key Project Data (amounts in US$ million) Project Component Appraisal Estimate Actualor Current Actual as Percentage of PrOeCtEstimate Appraisal Estimate Total Project Costs 18.32 10.05 54.9 Loan Amount 14.34 8.02 55.9 Cofinancing - - - Cancellation 6.0 Date physical components Feb. 1999 June, 1998 completed Economic Rate of Return - Cumulative Estimated and Actual Disbursements (in US$ thousands) FY93 FY94 FY95 FY96 FY97 FY98 FY99 FY00 Appraisal Estimate 0.4 1.4 3.7 6.0 8.3 10.6 12.9 14.3 Revised Estimate' Actual 0.60 1.36 2.34 3.68 5.71 7.99 8.02 - Actual as % of Estimate 150 97 63 61 69 75 62 - Date of final disbursement September 30, 1998 1/ US$6.0 million was cancelled in May 1998 Project Dates Steps in Project Cycle Planned Actual Identification June 1989 January 1989 Preparation November 1990 February 1991 Appraisal June 1991 July 1991 Negotiations April 1992 April 1992 Board Approval June 1992 August 1992 Signing Effectiveness September 1992 February 1993 Midterm Review May 1995 March 1996 Project Completion February 1999 June 1998 Loan Closing June 30, 1999 June 30, 1998 12 Annex A Related IDA Credits Loan/Credit Title Purpose Year of Approval Status Preceeding Operations: 1. Agricultural Extension and Develop smaliholder agriculture 1982 PCR issued in Adaptive research project sector by improving and April 1992 (Cr. 931-CE) strengthening physical and institutional capabilities the research and extension system. 2. Agriculture Research project (Cr. Improve and strengthen the 1987 PCR issued in 1776-CE) Agricultural Research System June 1997 3. Following Operations: None Staff Inputs (Staff Weeks) Stage of Project Cycle Planned1' Revised ' Actuall/2' Weeks US$ (000) Weeks US$(000) Weeks US$(000) Preparation to Appraisal N/A N/A N/A N/A 59.8 157.6 Appraisal N/A N/A N/A N/A 67.8 207.6 Negotiations through Board Approval N/A N/A N/A N/A 8.6 29.4 Supervision N/A N/A N/A N/A 94.2 241.1 ICR Completion 3/ N/A N/A N/A N/A 14.0 3.9 Total N/A N/A N/A N/A 244.4 639.6 1/ Estimates for original and revised staff weeks and dollar budgeting introduced only in FY95 and therefore total original and revised staff weeks and costs cannot be computed. 2/ Data from World Bank MIS 3/ Actual cost relates only to staff in Colombo office. Four weeks contributed by FAO/CP 13 Annex A IDA Resources: Missions Performance Rating by Stage of Project Cycle Month/Year No. of Persons laye in S ee Sa/ Pemn ation D o p bm Field Skills Represented a! Implementation Development Problems d/ Status Objectives FAO Initial Preparation Assistance Jun/Jul. 1989 1 9 Ag. Fact Finding Mission September 1989 2 12 Ag. (2) Initial Preparation Sep/Oct 1989 5 25 M '. (2), Se 1)5 Preparation Sept/Oct 1990 6 30 Ag. (5), Mk (1) Preappraisal March 1991 5 20 Ag (4) Ec. (1) Appraisal Jun/July 1991 7 19 Ag (3), Ec (), Se (2), Fe (1) ________ ______ Supervision 1 Oct/Nov. 1992 3 11 Ag. (2), Pr. (1), 1 1 - Supervision 2 Mar/Apr. 1993 1 17 Ag. 1 1 - Supervision 3 Oct/Nov. 1993 2 23 Ag. (2) 2 1 - Supervision 4 Mar/April 1994 1 17 Ag. 2 1 - Supervision 5 Oct/Nov. 1994 2 14 Ag. (2) S S - Supervision 6 Apr/May 1995 1 16 Ag. S S - Supervision 7 March 1996 3 2 Ag., Ec., Gn. U U - Supervision 8 Oct. 1996 1 17 Ag. U U M ICR March 1998 3 8 Ag. (2) En (1) U U M a/ Staff specialization: Ag. - Agriculturist; EC - Economist; FA - Financial Analyst;En - Engineer; Gn - Gender Specialist; Mk - Marketing; Pr. - Procurement Specialist; Tr - Training Specialist; b/ Performance ratings based on IBRD and IDA - implementation summary Form 590 (before FY94) = 1 - Problem free; 2 - Moderate; 3 - Major problems, Implementation/development impact status (from FY94) = HS - Highly Satisfactory; S - Satisfactory; U - Unsatisfactory; HU - Highly Unsatisfactory c/ Types of problems: T - Technical; M - Managerial; and F - Financial d/ Project Identification e/ Information not found in project files f/ Mid-term review  Annex B 15 Borrower Comments Qmeu} 329623 sod*4emo Telephone ] Gwarp &-W.I My No. J GrQome FINMIN-CE 21409 6ea q Telex &m5 . Your o Z.xd 1443063 d ~@ C )E C JtC .mtc Fax 3 oo C@aiSet qtea5 @qd! w sageealm ge3mm*0 (MeGM *0). *m 1. a0w SQomwar. (l460 eoff4' The Secretariat (Ist Floor) DEPARTMENT OF NATIONAL PLANNING Colombo 01, Ministry of Finance and Planning 19 .................................. June 30, 2000 Mr. Ridley Nelson, Acting Manager, Scetor and Thematic Evaluations Group, The World Bank, U.S.A. Dear Mr. Nelson, Re: Sri Lanka- Second Agricultural Extension Project (Cr.2380 CE) Draft Performance Audit Renort The following comments are made on the report- . The concept of the new extension methodology on which the project was based, had not been proven anywhere in the world at the time of its introduction to Sri Lanka. Therefore the implementation of the project at the field level faced severe problems. The appropriateness or otherwise of the concept for societies where the beneficiaries come from diverse social and economic environments had not been taken into account when designing the project. * This extension methodology, seems to be more relevant to commercial agriculture. In countries like Sri Lanka, where the vast majority of farmers are small farmers, and where the profitability of farming is, comparatively speaking, very marginal, and where the transition to commercial agriculture is still at an infant stage, it cannot be implemented successfully. * During the project implementation, the difficulties faced by the management were repeatedly pointed out. One of the reasons for this was the absence of enthusiasm on the part of participating agencies because of their involvement in their widely differing extension delivery systems, each of which is relevent to the field it covers. Tlhis diversity among the participating agencies and the reasons for this should have been considered at the preparatory stage of the project, at which point, a suitable mechanism to meet the situation should have been evolved. Yours sincerely, Dr. P.Ali.ma Director General Annex C 16 OED Response to Borrower's Comments The comments raise two issues: the first two bullets are concerned with the extension methodology adopted by SAEP and the third bullet notes the lack of enthusiasm on the part of the participating agencies, which should have been investigated more thoroughly during project preparation. On the issue of methodology, the audit notes that the approach adopted was innovative. It was developed in the light of past experiences with agricultural extension services in Sri Lanka and in other countries. The central focus of the PC/PS methodology is to make the extension system farmer centered and make the service more demand driven. As such, the methodology is neutral to the nature of agriculture or the scale of farming. Almost every official met during the audit mission, from all agencies, agreed that the concept and approach were good and quite relevant in the Sri Lankan circumstances. They also repeatedly noted that the issues facing SAEP were more institutional, particularly with respect to the inter-agency cooperation and commitment to project objectives. These issues are discussed in detail in the audit. The audit agrees with the second issue, that greater attention should have been paid during project preparation and appraisal to the institutional and coordination issues, and to the respective missions and objectives of participating agencies. However, it must also be noted that the agencies had accepted the project as it was designed and hence had agreed to the basic project concept and objectives. Thus, while the project preparation can be faulted for inadequate stakeholder analysis, the individual agencies must also bear the responsibility for the lack of commitment to the project after agreeing to participate in the project. Also it must be noted that, albeit very limited, the project was successfully implemented in some areas in the initial years before many of the implementation problems emerged.