Malawi - Agricultural Extension and Planning Support Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 16512 PERFORMANCE AUDIT REPORT MALAWI AGRICULTURAL EXTENSION AND PLANNING SUPPORT PROJECT (CREDIT 1626-MAI) April 23, 1997 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents (annual averages) Currency Unit = Kwacha (MK) 1985 US$1.00 1.80 (appraisal) 1986-92 US$1.00 2.53 (average) 1993 US$1.00 4.39 (completion) Abbreviations and Acronyms ADD Agricultural Development Districts ASP Agricultural Services Project BES Block Extension System DOA Department of Agriculture EPA Extension Planning Area EP&D Department of Economic Planning and Development ERR Economic Rate of Return FA Field Assistant FAO Food and Agriculture Organization of the United Nations FRR Financial Rate of Return FAO/CP Cooperative Program of the Food and Agriculture Organization GOM Government of Malawi GTZ Deutsche Gesellschaftfuer Technische Zusanrnenarbeit IFAD International Fund for Agricultural Development IGP Income Generating Project IRDP Integrated Rural Development Project MAEPS Malawi Agricultural Extension and Planning Support Project MARE Malawi Agricultural Research and Extension Project (USAID) M&E Monitoring and Evaluation MOA Ministry of Agriculture MOALD Ministry of Agriculture and Livestock Development NARP National Agricultural Research Project NRDP National Rural Development Project OED Operations Evaluation Department OFD On Farm Demonstration OFT On Farm Trial PAR Performance Audit Report PCP Pilot Communication Program PCR Project Completion Report RDP Rural Development Project Area SAR Staff Appraisal Report SDR Special Drawing Rights SFSP Smallholder Food Security Project (IFAD) SIG Special Interest Group T&V Training and Visit (Extension) System TA Technical Assistance UNDP United Nations Development Programme USAID United States Agency for International Development ZOPP Zielorientierte Projektplanung Fiscal Year: April 1 to March 31 Weights and Measures: Metric System Director-General, Operations Evaluation: Mr. Robert Picciotto Acting Director, Operations Evaluation: Mr. Roger Slade Division Chief: Mr. Roger Slade Task Manager: Mr. E. B. Rice FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. Office of the Director-General Operations Evaluation April 23, 1997 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT Subject: Performance Audit Report on Malawi Agricultural Extension and Planning Support Project (Credit 1626-MAI) Attached is the Performance Audit Report on Malawi, Agricultural Extension and Planning Support Project (MAEPS), prepared by the Operations Evaluation Department (OED). MAEPS was supported by a credit for US$11.6 million equivalent approved in FY86. The credit was closed in FY93, after three extensions of the closing date, and an undisbursed balance of US$0.5 million was canceled. The basic objective of MAEPS was to improve the Ministry of Agriculture's (MOA) capacity to deliver services effectively and efficiently and to enhance the sustainability of its programs. Specifically, the project was to (i) strengthen MOA's analytical and long-term planning capability (15 percent of project costs); (ii) strengthen the national extension system, including the establishment of mechanisms for better defining extension strategies, for rationalizing existing institutional structures, and for linking the extension planning and budgeting processes (85 percent); and (iii) develop the human resources needed to deliver an efficient extension program (included in the other two). Within the extension component funds were included to support improvements in the performance of services in the Mzuzu Agricultural Development District (ADD), one of the largest of the eight ADDs in Malawi. When preparation began, the Mzuzu area development program was the primary focus. By appraisal, the emphasis had shifted to resource-allocation issues and capacity development at the ministry level. Importance was also given to limiting recurrent costs to levels consistent with government's own resource base. Activity in the Mzuzu ADD was redefined as a pilot operation to introduce improved planning and budgeting systems developed at the center. The technical assistance teams assigned to MOA's planning and agricultural divisions arrived late and had little impact. Most of the work carried out by these Bank-financed resident experts was below expectations, and they often found themselves in advisory roles with ministry officers who either did not know how to manage them or did not want their services. High rates of turnover of senior staff undermined weak borrower ownership of the capacity-building objectives. Initial progress at Mzuzu ADD was much better, and that is where Bank supervision concentrated its attention. The principal activity was a pilot extension operation intended to upgrade the effectiveness of a rudimentary training and visit (T&V) extension system that had been established at this ADD without Bank support at the beginning of the 1980s. The appraisal team had not anticipated a T&V pilot; the appraised pilot was appropriated by Bank supervision for this new and dynamic program. The leading feature was the on- farm demonstration (OFD), which was to demonstrate new technology to farmers in their own fields. The pilot and its OFDs were quickly extended to other ADDs, but faced mixed enthusiasm from ADD and MOA managements. By the early 1990s, however, financial and other problems emerged that limited This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 the spreading and deepening of the OFD program, coupled with evidence that its farm-level impacts had been overstated. The outcome in terms of capacity building at the center was insignificant. Apart from weak performances by most of the experts, the majority of the economists sent abroad for MSc degrees were transferred out of MOA soon after their return, and there are doubts also about the effective use of the training provided to staff in the agricultural professions. The extension pilot at Mzuzu was an important initiative that enhanced the effectiveness of that ADD's T&V system. But government is unable to finance similarly intensive operations throughout the country and is dependent on declining support from the Bank and other donors. The impact of the T&V program has also been reduced by the eroding profitability of the hybrid-maize/fertilizer package-the leading technology of the OFD program. The ADDs, using the T&V protocol, have found it hard to adjust their kit of packages to low-cost alternatives and poor clients. In the last few years, and propelled partly by the Bank's follow-on Agricultural Services Project, MOA has reached out to research, NGOs, traditional church groups, and other formal and informal associations to redesign the extension program. These exciting initiatives have given new life to the ADD T&V extension model. The audit rates project outcome as unsatisfactory, a rating determined by the failure to achieve any measurable capacity building at the center and the disabling of an otherwise promising pilot T&V program in the field. Institutional development itself is rated moderate, in recognition of the positive impact of the pilot on the earlier T&V model. The audit rates Bank performance as unsatisfactory, due to serious lapses at appraisal in putting in place an effective technical assistance program and in underestimating the inevitable erosion of government ownership. The "goal displacement" that occurred during supervision-substituting an action-oriented field program for capacity-building programs at headquarters-invites a mixed reaction. Supervision can be applauded for its opportunism, but it helped to establish what has proved to be an unsustainable extension operation. The outcome rating of unsatisfactory replaces the satisfactory rating set by OED in the Evaluation Memorandum accompanying the Project Completion Report. The downgrading is warranted because of the poor performance of all the headquarters' components. If capacity building is taken seriously ex ante, it must be taken seriously ex post. Partial progress on physical investments is not a substitute. The audit accepts the Evaluation Memorandum's rating for sustainability as uncertain. Lessons that are drawn from this project are: (i) the Bank has to arm itself with an improved set of capacity-building strategies; (ii) unsatisfactory ratings of capacity-building projects can be defended, despite progress of parallel investment operations in the field; (iii) extension services are necessarily expensive: provided they are cost-effective, operational expenses are worth supporting; and (iv) when properly managed, the T&V system of extension can serve exciting new campaigns such as those set in motion in Malawi in the last few years. T&V extension is ineffective when it is treated as the target, not the tool. Attachment FOR OFFICIAL USE ONLY Contents Principal Ratings and Key Staff Responsible for Project......... ................ 3 Preface...... ..................................................... 5 1. Introduction..................................................... 7 2. Project Objectives And Design A. Objectives ..................................................... 9 B. Design....................................................... 10 3. Implementation A. MOA Headquarters: Capacity Building. ....................... ............ 13 B. Mzuzu ADD: the Extension Pilot Operation ......................... 16 4. Outcomes A. MOA Headquarters .............................................. 23 B. Mzuzu ADD ................................................... 25 C. Ratings...................................................... 28 5. Findings and Lessons A. Findings...................................................... 30 B. Lessons ...................................................... 32 Annexes A. Basic Data Sheet ................................... ...... ....... 35 B. Other Components ..................................................... 37 Map Agriculture Extension and Planning Support Project (IBRD 18833) This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  3 Principal Ratings Principal Ratings Outcome Unsatisfactory Sustainability Uncertain Institutional Development Modest Bank Performance Unsatisfactory Borrower Performance Unsatisfactory Key Staff Responsible for Project Director Division Chief Task Manager Appraisal H. Wyss E. Senner N.C. Hamann Implementation H. Wyss A.H. Uhlig N.C. Hamann S. Sandstrom R. Ali R. Clough S. Denning C. Helman G. Stern Completion S. Denning C. Helman R. Anson PCR Prepared by: FAO/CP (R. Suppa, G. Stem)  5 Preface This is a Performance Audit Report (PAR) on the Agricultural Extension and Planning Support Project, Malawi, involving a Bank Credit in the amount of SDR 11.7 million (US$11.6 million equivalent). The credit was approved on December 20, 1985. It was closed on June 30, 1993 after three extensions of the Closing Date. The undisbursed balance of SDR 510,000 was canceled. The United States Agency for International Development (USAID) supported the project with parallel finance, disbursing approximately US$5.6 million. The PAR is based on the Project Completion Report (PCR) prepared by the Africa Regional Office and submitted to the Board on October 25, 1994, Government's comments included in the PCR, USAID's final evaluation report, the Staff Appraisal Report, the President's Report, the Credit documents, a study of project files, and discussion with Bank staff. An Operations Evaluation Department (OED) mission visited Malawi in July 1996 and discussed the effectiveness of the Bank's assistance and project execution with the Ministry of Agriculture and Livestock Devzlopment (MOALD) and other relevant agencies. MOALD's kind cooperation and valuable assistance in the preparation of this report is gratefully acknowledged. The PCR provides a good account of the operation of the project and the performance of the Bank and project executing authorities. This is one of four projects in the Africa Region, audited as a cluster by OED, which at appraisal gave priority to capacity building in the central ministry. This report pays special attention to that issue. Following standard OED procedures, copies of the draft were sent to the Government for comment. None were received.  7 1. Introduction 1.1 The genesis of the Malawi Agricultural Extension and Planning Support Project (MAEPS) was an agreement between government and the Bank in early 1983 to prepare the fifth in a series of projects making up the National Rural Development Program (NRDP). NRDP had started in 1978. It was the second phase of a group of integrated rural development "area" projects (IRDP) which began in 1965 in the smallholder maize belt west of the city of Lilongwe. The IRDPs in Malawi were the first of their kind in Africa and the models for subsequent Bank and other donor support to integrated rural development in that and other regions. By the mid- 1970s it had become apparent that the integrated area-based projects needed to be redesigned if they were to be extended throughout Malawi, and larger numbers of smallholders reached more quickly and effectively. NRDP continued most elements of the basic design of IRDP but at less cost-by reducing the intensity of services and infrastructure. NRDP V as originally conceived conformed to the revised design and was to cover the Mzuzu Agricultural Development Division (ADD)' and parts of Lilongwe ADD. There were no other components. As preparation and appraisal progressed the area development components receded in importance in favor of central capacity-building components, only to return to dominate project implementation as the newer components stalled. 1.2 A related element during preparation, which helped set the stage for the realignment of project priorities during implementation, was the recent introduction of the Training and Visit extension system (T&V) in Mzuzu ADD. The program manager of this ADD had visited Sri Lanka in 1980 to witness the results of T&V. He returned impressed, and in 1981 established in his ADD a modified form of the system, called the Block Extension System (BES). His initiative introduced T&V to Malawi. The Mzuzu system was home-grown; foreign consultants were not at that time involved. In the system, the field extension agent's work area and schedule conformed roughly to the standard T&V design. Each "field assistant" (FA) covered eight clusters ("blocks") of communities, and visited each block once in two weeks. Training sessions for the FAs were also conducted fortnightly. The primary client of BES was the Block Group, an informal association open to all farmers in these clusters but in fact dominated by the members of the ADD credit clubs. The agents carried out their demonstrations on Block Gardens, in theory one per block in a field set aside by mutual agreement of the farmers. The Mzuzu system differed in that respect from the standard T&V model, which identified individual "contact farmers" who would attract neighbors to demonstrations on the contact farmers' own fields. Most of the Mzuzu agent's time, however, continued to be spent in assisting credit delivery and recovery from individual club members, as it was in the rest of Malawi. The Mzuzu T&V system had gradually taken root in that ADD. There was some outreach to other ADDs, though it was not yet accepted by the Ministry of Agriculture (MOA)2 as a national program. 1. For the implementation of NRDP, the Ministry of Agriculture divided the country into eight ADDs. The ADDs were subdivided into Rural Development Project Areas (RDPs) and these in turn were subdivided into Extension Planning Areas (EPAs), the EPAs into Sections, and the Sections into Blocks. A field agent was responsible for a Section and its 8 Blocks. 2. Throughout the project period the ministry was called the Ministry of Agriculture. In 1994 it was renamed the Ministry of Agriculture and Livestock Development (MOALD). MOA and "ministry" are used in this report. 8 1.3 A third important part of the background is Malawi's population density, one of the highest in Sub-Saharan Africa. It is a consequence not of exceptionally fertile soils but of this strip of land's unique role--during and since the colonial era-as an island of relative calm in a region traditionally inflamed by slavery and war, a condition which still attracts refugees. Increasing density has forced smallholders to abandon the fallow, resulting in continuous cropping and the exhaustion of once-reasonably productive soils. The situation is aggravated by Malawi's single, short and erratic rainy season. In fact, this is the only place on the continent where a mono-modal rainfall pattern combines with a population density of this level. The land cannot support the inhabitants. Malawi ranks as one of the poorest nations on earth defined by per capita incomes, with high indices of disease, malnutrition and infant mortality. Most households produce less food than they require for subsistence, and their cash incomes are too low to maintain diets at minimum standards. Frequent droughts have a devastating impact on these people, reckoned to comprise at least 60 percent of the population.3 MAEPS was implemented in a period when the government and the Bank began finally to focus on these stark realities. The emphasis in the last years of the audited project, and more importantly during the follow-on project, has been to reach out to this impoverished sector of the rural population with technologies it could afford.4 3. The Staff Appraisal Report for the follow-on Agricultural Services Project, issued in May 1993, says 75 percent. 4. OED's audit report on the Malawi Agricultural Sector Adjustment Credit (ASAC, Cr. 2121 -MAI) discusses the condition of poverty of the majority of Malawi's farmers and the steps agreed with government under the umbrella of ASAC to enhance the smallholders' incomes. The report describes the discriminatory and oppressive price and marketing policies-the primary targets of ASAC-that combined with the geographic features to perpetuate that poverty (Report No. 15641, May 23, 1996). 9 2. Project Objectives And Design A. Objectives 2.1 The area-based NRDP V project was "identified" in January 1983. During the year, however, the Bank shifted its attention from area specific operations to reforming MOA. Bank staff argued that because resources were exceptionally scarce, even for the Africa region, it was more important to establish national priorities for further expenditures in the agricultural sector, and to improve the overall planning and budgeting processes to effect economies and eliminate wasteful expenditures from the ministry's portfolio. Resource allocation became the key issue. That meant upgrading the ministry's capability to identify and finance its priority objectives. The Bank incorporated into NRDP V an MOA proposal for direct support to the extension service as a vehicle for process reforms, adding elements of technical assistance to the project that had not been included before. The Bank and the ministry then added another component-also featuring institution building and technical assistance-supporting the ministry's Planning Division. The two new components displaced the Mzuzu area operation as the primary business of NRDP V. At that point NRDP V was renamed MAEPS, to reflect the emphasis on core extension and planning capacity components. 2.2 MAEPS was one of three projects which the Bank and the United States Agency for International Development (USAID) developed in tandem as a package aimed at relieving key institutional constraints. Nine months before approving MAEPS in September 1985, the Board of the Bank approved the National Agricultural Research Project (NARP, Cr. 1549-MAI). USAID paralleled both Bank operations with the Malawi Agricultural Research and Extension Project (MARE), also authorized in 1985. One of USAID's objectives in combining its support for the two services in one operation was to help ensure that research and extension were integrated in practice.5 2.3 The basic objectives of MAEPS, as submitted for Board approval, were defined at the national level. The overriding goal was to improve MOA's capacity to deliver services effectively and efficiently, so as to enhance the sustainability of its programs. MAEPS was to carry the NRDP emphases on affordability one step further, remodeling the ministry's whole operating system. Specific objectives were to: (1) strengthen MOA's analytical and long-term planning capability; (2) strengthen the national extension system, including the establishment of a mechanism for better defining extension strategies, priorities and programs and rationalizing existing structures; and (3) develop the human resources needed to deliver an efficient extension program. 2.4 The project redefined and downgraded Mzuzu's role. It stripped away some of the elements of the revised NRDP model, in particular the rural road and water supply components. 5. This cluster of three projects-two by the Bank and one by USAID-was often referred to in subsequent Bank reports as NRDP V. Although the cluster was presented as a consistent package, some inconsistencies soon became apparent. In its comments to OED on the draft PAR, the Bank's Field Office points to "conflicts" between the Bank's projects (MAEPS and NARP) and USAID's MARE. In notes in particular that MARE sent out of the country for advanced training many of the able extension and research staff that were needed to operate MAEPS and NARP, overstretching the capacity of the ministry to manage all three of these "complex" projects. Memorandum from F. Mbuka to E.B. Rice dated March 26, 1997. 10 The area program would focus on the core agricultural activities. More important, it treated Mzuzu ADD as a testing ground for process reforms generated by the technical assistance at the ministry level. The Bank referred to the downsized Mzuzu component as a "pilot." The government called it a "first phase." The notion of converting Mzuzu to a pilot may even have saved its place in the project. The Bank's senior vice president for operations had questioned during his review of the project whether there were too many components and whether Mzuzu was even necessary, given the laudable emphasis on capacity building components at the center. The Bank's single-page internal information sheet "Status of Negotiations," issued just after negotiations had been substantially completed, mentioned the three primary objectives listed in para. 2.3, but said nothing about Mzuzu. 2.5 Another prominent feature of the discussion leading up to Board presentation was the emphasis, not only on economies in the ministry's budget at the national level, but on cost- reductions in the proposed area operations at Mzuzu. The appraisal team was complimented for its vigorous efforts not only to keep total project costs down but to reduce wherever possible the capital and recurrent expenses of the field extension program for that ADD. Again, the intention was to cut the NRDP package down to a level consistent with government's projected future fiscal capacity. Thus: "To ensure that the present Project's recurrent cost requirements are built into the recurrent budget, the Project provides for the Government to finance all Project incremental operating costs starting in Project year 1 and continuing throughout the Project implementation period" (SAR, para. 5.11). B. Design 2.6 The IDA Credit was approved in September 1985 for SDR 11.7 million (US$ 11.6 million). The Staff Appraisal Report (SAR) divided the project's components into two categories-support for MOA's Planning Division and support for the national agricultural extension system through MOA's Department of Agriculture (including Mzuzu). The SAR allocated fifteen percent of project costs to the planning category, and 85 percent to the extension category. 2.7 Support for the Planning Division was to include three long-term resident experts, short- term consultancies, degree and short term training either outside or inside the country, recurrent costs of six additional staff, and a few computers and other equipment. The expert group was to comprise a senior planning officer, a pricing policy analyst, and an advisor on computerized management information systems. The training was to include 6 overseas MSc degrees. The purpose of this package was to strengthen the Division so that it could formulate a comprehensive long-term development strategy for the agricultural sector and provide better technical advice on agricultural policies and investment priorities. Other donor-supported technical assistance programs had for many years provided expatriates and graduate training to MOA and its Planning Division, and that support would continue. In 1989, for example, there were still seven expatriates working in the Division. The Bank-supported technical assistance package was expected to make a significant difference over and above other donor support in reaching the project's specific objectives. USAID's MARE project did not participate in this planning category. 11 2.8 Support for the extension service was to include two long-term resident experts, a much larger training program than that assigned to the Planning Division, facilities for production of technical and training materials, and a package of goods and services, including credit, for Mzuzu. The experts were to be an extension management/planning advisor and a financial advisor. Together they would help establish within the Department of Agriculture (DOA)6 a program planning capacity and a budgeting system integrated with MOA overall planning operation. The work with DOA, the largest department in the ministry, was to reinforce the work on planning/budget reform within the Planning Division. To secure these process reforms, government agreed to modify certain civil service procedures specific to DOA, in particular to establish a professional career stream for subject matter specialists and a promotion system for extension professional and technical staff. Ten overseas MSc degrees were funded (six by USAID). 2.9 The redefined role of Mzuzu was described in the SAR as follows: "Since the Project would involve significant improvement in the present agricultural extension system with potentially far-reaching implications for ADDs, it would be reasonable to phase implementation to minimize abrupt changes and unnecessary disruptions in the current system. The Mzuzu ADD, as the second largest ADD and representative of most of the country's agricultural areas, has been selected as first phase of a national program to improve the extension service" (SAR, para. 3.17). The phasing would be rapid. An improved annual extension planning and budgeting system was to be implemented in Mzuzu ADD within a year of the effective date of the IDA Credit, and extended to the other ADDs shortly thereafter. 2.10 The SAR provides little detail on the Mzuzu package. The area component was expected to "support innovative approaches to agricultural extension and information transfer," as well as "provide credit to smallholder farmers in the Mzuzu area for agricultural inputs to encourage their adoption of improved technical recommendations." In another passage, the SAR says that "under the Project, the 'Block Extension System' would be improved." The cost tables allow for construction of extension agent houses, market depots, office space and other infrastructure, and for procurement of vehicles, equipment and furniture. Goods and services for the Mzuzu ADD would absorb 20 percent of total project costs, and short term credit for participating Mzuzu farmers another 15 percent, giving Mzuzu 35 percent of total costs.7 2.11 None of the USAID funds were specifically allocated to Mzuzu. But USAID planned to use Mzuzu to promote three of its prominent initiatives under MARE, in particular a pilot mass communication scheme (exclusively for Mzuzu), a women's extension program, and an "adaptive research team" approach to linking research and extension services.8 The other prominent USAID component was the massive DOA training program, under the control of the Principal Secretary's office. These USAID-financed components were incorporated in its MARE 6. In 1994 DOA was renamed the Department of Agricultural Extension and Training (DAET). DOA is used throughout this report. 7. Or 41 percent of DOA's share (para. 2.6) and around 50 percent of projected Credit disbursements. These percentages are derived from the SAR. The Credit Agreement allocated no funds specifically to the extension service. 8. GTZ is credited with having first introduced the ART approach, in one of its area-based projects. 12 project but eart of their costs were also included in the MAEPS project description and appraisal cost tables. The total long-term expatriate consultant input proposed by USAID and incorporated in MAEPS as well as MARE-for mass media, women's programs and training advisors-was exactly equal to the 180 person/months proposed by the Bank for its components.10 9. USAID eventually disbursed US$14.8 million for MARE. Of this, the PCR identifies US$5.6 million as a contribution to MAEPS. 10. The USAID figure excludes some unrelated long-term experts. 13 3. Implementation 3.1 This chapter covers the two principal project components-capacity building at ministry headquarters and the extension pilot operation. Other important components are discussed briefly in annex 2 (the Mzuzu ADD credit program, the Food Security and Nutrition Unit, research/extension linkages, women's programs, communications assistance, and M&E). A. MOA Headquarters: Capacity Building 3.2 All five technical assistance experts arrived in 1987, at least a year later than planned. They were recruited through a single British agency. The agency was one of those recruiting firms referred to-and employed by the Bank-as a "post box." The experts operated independently of one another, and without professional oversight from the agency. The senior planning position for the Planning Division was dropped. The arrival of an expert financed by the European Economic Commission on a grant basis and with similar terms of reference rendered the project position redundant. Instead, it was converted to an advisor to the Division's Monitoring and Evaluation Section. 3.3 The overall record of this expert group-performance and impact-was well below expectations. Their effectiveness was compromised partly by poor selection, partly by poor working conditions at ministry headquarters and an ill-advised dispersal to different MOA units, and partly by the ministry's reluctance or inability to put them and their recommendations to work. The first pricing advisor to the Planning Division left disillusioned before the end of the year. The financial advisor to DOA also proposed to leave early, claiming his services were not being used. The Bank thought so too. He stayed on however, but when he later asked for an extension of his contract the government and Bank declined. With the exception of the extension planning advisor none of the five stayed beyond their 30-month contracts. He was asked to stay another six months, until July 1990, to finish an extension strategy plan that was by then almost three years overdue. 3.4 Ministry officers interviewed during the audit gave uniformly good marks to only one of the five, and even that rating is slightly tarnished by comments in the Bank files that he also had trouble with some counterparts. Unlike the others, he was the only one whose defined job put him into a quasi-operational position, responsible for establishing computerized systems for the Planning Division to generate the overall ministry budget, the payroll and other applications. The others were initially treated as they were supposed to be, as advisors, but their influence was much less than anticipated in their terms of reference. One of the ministry officers interviewed at audit said that the typical pattern would be for an expert to be paid little attention in the prominent role for which he was hired, and then settle back into an operational job of lower import until his contract expired. He was referring to all highly paid advisors arriving with agendas for institutional reform, not just the MAEPS group. This description also fits the EEC planning advisor, since he concentrated on project planning and oversight of the EEC's own portfolio in Malawi, and did not perform the role anticipated in the SAR. 14 3.5 The government's comments on the PCR (included as Part II), describes the performance of the Bank-funded experts as "mixed." It recommends more careful selection of long-term experts, and faults Bank supervision for having let all of them11 complete their assignments.12 3.6 Other comments made by government officials during the audit show that the other side of this two-way engagement was also defective. Two of the experts rated poorly by their counterparts were rated by others outside the immediate office as potentially satisfactory performers who had been unfortunate to be given advisory roles with persons who either did not know how to manage them or did not want them.13 3.7 That last opinion, suggesting the Malawians felt at least some of the positions and advisors had been imposed on them by the Bank, was repeated too often to ignore. The critics added that the advisors were not needed, that their advice offered little value added, and that MOA had been in the IRDP/NRDP long enough to work out its own solutions to emerging problems. The advisors, in other words, were benefits they had to accept to get the benefits they wanted. This is not an uncommon complaint in the Bank's borrower countries. But the frequency of the remarks from MOA in Malawi took the OED audit mission by surprise.14 3.8 One explanation is the rapid turnover of senior staff. In almost all cases these were the opinions of officers who had not participated in preparing the project. Indeed the Principal Secretary, the Chief Planning Officer, and almost all of the other MOA officials who helped with preparation of the project left before the end of the first year of implementation, the first two even before the Credit became effective. There were five different occupants of the post of Chief Agricultural Officer, the senior extension position, during the period of the project. Replacements were unconvinced the experts were needed. They might have later changed their minds, but then they also were likely to be replaced. The replacements started "with a clean slate." This lack of continuity among managers undermined the integrity of project design. 3.9 By contrast, USAID's long-term experts were generally rated highly by their Malawian colleagues. All brought with them budgets to execute the programs they recommended.15 The Americans were managed by a consortium of US universities under contract to USAID. The contractor, backstopped by USAID's resident mission, provided close-in professional support and treated its experts as a team. Although the long-term impact of these three advisors was somewhat less impressive than their reputations (see, for example, paras. 3.10-3.12 and annex 2, para. 5), nevertheless they did deliver the short-term outputs expected. 11. The expert who left during his first year had signed only a one-year contract, intending to renew. 12. This remark by government in the PCR clearly reveals a lack of ownership of the consultant team. 13. Exactly the same problem surrounded the financial advisor to the ministry supported by the follow-on project to MAEPS (ASP). He was described by one Malawian member of the Bank's Field Office as the best financial expert he knew whose services "were never used." 14. However, it was not shared by all senior officers interviewed. One felt that with some exceptions the advisors had accomplished about all that could have been expected. He said it was wrong to allege they had been imposed on the ministry, in fact he saw the Bank as the most "understanding" of the donors. By contrast, the majority opinion was best captured by the following remark from another one of the principal actors: "We didn't need this post, and then you send us this chap who does poorly and gets paid 100 times what we do." 15. That applies to the principal training advisor and the mass communication and women's program advisors. 15 3.10 The training advisor and his ministry counterparts were responsible for planning and implementing the massive training program targeted in the USAID and Bank documents for all three projects, MAEPS, NARP and MARE. There were two weaknesses in the program which the USAID team was unable to address, and which limited their effectiveness. 3.11 First, the overseas degree training program for MOA's economists was self-defeating. MOA has been unable to retain the services of the majority of the economists it has sent abroad, under MAEPS or any other technical assistance program. MOA loses its graduates mostly to other ministries. Government economists are pooled in a common service, controlled by the Department of Economic Planning and Development (EP&D) in the Office of the President and Cabinet. EP&D has authority to transfer members of the service to other public agencies.16 Because of the large amount of donor aid that has been allocated to MOA, this ministry has sent by far the largest contingent of staff abroad. The economists return with a marketable degree. The other professionals-agriculturalists--do not attract such attention, and generally remain in MOA. The ministry complains with a mixture of pain and pride that it is the "nursery" for trained economists throughout the civil service. Of the twelve Planning Department economists sent abroad in the period 1990-93, onl three remain in the ministry. Of 11 agriculturalists sent abroad for higher degrees, 10 remain. 3.12 Second, government counterpart contributions were insufficient to complete the training program. The majority of funds for local training were committed at first to training of trainers. This apparently logical plan cost so much that the follow-up training of field staff had to be curtailed. MOA is still unable to exploit new initiatives, that require intensive training of field staff in all ADDs, after the pilots have been completed. 3.13 Bank supervision staff throughout implementation reported lack of progress toward the staffing and output targets established for the Planning Division. It attributed this variously to the turnover of senior staff, the loss of returning graduates, the consequent high vacancy rates, and the mediocre performance of most of the advisers. These weaknesses (presently there are no Bank-supported advisors) persist.'8 3.14 Bank supervision reports show less concern about the lack of progress toward planning and budgeting reforms in DOA, although occasional references indicate this component was also under close observation. The shortfall from expectations is equally disturbing. Neither the extension planning advisor nor the financial advisor contributed significantly to the primary objective. This was to improve and integrate the planning and budgeting systems: so that planners at headquarters and the ADDs could count on funds being budgeted and made available for priority activities, and the budget officers could concentrate limited resources where they were most needed. The SAR called for an improved extension planning and budgeting system to 16. In some other countnes-Kenya is a good example-the private sector takes them first, at substantial salary increases. In Malawi, the private sector is smaller and salary advances are normally available only to those staff who are promoted, usually through inter-ministerial transfers. 17. MAEPS actually financed 12 MSc economists and 22 MSc (and 1 PhD) agriculturalists, excluding 13 graduates financed by USAID (under the MAEPS umbrella). 18. The follow-on project (ASP) does not include advisors or overseas degree training for the Planning Division. 16 be established, and installed at Mzuzu, by 1987. The last supervision reports, in 1992, said the job had still not been done.'9 3.15 The extension planning advisor faced one unanticipated hurdle which partly explains lack of progress toward the extension objectives at MOA headquarters. Almost as soon as he took up his assignment in late 1987, Bank staff began to reshape the T&V system at Mzuzu, thereby establishing a new extension operating model for the rest of the country without his assistance and in the absence of guidelines from MOA headquarters. He is alleged to have complained later that the Bank "overpowered" him. This story is picked up again in the next section. 3.16 The improvements to the MOA schemes of service never took place. Government had agreed to them at a time when a comprehensive civil service reform seemed likely. It did not take place, and government refused to act on one ministry alone.20 B. Mzuzu ADD: the Extension Pilot Operation 3.17 Genesis. The Mzuzu experience under MAEPS is a remarkable story of an unanticipated initiative, barely visible in the appraised design, suddenly overwhelming and recasting the character and image of the entire project. The event was the development by Bank supervisory staff in 1987/88 of a new T&V extension pilot program-unique to Mzuzu ADD, initially resisted by ADD management, and never blessed with full MOA support. The Bank's intent was to reinforce and build outward from BES and the Block Garden model, by moving extension closer to the farmers' fields. The adjustment was conceptually simple. Specifically, the proposal was that each FA would establish on-farm demonstrations (OFD) on five volunteer "contact" farmers' fields in each block, or 40 per agent.21 This had nothing to do with the pilot concept, established by the Bank at appraisal two years before, to use Mzuzu to introduce extension reforms generated by the technical assistance programs at MOA headquarters.22 The word "pilot" was reinterpreted by the Bank's supervisory staff. Early success with the pilot operation at Mzuzu, and the spread of the model to other ADDs, became the focal point of supervision 19. The PCR, prepared in 1994, refers to an improved planning system which had recently been established in all ADDs but whose impact was as yet "uncertain." This system is based on the logical framework, and had been under development for the ADDs since 1988. Its origins in MOA can be traced to German and American technical assistance programs. The MAEPS extension planner played almost no role in promoting it. A consultant firm, financed by a Japanese International Development Fund grant administered by the Bank, reported in 1995 that this logframe system was flawed and needed to be substantially modified. Ministry officials, commenting to OED on this report, said they were upset about donors always changing their minds or contradicting each other. 20. The integrity of this argument has been compromised by recent action taken to improve the scheme of service for school teachers. Village teachers have had a significant increase in salaries, partly through "horizontal" promotions (to higher grades with no change in function). The FA live in the same villages, are aware of the discriminatory adjustment, and resent being left out. 21. The system differed from standard T&V methodology in that the "contact farmer" was only designated for a specific OFD. The FA might or might not (usually not) use him/her again for the subsequent season's demonstrations. 22. It also had nothing to do with another use of the word "pilot" during the appraisal period. USAID's intention was to create an improved participatory system for developing and communicating acceptable new technologies to smallholders. The system was described in the SAR as a pilot and in fact came to be called the Pilot Communication Program (PCP). At one point in the SAR, it would appear this was to be the key "pilot" activity. Elsewhere in the SAR, and in the files of the period, all process reforms for the national extension program, that were to be piloted at Mzuzu, are highlighted. 17 reporting. In the ratings it more than offset the unimpressive performance of most of the technical assistance programs aimed at capacity building at headquarters. 3.18 The pilot was started by a newly appointed supervision task manager from Washington, who first visited MAEPS and Mzuzu in March 1987. This was only one year after the IDA Credit became effective, and in fact before he assumed responsibility for MAEPS. However, his March visit coincided with one of the early MAEPS supervision missions, two of whose three members were well-respected agriculturalists, sent by Bank headquarters, who had never visited Mzuzu before and were never to again. But they identified fundamental weaknesses in both the Block Garden strategy and the way it was being executed. They said so there and in their report, offending ADD staff who resented being treated as "fools" by first-time visitors. But the future task manager was also persuaded that changes were needed. Although the project was only a year into implementation, he had come to the conclusion that the planning and budgeting process reforms at headquarters were only partial solutions to the ministry's problems, and in any case would make no headway unless the field operations were restructured and the extensionists' motivation restored. That foresight was later to prove correct. But it was certainly not based on experience, since the expert team was only then assembling at headquarters (late 1987). 3.19 The objective at first was modest: simply to make adjustments that worked for Mzuzu ADD. Outreach to other ADDs would follow if the pilot succeeded. Support was sought and provided by a T&V extension expert at the Bank's Regional Office in Nairobi, who still supervises extension in Malawi. The Bank also invited Daniel Benor, the founder of the T&V system, to Mzuzu in 1988 and occasionally thereafter to assess and advise on the establishment and expansion of the modified BES T&V program.23 In 1989 a Malawian joined the staff of the Bank's Field Office in Lilongwe specifically to closely guide the development of the extension system.24 3.20 Early Years. The MAEPS pilot operation took off rapidly. Mzuzu ADD extension field staff were happy with it, partly because it promised them the transport and other resources they lacked. Also, OFD gave them more job satisfaction, since the impact of technology transfer on farmer practices was stronger than through the exclusive medium of the Block Gardens. The Block Extension System-the Groups and Gardens-was not discontinued. It was now complemented by OFD, and some agents were especially good at coordinating the use of both instruments. As stated in the PCR: "The Block Garden/OFD combination remained the most desirable option, with a range of recommendations shown on Block Gardens and specific ones, 23. Neither the members of the March 1987 supervision mission, nor the future task manager, considered themselves to be T&V advocates. They simply wanted to make the BES operating system work. Nevertheless, Daniel Benor's repeated visits to Malawi in the company of the Nairobi resident mission T&V expert helped ensure attention from the highest level of government. The Malawi pilot is sometimes, mistakenly, referred to in MAEPS supervision reports as the "Benor initiative." 24. In a thoughtful series of comments on the draft PAR, this Malawian staff member provides another perspective on the evolution of MAEPS. He says that the shift in priorities during implementation can be attributed to an "omission that was made at the design stage". He says that "MOALD had already sent an SOS to the bank indicating that they had noted that their extension system was not effective, but that they had no clue as to what needed fixing. They then asked the Bank to look at their extension system and suggest ways of improving it". The Bank's response, as embodied in the project design, was to create a position for a national extension planner and provide funds to Mzuzu ADD to carry out any recommendations. It did not offer any insights on the ongoing extension operations, or suggest the nature of procedural reforms called for in field activity. That "omission" invited and indeed demanded the apparent shift in priorities during supervision. (see reference in footnote 5). 18 selected by specific farmers, on OFDs" (PCR para. 5.12). Mzuzu ADD management swung its support behind the pilot after the next supervision mission, in October 1987. The supervision reports and the PCR describe the overall ADD response as "enthusiastic," and OED's audit interviews confirm that early support. 3.21 Within a year the Bank had agreed with MOA that two other ADDs should adopt the new system, in several of their Extension Planning Areas.25 In the third year (1989/90 cropping season) all three ADDs extended the pilot throughout their areas and the other five ADDs entered the program. Though aggressive enthusiasm and pressure by the Bank played a role in this rapid conversion process, at least some of the entering ADDs were eager to convert. The field extension staff everywhere favored the program, because of the extra resources and job satisfaction. 3.22 In the early years, the volunteer OFD farmers were mostly members of the credit clubs that the FAs had been accustomed to work with. OFD did not provide free or subsidized inputs. The volunteers had to finance their own demonstrations. But those with access to club credit had subsidized finance-if they needed it26-for the few inputs required to set up an OFD mini- 27 plot. In OFD's first five years the great majority of the demonstrations-several persons said 80 percent 28-were dedicated to elements of the hybrid-maize/fertilizer technical package. This technology had provided the basic set of extension messages for IRDP, NRDP and the Mzuzu ADD Block Gardens since the first project in Lilongwe, and was highlighted again by the OFD pilot. The emphasis swung dramatically away from "dent" hybrids to "flint" hybrids after the latter were introduced in 1991. But yields continued to depend heavily on fertilizer. The spread of this technology from the OFDs to neighboring fields was limited to those farmers who had similar resources. 3.23 The OFD pilot was designed to make a substantial capital injection into the Mzuzu ADD's extension program. This had originally been intended to put BES to work, with better transport, supervision and supporting materials and equipment. The OFD pilot, also based on adequate mobility for and supervision of all FAs, revealed that the level of finance estimated at appraisal and sanctioned by government was woefully inadequate to run an effective field extension service with full area coverage. After the pilot expanded in 1988, and with the prospects of further expansion in 1989, it became clear that the ministry could not implement the program without additional funds at least for operational expenses. MOA presented a list of proposed additional expenditures-on vehicles and operating expenses-which the Bank's Field Office called "massive." In May 1989 the Credit Agreement was amended to provide SDR 1 25. See footnote 1. 26. As the supply of cheap credit dried up after 1993, the cost of OFDs became a more important constraint on the program. Farmers were reluctant to volunteer without the subsidies, and many of those that did were unable to provide adequate and timely inputs for the demonstration. 27. As a rule, crop demonstrations were confined to one hundredth of a hectare, hence the name mini-plot. In reports "OFD" and "mini-plot" are used interchangeably, although OFD is a broader concept and includes demonstrations not involving crops or mini-plots. 28. Probably an exaggeration. The task manager of the period says the percentage was much smaller in the initial phase of OFD, before the flint hybrids were brought in. 19 million for recurrent costs of the pilot operations.29 The Bank's concern at appraisal concern for cost cutting, and government's agreement to cover all recurrent costs to help ensure sustainability, were set aside.30 Bank supervision legitimately argued that even in the absence of the pilot the Mzuzu ADD program was underfunded and incapable of reaching its production objectives. A guiding principle of the appraised design had proved to be unrealistic and was abandoned. 3.24 Staff at MOA headquarters did not fully31 support the pilot, a position which invited repeated complaints in the Bank's supervision reports (and in the PCR). One reason was historical, and reflected some officers' deeply held suspicions about singling out individual farmers for special attention. This reminded the veterans of the old "master farmer" system of the colonial era, from which they had always wanted to make a definitive break by stressing group approaches.32 Another reason was reluctance to be railroaded into the T&V program, particularly the new variant introduced by Bank staff, faster than the "pilot" could prove itself. The last, and probably most important reason, was the escalating costs of extension and the conviction that the ministry could not afford to finance the new program. This had been a problem with IRDP/NRDP from the beginning, and the pilot only aggravated it. 3.25 At the end of the government's comments on the Bank's PCR (Part II) the ministry takes the Bank to task for "forcing.. .the wholesale expansion of the T&V Extension system with its associated OFD" without due appreciation "of the availability of local resources." The ministry did not claim the program was inviable. It did not insist, as the Bank had at appraisal, that recurrent costs be limited to levels that could be financed through the government's Revenue Budget. It did say, however, that the Bank ought to have been prepared to absorb a larger part of incremental costs. In late 1988, at the start of the expansion of the pilot into two additional ADDs, the ministry felt it had been promised just that-Bank funds "as long as they are needed."33 Those expectations were to be disappointed. While recognizing that the Bank had agreed to provide extra resources, and reallocated SDR 1.0 million in 1989, the ministry asserts 29. A new category for operational expenses was established in the Development Credit Agreement. It covered the pilot and another, new program for monitoring food nutrition and security which had been added to the DCA in an earlier amendment (see annex A). 30. Less than three years after Board approval. 31. In the start-up years, although there was at least one senior official at headquarters who favored the pilot and was in a key position to support it, the majority of senior officers appear to have been hostile. 32. The Bank-financed extension planning advisor in DOA supported the Block Extension System but opposed the OFD pilot. He played practically no role in the take-off and spread of the pilot, and his Five Year Extension Strategy Plan hardly mentions it. 33. This alleged promise was cited in the ministry's minutes of a wrap-up discussion with Bank supervision staff in August 1988. The full passage is: "The meeting wanted to know what will happen after the funding period as Malawi government budgetary problems may not allow sustenance of the extension education to farmers. In reply it was reiterated that so long as assistance was needed the World Bank will always come in and can stay as long as they are needed. They gave examples of India, Thailand and Sri Lanka where they have been for 15 years now and are not intending to go because the financial and Technical Assistance they provide are still needed." (Minutes of the meeting, chaired by the Deputy Secretary, MOA, on August 16, 1988, and attended by 21 other MOA officers and three Bank staff.) 20 in the PCR that this was "insignificant to effectively assist eight ADDs" (PCR Part II, pages 2 1- 22). 3.26 Complicating the financial problem was government's archaic system of budget management. Although the IDA funds were made available, the rigidities of financial controls blocked easy ADD access to these incremental resources. The first of the reallocated funds reached the ADDs in mid-1992, three years after the Development Credit Agreement had been amended.34 3.27 Later Years. The pilot operation began to lose momentum in the early 1990s. In 1993 the MOA M&E Units at Mzuzu and other ADDs presented evidence that the number of actual OFDs was well below planned levels, i.e., nominal levels based on the number of Extension Planning Areas and FAs reported to be involved. Also, only about 30 percent36 of the contact farmers listed by the FAs and interviewed by M&E enumerators claimed they had OFDs. This indicated that many FAs had either misunderstood the OFD instructions (all contact farmers by definition should have at least one OFD) or were mis-reporting one or both figures. 3.28 Moreover, the M&E data from Mzuzu, the cradle of the pilot, indicated that the number of OFDs was declining. The PCR reports, but does not try to explain this reversal, attributing it to problems "that have not been diagnosed" and suggesting that "several project cycles may be needed" to solve them (PCR para. 6.6). Yet, several explanations for the reversal in Mzuzu, and in other ADDs in later years, have emerged since the PCR mission. Four of them follow. 3.29 First, there is general agreement at Mzuzu ADD that the program tried to do too much too quickly. Forty OFDs each season were too many for one FA to manage. Some of the early OFDs had been simply invented, and many more had been imposed on indifferent contact farmers in order to meet those targets, even when the technical message was not of interest to the "volunteer." Mzuzu ADD officers told the audit mission that a decline in OFD numbers to a realistic level was inevitable. Second, the failure of OFD numbers to approach targets in Mzuzu and other ADDs can be directly related to the shortage of operating funds. BES T&V should have been training and travel intensive. The pilot operation secured the funds to fulfill that objective. As mentioned above the ministry did not continue to fund the project on the scale required to keep the pilot expanding. Third, foot-dragging at headquarters, and at those ADDs where the program managers were also skeptical about the philosophy and efficacy of OFD, is Whether the Bank's team had promised anything more than to maintain a presence in the extension program is unclear. 34. The follow-on project, ASP, accepts that the Bank must support the recurrent costs of the T&V extension system. The legal documents provide funds to cover operational costs other than salaries (and maintenance, electricity and other utilities for offices), on a declining basis. This (1996) is ASP's third operational year, and the ministry has presented a request that the Bank postpone the first scheduled reduction in the IDA share of recurrent costs. 35. The Block Garden program, which MOA had been promoting countrywide during the late 1980s (only a few years ahead of OFD), had already stalled. The Five Year Extension Strategy Plan, issued in draft in 1990, provided the first documented evidence that the number of Block Gardens was far below target and only a third the number of officially reported blocks. 36. The ratio was higher at Mzuzu ADD. 37. In fact the PCR mission, in the field in December 1993, followed the 1992/93 cropping season when there had been a spectacular resurgence of maize production after two years of drought. The surge of production was accompanied by a reversal in the decline of Block Gardens and OFDs, but it was short lived. 21 another important factor. Apparently the key position in the extension system at the Mzuzu ADD was taken over in this period by an official who had always objected to the pilot. 3.30 A fourth explanation relates to broader changes in the agricultural sector that were less evident to the PCR mission in late 1993. Farm production in the two years preceding a 1992/93 bumper crop had been seriously depressed by prolonged drought, one of the most severe of the century. Maize production, and farmer cash incomes, collapsed. The depressed state of farming affected the popularity and survival of maize demonstrations unsupported by free inputs. Drought returned in 1993/94, after the bumper year. And 1994 also saw much higher fertilizer prices, as government removed the subsidy as part of a program of structural reforms. Fertilizer costs in Malawi are extraordinarily high, due to the long transport haul from the docks for imports of this bulky commodity. As a consequence, the margin between costs and benefits of the hybrid-maize/fertilizer package is extraordinarily narrow. As the demand for fertilizer fell, the demand for hybrid seed fell as well. This seed is expensive, and without fertilizer the hybrid yield increment is too small to justify the cost. The droughts, accompanied by the removal of the subsidies, prevented further progress of what had hitherto been the most popular OFDs. The era when the hybrid-maize/fertilizer package dominated extension practices had closed. 3.31 The OFD program had to be reformulated to meet the needs of farmers who once-but no longer-could afford a high input package. Moreover, in response to growing recognition that the majority of farm families had never been included under the IRDP/NRDP umbrella, including OFD, the overall T&V strategy has itself had to be revised to achieve greater outreach. The Malawi extension service, the BES T&V system, and the OFD methodology are turning to face a broader clientele with new packages. The Bank's follow-on project, the Agricultural Services Project (ASP), is supporting that adjustment. MAEPS was completed during the last, unlucky years of an older, hitherto durable, strategy keyed on an objective-self-sufficiency in maize-that was no longer viable. 3.32 Supervision reports for the final years of MAEPS reflect the pilot operation's loss of momentum. Declining performance of what the PCR calls the project's "centerpiece" heavily influenced supervision ratings of the project, since there was little good news from other components. The last of the group of technical experts financed by the Bank departed Malawi in mid-1990, effectively ending the Bank initiative in capacity building at MOA headquarters. At the end of 1990 a supervision report for the first time rated the project a "problem project." It cited in particular the lack of commitment to the new extension approach from headquarters and from "most ADDs." The ratings fluctuated in and out of the problem project category ending up with a satisfactory grade in the last report. 3.33 The Credit was extended three times for a total of 22 months beyond the original closing date of September 1, 1991, but only in order to ensure that as much as possible of the SDR 1 million reallocated to recurrent costs could be disbursed, bridging the gap until the recurrent funding promised by ASP became available. That the Bank agreed to three extensions of relatively short duration reflected another disturbing feature of this project: government's failure to comply with an unusually large number of legal covenants. The project was completed and the 38. Stephen Carr has calculated that a farmer has to produce about 19 kg of maize to purchase I kg of nitrogen at unsubsidized prices. This "grain:N" ratio of 19-1 compares with ratios between 2-3:1 in both North America and India. (Carr: "The Unique Challenge of Malawi's Smallholder Agricultural Sector"; undated draft, approx. June 1996.) 22 Credit closed in 1993. SDR 510,000 was canceled, including SDR 322,000 from the new recurrent cost category. 23 4. Outcomes A. MOA Headquarters 4.1 The experts financed by the Bank at MOA headquarters had little impact on the ministry's planning, budgeting and financial management processes. Mzuzu ADD officers recall no process reforms of any importance associated with MAEPS, and originating from MOA headquarters, that they were supposed to have piloted. That is not altogether fair, because each of the long term experts did make contributions that would have impacted on Mzuzu, whether or not that ADD was used as the pilot for reform. The DOA financial advisor contributed to the development of the activity and cost center concepts for budget organization, building blocks for subsequent improvements in the budget process. The extension planning advisor, apart from managing the production of the largely descriptive 1990 draft Five Year Extension Strategy Plan,39 calculated average activity costs of extension operations which were subsequently used as standard yardsticks. The M&E advisor was mentioned as having trained field staff in preparation of the annual work plans. As already stated, interviewees applauded the work of the systems analyst assigned to the Planning Division. In terms of overall output and impact, however, the technical assistance rendered by this disparate team of resident experts delivered little of the enhanced capacity previewed in the SAR and fundamental to the justification of the project presented to the Board. 4.2 The training program, in-country and overseas, was better. The senior training expert's assistance was well considered. The major drawbacks were the relatively high expenditures on training trainers, rather than field staff, and the loss of the majority of MOA's economic degree graduates to other ministries. Though not a loss to the government and the economy, it was a loss to MOA. The impact of the rest of the degree training program on institutional capacity building is less clear. It was valuable only to the extent the training was put to use. The audit mission was told about graduates returning from overseas to the same job, room, cracks in the windows and holes in the chairs they had left two years before, with no financial reward and little recognition for their higher skills. The schemes of service do not allow for automatic adjustment of a salary to reward the higher degree. That only comes with promotion, which often means moving to another ministry with different training requirements. Just as some interviewees criticized senior MOA officials for mismanaging the expatriate long-term experts, so they criticized them also for not knowing how best to deploy their own graduates. The audit discussions mostly focused on the economist cadre, and it is not clear whether the same criticism can be leveled at the managers of agricultural graduates. The PCR suggests that it can: "the lack of discernible impact from high training expenditures is troubling" (PCR, para. 10.2). 4.3 In any case, neither the experts nor the trainees stimulated any of the substantial process reforms targeted at appraisal. During the project period there were improvements, but these were mostly attributable to other donor activities. In particular, the logical framework was introduced as a basis for annual work plans throughout the ministry, including at EPA and RDP40 field offices of the ADDs. A German GTZ team had adapted the ZOPP program, based on the 39. Which one supervision report described as too "prescriptive," with little supporting analysis of critical issues such as extension's low impact on smallholder production. 40. See footnote I for definitions. 24 logframe, for the Liwonde ADD. From that base, and starting in 1988, USAID and its ministry counterparts promoted the use of the logframe routine at all the other ADDs, using a few experienced Liwonde staff as part of the training team.41 The logframe was brought in along with "zero-based activity budgeting," a system that builds each year's budget upon activities approved in the annual work plan, rather than basing it on the previous year's budget. "Yardsticks" have been introduced to standardize activity cost estimates and non-salary/salary cost ratios across all ADDs. More recently, the government has started to switch to a "cash" budget, which prevents operating units like the ADDs from incurring commitments unless they already have the funds. MOA does not associate MAEPS closely with any of these subsequent planning, budgeting and spending reforms, except insofar as some of the foundation was constructed under MAEPS and Credit funds were used to finance in-service training on the new procedures. 4.4 Even with these post-project improvements, the overall impact on ministry planning, budgeting, and financial management processes has been modest. Indeed, some recent reports suggest the situation has adjusted very little since the mid-1980s. Thus, for example, the consultant's final report on phase 2 of the Management Strengthening Programme, dated August 1995 (see footnote 19) refers to the "disappointments" that have characterized other government management improvement efforts in recent years: "These Reports are generally filled with impractical recommendations that were developed by external consultants who worked in virtual isolation from their clients and/or those who would be directly affected by their proposals."42 Elsewhere the report on phase 2 says the ministry's work plans and budgets were still being carried out as two separate exercises. Although the Planning Division was responsible for coordinating these functions, continued erosion of the capacity of this "disintegrating" unit had blocked effective action. Thus the Bank's most recent supervision report on ASP concludes "little has been done to strengthen extension or financial management within MOALD."43 This is the same language used by the Bank 12 years earlier. 4.5 Ministry officers told the audit mission that even some of the improvements that had been made were breaking down, the result of continued pressure on the budget. Staff in the Planning Division said the logframe had effectively been "shelved," and the cost center concept was largely irrelevant. They felt the disconnect between the work planning exercises, and the budgets that were eventually approved and released, was so large as to make the elegant planning procedures a waste of time. 41. The PCR seems to attribute the application of the logframe-based planning and budgeting system to MAEPS (PCR para. 5.8). This is too generous. Although technically under the MAEPS umbrella, USAID's initiative, building upon GTZ's work with ZOPP, would be more appropriately claimed in the NARP completion report. See footnote 19. 42. Development Management Associates; MSP, Phase 2, Final Report, page 2. 43. Form 590, Section 5. June 27, 1996. 25 B. Mzuzu ADD 4.6 The outcome at Mzuzu has two parts. First, the capital investments at ADD headquarters and its field stations, the training programs, and the specialized activities such as the Pilot Communication Program (PCP), gave a substantial boost to Mzuzu ADD's implementation capacity. Half of the ADD area had been covered by NRDP I. The headquarters building itself had been financed from another program. But MAEPS provided much needed office, extension and audio-visual equipment, in addition to vehicles (28), motorcycles (107), and bicycles (100). MAEPS also financed 117 houses for FAs, 16 marketing sheds and two office buildings.45 4.7 Of greater importance is the outcome of the extension pilot operation and its impact on farm production, both at Mzuzu and in the other ADDs to which the pilot was extended. During the project period, i.e., through mid-1993, this impact is best represented by, if not confined to, trends in maize production. Maize dominates the Malawi economy. It covers 80 percent of the nation's cropped area, is grown by the great majority of farmers, and is the foundation of food security. Maize dominated the Block Extension System throughout most of the Mzuzu ADD area.46 The OFD program maintained the emphasis on maize during the project period. 4.8 The project period also saw the introduction of flint hybrid maize seeds on a wide scale. The flints are an improved hybrid variety which had initially been disregarded by the Malawian research establishment but subsequently proved to be superior. The flints were as popular with farmers for home consumption as for sale. The dent hybrids which the flint hybrids replaced dominate world trade but are not eaten by Malawians. 4.9 The PCR team, following a visit to Malawi in December 1993, reported on the remarkable expansion of maize production in the 1992/93 boom season, driven by high yields harvested from the larger area sown to the fertilized flints and dent hybrids. The PCR makes a rough pass at a break-even analysis of the economic rate of return (ERR) for the Mzuzu component of the project, based on maize production. It calculates that to reach 13 percent returns,47 the proportion planted to hybrid of the total area in maize in the Mzuzu ADD would have to increase from 30 percent in 1992/93 (and a 20 percent average for 1990/91-1992/93), to 40 percent in 15 years. It calls this a conservative projection-and so it might have appeared after the exceptional rebound in agriculture in 1992/93 from the previous years' drought. That rough arithmetic did not include the other crops featured in the Block Gardens and OFDs. Nor did it include the impacts on production (and associated incremental costs) of the pilot operation in other ADDs. The PCR team was comfortable with the conclusion that the directly productive components of the project were economically efficient. Of course, that finding depended on the assumption that the incremental production could be associated with extension activities. 44. See footnote 22. The Bank's supervision task manager was less generous in describing PCP, calling it "a flop with negligible impact." 45. The house designs have been widely criticized in Mzuzu as being small and inconvenient, factors attributed to Bank-prompted cost-cutting. Nevertheless they are all occupied. No houses were budgeted for storekeepers for the marketing sheds, which resulted in the sheds standing idle for most of the project period. 46. The eastern part of the ADD is lower, hotter, and dominated by cassava and vegetables. 47. The SAR also estimated a project ERR only for the Mzuzu component, of 22 percent. In that analysis it attributed 24 percent of total project costs to Mzuzu, i.e. not the full 35 percent shown in the SAR cost tables (including credit. See para. 2.10). The SAR does not explain the discrepancy. 26 4.10 Subsequent events have undermined those calculations. The area planted with hybrid maize has shrunk, along with the level of fertilizer applied per hectare of hybrid, nationwide and in Mzuzu. In 1995 hybrids occupied only 10 percent of the area planted to maize nationally and 15 percent in Mzuzu. Fertilizer sales for use on maize dropped from 26,000 tons of nutrients in 1990/91 to less than 12,000 tons in 1995/96.48 The removal of the subsidies was the primary factor explaining the reversal (para. 3.30). The OFDs could not prevent it. 4.11 MAEPS was completed three years ago. It made an important contribution to extension by breaking the constraints limiting the Block Garden program: by moving to the contact farmers' fields and refocussing the extension messages to more closely meet their farming conditions. But the stream of incremental benefits from the present maize crop, assuming they are all directly attributable to the MAEPS investments, is a fraction of that anticipated in the PCR. The streams of benefits from incremental production of alternate crops, to the extent they can also be attributed to MAEPS, lag far behind the MAEPS cost streams and are unlikely to significantly improve the viability of that investment. 4.12 However, there is no evidence to support the attribution of the bulk of incremental hybrid maize production to T&V extension, as in the SAR and PCR calculations. Other observers attribute most of that dramatic development variously to the flood of subsidized credit in the early 1990s and to successful promotion by the hybrid seed industry. Extension played a role, but, according to this view, not the decisive one. The one area where the extension influence should have been felt was in the correct application of the higher nutrient fertilizers that were introduced starting in the late 1980s. Yet a PhD study in Mzuzu has shown that less than 20 percent of the fertilizer users knew the correct rates and time of applications. As a consequence, the average yield of hybrids has been declining,49 hardly the basis for a positive rate of return on an extension project aimed at raising yields. 4.13 MAEPS also cannot take much credit for some of the most interesting developments under the ASP. Together with IFAD's Smallholder Food Security Project (SFSP, SRS 35-MW), these companion projects have departed in significant ways from the MAEPS framework. Not only have they successfully integrated research and extension, but their extension strategy is determined by research results. MAEPS was in theory complemented by the Bank's and USAID's research projects (NARP and MARE). But the pilot extension program was self-driven and largely committed throughout the project period to the hybrid-maize/fertilizer technologies. These, in turn, were based after 1990 on releases of the new flint varieties. The partnership with research effectively stopped there.50 Under ASP/SFSP, research drives the whole program. On farm trials (OFTs) have taken the lead, not OFDs. 4.14 A remarkable surge of group activity, both traditional community associations such as church groups, and newly formed "special interest groups" (SIG) dedicated to a common economic activity, is also only loosely related to MAEPS, OFDs and other routines of the pilot 48. A commentator on an early draft of this report wrote: "sales of fertilizer for use on maize worked out at the equivalent of an average of 2.3 kg of nutrients per head of population during the 1980s and is now expected to settle to its 1995/96 figure of I kg." Memorandum from S. Carr to E.B. Rice, dated October 28,1996. 49. S. Carr reports that "the three year running average of hybrid maize yield had fallen form 2.94 tons in 85/86 to 2.42 in 92/93." Carr also provides the reference to the PhD study. See the previous footnote, regarding his memorandum. 50. See the brief discussion of the failure of the Adaptive Research Team approach in annex A. 27 operation. At first the OFD emphasis on contact farmers slowed the growth of group activity. Now, most of the FA in the T&V system support at least some of these traditional and special groups. But few of these groups are closely tied to Block Groups and Gardens. The "income generating projects" (IGP) developed collaboratively by the ADDs and the SIGs usually do not include OFDs. Only a few are counted as OFDs in ADD reports. The productive relationship between an IGP and the FA is determined by the time-bound requirements of that particular economic activity, rather than by a biweekly visit schedule. 4.15 ASP has also expanded well beyond the hybrid-maize/fertilizer package. The Bank's concern for the lack of viable extension messages apart from the maize packages dates at least as far back as the identification of NRDP V in 1983. MAEPS did not significantly extend that base. ADD headquarters, extension agents and farmers are looking now for affordable low input crops and technologies. ASP, and the companion IFAD project that is aimed specifically at the poorest households, are leading this campaign. The emphasis has shifted to soya, composts, interplanted leguminous trees and other methods of raising soil fertility without the use of fertilizer. The yield increments of most of these alternative low-input packages do not come close to those of the fertilized packages. Nevertheless the low-input/low-output improved technologies provide a boost to farm food supplies and income. The rapid expansion of smallholder soya production is especially valuable, since it restores rather than removes nutrients from the soil while providing a highly nutritious foodstuff to the family and particularly the children. For club members who can no longer afford fertilizer, and for the poorer farmers who never could, these are welcome developments. 4.16 This story of project-related impacts leaves out the most exciting development in smallholder agriculture in the last five years: the rapid expansion of burley tobacco production. Burley is the only highly profitable cash crop suited to smallholder production conditions in the Malawian mono-modal rainfall environment. But from Malawi's independence in 1962 to 1990 only estates were licensed to grow burley.51 The Bank declined to finance Malawi's tobacco crop. But it did provide crucial support at the policy level to secure the repeal of laws preventing smallholder participation, and to get the first licenses issued and later extended. USAID helped lead that development in the early 1990s. Smallholder production has responded handsomely, and about 100,000 farmers now produce burley, with and without licenses. Their production 52 reached 40,000 tons in 1995, 40 percent of the national burley crop. 4.17 ASP is not directly involved with burley, but half the IFAD companion project also supervised by the Bank is aimed at supporting that crop. This is not a T&V extension success story. In burley areas the ADD extension service-FAs and, particularly, the ADDs' tobacco subject matter specialists-supports the burley program and some OFDs are devoted to tobacco.53 But the smallholder burley clubs are different from the ADD credit clubs, and the T&V visit routine is almost immaterial. The extension service's contribution to the rapid growth of this (once-familiar) cash crop, following the decision to restore licenses, has been equally as important in ADDs and EPAs outside the BES and OFD areas as in them. 51. Although there was considerable illegal planting by smallholders. 52. OED's audit report on the Malawi Agricultural Sector Adjustment Credit discusses the burley success story, attributing it more to USAID's follow-through than to the Bank's initiative (see footnote 4). 53. Usually for management of the sheds, not a mini-plot of tobacco. 28 4.18 In short, MAEPS made important modifications to BES that set the stage for improved performance of the T&V system. But neither MAEPS nor the T&V system explain the subsequent resurgence of research, group action,54 low-input technologies and burley tobacco. Of course T&V is not supposed to produce low-input technologies and burley tobacco: it is supposed to extend them. The record of the last few years suggest they have extended relatively well independently of T&V. 4.19 OED did not carry out a formal recalculation of the ERR for the Mzuzu component, based on re-estimates of actual areas and yields of all targeted crops. As just mentioned, the PCR did a break-even analysis and concluded the ERR, based on a modest expansion of hybrid area, and assuming extension was responsible for much of that progress, was likely to be well above 13 percent. Its optimism was fed by the bumper crop of the previous year. The subsequent decline in area committed to hybrid-maizes would result in a negative ERR according to the PCR calculus. The reversal eliminated any long-term impact of the project's extension program on the maize economy. A more flexible, diverse set of messages would have put the ADD in a better position to offset the retreat from fertilized hybrids. But it is unlikely this would have compensated enough to maintain the economic viability of the Mzuzu investments. C. Ratings 4.20 The audit rates the overall outcome of MAEPS "unsatisfactory." The PCR did not provide a rating. But by giving priority to the T&V pilot operation over the MOA headquarters capacity building components, and by assuming a favorable ERR at Mzuzu, the PCR implied the outcome is satisfactory. That is the rating assigned in OED's Memorandum introducing the PCR.55 The downgrading is warranted for two reasons. 4.21 First, the PCR evaluates a project different to the one presented to the Board. The PCR admits as much: "considering that the project had not been designed to remodel extension procedures in the field" (PCR para. 5.12). There is no doubt that the project failed to reach the objectives of the capacity building components. It also abandoned the low-cost criterion applied at appraisal. Appraisal objectives proved unrealistic. But the PCR should not have casually displaced them, to reward nimble task managers. If supervision had called for a mid-term review, and the project had been redesigned and approved with a new ordering of priorities, then the implicit objectives underlying the PCR rating would have been appropriate.56 The Bank should still have been concerned with the lack of progress in capacity building, but that problem would not have prevented ratings based mainly on the pilot operation.57 54. The Bank's task manager for most of the project period says that he was wary of pushing group activity too much because he knew how strongly the Mzuzu ADD felt about preserving the homegrown BES system . An earlier attempt to downgrade BES had invoked strong resistance (para. 3.18). 55. The FAO/CP staff member who led the PCR mission does not agree that the original PCR "implied" the outcome was satisfactory. (Memorandum from R. Suppa to Ben Van de Poll, dated January 17, 1997). The relevant passage was modified in a subsequent draft (the phrase "was generally satisfactory" was added), and OED finds the tone of the whole final document positive. 56. The amendment to the Development Credit Agreement, dated May 10, 1989, which reallocated SDR 1.0 million for recurrent costs, also added "operation of pilot schemes" to Schedule 2. But there was no attempt to realign the original priorities. 57. One commentator on the draft PAR questioned the "somewhat bureaucratic" logic of this rating procedure, keying the overall outcome rating to whether the adjustments were formalized by a mid-term review. Other readers may have 29 4.22 Second, the pilot operation failed to live up to its potential. Partly because of its own deficiencies, partly because the government did not support it, and partly because the technical packages it promoted during the project years proved later to be inviable, the economic returns to the program have not matched the PCR's expectations. The OFD innovation was highly appropriate, and it has positioned the extension service to take better advantage of the BES T&V routine. But that is not enough to justify a satisfactory rating. 4.23 Similar considerations apply to the ratings of institution building and sustainability. The PCR did not rate institution building, but OED's covering Memorandum called it "modest." OED's definition of this rating is that the institutional objectives "were met only to a limited extent." That seems about right. On the one hand, there was the disappointing performance at MOA headquarters attributable to the MAEPS technical assistance. On the other hand, by strengthening both the BES T&V program and the Mzuzu ADD, the project added to the ministry's capacity in the field. This partly offset the lack of progress towards the dominant capacity building objectives. 4.24 The PCR did discuss sustainability. Referring again to the pilot operation, the PCR anticipated that further cuts in the recurrent budget posed a threat to "sustainability of the project's activities" in the absence of cost savings at the ADDs. Despite support from ASP, the ADD budgets have indeed deteriorated as the government's fiscal situation has worsened in the last couple of years. During the audit, OED observed many signs of the budget crisis at Mzuzu: the recent release of over half of the M&E enumerators (21 of 40); the lack of funds for Visual Aids staff to go to the field to generate new extension messages (forcing them to work with "repeat" messages); curtailment of the training program; and, most important in terms of staff morale, the fact that most staff had not been paid for three months. OED's Memorandum reflected the PCR's "doubts" by rating sustainability as "uncertain," and this audit agrees. been equally uncomfortable. The PAR adopts this position partly because of the importance of the capacity-building objectives that were displaced, and partly because the adjusted program failed to reach the new objectives, as discussed in the following paragraph. 30 5. Findings and Lessons A. Findings 5.1 Conditions for Successful Capacity Building. The Bank promised major advances in capacity building and did not deliver. The question is whether: (1) at appraisal it overestimated the potential of the technical assistance program, the interest and ability of the ministry to put it to use, and the competence of Bank supervision to keep capacity building on track, or (2) in this case supervision turned prematurely away from institutional developments at the center. Poor outcomes are common in this class of projects, and it is important to identify the limiting factors. With MAEPS, the problem appears to have been inherent in the design: Long term advisors. The SAR sought reform of two of MOA's core units through five resident experts financed by the Bank and a sixth by EEC, for an average of 30 months. The individual terms of reference and qualifications were heroic. The messy and low- budget consultant procurement process that followed should have been anticipated. Using a "post box" recruitment firm, placing the experts in different units, and failing to create a sense of teamwork, were other unfortunate features. Even with recruits fitting the description, and even if they had been adequately backstopped by the supplying agency, 30 months was too short a time to produce sound and sustainable results. Given these sobering facts, the terms of reference should have been written less ambitiously and the objectives of the capacity building components correspondingly reduced. Ministry absorptive capacity. It is unclear whether the senior officers who helped prepare the project embraced the capacity building objectives. Training, funds for Mzuzu ADD, and material support for offices at headquarters were undoubtedly of primary importance. Certainly the officers who replaced government's preparation team showed little commitment to the long term advisory component. They did not object to improvements, but to unproven and potentially costly tools of modern management that threatened established-and what they felt were sensible-procedures. This reaction, and the turnover of senior staff, should also have been anticipated. In particular, staff instability at the Planning Division seems to have been inevitable, in the absence of reform of the economist's scheme of service. Bank supervision skills. Getting better performance out of the six resident experts also called for heroic supervision skills, well beyond the Bank's capacity to deliver. Thus, supervision reports paid attention to the experts' activities and the process reforms expected of them (as well as commenting on problems of recruitment and, later, on the disappointing end results), but had almost no impact on the utilization of these experts or their output. The Bank's Field Office provided substantial support to MAEPS, joining all the supervision missions and keeping up continuous contact with project authorities. But the pilot operation became its priority too. In retrospect, resident mission staff seem to have been unable to focus and materially strengthen the technical assistance program in the ministry. 5.2 Ministry Management ofResident Experts. MAEPS is a good example of the difficulty experts have in making an impact when the ministry fails to take ownership and manage their 31 time. One of the Malawian staff in the Banks' Field Office believes this to have been the decisive factor explaining the "so-called" poor results. The officers who had to work with the experts did not know why they were there, did not know what to ask them to do, and later complained about the results. The staff person insists these officers did not have the basis for a proper evaluation: "if you don't use a knife, you can't tell whether its sharp or blunt." 5.3 Donor Support for Resident Experts. The USAID project MARE provides a good example of the positive effects on resident experts' performance if they are well backstopped and supported by the donor agency, its contract firms, and other members of the expert team. Material donor support-training grants, computers, desk-top and other audio visual equipment-increases the likelihood that the experts will have positive impact. By contrast, the recruiting firm that delivered the MAEPS experts was nothing more than a "post box" agency, with no capacity to provide technical support. 5.4 Bank Commitment. The resulting displacement of project goals set at appraisal was perhaps inevitable. It is possible that if an institution-building expert had been assigned early as supervision task manager, and, strongly supported by equally well prepared institutionalists at the Bank's Field Office, had concentrated on enhancing the effectiveness of the technical assistance program rather than on Mzuzu, the results at MOA headquarters would have been different. Given the constraints mentioned above, and the Bank's skill profile, that outcome seems unlikely. If the Bank wishes to achieve ambitious capacity building objectives, it must make a major commitment to upgrading its own skills in this field. The alternative is evident in this project, where a task manager-an agronomist with formidable farming and operational skills-saw an opportunity for process reforms in the field rather than at headquarters, pursued them, and succeeded. The other job was effectively shelved. 5.5 Costs ofExtension. This audit report refers several times to the "high costs" of T&V extension, of the BES model, and in particular of the OFD initiative. During the period of this project these were never extravagant expenditures. For example, the FA still use bicycles, despite often being responsible for huge working areas. This means that Malawi extension lags behind some other African countries where motorcycles have begun to replace bicycles for front line staff. High costs are simply an inevitable consequence of efforts to put any staff intensive field extension system to work. T&V extension appears to be relatively expensive because it takes the training and visit schedules more seriously than other systems. When an extension program compromises on mobility for front-line and supervisory staff, for the travel and lodging costs of their training, and for stationary and other equipment to support their field visits, it ensures a low level of impact. There are ways to economize. The rapid shift toward group contact is one of them. The PCR suggests a few more, including cutting back on the unusually high number of subject matter specialists and abandoning the idea that FAs have to be spread uniformly over all farming areas. But extension will always remain a high cost development service, especially where the clients are poor, illiterate, without television or good road access. 5.6 Role of T& V. Neither of the T&V systems introduced at Mzuzu-the homegrown BES adopted in 1981 or the Bank's OFD initiative in 1987-have lived up to their claims. The government's desperate fiscal position goes a long way to explain the shortfall. Government simply could not support these high cost operations without levels of financial aid the donors were unwilling to provide. The government found it difficult to support any of its extension services at levels that made them fully effective. 32 5.7 Nevertheless, the Mzuzu experience shows that the problem with T&V was more than just budgetary. The rigidities, narrow perspective and lack of sophistication and specificity of both the training and visit routines limited the potential flexibility and outreach of the system. Recent Mzuzu and Malawi experience shows that the T&V system is neither a necessary nor sufficient condition for smallholder progress and an effective extension contribution. The innovations inspired in the last four years with aggressive support from USAID, IFAD and the Bank's ASP project, the ones described in paras. 4.13-4.16, have pulled T&V along rather than been propelled by T&V. In fact they have given the T&V system a fresh mandate. The downward trend in the number of effective OFDs has apparently been reversed in the last year, as the spate of new low-cost technologies has taken hold and the numbers of receptive traditional and focused club groups has exploded. The ADD contribution to these successes has unfortunately been limited by the lack of funds. B. Lessons 5.8 Capacity Building. The Bank should not keep putting off following through on its professed commitment to capacity building. This is yet one more agricultural project which OED has reviewed where capacity building objectives that were taken seriously at appraisal were casually relaxed later on. If the Bank truly intends to build capacity among borrowers, it has to arm itself with a new set of weapons. 5.9 Rating of Overall Outcome. The audit rates the outcome "unsatisfactory," because it assesses results against the declared primary objectives and because the continuing slippage by capacity building components in the Bank's portfolio must be highlighted and reversed. That may seem rough treatment of the supervision effort on this project, which reconstructed an impractical appraisal design. The SAR and Development Credit Agreement left a loophole for just the sort of nimble realignment that took place. The language quoted in para. 2.10 of this audit report opened the way for the pilot operation, even though that was not intended. Nevertheless, even while applauding supervision, the audit cannot dismiss the poor performance of a program of paramount importance to the Bank's goal of sustainable development. The lesson is that judgments on the outcome of capacity building projects may have to be carefully explained, but should resist the temptation to settle for progress on secondary assignments. 5.10 Costs ofExtension. The Bank's passion at appraisal to cut field costs to the bone was ill- advised. The Bank's refusal then to help finance the government's recurrent extension budget was also inappropriate, and ultimately self-defeating. These extension expenses are for legitimate operation and maintenance costs of implementing an investment program. Provided the extension operation promises to deliver positive net benefits, and extravagant costs have been excised from the design, Bank participation can be justified. ASP supports Bank contributions to the operational costs of T&V extension; the PCR does too. So does the audit. 5.11 Role of T& V When properly managed, T&V extension serves the innovative and promising crop campaigns set in motion in Malawi in the last few years. They did not emerge from the T&V program, but from other initiatives. In fact, they helped put T&V back to work- after the attractions of the packages of hybrid flints and fertilizers disappeared. T&V extension is 58. To be sanctioned later in an amendment to the Development Credit Agreement: see the preceding footnote. 33 not a "solution" to rural poverty, it is ineffective when treated as the target rather than the tool, and operations which feature only T&V are likely to founder.  35 Annex A Basic Data Sheet AGRICULTURAL EXTENSION AND PLANNING SUPPORT PROJECT (CREDIT 1626-MAI) Key Project Data Actual as % of Appraisal Actual or Appraisal Estimate Expectation Current Estimate Total Project Costs (US$ m) 20.2 23.0 114 Credit Amount (US$ m) 11.6 15.0 129 Cancellation - .5 - Institutional Performance ... Modest Follow-on Operation Not Discussed ASP ... Not available. - Not applicable. Cumulative Estimated and Actual Disbursements FY93 FY86 FY87 FY88 FY89 FY90 FY91 FY92 /FY94 Total Appraisal Estimate (US$ m) 0.7 3.3 7.2 9.3 10.8 11.6 - - 11.6 Actual (US$ m) 0.5 0.7 1.8 4.0 6.9 9.2 11.8 13.5 15.0 Actual as% of Appraisal 71 21 25 43 64 79 102 116 100 Date of Final Disbursement: November 16, 1993 - Not applicable. Project Dates Appraisal Actual Identification 01/83 Preparation 06/83 Appraisal 09/84 09/84 Negotiations 07/30/85 07/30/85 Board Approval 09/19/85 09/19/85 Signing 12/20/85 12/20/85 Effectiveness 02/01/86 03/20/86 Project Completion 03/01/91 06/30/93 Credit Closing 09/01/91 06/30/93 Annex A 36 Staff Inputs (weeks) Stage of Project Cycle Planned Revised Preparation - 62.7 Appraisal - 101.3 Negotiations - 10.5 Supervision - 246.2a PCR - 0.3 Total - 421.1 - Not applicable. a. Including full-time Malawian agricultural services specialist in the Resident Mission. Mission Data Date No. of Specializations Staff days in Performance (month/year) persons represented a field ratingc Types of problems Pre-appraisal Jun/Jul 84 - - - - Appraisal Sep/Oct 84 - - - - Supervision 1 Mar 86 1 A 5 1 - Supervision 2 May 86 1 A 4 1 - Supervision 3 Oct 86 2 A, E 5 2 PM, 0 Supervision 4 Feb/Mar 87 2 A, E 20 (10) 2 PM, 0 Supervision 5 Oct/Nov 87 4e A, A, E, EX 26 (9) 2 PM Supervision 6 May/Jun 88 4e A, A, E, AR 24 (6) 2 PP Supervision 7 Oct/Nov 88 40 A, A, E, EX 31(6) 2 PM, PP Supervision 8 Feb/Mar 89 3 A, E, EX 31(10) 2 PM, PP Supervision 9 Nov/Dec 89 3e A, A, AS 25 (9) 2 PM, PP Supervision 10 Jun/Jul 90 2 E, AS 26 (9) 3 PM, AF, PP, LC, DO Supervision 11 Dec 90/Jan 91 2 E, AS 30 (11) 2 PM, AF Supervision 12 Apr/May 91 5 A, E, AS, F, P 22 (6) 3 PM, AF, PP, LC, DO Supervision 13 Oct/Nov 91 6e A, A, E, AS, EX, P 26 (6) 3 LC Supervision 14 Jun/Jul 92 4e A, A, AS, EX 28 (6) 2 - Supervision 15 May/Jun 93 2 AS, AS 18(9) - Not applicable. a. A = Agriculturist; AR = Architect; AS= Agricultural Specialist (resident staff); E = Economist; EX = Extension Specialist; F = Financial Analyst; P = Procurement Specialist. b. From the fourth Supervision mission, the World Bank combined the supervision of other IDA assisted projects; e.g., NARP, NRDP III, NRDP IV, Agricultural Marketing and Estate Project, Dowa West Rural Development Project, Kasungu Agricultural Development Project and Smallholder Agricultural Credit Project. Figures in brackets are attributed to MAEPS. c. Performance rating status: 1 = Problem free or minor problems; 2 = Moderate problems; 3 = Major problems. d. AF = Availability of Funds; DO = Development Objectives; LC = Compliance with Covenants; 0= Other; PM = Project Management; PP = Procurement Progress. e. Including consultants. f. No Form 590 was prepared. 37 Annex B Other Components 1. This main report focuses on the Bank-financed capacity building components and the extension pilot operation. Among the other activities included in the project, most of which are discussed adequately in the PCR, are: 2. Mzuzu ADD Credit Program. About US$2 million of the Credit was disbursed for seasonal loans to Mzuzu credit club members. This program was independent of the pilot operation, although it occupied most of the FA's time and included most of the OFD contact farmers. The Bank was committed to disburse only against incremental lending, but the definition of a base-line for measuring the increment was disputed. In any case the lending volume declined in later years, as delinquency increased and club members and clubs were rejected for repeater loans. The droughts in the 1990s are part of the explanation, as well as the rapid expansion of lending volumes in the early years and the politicization of the credit system and softening of rules on debt rescheduling in later years. (PCR paras. 5.24-25). 3. Food Security and Nutrition Unit. The government set up FSNU in EP&D in 1987, contracted with the Harvard Institute for International Development (HIID) to provide technical support, and requested Bank support. The MAEPS Development Credit Agreement was amended to include EP&D. The Unit has earned a good reputation, starting from assessments that helped mobilize donor support to avert disaster during the 1987 food crisis. The PCR calls it an "indispensable" agency and a "vital tool for the Government to monitor national food supplies and nutrition status and design remedial actions if and when needed" (PCR page iv). But the Unit's effectiveness has been undermined by budget and staff shortages. The 1989 amendment to the DCA that provided for recurrent cost financing for the extension service also included FSNU. HIID technical assistance continued after the project, but on grant finance from another donor. (PCR paras. 5.26-27 and 6.11). 4. Improving Research/Extension Linkages. The Adaptive Research Team initiative was financed entirely by USAID under MARE. The design seemed ideal, but the system did not work. Senior research scientists were reluctant to participate, and the economists needed for diagnostic work on farmers' constraints found other jobs. (PCR para. 5.28) 5. Women 's Programs. This was a highly applauded USAID component. The program, national in scope, with six years of USAID support, provided extension and credit to many women farmers previously deprived of these facilities. It changed the emphasis from almost exclusive attention to home economics to equal attention being paid to agricultural production activities. In Mzuzu ADD alone the number of women's extension clubs increased from 175 to 530, and the number of women who received seasonal credit increased from 3,500 to 14,500. (PCR paras 5.16 and 6.9). Unfortunately, few of the women were able to repay their loans and are precluded from participation in the new, Bank-supported, credit program, which is being strict about old defaulters. 6. Communications Assistance. The Bank had originally intended to finance this component, but turned it over to USAID. There were two parts: assistance at MOA headquarters to the Agricultural Communications Branch (ACB), and management of the Pilot Communications Program (PCP) at Mzuzu ADD. The technical assistance among other impacts Annex B 38 gave ACB a much strengthened desk top publishing capacity. The PCP was effective in the areas within the few Extension Planning Areas included in the pilot (a total of 16 blocks), building on participatory approaches to identifying extension messages. However, training and operational costs were high and, due to the ADD budget shortage, the pilot was neither expanded during the project's lifetime nor extended after USAID's support terminated. (PCR paras 5.17 and 6.10). 7. Monitoring and Evaluation. In the eras of IRDP and early NRDP, Malawi's M&E system was renown throughout the development world. That luster has faded. The system is coordinated by a Section in the Planning Division. Each ADD has a small M&E Unit at headquarters, supported by two groups of enumerators resident in the field. The larger group collects agricultural data, the smaller one collects market price data. Mzuzu has a headquarters establishment of five professional posts, including clerks responsible for field supervision, and groups of 19 and 2 enumerators. Crop cuts, routine in earlier years, were abandoned in 1987. The enumerators now make pre- and post-harvest estimates based on visual inspection and farmer recall. There is very little analytical output from the system, although for the 1992/93 crop year the ADDs produced comparative studies of extension "impact" that were aggregated and summarized by the Section at MOA headquarters. The studies were being repeated at the time of the audit mission in mid-1996. The descriptive material is neither valid as measures of attributable impacts, nor rich in insights on the factors determining success and failure in extension.2 But the tabular material provides a large body of evidence on the progress of the T&V program, such as numbers of and attendance estimates for Blocks, Block Groups, Block Gardens and OFDs. The 1994 reports were the ones that revealed the lack of momentum in the OFD program, and the large number of so-called contact farmers who did not have OFDs (para. x.xx). That national report, and the one specific to Mzuzu ADD, were very useful inputs to this audit. But the Bank expert assistance has been disappointing. One advisor contracted under NRDP III was terminated a year early and replaced by the expert recruited under MAEPS. The Bank described the latter's work as "unimpressive" and his request for an extension was denied. 1. "Extension Monitoring Survey, 1992/93, Vol. 1: National Report", Central Monitoring and Evaluation Unit, Planning Division, September 1994; and "Agricultural Extension Monitoring Survey Report for 1992/93," Monitoring & Evaluation Section, Mzuzu ADD, June 1994. 2. The PCR says: "lack of recent impact evaluation makes it impossible for the Borrower and the Bank to judge the merit of ongoing extension work" (PCR, para 13.1). IBRD 18833 32 .70T 00A ES SOLAAM34 MALAWI AGRICULTURE EXTENSION AND PLANNING SUPPORT PROJECT Korongo 0* MZUZU ADD - OUNDARJES OF AGRICUlTC-PAL DEVELCPMENT DLVISICNS (A D D I ADD HEADQlARTERS M - -MAIN ROADS gon Khondowe ¯'----,- RAILWAYS Chiwet -- RIVERS -- - INTERNATIONAL BOUNDARIES 0 /;0 0u-y. 9Nhhoto Boy • Z A M B I A CHIZUMULU ISLAND M O Z A M B QL E NLIhotokoto LScgng Chipata -LILONGWE Monkea OtBt, Dedz Ncheu - 1Non/a go Ksupe M O Z A M B IQ U E Z0kMA Cleka hrdzl ThLsNmap has bee preard by BLANTR Lmbe Te & Tn k 0s stat e s -ANZI N - C-,ooeoo &- e I. ro, reaEder ad - secuwt o h M ulanA eet ~nt km t ss-fT e wol a AZ 'IU usd,nNtebongnsshw 5 .. of ay te tory-r'an NkAMIBIA \ZMABE r orSWANA< / SwAZ*LAND 0 20 40 60 80 100 S TH AFCAIMILES 0 20 40 60 16 1 0   IMAGING Report No.: 16512 Type: PPAR