Malawi - Lilongwe Agricultural Development Project

CONFIDENTIAL Report No.75t1&AT. INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT INTERNATIONAL DEVELOPMENT ASSOCIATION PROJECT PERFORMANCE AUDIT REPORT on LILONGWE LAND DEVELOPMENT PROGRAM PHASE I (CREDIT 113-MAI) May 23, 1975 Operations Evaluation Department   PREFACE Credit 113-MAI of February 1968 was closed in August 1972. This paper reports on a performance audit of achievements under this credit against the objectives on the basis of which it was approved, as described in the Bank appraisal report dated January 3, 1968. This performance audit is in part an outgrowth of previous research work done on the Lilongwe Land Development Program (LLDP). The program was included among the case studies in the Bank's Africa Rural Development Study, and a follow-up study has also been carried out for the Rural Development Division of the Development Economics Department. The complexity of the project and the existence of these exten- sive studies have affected the scope of the present report. While all the main features of the 1968 project were subjected to performance audit and are touched on here, there is very little elaboration in this report on some important aspects - such as institutional arrangements, training, trends in use of new inputs - which are fully covered, along with others, in the larger reports; while, on the other hand, quite disproportionate space is given here to the LLDP's Evaluation Unit, not very much covered in the other reports, and related questions. While such units have been quite common features of more recent IBRD/IDA-assisted rural development projects, the LLDP one was among the earliest to be established. Thus the report gives essentially an overview of the 1968 project and a fuller assessment of the performance of the Evaluation Unit. The valuable assistance provided by the LLDP staff and the Malawian authorities is gratefully acknowledged. Currency Equivalent (Malawi Kwacha) 1966-71: MK 1.00 = US$ 1.20 1972: MK 1.00 = US$ 1.30   PROJECT PERFORMANCE AUDIT REPORT LILONGWE LAND DEVELOPMENT PROGRAM PHASE I (CREDIT 113-MAI) TABLE OF CONTENTS Page No. Summary i I. Introduction 1 II. Project Implementation 3 Administration and Staffing 4 Costs and Accounts 5 Land Development 6 Land Reorganization and Registration 7 Marketing and Storage Facilities 8 Extension, Supply and Credit Services 8 III. Impact of the Program in Production Terms 9 Yield Objectives 10 Acreage Objectives 13 Rate of Return Estimates 15 IV. The LLDP Evaluation Unit 17 V. Conclusions 21 Annex 1: Description of the Project Map: Malawi - Lilongwe Land Development Program - Phasing of Road and Conservation Works   SUNMARY The Bank first became involved in the planning of the Lilongwe Land Development Program (LLDP) in 1965. A first IDA Credit, 113-MAI of February 1968 ($ 6.0 million), was extended to finance 85% of total esti- mated costs of a Phase I four-year project. The major objectives of the Project, and of the larger 13-year Program, were to increase agricultural productivity in a large area of high potential in central Malawi and to further the transition from a traditional to a market economy. Justifica- tion was based on rapidly growing population pressure on the land base and the threat of food shortages. The significance of the project for Malawi lay not only in the stated objectives but also in the fact that LLDP was the first ID)A-funded agricultural project to be located in the country. The experience of designing LLDP and putting together proposals in integrated project form have helped to establish within the Malawi Government a planning competence which has led to two further agricultural programs (Shire and Karonga) established with IDA funding and other projects assisted by bilateral lenders,as well as to two further phases of the Lilongwe Program; Ph-iase II being supported by IDA Credit 244-MAI of May 1971 and Phase III by a further IDA Credit approved by the Executive Directors in March 1975. LLDP has evolved substantially in scope and objectives through these various phases, and one characteristic of it has been rather strong and flexible management, adapting to changing circumstances. In 1969 planning started for expanding the coverage of the full'Program from the originally envisaged 500,000 acres to over 1 million acres, accounting for about 10% of Malawi's rural population. Though not originally intended as a model of a particular approach to rural development, LLDP has evolved in objectives and has with the passage of time become a showpiece for package-style integrated development; Mason and Asher, in their recent volume on the Bank, "The World Bank Since Bretton Woods", treat LLDP at some length as the prime example of one of the Bank's two principal strat- egies for rural development. These changes make performance audit of the Phase I project itself more difficult. The project supported by Credit 113-MAI consisted of land development for a first portion of the Program area, land demarcation, construction of marketing and storage facilities, and provision of exten- sion services, credit and inputs (mainly fertilizer and improved seed) for farmers of the project area. Special conditions to the IDA Credit em- phasized the importance of adequate program management and staffing and called for creation of a permanent Evaluation Unit at Program headquarters. Planned land development works - mainly roads, drains and boreholes - were completed early, in September 1971, and in significantly greater quantity than forecast, to service 280,000 acres rather than the planned 163,000 acres, as a result of use of capital-intensive construc- tion methods under efficient, expatriate management. Construction of markets was also completed on schedule, a training center was built, and a large  - ii - number of staff houses were constructed. The accelerated pace of devel- opment did not apply however to other elements of the project, with the result that faster-than-expected expansion in physical terms led to some loss of coordination with other components. The impact was particularly felt in those project sections - credit and extension, and training - where the quality or number of staff were the most important constraints to a better performance. Throughout Phase I it proved difficult to achieve desired staffing levels. Expatriates, expected to fill management posi- tions at first, could not be attracted -in adequate numbers due to unsatis- factory Government contracting procedures and low salaries, and the few qualified Malawians were discouraged by Civil Service procedures as well as low salaries. As a result, many high- and middle-level positions were left vacant for extended periods of time, and a large number of lower- level posts originally intended for expatriates had to be filled by Mala- wians who had little practical experience. The ratio of extension workers to farmers was greatly increased over the average level in the country but did not reach the planned intensity, over the expanded area developed. In the area of credit and distribution of farm inputs staff numbers were generally as planned, but the Farmers' Marketing Board (FMB), a semi- autonomous national agency meant to assume responsibility for credit and inputs, partially failed to perform, forcing management to internalize these functions within the Project and thereby seriously overburdening existing staff. Actual costs are not fully comparable with estimated, due to incompatibilities between the Project's accounting system and Government and IDA reporting categories. However it appears that total costs of the works and activities described above were less than expected - due principally to the staff shortfall and to economies in project works. By means of substantial advance procurement for Phase II the IDA Credit was brought to full disbursement in August 1972, four months ahead of original schedule, and total Program expenditures over the period were close to the origi- nally estimated $7 million. As regards progress towards the major original objective of Phase I of LLDP, to increase maize and groundnut productivity through higher yields and expanded acreage, the data available, for the first five years of the Program through 1973/74, do not show broad impact so far, but this may be partly a result of poor weather in particular years, especially 1972/73. Among the most important parameters, maize yields have shown no sustained increase and groundnut yields declined from 1969 through 1972, recovering somewhat in 1973 but remaining much below appraisal expectations. The reasons for the poor groundnut performance are not clear but it appears generally to be considered that, even though prices to the farmer rose more for groundnuts than for maize between 1969 and 1972, nonetheless farmers concentrated their main attention on maize, a food staple with yields that could be increased more easily than in the case of groundnuts. Original targets for yields are still being maintained, and it is hoped that factors such as the further increase in producer price for groundnuts in 1973, the increased extejlsion effort with women who, with children, do most of ,he work on this labor-intensive crop, and development of a more disease- resistant variety will reverse the past trend.  - iii - It is not possible to make a reasonable reassessment at this stage of the economic return to the project investment. This was origi- nally estimated at 9% for the project, if not extended beyond 1972, and 13% for the whole 13-year program. Offsetting the apparent yield shortfall are export prices for groundnuts and maize substantially above projected levels; and among nonfarm inputs, which have anyway been used in rather lesser quantity than expected, only fertilizer appears to have risen substantially more in price than the crops produced. On the other hand there is some evidence of yields growing better in areas outside the Program area without benefit of direct Program assistance. More generally, there is a problem of considerable uncertainty about both baseline and present data, particularly acute for the composite estimates crucial for cost-benefit analysis such as gross production, the product of total cropped acreage, proportion devoted to the particular crop, and average yield for that crop, each with its own degree of uncertainty. The number of families basically affected by the Programn seems to have expanded commensurately with the increase in area developed, and the credit program has been extended to reach substantially more farmers than originally planned. Some of the most important effects of LLDP may be resulting from fundamental changes which necessarily take time to accom- plish and to affect behavior - for instance registration of family land holdings, which proceeded cautiously, and the involvement of large numbers of farmers in the Program's elaborate committee structure. An increasing amount and variety of self-help activities are becoming visible in the area. Staff training - both formal and on-the-job - has made a major con- tribution towards the creation of a large nucleus of experienced Malawian agriculturalists and low- to middle-level administrators. The Program also seems to have had a useful impact in helping to make certain Govern- ment procedures more flexible, particularly in procurement, expenditure authorization, and civil service regulations. The LLDP Evaluation Unit has carried out a large amount of survey work, produced a sizeable number of useful reports and played a useful role in support of Program management, but it has suffered from a late start, serious staffing problem at the supervisory level for extended periods, some diversion from its main evaluation role, and some tendency for data` collec- tion to get ahead of analysis of data already collected. While there were factors that made it imperative to start the Program early, the lag between project initiation and full operation of the monitoring unit encompassed most of the four-year project implementation period, precluding improvements to the Program as a result of the unit's work in the course of execution of Phase I or between PhaseI and II. Because of inadequacies in the Pro- ject's accounting system, the Evaluation Unit had to perform cost analysis work which was outside its proper sphere of operations. The farm manage- ment data, on collection of which most of the funds originally allccated for evaluation were spent, have not been analyzed - although they might help to explain phenomena such as the decline in groundnut yields - and the collection effort was abandoned because of the small scope that farmers in the project area were felt to have for changing cropping patterns and im- proving productivity through reorganization of resources. There appears to be a need to do evaluation work on the use of credit and the effectiveness of extension work.  - iv - While the Bank made a major contributia to the preparation of this project and showed imagination and flexibility in undertaking to finance it, it is unfortunate, considering the project's complexity, that, presumably due to staff constraints, no two supervision missions contained the same personnel and supervision was in general less systematic and thorough than would probably have been useful and than project management would apparently have desired; supervision missions were also felt to lack powers, so that questions only got resolved long afterwards, from Washington. The Bank might have become more deeply involved as regards resolution of the serious staffing and accounting problems and promotion of earlier establishment of the Evaluation Unit. As regards Program design, it appears that the relatively capital-intensive construction techniques introduced, although essential for some works, efficiently applied and useful for training of operators, should have had a somewhat reduced scope and application, leaving more of the works - such as feeder roads and some conservation works - to slower labor-intensive self-help effort, with useful advantages in terms of direct income-distribution effect, more rapid development of self-help experience and training in normal maintenance. More labor-intensive methods should at least have been tested on a pilot basis. Finally, the case suggests some important lessons about the kind of infonmation that should be sought, presented and targetted about such a rainfed agricultural development project. Even with considerable expenditure on a built-in monitoring unit, strong field supervision and good techniques, the data that can be produced on actual progress is subject to a considerable range of uncertainty, which needs to be explicitly recog- nized; it is especially serious for the major composite parameters that are important in project appraisal. Overemphasis on the apparent yield shortfall which results from comparing original single-valued projections with estimated actual averages as discussed above seems to have been a factor, in 1971/72, in causing temporary abandonment of useful experiments in group credit and somewhat excessive expansion of individual credit in that year. More explicit attention could also usefully have been given to the weather assumptions underlying target values, perheps leading to target-ranges, spanning good and bad climatic conditions. Analysis of the statistical data available from Lilongwe also suggests that yield distributions are sufficiently skewed that they need to be given careful attention in any analysis of the distributive impact of such a project.  PROJECT PERFORMANCE AUDIT REPORT LILONGWE LAND DEVELOPMENT PROGRAM PHASE I (CREDIT 113-MAI) I. Introduction Bank involvement with the Lilongwe Land Development Program (LLDP) began during 1965 when the Government, emphasizing the need for development in the country's central regiont requested planning assistance from the Bank's Permanent Mission in Eastern Africa (PMEA). The planning, largely completed in 1966, encompassed an area of 500,000 acres in the Lilongwe Plain. PMEA, and subsequently the newly established Agricultural Development Service (ADS), was to prepare the development program in a manner suitable for Bank project financing. The resultant rainfed land and agricultural development plan was first presented to the Bank in November 1966. The area to be included in LLDP was about a third of Lilongwe District (see map). The rural population was estimated to number some 190,000 with a forecast growth rate of 3% per annum, and an existing density of about 250 persons per square mile, almost twice the national average. The number of holdings was put at 38,000, with an average of six arable acres each. Production methods for food crops were largely traditional, but tobacco growers utilized substantial quantities of nonfarm inputs. Though physically reasonably homogenous, the proposed project area supported holdings of widely varying size and productivity. Under customary rights of usufruct, each village controlled a given land area and a matrilineal succession to cultivating rights prevailed, though the first signs of erosion of this system had appeared. Marketed surpluses of crops had declined during the Federation period to the mid-1960s, but were beginning to show signs of recovery. While the human and agricultural potential of the Lilongwe Plai-n had already been demonstrated by smallholders farming without outside assis- tance and was considered to be among the highest in Malawi, the area faced, in the view of local officials, two principal problems, both stemmFing from the same source - rapid and uncontrolled population growth, which made it imperative to preserve and enhance this potential. The first probllem was that high concentrations of population in certain locales were causing rapid deterioration of the natural vegetation cover, with the threlat of serious erosion. The second problem was that population growth was in- creasing the pressure on the land base and reducing holding and plpt sizes, thereby forcing farmers back toward a subsistence position. Given the background of the program area, the major plan objec- tive (as originally conceived) was to raise agricultural productivity, by increasing yields and making more effective use of all arable land, for the region's major crops -- maize, groundnuts and tobacco -- and to accelerate the transition from a subsistence to a market economy. Targets were set for a twelve-year period and total cost was estimated at US$ 16.25 million  -2- equivalent. The program, originally submitted in its entirety for single project financing, suffered little alteration in content prior to the Credit Agreement. The Bank's major objection to the proposal was the ex- tended implementation period, for which it recommended phasing. Lesser objections were raised with regard to the lack of detailed data in support of forecast production targets, and the Bank requested further information based on the outcome of a UK-financed pilot project begun near Lilongwe in 1965. For other supportive activities, notably land registration and credit services, the Bank requested clear delineation of implementation procedures. A final important recommendation of the Bank was the crea- tion of a new administrative body within the Department of Agriculture, the Agricultural Development Branch, to assume responsibility for all major agricultural development schemes undertaken by the Government. The Lilongwe Program was to be administered by this new body. Project appraisal took place during May 1967 and the resultant development Program and Phase I Project are described below. The Program, then expected to be implemented over a 13-year period, in three phases, consisted in technical assistance in conjunction with simple infrastructural and working capital investments. The targets of the LLDP were: 1. Completion of detailed land use plans, provision of crop extraction roads, construction of soil conservation works, water supply facilities and markets for 500,000 acres. 2. Development of 60,000 acres that were idle and improved use of 40,000 acres lying fallow. 3. Provision of credit and land registration services. 4. A tenfold increase in the marketable surplus of maize; a doubling of groundnut production; and a constant production of tobacco, but decreasing by half the area of cultivation. The estimated cost of the 13-year program, as projected in the Phase I appraisal, is shown below: US$ million equiv. Buildings, Equipment, Machinery 4.397 Staff and other Recurrent Costs 9.847 Farm Inputs 6.967 21.211 The Phase I Project, for which IDA financing was proposed, consisted of six parts, detailed in Annex I: a) Land Development (soil conservation and drainage works, roads and boreholes) for 163,000 acres;  -3- b) Land Reorganization and Registration (establishment of Land Survey and Land Registry Offices in Lilongwe, and aerial topographic and cadastral surveys); c) Marketing and Storage Facilities (11 markets and an 8,500 ton crop storage facility); d) Extension, Supply and Credit Services; e) Provision of Program Management, Administrative Services and Staff Management, including an element for in-service training; and f) the review and evaluation of progress achieved under the Project and the preparation of detailed plans for subsequent phases of the Program. For implementation purposes, the area was divided into development units of about 20,000 acres each. Land development, construction of storage and marketing facilities and provision of extension, credit and land reorgani- zation services were organized around these units. The project was to be implemented over a four-year period at an estimated cost of US$ 7 million equivalent, of which half were for capital costs, 44% for recurrent expendi- tures, and the remainder for farm inputs, including fertilizer and implements. Credit negotiations took place in November 1967. The Agreement was signed on February 5, 1968. The Credit, 113-MAI, in the amount of US$ 6.0 million, was to cover 100% of foreign exchange costs of Phase I of the Lilongwe Program and, given Malawi's financial constraints, alpprox- imately 86% of total project costs. The project was to be implemented betweeen 1968 and 1972, with the closing date set at December 31, 1972. Special credit conditions and covenants included Bank approval of: appointments to the positions of Program Manager, Deputy and Assistant; procedures for on-lending to the Farmers' Marketing Board (FMB), a semi-autonomous agency assigned by statute to purchase all marketable produce grown on smallholdings and, under the project, responsible for implementing all marketing and credit services, including supply and distribution of inputs. A final covenant called for independent annual auditing of project and FMB accounts. Supplementary letters covered procurement, to be carried out through international competitive bidding for larger items; administration of the credit scheme; staffing needs; and establishment of three special units responsible for training, accounts and evaluation. A single condi- tion for Credit effectiveness called for agreement on the amount of credit proceeds to be on-lent to the FMB. II. Project Implementation Project implementation was generally expeditious and flexible. The Credit became effective on March 15, 1968 and, although disbursements  -4- lagged initially because of procedural problems connected mainly with the novelty of this type of project, the Credit was finally closed, fully dis- bursed, in August 1972, four months ahead of original schedule. Land development works, covering nearly 280,000 acres compared with the origi- nally planned 163,000 acres, were completed ahead of schedule, by September 1971. Other project components, however, did not follow the accelerated pace of development so that for a certain period items were out of phase with one another. Largely the difficulties encountered stemmed from per- sistent problems in recruiting and retaining qualified staff. Delays were experienced primarily in the extension services and in training and evalua- tion. Major changes made during implementation included the hiring of Malawians instead of expatriates for certain field positions, alteration in construction specifications for markets, deletion of construction of a project headquarters, and inclusion of the first steps on a ranch for producing feeder stock. A major change in the Project's planning horizon occurred in 1969 when preliminary docu.ents for Phase II proposed expanding the Program area from the originally envisaged 500,000 acres to over one million acres, a proposal that was eventually accepted by the Bank. IDA Credit 244-MAI, in the amount of $7.25 million, was signed in May 1971, in support of a Phase II Project, which was intended to extend land development and program activities over a further 240,000 acres; this target has again been ex- ceeded, and it was expected that by about mid-1975 a total of about 660,000 acres will have been developed under Phases I and II combined. A further IDA Credit, in the amount of $8.5 million, was approved by the Executive Directors in March 1975, to assist execution of a Phase III Project extending program coverage to a further 210,000 acres. Additional to these smallholder areas directly developed by the Program are the 160,000-acre Dzalanyama Ranch and other smaller areas, bringing aggregate area involved to nearly 1.2 million acres, containing over 400,000 people or nearly 10% of the country's rural population. Administration and Staffing As it emerged from the Bank's original appraisal, LLDP was to be supervised by the Agricultural Development Branch (ADB) of the Department of Agriculture through the Chief Agricultural Development Officer (CADO). This new oranch was established to supervise all major development schemes, to coordinate procurement and reimbursement of expenditure, and to lay down priorities for recruitment, building, etc. The ADB, which was viewed by management as an effort to restrict project autonomy, was funded through the Credit for LLDP. Originally the ADB was to consist solely of the CADO and an assistant, but the Branch aggravated LLDP staffing problems in that it was staffed by building in posts which had been intended as Project posts. Altogether five senior posts, including finance, credit, adminis- tration and personnel officers, were transferred from the Project to ADB and, while paid by the Project,failed to perform adequately their functional duties in respect of LLDP. The ADB was ultimately judged a failure and was disbanded before Phase I ended. Responsibility for the project has therefore been mainly in the hands of the Program Manager, assisted by a divisional structure assigning  responsibility for the various major functions to be carried out and by an elaborate structure of committees, involving the farmers themselves and with Village Planning Committees at the base of the pyramid. Management has been largely in the hands of expatriate staff, mainly on short-term contracts. It has adapted flexibly to the needs of the situation, taking over at various times, as necessary, functions that should have been ful- filled by other agencies cut temporarily could not be, and developing and elaborating the LLDP concept. Senior staffing has been a problem throughout the Program, but particularly during Phase I execution. Difficulties in hiring expatriates resulted from insufficiently attractive Government contract terms, low salaries and a lack of "topping up" arrangements. Delays in the Govern- ment's recruiting procedure were also common, and key posts were left vacant for up to a year or more, although by 1972 all high-level positions were filled. Recruiting was equally difficult in the case of qualified Mala- wians due to Civil Service procedures, higher salaries in the private sec- tor and a real shortage of qualified personnel - particularly as the various other major agricultural development projects got under way in the country. During Phase I only one of the four management trainee positions was filled, and the man so trained had to be transferred before long to another project. Even the more junior extension staff could not initially be recruited in the numbers desired, partly due to the Government recruitment policies mentioned above, and partly due to resistance to hiring workers who had not graduated from one of the region's two agricultural schools. At the Bank's suggestion it was agreed that non-graduates with the relevant basic skills could be trained to carry out extension work at the program's training center, operating since 1969. The shortage of extension workers has been largely overcome, in significant part thanks to the major training efforts undertaken under the Program. Costs and Accounts Actual total project costs coincided almost exactly with the esti- mated $7 million, but cost composition was somewhat different due to savings in some categories and forward spending in others. Savings arose largely from difficulties impeding the hiring of staff, both expatriate and Malawian, from parts of the land development scheme, and from the use of structures less expensive than planned for markets and some housing. With Bank approval, project savings were reallocated, largely to construction of storage units for farm inputs, forward buying of inputs for the credit program; and to initiating a livestock ranch, originally to have begun under Phase II. A detailed cost breakdown cannot be obtained from the published figures. Three different and mutually incompatible accounting systems have been applied to the LLDP cost data. Project management had its own accounting system which served a cost control function but the categories of which did not correspond to the vote subheadings in Government accounts  -6- nor to the IDA reporting categories. Thus the consolidation of project accounts and the allocation of costs to IDA headings obscured the actual pattern of expenditure because certain "activity" expenditure could fall under a number of headings. Examples include "Housing" under Category 1 expenditure (100% reimbursable), when total housing expenditure was actually divided among this item and five other items in Category 2 (75% reimbursable) and commodities like cement, which were included in almost every sub-item in each category. The Government did not recognize project accounts, and insistence upon allocating costs to often inappropriate IDA disbursement categories and classifications distorted the true pattern of expenditure. An additional element that complicated further the available cost data from Phase I was the Bank's insistence upon a strict separation between the accounts for Phase I and the following phase. Considerable savings were realized during implementation of Phase I (in the neighbor- hood of $2.0 million), but these could not be carried forward into the next phase and so large amounts of funds had to be committed in a short time in the form of forward buying for Phase II. These advance purchases in large measure explain the apparent "over-expenditure" in certain budget categories in the Phase I accounts ultimately accepted by the Bank. The principal items purchased in advance were vehicles, machinery and equip- ment, farm inputs and cement. Just as the costs of Phase I are over- stated 'oy this technique of transferring available funds forward, the costs of Phase II will similarly be understated. The lack of reliable accounts and adequate accounting procedures has been a serious problem since the beginning of the Program and explains why it is not possible to obtain a clear picture of costs and expenditures and hence evaluate the cost-effectiveness of various Program activities. To some degree these inadequacies arise because of the use of incompatible accounting systems and Government's insistence on an audit format that did not permit cost accounting by the only trained personnel available to do it. Thus the independent audits, as eventually carried out (audited accounts for 1968/69 and 1969/70 were not submitted to the Bank until late in 1970, after the matter had been drawn to the Auditor-General's attention) were still of limited use as a management tool. Program management considered it impracticable to introduce cost accounting given the lack of trained per- sonnel, but more use might have been made of the Program's training section to teach staff rudimentary accounting and cost-benefit techniques. PMEA has con- sistently been critical of the quality of LLDP accounts but has seldom urged that either the Government or the Program train the required staff. Only in 1974 was action taken when the Bank seconded a staff member to help improve accounting procedures and systems for all the IDA-funded agricultural projects in Malawi. In retrospect, it seems that the IDA-sponsored accounting system was too sophisticated in light of the staff constraints and the experimental nature oi the Program. Land Development Capital-intensive land development works, managed by expatriate staff, were carried out very efficiently. As shown below, road-building  -7- and borehole drilling exceeded targets in quantity and intensity; construc- tion of drainage works was ahead of schedule in total miles completed, but intensity was below planned, as a result of a decision to alter the stand- ards to allow for longer terraces, with a higher run-off; the original 1.5' terraces had shown early signs of erosion. More efficient use was made of heavy machinery by separating road building and conservation works and carrying out the one during the dry season and the other in the rainy season. It also proved possible to make dual use of many of the stormdrains constructed, since the earth excavated was compacted and formed a simple but satisfactory route of access to many villages. Total acreage developed, at nearly 280,000, was 70% greater than targeted. The quality of work was satisfactory, with the exception of boreholes, where ini- tial results showed the need for concrete surrounds to prevent contamination. It was agreed that the surrounds be built, but contruction proceeded slowly until 1971-72. Estimated and actual land development works were as follows: Appraisal Actual Amount/ Amount/ Amount 1,000 acres Amount 1,000 acres Roads: 165 miles 1.0 miles 642 miles 2.3 miles Stormdrains: 2,038 miles.L112.5 miles 2,271 miles 8.2 miles Boreholes: 125 .7 232 .8 Total Acres Developed: 163,000 -- 276,800 -- 1/ Derived from the Phase I Appraisal Report, Annex 1, p. 4. The average length of a drain was to be one mile and the area protected by each drain would be 80 acres. A major departure from the original project design was the in- clusion, under the Land Development Division, of a section responsible for building. It had been planned to have building work done under contract or under force account by the Ministry of Works, but neither system proved reliable enough to ensure coordination of construction with other eIlements and both were far more expensive than envisaged. With Bank approval, management internalized the building function within the project and estab- lished a construction and bridges section. Both land development and construction activities within the project have contributed significantly to developing skills that are in short supply in Malawi. These include surveying, heavy machinery halndling, building and general mechanical and-.engineering capability. Land Reorganization and Registration Implementation of this item was slower than expected mainly as a result of the inability of the Land Survey Office to attract qualified  - 8 - staff and the need to proceed cautiously with changes afEecting land tenure arrangements. The aerial topographic surveys were completed only by 1971, and the 1:5,000 scale maps, necessary for demarcation and subsequent regis- tration of holdings, were not completed until early 1972, after a qualified photogrammetrist had finally been retained. Some 13,600 acres were regis- tered to both individuals and family groups during 1972, and a much larger area had been demarcated in preparation for registration. About 120,000 acres were registered, or about to be registered, by the end of 1973. The Bank considered that registration was proceeding too slowly in view of the urgency of stabilizing holdings to prevent further subdivision, but Project management wisely rejected setting explicit annual targets for what was a novel, politically sensitive and -- to the local people -- potentially threatening program. Because of the necessity to explain the concept of land registration thoroughly to all those involved, this aspect of the program has proceeded slowly and at the same time has been very demanding in terms of staff time required. Marketing and Storage Facilities Prior to project implementation, 17 temporary markets were operating in the Lilongwe area. In the Phase I project, 11 were to have been relocated and constructed on a permanent basis. The markets were to serve as centers for extension and credit services and distribution of farm inputs in addition to providing regular marketing services. The first two markets constructed by FMB (later called ADMARC) were considered by project management to be overly elaborate and too expensive, and minor delays in construction resulted from the change in specifications to cheaper prefabricated buildings, though substantial savings were realized through the change. Construction began in 1970, and all planned market and storage facilities were completed by 1972. Extension, Supply and Credit Services Extension services were to focus on use of improved seed, fertilizer and improved cultivation methods. Introduction of the new techniques was expected to require high staff intensity, and for the first two years one extension worker was to be provided for each 200 families, some 6 or 7 times the average extension intensity in the country at the time. New, imported inputs, especially fertilizer, were to have been purchased by the program and distributed to the farmers on credit. Repayment would be made in cash, or in kind, at an interest rate of 10%. Although extension intensity was increased greatly in the area, the desired ratio of extension staff to farmers was not attained due to the difficulty encountered in recruiting staff and to the more rapid than planned expansion of the program area. Moreover, many of the extension workers were Malawians who lacked practical experience, instead of expa- triates, as originally planned. Thus, the quality of the service was also lower than expected. Extension staff have been much more successful in some areas -- e.g. in achieving almost universal adoption of contour planting and promoting the use of new inputs -- than in others -- e.g. .improving overall crop husbandry or convincing farmers to adopt improved on-farm storage.  - 9 - FMB was meant to be in charge of supplying credit and inputs to farmers but failed to perform this function during the 1968/69 season, forcing management to respond quickly and incorporate a credit aird input- supply system within the project. Though there had previously been a Government-run credit program in the area, it supplied predominantly capital inputs and reached only a tiny fraction of the farmers. In con- trast, the LLDP credit scheme emphasized seasonal inputs and was broader in scope, though less than one-fifth of eligible farmers received credit in the first two seasons due to the rigid application of creditworthiness criteria. Largely at the Bank's insistence, the credit program was drama- tically expanded in 1970/71, when it reached over 4,000 farmers, and again the following year, attaining 20,000 farmers, in order to stimulate a more rapid adoption of improved inputs. This useful expansion was achieved mainly by relaxing the creditworthiness criteria, although demand also increased because farmers became more aware of the benefits of the program, but it carried a cost in the form of higher default rates and greater demands on staff time to enforce repayment. With major effort recovery had been brought by the end of 1973 to about 95% of 197L/72 credits and 88% of 1972/73 credits. A program to provide medium-term credit was also initiated and has consistently realized a 100% repayment rate. III. Impact of the Program in Production Terms The initial overall objective of the Lilongwe program was to increase agricultural productivity on a scale large enough to have a significant impact at the national level. The appraisal report cited two factors -- rapid population growth and the imminent threat of localized food shortages -- as creating a need to intensify effort to enhance agri- cultural productivity, with particular stress on better yields oE food crops. Due to the restricted area of agriculturally suitable land, the report viewed as limited the potential of increasing production through an expansion of acreage. Furthermore, the report noted that soiL conser- vation would be necessary in many areas just to maintain the existing level of output. Special emphasis was given to the need to raise the yields of maize and groundnuts, two of the most important food crops and crops which, the report declared, are most economically grown, in MalawL on the Central Plateau near Lilongwe. Citing selected research.findings and experience in a pilot program, which had been underway'in the region since 1965, it was further concluded that practical means existed for achieving such yield increases. In the words of the appraisal report, then, "the aim of the Lilongwe Development Program is to raise agricultural production, by increasing yields and ensuring the effective use of all" suitable land, and by furthering the transition from a subsistence to a market economy". Virtually all other benefits ascribed to the program -- increases in income, export surpluses, more government revenues, etc. -- were to be generated by the direct and indirect effects of increased yields, partic- ularly of maize and groundnuts. Thus, although the scope of the program has been expanded in Phases II and III, yields are the key variable in determining the impact of Phase I of the LLDP in strict appraisa'L terms. The analysis that follows has to be interpreted in the proper context. In this audit we have been able to analyse only four or at most  - 10 - five years of the evolution of a program that is expected to last 13 years. Four years is a short period, particularly for agriculturaliprojects in rain- fed areas. Yield Objectives The appraisal report's baseline (necessarily quite rough) and target yields, together with data on acreage, production and marketable surplus, are summarized below: Marketable Acreage Yield Production Surplus1/ Crops (000's) (lb/ac) (tons) (tons) Before Development Maize 133 1,100 73,150 10,000 Groundnuts 38 500 9,500 7,000 Tobacco 47 200 4,700 4,700 After Development Maize 203 2,000 203,000 98,000 Groundnuts 66 650 21,000 17,000 Tobacco 23.5 400 4,700 4,700 1/ After allowing for on-farm consumption, seed retained and storage losses. In the case of maize the rate of storage loss was assumed to decrease slightly as a result of the program. Data appear to have been available which indicated the technical feasibility of meeting Lhe target yield for maize. The mean yield for fertilized synthetic maize in seven years of research station trials in the region was 6,800 lb/ac. Citing 51 trials on farmers' land in Lilongwe District, it was observed that the growers obtained an average yield of 3,366 lb/ac (48% of the research station yield in the same year). Thus, it was assumed that in the future the best farmers should be able to attain 40% of the research yield. Based on experience in the pilot project area, it was also noted that yields could be increased 45% (to 1,600 lb/ac) simply by switching to synthetic maize and planting early, weeding thoroughly and spacing properly but without using fertilizer. The entire yield increase of 30% for groundnuts was to stem from improved cultivation practices alone, though no research or trial data were cited to support this increase. Groundnuts were said to respond partic- ularly well to close spacing, which reduces the incidence of rosette, as well as to timely planting and careful weeding. While no new inputs were proposed, attention was drawn to the fact that there are important inter- action effects between groundnuts' need for micronutrients -- especially sulphur -- and the residues left in the soil by certain types of fertilizer used on maize and tobacco. The doubling of tobacco yields was to result from the use of balanced applications of nitrogen and phosphate fertilizer and more careful  cultivation, particularly tbe proper techniques for raising tobacco seedlings in nurseries before transplanting them into the fields. The appraisal report specifies only baseline and target yields and does not trace out in detail the growth path that yields must follow to reach the target level. Only certain assumptions are made explicit. For example, it is assumed that 10% of all growers will refuse to adopt any changes, but that five years' exposure to intensive extension campaigns will induce 90% of farmers to use improved seed and 80% to improve their cultivation methods. The expansion of fertilizer use is projected to go rather slowly and only 45% of the maize crop is expected to receive ferti- lizer in the twelfth year. The only possible method of estimating annual yield targets, therefore, is to use the figures for before- and after- development set out above to derive implicit growth rates. The figures imply the following annual compound growth rates over 13 seasons: Percent Maize -- Acreage 3.3 Yields 4.7 Production 8.2 Marketable surplus 19.0 Groundnuts -- Acreage 4.3 Yields 2.0 Production 6.3 Marketable surplus 7.1 Tobacco -- Acreage -5.5 Yields 5.5 Production 0.0 Marketable surplus 0.0 In order to get some indication of how the program is going, compared with expectations, these compound growth rates have been used to estimate annual yield "targets" for maize and groundnuts"1, for comparison with "actual" reults as derived from the Lilongwe Evaluation Unit's annual surveys.- Table 1 gives the figures. From an inauspicious begin- ning in 1969/70 (an exceptionally bad season for this crop), maize yields in- creased the following year to 90% of the "target" figure and increased again in 1971/72 to 97% of the calculated "target" figure for that year. 1972/73 was exceptionally dry and 1973174 exceptionally wet; results were 1/ Aside from stating what the yield targets are for tobacco, there is little more that can be said about yields for this crop. The LLDP Evaluation Unit, despite repeated efforts, has been unable to overcome serious methodological problems that stand in the way of devising a satisfactory technique of measuring yields of cured leaf per acre. 2/ See below for discussion of the confidence limits of these figures.  - 12 - disappointing in both. For the five years for which estimated actual yields are available, then, the mean yield achieved is 85% of "target". The picture Table 1 Lilongwe District. Projected and Actual Yields for Maize and Groundnuts Maize Yield (lb/ac)11 Groundnut Yield (lb/a_ Year Season Projected Actual- Projected Actual- (1) 1966/672/ 1,100 __ 500 __ (2) 1967/68 1,152 -- 510 -- (3) 1968/69 1,206 -- 520 -- (A/P) (A/P) (4) 1969/70 1,263 1,028 (81%) 531 533 (104%) (5) 1970/71 1,322 1,196 (90%) 541 519 (96%) (6) 1971/72 1,384 1,343 (97%) 552 483 (88%) (7) 1972/73 1,449 1,161 (80%) 563 241 (43%) (8) 1973/74 1,517 1,172 (77%)3/ 574 403 (70%)3/ (14) 1979/80 1,998 -- 647 1/ Mean Yield for each evaluation area (EVa) weighted by the number of observations. The EVa's used for each year are as follows: 1969/70 - 1-6; 1970/71- 1-5, 7, 8; 1971/72 - 1-4, 6, 7; 1972/73 - 1-4, 6-8; and 1973/74 - maize 1-6; groundnuts 1-4, 6-8. 2/ Assumed to be the base year. The appraisal report does not spell out how the base yields used were derived nor to what seasons they applied. 3/ Latest LLDP estimates. Groundnut yield is comparable with previous sea- sons, but maize yield is derived from a somewhat different and slightly smaller area than in previous seasons. Source: LLDP Reports, Nos. 8 and 11, and data supplied by the LLDP Evalua- tion Section. is similar for groundnuts,with calculated mean yield over the five seasons averaging 80% of "target", but, even before the difficult climatic conditions of 1972/73, yields showed the reverse pattern of that in maize -- a gradual decline, both absolutely and relative to "targets". The reasons for this  - 13 - disappointing trend are not clear. The average price received by the farmer for groundnuts rose significantly more between 1969 and 1972 than for maize - 26% against 8%. It appears to be generally assumed that farmers have nonetheless given higher priority to maize, because yields can be increased more easily and because it is one of the main staples. Secondary factors may be disease problems with groundnuts and the fact that the Program had difficulty in the early years in developing agricul- tural training for women - who are particularly important, along with children, in the much more labor-intensive groundnut crop. There is no basis to evaluate the relative importance of these factors. It is hoped that the sharp increase decreed in 1973 in the farmer price fcor groundnuts may serve to increase interest in that crop, and the same ultimate yield targets for both maize and groundnuts are still being maintained. Acreage Objectives Though the potential for increasing production through expansion of acreage was considered limited the appraisal report contained various explicit assumptions regarding changes in cropping patterns and total acreage cultivated. These changes are shown in Table 2. The post- development objectives given in the table assume only one phase for the program and therefore depict changes anticipated as a result of only four years of intensive development effort, succeeded by a longer follow-up period.  - 14 - Table 2 Lilongwe Program. Anticipated Changes in Land Use During Phase I Pre-development Post-development Actualll (c. 1966-68) (c.1979/80) (c. 1969-71) 1. Total Area (000 acres) Non-arable 150 150 1492/ Arable but idle 80 0 1072/ Roads, drains, etc. 0 20 - Existing farm land 270 330 2652/ Total area 500 500 521 2. Cropping (000 acres) Fallow 40 0 5 Maize 133 203 151 Groundnuts 38 66 41 Tobacco 47 23 18 Other 12 38 16 Total cropped area 230 330 226 3. Population Farm families 38,000 53,560 40,626 Cropped land, acres per farm family 6.1 6.2 5.6 1/ Figures under "Total area" are based upon random transects across serial photographs taken in 1966-67. Figures under "Cropping" are derived from the mean area planted to different crops during the 1969/70 and 1970/71 seasons. 2/ Of the total area, 44% is cultivated. Of the area uncultivated, 51% is non-arable and 49% is potentially arable. Of the potentially arable land, 25% has been previously cultivated and is included here as "Existing farm- land" while the remaining 75% is included as "Arable but idle." Source: From appraisal report, Annex 1, Table 2. Figures representing the actual situation are derived from the proposals for Phase III of the Program and data supplied by the LLDP Extension Section. The appraisal report indicates that through the net addition of 9'0,000 acres of land formerly idle, existing farmland was to increase by 22% to 330,000 acres. Maize acreage was to expand by 53% to 203,000  - 15 - acres, groundniut acreage was to increase by 74% to 66,000 acres, and tobacco acreage was to be cut in half. The acreage of other crops was anticipated to triple. The combined effect of these various changes was to be an increase in cropped area of 100,000 acres, or about 45% more than the pre- development acreage. The newly cropped land was to come from the complete elimination of fallow land, the cultivation of 60,000 acres of arable land previously idle, and the reduction in acreage planted to tobacco; and the area cropped per family was to increase slightly despite growing population pressure on the total land area. The appraisal projections indicate an 8% increase in maize acreage per holding and a 23% increase in groundnut acreage. Whereas the projections made for yields had their basis in agronomic research, there was no empirical basis for assuming that cropping patterns and acreage could be adjusted in the manner suggested by the appraisal report. A possible constraint to attainment of the acreage projections is land availability. The appraisal projections indicated that the "average" holding would plant 1.2 acres of groundnuts and 3.8 acres of maize, for a total of 5.0 acres under these two crops. Actual figures for 1969-71 indicate that the average holding was planting 1.0 acre of groundnuts and 3.7 acres of maize. However, in 1971-72, for the combined Phase I and II area, the mean cropped acreage per holding was 4.55. These 1971-72 data also show that the median holding size is below the mean and that approxi- mately one-half of all holdings plant less than four acres to all crops together. Thus, even leaving aside consideration of tobacco acreage or the area planted to other food crops, it appears that the average holding may be experiencing a land constraint which will make it difficult to attain the projected acreages of maize and groundnuts. Another bottleneck to the expansion of groundnut production is labor requirements, which are much greater for them (both for actual cropping and for pre-marketing processing) than for maize. The appraisal took account of the tillage/cultivation bottleneck and proposed medium- term credit for the purchase of draft oxen and implements. If suitable equipment were available for lifting groundnuts, the use of oxen would very considerably reduce the significance of this bottleneck on the larger holdings. The report notes that a team of oxen and a set of implements is sufficient for 20 acres but fails to point out the fact that draft oxen will be uneconomic on most holdings. It is possible, therefore, that the expansion of groundnut acreagel/ permitted through breaking this bottle- neck would occur chiefly on the larger holdings. This in turn could1 have a further unfavorable income-distribution effect by inducing the hiring, at low wages, of large numbers of people for shelling (a process it has not yet been possible to mechanize effectively) at times when they should be giving attention to their own small farms. Rate of Return Estimates It is not possible to make a sound reassessment at this stage of the projected economic return to the project. The 1968 appraisal report calculated economic rates of return for Phase I on its own, in case the project were not extended beyond 1972, and for the whole 13-year program; 1/ The Phase II appraisal projects a larger expansion of groundnut acreage, to an "average" of 1.6 acres per holding.  - 16 - the estimates were 9% and 13% respectively. A major offsetting factor to the important yield shortfall that seems to be occurring so far is that world market prices for both maize and groundnuts have risen more than either the general level of prices in Malawi or the prices of nonfarm in- puts other than fertilizer, which appear anyway to have been applied in lesser quantity than expected. The general levels of prices in Malawi, and of its imports, appear to have risen about 30% between 1968 and 1973-74. But the average export price received for maize since 1968 has been about K57 a ton, some 68% above the appraisal report's long-term projection of K34 a ton, and current Bank projections indicate a price of some K47 at present falling to perhaps K38 (in terms of 1974 prices) towards the end of the current decade. Groundnut export prices were only about 15% above the appraisal report projection of about K140 a ton (ex Limbe) for 1968-71, but in 1973/74 they rose to about 166% of projection, and curffent Bank forecasts sliggest their declining not below about K170 a ton (in 1974 prices) by the end of the decade. But there are also some negative factors and, in general, consid- erably too much uncertainty about the real validity of the baseline and estimated actual data to warrant a sound judgment on the likely return to the investments so far made. Data collected, by the Program staff give some indication that, at least during Phase I, each year's expansion of program acreage embraced areas of higher productivity and that yields out- side the program area have been growing at rates of some 5-20% without benefit of direct program assistance, 'so that there would be a serious question of what might reasonably be causally attributed to the program. Volumes of produce marketed, according to the official statistics, have fallen below projections, but this may reflect only smuggling or higher on-farm consumption. As regards the uncertainty of the data available, quite apart from the considerable doubts about the validity of the baseline data, it must be remembered that each statistic cited is subject to a certain margin of error so that a composite figure such as overall production - crucial for any cost-benefit calculus - is subject to the cumulative un- certainties of all component elements. Overall production has to be derived from acreage and yield data. The acreage of a given crop is it- self obtained through a chain of estimates: the program area is such, x percent of this area is cropped, and y percent of the cropped area is planted to z . Study of information available about Lilongwe in the early 1970s shows that total cropped area and the proportion planted to maize. for example, may each be subject to about + 10% margins of error, whereas Table 2 above cited only the estimated mean values for each. As regards yields (for all varieties together as cited in Table 1) the large samples taken at Lilongwe give confidence limits of also about + 10%, which is relatively high. But when all these figures are multiplied together then the resulting estimate of overall production is subject to a margin of error of as much as + 40%. Clearly these problems are compounded when calculations are based on comparison of what is ' actually' happening under the project with what would have happened without it.  - 17 - It would seem that project appraisal and supervision could use- fully haxe given more attention to the full range of yields that might be achieved instead of mainly to estimated and projected overall averages. Whi1e LLDP management has generally been very broad-minded and kept in mind all dimensions of development, social as well as economic, there is evidence that others' excessive focus on the rather uncertain trends of overall average yields, compared with targets, may occasionally have pressured management. into decisions that were not entirely wise: for instance, the temporary sacrifice of the experimentation with group credit begun in 1970/71 in favor of very rapid expansion of individual credit the following year, in the effort to get the level of inputs quickly up to that needed to permit target yields to be reached, with consequent logjams of credit cases in local courts and heavy administrative costs and burdens on project staff. The fuller distribution of yields is also important from the point of view of distribution of project benefits. The evidence indicates that yields are in fact highly skewed such that almost two-thirds of project farmers may achieve less than the mean yield. Typically the modal yield is one- third to one-half the mean, while the median may be 20% or more below the mean. As regards the number of persons affected by the Program, avail- able figures suggest that this has been at least in line with original projections (see,for instance, Table 2) and probably substantially above - both as a result of the more rapid than expected expansion of land develop- ment works (where all the farmers would at least benefit to some extent from the better roads and drainage provided) and as a result of the more ambitious than originally planned credit program. IV. The LLDP Evaluation Unit The Evaluation Unit in the Lilongwe Program was established, according to the appraisal report, with the following terms of reference: A small unit, headed by an agricultural economist, with a staff of enumerators will conduct surveys to determine the effect of changes and the effectiveness of techniques employed in the Program... (and) ... to study the effects of the Program on the individual farmer. The main pur- pose is to provide information for updating the present program but the unit will also produce data of use in planning other projects else- where in Malawi... (and) ... for the IDA review... The Evaluation Unit was allocated for Phase I $135,000 -- 1.9% of total project costs, 7.6% of total administration costs, and 12.4% of management costs. The final terms of reference provided in the appraisal report are somewhat less specific than the original proposals contained in the application for funding which placed more emphasis on practical application in farm management terms: "The information ..... colliected will... .form the basis for advice to the extension staff as to the recom- mended pattern and distribution of crops that should be followed on the  - 18 - various sized holdings and produce concrete evidence as to the effects of the different measures carried out under the Project". By about mid-1974 the Lilongwe Evaluation Unit had produced a series of 13 major reports plus numerous discussion and planning papers, notes and analyses, and operational reports to management. The fairly vague and general nature of the terms of reference provided for evalua- tion at LLDP are open to the broadest possible interpretation, and it is to the credit of the evaluation staff that a crucial deficiency was recog- nized and major efforts were directed toward evaluating the reliability of the physical planning data used in the application to IDA. Only through such an assessment could the reality of projected program benefits be ascertained. Given the dearth of objective information about small- holder agriculture in the Lilongwe area, the Evaluation Unit also broadened the scope of its investigation to include a detailed socio-economic study of the target population. When the actual work program of the Evaluation Unit was being formulated in early 1969, it was agreed that the evaluation effort should provide information in five specific categories: 1. Baseline data from the undeveloped portions of the program area so as to provide a benchmark for future evaluation work and to verify assumptions made in the initial program proposals. 2. Survey data from the areas in the process of development in order to monitor progress during the five-year phase of intensive development activities. 3. Survey data to provide information on the final achieve- ments after the completion of the intensive development phase in each area. 4. Specific data on various sections within the program in order to estimate optimum operational capacities and to provide a foundation on which proposals could be formulated for the continuation of certain sections -- credit, for example -- at the completion of the program. 5. Data on specific enterprises, such as livestock, for which insufficient data existed for planning purposes. The data required to produce these various types of information were grouped under three broad headings: (1) land classification and cropping patterns, (2) farm production and consumption figures, and (3) miscellaneous, includ- ing principally data relating to the operations of the various sections -- credit, extension, construction, land allocation, etc. -- within the overall program. Data in categories 1 and 2 were to be collected through sample surveys carried out in the field by teams of enumerators. Unfortunately, many of the ambitious and commendable objectives set out in the paragraph above were not fulfilled, or were met in such a  - 19 - way that the results are of dubious value. The various reports and docu- ments produced by the LLDP Evaluation Unit provided a large part of the input for the review of the program under the Bank's Africa Rural Develop- ment Study, and the necessity of relying upon them revealed both their strengths and shortcomings. Without the reports, a meaningful review would have been impossible and far more praise than criticism is deserved. Nevertheless, there were some important questions the review hoped to answer on the basis of the data collected by the Lilongwe Evaluation Unit which could not be answered. A first problem was that the Evaluation Unit was set up too late and that it had partially to be diverted to fill other gaps. Because of the delays associated with staff recruiting and training, the Evalua- tion Unit became operational only well after the program was underway. Due to the accounting difficulties mentioned earlier it had to give sig- nificant attention to processing of cost data which was very necessary and useful for project management but not of direct priority under the evalua- tion objectives mentioned above. The Unit produced its first discussion paper only shortly before the preparation of Phase II began and, after a major effort to assemble the proposals for Phase II (a job accomplished within abcut a year of the Unit's establishment), it turned to its first major report which appeared only a month before the Phase II appraisal report. The delay in establishing the unit, together with the normal lags in evaluation work, meant that there was very little opportunity for design of Phase II to benefit from analyzed experience under Phase I. Staffing continued to be a problem. For a period of almost a year and a half toward the end of Phase I, there was no senior evaluation officer at LLDP because of recruiting difficulties. As a consequence, the Evaluation Unit suffered from lack of direction and progress was slow. No major reports were published from October 1971 until mid-1973, and few other materials were produced during this period. Another aspect of the staffing problem relates to the quality of field staff. Experience at Lilongwe indicates that the men who ask the questions of farmers and who measure their fields must be highly trained and strongly motivated if meaningful data are to be collected. In addition, a high degree of super- vision is essential to prevent data collection from degenerating into a process in which enumerators supply answers for farmers. Such supervision is costly in terms of senior staff time. A major piece of work started by the Evaluation Unit was unfortu- nately abandoned half way through. Detailed farm management data were collected in continuous surveys over a sample of 100-120 farmers throughout a period of three years as the farmers were exposed to the program's influence. These data were collected with the objective in mind of permitting the ex- tension section to undertake farm planning work. After consuming a large proportion of Evaluation Unit resources and staff time ($80-90,000 and some 40 man-years), the farm management surveys were dropped when management decided that elaborate farm planning had little practical place in the con- text of the very small holdings existing in the area because the range of alternative cropping patterns is very limited. This decision, however, ignored the fact that there is a place for farm planning even on very small holdings, particularly in working out improved rotations and integrating  - 20 - other activities with cropping. With resources fully taxed by subsequent on-going survey work, the Evaluation Unit has never been able to analyze. the farm management data collected, despite the fact that the data are already on some 150,000 computer cards. There would be merit in analyzing the farm management data even if only on the grounds that such comprehensive surveys are extremely rare in Africa, but a far more important reason is that such analysis should be capable of helping explain several operational questions: for example, the decline in groundnut yields, the fall in cash sales of inputs asso- ciated with the credit program, or the fact that the marketed output of maize has been below projected levels. Other unanswered questions at LLDP relate to the distribution of credit funds and the use of and need for seasonal production credit. Both loans and defaults are distributed according to distinct geographical patterns, but the determinants of these patterns are unknown, though it might be possible to ascertain them from further analysis of existing data. As the program keeps no records of those who apply for credit but do not receive it, a study of the need for credit would be exceedingly difficult without further fieldwork on a substantial scale. Likewise there has been no analysis of the only data available which could reveal irmportant secondary effects arising from the project -- such as changes in income, investment and consumption -- or whether benefits have been generated according to the pattern set out at appraisal or have arisen through other processes. One survey has been carried out to deter- mine farmers' knowledge about extension recommendations, but this survey of knowledge has not been related to the actual practices adopted by farmers. Considerable work in the area of rural sociology elsewhere suggests that the gap between knowledge and application may be a sizeable one. It has been difficult, therefore, for management to establish guide- lir.es for improving the extension service. Having noted the problems and shortcomings of the Evaluation Unit at LLDP, a comment should be made on its major strength. Explicitly recognizing that rural development is a multi-dimensional process, the Evaluation Unit has consistently collected a range of sociological infor- mation despite the fact that such information frequently has little short- term operational content. Operating in this manner, the Evaluation Unit has helped to counteract the economic and agricultural bias built into the original design of the Program and to promote the recognition that the major determinant of the pace of development is the response of the people toward whom the program is directed. The Bank is to be commended for promoting the inclusion of an evaluation unit in the original program, but its interest in the evalua- tion process seems to have dwindled during implementation. Even during Phase II, interest seems to have been displayed only sporadically, mainly when supervision missions required data. The Bank did not pay enough atten- tion to the delays in setPing up the Unit nor to solution of staffing problems, and made no effort to see that conflicting objectives were resolved  - 21 - and that data of high utility were analysed. There is little doubt among professionals that evaluation units can have a worthwhile role to play in rural development projects, both in promoting more effective management and in advancing knowledge about the needs and responses of rural peoples, but the Bank did little until Phase III, to strengthen the role of the LLDP Unit. V. Conclusions A review of the developments at the Lilongwe Land Development Program during Phase I reveals that the program was extremely effective in all aspects relating to physical infrastructure development. However, numerous problems -- especially those relating to staffing -- impeded operations in other areas, and the accelerated pace of infrastructure development resulted in inadequate coordination with other parts of the program which developed less rapidly. The appraisal's principal yardstick of accomplishment at LLDP was increased crop productivity and production, but the picture that emerges from the available data is a mixed one. The data suggest that a potential exists to increase maize yields, though t'is potential has only been realized for a small number of program farmers and is subject to very considerable inter- and intra- seasonal variation. The data also indicate a decline in groundnut yields but the reasons for this decline are unclear. Methodological problems have prevented any assess- ment of tobacco yields. A range of secondary and tertiary benefits, such as favorable changes in consumption and investment patterns, was expected to result from increased production and incomes; but, although data have been collected which could reveal the emergence of such benefits, these data have not been analyzed. Finally, it has not been possible to assess the effects of the delays and shortfalls in parts of the Program, partic- ularly in the extension service and training. In short, no convincing evidence is available from Phase I either to repudiate or to confirm the economic validity of the investment undertaken. Given the strong emphasis in Phase I on the development of infrastructure, it may be some time yet before such evidence is Eorthcoming. Not only were the full benefits of the program's activities expected to materialize only over a period of 13 years but also, in this particular case, the program is very much a living and continuing organism with a management adapting the project through expanding phases to changing cir- cumstances and new knowledge. A performance audit of Phase I aLmost in- evitably fails to detect more subtle yet profound changes that may be occurring slowly, particularly for example as a result of the participation of large numbers of people in active committees or as a result of the reg- istration of family land holdings. An increasing amount and variety of self-help activities are beginning to be visible in the area. Moreover there are signs of other non-quantifiable, indirect benefits of LLDP activi- ties. For example, the project itself appears to have had a significant impact on certain Government procedures, encouraging responsive flexibility in procurement, expenditure authorization procedures and civil service regulations. Likewise LLDP appears to lhave had a positive impact on FMB staff and operations in the area. Besides formal training, the Prograrn s nonformal in-service training seems to be making a major contribution to the creation  - 22 - of a large nucleus of experienced Malawian agriculturalists and low- to middle-level administrators. LLDP also demonstrates dramatically that, when existing staff are motivated by management commitment and unhampered by cumbersome regulations, the effects of shortfalls in staff numbers can in considerable measure be offset by extra effort willingly supplied. As regards the Bank's participation in the project, it evidently played a major and constructive role in preparation, even if some of its initiatives such as the creation of the ADB failed to work satisfactorily and had to be abandoned. It showed flexibility and imagination in adapting to the inevitable lack of solid underlying data at the time of project appraisal and in promoting the establishment of the evaluation unit. But it seems unfortunate, given the novelty and complexity of this type of project, that, no doubt due to staff constraints, it dedicated relatively little effort to follow-up supervision. While the frequency of missions (of all types) from the Bank was adequate, possibly even excessive since project management sometimes complained about the time required to show people around and the diversity of opinions expressed to Government, it seems that there was scope for systematic supervision more thorough and more fre- quent than the approximately once a year attained and for more Bank assistance with the problems that arose during implementation. Project management felt, for example, that the Bank should have encouraged early and continuing debate on development of institutions to succeed the project. It would also have been useful if supervision missions had worked with a project-specific pro forma, which would have detailed items for review, actions needed and action dates. The delay between identification of a problem by the supervision missions and its resolution suggests that the missions could usefully have been given more authority to secure approval of certain types of alterations in the field and provide definitive assurances so that management can proceed with making necessary commitments. The fact that no two supervision missions included the same personnel meant that much time was lost in basic familiarization and collecting background information. Project management would also have liked to have had a sociologist included in the supervision missions, but this expertise was not available. It is likely that a closer relation with the Program would have allowed the Bank to take positive actions, for example, in helping with the Program's staff- ing problems, in asking more forcefully for better accounting procedures, in promoting the earlier establishment of the Evaluation Unit and in urging analysis of collected data. On basic project design the main question that emerges from this case is whether a less capital-intensive approach could not have been taken, with advantage not so much in financial terms and certainly not in terms of works construction time but in terms of participation, self-help ex- perience, income-distribution -- and better coordination of the various dimensions of the project. Efficient use of heavy eq.'ipment did enable more roads and drains to be built than expected, by appropriate use of the equipment in both wet and dry seasons and because bunds were wisely built as roads, and it was probably also necessary forthe more important roads, e.g. those between unit centers, which were crucial in opening up the area for the continued development effort. Training in equipment operation was also a benefit of the project. Yet there is strong opinion inside and outside  - 23 - Malawi that some works could better have been organized on a self-help basis, there has been a move over time in this direction and it would seem that the Bank could have contributed valuably by promoting this approach and its testing, at least on a pilot basis, in the project. There were works - feeder roads, some conservation works and construc- tion - that could have been carried out in a more labor-intensive fashion, taking advantage of the seasonality of agricultural production requirements. Indeed, in some cases the program actually pre-empted the activities of local people who had organized themselves to provide their own conservation works. An opportunity seems to have been missed to stimulate a powerful local commitment to the program and to promote early on a sense of local self-identity. This course would have channelled some direct benefits to the poorer people and would have helped provide them with the knowledge and skills required for later maintenance work. The concept of direct involvement of the people in major develop- ment activities on a self-help basis was not established in Malawi to the extent that it could have been included at the time when the project was conceived, in 1966-67. But it is the kind of matter on which an international agency can usefully propagate information about successful experience in other countries. The second principal point that emerges from study of this project, with its interesting early effort at built-in evaluation, is to underline the importance of such work, including its commencement at the very early 'baseline' stage, in a rather elaborate way, if usable infor- mation is to be available for judging the value of the investments under- taken and for appropriately reorienting those that follow. For primarily agricultural projects crop yields are the most important variable to monitor; to measure them is conceptually easy but in practice difficult. And even with very large (and consequently quite expensive) samples, as used at Lilongwe, the variation in yields is so great that the mean figures alone have little value and need to be complemented with weather-related ranges to be useful as targets and with much fuller detail on statistical distribution to be useful for indicating the breadth of impact the project may be having; and all such figures need to be treated as subject to a significant margin of error. These points appear to apply particularly strongly to projects in rainfed areas.   ANNEX 1 Page I of 2 Description of the Project The Project is the first four-year phase of the Lilongwe Develop- ment Program, a long-term program, for the developnent of infrastructure and the improvement of smallholder agricultural production in the Program Area. The Project consists of the following: I. Land Development. This includes, in about one-third of the Program Area: (a) the preparation of detailed land use plans and the construction and maintenance of soil conservation and drainage works; (b) the construction and maintenance of about 165 miles of crop extraction roads; and (c) the drilling and maintenance of about 125 working boreholes, equipped with hand-operated pumps, for village water supply. II. Land Reorganization and Registration. This includes: (a) the establishment, staffing and equipping of a Survey Office in Lilongwe; (b) the completion of aerial topographic surveys and the preparation of large-scale maps of the entire Program Area; (c) the initiation of aerial cadastral surveys and the demarcation, reorganization and registration of holdings in part of the Program Area; and (d) the establishment and staffing of a Land Registry Office in Lilongwe. III. The Provision of Marketing and Storage Facilities. This includes: (a) the construction and equipping of 11 markets within the Program Area; and (b) the construction and equipping of a new 8,500 ton crop storage facility in Lilongwe. IV. Extension, Supply and Credit Services. This includes: the provision of intensive agricultural extension services, fertilizer and other farm inputs, credit to farmers for such goods and the supervision of the use and collection of the  Annex 1 Page 2 of 2 credits in the one-third of the Program Area in which land development will be concentrated during the first phase. V. The Provision of Program Management, Administrative Services and Staff Training. This includes: (a) the establishment and staffing of an Agricultural Development Brarnch located in Lilongwe, within the Department of Agriculture; (b) the establishment and staffing of a Program Organiza- tion under the Agricultural Development Branch with headquarters in Lilongwe; (c) the recruitment of qualified managerial, professional, technical and administrative staff and the provision of in-service training; (d) the construction, equipping and maintenance of houses and offices for Program Staff in Lilongwe and in the Program Area; (e) the purchase, operation and maintenance of vehicles, machinery and equipment; and (f) the provision of administrative services. VI. The review and evaluation of progress achieved under the Project and the preparation of detailed plans for subse- quent phases of the Program. Source: Development Credit Agreement (Lilongwe Agricultural Development Project), February 5, 1968, pp. 12-14.