Malawi - Second Technical Assistance Project

Doan MICRC)FICH: POVY nCe WC Report N. 974t MAI Type. (1P1 ) von omtc eport No. 9743 PROJECT PERFORMANCE AUDIT REPORT MALAWI SECOND TECHNICAL ASSISTANCE PROJECT (CREDIT 1428-MAI) JUNE 28, 1991 Operations Evaluation Department This document has a restricted distribution and may be used by reciplents only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank sathorization. CURRENCY EQUIVALENTS (Period Average) Currency Unit: Kvacha (K) US$1.00 - K US$1.00 - K 1982 - 1.0555 1986 - 1.8611 1983 - 1.1748 1987 - 2.2087 1984 - 1.4113 1988 - 2.5613 1985 - 1.7191 1990 - 2.7289 LIST OF ACRONYMS AND ABBREVIATIONS ADMARC Agricultural Development and Marketing Corporation DPMT Department of Personnel, Management and Training DPU Data Processing Unit EPTD Economic Planning Division ESCOM Electric Supply Commission PAO Food and Agriculture Organization IDA International Development Association IDP Institutional Development Project ILO International Labor Organization kwh kilowatt hour MIM Malawi Institute of Management MOF Ministry of Finance NRDP National Rurs 1 Development Program OED Operations Evaluation Department PCR Project Completion Report PPAR Program Performance Audit Report PPF Project Preparation Facility SAL Structural Adjustment Loan SDR Special Drawing Rights TA Technical Assistance TRA Tobacco Research Authority UNDP United Nations Development Program SYSTEM OF WEIGHTS AND MEASURES: setric Metric US Equivalent 1 metric ton (t) 1.102 tons (t) 1 kilogram (kg) 2.205 pounds (lb) 1 cubic meter (mW) 35.32 cubic feet (cf) THE WORLD BANK FOR OFFICIAL USE ONLY Washington, D.C. 20433 U.S.A. Offere of Dirctor*GWWaI Opwatkms Ivaluatim June 28, 1991 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Project Performance Audit Report on Malawi - Second Technical Assistance Project (Credit 1428-MAI) Attached, for information, is a copy of a report entitled "Project Performance Audit Report on Malawi - Second Technical Assistance Project (Credit 1428-MAI)" prepared by the Operations Evaluation Department. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be discksed without World Bank authorization. FOR OFFICIAL USE ONLY PROJECT EEREORBWCE AUDIT REPORT MAIAWI SECOND TECHNICAL ASSISTANCE PROJECT (Credit 1428-MAI) TABLE OF CONTENTS Page No. PREFACE............................. ............... . . . .i BASIC DATA SHEET.............. ..... . . . . . . . iii EVALUATION SUMMARY................. ..... . . . vii I. BACKGROUND................ ...... . . . .1 II. OBJECTIVES AND DESIGN........... ..... . . . . 3 III. IMPLEMENTATION.............. ...... . . . . 5 A. The Energy Planning Component. ................5 B. Pilot Fuelwood Savings Program at the Tobacco Research Authority (TRA) ................... ....6 C. Study of Power Tariff for the Electricity Supply Commission of Malawi (ESCOM)..... ..................8 D. Strengthening the Financial Management of the Agricultural Development and Marketing Corporation (ADMARC) ..... 9 E. Support to the Ministry of Finance Debt Management Unit 9 F. Other Studies and Components.................10 G. Implementation and Supervision................14 IV. RESULTS........................................................15 A. Results of individual Components...............15 B. Overall Results.. ......................18 V. SUSTAINABILITY. .........................23 VI. LESSONS OF EXPERIENCE ......................23 This document has a restricted distribution and may be used by recipients only in the performance inf their official duties. Its contents may not otherwise be disclosed without World Bank authorization. PROJECT PERFORMANCE AUDIT REPORT MALAWI SECOND TECHNICAL a.SSISTANCE PROJECT (Credit 1428-MAI) PREFACE 1. This is a Project Performance Audit Report (PPAR) on the Second Technical Assistance Credit 1428-MAI. The credit, in the amount of US$1.5 million to the Government of Malawi, was approved on December 20, 1983, became effective on January 6, 1984, and was closed on June 30, 1989, two years behind schedule. The last disbursement took place in October 1989. The credit was fully disbursed. 2. The audit was prep&red by the Operations Evaluation Department (OED). The Project Completion Report (PCR) for the above project, prepared by the Africa Regional Office of the Bank, has already been issued to the Board as an OED Report No. 8958 (dated August 13, 1990). This audit is based on the PCR, the President's Report, sector and economic reports, study of the project files, and discussions with Bank staff. The audit has also relied on a case study of the First and Second Technical Assistance Projects carried out by OED in conjunction with an Evaluation Study of Free-Standing Technical Assistance for Institutional Development in Sub-Saharan Africa (April 1990). An OED mission visited Malawi and discussed the effectiveness of the project with Government officials and representatives of the business and financial community. Their kind cooperation and valuable assistance in the preparation of this report is gratefully acknowledged. 3. The PCR provides a good account and assessment of the project experience with regard to the implementation of the various action programs; achievement of the components, administration and use of the credit; and draws lessons of experience. The audit discusses the evolution of technical assistance in Malawi, portrays the critical issues of concern that led to the genesis of this credit in conjunction with structural adjustment programs; evaluates the progress made in creating and implementing the various components of the project; determines the purposefulness and effectiveness of technical assistance measures; and ascertains the key factors that determined the credit's outcome and effectiveness and sustainability of the project. The audit, then, draws further lessons from the project experience. 4. The draft case study referred to above (para. 2) was sent to the Borrower for review and comments, but none were received. - iii - PROJECT PERFORMANCE AUDIT REPORT MALAWI SECOND TECHNICAL ASSISTANCE PROJECT (Credit 1428-MAI) BASIC DATA SHEET CREDLT POSITION (amounts in US$ million) As of April 30. 1991 Ctedit Original Disbursed Cancelled Reaid Outstanding 1428 1.501' 1.671' -.- -.- 1.87 CUMULATIVE ESTIMATED AND ACTUAL DISBURSEMENTS S.6 EX.Z EIY8 Appraisal Estimate (US$M) 0.10 1.50 -- Actual (US$M) -- 1.17 1.67 Actual as % of Appraisal (%) - 78% l11%!, Date of Final Disbursement: October 3, 1989 PROJECT DATES Original &ctual Initiating Memorandum 02/83 04/23/83 Negotiations 11/83 11/11/83 Board Approval 12/83 12/20/83 Credit Agreement 12/83 12/20/83 Effectiveness 01/84 01/06/84 Credit Closing 06/87 06/30/89 1 Equivalent to SDR 1.40 million. i Actual as % of appraisal was 100% when expressed in SDR. - iv - STAFF INPUTS (staffweeks) EM5 EHA E2. EA E.2 £EQ .T.L Appraisal 3.3 - - - - - - 3.3 Negotiations 1.4 - - - - - - 1.4 Supervision 0.9 4.0 4.4 3.9 0.7 2.0 - 15.9 Completion - - - - - - 3.8 3.8 Total 5.6 4.0 4.4 3.9 0.7 2.0 3.8 24.4 MISSIONDAT No. of No. of Staff Date of Month/Year Weeks Persons Weeks Report Preparation 06/82 2 3 6 07/21/82 02/83 2 1 2 02/24/83 Appraisal 06/83 3 1 3 07/06/83 Supervision I 05/84 1 3 3 06/01/84 Supervision II 07/86 2 1 2 08/12/86 Supervision III 07/88 2 1 2 08/05/88 Completion 02/90 1 1 1 08/13/90 Note.- Separate Supervision missions for various subprojects are not included. OTHER PROJECT DATA Related Projects Borrower: Government of Malawi Project: Structural Adjustment I Loan No.: 2026-MAI Amount: US$45.0 million Approval Date: July 2, 1981 -V- Project: Technical Assistance I Loan No.: 2027-MAI Amount: US$1.0 million Approval Date: July 2, 1981 Project: National Rural Development Program Credit No.: 1183-MAI Amount: US$7.3 million Approval Date: October 13, 1981 Project: Structural Adjustment II Credit No.: 1427-MAI Amount: US$55.0 million Approval Date: December 20, 1983 Project: Structural Adjustment III Credit No. 1644-MAI Amount: US$30.0 million Approval Date: December 20, 1985 Project: Institutional Development Credit No.: 2036-MAI Amount: US$11.3 million Approval Date: December 14, 1989 - vii - PROJECT PERFORMANCE AUDIT REPORT MALAWI SECOND TECHNICAL ASSISTANCE PROJECT (Credit 1428-MAI) EVALUATION SUMMARY Introduction 1. Two associated Technical sectors (Energy, Indus.ry and Assistance Projects have supported Agriculture) and in the fields of improvements in the public sector resource rabilization, management and management, in addition to reforms in institutional development. A number the agricultural, energy, and of proposed actions were to be parastatal sectors. The US$1.5 carried out with bilateral and million Second Technical Assistance multilateral technical assistance Credit was approved by the Board in since Malawi still had a shortage of 1983 in conjunction with the Second trained personnel in both the civil Structural Adjustment Credit. It service and key private sector areas. followed on the First Technical A separate technical assistance Assistance Credit approved earlier in proje,,t was thus prepared to help 1981, also in support of a Structural implementing some of the actions Adjustment Operation. supported by the SAL (para. 2). Structural adjustment reforms ObJic.i ware envisaged to cover a number of years due to the fact that it was 4. The project aimed at assisting realized in the early 1980s that the the Government in achieving its goals adjustment process in Malawi would of improved gn.e.gy planning and take time to implement, and that a coordination, substantially reducing series of credits from the Bank would its consumption of fuelwood, be necessary if the Government was to improving its pricing of power, have the resources to carry out its strengthening the debt and financial program of economic recovery. F,r capabilities of the Ministry of this reason, SAL I was followed by a Finance and the Agricultural second SAL in 1983 and a third SAL in Development and Marketing Corporation 1985. The latter also provided some (ADgARg) (para. 16). room for technical assistance by earmarking a small fraction of the 5. Five specific components of the total Credit for a number of studies project were identified, in addition on budgeting, public investment to various studies. These components programming, export promotion, and were: tax system (paras. 1 and 4). * strengthening the energy 3. This Second Adjustment Credit planning capabilities of the Economic was to support a number of actions Planning Division through a long-term and steps to be taken in specific expert, training and equipment; - viti - * carrying out a pilot fuelwood and the Government to find savings program in order to reduce alternative grant financing for all fuelwood consumption in flue-curing the components of TA-TI (paras. 21- of tobacco through the financing of 22). consultant services and vehicles; Implementation * carrying out an electric power tariff study and an industrial price 8. Delays occurred in the decontrol study, through the implementation of two components: provision of consultant services; the energy planning and the fuelwood savings program; but on the whole, * strengthening of the progress was made in implementation Agricultural Development and of all components. Only one of the Marketing Corporation's financial two experts who were to help management capabilities by financing strengthening ADMARC's financial assignments of one financial analyst management was hired. The Debt and one financial controller to head Management Unit of the Ministry of the Finance and Accounting Finance did receive a microcomputer Department; system and other equipment, but thera is no indication that advisory * strengthening the Ministry of assistance was provided to firmly Finance's debt management establish the analytical capacity to capabilities through consultant make projections and simulations to services, training and acquisition of facilitate decision-making and choice computers and other equipment (para. of Debt management options. Finally, 18). the study on power tariff was implemented, but the analysis of the 6. The various components varied data and drafting of the report took widely in respect to their specific place abroad, with members of the objectives (type of results and ESCOM management team participating impact to be achieved) and in the only in the final phase which reduced actual form in which technical the degree of commitment to the study assistance was to be rendered (para. 24-52). (foreign experts in line position, advisers, trainers or consultants 9. Several studies were carried producing a study, --tc.). out using unallocated funds for various activities. The most 7. The five components under TA-II important ones were: were all explicitly mentioned in the Letter of Development Policy * The Comprehensive Human supported by SAL-II and the attached Resources Study (Manpower Survey). TA project, providing a convenient "umbrella" for the different * Agricultural Diversification components. Apparently, the three Study (analysis of the viability of a energy components (out of a total of proposed agricultural credit five) were included in the Project in institution for the large estate order to facilitate the subsector). implementation of useful recommendations made by a Bank Energy * Study on the revision of the Assessment mission in 1981. It is, Code for Government Procurement however, difficult to know if greater (paras. 53-61). efforts would have enabled the Bank - ix - 10. Unallocated funds were also has been reduced by 50%. On the job utilized to finance parts of project training effects were very limited, preparation activities for an however (paras. 79-80). Institutional Development Project to be financed by the Bank, and to C. Study on Power Tariffs provide support for training abroad of sixteen staff members of the 15. The resulus of the study were Ministry of Finance (paras. 62-69). not implemented because ESCOM was unconvinced of its usefulness from 11. The individual subprojects the start, and there was insufficient which were executed by various dialogue between the consultants and Government institutions were to be ESCOM (paras. 81-82). supervised by ths Ministry of Finance. This super--.sion, however, D. Financial Management ofADMARC was largely confined to the financial and administrative aspects. It was, 16. The Financial Controller played in fact, unrealistic to expect the an important role in ADMARC's Ministry to play a very active role financial management operations. in managing an "umbrella" project However, he also tended to work such as TA-II (para. 71). independently, and did not provide guidance to his counterparts. It is 12. The overall project was difficult to ascertain if the extent supervised three times in six years. of his contribution has been a Given the varied nature of the lasting one (para. 83). components, supervision tasks had to be assigned between several divisions E. Debt Management in the Bank, which inevitably complicated the coordination of 17. The planned technical activities and decreased central assistance in the form of control and responsibility. However, consultancies did not materialize, Bank supervision of the energy- but free advice and support was related components had much higher provided by the External Debt frequency (para. 72). Division of the Bank. The professional capacity of the local Results staff was only partly developed to make full use of the facilities for A. Energy Planning simulations and analysis of alternative debt strategies. 13. The major achievement was the However, progress was made in inter- completion of a fairly comprehensive agency communications on debt issues National Energy Plan. However, staff and the computer system was, in most training benefits were not fully respects, completed (paras. 84-86). sustainable, and the results of energy data collection were F. Human Resources Study disappointing (paras. 74-78). 18. The study provided useful data B. Fuelwood Savings Program and a basis for setting priorities influencing future investments in 14. There were substantial manpower training. However, the achievements. Technical solutions training part of this subproject was have been developed, and fuelwood not sustainable, as the Malawian consumption in flue-curing tobacco staff left the Manpower Unit and nobody is available to assist in S nbility "selling" the product to potential users and to serve as a bridge for 23. Sustainability can only be future updating of the data generated ensured where a professionally sound in the survey. The latter output has been produced and has been experienced a 40% cost overrun accepted by the recipient (paras. 87-89). institution, and wheve tangible training and know-how transfer has G. Overall Results taken place. A striking example of non-sustainability is the Human 19. With the exception of the Resources Study where all local fuelwood savings project, which was professionals left the Manpower Unit. the most concrete and technical Only in the case of the fuelwood component, none of the subprojects savings project sustainability is can be regarded as a complete likely to be achieved within the success, However, none of them target group (tobacco growers), should be .egarded as a total failure Overall, the TA-11 project is either. All have had at least unlikely to maintain an acceptable partial success; they have made some level of net benefits throughout its contributions, or there is still a economic life. Moreover, its impact reasonable probability that some on institutional development has been benefits will accrue in the near generally negligible (paras. 105- future (para. 98). 107). 20. In respect to the professional Lessons of ENerience quality of the personnel involved and the quality of the job performed, no 24. The TA-It project was not strong critical comments were voiced. designed to provide needed However, some doubts were l-ed in institutional support over the several cases by the formt_ local considerable period of time necessary counterparts and/or Bank supervision to internalize fully reforms in reports, whether an optimum choice of economic management. Instead of personnel had been made, e.g. Energy grouping in an "umbrella" type Advisor, Human Resources Study team project overly diffused components, (para. 100). one should rather concentrate on a limited number of high priorit.y 21. Lack of commitment was evident reforms to be carried out over a in ESCOM, which was never really sufficiently long period of time. convinced that the tariff structure Such an integrated form of technical was an issue of urgent concern (para. assistance would better mer Malawi's 101). pressing needs to strengthen national economic management (para. 109). 22. The lack of training and know- how transfer was a negative feature 25. There has been a tendency to common to all subprojects, with one use a rigid, "blueprint" approach to exception- -the Human Resources Study, technical assistance projects which was done in a form conducive to because, for administrative reasons, familiarizing the local professionals both the donor and the recipient with relevant techniques for manpower prefer to have the objectives, surveys and planning (para. 103). support and activities of technical - xi assistance already defined for the the three- or four-year time horizon whole contract period. However, this of a single technical assistance approach is more suitable for project. More narrowly defined, investment projects than for successive technical assistance assistance to improve social and projects planned over a longer period management processes. Future TA could, thus, could lead to better projects oriented to institutional fulfillment of long-term development should be molded in such institutional objectives (para. 112). a fashion, so that both sides learn from the experience of implementing 28. In sum, efficient free-standing the project. And provision should be technical assistance projects made for more frequent supervision of require: both the overall project and possible subprojects (para. 110). * preparation of realistic institutional development strategies; 26. Based on the TA-II project experience, future technical * a clear and precise statement assistance should concentrate in such of objectives; fields where the Bank has a comparative advantage over donors, * establishment of performance for example, projects which help indicators for subprojects; develop macro and microeconomic analysis capal-ilities in such areas * detailed differentiation as debt management and energy between various types of TA inputs, planning (para. 111). implying preparation of a detailed program and on-the-job training 27. Funds provide, under Technical components, a thorough analysis of Assistance projects should not be the respective expected contributions looked upon by both sides as free of short- and long-term experts, and resources to be used in an a careful definition of the local TA uncoordinated way. Similarly, it project coordinator's seems more appropriate to organize responsibilities; technical assistarce on a country or sectoral basis, with only one or few * a clear definition of the major partner institutions being Bank's comparative advantage in the involved administratively and in provision of technical assistance on terms of staff contributions. To a credit basis vis-&-vis grant achieve this goal, an institutional financing provided by bilateral development strategy should be donors and the UNDP; planned taking into account the time needed to achieve reforms in public * a clear link between TA and sector management, often going beyond SALs (para. 113). PROJECT PERFORMANCE AUDIT REPORT MALAWI SECOND TECHNICAL ASSISTANCE PROJECT (Credit 1428-MAI) I. BACKGROUND 1. The Second Technical Assistance Project was approved together with a Structural Adjustment Operation in December 1983. A first Technical Assistance Project had been approved earlier in July 1981, also in support of a first Structural Adjustment Project. Soon after the 1981 project was approved, it was realized that the structural adjustment process in Malawi would take a number of years to implement and that a series of loans/credits from the Bank would be necessary if the Government was to have the resources to carry out its program of economic recovery. This program was designed to diversify the export base, encourage efficient import substitution, adjust incentives, improve the public sector's financial performance, and strengthen policy-making capability. To provide support for this program, a second Structural Adjustment credit was consequently approved for an amount equivalent to US$55.0 million, in 1983. 2. This Second Adjustment Credit was to support a number of actions and steps to be taken in specific sectors (Energy, Industry and Agriculture) and in the fields of resource mobilization, management and institutional development. A number of proposed actions were to be carried out with bilateral and multilateral technical assistance since Malawi still possessed a shortage of trained personnel in both the civil service and key private sector areas. A separate technical assistance project was thus prepared to help implementing some of the actions supported by the SAL. 3. The agreement on TA-II provided for an IDA credit of SDR 1.4 million equivalent to US$1.5 million. Three categories of items were to be financed out of the proceeds of the credit: (1) vehicles, equipment and office supplies (SDR 230,000); (2) consultants' services and training (SDR 990,000); (3) unallocated (SDR 180,000). The credit agreement defined five specific components (three in the energy field, one dealing with agriculture marketing, and one with the management of external debt) and left a sixth one open for "other studies associated with the Second SAL." 4. Following SAL-II and TA-II, a third Structural Adjustment Program, approved in December 1985, also provided some room for technical assistance by earmarking a small fraction of the total SDR 28.0 million credit for a number of studies on budgeting, public investment programming, export promotion, and tax systems. 5. However, neither the SALs nor the TA operations were designed to provide institutional support over the considerable period of time needed to fully internalize reforms in economic management. The need was, however, felt at a later stage, to provide more consistent and long-term support for the reforms that were to be made, and to concentrate on a limited number of high priority reforms. As result of this new, less broad-based approach to technical -2- assistance, an Institutional Development Project was approved in December 1989 and is now being implemented over a five-year period. 6. One of the structural problems faced by Malawi and addressed by SAL I and II, was the rising cost of energy resources. With respect to petroleum, Malawi is totally dependent on imported resources. While the volume only increased by 8% between 1977 and 1980, the value of imported petroleum doubled and the share of imported petroleum in total imports rose from 10% to 15%. Thus, the Government's difficult task in the sector was to reduce the country's dependence on imported oil. 7. Another energy issue was related to the needs of the rural areas. Fuelwood represents some 80% of the total energy consumed in Malawi. Wood is used for cooking and for curing tobacco and tea. As a result of population growth and expansion in tobacco and tea production, the demand for fuelwood was outstripping available supplies, leading to deforestation and environmental problems. 8. The Government recognized that a comprehensive Energy Sector Study was needed to assess available resources, project future demand and supply, and examine alternative sources of energy and the possibility for intrasectoral substitution. The Government asked the Bank to carry out such a Study which was financed in part by the UNDP Energy Account. During negotiations of SAL-I, the general scope of work and timetable for completing the sector Study were agreed on by the Bank and the Government. An energy sector Mission visited Malawi in September 1981 and June 1982. 9. The mission recommended a number of measures and in particular in three areas: demand management, energy planning and power pricing. 10. The main thrust of the Government's strategy to tackle the fuelwood problem had so far comprised measures to increase the supply of this resource, with little being done to improve the management of fuelwood demand. Tackling the demand side was more difficult because, for most users, wood was not a commercial fuel. However, one aspect of demand management which held considerable potential and which had been comparatively neglected was a program to increase the low efficiency with which the tobacco industry used fuelwood. Considerable energy savings were thought to be obtainable by improving the efficiency of the furnaces and barn design and by developing new designs for the flues themselves to achieve better heat transfer and thus lower the fuel requirements for producing flue-cured tobacco, a major export for Malawi. 11. The Bank Energy Sector Report also recommended a strengthening of the institutional framework for energy planning and capability. In particular, it was felt that the Energy Unit of the Economic Planning Division (EPD) needed to be strengthened through additional national staff and the provision of technical assistance, including a senior energy economist/planner and the budget for carrying out energy studies and training national staff working in the Unit. 12. Finally, the Bank Report recommended a study on power tariffs due to a number of anomalies which had been found in the rate structure. There was a need to determine long-range marginal costs, to evaluate the effects of - 3 - introducing a single kwh rate and the effects on low income users of a monthly fixed charge. 13. The above recommendations were accepted by the Government and included in a Letter of Development Policy drafted in conjunction with SAL-II and TA-II. 14. Although energy issues represented the most important component of the TA project, two other unrelated topics were also included: improved operation of the agriculture marketing organization and management of the external debt. 15. The Second Technical Assistance Project was evaluated in conjunction with SAL-II. The latter, as well as SAL-I, was broad-based with simultaneous actions on several fronts to enhance resource allocation in the productive sectors (energy, agriculture, industry) and to improve efficiency in supporting institutions, including the Government itself (PCR, para. 11). The role of the Technical Assistance Project, coupled with SAL-II, was thought to provide the expertise where it was lacking for selected areas. II. OBJECTIVES AND DESIGN 16. The Project aimed at assisting the Government in achieving its goals of improved energy planning and coordination, substantially reducing its consumption of fuelwood, improving its pricing of gower and industrial commodities, strengthening the debt and financial management capabilities of the Ministry of Finance and the Agricultural Development and Marketing Corporation (ADMARC). 17. As a result of the project, Malawi was assumed to have developed a medium-term energy investment plan which addressed its major energy problems, a fuelwood savings program to reduce the level of wood consumption, a revised power tariff structure reflecting the long-run marginal costs to the economy of meeting uhe demand for electricity in Malawi, a phased program for decontrolling industrial prices, and an improved system in financial and debt management. 18. Five specific components of this project were identified at the time it was approved, with a sixth category left for other studies. They were: (1) Strengthening the energy planning capabilities of the Economic Planning Division of the office of the President and Cabinet, including training. This component included mainly the financing by IDA of a long-term expert assignment at a cost of US$130,000; there was cofinancing from UNDP mainly for training and equipment. (2) Carrying out by the Tobacco Research Authority (TRA) of a pilot fuelwood savings program. This component involved the financing of vehicles, operating expenses and engineering and consultant services for a pilot fuelwood savings program. The component may be regarded as the partial financing (US$313,000) of a project strongly supported by UNDP. (3) Carrying out of an electric power tariff Study by the Electric Supply Commission (ESCOM) and an industrial price decontrol Study by the Ministry of Trade, Industry and Tourism. Consultant services for the power tariff Study were estimated at US$130,000. Consultant services were also included to examine price controls for industrial commodities (US$40,000). (4) Strengthening of the Agricultural Development and Marketing Corporation's financial management capabilities. This component assisted in financing assignments of one financial analyst (US$220,000) and one financial controller (US$110,000) to head the finance and accounting department. (5) Strengthening the Ministry of Finance's debt management capabilities, including consultant services, training and acquisition of equipment. This was a continuation of technical assistance started under the TA- I projt:.t. The assumed foreign exchange cost of this component was US$98,000. 19. The total foreign exchange cost of these components was US$1,121,000 and US$1,255,000 after adding local costs to be incurred. Since the amount of the credit for TA-II was US$1.5 million, the difference was allocated to consultant services for other studies needed to carry out the structural adjustment program. 20. The five components of the TA-II project had the following characteristics: (i) three were dealing with energy sector issues (planning, fuelwood conservation, and power tariffs) which had been raised by a Bank Energy Sector mission; and (ii) two were expanding on work already started under TA-I (diagnostic study of ADMARC and support to the Ministry of Finance's Debt Management Unit). However, the various components differed widely in respect to their specific objectives (types of results and impact to be achieved) and in the actual form in which technical assistance was rendered (foreign experts in line position, advisors, trainers or consultants producing a study, etc.). 21. The components also differed widely in the scope of their ambitions to contribute to institutional development. Some of the subprojects (Energy Planning Unit, Debt Management Unit, support to ADMARC) really aimed at a broad- based strengthening of an institution or organizational unit. Others may be seen as a contribution to certain strategic decisions (improved energy management for fuelwood, better power tariffs) without institutional development as a basic objective. 22. The Bank's assistance given under TA-II, although assumed to provide support to SAL-II (and this was done to some extent), included a wide spectrum of relatively small technical assistance components greatly differing in their objectives, means of delivery, and comprehensiveness of the resources provided. As it will be seen below, unallocated funds provided under TA-II, in order to increase flexibility, were sometimes used for purposes loosely or sometimes not at all related to the policies of Structural Adjustment. However, the five main components financed under TA-II were all explicitly mentioned in a 1983 Letter of Development Policy supported by SAL-II and the attached TA project, providing a convenient "umbrella" for the different components. It is, however, tempting to believe that the three energy components were conveniently included in the project to facilitate the implementation of useful recommendations made by the - 5 - Bank Energy Assessment missions of 1981-82. The Assessment missions' report had been partly financed by the UNDP Energy Account and, indeed, the energy planning component was subsequently picked up under grant financing. However, as stated in the PCR (para. 17), it is not possible to know if greater effort at the time of TA-II preparation would have enabled the Bank and the Government to find grant financing for all the components of TA-II. III. IMPLEMENTATION 23. Given the diversity of components, implementation aspects have been first reviewed for each of them, followed by an evaluation of global implementation and supervision problems. A. The Energy Planning Component 24. Keeping in mind that the various energy subsectors (fuelwood, petroleum, power, coal and non-conventional energy sources) were taken care of by different Government institutions (and not by the Ministry of Energy), the strengthening of a coordinating Energy Unit in the Economic Planning Department (EPD) of the office of the President and Cabinet appeared to be the rational approach. Towards this end, the Government agreed to employ additional professionals (in particular an economist as head of the Unit) and requested IDA to finance the assignment of an experienced energy economist/planner for a period of eighteen months. 25. The original request for the advisor (March 1984) emphasized product- oriented studies like preparation of an overall National Energy Plan, identification of energy resources, identification and priority ranking of projects, etc. In addition, the resident advisor was to help managing che Unit, define its work program and provide on the job training to counterpart staff. The team which the expert was to work with was planned to consist of up to five professionals. 26. According to the Credit Agreement, the work was to start in mid-1984. In fact, there were considerable delays until a contract with an American consulting firm was concluded (mid-1985) and until the energy advisor actually arrived on the spot (June 1986). It took another several months until a core group of Malawian professionals (less than planned) joined the Energy Unit. 27. The work of the advisor, in overall terms, seems to have been adequate, although there were some critical comments in two Bank Supervision Reports (September 1986 and August 1987). As planned, the work concentrated on producing strategic documents (contribution to the Development Plan 1987-1996, and as the main "product": The National Energy Plan 1988-1997). At the same time, efforts were made to strengthen the Unit's internal organization (including rudimentary data banks) and its external links to the various institutions involved in the energy sector. - 6 - 28. The National Energy Plan became ready in draft in October 1988, necessitating an extension of the advisor's assignment by three months. This was not financed by the TA-II project but by UNDP funds which were also used for acquiring computer equipment and sending Malawian staff members for training abroad. 29. The assignment of the Energy Advisor appears to have been supervised by Bank regional staff only once, at the very beginning. More regular and intensive supervision came from the Bank's Central Energy Department Staff who was also involved in supervising or preparing other activities in the energy sector. Any professional input does not seem to have been given by the Consulting firm to the Unit (except direct discussions with the advisor on the work program). 30. For some time cooperation between the advisor and his Malawian colleagues was made difficult by the former's double (and not clearly defined) role of a performer being charged with specific own tasks, and of an advisor supposed to assist the Malawian staff in getting the Unit's work done. B. Pilot Fuelwood Savings Program at the Tobacco Research Authority (TRA) 31. Already in 1980, Phase II (wood energy) of the National Rural Development Program (NRDP) financed by IDA Credit 992-MAI, embarked on various measures to increase forest resources and the creation of an Energy Unit in the Forestry Department of the Ministry of Agriculture and National Resources. No specific efforts relating to fuelwood savings in the tobacco sector were included in this project. 32. As indicated above (para. 10), the Bank Energy Assessment Report gave specific attention to the fuelwood savings potential in the process of curing tobacco. Tobacco curing accounted for an estimated 40% of Malawi's fuelwood consumption, some 400 growers of flue-cured tobacco used 75% of the wood and 42,600 growers of fire-cured tobacco used the remaining 25%. While energy savings were possible in both, the greatest immediate potential obviously laid in the flue-cured tobacco. Malawi's TRA estimated that the flue-curing tobacco estates consumed between 0.02 cubic meter and 0.13 cubic meter of wood per kilogram of tobacco in their barns. Even the most efficient barns, however, used about three of four times more wood energy than the best ones in the United States. 33. It is against this background that a pilot project was proposed to: (i) survey wood use as well as the number and condition of the barns used in the flue-cured tobacco se-tor; (ii) to determine and demonstrate the possible savings in fuel usage made by improving furnaces and vents, and to show that such improvements were economic; and (iii) to disseminate the findings and results to growers for implementation. Phase I of the project was intended as a pilot scheme to start in April 1984 and continue for two years. 34. This two-year period was not meant to be a complete fresh start. It was to intensify and to widen the work already begun by a FAO-financed research officer who had been working with TRA since 1980. Because of this, the one-year delay in implementing this two-year project phase did not lead to the full loss -7- of one season. The FAO expert was able to initiate the necessary survey work and some preliminary work on research equipment, the latter in informal cooperation with an instrumentation technician who had already been selected and was waiting in Malawi to get his contract. Due to various administrative delays, the actual project phase financed under TA-II began about mid-1985 and comprised two main lines of action: a. Research, mainly done by the instrumentation technician in cooperation with the FA0 Research officer, related primarily to barn trial measuring wood consumption for different existing and improved barn types. For one season this work was done Ln five different regions of Malawi and for the following season research was concentrated in one location, allowing a much more controlled and systematic comparison of different technical solutions and firing practices, b. Extension, done by an extension expert, with more than 20 years of experience in Malawi, comprised the development of extension material (and a monthly bulletin) on the research findings and their applications, as well as the active dissemination through workshops and visits to tobacco growers. 35. It had been planned from the outset that a second project phase (on a wider scale) would have to follow. Therefore, the instrumentation technician was given a second two-year contract, and the former FAO advisor to TRA, after the end of his IDA-financed contract in November 1987, stayed with the project under a special trust fund agreement with FAO financed by a UNDP/Bank Energy II Project. From the same funds some further equipment was provided (partly even before the end of the TA-II project phase). 36. While the team leader was working under a UNDP/FAO contract, the two experts financed under TA-II were working under individual work contracts with the Government of Malawi, represented by the Ministry of Finance. In organizational terms, the project suffered considerably from the following weaknesses: a. The financing under TA-II provided but one phase (about two years) of a project which actually started before that phase and which had to go beyond that phase. This created a substantial amount of uncertainty as to the long-term perspectives of the project, and it certainly did not facilitate a consistent project management by the Bank and the Government of Malawi. b. At the time of the TA project phase, TRA management and administration were weak and not really able to actively support the project in terms of logistics and financial administration (including liaison with the Ministry of Finance). c. The Ministry of Finance did not take an active role in the administration of any of the individual subprojects of TA-II and the fuelwood subproject was no exception. Responsibilities for dealing with the subprojects were not defined; the project personnel often did not know who to approach in matters relating to their work contract. The Bank field office did not offer assistance on such matters. -8- 37. No counterparts were provided for the experts financed under TA-II. Even after establishing the reqvired posts, it was not possible to get them filled. Potential candidates were reluctant to join the Project because of its uncertain perspective. Only from April 1988 were counterparts employed after posts were established with TRA proper (not with the Project as such). 38. In contrast to some other TA-II subprojects, this component was fairly closely supervised or guided by the Bank Control Energy Department staff who took a quite active interest in the project and its results. Supervision missions took initially place about once a year, later twice a year. The project team reported fairly regularly on the project's progress. In fact, this is the only subproject on which some progress reports could be found. C. Study of Power Tariff for the Electricity Supply Commission of Malawi (ESCOM) 39. The Government's medium term program (1981/82-1985/86) which was formulated in the context of the First Structural Adjustment Program made the review of price systems and tariffs one of four areas of priority concern. A study on power tariffs and suitable formulas for future adjustments was rational in that context. It appears that the Bank, at the same time, was interested to use the Malawi case (among others) as a test case for a fairly new concept of developing a tariff structure which reflected, as closely as possible, the long- run marginal cost to the economy of meeting power demand. Considering the fact that, in comparison to other Malawian parastatal organizations, ESCOM had maintained a relatively sound economic position, no particularly pressing reasons for introducing a new power tariff structure seem to have existed at the time of conceiving the TA-II Program. However, this study, though being the only ESCOM- related activity under the "free-standing" TA-II agreement, was but one of many Bank-supported activities related to ESCOM. There had been TA components in various capital projects, and there are still such components in the Bank's more recent sectoral projects. The study was thus not as "free-standing" as it might appear at first glance. 40. The contract for the study was given to Coopers & Lybrand Associates (London) who, in cooperation with Merz & McLellan, provided a team of three experts (two economists, one engineer). The team visited Malawi for about three weeks primarily for fact finding. During this period ESCOM staff was active in providing data, primarily on the demand structure and cost characteristics of the various elements of the power system. The analysis of the data and drafting of the report took place mainly in Britain with members of the ESCOM management team participating only in the final phase (February/March 1985). 41. Bank sector staff obviously was quite active in initiating the study and financing it under the TA-II agreement. It seems that during a Bank supervision mission in September 1986, discussions were held with ESCOM on the results of the study. ESCOM did not agree to some of the assumptions and conclusions of the study. The Bank's mission report stated that "it was agreed that the results of the study should be held in abeyance until some time in 1987, when firmer information becomes available." - 9 - D. Strengthening the Financial Management of the Agricultural Develoment and Marketing Corporation (ADMARC' 42. ADMARC is one of Malawi's large parastatal organizations, involved in various agricultural development schemes and particularly in helping the smallholder farmers to market their cash crops. In the context of SAL-I and SAL- II the strengthening of ADMARC was important in respect of two objectives: providing better price incentives to the farmers and reducing the government budget deficit to which losses of various parestatals had been contributing. 43. In the same way as other companies such as the Malawi Development Corporation (MDC) and Press Holding, ADMARC was to screen and streamline its portfolio, an operation which required capacity for financial analysis and carrying out investment/disinvestment decision. Assistance to ADMARC was not explicitly mentioned in the TA-I Agreement. However, a short-term consultancy was financed unde. this agreement in 1982 (carried out by an expert who had previously served as ADMARC's Company Secretary). Apart from other recommendations, the consultant strongly emphasized the necessity of strengthening ADMARC's financial management. This was taken up in the President's Report for TA-II proposing assignments of two financial experts: (a) a financial analyst, (b) a financial controller (to head the finance and accounting department. 44. The financial analyst was never hired while the financial controller had already been recruited before the TA-II project had been signed. From mid- 1983, -he controller was financed through the US$7.3 million NRDP-I project for a peri3d of two years. The TA-II project intended to further finance the controller's services beyond mid-1985. 45. According to a 1986 Bank Supervision Report, the financial controller was actively involved in developing ADMARC's divestiture program in collaboration with IDA and with USAID. However, given the preparation of an ADMARC separate project, Bank staff did not actively supervise this component. E. Support to the Ministry of Finance Debt Management Unit 46. During preparations for SAL-I and TA-I, the need for strengthening Malawi's debt management capacity was already emphasized. The President's Report on TA-II (para. 62) stated: "Although Malawi's debt reporting and management performance had been satisfactory in the past, there is a need to strengthen debt management and to monitor more closely external borrowing ...; it was further agreed during negotiations that a Bank staff member would visit Malawi to assist the government in standardizing and streamlining its debt reporting system." However, no explicit provision was made under TA-I for technical assistance in this field. 47. It seems, however, that fuads were utilized under TA-I to purchase some computer hardwares and hire a consultant to help set up a computerized recording and monitoring system. This system was to help not only in recording debt - 10 - already contre,7ted but also help in the formulation of future borrowing policy that would reduce the debt service ratio. 48. The President's Report on TA-II emphasized the "continued improvements in institutions dealing with external debt and investment screening" as one of the important aspects of the Government's development policy in the context of the Second Structural Adjustment Credit, and it made the "strengthening the debt management capabilities of the Ministry of Finance', an explicit component of the TA-II project (para. 102): "The project includes funds to computerize the system (purchase of micro-computer and associated equipment) and six to eight months of consultant services to strengthen further the debt management system including helping formulate an appropriate debt management policy." 49. The Ministry of Finance (MOF) acquired a microcomputer system for use in developing its debt management capabilities. Additional equipment was also purchased to: (i) increase capacity to enable projections and analysis of the recurrent and development budget; and (ii) develop the capability for simulations, projections, and the incorporation of microeconomic variables into the analysis of debt and thus enhance capability for decision-making and choice of debt management options. 50. Although a 1986 Bank mission suggested that advisory assistance be provided under TA-II to firmly establish the above described analytical capacity, there is no indication that such technical assistance took place. In fact, technical assistance was actually replaced by free assistance from the Bank's own External Division. The system was developed largely by Malawian professionals, and in particular by a system analyst who had joined the Unit in 1982 (coming from the data processing department) and left the Unit in spring 1987. Foreign training of MOF personnel as financed under TA-II (see below) did relate to a variety of subjects but not really to debt management. 51. The debt management's system was part of MOF's Economic Affairs Division. In terms of personnel and equipment, it was highly integrated with the other tasks of that Division, and could not be regarded as a separate Unit. 52. On the Bank's side, resp>nsibility for supervision rested with the Africa Regional Office and the Debt and International Finance Division, International Economics Department. The latter undertook seven missions from 1983 to 1988 to provide assistance in setting up and running the Debt Management Unit. F. Other Studies and Components 53. The TA-II project included unallocated funds to be tapped for various activities for which other funding could not be found, even if the links to the SAL were somewhat tenuous at times (PCR, para 18). - 11 - a. Comprehensive Human Resources Study (Manower SurveX) for the Department of Personnel. Management and Training (DPMT) 54. The context of this Study is not quite clear. None of the various reports or agreements on SAL or technical assistance ment.ons the non- availability of data on human resources or manpower (existing or missing) as a problem deserving specific attention. An issues paper on SAL-II and TA-II dated July 29, 1983 refers to a proposed consultancy on personnel policy guidelines and a public service manpower and training program. However, this proposal is neither identical with the comprehensive human resources Study nor made it its way into the actual agreement on TA-II. 55. The official request (telex from May 29, 1985) reads as follows: "... we propose to carry out a human resource development Study to produce a public and private sector manpower development program to enable the Government meet the national objectives of self-sufficiency in skills in short supply. This study is related to the achievement of the institutional development objectives of the Government's ongoing structural adjustment program." This Study was not mentioned in the Credit Agreement on TA-II; however, that agreement envisaged (under part F of Schedule 2) "other studies associated with the second structural adjustment project...." This connection to SAL is not obvious. 56. By mid-1986, the Government entered into an agreement with ILO to carry out the Study, based on a proposal elaborated by a consultant before. Under this contract ILO was to provide two long-term experts for about one year each plus short-term assignments of two consultants (among which the consultant who had drawn up the proposal). The Chief Advisor (from Ghana) arrived by November 1986, the second advisor (statistician, from Sierra Leone) by March 1987. The survey involved substantial field work; more than 1,100 establishments and more than 30,000 skilled employees were covered. Processing of data was planned to be fully handled by the Data Processing Unit (DPU) of the Ministry of Finance, but had to be done partly with DPMT's own resources. A draft report was produced by end-1987; the final version was printed by end-1988. 57. The remarkable feature of this subproject is the discrepancy between the original cost (and time) estimate and the actual requirements. It appears that the original proposal grossly underestimated the work and time required to get the sufficient number of enumerators and supervisors selected and trained, and to get the required number of questionnaires returned from the respondents (establishments and individuals). There may have been some differences of opinion in interpreting thF original work proposal, leading to more work than originally intended. The files show that the Government asked for three (March, August, and October 1987) additions (total approximately US$127,000) to the project's budget of originally US$321,000. The Bank's reaction cannot be fully ascertained from the files. According to information from the Ministry of Finance total disbursements amounted to approximately SDR 350,000. 58. According to information from DPMT, there was no real supervision from the Bank (except for casual talks during missions having a different focus). ILO representatives came twice to assist in overcoming the difficulties the project had run into. . 12 - b. AgXicultura1 Diversification Study 59. This Study was to analyze the viability of a proposed agricvltural credit institution for the estate subsector (as opposed to the traditional small- holders subsector). SAL-II included a provision that, after a review by the Bank, the Government would develop an estate credit project. The reason was the need to provide medium- and long-term financing to the estates and to study whether a new institution was necessary or whether commercial banks could handle this need if the Government would provide assistance such as a guarantee scheme or rediscounting facilities. 60. The Study was carried out at a very early stage of TA-II (1984-85), at a cost of approximately SDR 82,000. The study showed that credit demand could be met without creating a new institution (PCR, para. 16-Fl). c. Study on the Revision gf the Code for Government Procurement 61. This Study (cost approximately SDR 33,000) was written as a draft which was then completed by end-1990 (PCR, para 16-F3). d. Support to the Ministry of Finance for Training Abroad and other Activities aiming at Institutional Development 62. Apart from strengthening the Debt Management Unit, the Bank has been assisting the Ministry of Finance in two other lines: in financing short-term training abroad for MOF staff members (under Technical Assistance II), and in financing a number of studies and consultancies as preliminary steps to a more long-term strengthening of the MOF. The latter was expected to take shape in an Institutional Development Project (IDP) which Zombines Bank assistance with assistance from other donors (particularly UNDP) and which relates to strengthening MOF as well as establishing a Malawi Institute of Management (MIM). The studies and consultancies relating to IDP have been partly financed under SAL-III and mainly under a Project Preparation Facility (PPF). i. Training Abroad 63. Schedule 2 of the TA-II Credit Agreement lists the various project components. Training is included in two of the five explicitly defined project components: strengthening the EPD Energy Unit and strengthening the Ministry of Finance's debt management capabilities. There is no provision for training in general, and the most flexible part ("other studies associated with the Second Structural Adjustment Project....") provides for studies and not for training. 64. In 1987, MOF asked the Bank to agree to financing under TA-II some individual short-term training measures (MOF staff members attending courses abroad). The Bank agreed although the link between the course topics and debt management appears to be somewhat vague. The Bank-financed MOF training "program" obviously gained momantum in mid-1988 when it became obvious that commitments and disbursements under the TA-II Agreement were lower than expected. A proposal for extension of the closing date of the Agreement to June 30, 1989 [memo from the Country Department (AF6CO) dated June 17, 1988] gives the following information: "Almost US$200,000 is still to be committed under the - 13 - project (including approximately US$90,000 which is being reimbursed by the Government from a previous duplicate disbursement). Funds are still left in the project because expenditures have not been as high as expected and new uses have not come up as quickly as anticipated. All of these funds should be committed and disbursed during the next year. Most of them will be used for foreign training and seminars, with some to also go for additional computers for the Ministry of Finance...." At end-1988, MOF had prepared a list of training cases to be financed under TA-II giving the names of 16 staff members (including the cases from 1987) with the topics and the cost (some in the range of K 5,000 - K 10,000, mostly in the range of K 25,000 - K 35,000) of their training. The total cost of this training "program" amounted to approximately SDR 100,000. According to the desk officer (multi-lateral) at MOF an agre,ement on funding this program under TA-II was reached during a supervision mission of the Bank's Country Officer in July 1988. The Aide Mdmoire on this visit does not explicitly refer to the subject of widening the training assistance to such a wide range of training topics (not related to debt management). The list of topics includes: * Investment planning, appraisal and management for development finance institutions. * Agricultural and rural project management. * Public budgeting and financial management. * Public enterprise policy and management. * Use of computers in financial management. * Use of computers in public budgeting and financial management. * Procurement and material management. 65. Such courses had a duration of 6-10 weeks and are held mostly in the UK or in the USA. The list also showed several participants in World Bank disbursement seminars, partly in Washington, D.C., partly in Kenya. 66. In the course of this evaluation, four of the trainees (two economists and two administrative officers) were interviewed. All of them stated and explained that the course content was relevant for (at least a part of) their task at MOF. However, all of them also stated that their participation in that specific training program was not he outcome of a deliberate planning effort on the part of the Ministry; it was mainly an outcome of own initiative after having heard of specific courses being offered by training institutions in the US or UK, and funds being potentially available under the TA-II scheme. This may be a quite normal procedure. However, it is somewhat surprising in a situation where, at the same time, a UNDP-financed consulting firm was drawing up a Development Plan for the Ministry of Finance whose implementation (as part of the IDP) was to involve a thorough planning of future manpower and related training needs of MOF. 67. This evaluation does not provide a basis for really criticizing the decisions made. However, there are reasons to believe that the actual shape - 14 - which this training took was heavily "supply-driven": The availability of unutilized fund3, and the fact that courses were actively advertised by institutions in the US and UK, very much intensified the 1988 MOF training "program" which, of course, was in line with the wide-spread individual preferences for attending training courses abroad. Although all the training cases were certainly endorsed by the superiors of the trainees, there are no indications that these endorsements were part of a well-thought of concept taking into consideration that the Bank's assistance to this training came in the form of a credit, a form of assistance otherwise rather rarely used for financing training abroad. ii. Other Assistance to the Ministry of Finance 68. Apart from the attempts to strengthen the Ministry of Finance's debt management capacicy, efforts were made for several years to lay the ground fo: a longer-term process of strengthening the Ministry's capacity in other areas, namely relating to a reform of Malawi's tax system and to an improvement of the accounting and budgeting processes. In the context of SAL-II, approved in December 1985, some preliminary studies were carried out for both of these areas. From the end of 1987 and the beginning of 1988, two groups of consultants worked on a follow-up which was meant to lead to a several-years' process of technical cooperation under a new Institutional Development Project (IDP) (approved in December 1989) which comprised the establishment of a Malawi Institute of Management and strengthening the Ministry of Finance. 69. The Institutional Development Project (now under implementation) was justified by the belief that previous Bank experience with improvements in economic management had demonstrated the need to provide consistent and long-term support for the reforms which were to be made. It was argued (cf. SAR, Institutional Development Project, 1989) that earlier reforms were too all encompassing, that too much was required at once rather than a proper sequencing. It was observed thal- reforms introduced under the pressure of SALs (supported by TA projects) and tied to tranche releases, lacked the mechanism to ensure that they were carried on after the immediacy had passed and the tranche released. The IDP project sought to learn from these experiences and concentrated on a limited number of priority reforms. It provided low-intensity technical assistance designed to support and not replace ez sting capacity. G. Implementation and Supervision 70. Planning, at the subproject level, mainly relates to the thorough definition of the terms of reference for the consultants or consulting firms to be employed. Only in some of the cases investigated in this study, the terms of reference were still traceable in the files. In general, they seemed adequate, sufficiently precise to give direction, sufficiently flexible to adapt to changing or unknown needs. A common feature seems to be that training tasks, though principally part of technical assistance operations, are not detailed in sufficiently clear terms in the terms of reference or contracts. 71. In principle, the individual subprojects which were executed by various Government institutions were to be superised, on the Malawi side, by the Ministry of Finance, as the agency responsible for the overall project. This -15 - supervision, however, was largely confined to the financial and administrative aspects. It was, in fact, unrealistic to expect the Ministry of Finance to play a very active and driving role in managing an "umbrella" project such as TA-II in terms of substance. The Ministry probably had more than enough to do with managing technical assistance projects in purely financial and administrative terms, reacting to initiatives and requests from the other institutions rather than taking the lead itself. 72. On the Bank's side there was some substantial supervision in respect to the energy-related subprojects. Other components (external debt, manpower study) were supervised by various departments of the Bank. The Bank supervised the overall project only three times, in 1984, 1986 and 1988. Given the varied nature of the components, supervision tasks had to be allocated between several Divisions in the Bank, but this inevitably complicated the coordination of activities and decreased central control and responsibility. IV. RESULTS 73. After analyzing results obtained by individual components, an attempt is made to discover common denominators and achievement of TA-II as a whole. A. Results of Individual Components (1) Energy Planning 74. The major achievement was the completion of a National Energy Plan. The latter was a fairly comprehensive and readable document providing data and concepts; it appeared to be a suitable basis for further planning and coordination within and between the various energy subsectors. 75. Another positive achievement was the working relationship established with the various institutions involved; the joint work on the Energy Plan (to which all the institutions contributed) was instrumental to this. 76. Three Malawian staff members benefitted from the cooperation with the expatriate Energy Advisor (primarily regarding work organization, establishing external links and use of computers). However, not all the benefits were fully sustainable. These staff members went abroad for training with returning colleagues replacing them. 77. It was planned to establish the core of a computerized data bank on the energy sector. No sustainable results were achieved in this field. In particular, data collected for an energy subsector were lost. 78. Finally, the Advisor left without submitting any final report on his assignment and apparently without proper information on how to follow up with his own work and contacts. - 16 - (2) Pilot Fuelwood Savings Program at the TRA 79. Up to now, the project made substantial achievements. However, it is not possible to clearly isolate those achievements which were made during the phase financed under the TA-II project agreement. The major achievements are: * The project has clearly established that flue-curing of tobacco is possible with an input of less than 10 cubic meters of firewood per ton of tobacco using low-investment technology. * Suitable technical solutions have been developed, tested and are being propagated through extension material. * The project has caused a general awareness of the fuelwood problem and the savings potential amongst growers. An extension system (with five agents in different regions coordinated by a extension engineer at TRA) has been made operational and will be able to further propagate the project's message for some time to come. * It is estimated that, with strong influence by the project, total fuelwood consumption in flue-curing of tobacco in Malawi has been reduced by 50% since 1984. This means that about 400,000 cubic meters are saved annually. Valued at 15 K per cubic meter (a price just reflecting the logging and transport operations, not the true value of the wood), this amounts to annual savings of K 6 million (i.e., about US$21 million). There is likely to be a potential for further cutting down the wood consumption to about 50% of the present level. 80. This project aims at achieving a specific objective (reduction of firewood consumption), not at establishing or strengthening an institution for carrying out long-term tasks. On-the-job training effects were very limited during the TA-II period since no counterparts were available to the Bank-financed experts. A substantial transfer of knowledge, however, has taken place since spring 1988 when counterparts finally were employed. (3) Study on Power Tariffs for ESCOM 81. The study seems to have created a certain understanding of the new concept of tariffs strongly reflecting future marginal energy cost. However, the results of the study as such have not been implemented probably for two reasons: * The process of the study (fact-finding mission and subsequent write-up of the report in London) was not conducive to a dialogue between the consultants and the client, convincing ESCOM management of the suitability of the concept and making the study team sufficiently aware of specific questions or constraints which ESCOM thought to be important. * Despite a fairly strong Bank role in initiating the study, there was little commitment on the recipient side. - 17 - 82. There was little participation of ESCOM staff in actually carrying out the analysis; therefore, training effects were very limited. (4) Strengthening the Financial Management of ADMARC 83. In the period in which the Financial Controller was financed from TA-II funds, unforeseen market developments (inter alia, increasing purchases at fixed prices, decreasing export prices) brought ADMARC into financial difficulties. The financial management had to play a particularly important role at that time. In the view of ADMARC management it had been the right decision (in 1983) to establish the post of the Financial Controller and to recruit an expatriate for this post since sufficiently qualified Malawians were not available at that time. The controller certainly played an important role in ADMARC's day-to-day operations. However, he also started to do things on his own instead of making other persons understand why and with which perspective things had to be done. (5) Debt Management Unit in MOF 84. Progress was made in inter-agency communication on debt issues and the computer system was, in most respects, complete. However, while only some minor problems existed in the computer programs, the system was not yet effectively used when the TA project component was completed. The causes of these problems were: (a) Inadequate data collection, partly due to insufficient communication between the various agencies having access to debt information and lack of clear guidelines on the responsibilities of each agency. (b) Lack of knowledge of the system capabilities by users, due to insufficient documentation and minimal communication of system capabilities to people who could use the information. (c) Poor data quality in the computer's data base, due to inadequate and inappropriate staffing in the debt unit, inadequate programmatic validation of data, and insufficient checking of data after it has been entered into the computer. (d) Insufficient knowledge and testing of system functions, due to staff-turnover and lack of systematic testing procedures. (e) Incomplete records in the ledgers maintained in the office of the Accountant General. 85. A task force was created which involved representatives from MOF, Accountant General, Department of Planning and Development, and the Reserve Bank of Malawi. The group tried to improve and secure the flow of data on which the system fully depends. The group has also analyzed the division of labor between the Debt Unit in MOF, the debt information system of the Reserve Bank of Malawi and the debt management system in the Department of Economic Planning and Development. Within MOF, apart from the need to check and clean up the existing data files, the main task still seems to be to build up an analytical capacity - 18 - sufficient to really make use of the system, i.e., a team of economists really versed in the mechanisms and possible strategies of debt management. 86. To conclude, two remarks can be made on results obtained by this component: * The planned TA in the form of consultancies did not materialize, but the External Debt Division of the Bank provided free, useful assistance, and undertook seven missions in 1983-1988. The professional capacity of the MOF staff was not sufficiently developed to make full use of the facilities for simulations and analysis of alternative debt strategies. (6) Human Resources Study for DPMT 87. There has been some disagreement regarding the quality of the study. In spite of some weaknesses, it can, however, be regarded as a useful basis for setting priorities influencing future investments in manpower training. It not only provided data on existing (and missing) skilled manpower but also comments on available training capacity and the priority needs for expanding such capacity. 88. The Malawian members of the team (four professionals) have learned in the process (including some short-term training abroad). However, this training effect was not sustainable: the four Malawian team members who have gained some experience in planning, organizing, and evaluating the manpower survey bave left the Manpower Unit of DPMT. None of them is available to assist in "selling" the product to potential users and to serve as a bridge for future updating of the data generated in this survey. 89. This study, together with other studies financed through unutilized funds from TA-II, had a somewhat tenuous link to SAL-II. The Human Resources Study was particularly expensive and cost approximately SDR 350,000 or 25% of the total credit amount. It experienced a cost overrun of over 40% (PCR, para. 18). B. Overall Results 90. A number of problems have been identified with technical assistance in general in Malawi and many of these are evident in the TA-II project. (1) The lack of sufficient Government coordinating capacity, able to define technical assistance needs on the basis of own personnel development plans, and to request and supervise technical assistance from various donors accordingly. (2) Difficulties in the Government's system of personnel management, leading to a loss (or insufficient use) of trained personnel (including personnel trained in the context of technical assistance. - 19 - (3) The unsolved issue of whether and to what extent foreign donors should make more use of local professionals instead of expatriates (advantages in terms of cost and training cpportunities, disadvantage of creating distortions in local pay scales). (4) Competition and non-coordination among donors; influence of own interests of donor agencies and experts (e.g., prolongation of assignments beyond actual needs). (5) Difficulties of donor agencies to react fast and flexibly to assistance needs and to recruit fully qualified personnel. (6) Insufficient planning and supervision of the individual technical assistance projects, both through donor and recipient agencies, particularly in respect to "soft" objectives like training of local personnel. (7) High cost of technical assistance (even if given on a grant basis); strong reluctance to use loans or credits for technical assistance purposes. 91. Issue (1) is regarded a key issue. Without strengthening the Malawi Government's capacity for a better coordination and supervision of technical assistance, little progress can be made in solving any of the other issues. 92. Looking at the subprojects financed under TA-II, it is very difficult to discover any common denominator except the following: * Funds from the TA credit were used for the import of expert services (including payment of salaries for expatriates already living in the country) and, in some cases, of goods with a high technology content. * These imported services (goods) were not directly linked to other Bank-financed capital projects (e.g., infrastructure projects) or import programs (as financed under SALs). However, in several cases there were quite strong indirect links (e.g., studies relating to sectors where the Bank is or intended to be involved). Three subprojects (ADMARC, ESCOM, Debt Management Unit in MOF) were linked to SALs, while two were not and two were doubtful. * In principle, all the subprojects referred to the import of know-how (thought to be non-existing in Malawi) which was to be brought to use, in quite different ways, within the Malawi Government's and parastatal institutional framework. 93. The latter seems important. There are, at least, four different ways of bringing to use the foreign know-how: - 20 a. Foreign experts work in line positions, making direct use of their know-how with or without transfer to others (Financial Controller). b. Foreign experts act as advisors and trainers, transferring their know-how to local professionals who actually bring it to use (in none of the cases existing in pure form; however, to some extent, transfer of know-how in the Energy Unit and the Fuelwood Savings Project). c. Foreign experts produce a study. This "product" incorporates the experts' know-how and work results, and it is left to the recipient institution(s) to make use of it (dominating form in the case of the Power Tariff Study and the Human Resources Study; there are strong elements of this "delivery system" also in other cases like support to the Ministry of Finance). d. Through training abroad, foreign know-how is transferred to local professionals who, at a later stage, will have to make use of that know-how in their own work situation (almost non- existing in the Bank-financed technical assistance to Malawi, except the recent, more or less unplanned, training for the Ministry of Finance personnel). 94. Thus, the subprojects differed widely in respect to the "delivery system" of the technical assistance, understood as the system through which foreign expertise is transferred to, and brought to use in Malawi. They also differed widely in their objectives or, one may say, in the scope of their ambitions to contribute to public sector management and institutional development. Since there are no generally accepted categories for a classification of such objectives, such differences can be described here only in a pragmatic way; one may distinguish the following types of cases: (1) There were only some cases where, at least in principle, the technical assistance was meant to lead to a broad-based and sustainable strengthening of a specific institution: this seems to have been the case with the strengthening of the Energy Unit of EPD and the intended support to the Ministry of Finance. (2) There were other cases where it was intended, through the provision of a study, to contribute to specific strategic decisions which would play a decisive role for the development of a certain institution: examples may be the Power Tariff Study for ESCOM. One may say that this was one-shot contribution to institutional development. (3) The Human Resources Study was not oriented to a specific institution; it was providing important basic information potentially relevant for many different institutions. (4) There were other cases where the main objective seems to have been the filling of a specific gap within the personnel - 21 - structure of an institution: this may have been true for the Financial Controller of ADMARC. This is a form of so-called "substitution" technical assistance. (5) The pilot Fuelwood Savings Project had no ambitions of institutional development and public sector management; it was a classical project meant to produce a certain output (research results) and to deliver those results to the target group. 95. Several cases combined elements of several categories; the given classification may not do full justice to them. It also should be noted that in some cases the Bank-financed assistance covered only a part of the foreign inputs; other important contributions to the same "project" had been made by other donors, for instance, by UNDP. In such cases, neither the success nor the problems can be fully attributed to the Bank's assistance. 96. Summarizing, one may say that Bank's assistance given under TA-II, although assumed to support SAL-II (and this was done to some extent), included a wide spectrum of relatively small technical assistance components, greatly differing in their objectives, means of delivery, and comprehensiveness of the resources provtded. 97. Only for some of the components one can see a fairly close link to the policies of Structural Adjustment. Such links, more obvious in the case of TA-I than TA-II, may be seen in the attempted strengthening of the parastatal organizations (ADMARC, ESCOM) or perhaps in the strengthening of core institutions (Energy Unit in the Department of Planning and Development, Ministry of Finance). In the case of the Fuelwood Savings Project and Human Resources Study, it is more difficult to see such links. Consequently, although the TA project was appraised at the same time as SAL-II and was presented to the Board simultaneously with the SAL, some of its components were in fact loosely or not at all related to this SAL. 98. With the exemption of the Fuelwood Savings Project, which was the most technical and practical project related, none of the subprojects can be regarded as a complete success, fully meeting the expectations which were originally formulated or which one reasonably could have had. However, one must also point out that none of the cases evaluated should be regarded as a total failure; all have had at least a partial success; as seen above, they have made some contributions, or there is still hope that such contributions will mature. 99. The subprojects had shortfalls in respect of one or more of the following success criteria which, though not explicitly formulated in a.y of the project documents, can be regarded a necessary condition of successful technical assistance activities. a. The professional quality of the personnel involved and the job performed. b. The correspondence to needs in the sense that the output really fits a felt need, making the recipient institution(s) to absorb and fully utilize the technical assistance output. - 22 - c. The efficiency/economy of producing the output at reasonable cost (particularly without undue and costly delays). d. The training and transfer effect: having local professionals to participate in the work process and absorbing the know-how brought in by expatriates. e. Sustainability: the recipient organizational unit is maintained after the end of technical assistance, and the local professionals trained under the project remain available with this unit for the work still to be done. 100. The cases investigated differ widely as to the extent to which those success criteria have been met or not. In respect to the professional quality (criteria a) no really strong critical comments were voiced. However, some doubts were raised in several cases by the former local counterparts and/or Bank supervision reports, whether really an optimum choice of personnel had been made, particularly referring to the Energy Advisor and the team working on the Human Resources Study. 101. The case of technical assistance not really matching the needs of the recipient institutions can be found in ESCOM which was never really convinced that the tariff problem under study 'tariffs based on long-term marginal cost) was an issue of urgent concern; the study remained an academic exercise. 102. Doubts concerning the efficiency/economy of TA have been and may be raised in respect to several subprojects. The Fuelwood Savings Project suffered long delays in the execution (this must be seen as the only major shortfall in this subproject) while the Human Resources Study ran into substantial difficulties due to budget overruns. 103. The lack of training and know-how transfer was a negative feature common to practically all the subprojects, perhaps with one exception: the execution of the Human Resources Study was done in a form conducive to familiarize the local professionals with relevant techniques for manpower surveys and manpower planning. 104. Project management has been done with very low intensity and with techniques which could be improved. * On both sides, in the Bank and in the Ministry of Finance, documentation on the project/agreement was kept only in the form of purely chronological files, taking in information if and when it comes up. Apart from the few Supervision Reports, there are no other instruments producing information on a routine basis in a standard form. * There is no standardized information on the individual subprojects (subproject data sheet) giving some major data like objectives, targets, responsibilities assigned to different institutions or persons, budget, commitments, actual disbursements. - 23 - * Disbursements were not made/classified by subprojects but only by types of cost (equipment, consultants, etc.); "project managers" have difficulties to find out from the files what commitments and disbursements actually have been made in respect of individual subprojects. * In the course of implementing TA-II, there were periods in which the responsible persons on both sides had great difficulties to decide how many funds were still available under the Agreement and how much room there would be to enter into further activities. V. SUSTAINABILITY 105, Sustainability of technical assistance projects may be defined as the recipient organizational unit being maintained after the end of technical assistance, and the local professionals trained under the project remaining available with this unit for the work still to be done. 106. Consequently, sustainability can only be applied where positive short- and medium-term effects have been achieved, i.e., where a professionally sound output has been produced and has been accepted by the recipient institution and where some training and transfer effects have taken place. A striking example of non-sustainability seems to be the Human Resources Study. Although definitely certain training effects were achieved during the study, such effects were not sustainable: all the local professionals who had participated in this project left the Department; the former Manpower Unit was practically dissolved. In case of the Energy Unit, training effects during the assignment of the advisor seem to have been below optimum, and their sustainability endangered by the fact that the local professionals who had worked with the advisor left the unit for further long-term training abroad. In the case of the fuelwood savings project, sustainable effects are likely to be achieved within the target group (tobacco growers); they were not intended to be achieved in the public institutions (TRA). 107. Overall, the project is unlikely to maintain an acceptable level of net benefits throughout its economic life. Moreover, its impact on institutional development has been generally negligible. VI. LESSONS OF EXPERIENCE 108. The Bank's three SALs and two associated Technical Assistance Projects have supported improvements in the public sector management in addition to reforms in the agricultural and energy sectors and in the operation of the parastatal sector. The Program Performance Audit Report, prepared by OED (Report No. 6833, June 13, 1987) on SALs I and II (Loan 2026-MAI and Credit 1427-MAI) and Technical Assistance I (Loan 2027-MAI) noted that the institutional capacity to -24 sustain reform in Malawi was fragile and required a longer timeframe that provided by the three above mentioned operations. The report also noted that while Malawi was politically committed to reform, it lacked sufficient manpower in order to predict confidently that the new concepts and processes set in train could be operated without risk of failure. 109. The lesson learned from TA-II is that previous observations on the fragile nature of the institutional improvements introduced apply to this operation as well. The TA-II project was not designed to provide institutional support over the considerable period of time needed to internalize fully reforms in economic management. The lesson is, thus, that instead of grouping in an "umbrella" type project overly diffuse TA project's various components, some of them loosely related to SAL operations, one should concentrate on a limited number of high priority reforms to be carried out over a sufficiently long period of time. Such integrated form of technical assistance should better meet Malawi's pressing needs to strengthen national economic management. 110. Where technical assistance is meant to support institutional development (which seems to be a key priority in Malawi) and this means to support social and management processes rather than projects defined in technical terms, a major drawback lies in the rigidity of technical assistance provided under contract basis. For administrative reasons, both sides, the donor and the recipient, prefer to have the objectives, output and activities of technical assistance, already defined for the whole contract period. In this sense, they prefer a "blueprint approach" as the one prevailing in technical projects. However, in the field of institutional development, changes cannot be defined in technical terms; they occur in the form of social processes. The exact path and speed of which cannot be fully predicted in advance. Quite often, even new issues, and the corresponding targets relating to their solution, are identified during the process; the same may refer to the identification of definition of specific needs for foreign expertise of training abroad. An example could be found in the preliminary phase of the broad-based support to the Ministry of Finance when the latter questioned the sustainability of the "blueprint approach." The lesson is that further TA oriented to institutional development should try to avoid or to minimize some other disadvantages of the Bank's present technical assistance, namely the tendency towards a "blueprint approach" (which is a more suitable for capital projects than for assistance to social processes), and the Bank's insufficient supervision of technical assistance projects. 111. Future technical assistance may concentrate in such fields where the Bank has a comparative advantage over donors (for example, projects which help develop macro and microeconomic analysis capabilities and debt reporting). An example can be found in the positive experience with the Energy Planning component. An in-depth energy assessment made several useful recommendations, one of them being to carry out detailed economic analysis of the energy sector and to prepare an Energy Plan. This recommendation was implemented and the preparation of an Energy Plan was financed under TA-II. The lesson, thus, is to concentrate TA on areas where the Bank is able to bring sufficient expertise and resources to obtain positive results. 112. Finally, an important lesson of the project is that technical assistance should not be given in the form of multi-sectoral, general agreement, - 25 - with too many loosely related project components. Extra funds should not be looked upon as a free resource to be used in an uncoordinated way. It seems more appropriate to organize technical assistance country or sectorwise with one or a few major partner institutions being involved, not only administratively but also professionally. To that effect, a long-term institutional development strategy taking into account the time needed to achieve reforms in public sector management, and thus, often going beyond the three or four-year-time horizon of a single TA project should be planned. Successive technical assistance projects planned over a longer period could thus lead to better fulfillment of long-term institutional objectives. 113. As demonstrated by the TA-II project inMalawi, efficient free-standing technical assistance projects require: * Thorough evidence of institutional development strategies; * a clear and precise statement of objectives; * performance indicators for subprojects; * detailed differentiation between types of TA inputs, implying preparation of a detailed program and on-the-job training components, a thorough analysis of the respective expected contributions of short- and long-term experts and a careful definition of the local TA project coordinator's responsibilities; * a clear definition of the Bank's role in the provision of technical assistance on a credit basis vis-h-vis bilateral donors and the UNDP; * a clarification of the link betieen TA and SALs.