Rwanda - Technical Assistance Projects

Document of The World Bank FOR OMCLAL USE ONLY Repot No. 12099 PERFORANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (CREDIT 1217-W) AND TECHNICAL ASSISTANCE PROJECT FOR THE IMPROVEMENT OF PUBLIC FINANCE NANAGEMENT (CREDIT 1565-RW) JUNE 29, 1993 MICROGRAPHICS Report No: 12099 Type: PPAR Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official daties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EOUIVALENTS Monetary unit: Rwanda Franc (RvF) 1981: US$1 - 92.84 1987: US$1 = 79.67 1982: US$1 = 92.84 1988: US$1 * 76.45 1983: US$1 - 94.34 1989: US$1 = 79.98 1984: US$1 - 100.17 1990s US$1 = 82.60 1985: US$1 = 101.26 1991: US$1 - 125.14 1986: US$1 w 8,.64 1992: US$1 = 133.35 ACRONYMS ADETEF Association pour le D6veloppement des Technologies 9conomiques et Financi4res (Association for the Development of Economic and Financial Technologies) BNR Banque Nationale du Rwanda (National Bank of Rwanda) BUNEP Bureau N&tional d'Etudes de Projets (National Bureau for the Study of Projects) CE Communeut6 tconomique Europ6enne (European Community) CIC Comit4 Interminist6riel de Coordination (Interministerial Coordination Committee) ISFP Institut Sup6rieur des Finances Publiques (Higher Institute of Public Finance) MINIFINECO Minist4re des Finances et de 1'Economie (Ministry of Finance and Economy) MINIPLAN MinistAre du Plan (Ministry of Planning) OED Operations Evaluation Department PIP Programms d'Investissement Pluriannual (Multiyear Investment Program) FISCAL YEAR January 1 - December 31 FOR OFFICIAL USE ONLY THE WORLD BANK Washington, D.C. 20433 U.S.A. Office of Director-General Operations Evaluation June 29, 1993 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Rwanda - First Technical Assistance Project (Study Fund I) (Credit 1217-RW) and Technical Assistance for the Improvement of Public Finance Management (Credit 1565-RW) Attached is the Performance Audit Report on Rwanda - First Technical Assistance Project (Study Fund I) (Credit 1217-RW) and Technical Assistance for the Improvement of Public Finance Management (Credit 1565-RW) prepared by the Operations Evaluation Department This Performanct- Audit Report (PAR) reviews these two projects as a "cluster," since both had the common institutional development objective of strengthening the "core" economic ministries, namely the Planning and Finance Ministries. The first Credit was intended to set up a Study Fund in the Ministry of Planning to finance preinvestment studies and to improve the coordination of the Government's project identification and preparation activities. The second Credit's main objectives were to improve budgetary procedures and public finance management in a much deteriorated economic climate characterized by a growing public sector deficit. The first project succeeded in financing a number of studies, most of which led to actual investments. It also had a beneficial impact on inter-agency coordination. However, its impact on institutional development in the Ministry of Planning was limited by excessive staff turnover and the Ministry's weak capacity to evaluate projects and integrate them into a coherent public investment program. Although studies financed in recent years have not led to additional investments owing to the country's current political situation, the audit confirms the PCR-based ratings of satisfactory for project outcome, partial for institutional development, and uncertain for sustainability. The outcome of the second project (for which the PCR has not yet been completed) is rated unsatisfactory. Existing procedures in the Ministry of Finance were not substantially improved, and the project had little impact in terms of solving budgeting problems. The institutional development impact was negligible in the absence of either a clearly defined strategy of public finance reform or specific operational objectives. The audit therefore rates the institutional development impact of the second project as negligible and the sustainability as unlikely. Both projects suffered from the absence of a coherent strategy for the institutional development of Rwanda's key economic ministries. A lesson of experience is that future efforts should be founded upon such a strategy. Attachment This document has a restricted distribution and may be used by recipients only in the perfort ,ince of i .eir official duties. Its contents may not otherwise be disclosed without World Bank authorization. FOR OFFICIAL USE ONLY PERFORMANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (Credit 1217-RW) and TECHNICAL ASSISTANCE PROJECT FOR THE IMPROVEMENT OF PUBLIC FINAICE MANAGEMENT (Credit 1565-RW) TABLE OF CONTENTS Page No. PREFACE . . ......... .. . . . . ........... . . i BASIC DATA SHEET . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii EVALUATION SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix I. IN1RODUCTION.. ............... . . . . . . . . . . 1 II. TECHNICAL ASSISTANCE PROJECT: STUDY FUND (Credit 1217-1W) . . . . . . 3 A. OBJECTIVES . . . . . . . .. . . . . . . . . . . . . . . . . . 3 B. PROJECT CONCEPT AND CHARACTERISTICS . . . . . . . . . . . . . . 3 C. PROJECT IMPLEMENTATION ... . . . . . . . . . . . . . . . 6 D. PROJECT RESULTS . . . . . . . . . . . ....... . . . 9 E. PROJECT SUSTAINABILITY . . . . . . . . . . . . . . . . 10 F. CONCLUSIONS . . . . . ...... . . . . . . . . . . . . . . 11 III. TECHNICAL ASSISTANCE PROJECT FOR IMPROVEMENT OF PUBLIC FINANCE HANAGEMENT (Credit 1565-RW) . . . . . . . . . . . . . . . . . . . . . 13 A. OBJECTIVES . . . . . . . . . . . . . . . . . . . . . . . . . . 13 B. PROJECT CONCEPT AND FEATURES . . . . . . . . . . . . . . . . . 13 C. PROJECT IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . 16 D. PROJECT RESULTS . . . . . . . . . . . . . . . . . . . . . . . . 18 E. PROJECT SUSTAINABILITY . . . . . . . . . . . . . . . . . . . . 19 F. CONCLUSIONS ...... . . . . . . . . . . . . . . . . . . . . 19 IV. LESSONS LEARNED FROM THE TWO PROJECTS. ........ . . . . . . . . 20 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. PERFORMANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (Credit 1217-RW) and TECHNICAL ASSISTANCE PROJECT FOR THE IMPROVEMENT OF PUBLIC FINANCE MANAGEMENT (Credit 1565-RW) PREFACE 1. This is the Performance Audit Report (PAR) for two technical assistance projects for which two credits were approved, Credit 1217-RW in the amount of 4DR 4.3 million on March 16, 1982, and Credit 1565-RW in the amount of SDR 4.9 million, on March 21, 1985. 2. The effectiveness date of Credit 1217 was November 22, 1982, and the closing date was June 30, 1989, two and a half years after the date planned. The last disbursement was in May 1991. 3. The effectiveness date of Credit 1565 was January 10, 1986. and the closing date was June 30, 1992, also two and a half years after the date planned. The last disbursement was on January 19, 1993. Total disbursements amounted to SDR 2.8 million, i.e. only 58 percent of the Credit, and the available balance was canceled in February 1993. 4. The PAR was prepared by the Operations Evaluation Department (OED). It is based on two Project Completion Reports (PCRs),! prepared in 1992 and 1993 by the World Bank's Africa Regional Office, on the President's Reports, on a study of the project files, and on conversations with World Bank staff. It is also based on a detailed study prepared by OED which forms part of a larger study of technical assistance projects in Africa (Report No. 8473). An audit mission visited Rwanda on this occasion, where, in liaison with Government officials, it examined the cost-effectiveness of the projects concerned. The officials concerned are to be particularly commended for their cooperation and assistance in facilitating the preparation of this Report. 5. The PCRs give a detailed account of how the various project components were implemented, and of the role played by the Bank. The PAR describes the principal subjects of concern leading to the approval of the credits, and evaluates the contribution made by the credits toward solving general problems in the area of institution building in Rwanda. The PAR also analyzes the key ' See Project Completion Report, Rwanda, Report No. 10385, for Credit 1217. The Project Completion Report for Credit 1565 was still under preparation when this PAR was completed. factors affecting the projects (Borrower's commitment, effectiveness of a twinning arrangement, monitoring, role of donors, etc.) and assesses the austainability of the two projects. The Report concludes by enumerating the lessons learned from the projects. 6. The draft PAR was sent to the Borrower and the cofinancing agency for comments but none were received. - iii - PERFORMANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (Credit 1217-RW) BASIC DATA SHEET CREDIT POSITION (Amounts in US$ Million) As of Apr. 30, 1993 Credit Original Disbursed Canceled Repaid Outstandini 1217 5.00 5.09 0.00 0.06 6.03 Note: Small discrepancies between varlous amounts are due to fluctuations in the exchange rate of the US$ in relation with SDRa in which the IDA Credit was expressed. CUMULATIVE ESTIMATED AND ACTUAL DISBURSEMENTS FY82 FY83 FY84 FY85 FY86 FY87 FY88 FY89 FY90 Appraisal Estimate (US$4) 0.10 0.60 2.10 3.80 5.00 5.00 5.00 5.00 5.00 Actual (US$H) 0.03 0.05 0.74 1.88 3.17 4.32 4.65 5.09 Actual as 2 of Appraisal (2) 5.0 2.4 19.5 37.6 63.4 86.4 93.0 101.8 Date of Final Disbursement: May 17, 1991 PROJECT DATES Original Actual Initiating Memorandum 01/13/81 Negotiations 12/07/81 01/15/82 Board Approval 01/26/82 03/16/82 Signing 04/05/82 Effectiveness 07/06/82 11/22/82 Credit Closing 12/31/86 06/30/89 - iv - STAFF IVPUTS (ataffteeks) Preappraisal Appraisal Nesotiation Supervision Other oW 1981 .2 .3 6.3 6.8 1982 1.0 3.5 .8 7.4 12.7 1983 4.3 .2 4.5 1984 9.4 9.4 1985 5.4 5.4 1986 7.1 7.1 1987 6.1 6.1 1988 2.0 2.0 1989 11.5 11.5 1990 8.7 8.7 1991 4.7 4.7 1992 1.0 1.0 Total .2 1.3 3.5 61.1 13.9 80.0 MISSION DATA No. of No. of Staff Date of MonthYear Weeks Persons Weeks Report Preparation 04/81 2 3 6 01/13/81 Appraisal n/a 02/18/82 Supervision I 02/83 0.6 n/a Supervision II 09/83 2 n/a Supervision III 10/84 3 n/a Supervision IV 10/85 2 2 4 Supervision V 02/86 1 1 1 Supervision VI 09/87 1 2 2 Supervision VII 08/88 n/a 1 Supervision VIII 02/89 n/a 1 Supervision IX 03/89 n/a 1 Supervision I 09/90 2 aborted!' 2 Completion 02/10/92 V Supervision was aborted due to fighting in Kigali while some members of the team were already in the field. - V - YEAR-END 590 RATINGS Eval. Development Legal management Availability year Overall Obiectives Covenmpt. Performance of punds 82 1 83 1 1 1 1 84 1 1 1 1 85 1 1 1 1 86 2 3 3 2 87 2 2 3 2 88 2 2 3 2 89 2 2 2 3 2 90 2 2 2 3 2 OTHER PROJECT DATA Borrower/Executing Agency: Republic of Rwanda Follow-on Proiects: Project: Public Finance Management Credit No.: 1565-RW Amount: US$4.8 million Board Date: 03/21185 Project: Sectoral and Preinvestment Studies Credit No.: 1796-RW Amount: US$7.4 million Board Date: 05/14/87 - vi - PERFORMANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT FOR THE IMPROVEMENT OF PUBLIC FINANCE MANAGEMENT (Credit 1565-RW) BASIC DATA SHEET CREDIT POSITION (Amounts in US$ Million) As of Apr. 30, 1993 Credit Original Disbursed Cancelled Reaid Outstandin 1565-RW 4.80 3.68 1.12 0.00 4.01 Note: Small discrepancies between various amounts are due to fluctuations in the exchange rate of the US$ in relation with SDRs in which the IDA Credit was expressed. CUMUIATIVE ESTIMATED AND ACTUAL DISBURSEMENTS FY85 FY86 FY87 PY88 FY89 FY90 U91 V92 FY93 Appraisal Estimate (US$4) 0.20 1.40 2.90 3.70 4.30 4.80 4.80 4.80 4.80 Actual (US$N) 0.33 0.88 1.00 1.65 2.62 3.16 3.43 3.68 Actual as 2 of Appraisal (2) 23.6 30.3 27.0 38.4 54.6 65.8 71.5 76.7 Date of Final Disbursement: January 19, 1993 PROJECT DATES Original Actual Initiating Memorandum 09/83 Negotiations 02/27/85 02/27/85 Board Approval 03/21/85 03/21/85 Signing 05/17/85 Effectiveness 09/16/85 01/10/86 Credit Closing 12/31/89 06/30/92 - vii STAFF INPUTS (staffweeks) Preappraisal Appraisal Negotiation Supervision Other Total i984 3.3 5.9 9.2 1985 1.5 5.7 .3 2.7 10.0 20.2 1986 8.2 .6 8.9 1987 14.6 14.6 1988 6.4 6.4 1989 3.8 3.8 1990 2.7 2.7 1991 .5 .5 1992 3.5 3.5 1993 3.8 3.8 Total 4.8 5.7 .3 46.3 16.6 73.7 MISSION DATA No. of No. of Staff Date of Month/Year Weeks Persons Weeks Report Through Appraisal 09/83 N/A N/A N/A Appraisal through Board Approval 10/84 N/A N/A N/A Supervision I 02/86 1 1 1 N/A Supervision II 04-05/86 1.8 2 3.6 N/A Supervision III 09/86 0.8 1 0.8 N/A Supervision IV 11/87 0.8 2 1.6 N/A Supervision V 04/88 0.8 2 1.6 N/A Supervision VI 03/89 2 2 4 N/A Supervision VII 11-12/89 2 1 2 N/A Supervision VIII 09-10/90 2.4 2 4.8 N/A Supervision II 07/91 1.4 1 1.4 N/A Notes: 1. Records of some missions may be missing and some missions supervised several projects. 2. Supervision No. VIII was aborted due to the fighting in Kigali while some members of the team were already in the field. - viii - YEAR-ED 590 RATINGS Eval. Development Legal management Availability Year Wveall Obiectivea Covenants Performance of Furds 85 1 1 1 1 86 2 2 2 2 87 2 2 2 2 8a 3 3 3 1 89 1 3 2 3 1 90 2 2 2 3 1 91 3 3 1 3 1 92 3 3 1 3 1 OTHER PROJECT DATA Borrower/Executing Agency: Republic of Rwanda Follow-on Proiects: Project: Technical Assistance Credit No.: 1217-RW Amounti US$5.00 million Board Date: 03/16/82 Project: Sectoral and Preinvestment Studies Credit No.: 1796-RW Amount: US$7.40 million Board Date: 05/14187 - ix - PERFOLMANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (Credit 1217-1W) and TECHNICAL ASSISTANCE PROJECT FOR THE IMPROVEMENT OF PUBLIC FINANCE MANAGEMENT (Credit 1565-RW) EVALUATION SUMMARY I. INTRODUCTION investment planning and project implementation capacities. 1. The present Report covers two technical assistance projects in 4. To achieve this objective, the Rwanda approved in March 1982 and project comprised: March 1985, respectively. The Performance Audit Reports on these a Fund to finance feasibility two projects were carried out and detailed engineering jointly, since they had objectives in studies; common and were both designed primarily to contribute to organization of training institution building at the seminars in Rwanda; Ministries of Planning and of Finance. provision of UNDP-financed technical assistance (one 2. While the purpose of Credit project economist at the 1217 was to set up a Study Fund at Ministry of Plauning, one the Ministry of Planning to finance financial analyst at BUNEP, one preinvestment studies leading to documentalist at the specific proiects submitted to Documentation Center); and donors, Credit 1565, granted at a time of serious macroeconomic construction and equipping of a problems, was intended to introduce a Documentation Center. number of reforms at the Ministry of Finance with the aim of improving B. CONCEPT resource mobilization and public finance management. 5. The project responded to Rwanda's concern to have a broad II. TECHNICAL ASSISTANCE PROJECT spread of preinvestment studies (STUDY FUND) (CREDIT 1217) relevant to the country's development priorities. Several donors were A. OBJECTIVE prepared to finance investments but were unwilling to finance the 3. The objective of the project necessary preinvestment studies; such was to improve interministerial studies consequently were rarely coordination and thus expand available in the requisite number and -x - quality. The Rwandese authorities of the Rvandese authorities, hoped that an increase in the number evidenced by the appointment of of studies would help increase active and dedicated officials, and investments. The creation of a Study by the provision of counterpart funds Fund responded to that concern. far in excess of the amounts originally planned. C. IMPLEMENTATION 12. The closing date of the credit 6. A portfolio of 42 studies was was June 1989, two and a half years established. In 1989, when the behind schedule. It was not until project was closed, 19 studies were the second half of 1984 that the completed, 9 were still in progress, studies started to be produced at a 10 were in course of preparation, and normal pace. The project design 4 were abandoned. This may be seen underestimated the time it would take as a respectable outcome. to hire experts and put new procedures in place for the 7. An Interministerial identification and preparation of Coordination Coumittee (CIC) was preinvestment studies. established under the aegis of the Ministry of Planning, with D. RESULTS representatives from the technical ministries concerned in the financing 13. The project financed a number of studies to be approved by the CIC. of studies that led to the This procedure proved very useful in preparation of an investment terms of improving contacts among portfolio currently valued at over officials of the various ministries US$400 million. Over half of the and avoiding duplication in the investment projects identified have selection of studies. obtained the necessary financing. 8. Twelve training seminars were 14. Overall, the project organized. However, a large number accomplished its objective of of the participants in those seminars improving interministerial have since left their jobs. coordination (establishment of the CIC), but investment capabilities 9. BUNEP's legal status and were still short of expectations at management style as a government the time of closing. The positive agency make it ill-suited for the effects of the training programs role of consulting firm. The fact increased the Goverment's capacity that no solution had been found to to prepare and program preinvestment this problem by the time of project studies, but staff turnover and the identification led to constant rapid rotation of professionals friction during project seriously undermined the impact of implementation. this effort. Moreover, the government capacity to appraise and 10. The Documentation Center was incorporate projects into an completed 18 months behind schedule investment program also remained because of a large number of weak. Bank supervision ratings for technical and administrative the years 1982-86 were "10 and for problems. the years 1987-90 were "2." 11. One very positive aspect of the 15. Thus, it may be said that project was the degree of commitment overall, despite the fact that the - xi - project led to considerable (d) weakness in incorporating strengthening of the Ministry of identified projects into a Planning's position via-&-via the public investment program; technical ministries, administrative changes and a dispersal of functions (e) persistent lack of cooperation somewhat weakened it internally, at between the Ministry of least during project implementation. Planning and the Ministry of Finance in developing an E. PROJECT SUSTAINABILITY integrated budgetary approach to investments. 16. There were doubts, upon termination of the first Study Fund, 18. The present PAR concludes (as about the sustainability of its did the PCR) that while overall outcomes. However, the results project performance was satisfactory, achieved since then (sectoral the project's impact on institution studies, multiyear programming, building (specifically at the central project pipeline, Ministry of Planning) was no more Documentation Center) indicate that than partial and that the the idea and mechanisms of a Study sustainability of the activities Fund will remain both worthwhile and undertaken remains uncertain. feasible for years to come. There is however a cloud on the horizon in Ill. TECHNICAL ASSISTANCE PROJECT that some studies are being used for FOR THE IMPROVEMENT OF PUBLIC investments whose prospects, given FINANCE MANAGEMENT (Credit Rwanda's present difficult political 1565-RW) and economic climate, remain somewhat bleak. A. OBJECTIVE F. COdCLUSIONS 19. The objective of this project was to improve public finance 17. In conclusion, the project's management. To this end, the project strong points were: (i) the provided for: preparation of sectoral studies and of projects leading to investments; (a) establishment of a pre- (ii) the valuable training activities employment Training Center and provided; (iii) improved coordination remedial training programs for of studies between the Ministry of current staff; Planntng and the technical ministries. But there were also weak (b) improvement of accounting, points, viz: budgetary, tax, and customs procedures; (a) weaknunts in the Ministry of Planning'sF internal (c) strengthening of parastatal organization; sector management, with the assistance of a business (b) excessive turnover of trained management specialist for three personnel; years; and (c) inadequate capacity to prepare (d) improvement of economic policy- cost-effective projects without making (short-term consultant external assistance; services). - xii - B. CONCEPT 25. Few results were obtained in the area of budgetary procedures. 20. The project focused primarily This was due to the lack of concrete on budgetary and accounting operational objectives and to the procedures, since other donors absence of clear instructions at the (Belgium, Switzerland, IMF) were ministerial level (for instance as already providing technical regards the preparation of a new assistance in the areas of resource budgetary nomenclature). No mobilization and parastatal sector significant action was undertaken, management. The project also gave other than the preparation, in 1988, priority to training, an aspect on of a very brief diagnostic study of which Ministry of Finance officials budgetary procedures. Budgetary had particularly insisted. decisions continued to be taken by the Government without consulting the 21. The component to strengthen Directorate General of Budget. capabilities for economic policy- Making consisted, in principle, of 26. The Ministry of Finance now has technical assistance (short-term a number of personal computers (PCs), consultant services) to the for which most of the financing was Secretariat of the CIC on Economic provided by the project. However, no Policy chaired by the Minister of coordinated effort to introduce Finance. No specific program of automated data processing has really studies was prepared, however, and it been made. is not clear whether the Directorate of Studies concerned was actually 27. The component to strengthen aware that it was supposed to be economic policy making capacity was involved with the project. hardly implemented. No specific program of studies to be financed by 22. The lack of a clear strategy the Credit was ever prepared and it and of a precise plan of action with would seem that resources available specific operational objectives for for short-term consultant services the various components were bound to had not been utilized, at least by jeopardize the success of the September 1991. project. 28. In general, the technical C. IMPLEMENTATION adviser posts were ill defined. ADETEF had only an incomplete idea of 23. The Training Institute started the requirements of the posts to be operations several months behind filled, resulting in wide variations schedule. Its annual enrollment in the quality of the experts averaged 30 students, and the cost proposed. This led to a number of per student was high because of such misunderstandings among the Rwandese low enrollment levels. authorities, the Bank, and ADETEF. The members of the 4-5 person expert 24. The impact of the training team initially planned could never be activities was seriously affected by brought together at the sam time, staff turnover. This problem can with the result that the team's only be solved within the framework performance was not very effective. of an overall civil service management reform. 29. Weaknesses in project preparation were largely to blame for the difficulties encountered at the -xiii implementation stage. These E. PROJECT SUSTNABILITY difficulties were heightened by the absence of reliable follow-up and by 32. The training component suffered the heavy turnover of project greatly from the staff turnover supervisory staff. It was no easy problem. To ensure sustainability of matter to get supervision missions to training activities, an across-the- correct mistakes made at the time of board overhaul of civil service project preparation. It would have management would have been necessary. been better to undertake a formal, major review of the project. This 33. Even though a few regulations procedure, considered a possibility were formulated in the area of public and recommended in an OED report and accounting and a few PCs were in an internal memo from the Region installed (haphazardly) at the (May 16, 1989), was regrettably not Ministry of Finance, the project had adopted. no lasting impact on the serious structural difficulties and budget D. RESULTS management problems identified at the start. Finally, the project did not 30. Even considering the positive really help solve the problems of the results achieved in certain areas, parastatal sector (actually dealt such as establishment of the Training with as part of another project) or Institute, and work done in the strengthen Rwanda's economic analysis public accounting sector, the project capabilities (this component has to be described as a failure, as remaining virtually undisbursed). also noted in the PCR (project unsatisfactor: overall, lack of F. CONCLUSIONS sustainability, negligible institutional impact). Although its 34. This technical assistance main objective was to improve public project experienced several serious finance management, there is nothing shortcomings, for the following main to show today that it introduced any reasons: effective procedures for accomplishing that objective. Nor (a) absence of a general strategy were economic policy-making for institution building; capabilities improved by any major actions directly attributable to the (b) underestimation of project. Finally, the project's institutional constraints; impact on institution building was negligible, owing largely to the (c) absence of a coherent program absence of any clear strategy for the of action to help relieve those overall reform of public finance and constraints (translation of to the lack of precise operational policy objectives into objectives. operational objectives, definition of the Government's 31. Bank supervision ratings were obligations and of a "1" for the years 1985 and 1989, "2" timetable); for 1986, 1987 and 1990, and "3" for 1988, 1991 and 1992. - xiv - (d) inadequate Government taken to ensure that projects commitment and assignment of are sufficiently well prepared insufficient borrower and Bank so that resident experts do not staff with reasonable have to improvise at the continuity to manage the implementation stagel project properly; (c) more intense and regular (e) relative failure of the supervision by the Bank is technical assistance, due clearly needed for public partly to poor analysis of the sector management projects. true potential of a twinning arrangement. (d) twinning is, in principle, an effective way of enabling a IV. LESSONS LEARNED FROM THE TWO weaker institution to benefit PROJECTS from the experience of a similar external agency. But 35. A number of lessons may be care must be taken not to leave drawn from the audit of the two the external agency with sole projects: responsibility for helping to implement a project that was (a) a coherent and systematic not designed and prepared in strategy for dealing with its entirety by staff of the problems of institution Bank and Borrower. It is building is a prerequisite for important to recognize the preparation of this type of limits of a twinning technical assistance project. arrangement; Such projects naturally cannot aim to solve the entire range (e) when a technical assistance of institutional problems, but project comes up against should rather be incorporated particularly serious into an overall analysis making difficulties during it possible to identify and implementation, and when assess the true constraints. supervision missions alone are On this basis, specific actions not able to remedy those (establishment of project difficulties, prompt action pipelines, strengthening of should be taken to undertake a budgetary procedures, etc.) can formal review of the project be envisaged; with the aim, in agreement with the Borrower, to look into the (b) such actions need to be soundness of the objectives and identified and prepared in the real prospects for accordance with a program of achieving them. Such a precisely-defined measures and reappraisal of the project to a rigorous timetable. Project assist public finance components (training, management would have been computerization, sectoral and eminently desirable, and it is macroeconomic analyses, etc.) regrettable that a decision to should be individually this effect was not taken in programmed before the credits 1989. are put in place. Care must be PERFORMANCE AUDIT REPORT RWANDA TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (Credit 1217-RW) and TECHNICAL ASSISTANCE PROJECT FOR THE IK1*ROVEMENT OF PUBLIC FINANCE MANAGEMENT (Credit 1565-RW) I. INTRODUCTION 1. The present report covers two technical assistance projects in Rwanda, the first of which was approved in March 1982 and the second in March 1985. The performance audit reports on these two projects were carried out jointly, since they had objectives in common and were both designed primarily to contribute to institution building at the Ministries of Planning and of Finance. 2. However, the two projects were born of different sets of macroeconomic circumstances. In the late seventies and early eighties, Rwanda had the reputation of being a well-managed economy and of being able to avoid wasting its resources. Consequently it was looked upon favorably by the donors, who expressed a willingness to contribute to its development. This required the identification and presentation of viable projects, for which preliminary studies would be needed. Since it would cost money to prepare such studies, it was proposed to set up a special Study Fund. The first IDA credit (Credit 1217) was granted specifically to finance this Fund, and it was understood that coordination of study selection and management would be entrusted to the Ministry of Planning, which would be strengthened for that purpose. 3. The second project was financed under different circumstances. The period 1982/83 marked the beginning of a serious decline in Rwanda's economic situation, chiefly because of sluggish growth in the agricultural sector, deterioration in the mining sector, and falling coffee export prices. 4. The continued increase in imports and the decline in export earnings produced a substantial current account deficit (10 percent of GDP in 1983), and led to the depreciation of the Rwandese Franc. 5. Rwanda became highly vulnerable to changes in the external environment, which demanded constant vigilance and readiness for prompt reaction. However, the country suffered from serious institutional weaknesses that prevented it from making the appropriate adjustments to cope with foreseeable economic changes. In particular, the slowing down of growth led to revenue losses (4 percent of GDP in 1983), which the Ministry of Finance was unable to control. This was the situation that gave birth to the second project to improve public finance management and strengthen the Ministry of Finance (Credit 1565). 6. In sum, while the objective of Credit 1217 was to create a Study Fund at the Ministry of Planning to finance preinvestment studies leading to specific - 2 - Rrojects for submission to donors, Credit 1565 was designed, in a different macroeconomic context, to introduce a certain number of reforms at the Ministry of Finance to help it improve its resource mobilization and public finance management capacities. - 3 - II. TECHNICAL ASSISTANCE PROJECT (STUDY FUND) (Credit 1217-RW) A. OBJECTIVES 7. The project objective, as described in the Credit Agreement, was to "improve interministerial coordination and thus investment planning and project implementation capacity." More specifically, the project was intended to achieve: (a) an increase in the number and improvement in the quality of projects to be submitted for external financing; (b) a strengthening of local project preparation capabilities; (c) improved coordination between technical ministries and the Ministry of Planning, thereby tightening the linkages between investment planning, programming and budgeting. 8. Thus, the project focused simultaneously on the financing mechanism, the coordination mechanism, and programming capabilities. The Ministry of Planning was to be strengthened through improvement of its capacity to prepare specific projects in cooperation with the technical ministries, and to incorporate those projects into an investment program 9. The project's objectives were to be achieved through the following five components: * a Fund to finance feasibility and detailed engineering studies; * organization of training seminars in Rwanda; * provision of UNDP-financed technical assistance (one project economist at the Ministry of Planning, one financial analyst at BUNEP (Bureau of Studies), one documentalist at the Documentation Center), with Rwandese counterparts to be appointed to work with and be trained by the experts; * construction and equipping of a Documentation Center; * provision of three vehicles. B. PROJECT CONCEPT AND CHARACTERISTICS 10. The project responded to Rwanda's concern to have a broad spread of preinvestment studies relevant to the country's development priorities. Several donors were prepared to finance investments, but were unwilling to finance the necessary preinvestment studies, which consequently were rarely available in the requisite number and quality. The Rwandese authorities hoped that an increase in the number of studies would help increase investments. The creation of a Study Fund responded to that concern. 4- (a) Study Fund 11. Project preparation was to be improved through preinvestment studies financed by the Study Fund. Such studies were also intended to be useful for macroeconomic and sectoral analyses (for which the Directorate of Studies was responsible). A related objective, therefore, was to achieve better coordination between macroeconomic and sectoral planning and project preparation. (b) Training seminars 12. The staffs of the technical ministries and of the Ministry of Planning participating in the preinvestment process did not have the necessary training for project identification, preparation and supervision. It was therefore necessary to ensure, through the provision of appropriate training, that standardized criteria were adopted in this area, with consequent improvements in terms of interministerial coordination. Eight two-week seminars were planned for this purpose. 13. It should be noted that the training component was designed in isolation, without any direct linkage to a permanent training mechanism (e.g. an institute for the training of professionals). The aim was simply to support the Study Fund by establishing a network of officials capable of playing a role in the process of preparing preinvestment studies. With the quick turnover of professionals at the technical ministries and the Ministry of Planning, there was a considerable risk that the professionals trained in this way would not be able to stay in post. (c) Technical assistance 14. The principal task of the Project economist was to help set up the Study Fund. He was to encourage the technical ministries to seek out ideas for projects, to help prepare the terms of reference and letters of invitation for the studies selected, take part in the appraisal of proposals and selection of consultants, keep an updated projects portfolio, assist with identification of possible sources of financing, organize training seminars and, in collaboration with his Rvandese counterpart, provide the technical secretariat services needed for management of the Fund. These terms of reference combined certain administrative tasks normally handled by the Rwandese authorities with the type of expertise for which recruitment of a project economist was justified. 15. One of the objectives of the project was to strengthen local project preparation and supervision capabilities. To this end, it was decided in principle to entrust one third of the project-financed studies to BUNEP, the Government Bureau of Studies, whose rigid administrative status made it unsuitable for flexible commercial-type operations. A financial analyst was to be seconded to BUNEP to help it participate more effectively in the procedures for procuring consultants to carry out studies under the Study Fund. BUNEP was thus in a particularly advantageous situation in relation to the other consultants, since it had free access to the services of a financial analyst. 16. The Rwandese authorities were very anxious to strengthen local consultancy capacities, and insisted that BUNEP be granted privileged status reflecting local "preference." The Bank accepted this principle, but without really assessing BUNEP's true capacity for perform the proposed studies. Thus there was both ambiguity and risk in the granting of special resources to BUNEP (secondment of a financial analyst for four years, provision of a vehicle). 17. A documentalist was to be provided to assist with launching of the Documentation Center financed by the project (para. 18). d. Documentation Center 18. The purpose of construction of this Center was to centralize the various studies conducted within the Government, thereby ensuring that new feasibility studies would not duplicate surveys and studies for which adequate data already existed. The ready availability of such data could reduce the cost of the studies. 19. The Center's specific role within the country's overall documentary system was not clearly specified. The President's Report (p. 40) apparently envisaged to set up a national Documentation Center to cover all aspects of economic and social development. This was a very ambitious objective; it might have been better to create a center designed mainly to gather documents related to preinvestment studies, feasibility studies, project reports, sector studies, etc., tied in with the main project objective. 20. It was planned to finance the various project components as follows: PROJECT FINANCING (thousands of dollars) Estimated Actual Resident experts 690a 826' Study Fund 3800 3990 (feasibility studies, etc.) Training seminars 160 194 Documentation Center 180 230 Vehicle, equipment 65 96 Miscellaneous 795 584 TOTAL 5000b 5094" a Financed by UNDP. b Excluding resident experts financed by UNDP and government expenditure estimated at US$1.36 million, compared to the original estimate of US$380,000. C. PROJECT IMPLMENTATION a. The Study fund 21. A total of 42 studies were initiated. By 1989, 19 studies had been completed, nine were in progress, 10 had reached the promoti.onal stage and four had been suspended, closed or abandoned. One might conclude from this that during the entire period, not counting the project start-up phase, studies were launched at the approximate rate of one every two months (42 studies over a 73- month period, or one study every 1.7 months). However, taking account of the preparatory work involved in the launching of the individual studies and the coordination and consultation procedures, the preparation and acceptance of terms of reference, the preparation and implementation of training seminars on study preparation etc., this represents a respectable achievement. 22. The sectoral breakdown of the studies, as planned by the project, was intended, in principle, to match the priorities of Rwanda's Five-Year Plan (1982- 1986), i.e. 35 percent for agriculture, 25 percent for industry, 15 percent for human resources and 15 percent for infrastructure. The actual breakdown was 33 percent for industrial studies, 27 percent for human resources, 25 percent for infrastructure, and only 15 percent for agriculture. 23. The Fund financed a large number of sectoral studies, although the project was intended to fund project studies only. This proved advantageous, since the Directorate of Macroeconomic and Sectoral Studies had carried out or supervised very few sectoral studies, which would normally have served as the framework for preinvestment studies. Two or three of the studies were very helpful in preventing costly investments from being made. 24. Of the 19 studies completed, 10 were considered excellent, two not very good, and the others about average in quality. The satisfactory overall quality of the studies was largely due to the work of the project economist, who paid special attention to the terms of reference for the studies and who did not hesitate to suggest that studies of dubious quality be sent back to the consultants concerned. 25. Pressure was sometimes put on the Fund by various Bank officials to carry out quick-response studies. Apparently this generally took the form of direct approaches to the technical ministries, with no regard for the procedures for coordinating with the Ministry of Planning or with the studies already started. A memorandum from the Division Chiefs involved (technical assistance program/projects) endeavored to resolve this conflict: "We understand that some of you, acting in all good faith and with the aim of speeding things up, have been discussing certain issues relating to the use of the Study Fund directly with the technical ministries. However, this type of approach, which bypasses the Ministry of Planning, is not only counter-productive but also results in delaying or confusing operations." 26. An Interministerial Coordinating Committee (CIC) was established under the aegis of the Ministry of Planning, composed of the technical ministries involved in the financing of studies requiring CIC approval. This procedure was found to be useful in improving contact among officials of the different ministries and in avoiding duplication where study selection was concerned. An indirect benefit was the creation of similar coordination mechanisms in six other sectors (agriculture, economic policy, industrial policy, infrastructure and energy, security, traAning and employment). Each CIC was chaired by the appropriate minister. The Ministry of Planning's CIC was intended to bring together the technical ministries concerned in a coordinated approach to the financing of preinvestment studies. Several ministers, seeing that this was a workable formula, decided to create their own CICs (the Minister of Finance, for example, formed an economic policy CIC chaired by him) in order to improve cooperation among ministries in other areas. b. Traininix Seminars 27. Twelve seminars were efficiently organized and provided training to 550 Rwandese officials. Although the seminars were not implemented exactly as envisaged in terms of topics covered and frequency, they resulted in the participants being better equipped to prepare and analyze preinvestment studies, to draw up terms of reference, and to coordinate action at the intersectoral level with the various ministries. A large number of seminar participants have already left their jobs, however, which jeopardizes the sustainability of the training. c. Strengthening of Local Consulting Firms 28. The expert hired to help strengthen BUNEP worked for less than two years (1985-86) on the assignment. He was primarily involved in helping BUNEP carry out three studies awarded to it within the context of the Study Fund. The goal of awarding one third of the study contracts to BUNEP was not achieved. BUNEP was found to be incapable of delivering the quality of work required or of keeping to agreed deadlines. It had no definite profile or proven specialty. Under such circumstances, the intended "promotion of local enterprise" remained problematic. 29. BUNEP's legal status and management style as a government agency made it ill-suited for the role of consulting firm. The fact that this problem had not been solved by the time of project identification was a constant source of friction during project implementation. d. Documentation Center 30. Because of numerous technical and administrative problems, construction of the Center was completed 18 months behind schedule. The Center became very well equipped and a highly experienced expert was recruited (also behind schedule) to organize it. Three types of difficulties were encountered, however, which hindered the Center's success, at least at the beginnings * for some curious reason, the Center was attached to the Ministry of Planning's Directorate of Statistics; * no provision was made for the Center's locally-hired management staff to receive any higher-level training; - 8 - * the Center's specific mission or classification within the Governmert and in relation to other existing documentation centers was clarified. e. Proiect Management 31. The project was remarkably well managed, but the resident technical advisers had to devote an excessive amount of their time to management activities, even though these were not officially included in their terms of reference. For the most part, Rvandese officials were not available during project implementation to take over project management. A solution was apparently found to this problem in 1988/89. 32. An encouraging aspect was the level of commitment on the part of the Rwandese authorities, as evidenced by the assignment of active and dedicated officials to take charge of document preparation and interministerial coordination, by the provision and equipping of physical facilities, and by compliance with the covenants of the Credit Agreement (delivery of reports, monitoring of procedures governing the use of consultants, audits, etc.). Above all, not only did the Government make a timely contribution of counterpart funds, but those contributions far exceeded (US$1.36 million) the amount initially planned (US$0.38 million). This was attributable to the fact that Rwanda agreed to shoulder a larger share of the local currency costs of the studies awarded to the consulting firms. 33. Project supervision by the Government took place primarily at the weekly meetings of the Interministerial Coordinating Committee. Some tripartite reviews (UNDP, as cofinancier of the technical assistance project, the Bank, as executing agency for the UNDP project, and the Government) resulted in positive conclusions, particularly concerning the need to renew assistance to the Study Fund (Credit 1796 of 1987 for US$ 7.4 million). These reviews skirted the institutional stumbling blocks. However, the above- mentioned institutional problems (BUNEP, Documentation Center, Ministry of Planning organization, turnover of trained staff) remained unresolved. 34. Follow-up on the part of the Bank was irregular. The weakness or outright absence of supervision gave the resident expert in charge of the Study Fund a large amount of influence. Mention must be made, however, of the constant involvement of the Bank's Resident Mission (at least from 1982 to 1986), which received a partial delegation of authority to approve project studies, thus helping to speed up the approval process. 35. A key role was performed by the project economist hired to work at the Ministry of Planning. He was the real authority behind the project to which he was extremely devoted. Later on, (i.e. in 1987) he was assisted by a macroeconomist directly financed by the Bank and who, from 1987 onwards, was directly financed by the Bank and who proved extremely valuable after the departure of the project economist and pending his replacement by a new project economist in 1989. 36. The credit was not closed until June 1989, i.e. two and a half years behind schedule. It was noL until the second half of 1984 that studies started to be produced at a normal pace, the project design having underestimated the initial -9 - time needed to hire experts and put new procedures in place for the identification and preparation of studies. D. PROJECT RESULTS a. Impact of Studies on Investments 37. The project financed or helped to finance a series of studies which resulted in the compilation of a pipeline of investment projects, currently estimated at over US$400 million. Financing has been secured for over half of these projects. Several large projects have been launched (cf. UNDP Evaluation Report, August 15, 1991. This report also covers the four completed studies financed by Study Fund II under Credit 1796). b. Effectiveness of Procedures 38. A framework for implementing and managing the financing plans for the studies was set in place. Meetings of the Study Fund's Studies Committee are held regularly. Standardized formats for terms of reference, contracts and bidding procedures are now in routine use, and reminders are sent concerning procedures and methodology for project appraisal and use of the study documents. The technical Ministries are routinely applying this methodology. 39. In 1990, two donor countries (Belgium and France) decided to finance Study Funds. These Funds follow the same procedures and the same system as the Bank- financed Study Fund: clearance of study proposals by the CIC, centralized management of all funds, use of the contract format and model terms of reference, letters of invitation, role of the ministries, etc. c. Institution Buildinx 40. Overall, the project accomplished its objective of improving interministerial coordination (establishment of the CIC), but investment planning capabilities were still short of expectations by closing. The positive effects of the training activities have certainly increased the Rwandese Government's capacity to prepare and program preinvestment studies, but staff turnover and the rapid rotation of professionals seriously undermined the impact of this effort. 41. The Ministry of Planning significantly improved its position both vis-&-vis other ministerial departments and in relation to concerned donors. However, while the Ministry of Planning thus strengthened its position in terms of its external relations, developments within its internal structure were not so positive. The number of units within the Ministry's organization increased considerably, leading to problems in terms of resources, circulation of information, coordination of actions, etc. This proliferation of new units came about without any prior action to determine the number and nature of staff needed for their operation or to assess the expected budgetary impact. 42. Although the rapid creation of the Fund can be attributed to the urgency of the situation and the impatience of donors and investors, it would in any case have been a good idea to include a general diagnostic review of the institutional environment among the first studies to be done. The true diagnostic study has - 10 - yet to be performed, along with the one on the Government as a whole. Such an evaluation of the administrative and institutional situation would not have been necessary had it simply been a matter of setting in place a mechanism to provide the Ministry of Planning with the funds to carry out studies. But the objective of the project was also -- and most importantly -- to improve interministerial coordination and investment planning capacity. The absence of an institutional diagnostic study therefore made i, impossible to get to the root of the problem. 43. The question of the consolidated investment budget (which should have included the Study Fund and even given it priority) could then have been discussed, instead of not being raised until several years later during negotiations for Credit 1565 (Ministry of Finance), Credit 1796 (Study Fund II), and the SAL. As the executing agency of the UNDP project (supporting the Study Fund) the Bank had undertaken to set in place a national directory of projects, but this commitment remained unfulfilled because no funds were earmarked in the budget to accomplish this objective. 44. A prior diagnostic study of the institutional environment would also have made it possible to analyze the situation of BUNEP, which the Rwandese authorities saw as being able to play an extremely important role in strengthening local capacity for project preparation. The absence of any analysis of the institutional aspects meant that all efforts to develop local capacity through BUNEP were doomed to failure from the start, since its legal status and resultant methods of management were exactly the opposite of what was needed to successfully operate and develop a consulting firm. 45. It is generally true to say, therefore, that although the Ministry of Planning's position was significantly strengthened in relation to its external environment (interministerial coordination) thanks to the project, its internal position was somewhat weakened by the proliferation of new units and fragmentation of responsibilities, at least during project implementation. E. PROJECT SUSTAINABILITY 46. The first Study Fund was supplemented by a second Fund (Credit 1796), which was approved in May 1987 but did not become effective until August 1988, primarily because sufficient funds were still available from the first Study Fund. Besides replenishing the Fund, the new project aimed to overcome a weakness that had come to light in the course of the first project, namely the absence of sectoral and sub-sectoral studies to provide better guidance in the selection of feasibility studies. 47. This project was also supported by a UNDP project aimed not erly at improving planning and investment capabilities through the execution of studies but also at putting in place a rolling investment programming system (Public Investment Program - PIP and Public Expenditure Program - PDP), along with a central project pipeline. The objective of Study Fund I, i.e. to strengthen linkages between planning, programming and budgeting was thus expressly reflected. 48. A central pipeline of over 500 projects (which the first Fund had been unable to establish) has been in operation since 1990. It makes it possible to - 11 - analyze a series of variables in terms of programming and forecasting, including impacts at the macroeconomic level. 49. The Documentation Center contains a large number of documents, including all the country's studies, the aim being to reduce the cost of pre-investment studies. It now has a budget and is operational, and efforts are under way to redefine management regulations and to train the personnel needed. 50. 1989 saw the beginning of a trend to focus more and more on macroeconomic issues and investment programming. The expatriate project economist and macroeconomist assigned to the Study Fund both contributed to this effort. After preparing the 1990-92 PIP, Ministry of Planning staff played an active role in the 1991-93 PIP. The PIP was first discussed in Council of Ministers in 1991. It was a key element in the agreements between the Bank and the Rwandese Government relating to the 1991-93 SAL. Improved coordination was introduced between planning, investment programming, and budget processing. 51. A certain number of studies were financed by Study Fund II. The project also helped to identify several studies for financing. Disbursements were slowed considerably by the political and economic situation, which impacted negatively on the new investments identified by the studies as suitable for funding. (By January 1993, only 28 percent of Credit 1796 had been disbursed.) 52. As mentioned earlier (paragraph 39), two donors have used the Study Fund model and procedures to create their own study funds, an evidenc of the austainability of the mechanism set in place. 53. While there may have been certain doubts, upon te-mination of the first Study Fund, about the sustainability of its outcomes, the results achieved since then (sectoral studies, multi-year programming, central project pipeline, Documentation Center) would seem to indicate that the idea and mechanisms of a Study Fund will remain both worthwhile and feasible for the years to come. There is however a cloud on the horizon in that some studies are being used for investments whose prospects, given Rwanda's present difficult political and economic climate, remain somewhat bleak. F. CONCLUSIONS 54. Overall, the project was well prepared and properly implemented. Nevertheless, it developed along lines different from the original concept. At the outset, the main focus was on preinvestment studies; increasingly, however, sectoral studies (which actually proved highly useful) and project preparation studies were produced. Owing mainly to the prevailing political situation, some uncertainties still remain concerning the financing of productive projects stemming from the studies conducted. 55. Training activities (seminars, counterpart training) contributed enormously to improved interministerial coordination and also to the establishment of procedures for the CICs. Sustainability of these outcomes, nevertheless, will depend largely on the way the government solves the problem of staff turnover. A strategy has yet to be formulated in this area. - 12 - 56. The project's institutional aspects were routinely underestimated. Shortcomings in terms of personnel, organizational units and procedures were still conspicuously present at project completion. A more coherent approach was adopted in subsequent projects financed by the Bank and UNDP, but an institutional "diagnostic study" of the public sector would have been useful from the outset in helping avoid the pitfalls encountered. 57. In sum, the project's strong points were: (i) the preparation of sectoral studies and of projects leading to investments; (ii) the valuable training activities provided; and (iii) improved coordination of studies between the Ministry of Planning and the technical ministries. But the project also had several shortcomings, viz: (a) weakness of the Ministry of Planning's internal organization; (b) excessive turnover of trained personnel; (c) inadequate capacity to prepare cost-effective projects without external assistance; (d) weakness in incorporating identified projects into a public investment program; and (e) a persistent lack of cooperation between the Ministry of Planning and the Ministry of Finance in developing an integrated budgetary approach to investments. 58. This Performance Audit Report concludes (as does the Project Completion Report) that, while overall project performance was satisfactory, the project* impact on institution building (specifically at the Ministry of Planning) was no more than partial, while the sustainability of the actions undertaken continues to be uncertain. - 13 - III. TECHNICAL ASSISTANCE PROJECT FOR IMPROVEMENT OF PUBLIC FINANCE MANAGEMENT (Credit 1565-RW) A. OBJECTIVES 59. The main objective of this project was to improve public finance management. To that end, the project aimed to: (a) train staff in the Ministry of Finance and Economy (MINIFINECO) and financial managers in other technical ministries, parastatals, and development projectsi (b) improve procedures-for budget preparation and execution; (c) strengthen Government capabilities for economic policy-making. 60. To accomplish these three objectives, the project had several components: (a) establishment of a pre-employment Training Center to provide training and remedial training for current staff; (b) improvement of budgetary, accounting, tax, and customs procedures, through% * technical assistance (budget and public accounting specialists, short backetopping missions); * phased introduction of automated data processing (with the help of a systems analyst and a computer programmer); (c) strengthening of parastatal sector management, with the assistance of a business management specialist for three years; (d) improvement of economic policy-making (short-term consultant services); and (e) vehicles and equipment. B. PROJECT CONCEPT AND FEATURES 61. The aim of the project was to bring about reforms in the public finance sector. It was noted that: (a) Rwanda's macroeconomic objectives remained essentially qualitative in nature in the absence of any clearly defined sectoral strategies and in view of the shortage of projects identified as economically profitable and the lack of solid data on available resources. As a result, the objectives set were rarely achieved, the order of priorities of the investments was inadequate and the allocation of scarce resources was inefficient; (b) The budgetary process was incomplete. In particular, current expenditures generated by investments and the financial impact of the parastatal sector were not fully taken into account. This stemmed from, among other things, outdated procedures, the dual nature of the budget, divided, as it was, into recurrent and investment expenses (each category being handled by a different Ministry), the fact that investments financed by donors were only partially taken into account, and the absence of any multi year public investment program (PIP) to link investment programs and their budgetary consequences; (c) The impact of internal and external factors on public finance in Rwanda had been underestimated. Adjustments were slow to come into effect and were made after the event. Capabilities for preparing economic policy were still at the embryo stage. 62. To help solve these problems, the project was to focus on four aspects in particulars (a) procedures for preparing and executing both the ordinary (or recurrent) budget and the development (or investment) budget and the linkage between these two budgets; (b) resource mobilizations strengthening of the tax and customs authorities and of economic policy-making (tariffs, exchange rates, taxes); (c) improvement of the financial management of the parastatal sector so as to reduce this sector's burden on the budget; and (d) strengthening the training of civil servants, particularly in the area of sector and budget programming and project appraisal. Criteria for the hiring, assignment and promotion of personnel were also to be clarified in order to enhance the professionalism of staff. 63. The project focused primarily on budgetary and accounting procedures, since other donors (Belgium, Switzerland, the IMF) were already providing technical assistance in the area of resource mobilization (to the tax authorities) and parastatal sector management (database to facilitate decisionmaking). The project also gave priority to training, an aspect particularly defended by Ministry of Finance officials, while stressing the linkages with various projects (UNDP, EC) and also with an IDA project, MINAGRI Institutional Development. 64. The concept of improving oversight of public finance management was thus intended to trigger a chain of comprehensive reforms, ranging from the act generating the revenue and expenditures (i.e. the budget, preceded by the formulation of a programming and budgeting policy) to the recording of the operations in the accounts. To do this, increasing use was to be made of computer resources for the automation of operations and management auditing (management charts for each Directorate General) and for guidance in economic policy-making (consolidated management chart at the Ministry of Finance). Finally, a linkage was to be established with the Ministry of Planning with respect to both the ordinary budget and the development budget. - 15 - 65. The training component consisted in establishing an Institut Supirieur des Finances Publiques - ISFP (Higher Institute of Public Finance). For such a small organization (30 students per year), this Institute was governed by a particularly cumbersome legal and institutional structure -- a Conseil de l'Institur (Institute Council), a Conseil de Direction (Governing Board), teaching appointments requiring the approval of the Ministry of Higher Education, etc. Its relationship to other Rwandese civil service training centers was not clearly defined. No real diagnostic study of the general status of civil service training, remedial training, continuing education or refresher training was performed before the ISFP was set up. Such a study would have made it possible to identify an across-the-board approach to training in general, which could have included the specific issue of training of professional staff of the Ministry of Finance. 66. The budget component focused particularly on improving procedures for budget preparation and execution and revision of the accounts nomenclature, which was to be computerized. This objective was to intended to lead inevitably to an analysis of the validity of the system as a whole and, based on this analysis, to the formulation of strategy for overall budget reform. The same was true of the component relating to public accounting procedures. The terms of reference for the resident specialists stipulated a certain number of tasks but, taken as a whole, these tasks did not amount either to a comprehensive program or to a phased plan of action. A strategy and plan of action remained to be decided on, operational objectives still needed to be set, a timetable had yet to be established, and specific resources remained to be allocated. No operational objective was formulated in the area of budgeting and no in-depth diagnostic study was made of the various steps in the phased plan of action. Finally, the actions envisaged in the area of data processing were supposed to be based on a Master Plan, which had not been prepared by the start of the project. 67. The project component aiming at improving the parastatal sector performance was not well defined and did not include any timetable of reforms (based, for example, on a diagnostic survey of institutional aspects, the gathering of data as a basis for forward management of the financial impact of decisions made by parastatal enterprises, management by directors (administrateurs] representing the Government, etc.). A clear mandate for the reform of the sector was also absent at the policy-making level. 68. The component to strengthen capabilities for economic policy-makinit consisted, in principle, of technical assistance (in the form of short-term consultant services) to the Secretariat of the CIC on Economic Policy, chaired by the Minister of Finance. No specific program of studies had been prepared, however, and it is not clear whether the Directorate of Studies in question was actually aware that it was supposed to be involved in the project. 69. The lack of a clear strategy and of a precise plan of action with specific operational objectives for the various components were bound to jeopardize the success of the project. 70. Bank officials had suggested that the Rwandese authorities use the services of ADETEF (a foreign aid agency of the French Ministry of Finance) to help them - 16 - carry out the project. This "twinning" offered the advantage of free technical assistance (except for travel expenses) for short-term consultants. ADETEF was also able to offer the services of long-term paid advisers, who would be paid but ADETEF would not have exclusive control over their selection. The Bank perhaps thought from the outset that ADETEF would tackle the general strategy issues of setting in place institutional support for the process of public finance reform. However, these issues did not fall within any of the project components (for which ADETEF was partially responsible, particularly where training and the provision of a specialist were concerned), but should rather have been dealt with at the time the project was being prepared by the Bank and the Rwandese authorities themselves. There is no indication that the latter really wanted to entrust ADETEF with activities related to the overall reform of the country's public finances. There was therefore a serious risk of misunderstanding from the very start as to the exact role to be played by ADETEF in overall project implementation and a danger of over-reliance on this agency for the accomplishment of operations for which it did not feel responsible and of which it undoubtedly had not fully grasped the internal logic. 71. Throughout the period of project identification and preparation, the Rwandese authorities took all the necessary measures to set the project in place. Work carried out in conjunction with the Bank's experts clearly defined the projects to be targeted, but the level of preparation was apparently insufficient with respect both to the individual components and to overall project implementation. C. PROJECT IMPLEMENTATION a. Training 72. The Institute came into operation in 1987, a few months behind schedule. It had an intake of an average of 30 students per year. The cost per student seems high (about US$3,000 per year, i.e. ten times the annual per capita income) because of the small number of students. The amount budgeted for developing the premises was not fully utilized because the costs had been overestimated. The Institute made a positive contribution toward achieving the objective of training future civil servants, but weaknesses in the area of personnel management (absence of linkages between budget management and personnel management, between organizational units and job descriptions) undermined the prospects of sustainability. The same holds true for the seminars organized as part of the remedial training program. None of the study trips abroad actually took place. 73. The impact of the training activities was seriously affected by staff turnover. This issue can only be solved within the framework of an overall review of civil service personnel management. b. Budgetary and Accounting Procedures 74. Since the operational objectives were not known, and since the Directorate had not received any clear mandate from the political level to accomplish a given number of procedures within a given time period (e.g. preparation and computerization of a new nomenclature), this component could hardly be expected to yield any results. No significant action was undertaken, other than the - 17 - preparation, in 1988, of a very brief diagnostic study of budgetary procedures. The budget continued to suffer the consequences of decisions taken without the Directorate General of Budgeting being consulted, and was unable to serve as an instrument of forward management of the public finances or as an uncircumventable component of all financial management. 75. The "accounting procedures" component is definitely the one where the most progress was made, thanks in particular to the hiring of effective specialist by ADETEF. The absence of progress in the area of budgetary reform seriously compromised the effectiveness of the work done in public accounting. While a new accounting regulation that deals with both budgetary and accounting issues has been prepared, a sound implementation strategy has yet to be determined. Certain procedures have been modified. For example, salaries are now paid through directly crediting employees' bank account rather than paying them in cash or giving them paycheks. c. Computerization 76. A "master" plan was prepared in 1986. This plan was general in nature and recommended an ambitious approach, involving the use of computers. Partly at Bank insistence, a more gradual approach was adopted, involving the use of a few PCs. Because of the lack of supervision, of sufficient familiarity with bidding procedures, and of staff training programs, it was only after an extremely long delay that the computer equipment was installed. The Ministry of Finance is now equipped with a number of PCs financed mainly out of project funds, but no coordinated action in terms of computerization (particularly at the Directorate of Taxation, as originally planned) was actually undertaken. d. Parastatal Sector 77. It was not until November 1988 that assistance from a resident specialist provided by ADETEF actually arrived, i.e. at least three years behind schedule. In May 1988, the Bank had begun to prepare a separate public enterprise reform project. To this end, the sum of US$450,000 from Credit 1565 was used to finance preparatory studies. The ADETEF specialist was transferred to the new project (Credit 2113) at the end of 1990 and stayed in Rwanda until December 1992. The "Parastatal Sector" component of Credit 1565, in any case, had been poorly identified and remained very vague. A diagnostic study of eleven enterprises was carried out but its conclusions were examined within the framework of Credit 2113. - 18 - e. Economic Policy-Making Capability 78. The component to strengthen economic policy making capacity was hardly implemented. No specific program of studies to be financed by the Credit was ever prepared and it would seem that resources available for short-term consultant services had not been utilized, at least by September 1991. f. Technical Assistance 79. Generally speaking, the technical adviser posts were ill defined. The work to be undertaken was described (incompletely) but no mention was made of the profiles of the specialists to be hired or of the precise professional experience required. ADETEF had only an incomplete idea of the requirements of the positions to be filled, resulting in wide variations in the quality of the experts proposed. This led to a number of misunderstandings among the Rwandese authorities, the Bank and ADETEF. g. Project Management 80. Weaknesses in project preparation were largely to blame for the difficulties encountered at the implementation stage. These difficulties were compounded by the lack of reliable supervision (except during a short period in 1988) and by the heavy turnover of project monitoring staff. It was no easy matter to get a supervision mission to correct mistakes made at the time of project preparation. It may have been better to undertake a formal, major review of the projects. This procedure, considered a possibility and recommended in an OED report and in an internal memo from the Region (May 16, 1989), was regrettably not adopted. 81. The Project Coordinator (Director General of Taxation) did not have the requisite institutional status to oversee all the missions involved in the project. He was not in a position to report to the Rwandese authorities and the Bank on the degree of achievement of the project's objectives or, in turn, to require a given level of performance from other Rvandese officials. As a result, nobody was really in charge of public finance reform, a fact that contributed to the growing disenchantment among Ministry of Finance staff and management in dealing with difficulties relating to the project. The fact that the technical assistance was financed by a reimbursable credit also contributed to the Ministry's hesitation to use available funds. At Credit closing on June 30, 1992, two and a half years behind schedule, only 58 percent of the funds had been used. D. PROJECT RESULTS 82. Even considering the positive results achieved in certain areas, such as establishment of the ISFP, and work done in the public accounting sector, the project has to be described as a failure. Its main objective was to improve public finance management. But there is nothing to show today that it introduced any effective procedures for accomplishing that objective. Nor were economic policy-making capabilities improved by any major actions directly attributable to the project. Finally, the project's impact on institution building was also - 19 - negligible in the absence of any clear definition of a strategy for the overall reform of public finance or identification of precise operational objectives. E. PROJECT SUSTAINABILITY 83. The training component suffered greatly from the staff turnover problem. This was not a localized problem that could have been solved within the framework of this limited project. To solve this problem, the general issues of personnel management (e.g., link between assignments and budgetary positions, promotion system, etc.) and employment procedures (recruitment, reassignments, dismissals) should have been tackled. But such action should be included in the broader framework of a general strategy for public sector management that will gradually remedy the institutional deficiencies, particularly in the area of personnel. 84. Even though a few regulations were formulated in the area of public accounting and a few PCs were installed (haphazardly) at the Ministry of Finance, the project had no lasting influence on the serious structural and budget management problems identified at the start. Finally, the project did not really help solve the problems of the parastatal sector (actually dealt with as part of another project) or strengthen Rwanda's economic analysis capabilities (this component remaining virtually undisbursed). F. CONCLUSIONS 85. This technical assistance project experienced several serious shortcomings, for the following main reasons: (a) absence of a general strategy for institution building; (b) underestimation of institutional constraints; (c) absence of a coherent program of action to help remedy those constraints (translation of policy objectives into operational objectives, definition of the Government's obligations and of a timetable); (d) inadequate Government commitment and assignment of insufficient borrower and Bank staff with reasonable continuity to manage the project properly; and (e) relative failure of the technical assistance, due partly to poor analysis of the true potential of a twinning arrangement. 86. This PAR concludes that the institutional development impact of this project has been negligible and that the sustainability is unlikely. The overall rating is unsatisfactory. * - 20 - IV . LESSONS LEARNED FROM THE TWO PROJECTS 87. A number of lessons may be drawn from the audit of the two projects: (a) a coherent and systematic stratev for dealing with problems of institution buildini is a prerequisite for preparation of this type of technical assistance project. Such projects naturally cannot aim to s-Ave the entire range of institutional problems, but should rather be incorporated into an overall analysis making it possible to identify and assess the true constraints. On this basis, specific actions (establishment of project pipelines, strengthening of budgetary procedures, etc.) can be envisaged; (b) such actions need to be identified and prepared in accordance with a program of precisely-defined measures and a rigorous timetable. Project components (training, computerization, sectoral and macroeconomic analyses, etc.) should be individually programmed before the credits are put in place. Care must be taken to ensure that projects are sufficiently well prepared so that the resident experts do not have to improvise at the implementation stage; (c) projects providing assistance for institution building are often complex. This means that ample resources and extremely competent experts have to be made available, not only at the preparation stage but also, and perhaps more importantly, during project implementation. In the area of technical assistance, monitorin is always important, but particularly so where public sector management projects are concerned; (d) twinning is, in principle, an effective way of enabling a weaker institution to benefit from the experience of a similar external agency. But care must be taken not *to leave the external agency with sole responsibility for helping to implement a project that was not designed and prepared in its entirety by staff of the Bank and Borrower. It is important to recognize the limits of a twinning arrangement; (e) when a technical assistance project comes up against particularly serious difficulties during implementation, and when follow-up missions alone are not able to remedy those difficulties, prompt action should be taken to reappraise the project in progress and, with the Borrower, to look into the soundness of the objectives and the real prospects for achieving them. Such a reappraisal of the project to assist public finance management would have been eminently desirable, and it is regrettable that a decision to this effect was not taken in 1989.