Uganda - Second and Third Technical Assistance Projects

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 14706 PERFORMANCE AUDIT REPORT UGANDA SECOND AND THIRD TECHNICAL ASSISTANCE PROJECTS (CREDITS 1434-UG AND 1951-UG) JUNE 28, 1995 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents Currency Unit = Uganda Shillings (USh) 1984 US$1 USh 3,597.0 1989 US$1 USh 223.1 1985 US$1 USh 6.7 1990 US$1 USh 428.9 1986 US$1 USh 14.0 1991 US$1 USh 734.0 1987 US$1 USh 42.8 1992 US$1 USh 1,133.8 1988 US$1 USh 106.1 1993 US$1 USh 1,195.0 Abbreviations and Acronyms CSRP Civil Service Reform Programme EFMP Economic and Financial Management Project ERC Economic Recovery Credit GDP Gross domestic product ICBP Institutional Capacity Building Project ID Institutional development IDA International Development Association IMF International Monetary Fund MOF Ministry of Finance MPED Ministry of Planning and Economic Development OED Operations Evaluation Department PAR Performance Audit Report PCR Project Completion Report PSCA Public Service and Cabinet Affairs SAR Staff appraisal report TA Technical Assistance UIS Unified Incentive Payments Scheme UNDP United Nations Development Programme Fiscal Year July 1 - June 30 FOR OFFICIAL USE ONLY THE WORLD BANK Washington, D.C. 20433 U.S.A. Office of Director-General Operations Evaluation June 28, 1995 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Uganda - Second and Third Technical Assistance Projects (Credits 1434-UG and 1951-UG) Attached is the Performance Audit Report (PAR) for the Uganda Second and Third Technical Assistance projects (Credits 1434-UG, approved in FY84; and 1951-UG, approved in FY89) prepared by the Operations Evaluation Department (OED). Credit 1434 (TA II) was closed in June 1992. Credit 1951 (TA III) will be closed in December 1995. Both projects had the twin objectives of facilitating the short-term design and implementation of economic reform measures and fostering longer-term institutional development (ID) and capacity building related to improving economic management. The main conclusions of the audits are that TA II had only a marginally satisfactory outcome whereas TA III had a highly satisfactory outcome in the context of difficult country conditions and constraints affecting its implementation. The audit's outcome rating for TA II differs from that of the PCR which rated it as satisfactory. The audit finds, however, that TA II sought to do too many things in a political and institutional environment that was disenabling. The audit also finds the ID impact of TA II to have been modest rather than substantial, as the PCR found it. Both the audit and the PCR agree upon a sustainability rating of uncertain for TA II. The audit and PCR are in agreement on all the ratings for TA III, i.e., highly satisfactory for outcome, substantial for ID impact, and uncertain for sustainability. An improved environment, strong ownership on the Government side, and good supervision on the Bank side were the most critical determinants of the improved performance of TA III. An encouraging finding is that there appears to have been a steady progression in the "quality at entry" of the Bank's TA projects in Uganda. This reflects a learning process aided by a high degree of continuity of staff involved, on both the Government and Bank sides. The principal lesson identified in the audit is that "management matters most." Good management and strong supervision are critical to making all the instruments of TA work well. For example, it was found that well-chosen and well-managed long-term expatriate advisers had in many instances been effective in fostering capacity building. Another lesson is that in a highly aid- dependent country such as Uganda, particular care must be taken to harmonize donor policies on salary supplements for civil servants in the context of an agreed exit plan to phase out those supplements. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  FOR OFFICIAL USE ONLY Contents Preface ............................................................ 3 Basic Data Sheet ..................................................... 5 Evaluation Summary ................................................. 11 1. Introduction and Overview ....................................... 19 Why Audit These Projects? ....................................... 19 Scope and Structure of the Report .................................. 19 A Bit of History: How Could TA Succeed in the Conditions of the 1980s? ..... 20 Evolution of Five TA Projects ..................................... 21 2. Objectives, Design, and Relevance .................................. 25 Main Findings and Conclusions .................................... 25 Objectives of TA II and III ....................................... 25 Design Features: Process, Ownership, Project Management Structure ......... 27 Relevance ................................................... 28 3. Implementation and Outcome Ratings ............................... 31 Main Findings ................................................ 31 Outcomes by Main Component/Beneficiary ............................ 31 Borrower and Bank Performance ................................... 38 4. Selected Issues and Sustainability .................................. 41 Salary Supplements ............................................. 41 Donor Coordination ............................................ 43 Home-grown Distortions ........................................... 45 Sustainability .................................................... 46 5. Conclusions and Lessons ........................................ 49 Main Conclusions: Summary ........................................ 49 Lessons ........................................................ 49 Attachment A: Letter from the Ministry of Finance and Economic Planning ....... 51 This report was prepared by Robert Armstrong (Task Manager) and Sola Mahoney (Consultant) who audited the project in December 1994. Norma Namisato provided administrative support. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 Tables Ratings Differences ............................................ 12 1.1 Key Dates. TA, Recovery, and SAC Operations ........................ 23 3.1 Planed and Actual Disbursements .................................. 32 3 Preface This is a Performance Audit Report (PAR) for the second and third technical assistance (TA) projects in Uganda. Credit 1434-UG (the second TA project, henceforth TA II), for US$15 million equivalent, was approved in December 1983 and closed in June 1992, four years later than the original closing date, having been fully disbursed. Credit 1951-UG (the third TA project, henceforth TA III), for US$18 million equivalent, was approved in August 1988 and had its final disbursement in May 1994, some 19 months before the closing date. Both projects were complemented by parallel- financed US$8 million UNDP projects for which the Bank was executing agency. One of TA III's components, US$2.3 million for strengthening the Agricultural Secretariat in the Bank of Uganda, was executed by FAO. This PAR was prepared by the Operations Evaluation Department (OED). It is based on the Project Completion Reports (PCRs) issued in 1993 and 1994 by the Bank's Africa Regional Office', a completion report prepared by FAO on the above-mentioned FAO-executed component,' the President's and Staff Appraisal Reports, credit documents, credit and country files, Bank and government reports, and interviews with Bank staff involved in the projects and/or the Bank's overall assistance program in Uganda. An OED mission visited Uganda in December 1994 to discuss the effectiveness of TA II and TA III and related matters with present and former Ugandan officials and donor representatives. Their kind cooperation and valuable assistance is gratefully acknowledged. The Government's comments upon an earlier draft of the PAR are shown on Attachment A to the report. These comments, as well as those of the Africa Regional Office, were taken into account in the final report. 1. Project Completion Report, Uganda: Second Technical Assistance Project (Report No. 12090 of June 28, 1993), and Project Completion Report, Uganda: Third Technical Assistance Project (Report No. 13590 of October 7, 1994). 2. Strengthening of the Agicubural Secretwiat: Project Findings and Recommanddons, Terminal Report, Rome: FAO, 1994.  5 Basic Data Sheet SECOND TECHNICAL ASSISTANCE PROJECT (CREDIT 1434-UG) Credit Position (Amounts in US$ million) As of May 31, 1995 Credit Original Disburseda Cancelled Repaid Outstandinga 1434-UG 15.00 18.30 0.002 0.31 22.04 Cumulative Estimated and Actual Disbursementsm IDA Fiscal Year Appraisal Estimate Actual Actual as % of Appraisal Annual Cumulative Annual Cumulative Estimate 1984 0.2 0.2 0.1 0.1 1 1985 2.5 2.7 1.9 2.0 13 1986 5.3 8.0 1.8 3.8 25 1987 4.5 12.5 4.1 7.9 53 1988 2.5 15.0 4.6 12.5 83 1989 n.a. n.a. 2.9 15.4 103 1990 n.a. n.a. 2.6 18.0 120 1991 n.a. n.a. 0.6 18.6 124 1992 n.a. n.a. 0.2 18.8 125 a. Disbursed and outstanding amounts differ from the original amount of the credit in terms of USS because of changes in the USS/SDR exchange rate. 6 Project Dates Planned Revised Actual Identification 02/83 02/83 Preparation 02/83 02/83 Appraisal 04/83 04/83 Negotiation 11/83 11/83 Board Approval 07/83 12/83 12/22/83 Credit Signature 03/84 03/08/84 Effectiveness 03/84 04/27/84 Credit Closing 06/30/88 06/30/89 06/30/92 06/30/90 06/30/91 06/30/92 Credit Completion 12/31/87 12/31/88 06/30/92 Staff Inputs (staff weeks) Preparation Appraisal Negotiation Supervision Total 1983 1.5 5.9 7.4 1984 2.9 2.1 8.7 13.7 1985 13.2 13.2 1986 16.0 16.0 1987 16.3 16.3 1988 10.9 10.9 1989 13.1 13.1 1990 12.9 12.9 1991 6.7 6.7 Total 1.5 8.8 2.1 97.8 110.2 7 Mission Datao Stage of Project Month/Year No. of Persons Specialization Performance Types of Cycle Repreented' Rating Status' Problems' Appraisal 04/83 1 TA Supervision 07/84 1 TA 2 M Supervision 03/87 1 TA 2 M, I Supervision 12/87 2 PA, Cons. * * Supervision 05/88 1 TA 2 M, I Supervision 03/89 2 TA, PSM 2 M, I Supervision 08/89 1 PSM 2 M, I Supervision 11/89 1 PSM 2 M, I Supervision 01/90 1 PSM 2 M, I Supervision 02/91 1 PSM 2 M a. No mission between 1985-1986 due to the civil war in Uganda b. Cons= Consultant; PA= Project Advisor; PSM= Public Sector Management Officer, TA= Technical Assistance Advisor c. 1= Minor problems; 2= Moderate problems; 3= Major problems d. M= Management; I= Impact * No Form 590 Other Project Data (Borrower: Republic of Uganda) Follow-on Operations Credit No. Amount Board Date (US$M) Third Reconstruction Program 1474-UG 50.0 05/15/84 Third Technical Assistance 1951-UG 18.0 08/23/88 Public Enterprise 1962-UG 15.0 11/08/88 Structural Adjustment 2314-UG 1.4 12/23/92 Economic and Financial Management 2418-UG 29.0 08/04/92 Institutional Capacity Building 8 Basic Data Sheet TmRlD TECHNICAL ASSISTANCE PROJECT (CREDIT 1951-UG) Credit Position (Amounts in US$ million) As of May 31, 1995 Credit Original Disburseda Cancelled Repaid Outstanding 1951-UG 18.00 18.98 - - 21.8 Cumulative Estimated and Actual Disbursementsa DA Fiscal Year Appraisal Estimate Actual Actual as % of Appraisal Annual Cumulative Annual Cumulative Estimate 1988/89 0.6 0.6 0.325 0.325 54.2 1989/90 3.6 4.2 1.560 1.885 44.8 1990/91 4.8 9.0 5.270 7.155 79.5 1991/92 4.5 13.5 4.992 12.147 89.0 1992/93 2.6 16.1 4.732 16.879 104.8 1993/94 1.5 17.6 1.121 18.000 102.3 1994/95 0.4 18.0 a. Disbursed and outstanding amounts differ from the original amount of the credit in terms of US$ because of changes in the US$/SDR exchange rate. 9 Project Dates Actual Identification March 1987 Preparation November 1987 Appraisal February 6, 1988 Negotiation July 18, 1988 Board Approval August 23, 1988 Credit Signature November 14, 1988 Effectiveness May 11, 1989 Credit Closing December 31, 1995 Credit Completion May 31, 1994 Staff Inputs (staff weeks) Preparation Appraisal Negotiation Supervision Total 1987 8.0 6.8 14.8 1988 2.1 2.1 1989 6.5 6.5 1990 16.3 16.3 1991 8.9 8.9 1992 11.2 11.2 10 Mission Data Stage of Project Month/Year No. of Persons Specialization Performance Types of C)le Repres&ed Rating S&aO Problem Appraisal 08/89 1 PSM 1 Supervision 11/89 1 PSM 1 Supervision 01/90 1 PSM 1 M Supervision 06/90 1 PSM 1 M Supervision 10/90 1 1 Supervision 02/91 1 1 M/I Supervision 05/91 1 1 M/I 08/91 Supervision 07/92 1 1 Supervision 03/93 1 1 M 06/93 Supervision 09/93 2 PSM/TA 1 a. PSM= Public Sector Management Officer; TA= Technical Assistance Advisor b. 1= Minor problems; 2= Moderate problems; 3= Major problems c. M= Management; I= Impact Other Project Data (Borrower: Republic of Uganda) Follow-on Operations Credit No. Amount Board Date (US$M) Public Enterprise 1962-UG 15.0 11/08/88 Second Economic Recovery 2087-UG 125.0 02/01/90 Structural Adjustment 2314-UG 1.4 12/23/92 Economic and Financial Management 2418-UG 29.0 08/04/92 Institutional Capacity Building 11 Evaluation Summary Overall Conclusions 1. These two projects were chosen for audit by OED mainly because they were described by the PCRs as success stories-particularly TA m, for which the PCR rated the outcome as "highly satisfactory." Bearing in mind that "success stories" in TA are hard to find, especially in Africa, this PAR sought first to provide an independent assessment of the extent to which the project outcomes had been satisfactory or otherwise, and second to identify lessons of experience that may prove to be particularly relevant to other African countries emerging, like Uganda in the late 1980s, from periods of internal political strife characterized by the breakdown of government services. 2. The main conclusions of this PAR are that TA H (approved in 1983 and closed in 1992) had a marginally satisfactory outcome, whereas TA M (approved in 1988, final disbursement in May 1994) had a highly satisfactory outcome in the context of the difficult country conditions and constraints under which it was implemented. 3. The story of these two projects is partly a success story and partly a cautionary tale. The main shortcomings of TA II were that the overall political and institutional environments were not conducive to either short-term policy reform or long-term institutional development, and that some project components were not very appropriate (relevant) to removing the (then) binding constraints to the country's development. The main strengths of TA III, on the other hand, were that all of its components have been highly relevant, that the Borrower was highly committed to achieving project objectives, and that project management on both the Government and Bank sides was good. PAR vis-a-vis PCR Ratings 4. This OED performance audit confirms the main findings of the PCRs, both of which were comprehensive, informative, and insightful. This PAR also confirms the validity of virtually all the important lessons, identified in the PCRs. The PAR does however differ significantly from the PCR for TA II with regard to its ratings. The following table shows how the PAR ratings for outcome, institutional development (ID) impact, and sustainability compare with the PCR-based ratings. 12 Ratings Differences TA H TA III PCR-based PAR PCR-based PAR Outcome Satisfactory Marginally [Highly] [Highly] Satisfactory Satisfactorya Satisfactory ID Impact Substantial Modest Substantial Substantial Sustainability Likely Uncertain Uncertain Uncertain a. Owing to the less-than-full achievement of some of the project's objectives, as described in the PCR itself, the OED Director-General's Transmittal Memorandum in the front of the PCR described the outcome as "satisfactory' (rather than highly satisfactory) and entered that rating in its database, pending the findings of this PAR. The main reasons for the PAR ratings are summarized below and explained in detail in the text. Objectives, Design, and Relevance 5. TA II and III were part of a series of five free-standing TA operations mounted by the Bank since 1980. In each case, the follow-up project was begun before the previous project was closed, so TA II overlapped with both TA I and m while TA III overlapped with TA II and with its follow-on operation, the Economic and Financial Management Project (EFMP). The EFMP will be succeeded by the Institutional Capacity Building Project (ICBP) that was negotiated in November 1994. 6. All five projects have had similar objectives and instruments. The central objectives have been: (i) to help the core economic ministries and agencies design and implement a progression of economic reforms; and (ii) to strengthen systems and capacities within those agencies. TA H was linked to, and overlapped with, the implementation of the Second and Third Reconstruction Credits and the First and Second Economic Recovery Credits. All of these credits, except for the Second Economic Recovery Credit, were found by OED audits to have had unsatisfactory outcomes. TA III was linked to and overlapped with the First and Second Economic Recovery Credits and with SAC I, not yet closed. Much of TA II was disbursed in a period of insecurity and political turmoil prior to the advent to power in January 1986 of President Museveni, and prior to the change from dirigiste to more market-friendly policies marked by the Government's adoption of its Economic Recovery Program (ERP) in mid-1987. 7. The design adopted in both projects involved a mixture of "blueprint" and "process" approaches, with some components being left "underdesigned" at appraisal. In TA HI, for example, a sizable component was devoted to "special advisory services for the ERP," allowing the project the flexibility to finance studies, activities, and other inputs as the need arose. It was a strength of the design of both projects that considerable flexibility was allowed to the project managers to modify project objectives and instruments during the course of the projects. 13 8. Borrower involvement during project preparation was limited in both projects. A major reason for this was the lack of Government capacity to articulate such priorities and to design TA projects. Hence, neither project had broad-based ownership across all economic management ministries and departments. Nevertheless, ownership was built steadily albeit gradually as between TA II and III and during implementation of TA III. A principal reason was that the Permanent Secretary of the project's coordinating ministry (MPED) became a committed champion of the Bank's TA projects, including the successor EFMP and ICBP. In addition, the mechanism of assigning managers for each component was an effective way of ensuring smooth project implementation and broadening ownership beyond those primarily responsible for project management. 9. Most of the objectives and components of TA II, and virtually all of the objectives and components of TA III, are judged to have been relevant. Both projects were appropriately directed to deal with fundamental impediments to good policies such as bad statistics and accounts, inadequate debt management information and analysis, and the need to make functional again what had become a dysfunctional civil service. With respect to TA II, however, considerable funds were allocated to potentially worthwhile undertakings (such as development of the Public Service Improvement Program, the Civil Service Reform, and a Parastatal Accounting Study) that did not lead to any significant results. In retrospect, it was unrealistic to believe that the (then) current government could or would follow up on those undertakings. TA II also contained some components that were, especially by the standards of the 1990s, of dubious intrinsic value, e.g. for strengthening the Planning Department of the Ministry of Industry and for public sector factory rehabilitations. In general, TA II was an unwieldy project that sought to do too many things in an environment that was still not ready, without a framework or focus on a central theme. 10. TA III was more relevant because it was mounted in a much more enabling institutional and governance environment. Also, by the late 1980s the Bank had a more coherent and cohesive country assistance strategy, and lessons about project management learned from TA I and II were applied to TA III. Implementation and Outcomes 11. The two projects had similar strengths and weaknesses in implementation. On the positive side, both projects supported considerable progress in improving Uganda's statistics and in undertaking relevant studies and policy analysis in agriculture (through the Agricultural Task Forces and the Agricultural Secretariat). The Economic Analysis Unit supported by both projects became a highly effective unit. It was also a positive outcome of both projects that pressures that must have existed to divert project funds to low priority equipment purchases and expensive study tours and external training (beyond those truly warranted) were resisted. Indeed, the proportion of actual expenditures on equipment was somewhat below the proportions indicated in the Development Credit Agreements. 12. TA II's shortcomings in implementation included such high-relevance components as preparation of civil service reform, the parastatal study, and accountancy training (the 14 last performed at extremely high unit cost). TA III had good implementation in terms of improving statistics, debt management, preparation of civil service reform, and macroeconomic and agricultural policy formulation. Implementation performance was more mixed in the program to encourage expatriate Ugandans to return and take up public service positions and was relatively ineffective in improving budgetary reform and expenditure control. Both projects were characterized by too much ad hoc decision- making and not enough by clear criteria and priorities, as should be established by an ID strategy. On the whole, however, both the relevance and efficacy of implementation of TA III's components were high enough to warrant an outcome rating of "highly satisfactory" for that project. 13. Both Borrower and Bank performance under TA II were only marginally satisfactory. Government compliance under TA II was unsatisfactory in regard to three covenants: the Review Committee within MPED that was supposed to approve sub- projects did not really function, the maintenance of project records was inadequate, and the frequency of audits was also less than required. 14. On the Government side, a strength of TA II project management, at least in the last half of the project, was that the project management was located in the Ministry of Planning and Economic Development (MPED) where the Permanent Secretary became the project's "champion." This Permanent Secretary developed over time an increasingly clear notion of how the project could be used to advantage and he took a direct role in steering it. However, getting and keeping the project on the constructive track it took in the latter half of the project period owed a great deal to the quantity and quality of supervision provided by the Bank. A strength of both projects was the continuity of management on both sides. 15. Bank performance in the early years of TA II was not very satisfactory, mainly because the process approach was not used to the extent it might have been to ensure that the project was kept relevant and realistic in the light of the changing circumstance. TA II should have been subjected to much closer supervision, during 1984-87, of the implementation of the poorer performing components. More importantly, it should have received more managerial oversight and stock-taking of (i) the relevance of the components as they evolved under the "process approach"; and (ii) the feasibility of implementation and the prospects of sustainability under the disenabling country conditions that prevailed while more than half of TA II's funds were being disbursed. The flexibility given to the project's managers was at the same time a strength and a source of vulnerability pending the establishment of transparent policies and procedures such as manifested in the "Implementation Manual" prepared for the EFMP project. 16. That the project became more relevant and realistic towards its conclusion was mainly due to (i) the "exogenous shock" (in the positive sense) of the launching of the ERP in 1987 by a new administration; and (ii) the learning process through which emerged "a commonly shared view of what needs to happen," facilitated by the hands-on, pragmatic and problem-solving approach of the Bank's frequent supervision missions. 15 17. Both Bank and Borrower performance under TA III were, on the whole, and in view of the difficult country conditions, highly satisfactory. Compliance by the Borrower with the covenants was satisfactory as was compliance by Bank staff with Bank policies and directives. TA III appears to have been well-managed and well-implemented, with no major implementation delays or problems. In TA III, as in TA II, the Government did not strictly follow the required administrative procedures, but the flexible process approach to project execution was used to good effect without the project's falling prey to the kind of indiscipline and abuse that sometimes befalls such projects. Most importantly, both the Bank and the Government deserve credit for selecting components that were highly relevant both to implementing the economic recovery program and to fostering ID. Sustainability and Related Issues 18. Neither of the PCRs on TA II and III gave an explicit sustainability rating for the projects, although the general thrust of both PCRs was that, in view of the progress being made in the economy in general and in moving ahead with civil service reform in particular, the projects' sustainability would be "likely." This was the rating entered provisionally into the OED database for TA II. Subsequently, OED entered a rating of "uncertain" for TA III, owing to "continuing difficulties to achieve appropriate public sector salaries," pending the findings of this PAR. 19. One of the most encouraging aspects of Uganda's recent history has been the progress made in moving ahead with the difficult task of civil service reform. The difficulties include the many distortions in the incentive structure that remain to be corrected, plus those inherent in reducing the country's overall heavy aid-dependency. Some of the distortions are "home grown" (e.g., compressed salary scales and nontransparent and nonmonetized allowances), while some stem from Uganda's very high aid dependence in general, and from a dependence on donor-financed salary supplements for civil servants in particular. Although an effort was made to standardize these supplements and make them transparent under a Unified Incentive Scheme, this scheme has not been implemented effectively. Recent steps have been taken to put in place an "exit plan" for phasing out donor supplements pari passu with increases in government revenues and measures to implement the civil service reform. But there is still some way to go to achieve a full harmonization of donor practices and commitment by all donors to the same exit plan. 20. Notwithstanding these limitations, impressive results were achieved in terms of: reducing the size of the civil service (cutting it in half over the past four years); preparing proposals for new structures and staffing levels; strengthening payroll monitoring and control functions in the Ministry of Public Service; formulating a capacity building plan; and securing donor funding (including from the Bank) for that plan. 21. Most importantly for sustainability, Uganda is fortunate to have a current political leadership with vision and a commitment to continued civil service reform. The "ethos of the bureaucracy" and the productivity of the civil service are improving. Yet as the forthcoming OED PAR on the Second Economic Recovery Project indicates, the near- term prospects are problematic for raising government revenues enough to sustain the 16 productivity improvements, even in a downsized civil service, while phasing out donor- financed supplements. Hence, the sustainability of the benefits of both projects is rated as uncertain. Conclusions and Lessons Main conclusions: Summary 22. It was a strength of both projects that they had a "process" rather than a blueprint design and were allowed to be managed with flexibility. But this very flexibility could have been a weakness unless sufficient "discipline in the process" was maintained by strong project management on both sides. This management was initially weak under TA II but improved over time. It was however a fundamental shortcoming of TA II that its risk assessment was very weak, mainly because the governance constraints were underestimated. 23. Similarly, government commitment/ownership was initially weak but it strengthened over time, partly because of the learning process whereby Ugandans perceived the benefits of well-managed TA. A key factor in the success of TA III was a project champion on the Government side. Another key factor was the intensive supervision provided by the Bank, with the quality of supervision (problem-solving, practical) being more important than the quantity. 24. Owing to the inadequacy of incentives (with civil service pay below a "minimum living wage"), there was justification for donors to pay temporary salary supplements. But these should have been more coordinated and transparent. No "exit plan" has yet been formulated in the context of the Government's fiscal outlook and civil service reform program. The progress to date on other aspects of this program is, however, encouraging and needs to be sustained if the benefits of the two TA projects are to be sustained. Lessons 25. These conclusions underlie some lessons of Uganda's experience with TA II and III. The main lesson is that: * Management matters most. The main complementary lessons are that: * The more unsettled a country's "institutional environment" and the more uncertain the prospects that project benefits will be sustainable (as in Uganda in the mid- and late-1980s), the more flexibility needs to be built into the project design, i.e. the more the project needs to have a "process" vis-a-vis a "blueprint" design. The more flexibility is built into the project design, however, the stronger must be the management experience and "field skills" of the Bank's task manager(s). The more the flexibility, the 17 greater the scope for abuses (e.g., diversion of project funds to low priority purchases of vehicles, study/travel abroad, etc.) and hence the greater the need for "discipline in the process." * It is critical in such projects (especially in countries emerging from political strife and characterized by a very poorly functioning public administration) to have a credible government commitment to reform (including civil service reform) plus a committed project "champion" on the Government side. Even without thorough upstream preparation, and even in the face of an initially disenabling institutional environment, ambitious institutional development (IDTA) projects can be made to succeed if there is a shared commitment and ownership among the key players, and sufficient discipline in project management. * Sustainability, civil service reform, revenue growth and expenditure containment (including through "right-sizing" of the public sector) are inextricably linked. Such links should be explicitly analyzed (and if possible modelled) both at the time of project design and on an ongoing basis with a view to enhancing the prospects for sustainability. An agreed "exit action plan" for donor-financed salary supplements should be a precondition for donors' entering upon the payment of such supplements in the first place. Some additional, supporting lessons are that: * The instruments are not the issues. TA III showed, for example, that under the right conditions, long-term expatriate advisers can be highly effective and beneficial and do not necessarily "undermine" capacity building. Under the wrong conditions, on the other hand (as prevailed during the early years of TA II), no TA modalities (whether short-term consultants, local consultants or twinning arrangements) are likely to be effective. * Harnessing existing capacities is sometimes more important than "building new capacities." Adequate capacities often exist in-country and even within the Borrower's public sector institutions. The real challenge is to develop a work environment with appropriate rewards and incentives that will get nationals to produce their best work. This may justifiably call for the temporary use of special incentives. * In a highly aid-dependent country such as Uganda, particular care must be taken to minimize donor-created distortions in local incentive structures and budgetary processes. Such distortions can be especially severe when donors pay (as in Uganda) varying salary supplements to Ugandan civil servants and bid against each other to recruit skilled Ugandans to work on their projects and staffs. Donor policies on salary supplementation must be harmonized in the context of an agreed exit plan. 18 Continuity and cohesiveness of the country team on the Bank side and of the project management team on the Government side enhance learning and the application of lessons of experience, and so improve prospects for project success. 19 1. Introduction and Overview Why Audit These Projects? 1.1 These two projects were chosen for audit by OED mainly because the PCRs (including the Part II's prepared by the Government) found both projects to have been highly successful-especially TA III. Lessons of success of such projects (involving TA to core economic ministries/agencies to prepare and implement economic reforms and to build capacity) are much harder to find than lessons of failure. 1.2 "Success cases" are all-the-more difficult to find in countries such as Uganda where long periods of political strife have gravely weakened the public administration, thereby making the absorptive capacity for TA especially weak. A characteristic of TA is that the countries that need it the most-like Uganda-tend to be those that are least able to use it effectively. One finds in such countries a "disenabling environment" for institutional development. 1.3 Thus, the purposes of the PAR are to verify the PCRs' very positive ratings, to expand upon the PCRs' discussion of the lessons to be learned from Uganda's experience, and to disseminate these lessons, particularly to other countries emerging from periods of turmoil and trying to rebuild gravely weakened public administrations. Scope and Structure of the Report 1.4 TA II and III are evaluated with reference to (i) the relevance of their objectives (the "appropriateness of their objectives in the context of the Bank's overall country assistance strategy); (ii) the efficacy of their implementation (the extent to which the projects' objectives were achieved or not); and (iii) their cost-effectiveness (where data permit judgments about efficiency to be drawn). 1.5 These criteria underlie the report's assessments and ratings of the project outcomes. Evaluations are made also of the projects' institutional development (ID) impacts, their sustainability, and of the performance of both the government and Bank at various stages of the project cycle. 1.6 The report is structured so as to consider the above-mentioned project attributes, outcomes and ratings in turn for both projects. Thus, it has a thematic structure rather than one that would treat the projects separately and chronologically. This structure facilitates a contrasting and comparing of the projects (as for example to show how and why TA III is found to be much more "relevant" than TA II). This in turn facilitates the identification of lessons of experience. The report does however distinguish clearly and separately the ratings for each project. 20 1.7 Chapter 2 describes and assesses the objectives of the two projects with a view to evaluating their relevance. This chapter deals with such design issues as: the extent to which the Government and beneficiaries identified and shaped the projects (the "ownership issue"); whether sufficient flexibility was built into project design to facilitate adapting the objectives and instruments in response to changing governance and institutional conditions; and whether the risk assessments were realistic and the means identified to manage the risks were appropriate. 1.8 Chapter 3 reviews the implementation of the two projects to establish the extent to which the objectives were realized. In a few areas the matter of cost-effectiveness (efficiency) is also addressed, although there is insufficient data on unit costs to permit a broad assessment of efficiency. The outcomes are assessed mainly in terms of individual project components (e.g. the components to strengthen statistics or to attract skilled Ugandans to return to Uganda). Chapter 4 considers several "special issues" and assesses the projects' sustainability. The final chapter summarizes the conclusions and identifies some lessons. A Bit of History: How Could TA Succeed in the Conditions of the 1980s? 1.9 The relevance and effectiveness of TA II and III have to be judged in the context of the conditions in which they were set. The PCRs, which were on the whole comprehensive, informative, and insightful, do not provide much sense of this context. Hence the following very brief review of some relevant history. 1.10 TA II was approved in December 1983, approximately in the middle of the regime sometimes known as "Obote II." From 1971 to 1979, Uganda had been ruled by the regime of Idi Amin. Prior to that regime, the Uganda Civil Service had been hailed as one of the best in Africa in terms of its motivation and performance. During the Amin regime, and continuing into the second Obote regime (that lasted until January 1986), this civil service had become inefficient, demoralized and unresponsive.' 1.11 Factors underlying this deterioration, which proceeded into the latter half of the 1980s, included a massive brain drain, inadequate pay, lack of leadership, dysfunctional organization, lack of a code of ethics, irregular recruitment of many unqualified persons, and usurpation of the functions of the Public Service Commission. The Civil Service Reform Programme (CSRP), intended to address these problems and restore the civil service to its former level of excellence, was initiated only in 1989. But as will be seen, the Bank's TA projects did have some role in developing the CSRP and then in promoting its implementation. 1.12 The political turning point of the 1980s was the advent to power in January 1986 of the National Resistance Movement government, headed by Yoveri Museveni. At that 3. The adjectives are those used by H.E. Sam Sebagerka, (then) Minister of Public Service, in his Foreword to an official document entitled Conte, Vision, Objectives, Strategy and Plan, a description of Uganda's CSRP, Ministry of Public Service, 1994. 21 time, the economy was in a state of virtual collapse with the transport network in an acute state of disrepair, agricultural lands in ruins, and most manufacturing plants closed or operating at a fraction of capacity. As mentioned above, discipline and accountability in the public service had all but collapsed. 1.13 Thus, the new government took over a country traumatized by civil war and state terror and reduced to penury through lawlessness, corruption and mismanagement.! The Museveni Government's first response (1986) was, however, to intervene heavily in the economy, and because of this dirigiste stance there was in fact no lending by the Bank to Uganda in that first year. 1.14 But then the Government changed its approach in May 1987, when an Economic Recovery Program geared to achieve stabilization and liberalization of the economy was launched. This program proved to be one that the IMF, the World Bank, and other donors could (and did) support with massive assistance that has grown to about 10 percent of GDP in recent years. This assistance to promote economic recovery, and real GDP has grown by an average annual rate of nearly 6 percent between FY88 and FY93. But the growing aid dependency has also had its "downsides" and raises questions about sustainability. 1.15 One of the conclusions emerging from some recent literature on TA is that TA projects cannot be expected to succeed unless some minimal governance preconditions are met, and unless the local "institutional environment" in which the projects are set is more enabling than disenabling.1 As noted above, however, Uganda's governance and institutional conditions were definitely unfavorable and seemingly unconducive to successful TA until well into the late 1980s. On the demand side, the unsettled conditions (including security problems) were not conducive to effective implementation, while on the supply side they were inhibiting both to the recruitment of expatriate advisers and consultants and to the establishment of effective counterpart relationships. Evolution of Five TA Projects 1.16 TA II and III were two in a series of five free-standing TA operations mounted by the Bank between 1980 and now. In each case, the follow-on project was begun before the previous project was closed, so TA II overlapped with both TA I and III while TA III overlapped with TA II and with its follow-on operation. This was called the Economic and Financial Management Project (EFMP), instead of TA IV. The EFMP, which is currently the only TA project being disbursed, will be succeeded by another project (with which it will also overlap) called the Institutional Capacity Building Project (ICBP). ICBP was negotiated in November 1994 and approved in June 1995. 4. See chapter on Uganda in Trends in Developing Countries, World Bank, 1994 and H. Hansen and M. Twaddle, Uganda Now, Between Decay and Development, 1989. 5. See especially M. Dia, A Governance Approach to Civil Service Refonn in Sub-Saharan Africa, World Bank Technical Paper No. 225, 1993, and D. Steedman, The Importance of Governance in Assessing the Potential Benefits and Risks of Technical Assistance for Institutional Development, Discussion Paper for PSM Network, The World Bank, January 1995. 22 1.17 Table 1.1 provides a chronology of approval and closing dates of all these TA operations, as well as of the reconstruction, recovery and SAC operations to which the TA projects were linked. The linkages were direct, as both the Government and the Bank view these TA operations as instruments needed for the implementation of ongoing, and the preparation of follow-up, recovery and of adjustment operations. 1.18 All five TA projects had similar objectives and instruments. The central objectives were: (i) to help the core economic ministries and agencies design and implement a progression of economic reforms; and (ii) to strengthen systems and capacities within those agencies.' These may be called, in short, the near-term reform implementation objectives and the longer-term ID objectives. 1.19 In view of the governance problems outlined in the preceding section, there was little progress made in achieving the overall reform objectives of the Bank's country assistance programs until 1987, and OED audits of the three reconstruction projects and of ERC I found the outcomes of all of these projects to be unsatisfactory. The Bank was clearly overestimating both the ability and willingness of the Government during the period of 1980-86 to implement reforms. The rationale and performance of TA II, approved in 1984 and in large part disbursed before reforms were really launched in mid- 1987, must be interpreted in this context. 1.20 In TA II and III, the reform and ID objectives were sometimes competing and sometimes complementary. They were competing, for example, when the short-run urgency to "get things done" sometimes resulted in expatriate advisers substituting for, rather than working with, local counterparts. In such cases, the modality of "substitution TA" tended to crowd out training and ID. And the "things that needed to get done" that sometimes caused this crowding out were frequently related to the meeting of conditionalities under the successive lending operations which moved from the very low conditionality reconstruction credits to the economic recovery credit and subsequent SAC operations having more demanding conditionalities. To a considerable extent, however, the reform and ID objectives were complementary, as when TA experts and consultants engaged in various forms of on-the-job training, and when those experts/consultants funded by the TA were Ugandans. 6. TA I and II also had investment project preparation objectives. 23 Table 1.1: Key Dates. TA, Recovery, and SAC Operations Operations Date First Reconstruction Project approved May 1980 TA I approved December 1980 Second Reconstruction Project approved May 1982 First Reconstruction Project closed December 1982 TA II approved December 1983 Third Reconstruction Project approved May 1984 Second Reconstruction Project closed June 1985 TA I closed December 1985 NRM government assumes power January 1986 ERC I approved September 1987 Third Reconstruction Project closed September 1987 TA III approved August 1988 ERC II approved February 1990 ERC I closed June 1991 SAC I approved December 1991 TA 11 closed June 1992 EFMP approved August 1992 ERC II closed June 1993 SAC II approved May 1994 TA III final disbursement May 1994 Capacity Building negotiated December 1994 TA III closed December 1995 1.21 One of the most positive aspects of the evolution of TA in Uganda is that the "quality at entry" of each succeeding project has improved. Each succeeding project has been characterized by greater participation and ownership by the beneficiaries, by better- defined objectives and performance indicators, and by improved processes aimed at putting more discipline and transparency into project management. These improvements have been achieved without giving up the flexibility needed to allow the projects to adapt to changing circumstances and evolving needs. The flexible, "process approach" was clearly 24 more appropriate to Uganda than a more rigid, "blueprint approach" would have been.' The story of how TA III improved upon TA II in virtually every respect is a main theme of this PAR. 1.22 Attribution problems among projects. One consequence of the overlapping of these projects (and of their having essentially the same objectives and same beneficiaries) is that is sometimes difficult to sort out just which project had which impact, especially in terms of ID. This "attribution problem" is compounded by varying leads and lags in the incidence of costs and benefits, and by the fact that the memories of those interviewed tend to sharpen in regard to more recent events.' The project files and supervision reports reveal quite a bit about what was happening, year-by-year, within each project in terms of logistics and at a technical and procedural level. But whereas TA inputs are relatively easy to identify both in time and cost, its outputs and outcomes are relatively hard to identify, much less quantify. For example, while it is relatively easy to ascertain the costs of this or that study or training program, it is hard to identify when and where the benefits ensued, and how to measure them. 1.23 Thus, it is relatively easy to ascertain that TA II and III and EFMP, taken together and cumulatively, had a substantial beneficial impact on the Statistics Department. But it is relatively hard to ascertain how much of the progress seen under TA IH was attributable to the foundations laid under TA II, and therefore how to apportion between the two projects the benefits of both. By the same token, it is still too early to know which of TA III's components will have which lagged effects. Many of these attribution problems will go unresolved in this report, especially since it constitutes a performance audit, not an impact assessment. 7. EFMP has not yet been evaluated and ICBP has not yet been approved, but this judgment (about improving quality at entry) is based upon reviews of documentation on all five projects plus interviews with Bank and Ugandan officials who are in a position to make comparisons across projects. 8. This report provides a much fuller accounting of TA III than of TA II. This is partly for the reasons mentioned above, partly because some of the "principals" involved in TA II have left the Bank, and mainly because the information base on TA Ill, both in Washington and in Kampala, is far more complete on TA II. 25 2. Objectives, Design, and Relevance Main Findings and Conclusions 2.1 TA II and III had similar objectives, components, instruments and other design features. Both were characterized by considerable flexibility allowed to project managers under a "process approach," and both had similar management/oversight structures. Neither project was set in the context of an ID strategy, and both had a design that enabled managers to cope with uncertain and rapidly changing conditions. But TA II took place under far less favorable governance/ID conditions, and whereas all of TA III's components are judged to be have been of high relevance, several of TA II's components are seen to be of low relevance. 2.2 TA II would probably have been more relevant had it concentrated more, in the period prior to 1986-87, on training activities, and had the project managers been more conservative about allocating funds to design or implement reforms that the (then) government was unwilling or incapable of undertaking. An underlying problem with the quality at entry of TA II was the overoptimism of the designers regarding Uganda's near- term prospects for improved security and governance, and the corresponding under- estimation of project risks. Objectives of TA II and III 2.3 TA II was appraised in April 1983 and approved in December 1983 at a time when conditions in Uganda were unsettled and insecure. Progress under the reconstruction loans and TA I had been slow, but the Bank assistance strategy that provided the rationale for TA II was based on an optimistic outlook regarding the government's commitment and ability to proceed with both reforms and reconstruction under the Revised Recovery Program for 1983-85. 2.4 The broad objectives of TA II, as set forth in the President's Report,' were "to strengthen the Government's economic decision making, planning, project preparation and implementation capabilities... [and] to transfer skills to Ugandans through training." The intended beneficiaries included mainly the central Ministries of: Finance (MoF), Planning and Economic Development (MPED); Public Service and Cabinet Affairs (PSCA); and several sectoral ministries including Industry, Power, Posts and Telecommunications. The Institute of Public Administration was also to be rehabilitated. 2.5 More specifically, TA II was aimed at improving the accounting functions in the Ministry of Finance, including the monitoring of the accounts of the parastatal enterprises; rehabilitating the Statistics Department within MPED (with a complementary UNDP 9. President's Report, Second Technical Assistance Project, Report P-3691-UG, December 5, 1983. No Staff Appraisal Report (SAR) was prepared. 26 project, executed by the Bank, being focussed on strengthening MPED's planning capabilities); undertaking studies and efficiency audits within PSCA; and assisting project preparation in the sectoral ministries. 2.6 Of the total project cost of US$17.1 million, IDA was to finance US$15 million of which US$8.8 million (about 60 percent) was designated for TA (advisers, consultants), US$3.2 million (about 20 percent) for training, and US$3 million (20 percent) for vehicles and support equipment. The concentration of TA II's disbursements were projected to occur in FY86-87 (about US$5 million in each of those years) with US$2.5 million being disbursed in FY85 and again in FY88. 2.7 TA III had very similar general goals and beneficiaries. Its more specific objectives were to: * strengthen the MoF's budgetary management, expenditure control, and government accounts; * improve the Bank of Uganda's external debt and financial management; * rationalize government procurement functions, including through establishment of a new Supplies Directorate and strengthening of the Central Tender Board; * help MPSCA improve its systems and management, and to prepare and start implementing a comprehensive civil service reform program; * strengthen MPED, including especially the capacities of the Economic Analysis Unit (EAU) and the Statistics Department (to prepare basic national accounts, household expenditure surveys, price statistics, and data on economic production); * strengthen the capacity of the Agricultural Secretariat (located in the Bank of Uganda) for policy planning, analysis, and monitoring in the agricultural sector; and * encourage expatriate Ugandans and local Ugandans in the private sector to take up Government positions directly relevant to the recovery program. 2.8 Of IDA's total US$18 million contribution to TA III, the SAR projected that about US$10 million (over 50 percent) would be used for consultant services, US$1.4 million (eight percent) for equipment and vehicles, US$1.8 million (10 percent) for training, about US$1 million (six percent) for staff costs and PPF reimbursement, and a large US$3.9 million (22 percent) for "unallocated." The disbursement schedule as projected in the SAR envisaged that half of the total credit would be disbursed by the end of FY91 and that the credit would be nearly entirely disbursed by the end of FY94. 27 Design Features: Process, Ownership, Project Management Structure 2.9 The design adopted in both projects involved a mixture of "blueprint" and "process" approaches, with some components being left "underdesigned" at appraisal. In TA III, for example, a sizable component was devoted to "special advisory services for the ERP," allowing the project the flexibility to finance studies, activities, and other inputs as the need arose. It was a strength of the design of both projects that considerable flexibility was allowed to the project managers to modify project objectives and instruments during the course of the projects. 2.10 This was particularly appropriate given the projects' ID objectives and the fact that they were both prepared at times when changing political and economic situations made it impossible to define the precise activities of the projects. The flexible/process approach allowed for the subsequent inclusion of certain subcomponents (in particular special studies and advisory services in support of the ERP) that could not have been foreseen at the time of project preparation. 2.11 A common problem with this approach is however that such flexibility can lead to a lack of discipline in the allocation process, especially when (as in the case of these projects) there is no ID strategy or well-defined criteria for prioritizing requests for allocations and when the country is in a crisis-coping mode. It is all too easy in such situations for flexibility to become "looseness" and "permissiveness."'o 2.12 Attempts were made in both TA II and III to address this potential problem by establishing review and vetting procedures intended to ensure that subprojects would be consistent with national priorities. Under TA III, all such special requests from government agencies had to be approved by a Project Review Committee (comprising the Permanent Secretary MPED, the Secretary to the Treasury, the Permanent Secretary MPSCA, the Project Coordinator, and a representative from the requesting agency) before submission to IDA for approval. In the event, the Project Review Committee only met during the early years and it was left up to the Borrower's project management unit (with MPED in both projects), together with the Bank's Task Manager, to decide which new activities and subcomponents were to be included. 2.13 As these individuals became a fairly close-knit group during the course of succeeding TA projects, however, the group's continuity facilitated project implementation immeasurably. Not surprisingly, perhaps, the lack of clearly-specified allocation criteria combined with the concentration of authority over project funds and project decision-making in the hands of a few individuals did give rise to allegations made in the press about the propriety of some allocations. The OED mission did not investigate any particular incidents that gave rise to these allegations, but neither its perusal of project 10. An example of a TA project where there was not sufficient discipline in project management and supervision was the Structural Adjustment Institutional Support Project, approved in 1987. In that project, rent-seeking became rife, as pressures were put upon project managers to (re)allocate funds for vehicles and trips abroad without much heed to the real need and developmental impact of such expenditures. See the OED Performance Audit Report, Report no. 13262, dated June 30, 1994. 28 documents nor its interviews held in Washington and Kampala yielded any evidence-even of a hearsay sort-of such impropriety. 2.14 Borrower involvement during project preparation was limited in both projects. As one senior official put it, the Government was basically "receiving" TA without defining its needs according to its own well-defined priorities. A major reason for this was the lack of capacity in the Government to articulate such priorities and to design TA projects. Hence, neither project had broad-based ownership across all economic management ministries and departments. Consequently, there was a lack of consensus on some principal project issues, and even MPED and MoF differed significantly (before their merger) in their respective approaches as to how TA should be used. 2.15 Nevertheless, ownership was built steadily albeit gradually as between TA II and III and during implementation of TA III. A principal reason for this positive development was that the Permanent Secretary of the project's coordinating ministry (MPED) became a very committed champion of the Bank's TA projects, including the successor EFMP and ICBP. In addition, the mechanism of assigning managers for each component was an effective way of ensuring smooth project implementation and broadening ownership beyond those primarily responsible for project management. Significantly, the components that were implemented the least successfully were those that did not have assigned component managers. Relevance 2.16 Most of the objectives and components of TA II, and virtually all of the objectives and components of TA III, are judged to have been relevant and appropriate insofar as they were addressed to help relieve the binding constraints to Uganda's economic development. The main constraints were inappropriate economic policies and devastated physical and institutional infrastructures. Both projects were appropriately directed to deal with fundamental impediments to good policies such as bad statistics and accounts, inadequate debt management information and analysis, and the need to make functional again what had become a dysfunctional civil service. Both projects were appropriately geared to cope with short-run crises and at the same time build capacities needed for sustainable ID in selected key public sector agencies. The flexible or "process" mode of execution was also entirely appropriate for the reasons mentioned above. 2.17 With respect to TA II, its most relevant components-and also the ones that were implemented most effectively-were the support provided to the Statistics Department and for the agricultural sector, especially in support of the Agricultural Task Forces. However, a number of factors made some of its components not very relevant. In some cases, the objectives were important but the country conditions, including the lack of commitment to reform, were unfavorable. And so considerable monies were spent on such potential worthwhile undertakings as development of the Public Service Improvement Program (US$4.2 million actual), Civil Service Reform (US$1.3 million), and a Parastatal Accounting Study (US$850,000) that in retrospect should not have been spent (at least not at such levels) because it was unrealistic to believe that the Government could or would follow up on those undertakings. 29 2.18 TA II also contained some components that are, especially by the standards of the 1990s, of dubious intrinsic value, whatever the likelihood or actuality of implementation. For example, TA II included an allocation of US$1 million for strengthening the Planning Department of the Ministry of Industry and another US$625,000 for the Industrial Enterprise Secretariat plus several hundred thousand dollars for public sector factory rehabilitations that were expenditures of dubious, if not negative utility. In general, TA II was an unwieldy project that sought to do too many things in a disenabling environment, without a clear framework or focus on a central theme. 2.19 TA III might also have proven an unwieldy project of mixed relevance, given its many components and its many different beneficiary agencies. But TA III avoided most of TA II's shortcomings because (i) it was mounted in a much more enabling institutional and governance environment; (ii) by the late 1980s, the Bank itself had a more coherent and cohesive country assistance strategy; and (iii) lessons learned from TA I and II were applied to TA III. 2.20 One of the most important features of TA III that serves to give it a relatively high "relevance rating" is the high priority given in the project to helping MPS to design and begin to implement the civil service (CSRP). For neither can the ERP reforms nor the ID strengthening benefits of TA III (and of all other projects and programs aimed at advancing both the reforms and ID, in whatever institutions) be sustained without an effective CSRP. 2.21 As discussed in Chapter 4, one of the problems relating to the sustainability of the benefits of TA II and III is the salary supplements that began to be paid (to the EAU) in TA III and are now paid in large volumes by the Bank and all other donors. More so than in other countries, the Bank and other donors faced the "lack-of-a-living wage issue" head on, recognizing that supplements in one form or another would simply have to be paid under Uganda's collapsed conditions of the late 1980s in order to enable government workers to come to work. 2.22 These interventions would have been more appropriate had they been made in the context of a medium-term macroeconomic framework incorporating a realistic and feasible CSRP that provided an "exit plan" for reducing the supplements in a predictable way. They also would have been more relevant/appropriate had there been more discipline and coordination amongst donors involving, at a minimum, a harmonization of salary supplement scales. This story will be picked up again in Chapter 4.  31 3. Implementation and Outcome Ratings Main Findings 3.1 The outcome of TA II was only marginally satisfactory, mainly because of "exogenous factors" in the form of governance conditions that did not permit significant progress to be made either in implementing economic reform or in ID until well over half of the project funds had already been disbursed. Some of TA II's components (to be identified below) were implemented in a reasonably efficacious and cost-effective way; this was no mean accomplishment in view of the impediments. But the implementation of other components, including some of the most important ones (also to be identified below), was neither efficacious nor efficient. There was also a lack of compliance on the government side with some project covenants. Putting the implementation assessment of this chapter together with the "relevance assessment" of the previous chapter yields an overall outcome rating that is, as mentioned above, no better than "marginally satisfactory." Both Borrower and Bank performance on TA If were also only marginally satisfactory at best. 3.2 The outcome of TA III was better than satisfactory, especially considering that TA III's implementation also took place under difficult conditions, albeit not nearly so difficult as those affecting TA II. Although a few components were not implemented satisfactorily, the implementation of most was quite satisfactory. And since virtually all components had high relevance ratings, the project's contribution to the country's economic reform and recovery was substantial. And more progress appears to have been made in terms of ID than is typical of such projects. Both Bank and Borrower performance were on the whole highly satisfactory. Outcomes by Main Component/Beneficiary 3.3 The findings of this PAR are largely in agreement with those of the PCRs with regard to the efficacy of implementation of the main components of the two projects. The PCRs did not comment on the relevance dimensions, as this PAR has done. It is therefore the lower "relevance ratings" given to TA II by the PAR explicitly than underlie the lower outcome rating given by the PAR to TA II. 3.4 The two projects had similar areas of strength and weakness. In both projects, little progress was made to strengthen the functions of the Ministry of Finance (accounting system improvement, budget reform), and these same objectives had to be incorporated in the successor EFMP. Training was somewhat crowded out in both projects and such training as did take place was not cost-effective. Both projects were characterized by too much ad hoc decision-making and not enough by clear criteria driven by well-articulated priorities. 32 3.5 On the positive side, both projects supported considerable progress in improving Uganda's statistics and in undertaking relevant studies and policy analysis in agriculture, both through the Agricultural Task Forces and the Agricultural Secretariat. The EAU supported by both projects became a highly effective unit. It was also a positive outcome of both projects too that pressures that must have existed to divert substantial proportions of project funds to low priority equipment purchases and expensive study tours and external training (beyond those truly warranted) were resisted. Indeed, the proportion of actual expenditures on equipment was in both projects even somewhat below the proportions indicated in the Development Credit Agreements. 3.6 The following table shows, for the major expenditure categories, the actual as compared with the planned distribution of funds. Table 3.1: Planned and Actual Disbursements (STAR equivalent) TA I TA III Plan Actual Plan Actual Consultants Servs. 7.1 13.1 7.4 11.6 Equipments/Vehicles 2.8 0.4 0.9 0.7 Training 3.0 1.1 1.4 0.6 Unallocated 1.3 3.0 O&M 0.2 0.2 Resettlement of 0.1 0.4 Expatriate Ugandans Staff Allowances 0.3 0.2 Refunding PPF 0.5 0.1 Total 14.2 14.6 13.8 13.8 3.7 TA II had the shortcomings in relevance remarked in the last chapter and some serious shortcomings in implementation as well. MPS officials acknowledge that the more- than-US$5 million expended on the public service improvement programs and civil service reform did not "take hold" during TA II, and it is not clear to what extent the groundwork was laid for the substantial TA provided under TA III and subsequently under EFMP to advance preparation and implementation of the CSRP. Some 137 Ugandans received accountancy training of uncertain quality and impact under the project, but at a very high 33 unit cost of US$9,300 per trainee for training provided within country-surely a very cost- ineffective use of resources." 3.8 A particularly unsuccessful component of TA II was the parastatal accounting study which had the shortcomings of being a low-quality study undertaken in an environment where the Government was not ready to participate in a dialogue on parastatals with the Bank. 3.9 Lack of adequate quality control applied to studies financed by TA projects was a more serious shortcoming in TA II, but also to some extent characterized TA III. There are several reasons for this. One is that, in a project involving many components and many studies, neither the project management unit of the government nor the supervision missions mounted by the Bank have either the time or the technical expertise to vet adequately either the relevance of the study terms-of-reference or the quality of the draft reports themselves. 3.10 On the Government side, the main problem is the scarcity of staff with sufficient skills and experience to provide the requisite quality control. TA-financed consultants are often engaged to vet each other's work, but sometimes at insufficient arm's length. Within the Bank, on the other hand, economists and sectoral specialists face an incentive structure which inclines them to spend much more time and effort fine-tuning their own studies than "rough tuning" studies of consultants financed under TA projects. Some dedicated Bank staff do provide such service, but this often tends to be service considered "above and beyond the call of duty." 3.11 The following section reviews the outcomes of the main components of TA III as grouped by main project objective. Objective: To make the Ministry of Finance's budgetary and financial management more effective by improving expenditure control and procurement rationalization, resource allocation, budgetary planning and monitoring, and by restoring the system of government accounts (high relevance, low efficacy). 3.12 Outcome. By reforming the budget process and improving the financial information system, TA III was designed to give the Ministry of Finance more effective management of government's finances and resources. Budget reform made very little progress under the project. A long-term consultant was recruited for the task but made little headway in designing and institutionalizing reforms. The consultant played a largely operational role, participating directly in budget preparations. Meanwhile, the expenditure control function was gradually moved out of the budget department to the Economic Analysis Unit. It is unclear how much "savings" was generated through the introduction of new procurement systems and procedures. Despite the setting-up of a new and 11. The PCR calculated, mistakenly, the unit cost at about US$1,000 per individual trained, and remarked that even this was a rather high unit cost. The OED mission was, however, unable to ascertain the causes of these high costs, or why they were allowed to go so high. 34 well-staffed Uganda Supplies Directorate, difficulties in implementing reforms at the level of the Central Tender Board made the component less successful than it might have been. 3.13 The development and implementation of new government accounting subsystems to improve financial information were also unsatisfactory. Progress was limited to the purchase of equipment and the implementation of a civil service reform-related exercise on payroll reform. Minimal institutional development took place, and it was decided to make budget reform and financial information management priority areas of focus under the successor project. Objective: To improve the Bank of Uganda's external debt management and the management of its own financial operations (high relevance, high efficacy) 3.14 Outcome. TA III provided essential and effective assistance in these areas. The BOU's accounting system had collapsed, with no accounts having been produced since 1986 and many records missing. External consultants established a new accounting system, computerized the recording of financial transactions, and provided training. With regard to the external debt, the BOU was also unable to keep track of its outstanding debts and schedule of interest payments. With the assistance of (high-priced) external consultants from a merchant bank, a detailed inventory of Uganda's debt was established. This provided a credible basis for Uganda to negotiate with its creditors, and among the benefits was the enabling of Uganda to mount Africa's first debt buy-back arrangement. 3.15 In both areas the technical assistance provided achieved more than just short-term outputs; it was effective in bringing about institutional development. Through training and on-the-job supervision, the external consultants developed capacities that have evidently extended to lower levels of management and are judged to be sustainable. In addition, top management in the BOU believe that having the BOU officials working side-by-side with the external consultants (as well as with missions from the World Bank and the IMF) was instrumental in improving BOU's work ethic. 3.16 One modality that is being used successfully in the BOU (and is becoming more widely used elsewhere) is what may be called the "coming and going expert" modality, to be distinguished from the long-term adviser or the short-term consultant whose inputs are often ad hoc and one-time. After an initial stay by an expatriate expert of varying length, in which the expert gets to understand better the nature of the problems to be solved and the local environment, and when initial steps are taken to put in place new systems and procedures and to train local staff, work plans are agreed and then the consultants "come and go" periodically (say for two weeks every three months, or three weeks every six months) to monitor progress made since their last visit, to help solve problems in situ, to provide additional on-the-job and formal training, and to update the work program for the coming period. This modality avoids the risk of having experts be drawn too much into operational (substitute TA) assignments, and it puts pressure on national counterparts to become fully responsible and accountable for performing the unit's functions. At the same time, it keeps in place a support system and tends to make the training provided during the experts' periodic visits highly focussed and cost-effective. 35 Objective: To strengthen the capacity of the core economic ministries, especially to analyze and monitor macroeconomic and agricultural policy issues and to development and implement a CSRP (high relevance, medium efficacy) 3.17 The Economic Analysis Unit, created under TA II and supported by TA III, was highly competent and productive in carrying out its analysis and advisory functions related to the recovery programs. It was also completely staffed by Ugandans, a phenomenon made possible because of the payment from project funds of substantial special salary supplements.12 EAU became an island of excellence and high productivity in a sea of much lower productivity. In fact, because its staff worked so well, top management in the ministry became increasingly dependent on the unit, transferring to it functions of other departments that were not being executed satisfactorily. 3.18 Although the unit performed well in the short term, there were problems with its sustainability, and probably some unmeasurable adverse effects related to jealousies and resentments created amongst civil servants not receiving salary supplements enjoyed by the EAU staff. The project design only envisaged financing the EAU for three years, after which it was to be absorbed into the existing ministerial structure. When the Ministry of Planning and Economic Development merged with the Ministry of Finance in 1992, the EAU was merged into the macroeconomic analysis department of the combined new Ministry. That led to a reduction in supplements and a change in status of the former EAU staff. There are widely varying views amongst government officials as to the costs and benefits of creating or disbanding such islands of excellence as the EAU proved to be. The OED mission was not able to sort out the complex costs and benefits. 3.19 The MPED's Project Monitoring and Evaluation Unit was intended to help monitor the recovery program's most important projects, and TA III was to develop this capacity in the line ministries. Beyond establishing a database of projects, this subproject was ineffective and had little impact, owing mainly to leadership problems. 3.20 In support of the recovery program's broader goal of reforming the entire public sector, TA III's assistance to the MPSCA led to significant gains in reviewing and reducing the number of ministries (by more than 50 percent), in removing 35,000 "ghosts" from the payroll, in authenticating payrolls, etc. The authentication of the teachers' payroll alone led to a savings of some US$10 million. Long-term TA experts were also able to establish capacity within MPSCA for managing computerized personnel statistics. And most importantly, the basis for more comprehensive civil service reforms was established. (Civil service reform has received considerable support under EFMP). Objective: To strengthen the capacity of the Statistics Department to prepare basic national accounts, price statistics, and general data on economic production (high relevance, high efficacy) 12. The issue of salary supplements is discussed in Chapter 4. 36 3.21 Outcome. Given the importance of reliable statistics for monitoring the economy and formulating appropriate economic policies, this was a highly relevant objective. Building on the statistics component of TA II (where the main progress was in getting the household budget survey underway), the rehabilitation of the Statistics Department under TA III is judged by many senior officials in the Ugandan Government (as well as by Bank staff and some donors) as one of the project's major successes. Through a combination of long-term TA experts, training, and physical rehabilitation, dramatic improvements were made in the national statistics capability and outputs. Two long-term TA experts, at least one of whom was en poste throughout the project, played an important role in this component, both operationally and in terms of ID. As of late 1994, these experts were on the verge of leaving Uganda, after which the department is to be managed by nationals exclusively. Objective: To strengthen the capacity of the Agricultural Secretariat for policy planning, analysis and monitoring in the agricultural sector (high relevance, medium/high efficacy) 3.22 A total of US$2.4 million was spent under this component, executed by the FAO. The PCR did not assess this component. Although the executing agency produced a final report,1 this report is descriptive rather than evaluative and does not provide a sufficient basis on which to judge the relevance, efficacy, or cost effectiveness of the component. 3.23 The OED mission had interviews with staff of the Secretariat in Kampala and with other government and World Bank officials familiar with the component, but it was able to perform only a superficial evaluation on this basis. Clearly the component's objectives were highly relevant in view of the importance of agriculture in Uganda and the distortions in agricultural policy. But since both the Agricultural Secretariat and other related agencies benefitted from many other parallel and complementary projects during the life of TA III, it is all-the-more difficult to isolate results specifically attributable to the TA III component. 3.24 One clearly attributable achievement, however, was the progress made in liberalizing the coffee sector, as the related institutional and policy work was done by the Agricultural Secretariat through TA III. Of particular note was the effectiveness of the project-financed study tour by high-level Ugandans to coffee producing countries. This was judged by the (then) head of the Secretariat to have made a critical difference in reorienting the thinking of the key decision-makers away from the government's dirigiste approach towards managing the coffee sector towards a market-determined one. Although the Secretariat is reportedly now performing its functions effectively, as the Secretariat continues to benefit from the operational input of a long-term TA expert (who has now been there for more than ten years), it is not clear whether the balance between operational support and capacity building in this component is appropriate or not. 13. "Strengthening of the Agricultural Secretariat: Project Findings and Recommendations," Terminal Report, FAO, Rome, 1994. This report provides a useful description of the outputs and accomplishments of the Secretariat. But it attributes to the project virtually all outputs and benefits of the Secretariat, and virtually all the outputs and processes are implicitly, if not explicitly, seen to be significant accomplishments. But since no judgments are offered regarding the quality or cost-effectiveness of the studies, training, and systems produced, it is difficult to see how to use the report for evaluative purposes. 37 Objective: To encourage expatriate and local Ugandans in the private sector to take up Government positions directly relevant to the recovery program. (high relevance, medium efficacy) 3.25 Outcome. Given the brain drain that Uganda experienced over many years, the great scarcity in government of technical and managerial skills, and the high cost of filling the skills gaps with expatriates (including through "substitution TA"), this was certainly a highly relevant objective. Under this component, some 40 Ugandans were assisted in returning to Uganda to take up employment in the public and parapublic sectors, at a cost of about US$1.9 million. The PCR indicates that the contracts of these returnees came to some 100+ person years, making for average cost of about US$50,000 per returnee and about US$18,000 per returnee per year. 3.26 The PCR claims that this scheme was of the TA III's most successful components. Some highly-placed officials support this conclusion and deny that there have been either any notable number of "misfits" as between returnees and the positions they filled, or any notable amount of resentment amongst colleagues in the institutions they joined. Other officials assert that some of the returnees, who were often placed in positions of considerable responsibility, were perceived to be "overpaid underperformers" within their institutions, with the consequent resentments amongst their colleagues being reflected in some lowering of the productivity of the non-returnees and/or some obstructionism of the returnees. 3.27 One of the problems with this scheme was that it took an approach to attracting expatriate Ugandans to return in which the returnee was first selected on the basis of his or her experience and qualifications and then fitted into a seemingly appropriate position. A more transparent and cost-effective approach would have been a more targeted one in which the Government would first identify specific positions that needed to be filled but could not filled locally, and then use the project funds to attract those specific expatriate Ugandans qualified to fill those positions. In any event, the scheme was not seen to be transparent in its criteria for selecting and placing the returnees, and the resultant perceived lack of equitability was certainly a shortcoming. 3.28 The Project Review Committee did not play the role envisaged for it, and the procedures through which salaries, benefits and resettlement allowance were negotiated were not uniform. In retrospect, it seems clear that there should have been more oversight of this component to ensure that decisions were made by objective criteria and not owing to patronage relationships. 3.29 In a similar vein, the recruitment of nonpublic sector Ugandans to government (including through consultant contracts) was facilitated by donor-financed salary supplements (though only to the EAU under TA III) and by TA funds made available for local consultants to work on all kinds of studies and projects. As will be discussed below, the payment of these supplements certainly enabled local capacities (including of regular government employees) to be harnessed, and the benefits were obvious. On the other hand, some of the disbenefits were not so obvious. Neither the original project documents nor the supervision reports (nor even the PCR) considered the potential or actual 38 "downsides" or "negative externalities" of the supplementation schemes, of which distortions in the labor market created by the supplements and donor-paid consultant fees were the most pronounced. 3.30 Given that the EAU was a highly productive unit, it is arguable that the supplements that enabled the government to attract Ugandans away from the university and other non-civil service activities facilitated the movement of such individuals from lower-productivity to higher-productivity pursuits. But neither the SAR nor the PCR gave any attention to the matter of the opportunity costs that to some extent offset the benefits of the supplements (e.g., the loss to the quality of teaching and learning at the university occasioned by the bidding away of university professors to work for donors, or to work for the government in donor-supplemented positions chosen by the donors). Borrower and Bank Performance 3.31 Both Borrower and Bank performance under TA II were only marginally satisfactory, although this performance improved considerably in the period following the launching of the ERP in mid-1987. By that time, however, some two-thirds of the total project cost had been disbursed, with a substantial proportion of these disbursements being directed (as described above) to uses of limited efficacy and cost-effectiveness. 3.32 Government compliance under TA II was unsatisfactory in regard to three covenants. The Review Committee within MPED that was supposed to approve sub- projects did not really function. The Permanent Secretary of MPED, the chairman of the Committee, in effect became a one-man committee; this created a lack of transparency that was one of the project's weaknesses. 3.33 The maintenance of project records was inadequate and the frequency of audits was also less than required. The lack of records was a handicap to the work of the PAR mission that could not track in detail, as it could in the case of TA III, actual disbursements by end use. No evidence was found of any non-compliance by Bank staff with Bank policies or directives. 3.34 On the government side, it became a strength of TA II project management, at least in the last half of the project, that the project management was located in the Ministry of Planning and Economic Development (MPED) where the Permanent Secretary became the project's "champion." This Permanent Secretary developed over time on increasingly clear notion of how the project could be used to advantage, and he took a direct role in steering it. 3.35 However, getting and keeping the project on the constructive track it took in the latter half of the project period owed a great deal to the quantity and quality of supervision provided by the Bank. As the PCR acknowledges, since the government's functions of prioritizing and vetting sub-projects was not being performed by the government oversight committee as required, a great deal of the hands-on management of the project fell to the Bank's supervision missions. The flexibility given to the project's managers was at the same time a strength and a source of vulnerability pending the 39 establishment of transparent policies and procedures such as manifested in the "Implementation Manual" prepared for the EFMP project. 3.36 Although the MIS data record an average of 13 SW per year devoted to supervision of TA II in the four years FY87-90, the PCR noted that "actual" in-country supervision, in conjunction with TA III and the UNDP project, between 1988-91 averaged six missions and approximately 20 weeks per year. 3.37 Clearly TA II should have been subjected not only to much closer supervision, during 1984-87, of the implementation of the poorer performing components, but much more importantly to more managerial oversight and stock-taking of (i) the relevance of the components as they evolved under the "process approach"; and (ii) the feasibility of implementation and the prospects of sustainability under the disenabling country conditions that prevailed while more than half of TA II's funds were being disbursed. 3.38 Bank performance in the early years of TA II was therefore not very satisfactory, mainly because the process approach was not used to the extent it might have been to ensure that the project was kept relevant and realistic in the light of the changing circumstance. That the project became more relevant and realistic towards its conclusion was mainly due to (i) the "exogenous shock" (in the positive sense) of the launching of the ERP in 1987 by a new administration; and (ii) the learning process through which, as the PCR aptly puts it, "a commonly shared view of what needs to happen" emerged, facilitated by the frequency and hands-on, pragmatic, and problem-solving approach of the Bank's supervision missions. 3.39 It should be noted that, coincidentally, mid-1987 was not only when Uganda launched its Economic Recovery Program, marking a turning point in policy under conditions of greatly-improved security and governance. This was also the time of a major Bank reorganization that led to changes in management in the department and divisions dealing with Uganda. The OED mission is of the impression that these management changes did not however much affect the course of TA II, given the continuity that prevailed at the working staff level in the Uganda country team at the time. 3.40 Both Bank and Borrower performance under TA III were on the whole highly satisfactory. Compliance by the Borrower with the covenants was satisfactory as was compliance by Bank staff with Bank policies and directives. On the whole, TA III appears to have been well managed and well implemented, with no major implementation delays or problems. As the PCR notes, however, in TA III as in TA II the Government did not strictly follow the required administrative procedures. "Many of the smaller subprojects, covering training..., procurement of equipment, funding for small studies, etc. were not discussed by the required project review committee and the updating and understanding of the project by interested government officials owed more to [Bank] missions and aide- m6moire than to outward communication from MPED." 3.41 Nevertheless, the flexible process approach to project execution was used to good effect without the project's falling prey to the kind of indiscipline and abuse that sometimes befalls such projects. Most importantly, both the Bank and the Government 40 deserve credit for selecting components that were highly relevant both to implementing the economic recovery program and to fostering ID. 3.42 Both the choice of relevant components and the relatively effective management were attributable to a considerable extent to (i) the "learning" that took place in the latter part of TA II, and the application of the lessons of TA to the design of TA IHl; (ii) the leadership on the Government side of an able Permanent Secretary with a strong sense of ownership of the project and a clear vision of how the project should and could be used; (iii) a practical, results-oriented task manager on the Bank side who provided intensive and extensive supervision; and (iv) the continuity and cohesiveness of the Uganda country team within the Bank coupled with the continuity of the Permanent Secretary and project management unit staff on the Government side. This continuity facilitated the development of close working relationships and understandings that are essential to the success of highly process-oriented projects such as TA II and III. 41 4. Selected Issues and Sustainability 4.1 This chapter discusses briefly a few issues that, while not being specific project issues, are important elements of what may be called Uganda's "institutional environment," and are therefore important to the sustainability of the benefits of TA II and III. The issues, which are all closely interrelated, are discussed under the headings of salary supplements, aid coordination, and home-grown distortions. The chapter concludes with a consideration and rating of the projects' sustainability. Salary Supplements 4.2 At the time that TA III was being prepared, years of inflation had drastically reduced the purchasing power of civil service salaries to a level that was well below a "living wage." The most important consequence of this development for the functioning of the civil service was the rampant (and somewhat institutionalized) phenomenon of moonlighting on government time, the extent of which brought productive civil service work to a near standstill. Under TA II, incentives were paid to task forces charged with specific assignments, and the UNDP-financed planning project in MPED was already paying supplements to a number of professionals and support staff. But TA III, through the salary supplement scheme financed for EAU staff, represented the first explicit Bank payment of salary supplements, and at a level (then US$300 per month) that seems low by absolute standards but was high by local standards. 4.3 The Economic Analysis Unit was originally set up at the request of the Minister of Finance who wanted a unit that would readily provide data and analytical information related to the economic recovery program and to improve the ministry's system of economic decision-making. TA III was to finance the EAU for a three-year period after which the functions of the Unit were to be absorbed into the existing ministerial structure. The scheme of salary supplements produced immediate results. Completely staffed by Ugandans, the unit worked long hours and made a recognizable contribution to some of the ministry's key functions, notably expenditure control and budgetary management. 4.4 As the Unit earned a reputation within the ministry for delivering the goods, more tasks and assignments were added to its existing functions. As an island of excellence within the Ministry, the EAU could have had an important demonstration effect through its performance. But the Unit's special incentive scheme and the special attention it received caused resentment among other units who collaborated and supported in EAU in its work, and as they were not beneficiaries they reportedly became less cooperative and productive. 4.5 Over the longer term the EAU approach was not sustainable for two principal reasons. Following the merger of MoF and MPED and the decision to integrate EAU into the mainstream of the ministry, it became difficult to align the salary supplement 42 levels of professionals who were going to be working side by side, largely because EAU staff benefitted from much higher levels. In fact, the attempt to reduce the level of supplements of some EAU staff resulted in them leaving. It was also suggested by more than one senior manager from the Ministry of Finance that the Unit's staff were so overworked that some "burnt out" and decided to leave. 4.6 The experience of EAU with salary supplements points to some lessons which may be of particular relevance to countries where the temporary use of salary supplements is being considered as a means of activating a dormant and dysfunctional civil service. These lessons are consistent with those identified in Chapter 21 on "Salary Supplements" in the World Bank's Handbook on Technical Assistance." That chapter begins with the following statement: The Bank does not support the practice of salary supplements. Instead of direct supplementation the Bank encourages borrowers to undertake a program of civil service reform, including salary scales which reflect levels of responsibility. In exceptional circumstances, however, the Bank might agree, with prior approval by the Regional vice president, to the use of Bank loans by the borrower for salary supplements. In these cases, direct supplementation should be part of a two-pronged approach-provision of salary supplements should go hand in hand with reforms of civil service pay and employment policies. 4.7 Salary supplements in the case of EAU were probably indispensable in getting competent economic management professionals in the Ministry of Finance to work productively. Although they may have been intended as a temporary measure, the project failed to identify how and at what stage these supplements would cease to be paid, with a phasing out of the supplements linked to a phasing in of civil service reforms and fiscal reforms geared to ensure that the Government could progressively pay itself an increasing share of the "living wage" paid to civil servants in a downsized or "right-sized" civil service. Thus, the supplements paid in TA III were made in an open-ended arrangement, without the specific "two-pronged approach" now called for by Bank policy. And while the EAU supplements did prove to be temporary, TA III was but one of a growing number of donor-financed projects that paid supplements to government staff working on donor- assisted projects. 4.8 As a second-best "quick-fix" solution, the benefits of salary supplement schemes lie in their short-term application. Under circumstances such as those that Uganda was facing (below minimum-living-wage salary levels and a non-functioning civil service), they can be indispensable to getting the job done. However, over the long term, the continued use of this instrument can give rise to serious systemic difficulties relating to financial sustainability, transparency, and equitability in civil service pay, dualism in the civil service structure, and the generation of a rent-seeking, patronage-dominated incentive system. 14. Handbook on Technical Assistance, OPD, World Bank, 1993. 43 4.9 A credible time-bound phasing out plan must have a number of key elements. The most important is the firm commitment of both Government and donors to a realistic trajectory of declining levels of donor-financed supplements matched by increasing government salary levels. This ensures that as supplements are phased out beneficiaries maintain their real salary levels. However, in addition, any phasing out plan, (the ultimate objective of which is to increase civil service productivity) must be underpinned by a civil service reform strategy which systematically addresses not only the issue of salary levels, but also the question of how to finance them through downsizing and other cost-saving measures as well as through increased government revenues; how to improve the relevant conditions of service; and how to move towards a more performance-related reward system. 4.10 One of the major risks identified in the SAR for TA III was the "difficulty of sustaining institutional reforms in face of inadequate civil service remuneration." This risk was, however, addressed only by making reference to a number of measures that the Government had initiated "to reduce the size of the civil service, to restore discipline and to increase wages for remaining employees," without an evaluation of the adequacy and bearing of these measures (in the face of proliferating supplements paid by the Bank in other projects and by other donors) on project sustainability. 4.11 Despite repeated declarations by the Government and donors of their commitment to phase out supplements and phase in minimum living wage salaries, little progress had been made up to December 1994, when the OED mission to audit TA II and TA III visited Uganda. One impediment to progress was the lack of any clear incentive for the Government to reduce donor support for such supplements; quite the contrary. UNDP sought to establish a target date of June/July 1994 for a 25 percent reduction in salary supplements, but this did not receive the support of other donors and no action was taken. A new multi-donor study of the system of incentives and supplements was launched in February 1995. Some of the principal findings and recommendations of the team that will prepare the new study are reported below in para. 4.14 ff. Donor Coordination 4.12 As donor-financed salary supplements became more pervasive in the late 1980s and into the 1990s, the problems created by these incentives mounted. One report described the situation as follows: Distortions arose from the ad hoc nature of donor policies, if indeed any existed.... Different levels of payments were made by donor on different projects and in different sectors. This led to situations of resentment between project and non-project staff, competition between donors to acquire civil servants for their projects, and movement of civil servants between projects to obtain the highest rate of incentive payment." 15. Quoted from "Statement for Unified System of Incentive Payments for Government Staff on Donor Assisted Projects", in "Implementation Manual" for EFMP. 44 One of the effects of the present Uganda "success story" is that many donors want to contribute, resulting in, as one donor representative put it, "more funding than projects, and more projects than can be implemented." 4.13 In competing for the most competent and productive civil servants for their projects, donors have employed a range of perks, incentives, allowances, honoraria and supplements which has increased over time because of the ratchet effect of the competition among donors. This has resulted in an uncoordinated and unregulated-and largely undocumented-system of rewards designed to attract and retain good people on donor projects and in the offices of donor agencies. Efforts to standardize salary supplements through the implementation of a Unified Incentive Payments Scheme (UIS) proved problematic as donors found it difficult to come to agreement on how to implement the new scheme. 4.14 The multi-donor mission on the UIS that visited Uganda in February 1995 found most donors in agreement that the UIS should be terminated, since it distorts salary structures and creates jealousy and morale problems within the civil service. Some donors expressed their view that top-ups are counterproductive insofar as they lead many civil servants to divert their time already remunerated to other unrelated activities. All donors were in agreement that, when most allowances become monetized and a living wage is also achieved, all salary supplements and other allowances (such as sitting allowances, or payments for attending meetings) should be terminated. The mission estimated the current level of donor-financed incentives budgeted under the Government's FY94/95 development budget at about US$20 million. 4.15 The draft of this PAR sent to the Government in March 1995 observed that there is an urgent need for the donors and Government to adopt an agreed "exit plan" for phasing out donor supplements of civil service salaries. In commenting upon that draft, the Permanent Secretary of MFEP noted that "there is [as of late April 1995] in place an agreed 'exit plan'," with the timing of the phase-out planned for the beginning of FY96/97.16 4.16 This timing-evidently for completion of the phase-out-was projected in the Government Budgetary Framework [as of early 1995]. In this Framework, a proposed wage bill of USh 220 billion would permit full attainment of a Government-paid living wage for the public service by July 1996. At that time, according to the Framework, the Government and donors would be able to terminate the UIS, since a gap between actual wages and the living wage would no longer exist. But doubts have since been cast by Bank and Fund staff of the feasibility of raising the wage bill beyond USh 190 billion in FY96/97, and hence the feasibility of achieving a phase-out until FY97/98. OED's forthcoming Performance Audit Report on the Bank's Economic Recovery Project II raises further doubts about the revenue outlook, and the extent to which the projected growth in the wage bill can be reconciled with competing claims on expenditure. 16. A copy of the letter of the Permanent Secretary is provided in Attachment A to this report. 45 4.17 There are many complex issues involved in phasing out the donor supplements that are beyond the scope of this present PAR to consider. These include the concept and measurement of the minimum living wage itself, the feasibility of the Government's financing of wage bills of alternative sizes over time, and options for the future operation of the UIS during the phase-out period. It appeared, as of mid-1995, that promising steps were being taken with regard to the "exit plan" and the harmonization of donor policies. But there remains still some way to go to actually achieve full harmonization and a full phase-out of donor supplements. Home-grown Distortions 4.18 The distortions in Uganda's incentive system are not, however, entirely created by the nonstandardized and nontransparent donor supplements. Indeed, many of the distortions and defects are home-grown and are merely exacerbated by the donor-paid incentives. 4.19 As the Public Service Review and Reorganization Commission concluded in 1990, the civil service is beset not only by its "bloated structure" (since reduced by about half-a notable achievement) and its inadequate pay and benefits, but also by "poor management skills, dysfunctional organization, [and] inadequate personnel management ... leading to abuse of office and government property, moonlighting and corruption, lack of discipline, an erosion of rules and regulations, obsolete procedures, lack of appropriate systems, poor public service attitudes, and massive bureaucratic red tape."7 Given this institutional environment, in which TA II and IHl took place, it seems all the more remarkable that the projects yielded the benefits they did. 4.20 As the Commission rightly observed, improving incentives is the most important prerequisite for a more effective civil service. But the incentive system is itself still highly distorted by 76 different types of allowances paid in the civil service-some relating to the nature of the job (e.g., housing allowances for senior civil servants), and some relating to expenses incurred in the course of duty (per diems, sitting allowances for attending meetings, honoraria, etc.). As the Commission noted, "the multitude of allowances creates problems in terms of defining when they should be paid, methods of payment, administrative complexity and scope for abuse." 4.21 In civil service environments such as Uganda's, where reforms have not yet established a meritocratic and performance-related reward system, it is especially difficult to operate a salary supplement scheme in an isolated area. Given the integrated nature of the work of a civil service, the scheme may miss out some very productive professionals who, in spite of being outside the targeted unit, may make important contributions to the unit's effective execution of its functions. This problem has been addressed only in part by TA III's successor project, EFMP, under which there has been a sizable broadening of the pool of beneficiaries. 17. "Management of Change", op cit, p.16 46 4.22 The system still rewards positions (as opposed to productivity). It also leaves out professionals outside the economic management area, raising a new problem. Ideally, and from the point of view of an equitable system of pay and grading, salary supplements should be used to reward those professionals in the civil service as a whole who are doing the most productive and important tasks. The present situation in which certain middle-level economic managers who qualify for supplements are paid more than senior managers and professionals outside the economic management area (irrespective of the importance of their role in the country's economic development) goes against some of the civil service reform's fundamental principles of equitability in pay and grading. Sustainability 4.23 Neither of the PCRs on TA II and III gave an explicit sustainability rating for the projects, although the general thrust of both PCRs was that, in view of the progress being made in the economy in general and in moving ahead with civil service reform in particular, the projects' sustainability would be "likely." This was the rating entered provisionally into the OED database for TA II, but OED entered a rating of "uncertain" for TA III, owing to "continuing difficulties to achieve appropriate public sector salaries," pending the findings of this PAR. 4.24 One of the most encouraging aspects of Uganda's recent history has in fact been the progress made in moving ahead with the difficult task of civil service reform. The sections above underline how difficult the task will be, given all the distortions that remain to be corrected plus the difficulties inherent in reducing what has become a heavy aid- dependency. But the result to date have been impressive, viz:"' a. The number of civil servants on the government payroll has been reduced from 320,000 in 1990 to 167,000 in March 1993. (Ghost workers were removed from the payroll and safeguards were built into the system to ensure that removal was permanent.) b. Functional reviews of all ministries have been completed and proposals for new structures and staffing levels prepared. Recommendations have been adopted by cabinet and are being implemented in five ministries, and the remaining sixteen are under cabinet review. c. All senior managers and common cadre staff were interviewed and evaluated by a panel as part of the ministerial review/downsizing effort. This not only helped to identify poor performers, but also signalled that downsizing was targeting poor performers at all levels, not just at the lowest levels of the civil service. 18. This paragraph is based largely upon a presentation made by Peter Longseth to a World Bank seminar. See also P. Longseth, Civil Service Reform in Uganda, paper delivered to Conference on Public Sector Financial Management, Ottawa: CIDA, June 1994. 47 d. Ministry of Public Service payroll monitoring and establishment control function has been strengthened. e. A capacity building plan has been drawn up, donor funding secured, and the first seminar ("Management of Change" for Ministers and Permanent Secretaries) has been held. f. The concept of ROM has been introduced. A baseline study is currently in the planning stages. When implemented, the study will yield indicators that can be monitored over time. g. The district review process has been initiated. h. A voluntary retirement plan has been initiated with donor funding of US$15 million. i. Agreement has been reached on a plan to monetize benefits. 4.25 The sustainability of the benefits of TA H1 and III depends largely on continued progress on the above fronts. The PAR rates the sustainability of the benefits of both projects as uncertain. On the positive side, Uganda is fortunate to have a current political leadership with vision and a commitment to civil service reform, on which front remarkable progress has already been made. The "ethos of the bureaucracy" and the productivity of the civil service are improving. Yet as the OED PAR on the Second Economic Recovery Project (forthcoming) indicates, the prospects are problematic for raising government revenues enough to sustain the productivity improvements, even in a downsized civil service, while phasing out donor-financed supplements. Hence, the sustainability of both projects is rated as uncertain.  49 5. Conclusions and Lessons Main Conclusions: Summary 5.1 It was a strength of both projects that they had a "process" rather than a blueprint design and were allowed to be managed with flexibility. But this very flexibility could have been a weakness unless sufficient "discipline in the process" was maintained by strong project management on both sides. This management was initially weak under TA II but improved over time. It was, however, a fundamental weakness of TA II that its risk assessment was weak, mainly because the governance constraints were underestimated. 5.2 Similarly, government commitment/ownership was initially weak but it strengthened over time, partly because of the learning process whereby Ugandans perceived the benefits of well-managed TA. A key factor in the success of TA III was a project champion on the Government side. Another key factor was the intensive supervision provided by the Bank, with the quality of supervision (problem-solving, practical) being more important than the quantity. 5.3 Owing to the inadequacy of incentives (with civil service pay below a "minimum living wage"), there was justification for donors to pay temporary salary supplements. But these should have been more coordinated and transparent. No "exit plan" has yet been formulated in the context of the Government's fiscal outlook and civil service reform program. The progress to date on other aspects of this program is however encouraging and needs to be sustained if the benefits of the two TA projects are to be sustained. Lessons 5.4 These conclusions underlie some lessons of Uganda's experience with TA II and III. The main lesson is that: * Management matters most. The main complementary lessons are that: * The more unsettled a country's "institutional environment" and the more uncertain the prospects that project benefits will be sustainable (as in Uganda in the mid- and late-1980s), the more flexibility needs to be built into the project design, i.e. the more the project needs to have a "process" vis-d-vis a "blueprint" design. The more flexibility is built into the project design, however, the stronger must be the management experience and "field skills" of the Bank's task manager(s). The more the flexibility, the greater the scope for abuses (e.g., diversion of project funds to low priority purchases of vehicles, study/travel abroad, etc.) and hence the greater the need for "discipline in the process." 50 * It is critical in such projects (especially in countries emerging from political strife and characterized by a very poorly functioning public administration) to have a credible government commitment to reform (including civil service reform) plus a committed project "champion" on the Government side. Even without thorough upstream preparation, and even in the face of an initially disenabling institutional environment, ambitious institutional development (IDTA) projects can be made to succeed if there is a shared commitment and ownership among the key players, and sufficient discipline in project management. * Sustainability, civil service reform, revenue growth, and expenditure containment (including through "right-sizing" of the public sector) are inextricably linked. Such links should be explicitly analyzed (and if possible modelled) both at the time of project design and on an ongoing basis with a view to enhancing the prospects for sustainability. An agreed "exit action plan" for donor-financed salary supplements should be a precondition for donors' entering upon the payment of such supplements in the first place. Some additional, supporting lessons are that: * The instruments are not the issues. TA III showed, for example, that under the right conditions, long-term expatriate advisers can be highly effective and beneficial and do not necessarily "undermine" capacity building. Under the wrong conditions, on the other hand (as prevailed during the early years of TA II), no TA modalities (whether short-term consultants, local consultants or twinning arrangements) are likely to be effective. * Harnessing existing capacities is sometimes more important than "building new capacities." Adequate capacities often exist in-country and even within the Borrower's public sector institutions. The real challenge is to develop a work environment with appropriate rewards and incentives that will get nationals to produce their best work. This may justifiably call for the temporary use of special incentives. * In a highly aid-dependent country such as Uganda, particular care must be taken to minimize donor-created distortions in local incentive structures and budgetary processes. Such distortions can be especially severe when donors pay (as in Uganda) varying salary supplements to Ugandan civil servants and bid against each other to recruit skilled Ugandans to work on their projects and staffs. Donor policies on salary supplementation must be harmonized in the context of an agreed exit plan. * Continuity and cohesiveness of the country team on the Bank side and of the project management team on the Government side enhance learning and the application of lessons of experience, and so improve prospects for project success. 51 Attachment A Telephones: Miister: Ministry of Finance anc Kampala 243054 & 232370 Economic Planning, Office: Kampala 234700S (10 Unes) P.O. Box 7086 Telex: 61170 Ka-pala Telegrams: -FINSEC.. Uganda. In any ccrrespondence on thissubtecteleasequote No... ..... THE REPUBLIC OF UGANDA tD/C/l 854/6/6/iI 20thApril, 1995. NvMr Carl Jayarajah Acing Division Chief Country Policy Industry and Finance Operations Evaluation Department 1818 H Street, N.W Washington, DC.20433 USA FAX: 202 522 3124 Dear Mr Jayarajah, RE: Second and Third Technical Assistance Proiects (Credits 1434-UG and 1951-UG) PAR I ara in receipt of yours March 29, 1995 to which you attached the draft PAR for my comments. I have read the draft PAR. I have the following two comments to make:- (i) There is, in place, an agreed "exit plan" for phasing out donor supplements of civil servants salaries. The timing is beginning offiscal year 1996/97. You may wish to check with Mr Ian Knapp and Ms Ritva. (ii) The draft PAR is, by and large, a true analysis of the two operations (TA II and TA I) Perhaps I should add this not so important comment- There are a number of editorial errors in the draft text. Thank you very much indeed for allowing me to have a look at the draft PAR. Yours sincerely, Fran' Tu uheirwe for: RMANENT SECRETARY/SECRETARY TO THE TREASURY     > ) -o 1 t r> 0 Ct COl.