Ukraine - Rehabilitation Loan Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No: 18852 PERFORMANCE AUDIT REPORT UKRAINE REHABILITATION LOAN (LOAN 3831-UA) January 25, 1999 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents Currency Unit = Hrivnya Period average rate 1996 US$1.00 H 1.8295 1997 US$1.00 H 1.8617 1998 I Qtr. US$1.00 H 1.9666 H Qtr. US$1.00 H 2.0494 Abbreviations and Acronyms CIS Commonwealth of Independent States DIC Direct Import Component FEC Foreign Exchange Component FSU Former Soviet Union GDP Gross Domestic Product GOU Government of Ukraine ICR Implementation Completion Report IPA International Procurement Company OED Operations Evaluation Department PCU Project Coordination Unit TA Technical Assistance Fiscal Year Government: January 1 - December 31 Director General, Operations Evaluation: Mr. Robert Picciotto Director, Operations Evaluation Department: Ms. Elizabeth McAllister Manager, Country Evaluations & Regional Relations: Mr. Ruben Lamdany Task Manager: Mr. Luis Landau FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. Office of the Director-General Operations Evaluation January 25, 1999 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Ukraine - Rehabilitation Loan (Loan 3831-UA) Attached is the Performance Audit Report (PAR) for the Ukraine Rehabilitation Loan (Loan 3831-UA, for US$500 million equivalent, approved in FY95 and closed on schedule in FY96) prepared by the Operations Evaluations Department (OED). The first half of the loan was released shortly after effectiveness in December 1994 and the second half was released on May 15, 1996. The loan was fully disbursed. Cofinancing for the project was provided by the Canadian Government for US$10 million and the Export-Import Bank of Japan for US$150 million. The main objective of the loan was to support the Government's economic reform program to stabilize the economy and create the conditions for a resumption of economic growth. Specifically, the loan aimed to support: (a) the development of competitive markets; (b) improved financial discipline and hard budgetary constraints for enterprises, farms, and banks; (c ) the targeting of benefits to the most needy. Key reforms supported by the project included price and trade liberalization, development of the private sector, abolition of the system of state orders, privatization of small, medium and large enterprises, and agricultural land; restructuring of the electricity sector; and improved targeting of benefits to protect vulnerable groups. Other objectives were: (i) to finance critical imports; (ii) to strengthen the social safety net; (iii) to improve the foreign exchange market; and (iv) to provide a framework for financial assistance from other donors. Progress in attaining the loan's objectives was mixed. The loan provided timely financial help to Ukraine during a financial crisis, thus helping to carry reforms forward. By the end of 1995 there was a successful macroeconomic stabilization program which, compared to 1994, almost halved the budget deficit, reduced the rate of inflation to 1-2 percent a month, replenished international reserves, stabilized the currency, liberalized the prices of major goods and services, and opened foreign exchange and trade regimes to market forces. But various administrative price controls and export restrictions are still in place. Moreover, new export quotas and export duties have been placed on some commodities. Medium and large firm privatization was a notable disappointment. At the end of 1995 the majority of medium and large enterprises were still government-owned. And those firms that were privatized did not improve much their governance of profitability. Little progress was made in imposing financial discipline on enterprises. In response to monetary tightening and reduced government subsidies, interenterprise arrears grew 240 percent in 1994 and another 128 percent in the first half of 1995. Although on a smaller scale, subsidies were renewed to ailing industries, especially the coal sector. Commercial banks were encouraged by the Government to make loans to state enterprises. No comprehensive reforms were introduced to strengthen the social safety net. A severe lack of analytical and institutional capacity, as well as inadequate statistical data on the recipients of social assistance, hindered the targeting and administration of social protection programs. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 11 Several factors help to explain the shortfalls of this ambitious operation. In spite of a successful public education campaign carried out by EDI and by the Bank's field office in early years, political changes in mid-1996 reversed government support for the reforms. There was a political struggle between decisionmakers that weakened support for the reforms. The need for certain liberalization measures recommended by the Bank was not fully understood by some government officials, who questioned the allocative ability of the private markets and feared that the privatization process would be open to corruption. Poor weather and shortfalls in external financing also contributed to weaken stabilization and liberalization measures. The ICR rated project outcome as satisfactory, institutional development as modest, and sustainability as uncertain. Given the formidable economic crisis at the time the loan was approved, OED concurs with these assessments except for project outcome. While good progress was made in stabilizing the economy through the project, the clearly disappointing results of enterprise privatization, the failure to impose hard budget constraints on enterprises, and the inability of the Government to improve the targeting of social assistance programs lead OED to rate project outcome as "marginally satisfactory." A number of valuable lessons can be learned from this project. Its timely support for the reform program was most valuable to help bring about macroeconomic stabilization and to launch the reform program. But overoptimistic expectations may have helped to undermine the Bank's credibility regarding enforcement of conditionality. More attention should have been given to putting in place institutional and legal mechanisms to carry out the structural reform, including providing effective technical assistance. In this respect, continuing to maintain a strong and authoritative permanent presence in the country could be most valuable. Finally, the Bank lacked a suitable lending instrument to offer quick support while addressing the need for structural reforms. Attachment Contents Ukraine at a glance.......................3........... .............3 Ratings and Responsibilities ................................. ........... 5 Preface ......................................................... 7 1. Introduction.............................9...... .............9 2. Objectives .......................................... ....... 10 3. Instruments .......................... ............... 11 Stabilization ................................................ 11 Support for Economic Reform Program. ..................... ........... 11 4. Project Design ...................................... ........ 12 Technical Assistance....................... ............... 12 5. Implementation and Outcome ............................. ....... 13 Macroeconomic Stabilization ............................ ......... 13 Structural Reforms........................ .............. 13 Tranche Release ............................................... 14 Cofinancing.................. ....................... ..... 14 6. Evaluation ......................................... ........ 15 7. Lessons Learned................................... .............. 17 Timely Support for a Reform Program can be Critical to Long-term Progress............17 Setting Overly Optimistic Goals Risks Undermining Bank Credibility........................ 17 Development of a New Lending Instrument Should be Considered .................. 17 Assessment of Risks .................................. ......... 18 Box 7.1 Views of Government Officials............................... 20 This report was prepared by Elliot Hurwitz (Consultant) and Luis Landau (Task Manager). The project was audited in April 1998. Norma Namisato and Barbara Yale provided administrative support. 2 Contents (cont.) Annexes A. Basic Data Sheet................................ .................23 B. Overview and OED Ratings......................................26 C. Comments received from the Export-Import Bank of Japan.................. 27 3 Ukraine at a glance Europe & Lower- POVERTY and SOCIAL Central middle- Ukraine Asia Income Development diamond* Population mid-1996 (milhons) 51.4 479 1,125 GNP per capita 1996 (US$) 1,180 2,180 1,750 Life expectancy GNP 1996 (bilfons US$ 60.7 1,043 1,967 Average annual growth, 1990-96 Population (%) -0.2 0.3 1.4 GNP Gross Labor force (%) -0.2 0.5 1.8 per primary Most recent estimate (latest year available since 1989) capita enrollment Poverty: headcount index (% of populadon) 32 Urban population (% of total populadon) 70 65 56 Life expectancy at birth (years) 69 68 67 Infant mortality (per 1,000 Ive births) 15 26 41 Access to safe water Child malnutrition (% of children under 5) Access to safe water (% ofpopulation) 97 .. 78 Illiteracy (% of popula0ion age 15+) 2 - Ukraine Gross primary enrollment (% of school-age population) 87 97 104 Male 87 97 105 - - Lower-middle-income group Female 87 97 101 KEY ECONOMIC RATIOS and LONG-TERM TRENDS 1976 1985 1995 1996 Economic ratios* GDP (billions US$) .. .. 47.5 46.6 Gross domestic investment/GDP .. .. 26.7 22.7 Openness of economy Exports of goods and services/GDP .. .. 47.1 45.5 Gross domestic savings/GDP .. .. 23.5 20.5 Gross national savings/GDP .. .. 22.9 20.4 Current account balance/GDP .. .. -3.2 -2.6 Interest payments/GDP .. .. 1.4 1.2 Savings investment Total debt/GDP .. .. 17.3 19.6 Total debt service/exports .. .. 6.9 6.3 Present value of debt/GDP .. Present value of debt/exports 47.9 Indebtedness 1975-85 1986-96 1995 1996 1997-05 (average annual growth) -Ukraine GDP .. -9.0 -12.2 -10.0 5.4 Lower-middle4ncome group GNP per capita .. .. -11.5 -9.6 5.4 Exports of goods and services .. .. 2.7 19.1 7.2 STRUCTURE of the ECONOMY (% of GDP) 1975 1985 1995 1996 Growth rates of output and investment (%) Agriculture .. .. 15.0 13.0 a Industry .. .. 50.3 46.2 -10r Manufacturing .. .. .. 20 Services .. .. 34.7 40.8 30t Private consumption .. .. 55.1 57.8 -40 General government consumption .. .. 21.1 21.7 _GDI --f-GDP Imports of goods and services .. .. 50.2 47.9 A 1975-85 1986-96 1995 1996 (average annual growth) Growth rates of exports and imports (%) Agriculture .. .. -4.6 -10.3 20 Industry .. .. -15.7 -10.2 i Manufacturing .. .. .. .. Services .. .. -6.4 -8.5 10 Private consumption .. .. -0.1 -5.6 s.4 General govemment consumption .. .. -3.5 -8.2 0 i Gross domestic investment .. .. -33.7 -23.5 91 92 93 94 95 96 Imports of goods and services .. .. 1.5 17.4 " Exports imports Gross Domestic Product .. -9.5 -12.1 -9.9 Note: 1996 data are preliminary estimates. Figures in italics are for years other than those specified. The diamonds show four key indicators in the country (in bold) compared with its income-group average. If data are missing, the diamond will be incomplete. 4 Ukraine PRICES and GOVERNMENT FINANCE Domestic prices 1975 1985 1995 1996 Inflation (%) (% change) 6.000 Consumer prices .. .. 377.0 80.0 4,ooo Implicit GDP deflator -0.5 -1.3 415.8 64.1 Government finance2.o (% of GDP) 0 Current revenue .. .. 37.8 37.2 91 92 93 94 95 96 Current budget balance .. .. -5.0 -3.1 GDP def. - CPI Overall surplus/deficit .. .. -4.9 -3.2 TRADE 1975 1985 1995 1996 (millions US$) Export and Import levels (mill. USS) Total exports (fob) .. .. 13,647 15,118 20,000 Commodity 1 - ferrous metals .. .. 4,484 4,847 Commodity 2 - ores, slags, ashes .. .. 560 605 15.O Manufactures .. Total imports (cif) .. .. 15,945 19,376 to,ooc Food .. 745 811 50oc Fuel and energy .. .. 6,946 8,107 Capital goods .. .. 3,281 4,891 o Export price index (1995=100) .. 100 102 90 91 92 93 94 95 96 Import price index (1995=100) .. 100 99 gExports aimports Terms of trade (1995-100) .. 100 103 BALANCE of PAYMENTS 1975 1985 1995 1996 (millions US$) Current account balance to GDP ratio (%) Exports of goods and services .. .. 16,436 19,935 Imports of goods and services .. 17,643 21,067 LA W Resource balance -1.207 -1,132 Net income -508 -579 Net current transfers 200 512 Current account balance, before official capital transfers .. -1,515 -1,199 -25 Financing items (net) .. 476 1,295 Changes in net reserves .. .. 1,039 -97 -3o Memo: Reserves including gold (mill. US$) 1,069 2,087 Conversion rate (local/US$) .. 1.11 1.83 EXTERNAL DEBT and RESOURCE FLOWS 1975 1985 1996 1996 (millions US$) Composition of total debt, 1996 (mill. US$) Total debt outstanding and disbursed .. 8,219 9,122 IBRD 491 859 G A IDA 0 0 231 859 F, Total debt service . .. 1,137 1.247 2144 IBRD . .. 8 32 IDA 0 0 C 2262 Composition of net resource flows Official grants 0 0 Official creditors .. 401 426 Private creditors -220 -384 D Foreign direct investment . . 266 436 E 266 Portfolio equity .. 517 350 3360 World Bank program Commitments - 146 1,260 A - IBRD E - Bilateral Disbursements .. 401 406 B - IDA D - Other multilateral F - Private Principal repayments 0 0 C-IMF G - Short-term Netflows .. 401 406 Interest payments 8.. 32 Net transfers 393 374 Development Economics Note: Estimates for economies of the former Soviet Union are subjectto more than the usual range of uncertainty. 5 Ratings and Responsibilities Performance Ratings Rehabilitation Loan (L3831-UA) Outcome Marginally Satisfactory Sustainability Uncertain Institutional Development Impact Modest Bank Performance Satisfactory Key Project Responsibilities TM Division Chief Director Appraisal Chandrashekar Pant Wafik Grais Basil Kavalsky Midterm Review Ritu Anand Wafik Grais Basil Kavalsky Completion Ritu Anand Wafik Grais Basil Kavalsky ICR was prepared by: Chandrashekar Pant 4 a 6 7 Preface 1. This is a Performance Audit Report (PAR) on the Ukraine Rehabilitation Loan (L383 1- UA) for an amount of US$500 million equivalent. The loan was approved in December 1994, became effective in the same month and closed in June 1996, the original closing date. 2. The PAR is based on the Implementation Completion Report (ICR) prepared by the Europe and Central Asia Regional Office and dated February 27, 1997, the President's Report for the project, the legal documents, a summary of the Board discussion, project files, related economic and sector work and discussions with Bank staff. Comments received from the Export- Import Bank of Japan are attached as Annex C to the PAR. 3. An OED mission visited Ukraine in April 1998 and discussed the effectiveness of Bank's assistance with government officials, representatives of the civil society and of other donors. This report was sent to the Borrower for comments on October 16, 1998. No comments were received. 4. The kind cooperation and assistance of those who contributed to the preparation of this report is gratefully acknowledged.  9 1. Introduction 1.1 From 1991 to 1994, Ukraine's economic structure changed little from the system of central planning inherited from the Soviet Union. This lack of structural reform, combined with loose financial policy and huge increases in energy costs, contributed to rampant inflation and plunging income. In mid-1994, the newly-elected President Kuchma indicated his strong support for macroeconomic stability and systemic reforms-a commitment that was opposed by some segments of Ukrainian society and many Parliamentarians. 1.2 In the fall of 1994, the Bank began discussions with the Government of Ukraine (GOU) on a US$500 million Rehabilitation Loan, and the loan was approved on December 22, 1994. An initial disbursement of US$250 million was authorized upon effectiveness (December 27, 1994), with disbursement of the balance authorized in May 1996, contingent upon a satisfactory review of progress in implementing reforms (which was granted). The loan was fully disbursed by May 31, 1996. Cofinancing was provided by a loan of US$150 million from Japan, and a grant of US$10 million from Canada. The loan followed approval, in October 1994, of a first purchase of US$365 million under the IMF Systemic Transformation Facility. 10 2. Objectives 2.1 The main objectives of the loan were to support the Government's economic reform efforts, stabilize the economy, and-after the severe drop in output from 1990-94--establish conditions for the resumption of economic growth. Other objectives were to: (1) provide foreign exchange for the purchase of critical imports; (2) provide budgetary support for strengthening the social safety net; (3) improve the functioning of the foreign exchange market; (4) catalyze financial assistance from other donors. 2.2 The loan supported an ambitious and comprehensive economic reform program outlined by the Government, following years of inaction by the previous regime. The Bank responded quickly and decisively as soon as there was evidence of government ownership for the reform program. These were sound and relevant objectives. The Rehabilitation Loan and its associated economic reforms were intended to--and eventually did-serve as a foundation for the Bank's subsequent lending program to Ukraine. 11 3. Instruments Stabilization 3.1 To reduce inflation from over 4,000 percent yearly to about 15 percent, the government agreed to cut its consolidated budget deficit to no more than 10 percent of GDP in 1994, mainly by reducing subsidies on coal, housing, and communal services, cuts in support for agriculture, and other expenditures. To protect vulnerable groups, a program of targeted assistance was developed. The 1995 budget deficit was expected to be reduced to half the level of 1994. 3.2 Monetary growth was also to be restricted, with growth targets of 40 percent in the fourth quarter of 1994 and 21 percent in the first quarter of 1995-with both figures considerably less than expected GDP growth. Support for Economic Reform Program 3.3 The Rehabilitation Loan became effective immediately after signing on the basis of a number of significant reforms implemented prior to Board presentation, including: * A Presidential Decree (Decree 699) outlining simplified privatization procedures. * Announcement of sectors to be included in the 1995 privatization program, and the statement that at least 8,000 medium and large enterprises would be included in the program. * Initiation of strategic policy papers on the social safety net. * Instructions to 20 large enterprises to prepare restructuring plans by mid-March 1995. * Increase in electricity prices to fully reflect recent fuel price increases. 3.4 After a satisfactory review of the implementation of the reform program, the remaining US$250 million was to be disbursed. Specific measures that were to be considered in the review included: * Satisfactory implementation of macroeconomic stabilization policies. * Termination of the state order system, and its replacement by a competitive system for state procurement. * Abolition of most export quotas and licenses. * Closure of selected large insolvent state enterprises. * Completion of a study evaluating alternative restructuring plans for former state banks. * Completion of strategic studies of the social safety net and implementation of appropriate actions in this area to protect vulnerable groups. 3.5 Certainly, this was a deliberately ambitious economic reform agenda---too ambitious, as it turned out-but there could be no quarrel that these were the right measures needed to help stop the economic decline and provide conditions for the resumption of growth. 12 4. Project Design 4.1 The original concept of a Rehabilitation Loan was a "quick, relatively small infusion of resources designed to get things moving, and based on some up-front actions and a statement of intentions."I In a departure, the Ukraine Rehabilitation Loan was larger and was designed with two disbursements, the second of which was based on an assessment of generally satisfactory performance in carrying oVt the reform program. This was the first Rehabilitation Loan with two-stage disbursement. The Bank made a decision that it was important to put a large sum into the hands of the GOU quickly, with only a loose attachment to progress in economic reform incorporated into this operation.2 Tighter linkage to achievement of specific reforms would be incorporated into subsequent operations, and it would be left for the managers of these operations to make further progress on the aspects of reform which had been started under the Rehabilitation Loan (which is what occurred, only more slowly than anyone in the first half of 1995 would have estimated). Thus, the Ukraine Rehabilitation Loan, in reality, was something between a rehabilitation loan and a Structural Adjustment Loan, which would have linked disbursements to achievement of specific reform measures. 4.2 Responding to this structure, during Board review an Executive Director asked, "Why is this loan called a Rehabilitation Loan and not a SAL? What was the rationale for tranching? Does this set a precedent for subsequent Rehabilitation loans?" Regional management responded that early financial support was vital, that some major reforms could not be made until several months later, and that staggering the disbursements would strengthen the hand of reformers. As for precedent, that would be determined later on a case-specific basis. This exchange highlighted the need for newer, more flexible lending instruments, whose structure watches more closely the needs of the situation at hand. Technical Assistance 4.3 The Rehabilitation Loan had been preceded by an Institution Building Loan, and by intensive public education efforts by the Bank, including private training of senior government officials carried out from the Bank's field office. While there was a clear consensus among staff who worked on the Rehabilitation Loan that weak Ukrainian institutional and technical capacity to implement reforms impaired the Government's ability to accommodate the Bank's agenda, the loan did not have a technical assistance component. The reason for this is that Ukraine was not willing to borrow for technical assistance, in large measure because considerable technical assistance was available on concessional terms from the European Union and bilateral donors. Beyond this reluctance, the GOU also declined to accept technical assistance offered by the Bank on a grant basis, seemingly because they preferred not to consider objective outside advice on certain issues (e.g., social safety net).3 I Office Memorandum, October 27, 1994. 2Although the Letter of Development Policy and the President's Report listed several specific performance parameters, these were not reflected as conditions in the Loan Agreement. This is a common problem. See Review ofPublic Enterprise Reform and Privatization Operations, Private Sector Development Department, August 1996, p. 5. 13 5. Implementation And Outcome Macroeconomic Stabilization 5.1 Progress in macroeconomic stabilization was good. Overall fiscal adjustment during 1995 was strong, with the cash deficit declining from 8.6 percent of GDP in 1994 to 4 percent in 1995. This was, however, somewhat higher than the 3.3 percent targeted, with the slippage mainly due to excessive expenditures on government employee wages, industry subsidies, and payment of energy arrears. 5.2 Monetary policy was drastically tightened, with the growth rate of broad money falling from 93 percent in the fourth quarter of 1993 to 8 percent in the fourth quarter of 1995. While the tightening in monetary policy was substantial, it fell short of the goals envisioned by the Bank. The shortfall was caused by credit extended by the banking system as part of an effort by the National Bank of Ukraine (NBU) to assist the enterprise sector. 5.3 Inflation, which had been over 4,500 percent in 1993, fell from an average of 16 percent per month in 1994 and nearly 20 percent in early 1995 (owing to price liberalization) to 5 percent by December 1995, and less than 3 percent in April 1996. Progress was sufficient for the Government to introduce a new currency, the hrivnya, in September 1996. This significant progress in stabilization was also sufficient for the IMF to approve Stand-by Arrangements for US$1.5 billion in April 1995, and US$900 million in May 1996. 5.4 Overall economic activity continued to fall, although at a slower rate. After falling by more than 23 percent in 1994, real GDP dropped 12 percent in 1995 and 10 percent (on an annual basis) in the first quarter of 1996. By contrast, exports to non-FSU countries in 1995 rose 20 percent over 1994, more than offsetting the fall in exports to FSU republics. Structural Reforms 5.5 The loan called for substantial liberalization of domestic trade and prices, as well as of exports and imports, and mixed progress was made in these areas. Significant initial progress was made in domestic prices, with most controls and profit margins lifted. The multiple exchange rate system was abolished, and export and import restrictions were lifted. 5.6 However, many controls and restrictions continued to operate, and in some cases the authorities replaced controls that had been lifted with new types of limitations. For example, while export quotas for most products were lifted, "pre-export notification" and a system of indicative prices was instituted. Quotas were re-introduced in early 1995 on exports of ferrous and non-ferrous scrap, and, in May 1996, export duties on certain commodities were introduced to be followed later by quotas on additional products. The Government continued to control much of trade in grains. 5.7 Progress on privatization was disappointing, with the pace much slower than envisaged. For example, the goal stated in the loan documents was for privatization of 8,000 medium and 14 large enterprises by the end of 1995, but by year's end only 1,015 firms had a majority of their shares sold to the private sector, and by June 1996, 2,000 such enterprises had been privatized.4 5.8 There was little progress in enforcing stricter financial discipline for enterprises. Budgetary subsidies were provided for insolvent firms (e.g., in the coal sector), and commercial banks were encouraged to provide credits. In spite of government pressure to relax overall credit, the monetary authorities continued to restrain overall monetary expansion and thus maintained macrostability, but at the cost of curtailing credit to the private sector. 5.9 The social area was also disappointing, because no reforms were introduced to strengthen the social safety net, and no real progress was made even on improving the inadequate information base. Tranche Release 5.10 The Bank was well aware of the disappointing progress in carrying out some of the promised reforms. Internal memoranda reveal the staff's concerns regarding the slow pace of medium and large privatizations, continuing softness in the budget constraints for the public enterprises, reintroduction of price controls, absence of social protection measures, and backsliding on trade liberalization and on removal of export restrictions.5 In spite of these grave concerns, the Bank felt that overall progress towards macrostability and structural reforms justified release of the second US$250 million tranche which was authorized in May 1996. It was left to subsequent adjustment operations in the country to continue advancing similar conditionality. Cofinancing 5.11 An unusual aspect of the cofinancing in this operation was that it came late in the process, already some months after the second tranche of the loan had been disbursed. An amendment to the Loan Agreement was required, and separate ratification by the Ukranian Supreme Rada. The timing of this intervention resulted in a lost opportunity to use this financing for a more effective purpose. 4 Privatization progress after the Rehabilitation Loan has continued slowly but steadily. The Enterprise Development Adjustment Loan, approved by the Board in June 1996, contains a condition that 3,500 medium and large enterprises be privatized before its second tranche (November 1996), and 5,000 by the third tranche (estimated as March 1997). Privatization of small enterprises was more successful, as the number increased to 14,000 in September 1995, from just 3,000 at the end of 1993. Because of noncompliance with audit covenants, the Bank threatened to discontinue statement-of-expense disbursement procedures in October 1995, and after two deadline extensions the required audit reports were received. 15 6. Evaluation 6.1 The results of the project were mixed, with positive aspects outweighing the negative. On the positive side, the GOU made a definitive break from prior years of dismal policies and moved toward establishing a market economy: * The loan, along with cofinancing that it stimulated, provided critical support to Ukraine's reform efforts, and reinforced the political position of reformers during a very difficult period. * After years of inaction, the loan helped stimulate the Ukraine to take the first few steps on the long path toward economic reform. * Macroeconomic performance improved dramatically, although not as quickly as expected by the Bank. * Complementing the innovative public education efforts spearheaded by the Bank in Ukraine, the loan opened a broad agenda for discussion of further reforms, and served as a foundation for further operations. * The loan provided the Bank with substantial credibility as an advocate of economic reform. * Ten additional Bank projects totaling more than US$1.2 billion were approved in 1995 and 1996, with implementation conditioned on continued progress on reforms. * Funds from the loan were used to pay for energy imports and strengthen the social safety net during the harsh winter of 1994-95. * The support of the Bank and IMF to Ukraine's reform program facilitated participation by other donors. At a March 1995 CG meeting, which the Bank chaired, pledges of US$5.5 billion in economic assistance were made, of which US$2.1 billion was from international financial institutions, US$2.5 billion in settlement of payment arrears by Russia, and US$850 million in bilateral assistance. 6.2 On the negative side, however, * In developing the reform actions requested of the GOU, the Bank was deliberately overly optimistic about what could be accomplished in the Ukrainian political context in this brief interval. Although this overshooting was meant to support the pro- reform elements, in the new government, it had the undesirable effect of undermining the domestic credibility of the Bank's conditionality. * While Ukraine took initial steps to institute structural reforms-including domestic prices, trade controls, and privatization-progress fell considerably short of what was promised, and in some cases was later reversed. * The important objective of introducing financial discipline on the enterprises was not achieved. * There was insufficient progress in strengthening the social safety net. * After making initial progress up to the point of the March review, the GOU stopped moving forward, and to some extent regressed, after full disbursement had been made. * The main reasons for this shortfall included: * Goals and an implementation timetable which were overly optimistic. 16 * Substantial resistance to reform among many segments of Ukrainian society. * Political changes which resulted in less government support for reforms after mid-1996. * Inadequate support to build and strengthen institutional capacity. 6.3 Ratings: * Outcome. OED rates the overall project outcome as marginally satisfactory, since the project played a significant role in initiating a reform program after years of inaction, and helped Ukraine to achieve a substantial measure of macroeconomic stability. In the context of the economic crisis the country faced, the financial assistance provided by the Rehabilitation Loan was timely and well appreciated by the borrower. These positive elements are assessed as outweighing the negatives of disappointing progress in some aspects of structural reform, and the reversals after mid-1996. * Sustainability. OED rates sustainability as uncertain, since a strong domestic constituency in favor of reform has yet to emerge, and the reform process has become increasingly fragile. * Institutional Development. OED rates institutional development as modest since it had a limited impact on the country's human and organizational capabilities. * Bank Performance. OED rates Bank performance as satisfactory, since the Bank moved rapidly to back reform when the opportunity arose, and utilized an innovative and appropriate lending instrument. However, because the reform goals were too optimistic and could not be attained, this exercise may have contributed to undermine the credibility of conditionality in Bank programs. 17 7. Lessons Learned Timely Support for a Reform Program can be Critical to Long-Term Progress 7.1 Ukraine in mid-1994 had changed little in its economic structure since the demise of Communism. The Bank was right in concentrating its efforts in providing public education and technical assistance, while withholding financial support. With economic policy in crisis, and with the accession of President Kuchma with his strong positions on economic reform, the Bank had an opportunity to join the policy dialogue, inject resources that would provide temporary relief, and provide reformers with a tangible benefit. The resident mission and EDI played a crucial complementary role in public education supporting the reforms. While it was evident that the Ukrainian political context would make rapid reform difficult, the first few steps toward a market economy were taken, and a foundation laid for further progress. In parallel, the macroeconomic situation was stabilized, providing relief from the earlier situation and an obvious benefit that people could see. Had the Bank not acted quickly or with major resources, it is difficult to see how reforms-incomplete though they are-could have progressed to the point where they are today. Setting Overly Optimistic Goals Risks Undermining Bank Credibility 7.2 The Rehabilitation Loan contained extensive and specific goals that Ukraine was expected to meet. While significant progress was made in some areas, many project objectives turned out to be too optimistic, given the extensive opposition to reform. Shortfalls in meeting objectives were especially large in privatization, export regulation, and the social sector, while greater progress was made in macroeconomic stabilization. The substantial gap between targets and outcomes may have undermined the Bank's credibility in subsequent dealings with the Government on reform conditionalities. Although this overshooting was deliberately meant to encourage fast progress, it remains an open question whether it would have been more effective for the loan to have had goals that were more modest, but more readily implementable. Development of a New Lending Instrument Should be Considered 7.3 The original concept of a rehabilitation loan was a quick, relatively small infusion of resources to get things moving, based on some actions taken before signing and an additional statement of intentions. This concept did not quite fit the Ukraine situation, so the Bank modified the instrument to enlarge it, and make a second disbursement (half the total) dependent on a finding of satisfactory progress toward reform approximately three months after loan effectiveness. In the eyes of the regional staff, there was no formal conditionality in the Loan Agreement, and this was left for subsequent operations. However, the Ukraine loan clearly incorporated features of both rehabilitation and structural adjustment loans. Although the Loan Agreement made disbursement of the second tranche conditional on compliance with the actions and measures described in the Letter of Development Policy, Bank staff accepted the very limited compliance as being satisfactory to the Bank. OED believes that the Ukraine Rehabilitation Loan 18 was well designed for its specific circumstances, but the Bank should consider development of a new instrument that can be utilized in situations such as those in the future.6 Assessment of Risks 7.4 The President's Report (PR) presented an ambitious program of reforms, both in the main text and in the attached Memorandum on Economic Reform Policies, signed by the First Vice- Prime Minister, which became part of the formal conditionality of the loan. OED communications with staff of the ECA Region revealed that the Bank did not expect full compliance with those reforms. Indeed, they indicated that there was a deliberate "overshooting" in the program, aimed at supporting the pro-reform forces in the country in their efforts. 7.5 Although the PR did not expressly recognize this overshooting, the section on Benefits and Risks did point out that there were four kinds of serious risks facing the loan: * First, the expected opposition to change within the cabinet of ministers, among the bureaucracy, in Parliament, and among large sectors of the population. * Second, the inadequate institutional capacity, which was likely to delay implementation of the reforms. * Third, the possible deterioration of Ukraine's relations with its neighboring countries, which could lead to interruptions and cut-offs in energy supplies. * Fourth, the risk that timely and sufficient external assistance might not be forthcoming, forcing a much faster pace of adjustment threatening the fragility of the social consensus in favor of reforms. 7.6 The PR spelled out the reasons why full implementation of the program supported by the Bank could adequately address those risks. Among other actions, the PR listed the proposed emphasis on the social safety net, mass privatization as a means of asset redistribution in favor of the entire population, development of the private sector, energy and agriculture reform programs, The Legal Department does not share this conclusion. Commenting on an earlier draft of this PAR, it has stated: "I would question whether the need for a new lending instrument has been proved as one of the Lessons Learned in this operation. This need is evidently based on the conviction that the flexible conditionality incorporated into this Loan is an inherently advantageous instrument for both parties. This I would strongly dispute. First, the statement made in paragraph 7.3 that 'there was no formal conditionality in the loan agreement,' is misleading. As you know, Schedule I of the Loan Agreement makes the second tranche conditional upon the satisfaction by the Bank 'that the Borrower has undertaken by [the date of the progress review, i.e. May 31, 1995], the actions and measures described in the letter referred to in Recital (A) to the Preamble to this Agreement.' (This refers to the letter of development policy.) So there was indeed an agreed set of policy conditions: just not stated explicitly in the Loan Agreement. The General Counsel has subsequently instructed operational counsel to eliminate this type of hidden conditionality from other adjustment loans, even single tranche adjustment (rehabilitation) loans. The reason, understandably, is that the side documents-the letter of development policy, the Bank's internal documents and MOP-in most cases have little public distribution outside the Bank. I strongly agree with the General Counsel's position on this issue, and therefore, in my opinion, the above formulation is fundamentally objectionable. Also we need to consider that expressions such as 'satisfaction of the Bank' are objectionable as a matter of legal and operational policy because it places the Borrower in the disadvantageous position of having the right to disbursement of the loan proceeds be dependent upon a non-objective standard, rather than the more neutral and enforceable sorts of objective standards which we try to use in international legal practice. The legal department expressed these concerns at the time the Loan was made, but it appears that there were overriding considerations. Be that as it may, I would still insist that the practice used does not conform to the highest standards of an international development banking practice, and that this Loan does not show any particular promise for becoming the inspiration for new lending instruments." 19 and Bank leadership in mobilizing solid financial support through the Consultative Group and other donor coordination efforts. 7.7 Was this an adequate presentation of the loan's risks to the Executive Directors? The enumeration of risks was quite comprehensive. However, by suggesting that the actions proposed in the program could provide sufficient protection, even though Bank staff believed that many of those critical actions were unlikely to be implemented during the life of the loan, the PR may have presented an overoptimistic risk assessment. 20 Box 7.1: Views Of Government Officials A few of the senior government officials interviewed by the OED audit mission offered the following comments, based on the Bank's performance on the Rehabilitation Loan. 1. The Bank's assistance was most valuable; by addressing early on the need to fight inflation it helped set the stage for the structural reforms that followed. Bank reports have been very useful and appreciated by the authorities. Cooperation with the Bank has been the best among IFIs. 2. In the early years the Bank's presence in the country was a major force to promote understanding and acceptance of the need for drastic reforms. The Bank's resident staff was seen as highly qualified, and able "to speak for the Bank," and was a powerful force driving the transformation efforts. More recently the Bank's field office has focused more on portfolio implementation. While this is a needed focus, there is also a need for staff well qualified to give guidance and advice on all important areas, and who can be seen by the authorities as able to speak for the Bank. 3. The privatization program (of medium and large enterprises) did not provide for enough follow up and support (technical and financial) to the privatized firms. The performance of the latter has not better than those that remain publicly owned. There is a need for "real owners" who will provide governance and improve efficiency. 4. The ambitious privatization program was imposed on the authorities, who did not fully realize at the time the implications and requirements in terms of institutions, legal framework, financial requirements. 5. On the other hand, the Bank should have pressed more firmly for certain reforms (e.g., changing the retirement age) early on, before the political opposition had time to get organized. 6. In the future, the Bank should be "forward looking" when proposing sector reforms, i.e., present a precise panorama of what can be expected to happen when the reforms are implemented and what complementary measures will need to be adopted later on. In other words, there should be a clear road map to show where the country is going, with a long horizon and with a full disclosure of all the likely implications, in order to achieve a complete and common understanding of the goals and expectations. 21 Annexes  23 Annex A Basic Data Sheet UKRAINE REHABILITATION LOAN (LOAN 3831-UA) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 660 660 100 IBRD Loan amount 500 500 100 Export-Import Bank of Japan 150 150 100 Canada 10 10 100 Source: Project files. There was no domestic contribution. Project Costs (amounts in US$ million) Appraisal Estimate Actual/latest estimates Item Local costs Foreign costs Total Local costs Foreign costs Total 1. Imports 0 660 660 0 660 660 Total 0 660 660 0 660 660 Source: ICR Cumulative Estimated and Actual Disbursements FY95 FY96 Appraisal estimate (US$M) 362.4 500.0 Actual (US$M) 364.16 500.0 Actual as % of appraisal 101% 100% Date of final disbursement: May 31, 1996 Source: ICR 24 Project Dates Initiating memorandum n/a Negotiations 11/16/94 Letters of Development Policy 11/30/94 Board approval 12/22/94 Signing 12/22/94 Effectiveness 12/27/94 Closing date 06/30/96 Source: ICR and interviews with project staff Staff Inputs (staff weeks) Stage of Project Cycle Planned Revised Revised as % Actual Actual as % ofPlanned ofRevised Preparation to Appraisal 10.0 42.4 424 76.8 181.1 Appraisal 0.0 12.3 -- 12.3 100 Negotiations through 0.0 20.2 -- 20.2 100 Board approval Supervision 20.0 47.6 238 50.3 105.6 Completion 10.0 2.8 28 2.8 100 Total 40.0 125.3 313.3 162.4 129.6 Staff Inputs (sooo) Stage of Project Cycle Planned Revised Revised as % Actual Actual as % ofPlanned ofRevised Preparation to Appraisal 35.6 164.2 461.2 239.7 146.0 Appraisal 0.0 23.6 -- 23.1 97.9 Negotiations through 0.0 39.9 -- 29.8 74.7 Board approval Supervision 57.6 185.2 321.5 214.6 115.9 Completion 34.0 15.0 44.1 15.0 100 Total 127.2 427.9 336.4 522.2 122.0 Source: ICR. 25 Mission Data Date No. ofpersons Days infield Specializations (month/year) Represented Preparation, Appraisal 08/94 7 14 E,F,A,O Supervision 03/95 7 8 E,O 06/95 7 8 0 E = Economist; F = Financial Analyst; A = Agricultural Specialist; 0 = Other (country officer, operations analyst, sector specialist, etc.) Source: ICR and Interviews with project staff Follow-On Operations Operation Amount ($US million) Approval Hydropower Rehabilitation 114 April 1995 Agricultural Seed Development 32 May 1995 Coal Sector Pilot 16 May 1996 Housing Sector 17 May 1996 Enterprise Development Adjustment 310 June 1996 Social Protection Support 3 September 1996 Electricity Market Development 317 October 1996 Agricultural Sector Adjustment 300 October 1996 Export Development 70 November 1996 Coal Sector Development 300 December 1996 Source: ICR 26 Annex B Ukraine Rehabilitation Loan 3831-UA Overview and OED Ratings Loan Amount US$500 million Approval Date December 22, 1994 Closing Date June 30, 1996 Delays None Ratings Outcome * Marginally Satisfactory Sustainability * Uncertain Institutional Development * Modest Bank Performance * Satisfactory Achievements * Substantial improvement in macroeconomic stability * Initiation of economic reform progress * Stronger position for economic reformers * Elimination of some price controls and trade barriers * Initiation of privatization program * Support for energy imports and social safety net spending Issues * Bank reform goals were overly optimistic * Reform progress fell considerably short of what was promised * Progress in many structural reforms was disappointing: trade restrictions, privatization * Little or no progress in enterprise financial discipline or strengthening of social safety net 27 Annex C Comments received from the Export-Import Bank of Japan Comments on the Perfornancv, Audit Rcpurt Ukrain Rubabilitation Loan lovember, 91 The Export-Import Bank of Japan 1. Although we did not join the OED audit mission, we would like to make somr comments not only as a co-Jender of the Rehabilitation Loan but also provider of the technicul asmistance to the State Export-Import Bank of Ukraine with the World Bank's PIIRD fund under Export Development Project. 2. As the OED audit mission pointed out, we are also aware of the successful progress in macrocconomic stabilization under the Lsan. We, a% a co-lender of the Loan, appreciate ihe efforts made by the World Bank, TIMF and Ukrainian authority In macrocconomic stabilization. 3. We agree that the flegativ: side of the evaluation in thLk report. which says that one of main reasons for the shortfall of the Loan is inadequate support to build and strcagthcn institutional capacity. From our expericuce through the above- mentioned technical assistance, we find that the institutional copacity of Ukraine is weaker than expected since the old Sovict rcgim nstill rxist dccply in the system, such as the confuting auwunting practice, untransparent decision making process, strong commitment of the government in the private sector. rnr example, when we discussed the Ukrainian uccounting problem jn banking buNiness %vth the Ukrainian EXIM bank, it took about onc and a half ycbt to reach the samc uderwstanding and the people whev could understand thC problm w4re vCrV liitd. Therefore, we highly recommend the World Bank to continuc to provide c-vices to develop the instittional capacity of Ukraine since it take% quite a long time to strengthen the institutional capacity. 4. We presume that one of the main reasons of setting over optimistic goal of the Loan is caused by the gop existing in Ukrainc, such as the formal sector and the informal setor, the old Sovict sys3tcm and the wcstcern system in the economy. In addition, we would lik% to express the fact obtained from our experience that we need to keep persistent dialogue with Ukrainian people to convince lhemn of thuir ;Xisting problems. Without this understanding, Uluainian peopl can nuver facu the reality of their problems and reach the ideia how to solve the problems by themselves. Als, lack of the realistic view may be a cause of the over optimism for both the 28 Ukrainan authority and the World Bank. Wc hope ihut the World Bank educates the Ukrainian authority persistently to find the reality while the World Bank designs program based on the realistic view concluded by both partic.4. 5. We also agree that Ukr;jinc rrccivzs a lnt nf technical assistancer programs from EU and bilateral donors on a grnnt basis. As a result, such programs are ovcrsupplied compared with the scarce human rN-ourMC in Ukraine and sometimes redundnt. We hope that the World Bank takes a strong leadership among donors to coordinate such programs in ordur to utilize the scarce resource in the rnot effecrive way. 6. In addition, as far as the next dcvclopmnct stage of the mkrainian economy is concerned since we believe that the dcvlopmcrt of the in-itiLutional CapaLity is the most imporan ispect for enhancing the econoiic ritfurmn, we hope that the World Bank should he a key player to achieve this object. Therefore, we hope tha the World Bank provides a comprehensive program to develop the insLitutionul capacity including lending instfuments, feasible technical assistance to sIategic sector so forth. In this regard, we hope that the World Bank designs comprehensivc policy for the strtegic scetoT and coordinates a feasible packaSc for technical assistance program provided by donurs for the dcvclopment of such sector-