Ukraine - Institution Building Loan Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 19303 IMPLEMENTATION COMPLETION REPORT UKRAINE INSTITUTION BUILDING LOAN Loan No. 3614-UA June 3, 1999 Poverty Reduction and Economic Management Unit (ECSPE) Country Unit: Ukraine and Belarus Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENTS 1/ Calendar Exchange Rate Year (end of period) (UAH/US$) 1992 0.0064 1993 0.126 1994 1.042 1995 1.794 1996 1.889 1997 1.889 1998 3.427 1999* 4.410 1/ In September 1996, the official currency of Ukraine was converted from the Karbovonets (Krb) to the Hryvnia (UAH), at a rate of 100,000 to 1. * World Bank and IMF forecasts. WEIGHTS AND MEASURES Metric System ABBREVIATIONS AND ACRONYMS AMC Anti-Monopoly Committee CAS Country Assistance Strategy CEM Country Economic Memorandum COM Cabinet of Ministers EDAL Enterprise Development Adjustment Loan EPS Electronic Payment System Goskomstat State Committee for Statistics ICB International Competitive Bidding ICR Implementation Completion Report ID APL Institutional Development Adaptable Program Loan IFC International Finance Corporation IMF International Monetary Fund LAN local area network MOE Ministry of Economy MOF Ministry of Finance MOS Ministry of Statistics NAUDEI National Agency of Ukraine for Development and European Integration NAURD National Agency of Ukraine for Reconstruction and Development NBU National Bank of Ukraine PIU Project Implementation Unit PTL Program Team Leader SCED State Committee on Entrepreneurship Development SPF State Property Fund STA State Tax Administration STI State Tax Inspectorate TTL Task Team Leader UAH Ukratnian Hryvnia USAID United States Agency for International Development UKRAINIAN FISCAL YEAR January I to December 31 Vice President: Johannes Linn Country Director: Paul Siegelbaum Sector Leader: Hafez Ghanem Teamn Leader: Rama Chandran  FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT UKRAINE INSTITUTION BUILDING LOAN Loan No. 3614-UA TABLE OF CONTENTS PREFACE EVALUATION SUMMARY ..........................................................i I. PROJECT IMPLEMENTATION ASSESSMENT ......................................................... 1 A. Project Objectives ..........................................................1 B. Achievement of Objectives .........................................................2 C. Implementation Record and Major Factors Affecting the Project .................................5 D. Project Sustainability ..........................................................7 E. Bank Performance .........................................................8 F. Borrower Performance .........................................................8 G. Assessment of Outcome .........................................................8 H. Future Operations ........................................................9 I. Key Lessons Learned ................................................9 II. STATISTICAL TABLES Table 1: Summary of Assessments ..11 Table 2: Related Bank Loans ..12 Table 3: Project Timetable ..12 Table 4: Loan Disbursements - Cumulative Estimated and Actual .13 Table 5: Key Indicators for Project Implementation . .13 Table 6: Key Indicators for Project Operation ..13 Table 7: Studies Included in Project ..13 Table 8A: Project Costs ............................................... 14 Table 8B: Project Financing ............................................... 14 Table 9: Economic Costs and Benefits ................................................ 14 Table 10: Status of Legal Covenants ............................................... 15 Table 11: Compliance with Operational Manual Statements . ............................ 15 Table 12: Bank Resources -Staff Inputs ................................. 15 Table 13: Bank Resources -- Missions .. .............................. 16 III. APPENDICES 1. Aide Memoire of Final Mission ................................ 17 2. Borrower's Contribution to the ICR ................................ 23 3. Map IBRD 27828R ................................ 45 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  IMPLEMENTATION COMPLETION REPORT UKRAINE INSTITUTION BUILDING LOAN Loan No. 3614-UA PREFACE This is the Implementation Completion Report (ICR) for the Institution Building Loan (IBL) to Ukraine, for which loan number 3614-UA in the amount of US$27 million was approved on June 8, 1993, and made effective on August 11, 1993. This ICR evaluates the implementation experience of this loan, the World Bank's first to Ukraine. Loan closing was postponed three times before it officially closed on April 1, 1999, three years and three months after the original closing date of December 1, 1996. Final disbursement took place on April 20, 1999, at which time a balance of US$68,817.23 was canceled. Cofinancing in the amount of $111,000 was provided by UNDP. Preparation of the ICR, begun during the final supervision mission from October 12- 23, 1999, was managed by Rama Chandran, Task Team Leader (TTL), with inputs from Maxim Ljubinsky (Project Officer in Kiev), Ali Hashim (Treasury component), Sandra Bloemenkamp (Task Manager 1993-95) and Lilia Burunciuc (Task Manager 1996-97). The report utilizes materials from the project files and interviews with the project management team. The borrower contributed to the preparation of the ICR by submitting project evaluations from the Anti-Monopoly Committee, the National Agency of Ukraine for Development and European Integration, the National Bank of Ukraine, and the State Committee on Statistics (attached as appendices). The ICR benefited from valuable comments provided by Dusan Vujovic (Program Team Leader), Lilia Burunciuc (Ukraine, Country Unit), and Paul Siegelbaum (Country Director, Ukraine and Belarus). The ICR was reviewed by Pervaiz Rashid, Lead Specialist at ECSPE.  IMPLEMENTATION COMPLETION REPORT UKRAINE INSTITUTION BUILDING LOAN Loan No. 3614-UA EVALUATION SUMMARY Introduction 1. Formal contacts between Ukraine and the World Bank was initiated during a Bank mission in January 1992, a few months after Ukraine's declaration of independence from the former Soviet Union. In September 1992, Ukraine joined the IBRD. The Institution Building Loan (IBL), approved in June 1993, was the Bank's first lending operation in that country. There was no Country Assistance Strategy (CAS) at the time of IBL's approval, but a country economic memorandum (Report No. 11 029-UA) was distributed to the Board a week prior to consideration of the IBL. Project Objectives 2. The IBL was designed to assist the Government to implement economic reforms and to strengthen the institutions essential to the transition to a market economy. 3. The project consisted of three components, Enterprise Sector Reform, Financial Sector Reform, and Public Economic and Financial Management Reform. The original programs were focussed on technical assistance for the implementation of reforms which would simultaneously build capacity to manage the reforms essential for efficient operation of a market economy through institutional building programs for the following areas: * Enterprise Sector Reform: (a) implementation of small enterprise privatization and regulatory reform in selected cities; (b) pilot privatization of medium- and large-scale enterprises; (c) national publicity campaign about privatization; and (d) initiation of a demonopolization program. * Financial Sector Reform: (a) modernization of the payments system; (b) accounting reform for financial institutions; and (c) institutional strengthening of banks. * Public Economic and Financial Management Reform: (a) establishment of a treasury system; (b) modernization of tax administration; (c) modernization of the statistical system; (d) strengthening economic analysis capacity of the Ministry of Economy; and (e) support to a project implementation unit for the IBL.  - ii - 4. The project objectives were clear, though somewhat ambitious. Selection of these components was appropriate at the time of the design of the project in view of the urgent need to build capacity in these sectors. 5. The IBL was launched in a very challenging environment given the fact that the political consensus on economic reformn within the country needed to be developed. At the same time, the project objectives were broad and flexible enough to respond to Ukraine's evolving priorities. Consequently, the project was restructured in 1995-96 within the framework of the original objectives to reflect the Government's revised needs and priorities. Following the restructuring in 1995-96, the focus of investments shifted to building up the individual institutions through the provision of training, and technical assi stance, which aimed at the development of information technology systems as well as building capacity to manage economic reforms. The revised components consisted of the following: * Enterprise Sector Reform: (a) development of a computerization strategy for the State Property Fund (SPF); (b) training for the SPF; (c) pilot demonopolization and development of a demonopolization strategy; (d) training for the Anti-Monopoly Committee (AMC); and (e) computerization of the AMC. o Financial Sector Reform: modernization of the payments system. * Public Economic and Financial ManagementReform: (a) establishment of a treasury system; (b) pilot modernization of tax administration; (c) computerization of the Ministry of Statistics (MOS) and training for MOS staff; (d) development of a computerization strategy for the Ministry of Economy (MOE) and training for MOE staff; (e) capacity-building of the National agency of Ukraine for Development and European Integration (NAUDEI) and Cabinet of Ministers to manage external financial resources; (f) computerization of Ministry of Finance (MOF) and training for MOF staff; (g) training of the SCED; and (h) support to a project implementation unit for the project. Implementation Experience and Results 6. The implementation of the IBL faced initial delays which were largely due to weak institutions, lack of consensus on policy approaches and the country's inexperience in managing Bank-financed projects. The conditions for project implementation improved by 1995 as the Government had adopted a comprehensive stabilization and economic reform program at the end of 1994 and over time the executing agencies had increased their capacity to implement the IBL. 7. In general, the IBL successfully initiated the strengthening of institutions essential to the efficient operation of a market economy. All activities, with the exception of the computerization of the SPF, were implemented. Additional investments would be needed in these areas in order to sustain the momentum for institutional development. The planned Bank investments (e.g., Treasury Systems Project, ID APL for Goskomstat and STA) and initiatives by individual agencies, are likely to maintain this momentum. A  - iii - key accomplishment of the project was specifically in changing the way these agencies think and do business, and this process is likely to be sustained as well. Overall, both the Borrower and the Bank performed satisfactorily. The IBL has been rated "Satisfactory" on both Development Objectives and Institutional Progress. 8. The outcome of the project was satisfactory due to the positive overall impact of the project, which goes well beyond the individual components. All the agencies involved in the IBL have changed the way they do business, train staff, build capacity to design and implement policies, and reinvent their own organizations. The Bank has learned a great deal about Ukraine, as has Ukraine about the Bank, allowing it to prepare follow-up institution-building and other operations that would build on the achievements and lessons learnt from the project. Summary of Findings, Future Operations, and Key Lessons Learned 9. Although the implementation of the IBL got off to a slow start and the loan closing date was extended three times, it had extremely positive outcomes. The IBL has helped to establish a broad constituency for economic reform, build up significant implementation capacity within the Government, and demonstrate the importance of institution building using a flexible approach. In retrospect, the original programs in the IBL may have been overly ambitious. However, the limited and focussed programs after its restructuring played an important role in engaging the Borrower. 10. Government ownership ofthe project is an important factor in institutional development projects. The IBL ambitiously set out to implement extensive reforms in enterprise, financial, and the public sectors. The mix of two objectives--implementing economic reforms and building institutions--and the absence of a clear consensus for economic reform within the government hampered the implementation during the initial two years or so. Implementation was slow and disbursement levels were at a low (14 percent of loan disbursed by end FY 1996) until the Government adopted a comprehensive structural reform program and restructured the project. These actions enhanced the Government's commitment to the project. 11. Because of the lack of experience with reforms and Bank projects, the Borrower was overwhelmed with the breadth, comprehensiveness and complexity of the project. The more focussed and limited programs included in the IBL following its restructuring serves as an important lesson for institutional development. Limited and focussed programs, particularly pilot programs, are effective tools to launch reforms in politically sensitive areas. Through such focussed programs, the IBL has created a core of mid-level public officials committed to reform. 12. Intensive supervision by experienced Bank staff is a key factor for IBL projects in new member countries. This was evidenced during the latter stages of the project. Close monitoring and supervision by the Bank's local staff was also a factor in achieving implementation progress. Flexibility in project design, and redesign during implementation through cancellation of components that were moribund and allocation of resources to agencies which were proactive helped to maximize the project's positive impact in an evolving country environment.  - iv - 13. The relatively short shelf life of information technology and other computer systems and the relatively long time required to complete the bidding process was frustrating to the borrower. The delay in procurement was because of the long learning curve in comprehending and internalizing transparent and open bidding procedures. This was a source of significant dissatisfaction with many of the agencies under the IBL. 14. The deep-seated cultural and political bias in Ukraine against foreign consultants presents a significant issue in all Bank-financed projects. Although this bias has been partly overcome through implementation of the project, there is a fundamental distrust of foreign advice and its value. This often leads to contractual disputes and a slowing down in project implementation. 15. Ideally, direct assistance to and communication with the beneficiary, should be the preferred choice for effective project implementation. This was not possible during the implementation of the IBL because of lack of capacity in the various institutions of the project. Consequently, a centralized Project Implementation Unit (PIU) at the Ministry of Economy was chosen to coordinate the implementation. This arrangement slowed down implementation due to poor communication between the PIU and the beneficiary agencies in the IBL.  IMPLEMENTATION COMPLETION REPORT UKRAINE INSTITUTION BUILDING LOAN Loan No. 3614-UA I. IMPLEMENTATION ASSESSMENT A. Project Objectives 1. In 1993, the World Bank approved and began to implement its first lending operation in Ukraine based on the Government's support of transition to a market-based economy. This first lending operation, the Institution Building Loan (IBL), would provide critically-needed technical assistance to the Government of Ukraine to erect the basic institutions of a market economy while simultaneously building the capacity of the Government to manage the transition. By a decision of the Hard Currency Credit Council in the Cabinet of Ministers, the Government certified its desire to borrow from the Bank for the objectives of the IBL: to support the implementation of economic reforms and to strengthen institutions essential to the efficient operation of a market economy. A Country Assistance Strategy (CAS) for Ukraine had not been prepared by the time of the IBL's approval in June 1993. However, Ukraine: Country Economic Memorandum (Report No. 11 029-UA) was distributed one week prior to the presentation of IBL to the Board. 2. The IBL addressed one of the major objectives of the Bank's strategy at that time, the initiation of systemic reforms, which was reflected in the project's three components: Enterprise Sector Reform, Financial Sector Reform, and Reform of Public Economic and Financial Management. The original programs were focussed on technical assistance for the implementation of reforms that would simultaneously build capacity to manage the reforms within the following institutions. * Enterprise Sector Reform: (a) implementation of small enterprise privatization and regulatory reform in selected cities; (b) pilot privatization of medium- and large-scale enterprises; (c) national publicity campaign about privatization; and (d) initiation of a demonopolization program. * Financial Sector Reform: (a) modernization of the payments system; (b) accounting reform for financial institutions; and (c) institutional strengthening of banks. * Public Economic and Financial Management Reform: (a) establishment of a treasury system; (b) modernization of tax administration; (c) modernization of the statistical system; (d) strengthening economic analysis capacity of the Ministry of Economy; and (e) support to a project implementation unit for the IBL.  3. The IBL was a demanding project for the Government to implement since it was Ukraine's first experience with a Bank-financed project. The volatile policy environment in which the project had its beginnings and the lack of internal consensus for economic reforms within the Government complicated matters. Further adding to the complexity, was that there were eight executing agencies for the various components in addition to several layers of beneficiaries. Implementation required coordination across many Government departments and agencies, and superior management. The lack of implementation capacity also contributed to slow implementation. B. Achievement of Project Objectives 4. Overall, the IBL was moderately successful in achieving both its objectives of assisting the Government of Ukraine to implement economic reforms and of strengthening institutions essential to the transition to a market economy. IBL's specific achievements are detailed in this section. Enterprise Reform Component 5. Privatization. The Government established the SPF in the fall of 1991 when they also adopted a number of fundamental laws that formed the legal foundations for privatization in Ukraine. Prior to its restructuring, the IBL included three distinct privatization programs that aimed simultaneously at implementing economic reforms and building the technical capacity of the SPF to manage the privatization process. They were: small enterprise privatization and regulatory reform in selected cities and oblasts, pilot privatization of medium and large enterprises, and a national public information campaign on privatization. 6. In 1995, all three of these programs were cut out of the IBL given either alternative sources of financing (for the national public information campaign and small-scale privatization) or changed priorities of the Government (such as the initiation of a mass privatization program). Given these developments, IBL assistance was refocused to provide direct institution-building support to the SPF, through which the SPF received financing for office equipment, training in privatization, and a computerization strategy. The computerization strategy, developed by Consultants, was not implemented due to the SPF's later dissatisfaction with the specifications of the bidding documents. The SPF asserted that the specifications no longer met their needs. However, the Bank did not agree with this conclusion, and the SPF ultimately cancelled the tender. 7. Despite the failure to implement the computerization strategy and the cancellation of the original privatization programs in 1995, this sub-component achieved considerable progress in strengthening the SPF as an institution and in implementing reforms. The IBL played a significant role in helping to launch the privatization program in Ukraine and built a foundation for other Bank assistance. For example, the Enterprise Development Adjustment Loans (I and II), approved in FY1996 and FY1998 respectively, deepened support to Ukraine in privatization and post-privatization. The original activities in this sub-component may have been ambitious given the absence of a consensus for reform. This experience demonstrated important lessons about Ukraine for the Bank. For example, the deep-seated reluctance to borrow for technical assistance; the  - 3 - difficulty in implementing reform programs without a clear mandate from the government for reform; and the potential complications in financing information technology systems given the relatively short shelf life of information technology products and the long lead time in procurement. 8. Demonopolization. The IBL's demonopolization efforts centered around strengthening the Anti Monopoly Commission (AMC), which was created in September 1992, providing assistance in initiating a program to break up some of the largest monopoly enterprises in Ukraine, and preventing practices aimed at restricting competition. Under IBL, the central office staff of the AMC received training and technical assistance to demonopolize a large-scale breadmaking enterprise, based on which a generally-applicable demonopolization methodology was developed. Although the breadmaking enterprise, Volin Hlyb, was ultimately demonopolized, political opposition from vested interests had slowed down the implementation of the program and led to the extension of the consulting services contract. In addition to technical assistance and training, AMC was computerized, which increased the efficiency of AMC operations, improved record management and bookkeeping, and accelerated case and data processing. The IBL's demonopolization program made substantial progress in initiating a demonopolization program as well as in strengthening the AMC. Financial Sector Reform 9. The Financial Sector Reform component originally set out to introduce major reforms in the sector through modernizing the payments system, overhauling the accounting system of financial institutions, and overall institutional strengthening of commercial banks. However, this component was eliminated from the IBL in early 1996 based on the lack of implementation progress and the Government's perception that the National Bank of Ukraine (NBU) could finance the modernization of the payment system from its own resources. However, the Government reversed its position on the latter and requested the reintroduction of the original program to modernize the payments system. With technical assistance and equipment provided by IBL, the NBU enhanced an electronic payment system (EPS), upon which the NBU has continued to build. This subcomponent successfully achieved its objectives, further building the institutional capacity of the NBU and at the same time establishing a market-oriented payments system. Reform of Public Economic and Financial Management 10. Treasury System. The IBL originally intended to finance technical assistance for the functional analysis of a treasury system, in addition to hardware and software to establish an interim system. Consultants completed the functional analysis and issued bid documents for the procurement of hardware and software, however, a newly appointed head of the treasury department wanted to modify the bid documents to include hardware only. The software would be developed in-house. This decision led to an impasse with the Consultants and resulted in the ultimate cancellation of the bid documents, the contract with the Consultants, and the sub-component in 1996.  - 4 - 11. In its first years, the treasury department was a growing but weak institution which was severely understaffed, suffered from frequent changes in management, and had limited institutional capacity, particularly to fully value the work of the consultants. These institutional weaknesses were eventually overcome, however, and although the sub-component suffered a setback following the contract cancellation, the treasury continued to develop the system using its own resources. In 1997, IBL resumed support to the treasury department with additional procurements of hardware and technical assistance to elaborate drafts of legislative and normative acts as well as to analyze the treasury department's functional and organization structure. There was also significant collaboration with the IMF, the Bank and the Treasury. The investments into the Treasury department established the basis for the Treasury Systems Project, which was approved by the Bank in 1998. In addition, equipment was provided to the MOF for a network to link several departments in the MOF with the treasury department. IBL thus achieved significant progress in introducing and initiating a functional and automated treasury system in Ukraine. 12. Tax Administration. The State Tax Administration (STA) implemented the roll- out of an IMF-financed pilot in 14 tax offices throughout Kiev using IBL-financed technical assistance and equipment. The original pilot upgraded and modernized the operations of the local tax office in Darnitsa rayon, city of Kyiv. Under the pilot, the administrative structure of the office was reformed, new procedures for assessment, collection enforcement and auditing introduced, and information technology integrated into its operations. Although the pilot is unlikely to be replicated and the hardware configured and software developed under the project, its experience established an important foundation for future investments into modernizing and improving itself is unlikely to be replicated beyond IBL, its impact goes well beyond the pilot. The STA improved its administrative and management skills base with the introduction of new processes, such as analyzing and re-engineering its business procedures in administering taxes, designing functional and structural improvements, optimizing human resources, assessing risk, contingency planning and using international advisory services. Although the pilot is unlikely to be replicated and the hardware configured and software developed under the project, its experience established an important foundation for future investments into modernizing and improving tax administration in Ukraine. The STA has begun the preparation of programs for potential financing under the Institution Development Adaptable Program Lending (ID APL). This sub-component achieved significant progress in building the STA's capacity even though it was only partially successful in achieving the objective of fundamentally transforming tax administration in Ukraine. There was also some collaboration with the IMF during the earlier stages of the IBL. This tapered off towards the latter stages of the project. 13. The System of Statistics. Under IBL, the State Committee for Statistics (or Goskomstat, formerly the Ministry of Statistics) received technical assistance, equipment, and computer training to elaborate and implement a new information system to modernize its statistical system, in addition to receiving equipment to improve the telephone system and the printing facilities. Building on the foundations of reform established by IBL and in accordance with a 1997 Presidential decree, implementation of a National Program on International Standards of Statistics began in 1998 which would begin the process of converting Ukraine's statistical system to international standards.  Moreover, Goskomstat is currently preparing future programs for potential financing under ID APL. This sub-component was partially successful in introducing institutional changes to and restructuring the statistical system in Ukraine. 14. Economic Analysis and Policy Formulation. Since the Ministry of Economy (MOE) was given a lead role in spearheading the country's economic reform effort, the IBL was designed to provide technical assistance and training to aid this process. Although MOE did not receive technical assistance for economic policy formulation, it did benefit from an extensive staff training program in economic analysis and policy formulation, in addition to the implementation of an integrated computer system. However, the success of the training program has been reduced by a high turnover of IBL-trained staff: by 1998, about 30 percent of those trained under IBL had left the MOE. Thus, this sub-component was only partially successful in achieving its goal of building capacity in economic analysis and policy fortnulation within MOE. 15. External Financial Resource Management. This sub-component, implemented jointly with UNDP, provided technical assistance, training, and equipment, separately to the Cabinet of Ministers (COM) and the National Agency for Development and European Integration (NAUDEI). Both entities have benefited from training as well as the computerization of their operations. Internal sources within the Bank report an improved dialogue with NAUDEI. This sub-component was successful in strengthening the capacity of COM and NAUDEI to manage external resources. 16. Project Implementation Unit. Technical assistance, training, and equipment were allocated in the IBL to coordinate project implementation as well as to build capacity in the Cabinet of Ministers to and later, the Ministry of Economy, to facilitate future World Bank lending. The performance of the PIU was mixed, particularly given the difficulty in maintaining clear lines of communication with the IBL executing agencies, which mitigated against timely and effective project implementation. Overall, this sub- component achieved some progress in meeting its objectives, particularly in building capacity for future Bank lending as demonstrated by the contribution of former IBL PIU staff to other Bank projects. C. Implementation Record and Major Factors Affecting the Project 17. Between project effectiveness in August 1993 and the initiation of a comprehensive stabilization and structural reform program by the Government at the end of 1994, there was very little implementation progress in IBL (little over 10 percent of the loan was disbursed against 20 percent of estimated disbursements). Until this time, the IBL had been used mainly to facilitate the policy dialogue with mid-level government officials, and overall, there was neither the political commitment nor will to implement reforms. Following the introduction of the Government's reform program at the end of 1994, however, the Bank's dialogue with its IBL counterparts dramatically improved and facilitated an effective process in project restructuring. There were many factors not subject to Government control which affected project implementation, most of which were related to their limited experience in implementing reforms.  - 6 - Factors Not Subject to Government's Control 18. Upheaval of the Transition Period. Project implementation was complicated by the rapidly changing social, political, and economic landscape in Ukraine in the first few years of its transition, particularly as many of the original activities became less relevant during implementation, leading to the cancellation or addition of some activities and the restructuring of others. 19. Reorientation to Independent Nation. The psychological effect stemming from Ukraine's newly gained status as an independent nation cannot be underestimated as a factor in implementation delays, particularly given the nature of its previous status under the former system. Ukraine, as with all the former republics of the Soviet Union, used to be rewarded for executing orders from the center, not by crafting its own policies. The newly-gained freedom to govern itself, as well as the reorientation in thinking, although heady at times, was very demanding at other times. 20. Weak Implementation Capacity. IBL, as the first Bank operation in Ukraine, was the Government's first meaningful experience with the World Bank. This was a significant factor contributing to implementation delays not only because of the Ukrainians' nescience vis-a-vis Bank's procedures (particularly procurement), but also in terms of its lack of experience in communicating and working with the Bank as an institution. 21. Inexperience with Competitive Procurement and Contract Management. IBL was the government's first exposure to the concept of competitive procurement and contract management. Given the radically different concepts behind selecting and managing contractors and suppliers in the past, learning the process of open, transparent and cost effective public procurement was a long exercise which will likely continue with other Bank projects. 22. Multiple Changes in Project Management. During the implementation of the IBL, both Bank management and the heads of some of the executing agencies changed several times. This was also a contributing factor in slowing down implementation. 23. Ambitious Project Design. Although there were only three components under the project, each component contained within it multiple activities involving far-reaching policy reforms. Given that there was no institutional experience with Bank projects, the complex project design significantly contributed to delays in jump-starting implementation. 24. Competing Assistancefrom other Donors. Some of the activities under the loan were delayed and ultimately abandoned after other donors provided grant financing in the same area. Factors Generally Subject to Government's Control 25. Unfocused Policy Environment. The unfocused policy environment in Ukraine adversely effected the implementation of the majority of IBL's programs until the end of  1994, when the Government adopted a comprehensive structural and economic reform program. The unfocused policy environment translated into unclear commitment to the project in the first years of IBL implementation. 26. Inefficient Project Management Structure. During project design, the First Deputy Minister of Economy was delegated overall responsibility for project implementation, with responsibility of day-to-day project administration given to the PIU, which was originally established within Cabinet of Ministers. Under this organizational structure, however, the PIU lacked the authority to manage the project because it had to receive approvals from the Minister of Economy for standard documentation. Until the PIU was transferred to the Ministry of Economy in 1995, this arrangement slowed down project implementation. Factors Subject to Implementing Agencies' Control 27. Lack of Interagency Collaboration and Cooperation. The project provided direct assistance to ten government entities, in addition to providing indirect assistance to many others. A coordination council was created to take executive decisions on the loan. Given this complex landscape of project beneficiaries, diligent attempts at interagency collaboration would have been necessary to ensure smooth implementation. The problem was particularly pronounced vis-a-vis communication between the PIU and the beneficiary entities, and even despite the efforts of the Bank's resident mission to help in coordination. D. Project Sustainability 28. Since the IBL provided only the basic initial investment, for the organizations included in the project, additional investments are needed to maintain the momentum of institution building. This process has already begun with other Bank loans (e.g., privatization, treasury system, tax administration, statistical system, etc.) or with the organization's own initiative (e.g., payments system). Furthermore, given the fundamental change the IBL-supported agencies have undergone in the way they think and do business, the IBL investments are likely to be sustained. 29. Enterprise Reform Component. The project's support to the SPF is being supplemented by EDAL I and EDAL II and thus is likely to be sustained. The sustainability of investments into the AMC, however, is uncertain given the political opposition to demonopolization. However, in view of the AMC's demonstrated commitment to public education in demonopolization, if the resources became available, the AMC could make significant progress in this area and sustain IBL investments. 30. Financial Sector Reform. Given the momentum generated by the IBL in enhancing the NBU's payments system, this sub-component is highly likely to be sustained. 31. Public Sector Economic and Financial Management. The investments into the treasury department, STA, Goskomstat, MOE, and NAUDEI. Additionally, the investments into the PIU are also likely to be sustained given that IBL PIU staff have  -8 - already contributed to the preparation of other Bank-financed activities, such as the Treasury Systems Project. E. Bank Performance 32. The Bank management team performed well in sustaining the IBL given the turbulent and uncertain political setting under which the project was prepared and implemented. Although not all the programs prepared under the IBL were ultimately implemented, the fact that some of these prepared prograrns were either financed by other donors or included as part of other Bank loans is a statement in praise of the Bank preparation team. This is remarkable since the IBL was prepared in a short time and in the same year, in which the Bank sent its first mission to Ukraine (January 1992). 33. The supervision effort of the IBL also deserves recognition. Implementation assistance for the IBL, particularly in the early transition period, was extremely difficult given the turbulent reform environment and the inexperience of the Borrower not only with the Bank, but with standard business practices of the commercial world. The patience and time devoted to supervision, particularly after the restructuring was completed in 1996, is commendable. The active participation of a project officer in the Kiev Resident Mission also significantly contributed to the quality of Bank supervision. 34. One factor that negates to some extent an otherwise estimable supervision effort was the fact that project management changed five times during the life of IBL. This was a contributing factor to delays in decision making and responsiveness to the Client's requests. F. Borrower Performance 35. Borrower performance was varied, depending on the capacity of various executing agencies. In general the executing agencies performed well, given their lack of experience with external assistance, reform, and Bank projects. There was a very steep leaming curve during the initial period of two years in trying to understand the nuances of a market economy, the requirements of transparency in public procurement, building of internal consensus on reform etc.,. The learning curve began to flatten during the later stages of the project and this was demonstrated by improved performance, more in some agencies and less in others. G. Assessment of Outcome 36. Although progress in meeting the development objectives of the project were moderate, the outcome of the project was extremely positive in that the IBL built a broad constituency for reforms as well as developing strong working relations with the executing agencies and institutional capacity to prepare and implement Bank-financed projects. More importantly, the overall positive impact of the project goes well beyond the individual components. All the agencies involved in the IBL have changed the way they do business, train staff, build capacity to design and implement policies, reorganize and reinvent their own organizations. Additionally, the Bank has learned a great deal  - 9- about Ukraine, as has Ukraine about the Bank, allowing it to prepare its own institution- building projects for consideration under the ID APL. H. Future Operation 37. Both Enterprise Development Adjustment loans (EDAL and EDAL II) which are currently under implementation are providing technical assistance to the SPF to build on the initial training they received on privatization under the IBL; in particular, EDAL and EDAL II address case-by-case privatization (CBC), foreign investment, and post- privatization restructuring. 38. Although it has not secured it to date, the AMC is actively seeking funding to disseminate information to the public at large about the benefits of demonopolization, with a focus on development of curricula on demonopolization in schools and universities. The AMC has received some technical assistance from TACIS, but would require additional funding in order to accomplish its objectives of educating the public about demonopolization. In spite of its desire, the political environment in Ukraine is not prepared for full-scale implementation of the demonopolization program developed under the IBL. 39. The NBU has already initiated various efforts in building on the EPS developed under the IBL, and have several programs currently under development, including increasing the capacity of EPS. In this regard, the NBU is seeking assistance in software development. In addition to its internal activities, the NBU held a conference on its EPS, one of the results of which may be the provision of assistance to Armenia, Azerbaijan, and Kazakhstan in the future for establishing EPS's in those countries. 40. Although the Treasury Systems Project, which would build on the initial investments made by IBL, was approved by the Bank in February 1998, it has not been declared effective since it has not been approved by the Ukrainian Parliament despite two readings. Given the strong political opposition to the project, it is unclear when the project will become effective. 41. Programs for Ukraine are under preparation at the time of this ICR. Two areas already identified as priorities were tax administration and statistical systems modernization based on the outcome of the IBL. The IBL played -a critical role in preparing both the STA and Goskomstat for future lending, and for helping both the Bank and the Borrower identify future investment needs. I. Key Lessons Learned 42. Although the implementation of the IBL got off to a slow start and the loan closing date was extended three time thereby increasing the implementation period, the overall outcomes have been positive. The IBL has helped to establish a broad constituency for economic reform, build up significant implementation capacity within the Government, and demonstrate the importance of institution building using a flexible approach. In retrospect, the original programs in the IBL may have been overly ambitious. However,  -10- the limited and focussed programs after its restructuring played an important role in engaging the Borrower. 43. Government ownership of the project is an important factor in institutional development projects. The IBL ambitiously set out to implement extensive reforms in enterprise, financial, and the public sectors. The mix of two objectives--implementing economic reforms and building institutions--and the absence of a clear consensus for economic reform within the government hampered the implementation during the initial two years or so. Implementation was slow and disbursement levels were at a low (14 percent of loan disbursed by end FY 1996) until the Govermnent adopted a comprehensive structural reform program and restructured the project. These actions enhanced the Government's commitment to the project. 44. Because of the lack of experience with reforms on the one hand and experience with Bank projects on the other, the Borrower was overwhelmed with the breadth, comprehensiveness and complexity of the project. The more focussed and limited programs included in the IBL following its restructuring serves as an important lesson for institutional development. Limited and focussed programs, particularly pilot programs, are effective tools to launch reforms in politically sensitive areas. Through such focussed programs, the IBL has created a core of mid-level public officials committed to reform. 45. Intensive supervision by experienced Bank staff is a key factor for IBL projects in new member countries. This was evidenced during the latter stages of the project. Close monitoring and supervision by the Bank's local staff was also a factor in achieving implementation progress. Flexibility in project design, and redesign during implementation through cancellation of components that were moribund and allocation of resources to agencies which were proactive helped to maximize the project's positive impact in an evolving country environment. 46. The relatively short shelf life of information technology and other computer systems and the relatively long time required to complete the bidding process was frustrating to the borrower. The delay in procurement was because of the long learning curve in comprehending and internalizing transparent and open bidding procedures. This was a source of significant dissatisfaction with many of the agencies under the IBL. 47. The deep-seated cultural and political bias in Ukraine against foreign consultants presents a significant issue in all Bank-financed projects. Although this bias has been partly overcome through implementation of the project, there is a fundamental distrust of foreign advice and its value. This often leads to contractual disputes and a slowing down in project implementation. 48. Ideally, direct assistance to and communication with the beneficiary, should be the preferred choice for effective project implementation. This was not possible during the implementation of the IBL because of lack of capacity in the various institutions of the project. Consequently, a centralized PIU at the Ministry of Economy (MOE) was chosen to coordinate the implementation. This arrangement slowed down implementation due to poor communication between the PIU and the beneficiary agencies in the IBL.  II. Statistical Tables Tables 1-13 Table 1: Summary of Assessments A. Achievement of Objectives Substantial Partial Negligible Not Applicable (x) (x) (x) (x) MACRO POLICIES O O x SECTOR POLICIES O O O x FINANCIAL OBJECTIVES O O O x INSTITUTIONAL DEVELOPMENT O x O O PHYSICAL OBJECTIVES X O O O POVERTY REDUCTION O O O x GENDER ISSUES O O O x OTHER SOCLAL OBJECTIVES O5 O1 x ENVIRONMENTAL OBJECTIVES O