China - First and Second Technical Cooperation Projects and the Planning Support and Special Studies Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 16803 PERFORMANCE AUDIT REPORT PEOPLES REPUBLIC OF CHINA TECHNICAL COOPERATION CREDIT (CREDIT 1412-CHA) SECOND TECHNICAL COOPERATION PROJECT (CREDIT 1664-CHA) AND PLANNING SUPPORT AND SPECIAL STUDIES PROJECT (CREDIT 1835-CHA) June 25, 1997 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents (annual averages) Currency Name = Renminbi Currency Unit = Yuan (9 1987 US$1.00 Y 3.73 1992 US$1.00 Y 5.52 1988 US$1.00 Y 3.73 1993 US$1.00 Y 5.76 1989 US$1.00 Y 3.78 1994 US$1.00 Y 8.62 1990 US$1.00 Y 4.78 1995 US$1.00 Y 8.35 1991 US$1.00 Y 5.32 Abbreviations and Acronyms DCA Development Credit Agreement GOC Government of China EDI Economic Development Institute ESW Economic and Sector Work ICB International Competitive Bidding ICR Implementation Completion Report ID Institutional Development IDA International Development Association MOF Ministry of Finance OED Operations Evaluation Department PAR Performance Audit Report PCR Project Completion Report PSSSP Planning Support and Special Studies Project SPC State Planning Commission TA Technical Assistance TATD Technical Assistance and Training Department in WBD in MOF TC I Technical Cooperation Credit TC II Second Technical Cooperation Credit UNDP United Nations Development Programme WBD World Bank Department in MOF Fiscal Year Government: January 1 - December 31 Director-General, Operations Evaluation: Mr. Robert Picciotto Director, Operations Evaluation Dept. : Mr. Roger Slade for Ms. Elizabeth McAllister Acting Division Chief : Mr. Ren6 Vandendries Task Manager : Ms. Laurie Effron FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. OWoe of the Director-Generat Operations Evaluation June 25, 1997 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on China - Technical Cooperation Credit (Credit 1412- CHA), Second Technical Cooperation Credit (Credit 1664-CHA), and Planning Support and Special Studies Project (Credit 1835-CHIA) Attached is the Performance Audit Report prepared by the Operations Evaluation Department on the China Technical Cooperation Credit (TC I, Credit 1412-CHA for US$10 million equivalent approved in FY83), the Second Technical Cooperation Credit (TC II, Credit 1664-CHA, for US$20 million equivalent, approved in FY86), and the Planning Support and Special Studies Project (PSSSP, Credit 1835-CHA, for US$20.7 million equivalent, approved in FY87). TC I was fully disbursed and closed one year late, on December 31, 1988. TC II was closed on June 30, 1994 after two years' extension and US$0.7 million equivalent was canceled. PSSSP closed two years late on December 31, 1994 and US$2.3 million equivalent was canceled. The main objective of TC I and TC II was to prepare projects for Bank Group financing. A secondary objective was to help build capacity for selecting, preparing, and implementing projects. The main objectives of PSSSP were to: (a) strengthen the Government of China's (GOC's) planning capabilities through carrying out sector studies; (b) provide assistance for intersectoral, innovative, or experimental studies; and (c) facilitate implementation of GOC's program of economic, financial, and institutional reforms. All three projects provided financing for external consultants, study tours abroad, short and long-term training abroad, in-service training, and conferences and seminars. The projects met most of their objectives. Under TC I and TC I, 48 projects were prepared and subsequently financed by the Bank Group for a total of US$7 billion of commitments. The quality of the projects, both completed and on-going, is considered high, with 96 percent of the ratings listed as satisfactory or highly satisfactory. In addition, considerable institutional development occurred in many agencies, mainly through on-the-job training and local seminars and conferences. Under PSSSP, the nineteen sub-projects had, overall, considerable impact on capacity building across diverse agencies, including areas such as auditing, national income accounting, macro-modeling, and sectoral planning for water resource management and electric power development, among others. PSSSP suffered, however, from an absence of clear criteria for sub-project selection, with the result that some of the sub-projects were of arguable priority. In addition, the Ministry of Finance's (MOF's) policy of on-lending funds for technical assistance discouraged some prospective clients and had a distortionary effect on the selection of benefiting agencies in favor of those with repayment capacity. The PAR rates the projects' outcomes as satisfactory, institutional development as substantial, sustainability as likely, and Bank performance as satisfactory. These ratings agree with those of the PCR and ICRs for the three projects. The PAR judged sustainability as likely because staff turnover in China is relatively low and, more important, the implementing agencies retained responsibility for the This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 objectives of their sub-projects, most agencies are continuing to build on or to use the tools and concepts obtained under the three projects. The main lessons from the three projects are: (a) the most important factors determining the success and sustainability of technical assistance (TA) projects are client motivation to obtain know-how and discipline to minimize rent seeking. These should be assessed carefully before proceeding with TA projects; (b) TA priorities should be geared to the most binding constraints to institutional development; (c) with the exception of TA projects for project preparation, umbrella TA projects covering many sectors and/or agencies should be avoided so as to ensure adequate supervision and quality control both within the Bank and the client country; and (d) TA funds should be allocated to where its rate of return is highest, not on the basis of an implementing agency's ability to repay. Contents Preface 3 Ratings and Responsibilities 5 Evaluation Summary 7 1. Introduction and Overview 13 Why Audit these Products? 13 How this Audit Differs from the PCR/ICRs 13 Main questions to be Addressed in this Report 13 Some Historical Context: China's Opening 14 Evolution of the Bank's approach to TA 14 2. Objectives & Design Issues 17 Objectives of TC I, TC II, and PSSSP 17 Were these Ojectives the Right Ones 17 Was the Design Appropriate? 18 Evaluation of Quality at Entry 21 3. Implementation and Outcomes 23 Caveats about Evaluating Outcomes 23 Outcomes by Main Objectives 23 TC I and TC II 23 PSSSP 25 Outcome by TA Instruments 26 Sustainability 27 Borrower Performance 29 Bank Performance 29 Key Factors Affecting Implementation 31 Conclusions 32 4. Selected Issues 35 When is a process approach for TA warranted 35 The Need for Defining Priorities for ID/TA versus Umbrella Projects 35 On-lending funds for TA to benefit agencies 36 5. Lessons and Recommendations 41 Laurie Effron (Task Manager) acknowledges with thanks the contributions of: Robert Armstrong (Consultant) who co-authored the audit; and Barbara Yale who provided administrative assistance. 2 Tables: 1.1 Bank Loans/Credits for Free -Standing TA, FY83-96 16 3.1 TCI and TCII: Credit use estimated at appraisal and actual 24 3.2 PSSSP: Credit use estimated at appraisal and actual 26 4.1 Characteristics of on-going TA projects 38 Boxes: 3.1 Examples of relatively successful TA Sub-Projects 28 3.2 Examples of Problems in Selected TA Sub-Projects 30 Annexes: A. Basic Data Sheets 43 Technical Cooperation Project (Cr. 1412-CHA) 43 Second Technical Cooperation Project (Cr. 1664-CHA) 45 Planning Support and Special Studies Project (Cr. 1835-CHA) 47 B. Sub-projects financed under TC I, TC II, follow on projects, ratings 51 Sub-projects and beneficiaries financed under PSSSP C. Benefiting Agencies under TC I, TC II and PSSP 55 D. Sub-projects financed under TC I and TC II aimed at Institutional Development 57 E. Questionnaire to MOF 59 F. Analysis of Ministry of Finance Cash Flow from On-Lending and 63 Repayment of PSSSP Funds G. Agencies and People Met During Evaluation Mission 65 3 Preface This is the Performance Audit Report (PAR) for the China First and Second Technical Cooperation Credit Projects (TC I and TC II, Credits 1412-CHA and 1664-CHA), and the Planning Support and Special Studies Project (PSSSP, Credit 1835-CHA), which provided funds for project preparation and institutional development across a number of sectors. TC I was approved on August 24, 1983 in the amount of SDR 9.4 million (US$10 million equivalent at appraisal). The credit was fully disbursed and closed on December 31, 1988, after a one year extension. TC II was approved on March 13, 1986 in the amount of SDR 18 million (US$20 million equivalent at the time of appraisal). The credit was closed on June 30, 1994, after two years of extension, at which time SDR 0.5 million (about US$0.7 million) was canceled. PSSSP was approved on June 30, 1987 in the amount of SDR 15.9 million (US$20.7 million equivalent at the time of appraisal). The credit was closed on December 31, 1994, after two years of extension, at which time SDR 1.5 million (about US$2.3 million) was canceled. During the period 1982 to 1991, about US$10 million in parallel financing was provided by UNDP for which the Bank (operational staff and EDI) was the executing agency. This PAR covers the three technical assistance credits and is based on President's Reports, credit agreements, project files, economic and sector reports over the period 1983 through 1996, discussions with a wide range of Bank staff involved directly or indirectly in the credits or in other aspects of technical assistance to China, Project Completion Reports (PCR) for TC I and TC II and the Implementation Completion Report (ICR) for PSSSP, all prepared by the East Asia and Pacific Regional Office, individual completion reports submitted by implementing agencies in China under PSSSP, and a mission that visited China in February/March 1997. Prior to arrival, the OED mission sent three questionnaires, one each for the two implementing agencies under the projects: the Ministry of Finance (MOF) for TC I and TC II, and the State Planning Commission (SPC) for PSSSP; and one for sub-project implementing agencies. The questionnaires were translated into Chinese and distributed in advance of the mission's arrival to the relevant implementing agencies. Some agencies prepared written responses, but in all cases the questionnaire helped to structure the interviews. A copy of the questionnaire is in Annex E. The PCR/ICRs for the three projects provide adequate accounts of implementation and lessons learned. In general, the PAR concurs with the PCR/ICR findings on the projects. The PAR, however, highlights a number of issues that should be explored further in a country context.  5 Ratings and Responsibilities TC I TC II PSSSP (Cr. 1412 - CHA) (Cr. 1664 - CHA) (Cr. 1835-CHA) Outcome Satisfactory Satisfactory Satisfactory Sustainability Likely Likely Likely Institutional Development Impact Substantial Substantial Substantial Borrower Performance Satisfactory Satisfactory Satisfactory IDA Performance Satisfactory Satisfactory Satisfactory Key Staff Responsible Appraisal Midterm Completion Review TC I Task Manager Albert Howlett n/a Albert Howlett (Cr. 1412-CHA) Division Chief David Turnham n/a David Pearce County Director S. S. Kirmani n/a Shahid Javed Burki TC II Task Manager Albert Howlett n/a Keiko Sato (Cr. 1664-CHA) Division Chief David Turnham n/a Zafer Ecevit Country Director S. M. L. Van der Meer n/a Nicholas Hope PSSSP Task Manager David Pearce n/a David Rix (Cr. 1835-CHA) Division Chief Baelhadj Merghoub/Ed Lim n/a Zafer Ecevit Country Director Shahid Javed Burki n/a Nicholas Hope PCR/ICRs were prepared as follows: Technical Cooperation Project Cr. 1412-CHA prepared by Albert Howlett Second Techical Cooperation Project Cr. 1664-CHA prepared by Keiko Sato Planning Support and Spec. Studies Project Cr. 1835-CHA prepared by David Rix  7 Evaluation Summary Why Did OED Evaluate These Projects? 1. OED chose these three technical assistance (TA) projects for performance evaluation because we hoped that their evaluation would yield some "lessons of success." This hope has been partially realized. 2. It is more difficult to find successful TA projects than to find unsuccessful ones. This is especially true in countries where conditions are changing rapidly but unpredictably, where relations between the Bank and client country are relatively new, and where the country has little experience in managing donor-financed TA. All of these conditions applied to China during the period of preparation and implementation of the three TA projects under review. 3. Countries that most need TA have often proven to be those least able to use it effectively or to sustain its benefits. Self-evaluations by the Bank's East Asia and Pacific Region and by the Government of China (GOC) suggested that China is an exception to this rule. OED wanted to find out if this was the case, and if so, why. Main Findings 4. This PAR finds the outcomes of all three projects to have been satisfactory and likely to be sustainable. It also finds that both Borrower and IDA performance was satisfactory and that the institutional development (ID) was substantial for all three projects (see Table on Principal Ratings on page 51. These ratings are the same as those of the PCR for TC I and of the ICRs for TC II and PSSSP. 5. This is not to say that this PAR is in agreement with the PCR and ICRs on all points. The PAR finds a more variable quality ofsub-projects and is more critical than the PCR/TCRs, particularly with respect to PSSSP regarding: (a) the vagueness with which the President's Reports specified project objectives; (b) the "ad hoc" nature of sub-project selection, reflecting a lack of clearly-articulated priorities; (c) distortions created by the Ministry of Finance's (MOF's) on-lending policies and by the lack of aid coordination; and (d) shortcomings in both Government's management and the Bank's supervision of the projects. Consequently, the PAR did not identify as many lessons of success from these projects as was initially hoped. 6. But if the PAR finds more to criticize, it also finds much to commend. China showed strong ownership of the projects, and the implementing agencies managed their respective sub- projects well. As OED has found in reviewing TA experience in many countries, it is difficult to get the right combination of flexibility and discipline into TA projects. Often the needed discipline is lacking, and projects become prey to rent-seeking and wasteful expenditure. In China, on the other hand, OED found that discipline and accountability were strong, with the result that rent-seeking and waste were minimal. The PCR for TC I did not contain formal ratings, but these ratings were implicit in the PCR. 8 Background 7. Since China joined the Bank in 1980, its demand for TA has been strong. This was the period during which China's leadership decided to shift from an inward-looking economic system modeled on the Soviet system of central planning to a more outward-looking, market- oriented system. 8. This change gave impetus to a surge of pent-up demand for foreign know-how. The Bank had a comparative advantage in serving as a "window on the world," particularly in the area of economic and financial management. As a result, TA became an important instrument by which the Bank helped expose Chinese officials to foreign techniques, approaches, and systems through a wide array of instruments (study tours and overseas training, consultant visits, in- country seminars, etc.). Throughout the 1980s and in addition to the TA projects discussed in this review, the Bank was assisting China on the policy front through complementary instruments such as economic and sector studies, a large, UNDP-financed EDI program, and Bank-executed TA financed by UNDP for policy discussions as well as project preparation. Objectives and Design 9. The main objective of TC I and TC II was to prepare projects for Bank Group financing; a secondary objective was to help build capacity for selecting, preparing, and implementing projects. These were narrow objectives, but appropriate. GOC and the Bank were at an early stage of their partnership; China had enormous needs for investments and ID, and TC I and TC II were effective tools to build up the lending program. 10. The objectives of PSSSP, by contrast, were broader: strengthening GOC's planning capabilities through carrying out sector studies, assistance for intersectoral, innovative, or experimental studies, and ID to "facilitate the implementation of GOC's program of economic, financial or institutional reforms." The objectives of PSSSP were highly relevant to the Bank's strategy of helping China to import foreign know-how in project preparation and economic and financial management. It was, however, a design weakness of PSSSP that its objectives were so vaguely specified. 11. All three of the projects were umbrella style, covering a wide number of sectors, each involving a blend of "blueprint" and "process" approach. At approval, each project had an illustrative list of sub-projects that included a mixture of well-identified, weakly-identified, and unidentified projects. The Box below indicates the main strengths and weaknesses of the projects' design, with particular reference to PSSSP. The main strength was, as noted above, China's strong demand for the types of TA offered by these projects, and the consequent strong Chinese ownership. The main weakness was the lack of clear priorities for ID/TA. For PSSSP in particular, the criterion for sub-project selection was vague, and led to an ad hoc selection of sub-projects, covering too many tasks, sectors and implementing agencies. On balance, however, OED found the "quality at entry" to have been satisfactory. Implementation, Outcome and Sustainability 12. The Box below lists the PAR's main findings on implementation and outcomes. In sum, TC I and TC II accomplished their main objective of helping GOC prepare projects for Bank Group financing. They also had positive outcomes in terms of strengthening GOC's capacity to 9 select, prepare and implement projects. Study tours and "embedded" training, imparted by consultants and through extensive project supervision, were important instruments in achieving these ID objectives. 13. With regard to PSSSP, OED judged the overall outcome to be satisfactory, even though some sub-projects were not as satisfactory as indicated in the ICR. The benefits of PSSSP are difficult to define and measure, especially because there were no performance indicators defined ex ante or measured ex post. This PAR concludes, however, that a number of sub-projects achieved considerable capacity building in auditing, national income accounting, macro- modeling, and other fields. In addition, some of the sub-projects had an impact in terms of helping to "change the concepts and the rules of the game"-- the broadest definition of ID. Examples of this include introducing economic parameters into project evaluation, defining the new role of the central bank in a market economy, and evaluating and assisting mergers and acquisitions. 14. The role of study tours in achieving this "consciousness raising" was particularly significant. Although skepticism has been expressed within the Bank and in China about the efficacy and cost-effectiveness of these tours, OED felt that the study tours were, on the whole, cost-effective. It is, of course, impossible to attribute specific institutional changes or momentous policy shifts in China to any particular TA input. But OED's qualitative judgment is that the rate of return to the package of TA products delivered by the Bank Group, including the three projects under review, was high. 15. By contrast, the use of foreign consultants was sometimes problematic, and is likely to remain so. Some of the issues include Chinese agencies' reluctance to: borrow funds to finance foreign consultants, share data or other information needed for analysis, pay high salaries demanded by consultants, accept designs or other advice from foreign consultants. On the other hand, the Chinese retain full ownership of the objectives of particular tasks, so that it is highly unlikely that foreign TA will create dependency effects. 16. The main factors explaining these outcomes have already been mentioned. The demand and ownership were strong for TA that, owing to China's particular situation, had a high prospective return. The resources made available were generally well-used, owing to the strong accountability and discipline exercised by the Government. 17. OED rates the sustainability of the projects' benefits as likely for all three projects. Implementing agencies had relatively low staff turnover and, more important, the agencies retained responsibility for the objectives of their sub-projects. The activities financed were only the first step for these agencies in improving their modeling, changing their analytic approach, reforming the regulatory framework in which they function. OED found evidence that all of the agencies interviewed during its mission are continuing to use the tools and concepts acquired under the sub-projects. At a more macro level, the Government has sustained policy reforms for the past two decades, and there are now many stakeholders committed to China's new directions. 10 11(4 I ~\I 5' I'<()IHl I11 All three poet:All three proects: Project objectives important to GOC: Ad hoc approach to ID: no priorities strong ownership established for sub-project selection Flexible, process approach Perfunctory risk/return analysis DESIGN TC I and TC II: PSSSP: Well-focused objectives Vague criteria for sub-project selection Criteria for sub-project selection clear, Many components and implementing except for ID agencies PSSSE Absence of performance indicators Focus on exposing senior officials to Absence of detailed implementation new approaches to economic/financial schedules for sub-projects management All three r j All three pMrjects: Projects' objectives largely achieved: Repayment requirement discouraged some -46 projects prepared for Bank prospective clients financing, US$7 billion committed Some implementation delays IMPLEMENTATION -quality of project, on-going and Some problems with consultants completed, 96% satisfactory -considerable ID occurred across Several sub-projects of arguable priority many agencies: on both technical and policy fronts, without dependency Several sub-projects with less-than-fully effects satisfactory outcomes -study tours particularly important Some study tours not justified Sustainability likely Inadequate progress reports, audits All three prjects: All three prjects: Strong demand for TA by GOC and MOF on-lending policy created distortions by agencies in selection of benefiting agencies Favorable institutional environment: Many first-time Bank borrowers: lack of competent civil service: generally familiarity with Bank procedures disciplined management by apex and High turnover of staff in MOF/TATD KEY FACTORS implementing agencies Strong accountability within GOC TC I and TC II and sector studies First-come, first-served selection of sub- under PSSSP: Strong incentive and projects, absence of established priorities adequate resources for Bank Except for sector studies, inadequate Bank supervision from project preparation incentives/resources to supervise projects budgets Problems in coordination between MOF and SPC 11 Issues 18. The Box on the previous page lists some of the main the issues addressed in the report. These include MOF's on-lending policy, difficulties inherent in the "process" approach, the need for future TA in China to be guided by more clearly-defined ID priorities and to avoid umbrella- style projects (i.e., many sectors, many implementing agencies). These issues, and the extent to which the lessons emerging from them have been incorporated into on-going TA projects in China, are discussed in more detail in Chapter 4. Lessons and Recommendations 19. The major lessons are presented here in summary form. A fuller discussion, as well as additional lessons and recommendations, are in Chapter 5. Most of the lessons listed below relate to TA project identification and design. The four most important lessons emerging from the experience of the three projects under review relate to: (a) the country's institutional environment; (b) the ad-hoc nature of the sub-projects under PSSSP; (c) the proliferation of tasks, sectors and implementing agencies in PSSSP; and (d) MOF's onlending policy. 20. Client motivation/ownership and accountability/discipline matter most: motivation to obtain know-how, discipline to minimize rent-seeking. These constitute the "litmus" test of the environment for TA. When these are weak, risks of failure in TA effectiveness are high. When these are strong and incentives are in place to produce results in an efficient manner, as they were in China, the likelihood of success is high. These are essentially "initial conditions," however, that cannot be affected by TA project design. They need to be carefully assessed before proceeding with a TA project. 21. TA priorities need to be geared to addressing the most binding constraints to ID. This PAR has criticized PSSSP because the choice of sub-projects was ad hoc. GOC presented the Bank with sub-projects for approval on a first-come, first-served basis, without reference to whether the sub-projects addressed the most urgent and important needs for strengthening China's planning capacity or carrying out its reform program. 22. Some ID constraints are more binding than others. TA needs to be identified and programmed in a strategic framework that links priorities and programs for capacity building to the pace and sequencing of policy reform. Both the Borrower and the Bank should agree on criteria for supporting TA that reflect the ID priorities. 23. The "process approach" need not involve many tasks, sectors and implementing agencies. This PAR finds that a process approach was appropriate for China in the 1980s. Nevertheless, it does not follow that an umbrella-style project, covering many sectors and agencies, is advisable. Such projects are extremely difficult for the client country and the Bank to supervise adequately. Because PSSSP involved many different tasks and sectors, supervision by both the apex institutions (MOF and SPC) and the Bank was spread too thin. Supervision became oriented more to monitoring than to professional problem-solving. Even in the absence of a strategic framework for establishing TA priorities, it should be possible to limit project objectives by defining the types of institutions that will have access to scarce TA funds. 12 24. MOF's on-lending policy introduces distortions into the allocation of TA funds: TA should be allocated to where its rate of return is highest, at on the basis of implementing agencies'repayment capacities. Well-used TA involves considerable positive externalities. The ultimate beneficiary of improved capacities for modeling, forecasting, designing, analyzing and implementing policy reforms, is the country as a whole, not the implementing agency. TA should be financed and allocated accordingly. Agencies with the highest priority need for TA may have limited or negligible repayment capacity, yet crowding them out by requiring repayment would forgo high potential returns. MOF should change its onlending policy in the allocation of TA funds. There are other means by which to preserve needed discipline and accountability. 13 1. Introduction and Overview Why Audit These Projects? 1.1 These three technical assistance (TA) projects' were chosen for audit by OED because the PCR and ICRs concluded that the projects were successful in meeting their objectives in spite of conditions that could have made it difficult to achieve satisfactory outcomes. Lessons of successful TA projects are harder to come by than lessons of less-than-successful ones. 1.2 The conditions mentioned above included the newness of Bank/Government of China (GOC) relations. There was a lack of familiarity by each side of the other in terms of working conditions, institutions, procedures, and approaches to economic development. 1.3 In addition, all three projects, and in particular the third one, PSSSP, were carried out during a time of considerable and unpredictable change in China -- conditions that posed special problems for the design and management of TA and required considerable flexibility. 1.4 As experience elsewhere has shown, however, it is difficult to get the right combination of flexibility and discipline into TA projects such as the PSSSP that pursue a "process" rather than a "blueprint" approach (these terms will be explained below). Given this experience, it was hoped that an OED evaluation might shed light on which factors are most important in determining the success or failure of "process approach" projects. How This Audit Differs from the PCR/ICRs 1.5 This report is similar to the PCR/JCRs of the three projects in that it reviews the design and implementation of the projects and provides ratings of the projects' outcomes, institutional development, sustainability, and Bank and Borrower performance. 1.6 One main difference between this report and the PCR/ICRs is the focus of this report on: (a) systemic country conditions that often influence the effectiveness of TA projects; (b) the question of how "relevant" and appropriate were the projects' objectives; and (c) selected issues, such as the appropriateness of using a "process approach" for TA projects, establishing priorities for institutional development (ID)/TA, and the MOF policy of on-lending Bank Group funds for TA to benefiting agencies. 1.7 Another difference is that the lessons identified by OED differ from those of the PCR/ICRs. Main Questions to be Addressed in this Report 1.8 OED sought to address the following questions: (a) how has demand for TA in China evolved since the early 1980s, when TC I was approved, through 1995, when final disbursements 1 Technical Cooperation Credit (TC I, Credit 1412-CHA), Second Technical Cooperation Credit (TC II, Credit 1664- CHA), and Planning Support and Special Studies Project (PSSSP, Credit 1835-CHA) 14 on PSSSP were made? (b) in what respects were the three TA projects most and least successful, and why?; and (c) what are the main lessons of experience and have they been applied to the design or management of the current set of TA projects? These questions were addressed through a questionnaire sent to GOC prior to the OED mission, a copy of which is found in Annex 4. Some Historical Context: China's Opening 1.9 One of the main reasons that TA has failed widely in the world is that TA projects are often donor-driven rather than client-driven, with weak ownership on the part of borrowing and implementing agencies. In the case of China, the demand for external TA, whether donor- financed or GOC-financed, was virtually nothing at the beginning of the 1980s but expanded enormously in the span of a few years. 1.10 Between 1949, when Mao Zedong proclaimed the People's Republic of China, and December 1978 (about two years after Mao's death), China pursued an inward-looking economic order modeled on the Soviet system of central planning. A turning point was marked in December 1978, when new, pragmatic leadership adopted agrarian policies aimed at expanding rural income and incentives, endorsed experiments in enterprise autonomy, reduced central planning, introduced legal reforms, and approved direct foreign investment in China. 1.11 In 1982, a new state constitution was adopted that provided the legal framework for ongoing reforms in China's social and economic institutions and practices. More recently, in early 1992, Deng Xiaoping's visit to southern China gave new impetus to the movement towards a market-oriented economy. The three major areas of economic reform have been the open door policy, movement to market-determined prices, and the devolution of economic decision making on investments, production and distribution. 1.12 The open door policy has resulted in a dramatic increase not only in foreign trade and foreign direct investments, but also in exposure to foreign technology, and, significantly, to new ways of economic and financial management. TA played a significant role in changing the approach of the Chinese leadership in these areas. Over time, the demand for TA shifted from improving efficiency within the socialist framework, which did not entail much institutional change, to changing more fundamentally the "rules of the game." TA was also required to develop institutional capacities to function effectively in an economic system increasingly characterized by administrative decentralization and market-driven development. Evolution of the Bank's approach to TA 1.13 China joined the Bank Group in 1980, and the Bank quickly became a significant "window on the world" for China. The role of the IMF and other international institutions was still relatively minor at this point. From the early 1980s the Bank's country strategy recognized the Bank's "unique opportunity...to increase China's access to foreign technology and practices and to assist in the development and implementation of reforms that will help to increase the efficiency of resource use and reduce poverty," notwithstanding "major uncertainties that still prevail concerning the pace as well as the pattern of reform". 1.14 The Bank also recognized the complementary role of various TA instruments, including TA components of investment projects, free-standing TA operations (including the three TA 15 projects under review in this audit), Bank economic and sector work (ESW), and Bank execution of LNDP-fmanced projects that involved project preparation, support to EDI for courses and seminars, and high-level economic policy advice. The focus of the Bank/IDA and UNDP funded/Bank executed TA was on: (a) -introducing technical improvements in project preparation so as to be in line with international standards; (b) improving efficiency of central agencies such as the SPC and MOF; and (c) "consciousness raising" about alternative economic policies and institutional frameworks for the top Chinese officials and selected central agencies. Throughout the decade of the 1980s, these various activities were planned and carried out within a context of centralized state planning and dominance of state-owned enterprises. 1.15 At the same time, Bank lending and policy dialogue proceeded along two parallel tracks, with little convergence. The lending focused initially on a fairly narrow range of equipment- intensive projects, while economic analytic work and dialogue were carried out under the Bank's ESW and UNDP-financed, Bank-executed policy reform projects, including a large program carried out by EDI, funded by UNDP, aimed at high GOC officials on various topics concerning economic analysis and macroeconomic policies. 1.16 During this first decade of the Bank/GOC relationship, there was a rapid buildup of Bank/IDA lending and related TA activities. The Bank's lending program went from US $200 million in FY81 to US$1.4 billion by FY87. As virtually all Bank/IDA-financed investment projects included TA components, the volume of lending for TA activities rose as well. By end- FY87, Bank-Group lending for TA (components and free-standing) had grown to US$400 million. 1.17 By the end of the 1980s, the Bank had carried out two internal reviews of Bank/IDA 2 financed TA activities in China. TA needs in China were large, growing, becoming more diverse and shifting from the micro level of project preparation to a macro level involving sector planning and national economic management. The Bank was increasingly being called on to deliver TA to a wide range of agencies at the central, provincial and municipal levels, across a diverse set of roles. Given these demands for TA, it was clear that GOC and the Bank needed to establish priorities for the use of Bank resources on TA, and the Bank had to improve its coordination of TA activities, both internally and with other donors. Almost ten years later, these are still issues for Bank provision of TA to China. 1.18 Beginning in the early 1990s, the pace of reform accelerated. As the scope of policy change broadened, away from the planning paradigm towards the market paradigm, the Bank's country assistance strategy shifted as well. It included a comprehensive policy dialogue at the national level, a more integrated approach to lending and non-lending services (ESW, seminars, conferences), and a lending program that sought to balance support for policy advice and institutional reform at various levels of Government, with support for technological modernization, environmental protection, physical and social infrastructure development, and poverty alleviation. The scale of the lending program increased as well, from about US$1.5 billion per year to more than US$2.5 billion per year between FY92 and FY95. The scale, objectives, and design of Bank-financed TA reflected these changes. 2 "Improving the Effectiveness of Technical Assistance in China: A Special Report" (March 1987), and "An Overview of Technical Assistance Activities in China" (June 1988). 16 Technical Assistance Project (Credit 2423-CHA), at US$60 million, the largest TA credit approved for China to date, which focused on improving the regulatory and supervisory role of the People's Bank of China and upgrading the payments system. This was followed by four additional free-standing TA projects, which together provided support for a broad range of institutions related to China's economic, legal, and environmental reforms, and included continued support for project preparation. By end-FY95, the Bank had approved a total of over US$1.4 billion for free-standing TA projects and TA components of projects, representing some 6 to 7 percent of the total cumulative Bank/IDA commitments in China since FY81. Of this amount, US$270 million was for eight free-standing TA projects, listed in the Table below.3 Table 1.1: Bank loans/credits for free-standing TA, FY83-96 Name LIC 9 $US m FY 83 84 85 86 87 88 89 90 91 92 93 94 95 96 Technical Cooperation Project C1412-CHA 10.0 X O Second Technical Cooperation C1664-CHA 20.0 X O Planning Support and Spec. Proj. C1835-CHA 20.7 X 0 Financial Sector Tech. Asalstance C2423-CHA 60.0 X Reform, Inst. Support and Pro-Inv. C2447-CHA 50.0 X Environmental Tech. Assistance C2522-CHA 50.0 X Economic Law Reform C2654-CHA 10.0 X - Fiscal Technical Assistance L3873-CHA 25.0 X - C2709-CHA 25.0 Total: 270.7 X - Year of BoAd Approval 0 = Project Closng - lrolementaton Perid 1.20 Now that new directions of economic reform have been sustained for some years, and the scale and complexity of the Bank's lending and non-lending program having reached such high levels, it is more important than ever to ensure that GOC and the Bank agree on an articulated strategy for TA in all its many forms. Such a strategy would be based on priorities for capacity building and ID, would include mechanisms to deliver timely, well-designed TA from the Bank and other donors for implementation of the reform program, as well as including rapid, ad hoc responses to specific requests from GOC. 3 The total picture on Bank-funded or Bank-executed TA activities should include: UNDP-financed, Bank executed projects, which have totaled some US$10 million since 1980; Policy and Human Resources Development fund of Japan, which has totaled over US$55 million since 1985; the Institutional Development Fund, for about US$3.4 million; and other bilaterally funded grants and trust funds of some US$30 million. To this it would be important to add the Bank's ESW. 17 2. Objectives and Design Issues Objectives of TC I, TC 11, and PSSSP 2.1 The main objective of both TC I (FY83) and TC II (FY86) was to assist GOC to prepare projects that were expected to be financed by the Bank Group; a secondary objective of both projects was to strengthen GOC's capacity to select, prepare, and implement projects and to administer TA. TC II went further than its predecessor in the objective of institutional development (ID), by specifying that certain sub-projects would be aimed at strengthening core agencies involved with Bank operations and at providing for economic and policy studies to be carried out by research agencies. 2.2 PSSSP (FY87) had three main objectives: (a) to strengthen GOC's development policy and planning capability through sector planning studies, as input into the next Five Year Plan (1991-95); (b) to prepare and carry out TA activities of an intersectoral, innovative, or experimental nature; and (c) to facilitate the implementation of GOC's program of economic, financial and institutional reforms through training to be provided to the central, provincial, and municipal governments. Were These Objectives the Right Ones? 2.3 This PAR concludes that the objectives were, in broad terms, the right ones, taking into account: (a) the realities of China's economic and political situation and of the Bank/GOC relations at the time the projects were formulated; and (b) that other instruments were being used to provide important windows to the outside world in key areas such as macro-economic management. In 1980s, when TC I and TC II were approved, China had enormous investment needs as well as a large scope for improvements in economic management. GOC was looking to Bank financing, however, primarily to support the investment needs. As a result, TC I and TC II were to be used to build up the pipeline for the Bank Group lending program, while other (grant) instruments were providing study tours, high-level policy seminars and training activities for top GOC officials. These instruments included the Bank's ESW and several UTNDP-financed, Bank- executed projects, which included support for EDI activities. 2.4 The ID objectives in TC I were focused mainly on investment projects, projects that were, in general, without significant policy content, reflecting the reality of the GOC/Bank relationship. By 1986 GOC accepted that it use Bank funding to support policy studies in China, so the ID objectives under TC II were a bit broader, covering "economic and policy studies." 2.5 By contrast, a key objective of PSSSP was to provide support to sectoral planning. Three sectors were identified where the Bank Group was likely to provide subsequent funding. In this respect, PSSSP can be seen as a logical successor to the first two TA projects: it was, in part, an "up-stream" project preparation facility, focusing on sectoral issues. With the benefit of hindsight, however, strengthening the existing apparatus of central planning was based on a false premise: that the planning paradigm would persist. 2.6 With the devolution of decision making and the increasing corporatization and autonomy of state enterprises, top-down approaches to investment planning and price setting have given 18 way to a large extent to market forces,4 and the role of sectoral planning has been greatly circumscribed. Nevertheless, in early 1987, when PSSSP was approved, sectoral planning was still a relevant objective. In addition, because PSSSP included other relevant and important objectives, such as ID for economic reform, PSSSP was able to adapt to changing demand for ID. Such adaptability proved to be both a strength and a weakness (see paras. 3.8-3.13). The third objective of PSSSP was to carry out TA activities of intersectoral, innovative, or experimental nature; this PAR finds this objective to have been so broad and general that, in the absence of any selection criteria to limit the possible choices, it invited an ad hoc approach to selecting sub-projects. Was the Design Appropriate? 2.7 Blueprint versus Process Approach. All three of the projects were a blend of "blueprint" and "process" approaches, because they each had an illustrative list of sub-projects that included a mixture of well-identified, weakly-identified, and unidentified sub-projects. All three projects were slightly more "process" than "blue-print" to the extent that even the well-identified sub- projects were not mentioned in the Development Credit Agreements (DCA). Given the uncertainties at the time of approval of the three projects, this approach to design was appropriate. 2.8 Implementation of sub-projects was to be on a first-come, first-served basis, provided that the sub-projects fit with the objectives and criteria for selection. In the case of TC I and TC II, this meant that sub-projects involving preparation activities were to be for projects that were part of both GOC's five year plan for investments and the Bank Group's lending program, and where grant financing was not available. Thus, the flexibility for sub-project selection was within a well-defined subset of projects that can be presumed to have had priority for the economic development of the country. 2.9 In addition to project preparation activities, TC I also explicitly recognized that a sub- project could also qualify for financing if it was a "program to strengthen the capacity... of institutions... for project planning and implementation". This was repeated in TC II, with the addition that a sub-project could qualify if it involved a program to strengthen economic development, which was not otherwise defined. The indicative list of sub-projects in both TC I and TC II included ID activities, aimed at, for example, strengthening external debt management (TC I) and assistance to the audit administration (TC II). According to the illustrative list of sub- projects, 33 percent of project costs were expected to be for non-project-preparation activities under TC I and about 40 percent under TC II. 2.10 Under PSSSP, over 60 percent of the credit was to finance sector studies to strengthen planning.5 The remaining 40 percent of the credit was to be used for ID and capacity building, particularly in Ministry of Finance (MOF), State Planning Commission (SPC), and the domestic consulting industry. PSSSP had extremely broad criteria for sub-project selection: the DCA Today, prices of more than 97 percent of consumer goods and 80 percent of producer goods are market determined. The three identified sectors were: iron and steel, petrochemicals, and electric power; other possibilities included transport, petrochemicals and mining, and water development and management. These were all sectors where the Bank Group had, or expected to have, significant lending activities. 19 specified only that each sub-project was to "be of high priority in [GOC's] development efforts and have no equally suitable financing available" from other donors. 2.11 A process approach for PSSSP can also be considered appropriate, given: (a) how rapidly China's economic conditions were changing; (b) how uncertain was the pace and sequencing of future policy reforms; and (c) how difficult it would have been for either GOC or the Bank to identify the tasks, beneficiaries and instruments that would yield highest returns per dollar of TA spent. 2.12 This design flexibility in PSSSP proved to be useful as the economy underwent fundamental reforms and the demand for ID/TA to support these reforms shifted. Using the process approach meant that when several of the sector studies failed to materialize, funds could be used to support other agencies and studies relevant to the economic reforms. 2.13 Nevertheless, a common problem with the "process" approach is that such flexibility can result in a lack of discipline in the allocation process, especially when there are no well-defined criteria for establishing priorities, as there were not in PSSSP. To avoid too much "looseness" or "ad-hocery" in the outcome, it would have been important to define more specific criteria for selection of sub-projects to ensure that they fit into the priorities for development, either by focusing on certain types of activities, such as strengthening regulatory agencies, or certain institutions, such as SPC and agencies associated with the State Council, that were likely to play a key role in managing the economy. The failure to do this under PSSSP affected the outcome (paras. 3.7-3.12). 2.14 Umbrella style versus sectoral focus. All three projects were umbrella-style, line-of- credit type projects, with a central agency serving as the apex unit for the benefiting agencies from many sectors. Under TC I and TC II, this was an appropriate design feature because: (a) the sub-projects were to come from a well-defined set; (b) there were strong incentives for both GOC agencies and Bank staff to ensure quality outcomes; (c) on the Bank side, the quantity and quality of resources for supervision were adequate because budgets for project preparation were available for the purpose. Under PSSSP, these factors were absent, except for sectoral studies where the Bank was likely to be involved in lending. Thus the risks of ad hocery, of spreading the funds too thinly over many agencies, and of inadequate supervision were much greater under PSSSP. 2.15 Structure of Project Management. As with any process approach or umbrella-style, line- of-credit type project, the role of the apex agency is potentially extremely important. Demand for the use of the funds under all three projects was to come from a large number of ministries and agencies, at the central, provincial, and municipal levels, research institutes, consulting firms, national corporations, and project offices. While it was theoretically important to ensure at the design phase that the apex institution was equipped to determine the eligibility of the sub- projects, appraise the specific proposals, provide support for implementation, and monitor and evaluate use of the funds, in practice it was unlikely that a department within MOF or SPC would have the expertise to handle, from a technical point of view, the wide spectrum of sub- 6Because there were few agencies that benefited from more than one sub-project, about 60 different agencies used funds under the three TA projects. 20 projects. There is no evidence that much attention was paid at the design phase to the issue of the ability of the apex agency to play these roles . 2.16 Under TC I and TC II, the criteria for selecting sub-projects were relatively well-defined, so in fact, the role of determining the eligibility of sub-project proposals was fairly straightforward for the apex agency. In addition, as noted above, both participating agencies and Bank task managers would have a strong incentive to ensure that the sub-projects for project preparation were well defined and executed. Therefore, by the nature of the design of TC I and TC II, the ability of the apex institution to play its various roles was less important and the likelihood that Bank staff would provide close and adequate supervision was greater than it was under PSSSP. 2.17 Under PSSSP, the project objectives and criteria for sub-project selection were extremely broad, as noted above (para. 2.10). The Bank was either going to have to rely on SPC and, to a lesser degree, MOF, to select projects with "high priority" and well-developed proposals and to supervise their implementation, or it was going to have to provide an unusually high degree of supervision, at all stages of sub-project selection and implementation. As noted below, it turned out to be a hybrid of the two, with mixed results (paras. 3.7-3.12), in some cases creating problems between GOC and the Bank. In addition, having two apex institutions, one to agree on the substance of the sub-project (SPC) and the other on the financial aspects (MOF), was potentially problematic for the benefiting agencies, who would have to deal with departments in two ministries. 2.18 With respect to PSSSP, Bank staff have commented that the main motivation for its formulation was to ensure that China's FY87 IDA allocation would be fully committed, in particular because, as of FY88, the repayment terms for IDA credits became less favorable. Evidence in the files lends support to this observation. Had the pressures of IDA commitments been absent, PSSSP might have been designed after more discussion between GOC and the Bank, with more specific objectives and criteria for sub-project selection, and more stream-lined apex arrangements. 2.19 Instruments. Under all three projects, the projected expenditures were to be mostly (63 - 70 percent) on consultants. Considering that much of the activity required technical expertise on design or modeling work, this was appropriate. In TC I, the remaining 30 percent was to be for "training and equipment," with no breakdown given between the two. Under TC II, training was expected to account for a little more than 25 percent of project costs, with only 4 percent for equipment. Under PSSSP, projected expenditures on training and equipment were about equal, at slightly less than 20 percent each. In all cases, these were clearly only indicative figures. Given the design flexibility of the projects, and the fact that the benefiting agencies had at most been identified but not appraised in terms of their ID needs, it was difficult, at the design stage, to provide detailed justification for one type of instrument over another. Nevertheless, Under TC I and TC II the apex unit was within MOF. Under TC I, a World Bank Department (WBD) was set up in MOF with specific responsibility for administering Bank Group projects and in 1989, a special unit within WBD, the Technical Assistance and Training Department, was set up to administer TA projects. For PSSSP, the apex responsibility was passed to SPC at the last minute (after negotiations), with MOF playing a secondary role of financier. 8 Information available on actual expenditures is presented in footnote 14. 21 there was clearly an appropriate attempt to avoid emphasis on equipment and to emphasize, by contrast, the training elements of the TA. 2.20 With respect to the various TA instruments to be financed under all three projects, one element that should have been emphasized at the time of project approval was the potentially high pay-off of exposure to other systems, whether in the form of study tours, short-term training abroad, or internal conferences and seminars. In addition, the pilot nature of the ID efforts under PSSSP should also have been noted, as well as the need to try various instruments, so as to adjust the design of sub-projects subsequently to suit the agency. Evaluation of Quality at Entry 2.21 Overall, the quality at entry was satisfactory for all three TA projects. This judgment reflects consideration of prospective rewards and risks of the outcomes, given the objectives and design of the three projects. 2.22 Prospective Rewards. Effectively designed and executed TA had, and still has, potentially very high economic rates of return in China. Given the scope for economic development, the scale of the country, the extent to which there were and still are key officials and staff with little or no exposure to the outside world, and the strong demand from GOC officials at many levels for such exposure, the potential impact from appropriate TA to relevant staff is huge.9 Nevertheless, and in keeping with the Bank's overall approach to lending services to China at the time, the focus of these three TA projects was relatively narrow and primarily technical: project preparation (TC I and TC II) and sector studies underlying centrally planned investments (PSSSP). The design of PSSSP, in particular, may have represented a missed opportunity to target TA more directly to the truly core agencies involved in economic reform (foregone reward). 2.23 On the other hand, the design of all three TA projects was a reflection of the reality of lending to China at the time: ESW and policy dialogue were separate from lending and were on a grant basis. The pace and direction of China's reform were still uncertain at the time these TA projects were put in place, and GOC may not have been interested in borrowing IDA funds to support a reform program whose nature was still uncertain. It was not until FY93, in fact, that another free-standing TA operation, in the financial sector, was put in place in China, and this was only after it had become clear that reforms in the financial sector were central to the overall economic reform under way. 2.24 Prospective Risks. Given the design of TC I and TC II, the risk offailure was minimal. Such failure could have included lack of demand for funds, misuse of funds, poor quality of preparation studies. Because the incentives were in place, however, for both the implementing agencies and Bank staff to oversee the use of the funds and ensure quality outcome, these risks were not significant. Under PSSSP, on the other hand, the risks were greater. Because criteria for sub-project selection were so broad, the risk was considerable that the sub-projects would be of arguable priority for use of scarce funds ("ad hocery"). For the capacity building objective, it A 1986 evaluation of EDI's China program stated, "The EDI program has had a profound and pervasive impact on the reform movement in China. Indeed, it is hard to think of another country or region where EDI training has had a comparable impact on both decision-making, particularly on investment, and in spawning the extraordinary number of training programs organized by former Chinese participants in EDI courses." 22 was possible that the sub-projects themselves might not be well designed, due to an inadequate needs assessment, inappropriate selection of instruments -- too many "toys and tours", or inappropriate staff benefiting. Because of the relatively modest amount of funds spread across many agencies, it was possible that sustainability of the sub-projects would be in question due to lack of follow-up TA. As it turned out, the presence of certain systemic factors, such as strong demand for TA, good discipline within GOC, staff stability, and close financial monitoring, meant that in fact, these risks were, relative to other countries, minor. These factors are discussed further in paras. 3.24-3.29. 23 3. Implementation and Outcomes Caveats about Evaluating Outcomes 3.1 The OED mission focused its efforts interviewing staff in agencies benefiting mainly, although not uniquely, from PSSSP. This was because a total of almost 60 agencies benefited from the three TA projects, and because many of the sub-projects under TC I and TC II were carried out eight to ten years ago. 3.2 Even with a narrower focus, however, there were three main difficulties in making an independent assessment of PSSSP outcome: (a) the nature of the benefits was difficult to measure, and PSSSP had no measurable indicators for output or impact; (b) PSSSP involved twenty agencies across as many sectors, and it was difficult to assess the impact of a given sub- project on the basis of short discussions with each agency, in spite of sending out questionnaires prior to the meetings to provide structure to the interviews; (c) few of the participants (GOC or Bank) provided constructive suggestions for what might have been improved in the design or implementation of PSSSP: everything was fine. With the exceptions of some complaints by GOC staff about external consultants and about Bank staff, and some complaints by Bank staff about GOC agencies, most participants judged all components to be equally and highly successful. 3.3 A full assessment of the impact of each of these three TA projects would require subject- matter specialists in each field spending considerably more time with the benefiting agencies. Outcomes by Main Objectives TC I and TC II Outcomes 3.4 TC I and TC II accomplished their main objective of helping GOC prepare projects for Bank Group financing. At the time of appraisal of both projects, it was expected that some 60- 70 percent of project costs/credit use would be for project preparation activities and the rest for ID or studies. In fact over 85 percent of disbursements under the credits (US$30.3 million out of US$35.0 million) were used for project preparation. Given that the funds for project preparation appear to have been relatively well-used, this shift in credit use was probably a good thing. Out of 58 sub-projects carried out under the two projects, 49 were for project preparation; these led to Bank Group financing of 46 projects during FY84 - 94, representing a total of US$7.0 billion in commitments, or almost 40 percent of total commitments over the period. Thus these two TA projects helped expand the Bank Group lending to a significant degree. 10 Of the 15 benefiting agencies interviewed in the field, 3 participated in TC I or TC II, the rest in PSSSP. The three sub-projects that were not followed by Bank Group lending were the Dalian Bulk Grain Terminal, the Jiangsu Provincial Industry Project and the Three Gorges Dam. The first two were canceled from the Bank's lending program. The Three Gorges Dam is nowhere mentioned in either the PCR/ICRs nor in the files, but disbursements of about USSO.47 million were made from TC I and TC II combined for an expert panel to review a Canadian $12 million feasibility study, financed by Canadian International Development Association (CIDA). The Bank accepted the study's findings, but GOC decided to implement a solution with much larger storage level and resettlement than what was proven feasible and economically desirable by the study. The dam is now under construction, with no Bank Group financing. 24 Table 3.1: TC I and TC II: Credit use estimated at appraisal and actual ----------Appraisal-------- --------Actual- - -- No. of Amount Percent No. of Amount Percent Sub- $m of Sub- SM of Credit projects Credit projects TC I" Project Preparation 15 8.1 68 22 10.0 92 ID & Studies 8 3.9 32 4 0.8 8 Total: 23 12.0 26 10.8 TC H1 Project Preparation 24 10.4 62 27 20.3 84 ID & Studies 16 6.5 38 5 3.8 16 Total: 40 16.9 32 24.1 "Amounts estimated at appraisal were for project costs. 2/Amounts at appraisal were estimated for only 85 percent of the credit use. 3.5 Of the 46 projects prepared under TC I and TC II and subsequently financed by the Bank Group, 25 have been completed. Annex B, Tables 1 and 2 present, for completed projects, the ratings on Bank and Borrower performance at preparation and on outcome, and for on-going projects, the ratings for ID objectives and project implementation. Out of 46 projects, 44, or 96 percent, have ratings of satisfactory or highly satisfactory, roughly the same as for the overall China portfolio as of FY96. On the basis of both the quantity and the quality of project preparation activity financed under the two TA projects, the outcome is considered satisfactory. 3.6 TC I and TC H also included the objective ofstrengthening GOC's capacity to select, prepare and implement projects. The evidence points to a substantial and sustainable impact on ID. On the input side, the two main approaches to ID consisted of on-the-job, "embedded" training and discrete sub-projects aimed at selected agencies. Under the first category, Bank staff held a number of seminars and provided direct TA on procurement procedures, drafting terms of reference for consultants, and selecting consultants. In addition, the contracts for external consultants often included study tours and seminars for GOC staff. Consultants "assisted" GOC agencies to prepare projects, with main responsibility for project design clearly resting with the GOC agency. While it is impossible to demonstrate that TC I and TC II alone were responsible for the improved capacity of local agencies, the consensus both in China and in the Bank is that local agencies' capacity for project preparation and implementation -- recruitment of consultants, management of consultants, procurement of equipment -- has improved enormously over the years in those sectors where TC I and TC II were active (e.g., power, ports). 3.7 On the other hand, the discrete sub-projects under TC I and TC II, amounting to about 15 percent of total credit use under the two projects, that were aimed at capacity building and/or economic and policy studies, generally involved modest amounts of money, were ad hoc (across a number of different agencies), and suffered from little or no follow up. See Annex D for a fuller discussion of these sub-projects. 25 3.8 While the quality of sub-projects under PSSSP is not as uniformly high as indicated in the ICR, the PAR rates the quality as being, overall, satisfactory. While no one sub-project was likely to have revolutionized the methods and approaches of the huge agencies in China that benefited from them, a number of the sub-projects certainly played a critical role in at least exposing staff to the concepts and rules used elsewhere and thus can be credited with beginning or reinforcing the ID process in its broadest sense. 3.9 One of the main objectives of PSSSP was to strengthen GOC policy and planning capability through sector planning studies. In the 1980s, planning was still largely within the socialist, centrally-planned paradigm. This objective was partially met. Table 3.2 below shows that at appraisal over 60 percent of the credit was expected to be used for sectoral studies, the remainder for technical assistance and training. In fact, only about 30 percent was used for sectoral studies. Given the shift during project implementation away from the central planning towards a market economy, GOC and the Bank took advantage of the flexibility offered by the design of PSSSP to make use of the credit for other purposes. Nevertheless, the sectoral studies that were carried out proved to be useful, at least in part, for both GOC and the Bank. GOC and the Bank have used the results of the Yellow River Study and the East China and Sichuan power planning studies as the basis for further discussions on policy issues (establishing investment priorities, optimal location of investments, pricing) and the Bank has financed projects in the relevant sectors following the sectoral studies. For the petrochemicals and electronics sector studies carried out under PSSSP, by contrast, the impact of these two sub-projects on planning is less clear: the paradigm has shifted since these studies were begun in 1989/90 away from centrally planned investments in these productive sectors towards corporatization and decentralized decision-making on investments. In addition, the Bank has ceased to be actively involved in financing in these sectors. So the extent to which these two studies have been useful for long-term planning purposes is difficult to judge. 3.10 A second objective of PSSSP was to prepare and carry out TA activities of an intersectoral, innovative, or experimental nature.12 This objective was partially met. A number of studies and training activities that didn't fit neatly into the other two categories were financed in the spirit of "innovation" or "intersectoral studies", particularly towards the end of the commitment period. By definition, it is a rather ad hoc set of sub-projects, covering poverty alleviation, science and technology development and scientific research, flood control, and quality assurance. While many of the sub-projects in this category seem to have been generally successful in meeting their objectives, there were no clear criteria for selecting them, and little Bank involvement in their formulation and/or supervision. 3.11 The poverty alleviation sub-project, for example, involved disagreement between Bank and GOC staff on the design of the sub-project (in particular, on the scope of the studies to be carried out and on the study tour destinations). On the outcome of the sub-project, there is disagreement between Bank and GOC staff on whether the resulting three poverty-related studies introduced any new concepts, approaches, or analysis. Bank staff disagree with the ICR that the 12 The judgment of this ICR (paras. 2.6) is that this objective was too general and vague. 13 In fact, one benefiting agency interviewed noted that its proposal would have never been approved if it hadn't been near the end of the project's commitment period and there hadn't been pressure to commit the remaining funds. 26 results of the poverty studies were used in designing the Bank-financed Southwest Poverty Alleviation Project, although the sub-project did enable the State Statistical Bureau to build up a database which is being widely used for analysis of the incidence of poverty and was used in the design of other projects aimed at poverty reduction. 3.12 The third objective was to facilitate implementation of GOC reform program through training for GOC officials at various levels. A number of sub-projects started or reinforced the process of building capacity in some key agencies, mainly at the central level. This objective was substantially met. The National Audit Office, for example, started the process of introducing fundamental change in the legal and regulatory framework for auditing, facilitated by the study tours, seminars, etc. financed under its sub-project. Similarly, the SPC introduced a new accounting framework for public finance, a process that was facilitated by the study tours, etc. financed under the project. The People's Bank of China (PBC) and several consulting firms (CICC and CIECC) that provide technical advice to core ministries benefited from study tours, overseas training, seminars and conferences. These sub-projects appear to have had an impact on introducing better methods and techniques in these agencies. 3.13 Overall, PSSSP was characterized by an ad hoc selection of sub-projects, with relatively modest amounts of funds, benefiting about twenty different agencies across many sectors. This was probably not the intention at the time of approval. During implementation, more than half the funds were used by about twelve agencies that had not been identified at appraisal. In addition, toward the end of the commitment period in December 1992, the Bank agreed to an allocation of the remaining uncommitted funds (about US$1.5 million) across a number of on- going sub-projects without any detailed proposals or rationale for the allocation. Some of the sub-projects are of arguable priority (see Box 3.2 below). In several cases, such as the audit sub- project with the National Audit Office, there has been lack of follow up for a potentially important agency. Several sub-projects may have suffered from too little Bank supervision, although GOC officials felt that some sub-projects suffered from too much Bank supervision! Table 3.2: PSSSP: Credit use estimated at appraisal and actual ---Appraisal - ----Actual-- No. of Amount Percent No. of Amount Percent s/projects $m of Credit s/projects $m of Credit Sectoral Studies .121 62 4 f.1 31 Identifiedu 3 9.0 43 2 3.2 16 Not identified 3.7 19 2 3.1 15 TA & Training M 31 15 13- 69 Identifiedu 5 5.0 24 5 6.3 32 Not identified 3.0 14 10 7.6 37 TOTAL 20.7 100 19 20.0 100 Identified at appraisal Outcome by TA Instruments 3.14 On the basis of the information available on actual expenditures by instrument (study tours, overseas training, internal conferences and seminars, consultants, and equipment), 27 relatively little was spent on equipment, and this was mostly office equipment and computers.14 For sub-projects involving mainly sectoral studies and project preparation, most of the expenditures were on consultants. For other sub-projects, most of the funds were spent on study tours, training, conferences and seminars. Even within consultants' contracts, study tours and conferences were often included, so that the actual figures underestimate the total amounts spent on local staff training in various forms. 3.15 Study tours. While some skepticism has been expressed within the Bank and in China about the efficacy ofstudy tours, others have noted that the benefits cannot be underestimated of exposing relevant officials (decision-makers, key implementers) to other systems, methods, etc. The potential payoff is enormous when senior officials decide to adopt or adapt new methods, systems, or concepts of design to project management or economic and financial management. 3.16 In addition, the effectiveness of training, including short-term, long-term, abroad, internal, formal, or on-the-job, appears to have been greatly enhanced by the commitment by Chinese officials to learning. 3.17 External consulants. One area that was, and will likely remain, problematic and complex is the effectiveness of external consultants. First, GOC staff were often reluctant or unable to allow consultants access to data and other relevant information needed for them to complete their tasks (e.g., Yellow River Basin Economic Modeling).i Second, and perhaps related to the first issue, is that the roles and responsibilities of foreign consultants were carefully and sometimes narrowly defined, with GOC staff retaining primary responsibility for the tasks. As a result, consultants (and Bank staff) were sometimes frustrated in what they perceived to be their circumscribed roles. On the other hand, there were no dependency effects of using external consultants: GOC retained full ownership of the results of the work. Third, the difference between salaries of GOC officials and external consultants was so great that GOC officials expected a great deal of their consultants. They were bitterly disappointed if the consultants failed to deliver according to expectations. It is difficult to know whether the expectations were too high, the selection procedure was faulty, or the consultants were simply not up to the task. A number of interviewees in China expressed the view that it was time to rely on Chinese consultants rather than foreign consultants. These issues in the use of foreign consultants are likely to persist. Sustainability 3.18 In all three projects, the sustainability of the benefits is rated as likely. Staff stability is high in most agencies. More important, from the beginning of the sub-projects, and as noted above, the agencies retained responsibility for the objectives of the sub-projects; ownership was strong. For many of the agencies, the activities financed by the credits were only the first step in improving modeling techniques, establishing forecasts as the basis for investment planning, 14No information is available on actual expenditures by instrument for TC 1. According to the ICRs for TC II and PSSSP, expenditures on equipment represented about 5 percent and 25 percent, respectively, of total project costs. These figures are close to appraisal estimates. One consultant heading a large team of consultants reported that for almost all of the three years he worked in a large agency in Beijing, he and his team always met and worked with their Chinese counterparts in rooms below the first floor. During his third year, he was escorted to visit offices above the ground level, and to his knowledge, was the first foreigner to have been permitted to do so. 28 Box 3.1 Examples of relatively successful TA sub-projects On institutional development: Support to Development Research Center (DRC (PSSSP). Various studies by DRC on industrial policy and futures markets were financed, as well as a number of study tours, overseas and local training and in- country seminars and workshops. DRC considered that the support provided under PSSSP was quite effective. Given that DRC is an advisory agency to the State Council, an important decision-making body in China, there could hardly be a more important target for capacity building. In addition, the TA provided under PSSSP closely complemented capacity building in DRC that was assisted through collaboration with the Bank on several studies funded through the Bank's ESW. Support to the National Audit Office (NAO) (PSSSP). As a result of overseas training and study tours and in-country seminars with external consultants, NAO staff drafted a new law, promulgated in 1994, that established the framework for auditing public revenues and expenditures, including those of state owned enterprises. NAO is now drafting 38 further regulations on auditing different categories of public funds, to be followed by more specific guidelines. The NAO credits the sub-project with introducing new audit approaches, methodologies, and procedures, in compliance with international standards, and enabling NAO staff to be trained in new accounting procedures. Bank supervision was frequent early in the sub-project, less so in last two years (1993/94). Staff stability is high (about 90 percent of benefiting staff still in post) and sustainability of results of this sub-project are considered highly likely. On sectoral studies: Y'ellow River Basin Economic Modeling StUdy (PSSSP). The sub-project financed consultants, overseas study tours and training, in-country seminars and workshops for the Yellow River Conservation Commission (YRCC), which produced a series of reports and a water system resource model system. Equally important, the YRCC gathered a full set of data on various aspects of the river basin which will help in planning future investments. The work carried out under the sub-project was closely supervised by Bank staff, and complemented by both a UNDP-assisted study and Bank-executed ESW. The Bank has financed several projects (Xiaolangdi Dam and Resettlement Projects-L3727/C2605) subsequent to the analysis and conclusions of the sectoral reports and studies. Planning of Electrical Power in Eastern China and Sichuan (PSSSP). The sub-project financed consultants to help develop models for peak load forecasting and power source planning for Sichuan and Eastern China, overseas study tour to Japan, and in-country seminars. The resulting studies have been provided the framework for long-term power development strategy and been incorporated into the Eighth and Ninth five year plans. The Ministry of Electric Power judged the results, both in terms of staff training and usefulness of reports for planning, highly satisfactory. A number of Bank projects (e.g., Ertan I and II, L3387/L3933, Sichuan Transmission, L3848, Tianhuangping Hydroelectric, L3606), prepared under TA projects (TC I, TC II, and CRISP) have followed on the basis of the studies carried out under this sub-project. It was closely supervised by Bank staff. changing the regulatory framework, and altering the fundamental approaches to the scope and methods of their work. There was strong evidence in all the agencies interviewed that staff are continuing to make use of the tools and concepts acquired under the sub-projects. At a more a more macro level, the policy reforms and the strong growth performance of the economy over the past two decades have encouraged these agencies to continue to try new approaches. 29 Borrower Performance 3.19 OED agrees with PCR/ICRs on its overall rating of satisfactory Borrower performance under all three projects, with three qualifications: MOF/SPC's overly-bureaucratic procedures, GOC staff's lack of self-evaluation, and GOC absence of aid coordination. First, MOF and, under PSSSP, SPC seem to have become overly-bureaucratic in their approval process for sub- projects. Specific examples include the need for one agency requesting sub-project approval to revise its proposal at least four times before it was approved by MOF. In addition, SPC made what appear to be arbitrary funding decisions. In the case of the Yellow River Basin Economic Modeling Study (sub-project no. 13 under PSSSP), for example, a proposal that had been discussed and agreed with Bank technical staff was made to SPC, for a funding request of US$1.4 million. SPC agreed to finance US$0.4 million, which was totally inadequate. The reasons were never clear. After considerable discussions within China and with Bank staff, SPC agreed to approve only $1.0 million. These discussions took ten months. 3.20 The second qualification relates to an absence of ex-post self-evaluation. Both MOF and SPC, which had roles of approving and monitoring the implementation of sub-projects, found all sub-projects to be equally satisfactory. The OED mission tried to get a sense of the better sub- projects, but MOF and SPC felt that all were equally satisfactory. When asked how the project could have been better designed or implemented, SPC and MOF answered that they wouldn't change anything, except possibly to reduce the role of Bank staff approval in the sub-projects. Benefiting agencies had a similar reaction: all instruments, except some external consultants, were equally satisfactory. If they had to do the sub-project over again, they wouldn't change anything, except in some cases reduce the role of Bank staff approval of the sub-projects. Such absence of self-evaluation makes it difficult to learn from past mistakes and to improve on project design and implementation in the future. 3.21 The third qualification is the vertical segmentation of GOC's bureaucracy and the consequent absence of aid coordination. Ministries in China are self-contained, as are agencies and departments within ministries. As a result, there is little coordination, cooperation, or communication between Ministries or between departments within ministries. Bilateral and multilateral funding and aid are channeled through different agencies: Bank Group activities go through MOF, and UNDP funds go through a department of a different ministry, Asian Development Bank funds go through PBC, and European Union funds and bilateral funds through still other agencies. There are no regular GOC meetings to coordinate aid. Although Bank staff have argued that because the financing and assistance needs are so great in China, the risk of crowding out is non-existent, the lack of donor coordination represents a lost opportunity for synergies and a continued risk of duplication of effort. Bank Performance 3.22 OED agrees with PCR/ICRs on its overall rating of satisfactory Bank performance in all three projects, with two qualifications. First, with respect to the project design of PSSSP, criteria for sub-project selection were extremely broad, and priorities were not established for ID or capacity building. This was a design weakness. 16Donors that have representatives in Beijing (such as the Bank and UNDP) do meet regularly to communicate on matters of common interest, including TA. But real coordination can only come about with Chinese participation and leadership. 30 Box 3.2 Examples of problems in selected TA sub-projects Issues of Bank supervision/quality of sub-projects: Poverty Alleviation (PSSSP). Because of disagreements between GOC and Bank staff on the scope of the studies to be carried out, Bank approval for the sub-project took about eighteen months. This was cited by GOC officials as an example of how differences in viewpoints between the Bank and GOC delayed project implementation. In addition, once the sub-project was approved, there were differences of view over its design (in particular, study-tour destinations). GOC and Bank staff differed in their views on the usefulness of the resulting studies. Institutional Development of the People's Bank of China (PSSSP). there were delays in implementation owing to a lack of attention from higher-level management. Task manager in PBC responsible for implementing the sub-project (study tours, training, in-house seminars, etc.) saw Bank staff only twice: once in 1988 to present the initial sub-project request and once six-and-a-half years later to request an additional allocation. Some evidence training under PSSSP was uncoordinated with that delivered under the larger free-standing financial sector TA, suggesting a lack of coordination within PBC and within the Bank on TA delivery to PBC. Issues of working with external consultants: Yellow River Basin Economic modeling atudy (PSSSP). Consultants hired to help develop a water resource model had difficulties in gaining access to monthly rainfall data; Bank staff had to intervene to get GOC staff to produce data. Grain distribution and marketing project preparation (TC II). Preparation for this sub-project went relatively smoothly. Serious problems developed, however, under the subsequent Grain Storage Project (Loan 3624). GOC agencies resisted accepting foreign technical advice on designs, and this caused major delays in the project. GOC officials in turn complained that the Bank over-emphasized the use of foreign consultants to the detriment of providing training to and/or relying on local professional staff. Problems of repayment of sub-loan to MOF: Development of Audit Capability (PSSSP). Problems of repayment to MOF arose (due at least in part to devaluation of the dollar against the yuan of over 100 percent in nominal terms since 1989 when the sub- project was approved), because of inadequate budgetary resources to repay. The National Audit Office (NAO) converted plane travel to train travel and cut down on office supplies and on communications. These restrictions affected the efficiency with which the NAO could function on a daily basis. In addition, there has been no support to follow up work begun under PSSSP 3.23 Second, and as a result of PSSSP design, supervision was not as thorough across all sub- projects as it should have been.17 During implementation relatively small amounts of funds (on average, about US$1 million) were committed to each agency, across many sectors. The Bank, although it made a concerted effort, provided neither the resources nor the incentives for staff to supervise adequately all sub-project proposals or their implementation, or to evaluate the likely outcome. The Table in the front of the report on Mission Data for PSSSP shows that there were about 30 supervision missions over a seven year period, but many of them were missions of one person for one day to visit one sub-project. While this shows the effort by to supervise the sub- 17The ICR for PSSSP noted that SPC and MOF did not exercise adequate quality control of sub-projects and should have taken a more pro-active role in monitoring and reporting on implementation. The difficulties of adequate supervision were inherent in the design and equally problematic for the Bank and GOC. 31 projects with appropriate expertise, such supervision was highly uneven.18 One interviewee from the implementing department of the benefiting agency reported that he met Bank staff only twice over a six year period. One Bank staff member reported frankly that he abandoned supervision of the sub-project after prolonged discussions with GOC counterparts over its scope and design, mainly because he had neither the incentive nor the resources to continue to supervise it. By contrast, supervision of TC I and TC II involved large input from the relevant Bank sector staff, which they were able to provide by drawing on resources for project preparation: they had both the incentives and the budgets to work closely with GOC counterparts. Key Factors Affecting Implementation 3.24 Initial conditions. TA projects often fail to establish realistic objectives and to achieve satisfactory or sustainable outcomes because the initial conditions are not favorable. The institutional environment is poor and the absorptive capacity for TA is low, frequently owing to governance and systemic problems. They are reflected in turn in underpaid and demotivated civil servants, instability of staff, distorted incentives, poor discipline and weak accountability for performance, rent-seeking behaviors, low commitment to the ultimate objectives of development or the proximate objectives of improving skills and institutional capacities, etc. The failure to recognize and the inability to manage these risks is at the heart of the failure of so much TA. 3.25 Normally, therefore, it is essential that the identification and design of TA projects give considerable attention to these systematic factors, in particular, the incentive system in which the intended stakeholders and beneficiaries operate. In the case of the three projects under review, there was no Mapliit analysis of these factors in the identification/preparation phase. There may have been implicit recognition that the essential ingredients for success existed, or it may have been just good luck that the systemic factors were favorable. 3.26 In any event, the success of these three projects is largely explained by two main factors: motivation and accountability. These include the strong demand of top GOC officials for improving capacity and introducing reforms, the strong demand among beneficiaries for improving skills and institutional capacities, the relative stability of staff, and the discipline of the civil service. 3.27 In terms of motivation, the fact that GOC was willing to borrow for TA funds under TC I was considered by the Bank to be significant: up to that point (1983), all project preparation as well as training and seminars, policy conferences, and policy studies had been on a grant basis (UNDP, EDI, ESW). TC I was therefore a sign of the importance that GOC, at least at a high level, attached to ensuring adequate project preparation and the first sign that GOC was willing to pay for external TA. In addition, at a lower level, there seemed to be genuine strong demand throughout the agencies for external TA. The fact that agencies were willing to borrow funds to finance these activities is only one indication of the interest in using external TA. By contrast, the same table on Mission Data shows a number of non-formal supervision missions with multi-person, multi-day input, in particular for the Power and Yellow River sector studies. The prospect of using the findings of the sector studies as a basis for establishing lending priorities was clearly an incentive for both GOC and the Bank to allocate appropriate resources to these sub-projects. 32 3.28 One explanation for this demand may have been the fact that most Chinese officials were essentially closed off from contact with the rest of the world for more than twenty years, and there was enormous pent-up demand to discover other techniques, technologies, approaches, and systems.19 It also may be that the rapidly growing economy created a positive environment for introducing changes: staff could have confidence that new approaches and policies were working. 3.29 These factors are consistent with analyses of the "ownership" question in various studies. In a 1993 World Bank Discussion Paper on the ownership of adjustment programs, the authors identify four variables that define ownership: the locus of the initiative for the adjustment program, the level of intellectual conviction among key policymakers, the expression of political will by top leadership, and the efforts toward consensus building among various constituencies. China would rank high on all of these measures in terms of measuring its ownership of the activities supported by the TA projects under review. In another World Bank study in 1995, three conditions were identified as being necessary for successful state owned enterprise reforms: ?,olitical desirability, political feasibility, and credibility (political promises must be believable). In the case of these three TA projects, the reforms and changes introduced or supported by the activities financed were not nearly as contentious or threatening as state enterprise reform could be, and it can be argued that all three conditions identified in the study would apply to the various techniques, methodologies and approaches being introduced. 3.30 In terms of accountability, Government officials in China tend to follow instructions from above in a highly disciplined manner. Once the top leadership decided that funds were to be used for TA to ensure high-quality project preparation, it could be expected that staff within line ministries and various agencies would implement this decision. Such discipline was reinforced by the genuine demand for TA. In addition, staff turnover was relatively modest which added both to the ability to hold staff accountable for the use of the TA funds and to the likely sustainability of the efforts at capacity building. Finally, GOC staff seem to take a relatively disciplined approach to spending funds, and there was little evidence of rent-seeking under the three projects. MOF believes that this discipline was due to the fact that the funds usually had to be repaid to MOF, but OED believes that such discipline is a general characteristic of the Chinese civil service. Conclusions 3.31 The value of all three TA projects should not be underestimated in the role they played in providing China with a "window on the world" at a relatively early phase of opening up. To some extent, the "success" of the TA projects is more than the sum of its sub-projects: in many of the benefiting agencies, exposure to new modeling techniques, new forecasting models, modern designs (in the context of project preparation), Western accounting and auditing 19In fact, during internal Bank discussion on TC II in late 1985, the idea was examined of broadening the scope of the project beyond project preparation; Bank staff reported that MOF "is reluctant to broaden the use of the [project] to prevent a flood of requests for all types of technical assistance." 20 From "Borrower Ownership of Adjustment Programs and the Political Economy of Reform", John H. Johnson and Sulaiman S. Wasty, World Bank Discussion Paper No. 199, 1993. 21 Bureaucrats in Business: The Economics and Politics of Government Owneship, World Bank Policy Research Report, 1995 33 procedures, new paradigms for the macroeconomic policy-making, for fiscal and monetary policy, etc., have had an immeasurable but potentially enormous impact. In some cases, such as the PBC, sub-projects have begun a process that has been continued on a larger and more intense scale under other Bank or donor-financed projects. 3.32 One interesting question to consider is the counterfactual: what would have happened if these three projects had never been implemented? Given the apparent funding and expertise gaps for project preparation, it can be argued that without TC I and TC II, project preparation would have been delayed: either the activity would have simply not taken place, or the Chinese would have tried to prepare the projects themselves, with consequent further delays as modifications would have been made to the feasibility studies and designs for the projects. The lending pipeline would have been constrained, investment would have been delayed, and the policy dialogue accompanying the lending would have developed much later. ID for project preparation and implementation would have been delayed or simply not occurred. The same can be argued for the sector studies and ID that took place under PSSSP: most of the sub-projects probably would not have been implemented, with the exception of agencies that had access to other foreign funds, including the PBC that later benefited from a much larger free-standing TA loan aimed at ID. Benefits accruing from the investments and ID would have not been realized at all, or on a more modest scale, or with considerable delay. 3.33 Finally, the question of the fungibility of funds must be addressed: how did funding for these three projects allow GOC to finance other projects and what was the rate of return likely to have been for these additional projects? While it is impossible to answer this with any degree of certainty, there are two important considerations to this question in the Chinese context: (a) vertical segmentation within the Chinese bureaucracy; and (b) enormous absorptive capacity of China for TA and investments. First, as noted above, Ministries and agencies within ministries are notably segmented, with little coordination or even communication among them. This constrains resource fungibility: if Bank Group funding for TA is unavailable to the TATD in the World Bank Department (WBD) in MOF, it is unlikely that the TATD will have access either to increased budget from MOF or to donor funds that are available to other ministries or 22 agencies. In this sense, Bank Group funds for TA truly are incremental for TATD and do not release funds to be used elsewhere. Second, even if the counterpart funding for the externally financed activities is fungible, Bank staff argue convincingly that the absorptive capacity in China is still so great that even most of the projects at the margin, to be financed only if additional funds are mobilized, are likely to have a satisfactory rate of return.23 22 With the exception of Japanese grant funds available for project preparation and some UNDP funds that have been Bank-executed. 23 This argument has to be tempered, however, with the recognition that GOC is investing in several large infrastructure projects which the Bank does not necessarily believe are technically optimal or economically justified, such as Three Gorges Dan.  35 4. Selected issues When is a process approach for TA warranted? 4.1 The process approach is generally a second-best solution to providing TA. It is the preferred solution when a blueprint approach can't be done: where there is instability, uncertainty, rapidly changing country circumstances, lack of knowledge by the Bank on the details required for a blueprint, and/or a need for "learning by doing". A process approach is best suited to a situation where flexibility is required to deal with contingencies, both positive opportunities that may come up and negative setbacks. In the case of China, a process approach to the TA projects was appropriate given the circumstances. 4.2 The downside of using the process approach is the risk that flexibility becomes a weakness rather than a strength. Advocates for low priority sub-projects crowd out the higher priority ones because of connections or political clout; rent-seekers clamor after "toys and tours"; allocations get made through a patronage system, "leakages" come from corruption, and so forth. The important question in using a process approach is how avoid these risks. The answer lies in ensuring that there is sufficient discipline in the process, through ensuring that the initial conditions exist in the country and through project design (clearly established objective criteria for project selection, requirement for sound justification and monitorable indicators of inputs, output and impact for sub-project proposals). 4.3 In the case of China, the risk of misuse of the TA funds was relatively low because of: (a) strong motivation of the leadership, the managers of implementing agencies, MOF, and individual beneficiaries to use the project for the "right" things, i.e., gaining and adopting the relevant techniques, modeling methodologies, policy tools, regulatory framework, and approaches; and (b) strong accountability/discipline: rules, regulations, norms, and monitoring that made it difficult for rent-seekers to corrupt the uses of the resources to unproductive ends. 4.4 The lesson from the experience of these three projects is that China is a special case of having both the strong motivation and the sanctions/restrictions of a disciplined bureaucracy in place. Given this, the process approach and the potential weakness of the project design, particularly for PSSSP, proved not to be major problems. The TA projects were successful, in spite of their process approach, largely because the initial conditions were right. The Need for Defining Priorities for ID/TA versus Umbrella Projects 4.5 Having concluded that the process approach to the three TA projects was the right one for China, it does not follow that the umbrella design, spanning many sectors, was also justified. Umbrella projects, by definition, lack focus and a strategic thrust. The risk of an ad hoc collection and sub-optimal quality of sub-projects is high. Umbrella projects are extraordinarily demanding on staff time and resources in both the client country and the Bank. Under normal circumstances, it would be impossible to harness the various types of expertise needed in sufficient quantities to supervise adequately a large number of relatively small sub-projects. 4.6 Under TC I and TC II, however, it can be argued that it was necessary to allow for preparation of projects across sectors. Given that the projects were to be prepared primarily if not uniquely for Bank Group financing, there was strong incentive on both GOC and the Bank 36 side to ensure quality work. In addition, the Bank provided far greater resources for supervision than it would have otherwise, because it had the resources under "preparation" budgets to do so. The same cannot be argued for PSSSP, and the outcome, while satisfactory overall, reflects an ad-hocery and variable quality of sub-projects that umbrella projects tend to suffer from. 4.7 For the future, the need for TA should be determined by the reform agenda and priorities for economic development of GOC, and an analysis of where the constraints to that agenda are likely to be greatest for institutional reasons. Delivery of future TA should be based on a strategy agreed between GOC and the Bank as to where ID and capacity building are likely to have the greatest impact. As a result of the agreed strategy, future TA should be focused on a selected few priority areas. The criteria for sub-project selection, if it is left to a process approach, should reflect those priorities. Umbrella projects, spanning many sectors or geographical areas, should be avoided: the amount of funds will be too modest, the subjects or areas too diverse, to ensure adequate profession dialogue on quality control, problem solving, and supervision. On-lending IDA funds for TA to benefiting agencies 4.8 Description. MOF on-lends IDA funds for TA to benefiting agencies. Under TC I and TC II, the funds were to be passed on as a grant, except where the funds were used for project preparation. Because over 90 percent of both credits were used for project preparation, in practice, virtually all of the funds were on-lent. The repayment terms were not defined in the DCAs for TC I and TC II, but according to MOF, they included a service charge of either zero 1.5 percent per annum, with a six year repayment period, including five years of grace and full repayment in the sixth year. Under PSSSP, by contrast, the DCA specified the on-lending terms 24 at twenty years, including five years of grace, and a service charge of one percent per annum. MOF gave the OED mission written information that the on-lending terms under PSSSP were as specified in the DCA. Both the principle and practice of on-lending funds for TA raises a number of issues. 4.9 On the positive side, MOF believes, and the Bank has implicitly agreed, that the requirement for repayment ensures discipline and accountability in the use of the funds. By requiring agencies to reimburse MOF, the agencies will ensure that their proposals are both commensurate with their needs and their ability to repay; they will be conscientious in the use of the funds and careful to avoid unnecessary expenditures for overseas study tours, training abroad, purchase of equipment, and so forth, areas where there would be temptations for abuse. Elsewhere this report has argued that discipline and accountability within the civil service are key factors in the successful outcomes of these projects. To the extent that the requirement to repay ensures discipline, this policy should be a positive force in ensuring the success of the projects. 4.10 This argument implies, however, that if the funds had been available on a grant basis, they would have been less well-used. Based on the experience of the Bank as executing agency 24The rationale for the different on-lending terms was not clear. In particular, the requirement for full loan repayment in the sixth year by an agency that had prepared a project seems particularly onerous. The project to be implemented would presumably only start generating funds (say, for a power plant) some six to eight years after the project was effective, and thus some eight to ten years (or more) after project preparation had begun. 37 for UNDP and Japanese bilateral funds that are available as grants, there is no evidence to suggest that grants are less well-used by implementing agencies in China than funds that are available only as loans. Incentives other than repayment requirement can be built into the sub- project design to ensure accountability and discipline in the use of TA funds.25 4.11 On the negative side, on-lending distorts the selection of eligible agencies in favor of those agencies that are either revenue earning and likely to increase their revenues as a result of the TA, or agencies that are reasonably assured of being able to increase their budget allocation from MOF for repayment. An indication that this in fact happened under TC I and TC II can be seen by analyzing the sectoral distribution of the use of the credits. There is a much heavier use of the credits by the energy, power, and industry sectors (about 65 percent of the credits), compared to their proportion of the lending program over the 1984-94 period (about 25 percent). Sectors such as education and health are virtually absent from the use of TC I and TC II, yet accounted for 7 percent of the lending program during the relevant period. 4.12 TA that is well-used involves considerable externalities. The ultimate beneficiaries are the Chinese citizens (for flood control, poverty alleviation, tax reform, planning for efficient power development, etc.), and not the borrowing agency. The agency's ability to repay the loan for TA in no way captures these benefits and is an unsuitable criterion for selection. Yet by imposing this requirement, potentially important beneficiaries are discouraged from applying for the funds. The policy thus prevents the selection of those agencies that should have priority access to the funds. As China shifts from an IDA blend country to IBRD, and TA funds become available only on IBRD terms, the on-lending policy will become even more problematic in terms of its distortionary effect on the selection of benefiting agencies. 4.13 The World Bank Department in MOF argues that it is required to be financially self- sufficient. Thus, it cannot afford to provide grants as it has no means of financing them. This is a narrow approach to budgeting, which ignores the fact that many of the benefiting agencies can repay only by receiving an increased budget allocation from MOF to do so. Thus one department in MOF is increasing an agency's budget so that it can repay another department within MOF. 4.14 In addition, MOF has passed IDA funds for TA under these three projects on harder terms those ofIDA. This has been explicitly agreed to by the Bank. The IDA credits for the three projects under review had fifty year repayment periods, including ten years of grace; the repayment terms to the agencies are up to only twenty years, and were harder in several other dimensions as well.26 There is no obvious rationale for the mismatch. A calculation in Annex 5 on the return to MOF on PSSSP inflows and outflows indicates that MOF makes a rate of return of some 28% on this mismatch, even allowing for 10 percent defaults and 10 percent arrears of 25 For ID, these can include, for example, providing modest amounts of funds under a first phase, establishing realistic performance indicators on inputs and outputs, and, if the performance indicators are met, providing additional funding under a second phase. For project preparation, the incentive is built in: to get the project financed. 26Repayment periods vary from six to twenty years. The service charge varies from zero to 1.5 percent, compared to IDA's service charge of 0.75 percent. The agencies must repay in dollar-equivalent, so they bear the dollar-yuan exchange risk. MOF repays in SDRs, and so takes the dollar - SDR exchange risk. IDA's commitment fee for undisbursed funds has effectively been zero for a number of years, and MOF does not charge a commitment fee to the agencies. 38 rates, the rate of return is still 28%. This does not include the Technical Assistance and Training Division's operating costs, but they would be spread over a number of TA projects and would account for very little in the calculation on a given project. 4.15 Finally, some agencies have trouble getting incremental funds allocated by MOF to reimburse the funds. One agency reported having to reduce its operating budget, affecting mode of travel, availability of supplies, etc., in order to find the funds to repay. This in turn affects the operating efficiency of the agency, an agency whose TA was designed to improve its effectiveness. Thus the very purpose of the TA is defeated by this repayment requirement. 4.16 In conclusion, MOF should change its on-lending policy for TA funds, and examine the possibility of requiring repayment only of revenue-earning entities, where the TA is designed to increase its revenue earning capacity and where the entity is unlikely to be eligible for normal, commercial bank credit. If it does not, demand for TA from agencies with potentially high returns may trickle to insignificance once China becomes an IBRD-only borrower. Characteristics of on-going TA projects 4.17 This PAR examined the extent to which the issues discussed above were dealt with in the on-going TA projects, which were approved between FY93 and FY95, a number of years after the three TA projects under review. The Table below summarizes some of the characteristics of these five projects. Table 4.1 Characteristics of on-going TA projects Project Loan/Credit Process Umbrella MOF on-lending Performance No. approach style" terms Indicators Financial Sector C 2423 no yes grant yes TA Project CRISP Project C2447 yes yes loan/grant yes Environmental C2522 partial partial2' loan (most) partial TA Project Economic Law C2654 yes yes grant (most) partial Reform Project Fiscal TA C2709/L3873 no no grant yes Project Defined here as covering six or more separate departments or agencies. 2' In the environmental project there were only two main implementing agencies, but many self-contained sub-projects to be implemented by various departments within the agencies. 4.18 With the exception of the Reform, Institutional Support and Preinvestment Project (CRISP), the other four projects are more narrowly focused, either on a sector, such as the Financial Sector and Fiscal TA projects, or on a theme, such as the Environmental and the Economic Law reform TA projects. Nevertheless, all these projects except the Fiscal TA project use an umbrella approach involving a large number of agencies or departments within agencies. This is likely because of the temptation to address as many problems and constraints as feasible 39 within a given sector, to use an integrated approach to the sector, and to try to achieve synergies across agencies working in the same sector. Supervision reports and task managers report difficulties in supervising such complex projects. 4.19 Second, at least part, if not all, funds under the various TA projects are being passed on as grants. This amounts to de facto recognition that on-lending is not always the appropriate policy for funding TA. Nevertheless, the on-lending issue has not been systematically addressed so as to establish sensible, objective criteria for choosing between the two forms of financing TA. 4.20 Finally, all the on-going TA projects have made efforts to define performance indicators, some more thoroughly (such as the Financial Sector Assistance TA project) than others. This represents an improvement over the three projects reviewed in this PAR. u 41 5. Lessons and Recommendations The most important factors determining the success and sustainabiity of these TA projects, and of most successful TA projects, are motivation to obtain know-how and discipline to minimize rent-seeking. 5.1 When these are strong, as they were in China, the likelihood of success is strong, even where there are design weaknesses, as there were particularly for PSSSP. In China, the motivation for acquiring know-how in various forms (modeling, technical designs, forecasting, accounting standards and practices, auditing, economic decision making, etc.) was (and is) strong. In addition, there was strong accountability within the Chinese bureaucracy for meeting objectives and strong financial discipline in the use of the funds. OED found minimal evidence of rent-seeking or obvious waste, and the funds appear to have been, for the most part, well spent. These two factors, ownership and discipline, are essentially "initial conditions", which need to be carefully assessed, preferably in a risk/return framework, before proceeding with a TA project. Their absence cannot be compensated by project design, while their presence reduces considerably the risks of failure. The process, or flexible, approach to providing TA is a second-best solution, when a blueprint approach cannot be done because of uncertainties and changing conditions. 5.2 All three of these projects involved a process, or flexible, approach that was appropriate, given the uncertainties of the demand for TA and of a changing environment. Nevertheless, there are many risks of using such an approach, including misuse of funds and rent-seeking. China may represent a special case where the initial conditions discussed above were such that these risks were minimal. The process approach was and still is appropriate for China, given continued uncertainties and the desirability of ensuring that a project aimed at TA has the ability to respond to needs of policy makers. Nevertheless, it does not follow that TA projects using a process approach should be umbrella in scope, covering many sectors and implementing agencies (see below). In addressing the needs for capacity building in China, the Bank and GOC should adopt a more strategic approach to identifying ID priorities, identfying where the institutional constraints to economic development and continued reform are most binding. 5.3 The criteria for sub-project selection should reflect those priorities and define clearly and objectively the areas where TA will be provided. The areas identified for strengthening should be limited under any given TA project. Umbrella projects, such as PSSSP turned out to be, that span many sectors, implementing agencies, and specialties will suffer: supervision, both within the country and by the Bank will be inadequate, and the quality of sub-projects uneven.27 There is also the risk that the impact of modest amounts of funds spread thinly over many agencies will be equally modest. 27 Funds for project preparation are an exception to this generalization. Relatively small amounts of funds are required per sub-project, and the incentives and resources within the Bank for strong supervision are adequate. 42 Once the overall strategy for ID and the priorities for TA have been agreed, it is equally important to establish the ID priorities of the organizations and institutions to be included in the project and to establish ex ante monitorable performance indicators, for inputs, output, and Impact 5.4 None of the projects evaluated in this PAR had an explicit analysis of the ID priorities of the agencies identified at the time of approval, nor had explicit performance indicators with respect to ID. As a result, it was difficult for the evaluation mission to assess whether the training provided under the projects addressed the most important ID issues in the most cost- effective way in the various sub-projects or whether the objectives of the sub-projects were the "right" ones for the agency in the sense of being the most important ones.28 Succeeding TA projects have generally had monitorable indicators for the participating agencies and this respect have already incorporated this lesson into their designs. The policy of MOF to on-lendfunds for TA is distortionary and should be re-examined 5.5 OED realizes that this is a country-wide issue and that MOF's policy of on-lending in general has repercussions for Bank Group operations well beyond that of TA projects. Nevertheless, on the basis of the analysis for these three TA projects, this evaluation concludes that the policy discriminates against potentially important agencies in favor of those that have repayment capacity. TA activities have large externalities, where the ultimate beneficiaries are outside the benefiting agencies. Using repayment capacity as one criterion for selection distorts the allocation of funds away from potentially priority agencies to agencies that are revenue generating or that have particular favor with MOF and will thus be able to increase their budget allocation so as to repay the loan. With a switch to IBRD funding for TA, the distortions in the allocation of TA funds will only increase. MOF should alter its policy of on-lending funds and consider requiring repayment only from revenue-earning agencies. The Bank and MOF should agree ex ante on the portions of the credit/loan that will be passed on as grant and as loan, and the Bank should monitor that the agreement is adhered to. Where MOF does on-lend, it should do so in a transparent manner, passing on the funds on IDA or IBRD terms, with a margin to cover its administrative costs and the possibilities of defaults. 5.6 MOF on-lending terms are different, and harder, than IDA terms. If defaults and arrears are not high, MOF will have a positive net cash flow from the mismatch and a high rate of return on the use of the funds. There is no clear rationale for this. 28 Except for project preparation activities under TC I and TC II, where Bank Group lending resulting from the preparation activities was examined, for size and quality. 43 Annex A Page 1 of 7 Basic Data Sheets TECHNICAL COOPERATION PROJECT (CREDIT 1412-CHA) Cumulative Estimated and Actual Disbursements FY84 FY85 FY86 FY87 FY89 FY90 FY91 Appraisal estimate (US$M) 0.5 3.0 6.0 9.0 10.0 Actual (US$M) 0.2 1.4 6.5 8.3 9.5 9.8 10.9* Actual as % of appraisal 40 47 108 92 95 98 109* * Devaluation of the dollar against SDR caused amount disbursed in dollar equivalent to exceed appraisal amount Project Dates Original Actual Project Brief 2/5/83 Issues Paper 4/13/83 Decision Memorandum 5/2/83 Appraisal 3/83 3/83 Credit Negotiations 7/83 7/83 Board Approval 9/83 8/24/83 Credit Signature 10/83 10/17/83 Credit Effectiveness 12/83 12/31/83 Credit Closing 12/31/87 12/31/88 Final Disbursement 6/30/87 7/12/89 Staff Inputs (staff weeks) FY83 FY84 FY85 FY86 FY87 FY88 FY89 Through Appraisal 7.0 Appraisal through 1.1 4.2 Effectiveness Supervision 14.6 21.3 8.7 6.1 5.4 6.4 Annex A 44 Page 2 of 7 Mission Data Date No. of Staff days in Performance 7)pes ofproblems (month/year) persons field rating* Through Appraisal 3/83 2 13 1 none Appraisal through 10/83 1 12 1 none Effectiveness 11/83 1 Supervision 3/84 2 13 1 none 10/84 2 14 1 none 6/85 1 11 1 none 6/86 1 11 1 none 9/86 1 4 n/a 3/87 1 12 1 none 10/87 2 13 1 none 6/88 2 12 1 none Completion 2/89 1 5 * Performance Rating: 1 - Problem Free/Minor Problems; 2 - Moderate Problems; 3 - Major Problems Other Project Data Related IDA Credits: Other Free-Standing Technical Assistance Credits Operation Credit no. Amount Board Status (US$ m) date Technical Cooperation Project C1412-CHA 10.0 8/24/83 closed Second Technical Cooperation Project C1664-CHA 20.0 3/13/86 closed Planning Support and Special Studies C1835-CHA 20.7 6/30/87 closed Financial Sector Technical Assistance C2423-CHA 60.0 9/29/92 active Reform, Institutional Support and C2447-CHA 50.0 12/15/92 active Preinvestment Environmental Technical Assistance C2522-CHA 50.0 6/22/93 active Economic Law Reform TA Project C2654-CHA 10.0 10/18/94 active Fiscal Technical Assistance L3873-CHA 25.0 4/ /95 active C2709-CHA 25.0 active 45 Annex A Page 3 of 7 SECOND TECHNICAL COOPERATION PROJECT (CREDIT 1664-CHA) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Project Costs 27.0 31.1 115 Cumulative Estimated and Actual Disbursements FY87 FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 Appraisal estimate (US$M) 2.0 7.0 13.0 18.0 20.0 Actual (US$M) 1.9 3.1 4.5 4.5 8.1 11.2 16.1 23.1* 24.1* Actual as% of appraisal 95 44 35 25 41 56 81 116* 121* * Devaluation of the dollar against SDR caused amount disbursed in dollar equivalent to exceed appraisal amount Project Dates Original Actual Project Brief Issues Paper Decision Memorandum Appraisal 12/85 Credit Negotiations 2/86 3/86 Board Approval 3/4/86 3/13/86 Credit Signature 4/86 Credit Effectiveness 7/86 7/86 Credit Closing 6/30/92 6/30/94 Final Disbursement 10/19/94 Annex A 46 Page 4 of 7 Staff Inputs (staff weeks) FY86 FY87 FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 Through Appraisal 1.1 Appraisal through 10.1 Effectiveness Supervision 3.9 14.3 6.5 5.6 5.0 5.4 4.8 3.6 Completion 7.8* * of which 0.4 staffweeks were spent in FY96 Mission Data Performance rating* Month/ No. of Staff days in Implement. Develop. Types ofproblems Year Persons field Status Status Through Appraisal Appraisal Through 5/85 1 Effectiveness Supervision 2/87 1 17 4/88 2 10 2/89 2 8 8/89 2 1 1 3/90 2 2 1 7/90 2 2 1 11/90 1 10 2 1 7/92 2 2 1 3/93 2 11 2 1 11/93 1 20 2 1 8/94 1 16 * Performance Rating: I - Problem Free/Minor Problems; 2 - Moderate Problems; 3 - Major Problems Other Project Data Related IDA Credits: see chart under Key Project Data for C1412 47 Annex A Page 5 of 7 PLANNING SUPPORT AND SPECIAL STUDIES PROJECT (CREDIT 1835-CHA) Key Project Data (amounts in US$ million) Appraisal Actual or estimate current estimate Local Foreign Total Local Foreign Total Project Costs 11.7 17.9 29.6 10.0 20.0 30.0 Cumulative Estimated and Actual Disbursements FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 Appraisal estimate (US$M) 1.2 5.2 10.2 15.2 19.2 20.7 20.7 20.7 Actual (US$M) 1.5 2.1 3.1 6.4 9.7 13.1 15.9 20.0 Actual as % of appraisal 125 40 30 40 50 63 77 89 Project Dates Original Actual Identification 6/86 Preparation 2/87 2/87 Appraisal 3/87 3/30/87 Credit Negotiations 4/87 5/22/87 Board Approval 6/87 6/30/87 Credit Signature NA 9/14/87 Credit Effectiveness 9/30/87 12/11/87 Credit Closing 6/30/92 12/31/94 Final Disbursement 12/31/92 1/26/95 Staff Inputs (staff weeks) FY87 FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 FY 96/97 Through Appraisal 33.1 Appraisal through 7.3 Effectiveness Supervision 20.0 14.1 15.6 6.4 2.9 5.8 3.0 3.0 0.5 Completion 3.5 1.5 Annex A 48 Page 6 of 7 Mission Data Performance rating* ** Month/ No. of Staff days in Implement. Develop. Status Year Persons field Status Through Appraisal 3/87 Appraisal Through 1 10/87 2 7 1 1 Effectiveness Supervision** 2 4/88 2 10 1 1 6/88 2 5 3 2/89 4 8 1 1 3/89 7 I 5/89 2 5 4 8/89 2 6 1 1 9/89 1 I 2/90 1 2 3/90 1 2 3/90 1 1 5 3/90 4 8 2 1 4/90 2 4 6 7/90 2 5 2 1 10/90 1 3 11/90 2 4 7 3/91 1 5 2 1 5/91 13 10 10/91 1 1 11/91 1 1 1/92 2 ? 8 3/92 2 3 2 1 9 7/92 1 10 2 1 10/92 4 5 10 2/93 3 5 2 1 4/93 1 1 5/93 2 15 11/93 1 1 2 1 11 12/94 1 6 S S * Performance Rating: 1 - Problem Free/Minor Problems; 2 - Moderate Problems; 3 - Major Problems ** Supervision missions that reviewed overall project progress denoted with number before date, all other supervision missions reviewed progress on specific sub-projects. 49 Annex A Page 7 of 7 Other Project Data Related IDA Credits: see Key Data Chart for C1412  51 Annex B Page 1 of 4 Sub-KOj0cts financed under TC I (Credit 1412-CHA) and follow-on projects financed by IDAIBank Amt. Amt PAR: ICR Bank4kVM~ S US$ m IDABRD Pod ACRoRn # US$ m outcome Outcome Prep sector 1 Dalian Bulk Grain Terminal 0.09 A 2 Karamay Petrol. Proj. Prep. 0.00 E 3 Rural Hth. & Med. Ed. Prep. 0.17 Rural Health & Med. Educ. C1472 8.0 S 8/8 0 4 Water Supply Proj. Prep. 0.11 Water Supply C1578 80.0 S S T 5 Motorway -Beijing -Tanggu 0.05 Motorway - Beijing to Tanggu C1792 125.0 T 12811 25.0 8 Maching Tools Proj. Prep. 0.98 ShanghaI Machine Tool L2784 100.0 S 81 I 7 Special Board of Consultants 0.20 Lubuge Hydroelectric Power L2382 145.4 S 5/8 E 8 Transmission Proj. Prep. 0.57 Second Power L2493 117.0 S S/S E 9 Forestry Pro). Prep. 0.05 Forestry Development C1605 47.3 S S Ul- A 10 Second Railway Proj. Prep. 0.11 Second Relway 12540 235.0 S T 11 Seed Proj. Prep. 0.07 Seeds Project C1577 40.0 8 S/S A 12 Asst to State AuditAgency 0.18 0 13 Port Master Planning, 0.09 T Feasiblity Study Sem. 14 Port Terminal Control 0.11 T System Training 15 Fertilizer Proj. Prep. 0.21 Fertilizer Rehablitation L2541 97.0 8 S/S I 16 Phosphate Proj. Prep. 2.00 Phosphate Project L2958 62.7 S SHS 17 Xinaha Terminal In 0.24 Huangpu Port Project C1845 25.0 S S/S T Huangpu Port 12877 63.0 S S/S 18 Bellungang Therm. Power 1.83 Bellungeng Thermal Power L2955 235.0 S S/S E 19 Missing Links Road Proj. 0.00 I 20 Pihthang-Chaohu Area 0.01 PIshlhang-Chaohu Area Ovip. C1606 75.0 S A Develop. Proj. L2579 17.0 S 21 Tianjin Port East Pier Dev. 0.56 Tianjin Port Project L2089 130.0 8 S/S T 22 Shulkou Hydroelectric 1.00 Shulkou Hydroelectric Power L2775 140.0 HS HSHS E Power Project 23 Yantan Special Board 0.12 Yantan Hydroelectric L2707 52.0 S S/S E 24 Economic and Finance 0.46 0 Pub. House 25 Freshwater Fisheries Proj. 0.05 Freshwater Fisheries C1689 60.0 HS A 28 Red Sol Project 0.04 Red Sol Area Development C1733 40.0 S 8 S/S A 27 Zinjiang Area DvIp. Proj. 0.06 Xlnjiang Area Development C1764 70.0 S HSIS A 28 Pharmaceuticals Project 0.07 Pharmaceuticals L2943 127.0 S S/S I 29 XIsmen Port Project 0.00 T B-1 Ertan Hydroelectric 1.04 Ertan Hydroelectric L3387 380.0 (aeeraigswuerTCIIchme E 8-2 Three Gorges Expert Panel 0.37 E Total: 10.65 2573A A * Aiclln, Fishades. and Area DwAlopmnt E - Energy/Power I * Industry T - Inrastucture 0 Otter Annex B 52 Page 2 of 4 Sub-projects financed under TC II (Credit 1684-CHA) and follow-on projects financed by IDA/Bank Amt. Amt ICR Bank4Barower Form 500 Sub-Ptfjed USSm IDA/BRD Prjec CRoLoan 0 USSm Outcame Prep Overa Status sector 1 Three Gorges Expert Panel 0.10 E 2 Reg. Cement Proj. Prep. 0.56 Regional Cement Industry L3443 82.7 8 I 3 VW|ng Thermal Power Proj. 0.95 Power VI Project L2852 190.0 HS S/S E 4 Shanghai Sewerage Proj. 0.18 Shanghai Sewerage Project L2794 45.0 S S/S U 5 Dalian Port Proj. Prep. 0.32 Dallan Part Project L2907 96.0 S SIS T 6 Ertan Hydroelectric Proj. 1.47 Ertan Hydroelectric Project L3387 380.0 HS E 7 Phosphate DvIp. Proj. Prep. 0.00 1 8 ShanghaI Indust. Dvip. Proj. 0.00 I 9 Shanghai Port Master Plan 0.20 Shanghai Port Restruc. & Dvip. L3552 140.0 S T 10 Henan Pharm. Quality Control 0.80 I 11 Jiangau Prov. Indust. Proj. 0.08 I 12 Strengthening CPI's Planning 0.34 0 Capabiltie 13 Medium Cities Project Prep. 0.38 Med. size Cities Dvip. L3286 168.4 HS U 14 Yanshl Power Proj. Prep. 0.42 Yanshi Thermal Power L3433 180.0 HS E 15 Tianjin Provincial Loan Prep. 1.97 1 16 Daguanba Hydropower 0.16 Daguanba Multipurpose L3412 67.0 U E Irrigation Project 17 Liaoning Urban Infrastr. Proj. 0.40 Liaoning Urban Infrastructure C2219 77.8 HS U 18 Zouxian Thermal Power Proj. 0.70 Zouxian Thermal Power L3462 310.0 S E 19 Xlaolangdil Multip. Dam Proj. 2.91 Xiaolangdi Multipurpose Dam L3727 460.0 S E 20 Jiangsu Provinc. Trans. Pro. 0.23 Jiangsu Provinc. Transport L3316 100.0 S T 21 BeVing Environ. Proj. Prep. 0.36 Beijing Enviromental Proj. L3415 125.0 S En 22 Henan Agricult. Dvip. Proj. 0.06 Henan Agricultural Dvip. C2242 110.0 S A 23 Zhejiang Multi Cities Dvip. Proj 0.38 Zhejiang Multi-Cities Dvip. C2475 110.0 S U 24 ZInjinag Tarim Basin Irrigation 0.00 U 25 Liaoning Environ. Proj. Prep. 1.00 Liaoning Environment Proj. L3781 110.0 S En. 26 Jiangsu Chemicals Proj. Prep. 0.00 1 27 Tianhuangping Hydroelec. Proj 2.63 Tianghuangping Hydroelectric L3606 300.0 HS E 28 Sichuan Gas DvIp. Proj. Prep. 1.68 Sichuan Gas Dvip. L3716 255.0 S E 29 Jiangsu Envir. Protection Proj. 0.35 Jiangsu Environ. Protection L3582 250.0 S En 30 Dvip. of Construction Industry 0.50 31 Regional Cement Proj. Prep. 0.00 32 Tianjin Indust. Dvip. Proj. Prep. 0.82 Tianjin Industry II L3572 150.0 S I 33 Changchun Water Supply and 0.93 Changchun Water & Environ. C2457 120.0 8 U Environmental Proj. Prep 34 Chinese Ecological Research 0.07 Environnment TA C2522 50.0 S En Network Project 35 Study - Dvip. of Securities 0.20 0 Trading Automated Quotations Syst of Stock Exch. Exec. Council 0 36 ShanghaI Environ. Proj. Prep. 0.06 Shanghai Environment L3711 160.0 S En 37 Grain Dist. and Mkting. Pro]. 2.97 Grain Distr. & Marketing L3624 490.0 U A Total: 24.16 4626.9 A - AgricLiukre En Enrvmnment E = Energy/Power U = Water/Sewerage/Urban Development I= Industry O= Other T Iriastructure 53 Annex B Page 3 of 4 Sectoral distribution of Sub-Projects financed under TC I (Credit 1412-CHA) Sector No of % of total Disbursement % of total Sub-Projects number US$ equivalent Disbursement Agriculture 7 24 0.37 3 Energy/Power 6 21 3.72 34 Industry 5 17 3.29 30 Infrastructure 8 28 1.24 11 Other 3 10 0.81 7 Total: 29 10.9 Sectoral distribution of Sub-Projects financed under TC II (Credit 1664-CHA) Sector No of % of total Disbursement % of total Sub-Projects number US$ equivalent Disbursement Agriculture 2 5 3.03 13 Energy/Power 9 24 11.02 46 Industry 10 27 4.72 20 Infrastructure 3 8 0.75 3 Environment 5 14 1.84 8 Water/Sewerage/Urban 6 16 2.25 9 Other 2 5 0.54 2 Total: 37 24.1 Annex B 54 Page 4 of 4 Sub-projects financed under PSSSP (Credit 1835-CHA) Sub-Project Amt. US$ m Beneficiaries I Reinforcing the Planning Commission 1.39 State Planning Commission (SPC) & Strengthening Planning Tools 2 Management of Government Finance 1.64 Ministry of Finance 3 Institutional Development of People's Bank 1.35 People's Bank of China, Human Resources Development Center 4 Training of Planning Officials 1.24 State Planning Commission (SPC) 5 Planning of Electric Power in 0.76 Ministry of Water Resources and Eastern China and Sichuan Electric Power 6 Petrochemical Development Plan 2.44 China Petrochemical Corp. (SINOPEC) 7 Promotion of China's Consulting Industry 0.63 China International Engineering Consulting Corporation (CIECC) 8 Development Strategy for CICC 0.29 China Investment Consulting Corp. (CICC) 9 Organization and Regulation of a Futures Market 0.08 Department of Market and Price, Development Research Center 10 Price Planning Program & 0.47 State Price Bureau, SPC Anti-Inflation Decision Support 11 Development of Audit Capability 0.69 Audit Administration of the PRC 12 Research on the Formation & Development 0.81 National Research Center for Science and China's Science & Tech. Development Plan Technology for Development 13 Yellow River Basin Economic Modeling Study 1.00 Reconnaissance, Planning and Research Institute, Yellow River Conservation Comm., Ministry of Water Resources 14 Improve Science Funding System & Promote 0.40 National Natural Science Foundation the Development of Basic Scientific Research of China (NSFC) 15 Planning of Electronics Industry Development 2.06 Ministry of Electronics Industry 16 Key Machinery & Basic Products 0.00 Ministry of Machinery and Electronics Ind. 17 Energy Price Research 0.35 Department of Market and Price, SPC 18 Poverty Alleviation Studies 0.43 SPC, in conjunction with other ministries and banks 19 Study of National Flood Control 1.48 State Flood Control Headquarters & Dispatch System, Phase I General Office 20 International Quality Assurance in China 1.85 China Council for the Promotion of Intemat. Quality Assurance, State Bureau for Technological Supervision Total: 19.99 55 Annex C Page 1 of 2 BENEFITING AGENCIES UNDER TC I, TC II, and PSSSP number of sub-projects and amount of funding in US$ million TC I TC II PSSSP Ministry of Communications 6 for - 1.1 Jiangsu Prov. Communications Bureau 1 for ~ 0.2 Ministry of Public Health 1 for - 0.2 Central Patriotic Health Campaign I for ~ 0.1 Committee First Bureau of Shanghai Mechanical & I for ~ 1.0 Electrical Industries Ministry of Electric Power 5 for - 3.7 1 for - 0.8 Ertan Hydroelectric Developmt Corp I for - 2.5 Shanghai Municpl Electric Power Bur 1 for - 0.9 Henan Provincial Electric Power Bur 1 for - 0.4 Hainan Electrical Power Company 1 for - 0.2 Shandong Provincl Electric Power Bur 1 for - 0.7 Yellow River Water and Hydroelectric 1 for - 2.9 Power Development Corp E. China Electric Power United Corp 1 for - 2.6 Ministry of Forestry 1 for - 0.05 Ministry of Railways I for - 0.1 Ministry of Agriculture, Animal 4 for - 0.2 Husbandry & Fisheries Henan Prov. Agricultural Bureau I for - 0.06 Ministry of Chemical Industries 2 for - 2.2 Chemical Industry Dept, Jiangsu Prov. 1 for - 0.08 Chemical Planning Institute I for - 0.3 Ministry of Finance 1 for - 1.6 (E&F Publishing House) 1 for - 0.5 State Pharmaceutical Admin. of China 1 for - 0.07 National Audit Office 1 for - 0.2 1 for - 0.7 State Administration of Building 1 for ~ 0.5 Materials Industry Shanghai Sewerage Construction Co. 1 for - 0.2 Shanghai Harbor Bureau 1 for - 0.2 Environment: Shanghai Environment Project Office 1 for - 0.06 Beijing Municipal Finance Bureau 1 for - 0.4 Liaoning Urban Constrctn & Renewal 1 for - 1.0 Jiangsu Provincial Staff 1 for - 0.3 Chinese Academy of Science (Bureau 1 for - 0.07 of Natural Resources) Henan Pharmaceutical Industry Corp 1 for - 0.8 Annex C 56 Page 2 of 2 TC I TC II PSSSP Ministry of Water Resources I for ~ 0.5 Yellow River Conservatn Commission I for - 1.0 State Flood Control Headquarters I for - 1.5 General Office Tianjin Finance Bureau 2 for - 2.8 Liaoning Urban Construction & 1 for ~ 0.4 Renewal Project Office Dalian Port Construction Authorities I for - 0.3 Municipal Governments in Luoyang, I for ~ 0.4 Changzhou, and Shashi Zhejiang Urban Development Proj Off I for ~ 0.4 Sichuan Petroleum Agency 1 for ~ 1.7 Ministry of Construction Construction Management Department I for - 0.5 Changchun Water Supply & I for - 0.9 Environmental Project Office Ministry of Internal Trade I for - 3.0 Stock Exchange Executive Council I for - 0.2 People's Bank of China 1 for - 1.4 State Planning Commission 5 for - 3.9 China Petrochemical Corp (SINOPEC) I for - 2.4 China International Engineering I for ~0.6 Consulting Corp (CIECC) China Investment Consulting Corp 1 for ~ 0.3 (CICC) Development Research Center I for - 0.8 National Research Center for Science & 1 for - 0.8 Technology for Development National Natural Science Foundation of 1 for ~ 0.4 China Ministry of Electronics Industry 1 for - 2.1 State Bureau for Technological Supervision: China Council for Promotion of 1 for - 1.9 International Quality Assurance 57 Annex D Page 1 of 2 Sub-projects financed under TC I (Credit 1412-CHA) and TC II (Credit 1664-CHA) aimed at Institutional Development 1. Out of a total of 25 sub-projects financed under TC I, 4 were specifically for institutional development (ID) rather than for project preparation. These four sub-projects are shown in Table 1 below. Two striking features of these sub-projects are their ad hoc nature and the modest amount of funds involved, usually used to finance one seminar, study tour and/or short- term training. The most substantial of these sub-projects, at almost US$0.5 million, was to purchase printing equipment for the Ministry of Finance's Economic and Financial Publishing House. It is questionable whether this was an appropriate use of scarce IDA resources for ID. 2. With respect to the other three sub-projects, the funds were used primarily for hiring consultants to prepare seminars and to conduct study tours. Although the benefits of such short- term exposure to external methods and systems should not be underestimated, particularly when the participants are high-level officials who may subsequently decide to adopt different approaches to design or management issues, such exposure is most effective when part of a larger package of training that also involves staff at different levels to help implement the changed approaches (modeling techniques, design features, policy approaches, etc.). In the absence of further information on whether these sub-projects were part of a larger effort at ID, that involved other forms of TA to provide more in-depth and broad exposure to new methods and so forth, it is difficult to judge whether such modest efforts can have had a substantial and sustainable impact.29 Table I Sub-projects under TC I aimed at ID Sub-project Sub-project name Amount number 12 Assistance to Audit Administration $183,651 13 Port Master Planning, Feasibility Study $ 91,089 14 Port Terminal Control System Training $113,481 24 Economic and Financial Publishing $456,589 House (Ministry of Finance) Total: $844,810 3. At the time of approval, the intention for ID under TC II was to finance economic and policy studies and to contribute to the professional development of research agencies In fact, little of this type of ID was carried out under TC II and this has been attributed to the overlap with PSSSP, approved shortly after TC II, with similar objectives. Nevertheless, out of a total of 37 sub-projects, five had ID objectives unrelated to project preparation, involving about 15 percent of total disbursements under the project. They are listed in Table 2 below. 4. The first of these sub-projects was used as the technical assistant component to an on- going investment project. It funded purchase of laboratory precision analytic instruments and training. The original investment project had also provided funding for purchase of similar 29The Audit Administration (now called the National Audit Agency- NAO), for example, held one seminar in 1984 and one seminar in 1986. Other than periodic contact with Bank staff, the NAO did not receive any additional TA from the Bank until 1990. Annex D 58 Page 2 of 2 equipment to the same quality control center, but the ICR for the investment project noted that the equipment turned out not to be needed and was never purchased. It is thus not clear why the instruments were purchased at all. The second of the sub-projects listed below (no. 12) involved the financing of computers which were never used for their intended purpose because of inappropriate software. The third sub-project (no. 15), in spite of its name, was the technical assistant component under the Tianjin Light Industry Project, Credit 3022-CHA). Funds were used under the sub-project to finance consultant contracts and the ICR reported "mixed results," with unsatisfactory work produced by one of the two firms. The fourth of these sub-projects (no. 30) involved training engineers in a neighboring country, but was criticized by the ICR for being implemented before a needs assessment was made. The fifth sub-project listed below (no. 35) was judged to have been highly satisfactory. Thus, the record of the impact of free-standing ID sub-projects under TC II appears to be at best modest. Table 2 Sub-projects under TC II aimed at ID Sub-project Sub-project name Amount number 10 Henan Pharmaceutical Quality Control $801,291 12 Strengthening Chemical Planning $337,441 Institute's Planning Capabilities 15 Tianjin Provincial Loan Preparation $1,971,828 30 Development of Construction Industry $500,247 35 Study on the Development of Securities $199,654 Trading Automated Quotations System Total: $3,810,461 59 Annex E Page 1 of 4 QUESTIONNAIRE TO MOF WORLD BANK TECHNICAL ASSISTANCE OPERATIONS IN CHINA, 1983-95 (IDA Credits 1412, 1664, and 1835) A World Bank mission from the Bank's Operations Evaluation Department (OED) will visit China from February 24-March 6, 1997. This mission, comprising Ms. Laurie Effron and Mr. Robert Armstrong, wishes to obtain views from the Chinese authorities regarding the relevance, effectiveness and efficiency (cost-effectiveness) of the three IDA projects mentioned above. The OED mission also wishes to identify related lessons of experience from China's use of technical assistance (TA) that may be relevant to improving the effectiveness of future TA to China and to other countries wishing to learn from China's experience with TA. Given these objectives, the mission would like to learn about China's experience with the projects mentioned above and also about experience with subsequent TA operations financed by the World Bank and other donors. The three projects above are known as Technical Cooperation Projects 1 and 2 (TC I and TC II, approved in 1983 and 1986, respectively) and the Planning Support and Special Studies Project (PSSSP, approved in 1987). A separate questionnaire is being sent to SPC regarding the PSSSP. The present questionnaire is directed to MOF and other Government officials who are familiar with either (i) the MOF-administered TA projects financed by the World Bank (TC I, TC II, CRISP, and/or TC IV under preparation); or (ii) China's general experience with donor- financed TA projects. The three main themes that the OED mission wishes to address are the following: 1. Demand for TA: How has the "demand" for TA in China evolved, from before TC I in 1983 to the present? How have the objectives of TA projects changed over time? 2. Performance: In what respects have the IDA-financed TA projects been most and least successful, and why? What appear to be the common factors associated with the most and least effective subprojects? 3. Lessons: What are the main lessons of experience from China's use of (i) the three completed IDA-financed TA projects; and (ii) donor-financed TA in general? How have these lessons been applied to the design or management of the current set of TA projects? More specific questions relating to the individual TA projects follow. The OED mission realizes that some of these questions may be difficult for current MOF staff to answer because they refer to events from long ago. If MOF knows of other people who were involved in the early- or mid- 1980s who could answer these questions, the OED mission would appreciate being able to meet them as well. Annex E 60 Page 2 of 4 Project Identification and Preparation 1. The Need for TA: Who decided that there was a need for the TC I and TC II projects, and why? The CRISP project? The TC IV project? What were the main similarities and differences in the nature of the "demand" for these operations? 2. The "Right" Objectives? With the benefit of hindsight, did each of these projects have the "right" objectives? For example, were the objectives of TC I and TC II perhaps too narrowly confined to preparing projects for World Bank financing? Were the policy support and institutional development objectives of later projects perhaps too loosely designed? If any of these projects could be done over again, how might its design be changed? 3. Why the World Bank? What kinds of issues were discussed between MOF and other GOC ministries or agencies leading to the decision to undertake these operations? Why was the World Bank (rather than UNDP or another agency, whether private, bilateral or multilateral) requested to finance these operations? 4. Experience with Other TA: What was the experience of GOC in using TA from other sources before, and at the same time as, TC I and TC II? How were lessons learned from the UNDP-financed umbrella TA operations applied to the design of TC I or TC II? 5. Participation: How participatory were the processes of identifying and-preparing TC I, TC II, and subsequent TA operations? What were the respective roles of World Bank staff, MOF staff, and staff of the beneficiary/implementing agencies, in the identification/preparation processes? How broad or narrow was GOC "ownership" of the projects? 6. Selecting the Beneficiaries/Implementing Agencies: What criteria did MOF apply in selecting beneficiaries and subprojects? To what extent did MOF take the initiative in identifying beneficiaries? How much flexibility was allowed to change the mix of investment projects to be prepared during the course of TC I and TC II? To what extent were funds made available, including in the CRISP project, on a "first-come, first-served" basis? 7. Onlendingi Terms: What were the onlending terms (interest rates, grace periods and repayment periods) between MOF and the implementing agencies? Under what circumstances did MOF provide funds to beneficiaries as grants? If there were different onlending terms to beneficiaries, what was the rationale for the terms? To what extent were some prospective beneficiaries deterred by the onlending terms? Were there other issues relating to the onlending terms? If so, what were they? 8. Other Design Issues: Was the design of any or all of these projects too complex, with too many implementing agencies? Too demanding on the managerial capacities of either MOF or the implementing agencies? Was enough attention given to training GOC officials in Bank procedures, e.g. guidelines for procurement of equipment and consultants, reporting and monitoring requirements, etc.? 61 Annex E Page 3 of 4 Project Implementation and Outcomes 9. Outcomes by Obiective: The main objective of TC I and TC II was project preparation. Later TA projects aimed at policy support and institutional development. Which of these objectives were most and least satisfactorily achieved, both in terms of quantity and quality, and why? 10. Implementation/Outcomes by Instrument: Each of the projects used a variety of TA "instruments" in different combinations. These instruments included the following: - equipment - short-term foreign consultants - longer-term foreign consultants - local consultants - study tours - formal training abroad - formal training (including seminars and workshops) within China - informal (on-the-job) training. In each project, which instruments were used most successfully? Least effectively? What were the main factors explaining why some instruments were more effective (and cost- effective) than others? What were the management and quality control problems with each of these instruments? 11. "Externalities": Were there any notable benefits from these TA projects to agencies or staff not directly involved in the subprojects? For example, did other agencies learn from the beneficiary agencies how to apply new appraisal or monitoring techniques for investments, or new procurement techniques? Were there unexpected costs of the TA projects, for example, lasting dependency on foreign consultants, or losses in productivity due to inefficient allocation of officials to training programs or study tours, or duplication of study tours or courses?? 12. Bank Staff Performance: From the standpoint of MOF, what were the strongest and weakest aspects of the performance of World Bank staff in the preparation and supervision of these projects? In particular, was Bank supervision too flexible or too inflexible? How good was Bank supervision in assisting implementing agencies to solve their problems? 13. MOF Staff and Implementing Agency Staff Performance: How does MOF evaluate the performance of its own staff in preparing and administering these projects? How were the implementing agencies made accountable for the outcomes? What aspects were most important in ensuring that the subprojects were well-managed? How does MOF evaluate the performance of the implementing agencies? 14. Aid Coordination: To what extent has TA provided to China by the World Bank been coordinated with TA being provided from other sources? In particular, how does GOC coordinate TA assistance provided through MOF on loan terms and through MOFTEC on grant terms? How are priorities for TA set on a national level, and what determines which donors will be requested to provide which kinds of TA? How has China's experience in using TA from the World Bank differed from the experience in using TA from other donors? Annex E 62 Page 4 of 4 15. Sustainability: Which benefits of the World Bank-financed TA projects appear most and least likely to be sustained? In particular, which kinds of TA and training, including study tours, are likely to be most long-lasting? Lessons of Experience: 16. If you could do any of the projects over again, how should they be done differently? 17. With regard to each of the following, what are the main lessons of experience of the TA projects? - how to design and prepare TA projects (including how to get the right balance of instruments and objectives) - how to manage and supervise TA projects (including getting managers with the right skill) - how to define performance indicators for such projects - how to disseminate lessons of experience, including across the boundaries of MOF- and MOFTEC-administered TA projects 18. In what specific ways were lessons learned from TC I applied to TC II? From TC II to PSSSP? From the TA components of investment projects to the IDA-financed freestanding TA projects? From earlier freestanding TA projects to later ones such as the CRISP, and now TC IV? 19. What is the single most important lesson from China's use of IDA-financed TA? 63 Annex F Page 1 of 2 ANALYSIS OF MINISTRY OF FINANCE CASH FLOW FROM ON-LENDING AND REPAYMENT OF PSSSP FUNDS 1. Introduction. The following is an analysis of the cash inflows accruing to the Ministry of Finance (MOF) from on-lending the funds from the Planning Support and Special Studies and the cash outflows paid by MOF to reimburse the IDA credit. The purpose of the analysis is to measure the impact on MOF's budget from the mismatch between the terms of borrowing from IDA and on-lending the funds. The two streams (in-flow from repayments from participating agencies and outflows from MOF repayment to IDA) are combined for a net inflow/outflow and a rate of return is calculated. 2. Bpayment Terms. The funds were borrowed from IDA with a repayment period of fifty years, including ten years of grace, at a service charge of 0.75 percent per annum and a commitment fee of 0.5 percent per annum on the undisbursed balance of the credit. The funds were on-lent to the participating agencies for a period of twenty years, including five years of grace, with a service charge of one percent per annum. 3. The repayments from MOF to IDA are based on actual billings in SDRs between March 1988 and March 1997. From September 1997 through March 2037 the repayments are based on the amortization schedule in Special Drawing Rights (SDRs). The repayments in 1988 include the commitment fee on the undisbursed balance of the credit. From 1989 through 1995, the commitment fee was waived. And from 1996 onwards the commitment fee is irrelevant because the credit was fully disbursed. 4. Assumptions. The repayments from each participating agency to MOF were based on the assumption that the repayment period started once the funds were approved. Thus, for a sub- project approved in 1988, repayment would start after the five year grace period in 1993. For a sub-project approved in 1994, repayment would start in 1999. In addition, the service charge to the participating agencies is assumed to be capitalized during this grace period, i.e., there are no payments to MOF from the participating agencies for five years, but the service charge accumulates and is amortized over the remaining fifteen years of repayment. 5. MOF on-lends the funds in dollar equivalent and repays the funds in SDRs. The participating agencies thus take the foreign exchange risk between the dollar and the yuan, while MOF bears the risk between the dollar and the SDR. In order to make a comparison between the cash in-flows from the participating agencies in dollars and the cash outflow from MOF to IDA in SDRs, the in-flows in dollars were converted to SDRs at the actual exchange rate prevailing at the time of the repayments up to December 1996, and from September 1997 onwards the in- flows were converted at the exchange rate as of December 1996. 6. It was further assumed that the default rate on the repayments due to MOF are 10 percent each year and that 10 percent of the repayments of interest and principle due are in arrears by one year. If in fact defaults and arrears have been significantly higher than these assumptions, it raises more fundamental questions about whether MOF should be in the on-lending business to start with. Annex F 64 Page 2 of 2 7. Results. The result of these calculations shows that MOF realizes a rate of return on the cash inflows and outflows of 28.1%. These calculations do not include the operating costs of the Technical Assistance and Training Division of the World Bank Department, but in any case these would have to be spread over the various projects and project components involving TA that are handled by the Division. It is unlikely that a pro-rated annual operating cost would change the results significantly. 8. Sensitivi analysis. Since FY88, IDA funds have no longer been available for fifty years. Thus PSSSP was one of the last credits to benefit from such a long repaymwnt period. For China, IDA funds are now available for 35 years, with a ten year grace period. Sensitivity analysis was carried out on the inflows and outflows assuming these latter terms and the results are remarkably similar: a rate of return of 27.5%. The reason for the similarity is that during the first ten years of the repayment period, the inflows and outflows are identical, and it is only in year 11 that the net flows begin to be somewhat smaller, and by small amounts. 65 Annex G Page 1 of 3 AGENCIES AND PEOPLE MET DURING EVALUATION MISSION Minisry of Finance State Planning Commission World Bank Department Department of Key Projects Construction Technical Assistance Division 1. Zou Zaihua, Deputy Division Chief 1. Feng Yushu, Division Chief (in Washington) State Planning Commission 2. Liao Renxing, Deputy Chief Department of Rural Economy 3. Zhang Xingping 4. Wang Liansheng (currently Vice 1. Wang Xinhuai, Division Chief Chairman of Import-Export of China) former head of World Bank Department in National Audit Office of PRC MOF Foreign Affairs Bureau Ministr of Finance 1. Hu Xuewen World Bank Department 2. Zhou Wenhua Energy Industry and Transport Division National Audit Office of PRC 1. Hong Houqi, Project Officer Foreign Funds Application Audit Department Ministry of Finance Department for International Cooperation 1. Wieping Yu, Deputy Director General 2. Dun Cao (in Washington) 1. Zheng Jiazhong, Deputy Division Chief 2. Liu Jinghai, Project Officer Ministry of Water Resources Foreign Investment Management Office Ministry of Finance Department of Local Taxes 1. Zhu Jiang 1. Zhang Zhi Hua Ministry of Water Resources Department of International Cooperation State Planning Commission Comprehensive Planning Department 1. Zhang Ximing 1. Zhang Fen, Division Chief Ministry of Water Resources 2. Li Ping, Deputy Division Chief Yellow River Conservancy Commission 3. Dai Riyong (currently President of BOT Investment and Development Corp) 1. Wang Yineng, Senior Engineer formerly Deputy Division Chief of CPD, and project manager of PSSSP Office of State Flood Control Drought Relief Headquarters State Planning Commission Training Center 1. Fu Zeng Ci, Chief Engineer 1. Kang Ning, Deputy Director Annex G 66 Page 2 of 3 China International Engineering Consulting Ministry of Electric Power Corporation (CIECC) Department of International Cooperation International Cooperation Department Division of Economic Cooperation 1. Zhang Shuwu, Deputy Director 1. Liu Jiayu, Project Officer 2. Li Shangwu, Senior Engineer, Training Department Ministry of Electric Power Department of Policy, Legislation and China Investment Consulting Corporation System (People's Construction Bank of China) Division of Policy Investment Advisor Department 1. Zou Chijia, Deputy Director 1. Ma Xiaohong, Project Manager 2. Gong Jianzu, Division Chief 2. Zhang Yuping, Chief Manager Ministry of Foreign Trade and Economic China Council for International Quality Cooperation (MOFTEC Assurance China International Center of Economic and Technical Exchanges 1. Hua Sheng, Secretary General Programming and Planning Division Development Research Center of the State 1. Sun Yongfu, Deputy Division Chief Council 2. Jiang Lingyuan, Programme Officer 1. Wang Huijiong, Vice President of United Nations Development Programme Academic Committee (UNDP) -- Beijing 2. Ren Xingzhou, Deputy Director 3. Li Shantong, Director, Department of 1. John Holcombe, Resident Representative Development and Forecasting 2. Chingboon Lee, Deputy Resident Representative (on secondment from World Ministry of Internal Trade Bank) Foreign Capital Management Office Price Waterhouse 1. Song Tingming, Director General 2. Chen Fei, Deputy Director General 1. John Campbell, former project manager 3. Zhang Chunxiao, Division Chief in PBC under Financial TA loan 4. Kenneth Luan, Deputy Division Chief 5. Yang Shizhong, Chief Engineer International Monetary Fund 6. Han Su, Project Officer 1. John Brondolo, Fiscal Affairs People's Bank of China Department International Department 2. N. Anderson - Beijing 1. Li Yue, Deputy Chief 2. Shi Jiyang (currently Director in Hong Kong Chinese Enterprises Association), formerly head of PBC sub-project under PSSSP 67 Annex G Page 3 of 3 World Bank: Resident Mission in China 1. Pieter Bottelier, Resident Representative 4. Kathy Krumm 2. Austin Hu, Deputy Chief of Mission 5. E.C. Hwa 3. Xiang Ying 6. Li Sheng World Bank -- Washington 1. Daud Ahmed 16. Vikram Nehru 2. Yves Albouy 17. Albert Nyberg 3. David Berk 18. Anthony Ody 4. Paul Cadario 19. David Pearce 5. Lauren Cooper 20. Alan Piazza 6. Rob Crooks 21. David Rix 7. Nicolette DeWitt 22. Claude Salem 8. Daniel Gunaratnam 23. Keiko Sato 9. Peter Harrald 24. Zhang Shengman 10. Mme Hua 25. David Steed 11. Al Howlett 26. Gene Tidrick 12. Bert Hoffman 27. Tom Tsui 13. Yo Kimura 28. Robert Van Der Lugt 14. Nicolas Mathieu 29. Shahid Yusuf 15. Fernando Montes-Negret   <CI zø hý