China - Rural Health and Preventive Medicine Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 18893 IMPLEMENTATION COMPLETION REPORT RURAL HEALTH AND PREVENTIVE MEDICINE PROJECT (LOAN 2723-CHA / CREDIT 1713-CHA) January 29, 1999 Health, Nutrition and Population Sector Unit East Asia and Pacific Regional Office This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENTS Currency Unit = Renminbi (RMB) At the time of appraisal: US$1 = RMi 3.20 At the time of project completion mission: US$1 = RMB 8.27 FISCAL YEAR January I - December 31 WEIGHTS AND MEASURES Metric System ABBREVIATIONS AND ACRONYMS CAPM - Chinese Academv of Preventive Medicine CMS - Cooperative Medical Svstem DPT - Diptheria-Pertussis-Tetanus Associated Vaccine EF - Engineering Firm EPI - Expanded Program on Immunization EPS - Epidemic Prevention Station FLO - Foreign Loan Office GMP - Good Manufacturing Practices HIV - Human lmmunodeficiencv Virus ICR - Implementation Completion Report IDA - International Development Association MCH - Maternal and Child Healtl MOF - Ministry of Finance MOH - Ministrv of Health NCD - Non-Communicablc Discasc OPV - Oral Poliomvelitis Vaccine PHC - Primarv Health Carc SAR - Staff Appraisal Report SDR - Special Drawing Right STD - Sexuallv Transmitted Disease TT - Tetanus Toxoid UNICEF - United Nations Children's Fund VPO - Vaccine Project Office WHO - World Health Organization Vice President Jean-Michel Severino, EAPVP Country Director Yukon Huang, EACCQ Sector Manager Maureen Law, EASHN Task Manager Herbert Boehm, EASHN  FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT CHINA RURAL HEALTH AND PREVENTIVE MEDICINE PROJECT LOAN 2723-CHA CREDIT 1713-CHA CONTENTS PREFACE .......................... , . ... . EVALUATION SUMMARY .................... ...... PART 1: PROJECT IMPLEMENTATION ASSESSMENT................................... . ..1........... A. Background ....I B. Project Objectives................... 3.... :3 C. Achievement of Project Objectives .4 D. Implementauon Record and Major Factors Affecting the Project .6 E. Project Sustainabilit .9 F. Perfornance .12 G. Assessment of Outcome .16 H. Future Operation .17 1. Kev Lessons Learned .18 PART II: STATISTICAL TABLES . . . 21 Table l: Summarv of Assessments .21 Table 2: Related Bank Loans/Credits .22 Table 3: Project Timetable .24 Table 4: Loan/Credit Disbursement: Curnulative Estimate and Actual .24 Table 5: Studies Included in Project .24 Table 6a: Project Costs (US$) .25 Table 6b: Project Costs (RMB) .25 Table 6c: Project Financing .25 Table 7: Status of Legal Covenants .26 Table 8: Complianece with Operational Manual Statements .26 Table 9a: Bank Resources: Staff Inputs (Actual Staff Weeks). 27 Table 9b: Bank Resources: Staff Inputs (Actual US$) .27 Table 10: Bank Resources: Missions .28 ANNEX A: BORROWER'S CONTRIBUTION TO THEI: ........................................... 29 Part I: Rural Health and Other Non-Vaccine Production Components . .29 Part 11: Vaccine Production Component ..32 Part HI: Bonower's Comments on Bank's ICR ..37 Map No. IBRD 29543 This document has a restricted distribution -and may -be used by recipients oaly in the performance of their official duties. Its contents may not. otherwise. be disclosed without World Banlc authorization.  IMPLEMENTATION COMPLETION REPORT CHINA RURAL HEALTH AND PREVENTIVE MEDICINE PROJECT LOAN 2723-CHA / CREDIT 1713-CHA PREFACE This is the Implementation Completion Report (ICR) for the Rural Health and Preventive Medicine Project in China, for which Loan 2723-CHA in the amount of US$15.0 million, and Credit 1713-CHA in the amount of SDR 57.2 million (US$65.0 million equivalent) were approved on June 19, 1986 and made effective on May 26, 1987. The loan and credit were closed on June 30, 1996. Both were fully disbursed. The last disbursement took place on October 10, 1996. An associated Grant from Rotary International in the amount of US$15.0 million, which was administered by the Bank, closed on June 30, 1997. The ICR was prepared by Herbert Boehm and David Dunlop, Consultants, with assistance by Darren Dorkin, Operations Analyst, in the Health, Nutrition, and Population Sector Unit of the East Asia and Pacific Region. It was reviewed by Alan Ruby, Acting Sector Manager, Jagadish Upadhvay, Senior Operations Officer, and Janet Hohnen, Senior Public Health Specialist, EASHN, and by Yukon Huang, Country Director, China. The Borrower's contribution is included as Annex A to the report. Preparation of this ICR was initiated during discussions between the Bank and the Ministry of Health in 1993 for the rural health and other non-vaccine production components of the project. However, the Bank and the Ministry of Health agreed to postpone completion of the ICR until the ongoing work under the vaccine production component was completed. A project completion workshop was held in Beijing in April 1998. The ICR is based on materials in the project file, including extensive summary and analyses prepared by the Borrower and the project teams, and comments by members of Bank task teams, the United Nations Children's Fund (UNICEF), and the World Health Organization (WHO).  IMPLEMENTATION COMPLETION REPORT CHINA RURAL HEALTH AND PREVENTIVE NIEDICINE PROJECT LOAN 2723-CHA / CREDIT 1713-CHA EVALUATION SUMMARY Introduction I . The Bank began working with China on health issues with the launching of the rural health and medical education mission in 1982. That mission led to a sector study in 1984, China: The Health Sector, and formed the basis of a sector lending program starting with the Rural Health and Medical Education Project (Health L Cr. 1472-CHA), approved by the World Bank Board on May 3, 1984. The Rural Health and Preventive Medicine Project (Health II) extended the work begun through the first health project. A project brief was presented to the Bank in July 1984, and the project was appraised in July 1985 and approved by the Board in June 1986. 2. The 1984 health sector report also provided the basis for crafting the design of Health II whose objective was to strengthen and expand the efforts of the Ministry of Health (MOH) to: (i) improve rural health care; (ii) improve the efficiency, coverage, and quality of the national immunization program for children; (iii) initiate improved drug quality control; and (iv) develop new communicable and chronic disease preventive strategies and health care financing systems in rural areas. To address this set of objectives, which primarily focused on the health of the rural poor, Health II was designed with four components: (i) rural health care deliverv improvement; (ii) vaccine production improvement; (iii) drug qualitv control improvement; and (iv) operational research in disease surveillance and disease prevention especially for non-communicable diseases (NCDs), along with testing new approaches in rural health insurance. Project Objectives 3. The main objectives of the project were to: (a) strengthen and expand the Ministry of Health's continued efforts to improve rural health care; (b) improve the efficiency, coverage and quality of the national immunization program for children; (c) initiate improved drug quality control; and (d) develop new communicable and chronic disease prevention strategies and health care financing systems in rural areas. 4. To accomplish these objectives, the project included four main components: (a) Rural Health -- expansion and qualitative improvement of preventive and curative health services in the poorer rural counties of five provinces, and strengthening of the disease monitoring and control  iii activities and immunization delivery and management of the epidemic prevention stations in these and three additional provinces; (b) Vaccine Production - improvement in the coverage and cost effectiveness of the national immunization program against the main childhood diseases through construction and rehabilitation of three national centers for the production of essential vaccines meeting international quality standards; (c) Drug Quality Control - improvement of the quality control of drugs used; and (d) Operational Research - undertaking research to (i) strengthen the capacity of the National Center for Preventive Medicine for disease surveillance, and (ii) test new approaches to rural health insurance. Implementation and Results 5 The project made significant contnrbutions in the area of rural health and disease control, but expenrenced severe problems and delavs in the development of vaccines and the construction of the production facilities. Within the Rural Health Care Delivery Component, a number of health facilities, including hospitals, village clinics, maternal and child health (MCH) stations, epidemic prevention stations (EPSs) and other training and administrative blocks, were built and equipped, mainly in the rural areas of five provinces: Gansu, Hubei, Jilin, Sichuan, and Ningxia Hui. The civil works were generally completed within the specified time and according to the budget estimates contained in the staff appraisal report (SAR). Preventive service delivery of the provincial and municipal EPSs was also strengthened in these five provinces as well as in the provinces of Heilongjiang, Jiangxi, and Shandong through training and by disease control and monitoring. 6. The Drug Quality Control Component also achieved its envisioned objectives of more skilled personnel due to improved training and better equipment for use in drug efficacy and quality testing. These improvements also contributed to the 1995 revision of the Chinese pharmacopoeia, as the results of improved testing were incorporated into the document. However, the project did not address the weaknesses of the national control authority, especially insofar as its authority for drug qualitv and its regulatorv enforcement capability are concerned. 7 The Operational Research Component assisted the Chinese Academy of Preventive Medicine (CAPM) in two objectives: extension of its disease surveillance system, and increased capacity to develop disease prevention strategies. The Academy launched the first of many investigations into the health impacts of smoking in China and contributed to assessing how various health education campaigns can be effectively designed to reduce smoking, especially among youth. This component also included a successful rural health insurance experiment which provided the Chinese govemment with experience and knowledge on the design of health financing mechanisms. 8. Finally, the project enabled the Government of China and other developing countries, because of the publicity of the project's implementation difficulties, to learn about the intricacies of development of high technology vaccine production. Three vaccine plants have been constructed: an oral polio vaccine (OPV) plant in Kunming, and two multi-vaccine production plants, in Shanghai and Lanzhou, designed to produce diphtheria, pertussis and tetanus (DPT), measles, and tetanus toxoid (TT) vaccines. The plants will begin to produce vaccines as soon as the validation of plant production processes and testing at normal production capacity levels are  iv completed. One year after the start of operation, the plants' output will be expected to meet the guidelines for vaccine production established by WHO. Although the component has experienced many frustrations in contracting with intemational vaccine manufacturers, significant cost increases over that estimated in the SAR, and difficulties in production process development for some vaccines, the three plants have been constructed and equipped, and are now awaiting full production testing. Issues 9. While the rural health and other non-vaccine production components had very favorable results, critical examination of implementation problems indicated a number of shortcomings. These shortcomings, primarily caused by health policy changes affecting the support to preventive and curative health care delivery, were outlined in the Borrower s project completion report, and included: (a) inadequacy of recurrent cost for: (i) the EPSs at the provincial and county level; and (ii) the provincial (health equipment) maintenance centers and county MCH stations, contributing to lowered personnel productivity in the immunization program, lower quality services than the public's willingness to pay via direct user charges, and lower than expected utilization; (b) lack of training materials for local staff based on the new primary health care (PHC) policies and changing pattern of health problems toward chronic diseases; and (c) insufficiency of implementation flexibility from one province and county to another, ignoring local differences in facility and service need or demand, and leading to relatively poor resource allocation from one assisted locality to another. 10. Although difficult to quantifv, these factors mav have contributed to lower health status and service use improvements than orizinally expected from the project's investments. The lack of project implementation flexibility, however, provided a learning experience for Chinese officials which contributed to strengthened provincial and lower level health planning and implementation in the World Bank supported Integrated Regional Health Development Project (Health III; Cr. 2009-CHA), which became effective in January 1990. l1. While the other project components were implemented with little difficulty, except as noted above, the vaccine production component was fraught with imnplementation problems from the beginning. Because of its highly complex technical requirements and long implementation delays, by 1992/3 the World Bank and MOH separated this component from the rest of the project for purposes of supervision and management. This separation inevitably reduced the project design linkages conceptualized at the start of project development. In addition, project closing, originally planned for June 1992, had to be extended three times for a total period of four years, to December 31, 1997. 12. Because of the Bank's and the Borrower's inadequate technical knowledge in producing vaccines in sophisticated facilities, the component was not properly designed, contracted, or  v managed at the outset. The Borrower and the Bank therefore obtained international expertise to facilitate the managing of the implementation process. Additional financing was obtained from the Rotary Foundation of Rotary International in the amount of US$15.0 million. The international group contracted from the Netherlands. after a nearly four year delay, experienced project financing problems due to a lack of adequate finds in the original design of the project, exchange rate fluctuations and domestic inflation. That difficulty sparked technical and managerial disputes between the Government of China and the contracting group. In spite of these difficulties, the plants have reached the next stage where production processes must be finally validated, full production testing is required, product quality must meet WHO guidelines, and production process must adhere to European good manufacturing practices (GMP) standards. Key Lessons Learned 13. The major key lessons learned from the project are: Rural Health and. Other Non-Vaccine Production Components (a) Particular attention should be paid to finding solutions to the common problem of inadequate recurrent costs in the design and implementation of "low-cost" rural health care projects. (b) As an integral aspect of the project preparation, the project monitoring and evaluation component should be designed and guidelines formulated as to how it will be implemented. Vaccine Production Component (a) Vaccine production is a hiahly sophisticated technical undertaking, for which the Bank has insufficient expertise in design or supervision. Further, production process development and up-scaling to large production volumes are difficult tasks that would pose daunting challenges to even the largest and most highly specialized firms in the industry. (b) Building three production plants of this size and complexity simultaneously should have been avoided. Rather, start with one and employ lessons learned for dissemination and application elsewhere. (c) Selected turnkey contractors should have successfully developed similar projects in the past. Also, envisioned production processes should be successfully tested in another location and the contractor should have firsthand knowledge of how to transfer that technology to the Borrower. (d) The Borrower should ensure that adequate technical and managerial staff are available to monitor project implementation, and if necessary should be willing to hire internationally qualified personnel to do the work. Involvement of WHO experts should be sought throughout the design and implementation phases of the project.  vi (e) Since a vaccine production project is not a community health project, but rather an industrial project, its feasibility should be assessed according to normal financial and economic methods of project appraisal prior to making a financial commitment. (f) Before starting a major vaccine project, a national strategic plan for viable vaccine production needs to be developed, and the national drug control authority should be strengthened to ensure drug quality and safetv.  CHINA RURAL HEALTH AND PREVENTIVE MEDICINE PROJECT LOAN 2723-CHA / CREDIT 1713-CHA PART 1: PROJECT IMPLEMENTATION ASSESSMENT A. BACKGROUND Macro Economic Performance and Policy Framework I. Since the introduction of economic reforms in 1978 designed to move the country from a command to a market driven economv, China has undergone one of the most rapid increases in economic growth ever experienced. During this twenty-year period, per capita income has increased more than fourfold at an average annual rate of 7. 1%, as individual responsibility, especially in the agricultural sector, has been fostered. As this unprecedented growth has been occurring, new challenges have emerged, with the rapid move from a predominately agrarian society to an urban, industrial one. To sustain the progress into the future, further reforms have been recently announced to strengthen the financial sector, extend the reforms to state-owned enterprises, and ease the social dislocation of prior and anticipated reforms by ensuring social protection through pension, housing and health financing reform. 2. At the same time China has been undergoing its rapid economic transformation, the health status gains achieved since independence have slowed. It appears, for example, that under-five mortality trends might have leveled off since the early 1980s at about 45 per thousand live births. The health status gains have generallv been attributed to the implementation of priority public health interventions, along with improved food security and nutrition, and by rising income. 3 At the same time, the epidemiological transition is fully underway. Communicable health problems such as immunizable childhood diseases. e.g., measles, and controlling endemic infections have largely been mitigated. Today, non-communicable problems, e.g., cancer and heart disease, in part due to an increase in smoking and other life style changes, have emerged as health challenges for the new millennium. 4. Finally, rapid increases in economic growth since the introduction of market reforms in agriculture have contributed to an increasing inequity in the delivery and financing of health care services, as many of the nation's poorest cannot afford critical health services under the present fee-for-service system. These issues were emerging as Health 1I was being designed. Rural areas were placed in jeopardy as the financial support to preventive and curative health care delivery was undermined by the dismantling of the rural cooperative medical systems (CMSs) without replacing that support with alternatives. Further, rural areas no longer received adequate financial support for preventive programs which had in the past enabled the country to improve its health status.  2 Health Sector Policy and Financing Framework 5. Policy.Framework. China's progress in improving the health status of its people has been grounded on not only improving the delivery of curative and preventive health care services, but also on stressing the importance of nutrition, water supply, sanitation, education, and reduced fertility. This broad approach taken by China, along with the delivery of basic preventive and curative health care services, forms the basis of China's policy to support primary health care. By virtue of its success in reducing communicable health problems, including immunizable childhood diseases, the country's policy basis for this project in the early 1980s was threefold, to: (a) promote prevention by strengthening anti-epidemic activities, improve the management of immunization campaigns, and identify preventive measures against chronic diseases; (b) strengthen and consolidate rural health services by upgrading country-level health institutions and major township health centers, and promote "barefoot doctors" to rural doctors after they have received sufficient training; and (c) intensify medical research and training of health professionals, improve the technical and administrative management of health delivery, and improve the quality, production, distribution and proper use of pharmaceuticals. 6. To address the second policy focus of strengthening rural health services, China had launched a national program in the early 1980s called the One-Third County Upgrading Program. This program was to strengthen the four county-level health institutions: (i) general hospitals; (ii) EPSs; (iii) MCH centers; and (iv) training centers. Further, selected centers were identified to serve an intermediate referral function between ordinarv centers and county hospitals. To this end, Health 11 provided continuing assistance to extend this national program to the poorest rural counties initiallv launched bv the Bank through Health I. In so doing, Health II contributed toward the implementation of the first two policies enunciated above. 7. Financing Framework. By the early 1980s, China had significantly altered how health was previousiy financed. From the 1950s to the late 1970s. the sub-national governmental levels were a major financier of rural based preventive health services, with the rural CMS providing a large share of curative health care financing support, especially at the hospital level. Out-of- pocket payments provided the balance of the financial support required. After the collapse of the rural medical insurance system in the late 1970s, it was expected that out-of-pocket payments would fill the gap and provide not only additional financial support to curative health facilities, but also to preventive programs such as those provided by the EPSs and MCH centers. Thus, as the Health II Project was undergoing implementation, the sector's financial base of support was experiencing a major transformation, especially in rural areas. During the period of implementation of the rural health and other non-vaccine production components (1986 to 1993), Government support to the recurrent cost of the sector declined in constant (1993) yuan per capita from RMB 9.9 to RMB 9.1, or 8%. For two of the major preventive programs, epidemic prevention and maternal and child health, per capita support fell even more, from RMB 1.82 to RMB 1.36 (in 1993 yuan terms), a decline of more than 25%. Finally, repeated concern was  3 voiced in project supervision reports and the Borrower's Completion Report for the Rural Health and Other Non-Vaccine Production Components of the Project (1996), about the lack of recurrent cost support for preventive (and even curative) services. B. PROJECT OBJECTIVES 8. The Bank began working with China on health issues with the launching of the rural health and medical education mission in 1982. That mission led to the sector study in 1984, China: The Health Sector, and formed the basis of a sector lending program starting with the Rural Health and Medical Education Project (Health 1), approved by the World Bank Board on May 3, 1984. The Rural Health and Preventive Medicine Project (Health II) extended the work begun by the first health project. A project bnref for the Health II Project was presented to the Bank in July 1984. It was appraised in July 1985 and approved by the Board in June 1986. 9. The sector report also provided the basis for crafting the design of Health II whose objective was "... to strengthen and expand MOH's continued efforts to improve rural health care, improve the efficiency, coverage and quality of the national immunization program for children, initiate improved drug quality control, and develop new communicable and chronic disease preventive strategies and health financing systems in rural areas" (pg. 7, SAR). To address this set of objectives, which primarily focused on the health of the rural poor, Health II was designed with four components: (a) Rural health care delivery improvement by: (i) extending access by constructing new facilities; (ii) addressing service quality by personnel training, and extending the availability of modern diagnostic equipment in five of some of the poorest provinces in China; (iii) improving disease monitoring and control and immunization delivery; and (iv) managing preventive health programs as part of epidemic prevention activities in three additional provinces, Heilongjiang, Jiangxi, and Shandong. (b) Vaccine production improvement via the rehabilitation and construction of facilities capable of producing essential childhood vaccines according to international quality standards in a cost-effective manner; (c) Drug quality control improvement by personnel training, new facilities, and using up- to-date testing methods; and (d) Operational research in disease surveillance and disease prevention by the Chinese Academy for Preventive Medicine (especially for NCDs) and by testing new approaches in rural health insurance. 10. Under Health 1, 46 rural counties were assisted, and through Health II, an additional 50 counties in five of the poorest provinces were supported. The vaccine production component was viewed at the time of project identification and design as a logical extension of supporting health status gains by improving the quality of childhood immunizations and extending coverage. The other two project components were viewed as vital to the long-term financial and service quality  4 sustainability of the health sector. Finally, the operational research sub-component was designed to test strategies for changing the pattern of disease by piloting various preventive approaches to the emerging rapid growth of chronic health problems. 11. The scope of the vaccine production component at the time of project design was to: (a) establish a new production center for oral poliomyelitis vaccine (OPV) in Kunming, Yunnan Province with an annual production capacity of 100 million doses of liquid trivalent vaccine; (b) establish two new production centers for essential vaccines, including diphtheria- pertussis-tetanus (DPT), TT, and freeze-dried measles vaccine in Shanghai and Lanzhou, Gansu Province, each with an annual production capacity of 100 million doses of DPT, 40 million doses of TT, and 20 million doses of measles vaccine; and (c) rehabilitate quality control laboratories in each of the three sites. 12. This component was to facilitate China's efforts to fully immunize its children in a cost- effective manner and to provide the country with internationally recognized quality vaccine production facilities. C. ACHIEVEMENT OF PROJECT OBJECTIVES 13. The investments undertaken in the rural health development component were consistent with the dominant policy objectives at the national and provincial level. The investment programs were successfully carried out. In addition, the project succeeded in expanding preventive health care service delivery along with improved MCH services and delivering basic health services in rural areas in the project provinces. However, lack of funding for recurrent costs at the local level prevented the project from fully achieving the originaliv expected objectives. In contrast, the training sub-component exceeded the project's original expected targets. 14. The drug quality control component met its project objectives by initiating many training and research efforts at institutional levels to improve the quality and safety of domestic and imported pharmaceuticals. However, the project design should have addressed the weaknesses of the national drug control authority, which has the mandate of developing appropriate policies and enforcing rules and regulations concerning drug quality and safety. In particular, improving the agency's deficit in enforcing GMP standards in the country could have been of substantial value to this project as well as for other pharmaceutical projects in China. The operational research component also met its objectives by providing financial support to CAPM, which enabled it to expand its capacity to conduct epidemiological research and improve its collaboration with other government agencies to formulate health related policies and improve its public information function. 15. In physical terms, the vaccine production component will have met the project objectives when full vaccine production commences. At present, the three plants are close to physical  5 completion with final validation of the plant systems and their equipment expected by end 1999. Before normal production can start, however, completion of process development for measles and DPT, trial production, process validation, GMP training of new staff, work process, standard operating procedures completion, work process training, and then managing each of the facilities as an integrated operation, remain to be accomplished. Barring unforeseen difficulties, all five vaccines should be in licensed production by the end of 1999 or early 2000. 16. The three new facilities represent state-or-the-art technology in the field of vaccine production. These extremely complex plants have been well designed and house some of the world's most advanced systems and equipment in the field. It is expected that the production targets for all five types of vaccine can be met. Due to the size of the facilities and the sophistication of the systems and machinery, production expansion should be achievable substantially above the design capacities. It is also expected that the quality of the vaccines will meet international standards for potency and safety as established by WHO guidelines and that the operation of the facilities will be in accordance with European GMP. 17. This component was the first vaccine production project financed by the World Bank. It is also by far the largest, most complex and most expensive project in the field of biological production in China. In terms of overall production volume (420 million doses of vaccine) and size of the facilities, it most likely is the largest project of this type in the world. Its complexity in terms of technology and management would undoubtedly be a daunting challenge to even the most experienced Western pharmaceutical companies. It is therefore in hindsight of little surprise that the project design, procurement arrangements as well as implementation and funding requirements often exceeded both the domestic and professional, managerial, and financial resources throughout the project implementation period. 18. In terms of financial objectives, i.e., yielding significant improvement in the cost effectiveness of the national childhood immunization program, the project upon completion may not exhibit the same positive results commensurate with its physical achievements. While it was always recognized that the new vaccine products would be more costly than those currently produced. as they are of higher quality and safetv, preliminary cost studies prepared by the Medical Technology Assessment Research Center of Shanghai Medical University indicate that the cost of vaccines would be substantially above the cost of the current production. It is essential that the cost study be revised as soon as actual data becomes available during the trial production phase, and that conceptual problems concerning the analysis be resolved as soon as possible. Moreover, MOH has agreed to carry out a study to assess the cost savings effects resulting from the improved-quality vaccines as well as from better product packaging, freeze- drying, and longer shelf life. It is, however, not expected that these effects will fully offset the substantially increased cost for the new products. 19. Of great concern to the Government is the possibility that the full cost of the new vaccine products may exceed the UNICEF price level. The Government has expressed interest in requesting assistance from U'NICEF experts to assist in cost accounting and analysis. Moreover, UNICEF may also help advocate the policy debate for additional funding needed to assure that effective immunization levels are maintained in China.  6 D. IPLEMENTATION RECORD AND MAJOR FACTORS AFFECTING THE PROJECT 20. Annex tables to this report provide quantitative information regarding the implementation record of the entire project, including supervision support, disbursement, and compliance with project covenants. Overall, all project components, with the exception of the vaccine production component, were successfully impiemented within the original timeframne, and in accordance with project design and objectives. When the vaccine production component suffered from significant impikmentation problems. and attainment of development objectives was a concern, the project received "Unsatisfactory" ratings to highlight these issues. Regarding the rural health and non- vaccine production components, the Foreign Loan Office (FLO) within MOH drafted a project completion report and submitted it to the World Bank in 1992 (see Annex A). It reported that civil construction had been completed as planned, personnel recruitment and training surpassed the project targets, and equipment procurement had been 80% completed by June 30, 1992, the original project completion date. Implementation of the Rural Health and Other Non-Vaccine Production Components 21. The issues encountered during the implementation of these components were outlined in the Borrower's project completion report and are summarized below: (a) lack of funding for recurrent cost for: (i) EPSs at the provincial and county level; (ii) provincial (health equipment) maintenance centers and county MCH centers, contributing to low personnel productivity in the immunization program, lower quality services than the public's willingness to pay, via direct user charges, and lower than expected utilization; and (b) shortage of training materials for local level staff based on the new PHC health policies and changing pattern of health problems toward chronic diseases, and (c) Insufficient implementation flexibility from one province and county to another, ignoring local differences in facility and service need or demand, and leading to relatively poor resource allocation from one assisted locality to another. 22. These factors contributed to lower health status and service use improvements than was originally expected from the project's investments. The lack of project implementation flexibility, however, increased the desire of Chinese officials operating at the provincial level and below to achieve greater decision making flexibility and thereby devolve greater autonomy to sub-national levels of government. This experience was an important lesson learned by local officials and helped facilitate the implementation of the Health III Project, which was based on local decision making. 23. From available documentation, it appears that the drug control and operational research components were implemented without major problems. The construction and equipment required to implement these components were procured without major difficulty and related program development was implemented in a timely manner.  7 Implementation of the Vaccine Production Component 24. While other project components were implemented without major difficulty, except as noted above, the vaccine production component has been fraught with implementation difficulties from the beginning. Because of its significant technical requirements and lengthy delays in project implementation, by 1992/3 both the World Bank and MOH effectively split off this component from the rest of the project for purposes of supervision and management. This reduced the project design linkages conceptualized at the start of project development. 25. Procurement and Contracting. Both the Bank and MOH agreed at appraisal to have the component implemented as a turnkey operation by a reputable international vaccine producer. However, the preparation of a short list of qualified international vaccine producers led to a dispute regarding the criteria used to prequalify firms. Further, the task of preparing very technically complex bidding documents created unforeseen delays within the Chinese import agency responsible for expediting the bidding process. Negotiations with the lowest evaluated bidder on the first round became protracted and eventually collapsed in late 1988 over price, a number of technical issues, and conflicts regarding the role of each party in a turnkey contractual arrangement. 26. A revised approach to contracting this component was formulated by selecting an engineering firm which would be associated with an experienced vaccine producer. The Bank approved this approach in January 1989. After a year, a contract was signed in 1990 with a Dutch consortium, comprised of an engineering firm (EF) as the leading partner which designed and implemented the construction of the vaccine production facilities, a vaccine production institute with the technical knowledge required to develop the vaccine production processes, and a supplier for bioreactor equipment. The vaccines were supposed to be developed in a "joint development program" with Chinese vaccine experts. 27. Financing. As the contracting difficulties were ending in 1990, it was learned there were insufficient funds from the World Bank project to fully finance this component as envisioned during appraisal. This problem was mitigated in part by a grant from the Rotary Foundation of Rotary Intemational which provided US$15.0 million for the OPV plant facility in Kunming. But as the contract between the Dutch consortium was beginning full implementation in 1992, a further financing gap of about US$50 million emerged after non-used World Bank project funds had been reallocated to this component from other project components. This gap was due to several factors: (i) devaluation of China's currency (RMB); (ii) local inflation; (iii) higher than expected facility equipment prices; and (iv) low estimates of the component cost prior to project approval. 28. Between the date of Loan/Credit effectiveness in 1987 and contract signing in 1990, the RMB devalued by 40%, and by full project implementation in 1992, the RMB had devalued by an additional 10% from 1990, or a total devaluation of nearly 55% from 1987 to 1992. The devaluation, in conjunction with international price increases of the vaccine production equipment required by the plants, delayed equipment procurement. Local inflation followed the currency devaluation by about a year and also increased the local cost of constructing plant buildings. Between 1987 and 1990, domestic prices as measured by the GDP deflator, increased by 29% and  8 by 1992 the increase from 1987 was nearly 49%. These price increases for international and locally procured items could account for a large share of the cost increase of the component. 29. An additional factor possibly at work in this large procurement was due to the typical bidding strategy employed by international firms seeking to obtain a major government contract. As the turnkey bids by a short list of suppliers were used to estimate the cost of this project component, which were based on the usual practice of "low ball" bidding, this likely led to lower than true cost estimates of the project component. Such bidding tactics are often employed by firms when asked to bid on high technology content projects where the production processes have not yet been developed or where the implementation of the process is not well understood in the new site. The delays in implementation caused by the financing problem created by a combination of the above mentioned factors further delayed project implementation, and caused increasingly strained relationships between the Dutch consortium. MOH, and the three Chinese vaccine institutes. 30. The financing problem was eventually addressed by obtaining a supplemental appropriation from the Chinese government of about US$44 million in equal parts of local currency and foreign exchange and about US$5 million from the Dutch bilateral assistance program. Also, the strained relationships were resolved in September 1994 in part by the formulation of a Mid-Term Project Modification Plan and a detailed Action Plan. These documents included the financial commitments of both the Chinese and Dutch governments, and modifications to the engineering contract which altered original onerous guarantees and lowered know-how fees, without reducing vaccine quality below European GWM standards. Finally, the Vaccine Project Offices (VPOs) at the MOH and institute levels were also strengthened and the role of the EF redefined to shift greater implementation authority to the institutes. The Bank approved these documents in September 1994. 31. Production Process Development. By early 1995, additional problems regarding payment disputes, engineering drawing delays, supervision inadequacies, and lack of progress in production process development again had strained relationships between the implementing parties. The Bank recommended a neutral assessment by an independent expert of the status of production process development to resolve this aspect of the strained relationship. The results of this assessment were used by the Chinese Government to release the EF from its responsibility to develop a new DPT production process and to use an improved Chinese process for DPT, and an available roller-bottle rather than an unproven microcarrier technology for measles vaccine. These production decisions were to be implemented according to the standards promulgated by WHO and European GMP guidelines, thus continuing to ensure the possibility that output from these facilities could be internationally marketed after Chinese immunization requirements had been met. The Government and the EF settled all of their claims regarding this decision in an amicable manner. 32. In spite of the aforementioned implementation difficulties, it is heartening to learn that the facilities have virtually completed construction and equipment installation. This phase was fully completed at all three sites by the end of 1998. An additional year planned to fully test how the vaccine production processes will work at fill capacity. Bank and Government officials are cautiously optimistic that full production will commence in late 1999 or early 2000.  9 E. PROJECT SUSTAINABILITY 33. Sustainability must be addressed from at least two dimensions: financial sustainability of the investments supported by the project; and institutional sustainability of the function or service created by the investment. Further, for this project, an assessment of sustainability must be disaggregated according to the various project components. 34. Vaccine Production. First, during full operation, the vaccine production institutes will face serious recurrent financial issues. As they have received considerable state investment funding, and since the immunization program for which they were originally built continues to be a policy pnrofity, the institutes through the medium-term will be able to garner financial resources from the state as may be required. At present, they are not viewed as a state enterprise soon to be privatized. However, the cost of continuing to maintain GNP production standards will be daunting, especially as the production processes for DPT, measles, and OPV have not yet been fully tested in any plant. 35. The three plants were initially incorporated into the project to help China realize its objective of fully immunizing its children against the basic package of childhood communicable diseases. During the project's design phase, an economic assessment'/ was conducted of the choice between locally producing or importing vaccines required to reach its immunization target. That report found that domestic production of vaccines for the Expanded Program on Immunization (EPI) would likely be about half as costly per dose as imported vaccine. These estimates were based on assumptions about the level of output envisioned by the new plants, the capital cost of developing the new production capacity, and the recurrent cost of producing the vaccines, once the new production facilities were in place. Thus, based on estimates of the investment cost of the vaccine plants in 1985, the decision was made to include the three plants in Health 11 36. However, the large cost overruns of the three plants increased the original investment cost from US$40.6 million to over US$90 million in 1985 dollars. This estimate does not include a fill accounting of the costs yet to be incurred, as the production processes have not yet been verified, production tested, or reviewed by WHO. This situation raises serious concerns about the future cost of vaccines. A revised assessment of this past decision must also take into consideration several other aspects, including: (a) the present price of vaccines from UNICEF or other suppliers; 1 F. Orivel, "Economic Analysis of Vaccine Production: Choice Between Domestic Production and Import of Vaccines in China", World Bank Consultant Report or the Preparation of the Rural Health and Preventive Medicine Praject, (Washington D.C.: World Bank, August 1985).  10 (b) the envisioned production costs of the vaccines in the three new plants when they become operational; (c) whether the Chinese Government will close the existing plants that produce vaccines for the EPI; (d) the likely change of domestic and international demand for EPI vaccines over the next two decades; (e) the examination of whether the global production capacity existing in 1985 would have been able to fully respond to the import requirements of China at that time without an increase in global prices; and (f) an assessment of the prospect that China will be able to attain and maintain GMP production standards, and thereby obtain and sustain WHO certification of its quality standards, so that the plants might compete on the international market for vaccine production. 37. Finally, it may be instructive to take a broader perspective on the economic feasibility of this project component. Without quantitative information to conduct a formal assessment, as suggested above, it may still be instructive to view this investment in terms of the transfer of technology from international producers of pharmaceutical products, particularly in terms of the potential of producing new biotechnological products in the future. When this project component was under initial consideration for inclusion in 1985, a number of international pharmaceutical firms were approached for their interest in inclusion on the turnkey short list. One firm expressed interest in the project, not because they wanted to develop vaccine plants, but because they saw it as the vehicle for becoming involved in producing high-tech biological products in China for domestic and international markets. Several others were intrigued by the same possibility. Part of the difficult discussions between the finallv chosen contractor consortium and Chinese officials was over the issue of technology choice and ultimate transfer to China of the production processes. The contractor consortium saw considerable economic possibilities in the growing biotechnology product area from successfully refining microcarrier technology for the large scale production of measles vaccine. This project would have provided the vehicle for supporting this research work without bearing the fill cost of it. The Chinese government and the Dutch consortium have agreed to share the results of the microcarrier development undertaken by this project, even though they may not be using it in the two plants where measles is intended to be produced. 38. The Kunming facility is the most financially at risk vaccine production institute, as it will enter full production of liquid polio vaccine (OPV) at a time when the country will have effectively eradicated the disease. Thus, it will serve the domestic requirements for vaccine during the eradication maintenance phase for about five years until 2005. Therefore, plans should be initiated soon as to how to modify the production process with attendant additional costs to produce other vaccines or other pharmaceutical products for the domestic market and possible export.  11 39. Vaccine Production Institutional Sustainability. One of the sustainability issues facing Chinese officials is whether these vaccine institutes, once completed, will be able to retain the trained staff necessary to maintain GMP production standards in a rapidly changing economy. This is especially true where pnvatization is one of the dominant economic poiicy objectives for the future evolution of the country. MOH is aware of the potential problems it might face in this regard, and has already had to replace some officials who received training via the project and left their institute for more lucrative positions elsewhere. MOH is working to establish institutional approaches consistent with the policy of alternative ownership, which may reduce the likelihood that trained staff would leave in the future. 40. Sustainability of the Rural Health Component. Expanding preventive health care service delivery, along with improving MCH services and delivering basic health services in rural areas of the country, comprised the second largest component of financial support from the Bank through the project2". This component has demonstrated the benefits of improved services and better management, however, the costs of these improvements, including recurrent expenditures, have been significantly higher than in the non-project areas. The sustainability and replicability of these programs would clearly depend upon increased budgetary allocations by the government at all levels. While the project did suffer from implementation delays and insufficient counterpart funding and recurrent cost support, additional costs for these areas are expected to be well within the reach of government budgets, especially considering the currently low level of budgetary allocations for health services in China compared to most other countries. This has been well recognized in recent years, and demonstrated by 1997 State Council directives to increase budgetary allocations for public and preventive health services. Sustainability of this program should also be viewed from the utilization of services. It was clear in the project areas that the poorer families had significantly lower utilization of health facilities. This will continue to be a concern until the health financing system is reformed to improve access for the poorer families 41. In terms of institutional sustainability of the rural health care system, the training sub- component exceeded its expected targets. But no assessment was conducted of the quality of the training or its impact on the quality of service delivery or on indicators of management performance. 42. Drug Quality Control Component. Institutes in Beijing, Shanghai and Tianjin, and medical universities in Beijing, West China, and Zhejiang were provided assistance under the project. The Government's Project Completion Report reported that a number of persons had been trained for field inspectors, administrators, and technical positions within the field of drug quality control. In addition, about 13 persons had obtained overseas training for high level research, technical and supervisory tasks. In addition, a number of research and methodological papers were written and published in Chinese technical journals by high level staff of each institute receiving support, which contributed to the dissemination of the new drug quality control methods throughout the country. Finally, this project component has contributed to improving the periodic 2 This component was initially envisioned as the largest component from a cost perspective, over USS75 million, with Chinese goveaninet fmancing amotmting to over USS53 million and the World Bank contributing over USS22 million.  12 revisions of the Chinese Pharmacopoeia, the nationally used reference document for pharmaceutical products. 43. According to the director of one institute, there was an occasional problem of recurrent cost support from the Government, but the institute adjusted to that reality by charging for drug quality tests conducted by the institute. By virtue of the testing equipment acquired under the project (US$0.6 million), the institute was in a position to charge fees as they had up-to-date equipment and trained staff with the know-how to conduct these tests in a professional and quality-minded manner. 44. Support to the Chinese Academy of Preventive Medicine. Finally, the project supported institutional development and operational research of the Chinese Academy of Preventive Medicine in an amount of about US$8 million, with the World Bank's contribution under Health 1I being about US$1.9 million. This support continued the assistance provided under Health I and extended the capacity of the institute to conduct epidemiological research, disseminate its findings, and work with MOH and other Government officials to formulate health related policy and nationally disseminate the results. 45. Through the institutional capacity building supported by the project, CAPM evolved from a traditional public health research institute, financed solely by the Government through MOH, into an organization whose mission, as defined by the State Council, has expanded from basic public health research. Its activities now include: (i) training public health professionals; (ii) providing public health surveillance technical assistance to sub-national entities of government; (iii) conducting infectious and NCD disease surveillance, (iv) providing technical assistance for the promulgation of health laws and regulations; and (v) formulating practical approaches for the dissemination of public health information. Its govemmental financial support has grown and it has become more active in attracting finds from domestic and international sources. F. PERFORMANCE 46. The project was in fact implemented as two projects after the technical and price lessons were absorbed from the initial unsuccessful effort to engage an international firm to implement the turnkey approach embodied in the project design. The Bank and MOH were naive at the outset about how technically complicated it would be to develop large-scale vaccine production facilities with untested methods of production and GMP-quality standard requirements underlying the task. 47. These problems contrasted with the rural health care delivery component, which was designed according to national policy guidelines, and benefited from the lessons learned and approach taken under Health I. At that time, it was thought that locating health facilities closer to where people lived would lead to a greater utilization of both preventive and curative services and subsequently improve indicators of health status. The project did not stress the importance of project monitoring and evaluation, but in the Borrower's Project Completion Report there were summarized monitoring and evaluation indicators from the assisted counties suggesting a mixed  13 picture of project outcome in terms of improved health status and perfornmance. However, there was little thought given to other factors which could have contributed to the changes noted. BANK PERFORMANCE Rural Health and Other Non-Vaccine Production Components 48. The Bank's overall performance during preparation and supervision of the rural health and other non-vaccine production components was satisfactory, although it is acknowledged that the Bank might have been more proactive about recurrent cost issues. While various references were made in supervision mission Aide Memoires regarding the lack of recurrent cost financing for the preventive rural health sub-components of epidemic disease prevention and MCH, there appears to have been little dialogue about possible solutions or how the project components could mitigate its adverse impact. A comprehensive review at mid-term of implementation could have been a desirable vehicle to address these issues. Such mid-term reviews have been included in all subsequent Bank financed health projects in China, beginning with Health 111. In addition, economic oversight or participation on supervision missions should have been strengthened, and the Bank should have addressed the need to strengthen the national drug regulatory authority as part of the project. Vaccine Production Component 49. The Bank's performance during preparation and implementation of the vaccine production component was flawed. The Bank recognized during the project preparation stage that it had no experience in the preparation and supervision of vaccine production projects in China or elsewhere. It had hoped that its limited experience in industrial projects in China would be of assistance. However, the Bank was not prepared to recognize the extremely complex, costly and highly specialized nature of a vaccine production project. 50. During the post-appraisal mission for Health I in February 1984, discussions were held between the Bank and MOH about a possible second health project. In the beginning the Government wished assistance for the strengthening of all seven regional institutes of biological production, but the Bank felt that resources were too limited for such a comprehensive approach. In May 1985, the present concept emerged, including building three new facilities for DPT, measles, and OPV. A WHO sponsored consultant team prepared the investment program, tentatively estimating it at US$26 million. Subsequent preparation missions increased the estimate to US$46 million, which was still far too low, given that final component costs amounted to US$141 million. To overcome the lack of experience with this type of project, the Bank anticipated assistance in form of a turnkey operation by a major international producer with appropriate qualifications. This became necessary because only very few top-level consultants work in this field and companies for proprietary reasons do not provide information unless there is a significant contractual relationship. It was hoped that such a contractor could prepare preliminary cost estimates before finalizing the SAR. However, the SAR was issued before such an arrangement was consummated, thus depriving the project of a realistic cost estimate. The Bank therefore did not set aside sufficient financing in the Loan and Credit to support the vaccine production component. However, this was not realized until much later. When the financial gap  14 became apparent, the Bank secured a Grant in the amount of US$15 million from Rotary Intemational and about US$5 million from the Dutch bilateral assistance program. 51. The dialogue between the Bank and the Borrower during the preparation and implementation phases of the vaccine production component was constructive, informative and professional. The Bank's supervision effort of 22 missions, two of which were carried out after Loan/Credit closing, was comparatively high. But in light of the experienced problems, this may have been insufficient, particularly in identifying problems in a timely manner. However, it is questionable whether additional Bank supervisory resources would have overcome significant technical and contractual obstacles. The Bank's repeated proposal for the Govemment to retain under project financing a long-term consultant with international expertise in vaccine plant construction and vaccine production to assist the VPO and the three vaccine institutes, was declined. The Bank played an important part in reconciling as honest broker differences between the project owner and the EF, and initiated the mid-term project restructuring as well as a number of studies, including the vaccine cost study as part of the Mid-Term Modification Plan, and several assessment reviews by experts. A Bank mission also traveled to Geneva to meet officials who could provide funding assistance and expert advice. On several occasions, the Government specifically requested Bank missions to be fielded to provide assistance. 52. The Bank and the Borrower benefited substantially from the generous assistance by UNICEF which made a vaccine production expert available for a period of about ten years for some of the preparation missions and for almost all of the supervision missions. In addition, the project also benefited from the advice given by a senior representative of Rotary International during several supervision missions. During the preparation and supervision period of 14 years, only three Bank task managers were assigned to the component, providing a large degree of staff continuity. They had various degrees of medical, epidemiological, and financial expertise. BORROWER PERFORMANCE 53. Overall, the Borrower's performance during project development and implementation was satisfactory. The Borrower learned from implementation of Health I how to address World Bank procurement regulations so that this often vexing problem did not arise, with the exception of the vaccine production facilities. However, as this procurement was an unusually large and complex matter, the two sides worked very hard together to deal with the one-of-a-kind problems presented by this task. Rural Health and Other Non-Vaccine Production Components 54. Project investments in health care facilities of poor rural areas were concentrated at the county rather than at the township or village level. Thus, access to facility based care was improving at the county level as facility per-capita ratios fell at the county level but rose at the township and village. Similarly, greater facility construction occurred in hospitals (and corresponding equipment procurement) rather than in health centers or related facilities at the village.  15 55. The training sub-component was successful in training more personnel than anticipated at project launch. It focused on technical skill acquisition, rather than on improving the quality of service delivery. However, neither focus was explicitly incorporated into a monitoring framework, thus, little has been learned from the legacy of the training investment. What is reported however, is that, over the life of the project, there was a systematic reduction in the numbers of trained paramedics available in the assisted counties, reflecting low pay and lack of status. 56. The monitoring of disease pattern change within the project areas was inconclusive, in part due to a lack of good information regarding health status prior to the project and partly due to difficulties in determining project attribution. This was due to the monitoring strategy employed where information was collected at points of service use under the jurisdiction of the local authorities, rather than from households in the various project localities. Greater attention on the design and gathering of information is required in future projects. 57. In spite of the lack of project monitoring of health status change in project areas, the operational research component provided funds to CAPM for the purpose of: (a) strengthening capacity to conduct disease surveillance, and (b) testing disease prevention strategies for emerging public health problems, especially for chronic diseases. No information was reported in the 1992 Borrower's Project Completion Report about these activities or their outcome. However, a separate Project Component Completion Report was issued by CAPM3/, which demonstrated considerable institutional capacity strengthening and role expansion from the support of the two World Bank projects amounting to about US$8 million, nearly US$6 million from Health I and about US$2 million from Health II over the period 1984 to 1992. The funds from the Health II Project supported personnel training, technical assistance, program activity support, and equipment to address the five tasks which it had been assigned by the MOH via the State Council: (a) conduct research on important public health issues; (b) train provincial level professionals in public health methods, (c) provide technical assistance to provinces and below levels of government regarding outbreaks of disease; (d) provide the scientific basis for health laws; and (e) develop and promote the dissemination of public health information to the Chinese people. Through these investments, CAPM now has a strong research record with highly qualified personnel and has demonstrated its importance to the Chinese Government in addressing its five tasks. Consequently, they are now better able to respond to the needs of the policy makers to translate epidemiological and other public health research findings into more relevant public health policy options. 58. Finally, the rural health insurance experiment was designed and implemented in a timely and professional manner, with a good degree of technological transfer between the external consultant group contracted to perform the experiment and Chinese counterparts. This transfer of health economic research skills was also strengthened at this time by the development of a network of health economics researchers which received financial and technical assistance from 3. Chinese Academy of Preventive Medicine, Final Report on the Implernentation of Health I and 11 Proiects, (Beijing: Chinesw Academy of Preventive Medicine, April 1992).  16 the Economic Development Institute of the World Bank. The findings from this research were published in the Rural Health Insurance Handbook and have been available to the health policy researchers and the country's emerging cadre of health economists, and it has positively influenced the thinking of many involved with formulating health financing policy in China. Vaccine Production Component 59. A leading group, headed by the Minister of Health, was established to oversee the implementation of the project. While the three vaccine institutes had a large degree of autonomy in project implementation, overall project coordination was carried out in a satisfactory manner by the VPO established specifically for this purpose. Personnel assigned to the VPO as well as to the institute level project offices were invariably highly motivated. None of them, however, had been previously involved in a project of similar dimensions and complexity. This was initially not seen as a deficiency as the vaccine scientists and production experts were supposed to work under the Joint Development Program to develop the technological processes in collaboration with the Dutch vaccine institute, and the engineering firm was supposed to be in charge of the design and construction of the buildings and the installation of systems and equipment. However, soon after construction started it was realized that project staff on the side of the project owner needed to be strengthened, but particularly at the center it was difficult to get additional positions, especially from the engineering field, for project supervision. Compared to the number of specialists and skills which would be normally assigned if a project of this size would be implemented in the private sector, the VPO and the project offices at the three institutes had to handle their tasks with but a fraction of such resources. That they were, nevertheless, able to ultimately fulfill their responsibilities, should be stated with admiration. G. ASSESSMENT OF OUTCOME 60. The two distinct parts of the project had different outcomes. Overall, the outcome of the rural and other non-vaccine production components was satisfactory but the vaccine production component remained deficient. The rural health development component succeeded in achieving its basic objectives of upgrading the quality of preventive, MCH and basic health services in rural areas and expanding access. However, recurrent cost deficiencies at the local level prevented the realization of its fuill potential. Through training and research efforts, the drug quality component improved the quality and safety of domestic and imported pharmaceuticals. In addition, research and methodological papers were wfitten and published, which contributed to nationwide dissemination of the new drug quality control methods, and improvements were made to periodic revisions of the Chinese Pharmacopoeia, the nationally used reference document for pharmaceutical products. The operational research component strengthened CAPM's capacity to conduct epidemiological research and improved its cooperation with other government agencies to formulate health related policies and augment its public information function. A key outcome of this component was to successfully evaluate new rural health insurance schemes, which provided the govemment with experience and knowledge on the design of health financing mechanisms, and helped contribute toward filling the vacuum left by the demise of the rural CMS.  '7 61. The vaccine production component had mixed results characterized by substantial implementation delays and cost overruns. Nevertheless, it would broadly achieve its goal when full vaccine production commences by the end 1999 or early 2000. Upon completion, China will have three of the largest, and among the most modem, vaccine production facilities in the world, which will enable full replacement of its current deficient production capacity in OPV, DPT and measles, and provide its children with safe and efficacious vaccines. It is expected that these vaccines will meet WHO standards and that the plant operations will meet European GMP standards. This would be important for exports since these plants are likely to have excess capacity. 62. Another important outcome of the project is the substantial transfer of biotechnology. Despite extreme difficulties and contract disputes with internationaL contractors, design engineers, vaccine experts, and equipment suppliers, the hundreds of Chinese officials people involved in the preparation and the execution of the project have gained invaluable experience in the state-of-the- art vaccine plant design, construction and operation. In addition, through the Joint Development Program, Chinese experts have also gained extensive expenrence to work with overseas counterpart scientist colleagues. These highly trained and motivated scientists and managers working for the three vaccine institutes are crucial for the operation of the new facilities. The remaining important challenge for China would be how far the production costs will be competitive with other local production and international competitors. H. FUTURE OPERATION 63. Project implementation succeeded in improving rural heaith infrastructure, equipping health facilities and training health workers, which improved the quality and delivery of rural health services. The quality of their fiture operation, including utilization and expansion, would greatlv depend on how fast local leaders change their priorities toward increased support for preventive and basic health care by rediscovering their virtues. The main issues are, first, whether the government will recognize the need to nurture these gains by allocating adequate budgets for continued public health and basic care and, second, how soon the arrangements would be established for risk-sharing and for assisting the poorest families to increase their access to these services. As noted in the Bank's recent health sector study report, CHINA: Issues and Options in Health Financing, 1996, the government's attention to these critical health areas has been inadequate, resource allocations have deteriorated, and health gains in some basic areas have stagnated and may even have reversed. On the other hand, recent government pronouncements on this subject have provided reassurances that the national mood has been reinvigorated in favor of investments in these areas. 64. Building on the experiences of the Health 1I Project, improvement of preventive and basic health care programs has been included in a number of subsequent Bank financed health projects in China. The Integrated Regional Health Development Project (Health III; Cr. 2009-CHA), the Comprehensive Maternal and Child Health Care Project (Health VI; Cr. 2655-CHA), the Disease Prevention Project (Health VII; Cr. 2794-CHA), and the Basic Health Services Project (Health VIII; Cr. 3075-CHA) have directly benefited from the design and implementation lessons of the  18 Health 1I Project and have progressively made firther improvements. Combined with improvement of health services, the Health 1I Project also provided MOH and the five project provinces with valuable exposure to modem management practices which has greatly benefited subsequent programs. CAPM has evolved into a large and well-respected institution, comparable in many respects to the best international institutions involved in health and medical research, and has become an important partner in many subsequent Bank supported projects. 65. Efforts to establish experimental health financing mechanisms and alternative delivery modalities, such as Health Maintenance Organizations and the building of necessary regulatory structures to ensure performance in line with service quality and equity guidelines, represent areas where Bank assistance may be helpful for improving the health status and service use of both rural and urban populations in China. 66. Given the recent policy guidance by the State Council to privatize much of the state enterprise structure, it would be well to assess how the Bank, possibly through collaboration with the International Finance Corporation, could assist China in developing a privatized vaccine production entity, with potential for producing other essential pharmaceutical items on a commercial basis. The country may be ripe for exploring alternative ownership arrangements for many activities within the domain of the health sector. Support for privatizing the vaccine plants may be a logical first step, along many, to begin implementing the policy guidance of the State Council within the health sector. However, MOH officials are not fully convinced about this approach. There are nevertheless already arrangements underway with international pharmaceutical companies which could accelerate technological transfer and assist in opening markets for Chinese vaccines overseas. In addition, an overall viability study for vaccine production in China would be helpful in determining the future use of the new vaccine facilities in light of their cost structure and the anticipated changes in global vaccine markets. 1. KEY LESSONS LEARNED 67. Through the experience gained from implementing this project, the key lessons learned can be grouped into two categories: Rural Health and Other Non-Vaccine Production, and Vaccine Production. Rural Health and Other Non-Vaccine Production: (a) Pay attention to recurrent cost in the design and implementation of "low-cost" rural health care projects, especially where there are externalities and public good benefits accruing therefrom. (b) Design project monitoring and evaluation frameworks and how they will be implemented as an integral aspect of the project fornulation phase. The use of the logical framework as an important design tool would provide immeasurable help in focusing attention on relevant indicators for regular and periodic managerial assessment of performance.  19 (c) Develop a research agenda to be financed via the project, which has some possibility of operational implementation testing and verification during the project's life. The findings from the rural health insurance experiment conducted during this project have been widely distributed and are still being utilized - a decade later - in the policy formulation process to address the problem of health financing. (d) The funds from this project which supported institutional development in the areas of drug quality control and preventive medicine have resulted in significant gains to the health development capacity of the countrv. Where there are similar opportunities the Bank should assist to the extent possible. Vaccine Production: (a) Vaccine production is a highly sophisticated technical undertaking, for which the Bank has insufficient expertise in design or supervision. It requires technical expertise normally only found in the industry, which, for proprietary reasons is not eager to share with others. Further, process development and up-scaling to large production volumes is a difficult undertaking and can pose daunting challenges to the largest and best firms in the industry. (b) To take on the challenge of building three large-scale production facilities in one project, without doing the project in phases with the first phase being a pilot undertaking with one facility, has added additional complexity and stress beyond the capacity of even the largest pharmaceutical companies. It would therefore have been desirable to start with one plant which is the least complex (in this case the OPV plant) and include a second plant in a follow-up project provided there is reasonable progress on the first. (c) Ensure turnkey contractors have successfully developed vaccine projects (or other technicaily complex products) of a similar size as the one contemplated in the project and the contractor has firsthand knowledge of how to transfer that technology to the Borrower. In addition, design and construction of vaccine plants should only start when the development process has been successfully established, tested and sufficiently scaled up to the anticipated production output. (d) If the Bank has decided to become involved in similar projects, it should insist, after careful review of potential local project staff, that the Borrower retain the necessary expertise from any locality, domestic or internationally, to manage the implementation of the envisioned undertaking and require the involvement of WHO experts throughout the design and implementation phases. (e) Since a vaccine production project is not a community health project, but to a large degree an industrial project, its feasibility should be assessed according to normal financial and economic methods of project appraisal prior to making a financing commitment. Such assessments will encourage realistic discussions about product pricing, potential markets and possible market changes as well as private sector  20 participation and other relevant economic development policy implications. Various altemative scenarios could be developed to test the sensitivity of various assumptions regarding the project's economic feasibility. (f) Improvements in vaccine production can only be successful if the respective control authorities are sufficiently strengthened in terms of authority, policy development, structure, management, and enforcement capabilities. In addition, physical investment should only commence after a national strategic plan ensuring viable production has been developed and periodically reassessed. including prioritization of improvements and investments.  2 1 PART II: STATISTICAL TABLES TABLE 1: SUMMARY OF ASSESSMENTS A. Achievement of Ohjectives Suhstantial Partial Negligible Not Applicable Macroeconomic Policics Sector Policies / Financial Objectives Institutional Development / Physical Objectives / Povertv Reduction Gender Issues Other Social Objectives Environmental Objectives Public Sector Management V Privatc Sector Dec%clopmcnt I Other (specidN) B. Project Sustainahility Likely Unlikely Uncertain C. Bank Perfornance Highly Satisfactory Satisfacton Deficient Identification V Preparation Assistance Appraisal 1/ / Supermsion I/ V Note: 1/ For the rural healthi and other non-vaccine production components. the Bank's perfornance was satisfactorv during appraisal and supervision. For the vaccine production component. however, the Bank's performance was flawed. This was demonstrated by failure to predict the highly complex technical requirements and large cost overruns of vaccine production. and to reduec their impacts in a more timelv manner. D. Borrower Perfornance Highly Satisfactory Satisfactorv Defacient Preparation Implementation V Covenant Compliance / E Assesment of Outcome Highly Satisfactor Unsatisfactory Highly Satisfacton UnsatisfactorY Vaccine Component Other Components V  22 TABLE 2: RELATED BANK LOANS/CREDITS Loan/Credit Title Purpose Year of Stahs Approval Rural Health and Medical Educaion Project Extend coverage of PHC and 1984 Completed (Cr. 1472-CHA) improve quality of medical education. Integrated Regional Health Dcvelopment Strengthen regional capacitv for 1989 Completed Project planuiing and management. (Cr. 2009-CHA) experiment with innovative wavs to improve health education. disease prevention and suvecillance. MCH cmergcncv scrvices. hospital sevices, rehabilitative services. medical education and training, and equipment management. Strengthen MOH capacity to oversee and manage innovation. Rural Health Workers Development Project Strengthen health worker planning. 1993 Ongoing (Cr. 2539-CHA) retrain health workers and improve training, improve management and working conditions of workers, and improve MOH capacitv to manage change in personnel manttrs. Infectious and Endemic Disease Control Support Tuberculosis and 1991 Ongoing Project Schistosomiasis control. and (Cr. 2317-CHA) improve institutional capacitv for infectious discasc control of all types. Comprehensive Maternal and Child Healtih Reduce maternal and child monalitv 1995 Ongoing Project and morbidity through improving (Cr. 2655-CHA) basic health services delivey,. health wvorker training. management improvemenL and improving MOH capacity to nmnage these changes. Iodine Deficiency Disorders Control Project Increase use of iodized salt to reduce 1995 Ongoing (Ln. 3914/Cr. 2756-CHA) iodine deficiencv disorders. strengthen the legal framework and improve management. Diseasc Prevention Project Reduce vaccine preventable disease, 19% Ongoing (Cr. 2794-CHA) and design and implement health promotion programs to control the rising prevalence of NCDs. sexually transmitted diseases (STDs), human immunodeficiency virus (HIV), and injury. Strengthen MOH capacity to manage these initiatives.  23 LoAnCredit Title Purpose Year of Stau Apprval Basic Health Scrvices Project Assist with achieving sustainable 1998 Ongoing (Cr. 3075-CHA) health improvement in poor mrual areas through improved allocation and management of health rcsources, upgrading health facilities, improved qualitv and ceffectiveness of health services. and grcater financial access to essential health care for the poor. Health Nine Project Improve maternal health and child Under development. and improve preparation prevention and control of HlV/AIDS/STDs and other blood born infections.  24 TABLE 3: PROJECT TIMETABLE Steps in Project Cycle Date Planned Date Actual Identification October 1984 October 1984 Appraisal May 1985 Julv 13. 1985 Negotiauions Fcbnzaly 1986 April 21. 1986 Board Presentation March 1986 June 19. 1986 Signing N/A Januarv 15. 1987 Effectiveness N/A Mav 26. 1987 Project Completion June 30. 1991 Junc 30. 1996 Loan/Credit Closing June 30. 1992 June 30. 1996 N/A: Not applicabic. TABLE 4: LOAN/CREDIT DISBURSEMENT: CUMULATIVE ESTIMATE AND ACTUAL (US$ MILLION) FY87 FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 FY96 FY97 Appraisal Estimate 8.9 24.1 45 6 64.0 76.0 80.0 -.- .. - -.- - _ Actual 9.0 21.4 23.3 39.1 50.4 56.0 62.0 78.2 83.5 90.3 93.85 Actualas%of 101.1 88.8 51.1 61.1 66.3 70.0 -.- -.- -- -- Estmate Date of Final October 10. 1996 Disbursement Note: Tnc difference bctwcen appraisal cstiiiatc and actual Loan/Crcdit disburscmcnt is duc to fluctuation of thc SDR exchange ratc. TABLE 5: STUDIES INCLUDED IN PROJECT No. Study Conducted By 1. Rural Cooperative Health Insurance Schcmes Ministry of Health 2. Vaccine Production Institutes Cost Stud% Shanghai Medical University 3. GMP Training Needs GMP Expert. United Kingdom. retained by the Ministry of Health  25 TABLE 6A: PROJECT COSTS (US$ MILLION) Project Component Appraisal Eslimate Actual/Latest Estimate Local ForeiLrn Total Local Foreien Total I/ Slrengthening Rural Healdi Scrniccs 53.3 22.3 75.6 52.6 31.6 84.2 Vaccine Component 11.2 29.4 40.6 44.7 %.5 141.2 Drug Qualitv Control 7.6 4.2 11.8 9.2 4.6 13.8 Operational Research 6.3 2.4 8.7 6.7 2.2 8.9 Physical Contingencies 8.5 10.0 18.5 Price Contingencies 11.8 10.4 22.2 Total Project Cost 987 787 177.4 113.2 134.9 248.1 Note: I/ Includes USS15.0 million Rotary International Grant. and USS4.5 million Dutch Development Assistancc. TABLE 6B: PROJECT COSTS (RMB MILLION) 1/ Project Component Apnraisal Estimate Actual/Latest Estimate Local Forcian Total Local Forei!n Total Strengthening Rural Hcalth Scrniccs 440.8 184.4 625.2 436.6 262.3 698.9 Vaccine Component 92.6 243.1 335.8 371.0 801.0 1,172.0 Drug Quality Control 62.9 34.7 97.6 76.4 38.2 114.5 Operational Research 52.1 19.8 71.9 55.6 18.3 73.9 Phvsical Contingencics 70.3 82.7 153.0 Price Contingencics 97.6 86.0 183.6 Total Project Cost 816.2 650.8 1.467.1 939.6 1.1!9.8 2.059.3 Note: 1/ Exchange ratc at project compiction used: US$ 1.00 = RMB 8.27. TABLE 6C: PROJECT FINANCING 1/ (US$ MILLION) Source Appraisal Estimate Actual/Latest Estimate Local Foreie-n Total Local Foreign Total Govenment 97.4 0.0 97.4 111.7 23.1 134.8 IBRD 0.0 15.0 15.0 0.0 15.0 15.0 IDA 1.3 63.7 65.0 1.5 77.3 78.8 Rotary International 1/ -.- -.- -.- 0.0 15.0 15.0 Dutch Development Assistance 2/ -.- -.- -.- 0.0 4.5 4.5 Total 98.7 78.7 177.4 113.2 134.9 248.1 Notes: 1/ In 1990, after the project was approved. Rotarv International provided a grant of US$15.0 million to bdp finance the foreign exchange costs of establishing the Oral Polio Vaccine Plant in Yunnan Province- 2/ Dutch Development Assistan ORET. wvas utilized for sole source equipment.  26 TABLE 7: STATUS OF LEGAL COVENANTS Agreement Section Covenant Present Desciption of Covenant Type Status Credit Article IIL 3.01 (c) 3 C Governnent to mnake proceds of credit available to local entitles. Credit Article IIL 3.05 10 C Borrower shall unplemcnt expernments in two counties in Sichuan Province regarding alternative approaches to rural health insur with terms of reference and time schedule acceptable to the Association. Credit Article 11, 3.06 5 C Borrower will maintain World Bank Loan Office in Ministry of Health. Credit Artiicle IV, 4.01 I C Adequate records. accountsandauditing procedures by the Borrower. Credit Article IV, 4.02 10 NY Production of vaccmes to meet intemnational guidelines established by the World Health Oranrzation on the qualitv of biological products, due one year after start-up of operation of the production centers. Project Article 1, 2.01 (a) I0 C Each local entitv shall provide funds, facilities services, and other resources required for the Project Project Article II 2.01 (b) 4 C Each local entitv shall make adequate and timely budgeting provisions for project implmaentation. Project Article IL 2.05 10 C Each local entity shall maintain a World Bank office with appropnate functions and staffing. Project Article 1m I c Adequate records, accounts and auditing procedures by the local entities. Covenant Class: Status: I= Accounts/audits 8 = Indigenous people C = covenant complied with 2= Financial perlormance/revenue 9 = Monitoring, review. and reporting CD = complied with after delav generation trom beneticiaries 10 = Project implementation not CP = complied with partiallv = Flow and utilization of project covered bv categones i-9 NY = not vet due funds 11 = Sectoral or cross-sectoral 4 = Counterpart funding budgetarv or other resources 5 = Management aspects of the allocation project or executing agency 12 = Sectorai or cross-sectoral policv/ 6 = Environmental covenants regulatorv/lnstitutional action 7 = Involuntary resettlement 13 = Other TABLE 8: COMPLIANCE WITH OPERATIONAL MANUAL STATEMENTS There was no significant lack of compliance with an applicable Bank Operational Manual Staternents (OD or OP/BP)  27 TABLE 9A: BANK RESOURCES: STAFF INPUTS (ACTUAL STAFF WEEKS) Fiscal Year Stage 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 . 1995 1996 1997 1996 Total Preparation 7.3 20.3 27.6 to Appraisal Appmisal 75.7 7.7 Negouation 24.4 24.4 to Board Presentation Supervision 10.6 2.0 6 h 14.2 6.4 10.9 10.1 15.7 14.1 15.9 17.6 2.8 127.1 Complicuon 13.1 13.1 Total 7.3 20).3 100.1 10.6 2.0 6.8 14.2 6.4 10.9 10.1 15.7 14.1 11.9 17.6 159 140.2 TABLE 9B: BANK RESOURCES: STAFF INPUTS (ACTUAL USS'000) Fiscal Year Stage 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 Total Preparauon 15.6 89 4 to Appraisal Appraisal 176.2 Negouauon 270.8 to Board Presenauion Supervision 28.4 5.6 18.8 40.3 20.6 37.9 34.9 55.3 49.4 39.3 51.5 4.3 386.3 Completion 527 52.7 Total 15.6 89.4 447.0 2&4 5.6 18.8 40.3 20.6 37.9 34.9 55.3 49.4 39.3 51.5 57.0 439.0  28 TABLE 10: BANK RESOURCES: MISSIONS Performance Rating Stage of Project Cvcie Month/ Number of Days Specialized Staff Imple- Devel- Year Persons in Field Skills Represented mentation *pment /a Status lh Objectives Pre-ldentification 02/84 2 n.a. EC. PiH Pre-Identification 04/84 2 9 EC, PI I Pre-Identification 07/84 3 n.a. PH. PL Identification 10/84 2 16 EC, PM Preparation 01/85 8 32 EC, HE, PH. PM Preparation 02/85 1 9 EC Preparation 03/85 2 n.a. EC, PH Preparation 07/85 3 6 EC, HE. Pli Appraisal 07/85 8 28 EC. HE. 00. PI-1. PR. PM Plost-Appraisal 12/85 3 5 1 lE. LO. PI I Supervision 1 10/86 1 5 HIE I I Supervision 2 09/87 1 n a. I'M I I Supervision 3 09/88 I n.a. H:E I 1 Supervision 4 12/88 1 15 HE I I Supervision 5 09/89 2 21 HE. PH 2 1 Supervision 6 10/90 I n.a. HE 2 1 Suipervision 7 12/90 3 9 AC, HE. VP 2 1 Supervision 8 09/91 3 9 AC, HE. VP I I Supervision 9 01/92 2 9 IIE, VP I I Supervision 10 04/93 3 11 AC, I-E. VP I I Supervision I1 05/93 3 n.a. I1E, OA. VP 3 3 Supervision 12 10/93 2 3 HE, VP 3 3 Supervision 13 11/93 2 3 HE, VP 3 3 Supervision 14 03/94 1 1 HE 3 3 Supervision 15 05/94 3 6 AC, HE, VP S S Supervision 16 08/94 1 1 IIE S S Supervision 17 (3/95 2 i IHE. VP S S Supervision 18 12/95 3 7 lIE. VP U S Supervision 19 0(/96 3 9 AC. IE. VP S S Supervision 20 03/97 4 9 AC. HE. OA. VP S S Supervision 21/Completion 0/.198 2 19 EC, IIE S S Completion 04/98 1 20 EC S S Notes: /a AC: Accounting and Auditing: EC: Economist: HE: Health Economist: HP: Health Promotion Specialist: LO: Loan Officer: 00: Operations Analyst: PH: Public Health Specialist: PL: Health Planning and Management Specialist: PM: Project Management Specialist: PR: Pharmacist: VP: Vaccine Production Specialist. A, I/HS: Highly Satisfactory: 2/S: Satisfactory: 3/U: Unsatisfactorv. n.a. Not available.  29 ANNEX A BORROWER'S CONTRIBUTION TO THE ICR Part 1: Rural Health and Other Non-Vaccine Production Components The Borrower's contribution to the ICR for the rural health and non-vaccine production components, "Project Completion Report," was submitted to the Bank in 1992. Following are two sections from the Report: Background and Project Results. The full document is available in the Project Files upon request. 1. Background I. I Since the early 1950s, China has made remarkable progress in improving its population's general public health and social sanitation conditions. However, there has been unbalanced development in terms of setting up health care deliverv systems and the provision of health services. The reason could be largely contributed to the geographic regional differences where social, cultural and economic situations have been shaped in its duet historical course. 1.2 As many of the health issues and health related issues are paralleled with the society's general economic stage, and although the reported national average health indicators/index reveal the successful control of the common communicable diseases and the fulfillment of the "First Health Revolution," those averages in fact conceal the wide variations among the different regions. Hence, the general consensus in China's health community is to promote the prevention and treatment of communicable and endemic diseases in remote and poor rural areas and at the same time, promote the early detection and prevention of risk factors of chronic conditions in urban/rural areas. 1.3 With the impact and the influence by the nation's economic reform, particularly the decentralization of financial responsibility, certain kinds of the so-called health reform are evident in some places in the 1980s, such as introducing fee-for-service mechanisms and inducing revenue-earning activities. But, on the other hand, the preventive work has shrunk because of the shortage of public funding and, due to the collapse of the commune system, the rural medical cooperative system which is supposed to serve the farmers is largely abandoned. The fact is since the country is trying to attain its economic/production goals through economic restructuring, its society's public services have comparatively been weakened. Aid, especially external aid, to support the public programs becomes necessarily important. 1.4 The formulation of health policies in the mid-1980s is aimed at existing issues that the health sector itself faces: (a) Consolidation of past gains; (b) Continued control of communicable diseases; and (c) Effective and efficacious responses to the new challenge of the changing  30 ANNEX A epidemiological profile and the emerging older age structure. 1.5 As the second health project financed in part by Bank loans and coming after assessment of China's achievement and prospects by the Bank, as reported in "The Health Sector in China," this project is to support the current policies in executing the national immunization program, improving rural health services, training health professionals and workers in different fields and at different levels, and strengthening preventive medicine. The project's rural health component is part of the national One-Third County Program. 2. Project Results 2.1 Overall, the project has been implemented as expected. The leading agency for managing the project operation, the Ministry's FLO, has been involved closely in every aspect of project activities, i.e., supervision of equipment procurement and program activities, financial accounting, as well as much of the painstaking coordination work. Observations and documents have been cumulated for files, and progress reports have been requested from the implementing entities for periodic evaluations. For the purpose of the project completion report, the FLO has made their special requirement for the project beneficiaries to conduct their own evaluations based on the agreed investigations and evaluation designs, and these data and investigation materials should be presented to the FLO for summary and analysis. 2.2 For the evaluation of the rural health component, a comprehensive investigation was conducted in the fifty project counties, and a before-after self-comparison evaluation indicators/index was constructed based on the following five categories: (a) Basic social economic status; (b) Health service conditions (in terms of civil construction, equipment, training); (c) Rural health service facilities/institutions and manpower; (d) Medical and health services/departments development; and (e) Health service delivery and its impact. 2.3 Based on the structure/process/impact evaluation design, the basic social economic status is a general framework or an overall structure which the evaluation will fit in; the health service conditions are the project inputs which are part of the structure indicators; the rural health service facilities/institutions and manpower are the extension of the service institutions coverage and rural manpower development/upgrading, which are process indicators on the health delivery part; the medical and health service/department development is the department/service items creations and articles publications, which are also process indicators on the health delivery part; and the health service delivery and its impact is self-explanatory. The before-after comparison duration is from 1986 to 1992, and the benefit population is as calculated, i.e., about 30 million which is 14% of the aggregate population in those five poor provinces. 2.4 The total evaluation indexes are 129, and only part of them were selected. For the health  31 ANNEX A service delivery and its impact, originally, there were a total 54 indexes, but in consideration of its comparative importance, we have only selected 14 index/indicators which have shown the devastating health conditions in rural areas. For the remaining 40 index/indicators, they have all shown the positive changes of health conditions and health outcomes. 2.5 With regard to the medical service and management, except the length of stay (LOS) in Sichuan has increased a little bit, all of the rest of the indexes show a positive change, which is an indication of health service quality improvement. Considering that the emergency case treatment success increase rate is not much higher, and the large investment at county level in Sichuan province, the negative LOS could be a point of serious case referral from the lower levels. 2.6 The health and public education indexes have all shown a good amount of work done, but they may not target specific risk factors in the local areas as we have already seen the increase of acute hepatitis A disease, so their effect needs to be further evaluated. But as general health knowledge dissemination, the middle school health education course is a good approach for health information sharing. 2.7 Overall, the project had the following merits: (a) Improvement of the income levels of those poorer counties; (b) Improvement of the service qualities and health facilities; and (c) Reduction maternal and child mortality.  32 ANNEX A Part tl: Vaccine Production Component 1. Background 1.1 Under the Rural Health and Preventive Medicine Project, or Health II Project, a vaccine component was established for which IDA and IBRD funding was provided to construct three new vaccine production facilities. The component was initiated in the middle of 1984 and was actually prepared in 1985, when a turnkey approach was envisaged to select a vaccine producer through open bidding to take overall responsibility for all steps of designing, constructing, equipping, commissioning and operating the facilities during the testing and initial production phase. Both the initial bidding and subsequent protracted negotiations with a Canadian vaccine production company ended in failure primarily due to underestimation of the cost for the facilities. 1.2 As a result of successful efforts to secure additional funds including a Rotary Grant, a new semi turnkey approach was designed to invite an engineering firm subcontracted to a vaccine producer as its partner. With foreign expert legal assistance, a contract was finally awarded to a Dutch Consultant firm combined with its subcontractors as vaccine production process tranferer and a Dutch Company as process equipment supplied, generally known as the Engineering Firm. In fact, the contract is a foreign engineering professional service and vaccine production know- how transfer project. 1.3 The Contract was signed between the EF, EEC, the Purchaser and the Ministry of Health (MOH) on January 25, 1990 and became effective on May 8, 1990. Hence the vaccine component or the vaccine project eventually began to move after 4 years of twists and turns. 2. Project Objective 2.1 The PO under the Rural Health and Preventive Medicine Project sought to establish three new Vaccine Production Facilities. One facility for the Production of OPV will be located within the Kunming Institute. The two others, each for the production of DPT vaccine, TT and lyophilized EVE have attenuated measles vaccine, will be located within the Lanzhou Institute and the Shanghai Institute. The quality of the vaccines should meet WHO requirements concerned and the performance and operation of the facilities will be up to the European Good Manufacturing Practice with a view to improve the quality of vaccines and cost effectiveness of the national immunization program against the main childhood diseases. 3. Project Design and Organization 3.1 The vaccine project is so far the largest one of technical renovation in the field of biological production in China. It was designed to upgrade the quality of vaccines under EPI in accordance with WHO requirements and performance and operation meeting European GMP. 3.2 It was envisaged that the existing production process for the five vaccines in China would be phased out after successful finalization of the vaccine project. According to the contract signed between the PO and the EF, microcarrier fermentation process with VERO cells will be  33 ANNEX A used for OPV production at 350L working volume in Kunming facility, current conventional technology with 800L scale-up fermenters will be applied for DPT production and microcarrier fermentation process will also be used for MV production. With production process improved, the annual output of the vaccines produced in the three new facilities should meet the requirements as indicated in the following chart. Host Institutes Products Annual Output (doses) Lanzhou/Shanghai Measles Vaccine 40 million Lanzhou/Shanghai DPT 200 million Lanzhou/Shanghai TT 80 million Kunming OPV 100 million 3.3 The contract started to be implemented in June 6, 1990 and was anticipated to be completed in 66 months with the prerequisite that the EF's three guarantees (Product Guarantee, Facility Guarantee, and Facility Performance Assurance) would be granted by the Project Owner. Due to mid-term modification of the contract, only Product Guarantee was retained. 3.4 To provide overall management and coordination for the project implementation, a vaccine project office (VPO) was established in Beijing under the Project Leading Group of MOH, chaired by Minister Chen Minzhang and composed of key members of FLO, DPF and CNBPC. A similar managerial structure was set up in the three host institutes to meet the project implementation requirements. All these organization set-ups ensure the smooth communications, coordination and implementation of the project. 4. Project Implementation 4.1 The project implementation generally went well in spite of its great complexity and difficulties. The project is now at the stage of facility and equipment validation. With acceptance of OPV and MV processes, trial production will start soon after validation. The indicators as listed in the contract to monitor overall project progress have been mostly met with efforts of parties involved in the Project. 4.2 From the effective date, the contract was originally anticipated to be completed within 66 months or by December 6, 1995. Now the project has more than two years behind the contractual time. The reasons for such delays were compounded by a number of factors which may well be listed as: (a) funding crisis due to underestimation of project budget, inflation, fluctuation in foreign exchange rates and higher bidding prices of equipment than originally estimated; and (b) lack of experience in managing and operating such a complicated project on the side of all parties involved. 4.3 The well-known engineering firm has great international experience in working with Asian  34 ANNEX A countries, but it seemed insufficiently prepared to develop a complicated project involving three independent institutes and other related parties in China, and had encountered one after another difficulty in addressing the problems experienced. Though known for its reputation in vaccine research and production, the subcontractor has no experience in transferring vaccine technology in such a scale and with such updated technique; and cultural differences and insufficient communication have added to the above difficulties. Therefore, the project delay has become an undeniable fact and the process development remains a bottleneck for successful completion of the whole project. However, efforts have been made between the PO and the EF to overcome all the difficulties experienced during the project implementation. 4.4 In its midway, the PO was confronted with a serious situation due to funding shortage, which consequentially led to the Bank's action to suspend the equipment procurement and disbursement and request to reassess the project. After reporting to the Govemmment and consultation with the Bank, an additional fund of USD 22 million was secured and a strategy in modifying the contract were agreed on. The Project Modification Plan was signed by the ,O and the EF and approved by the Bank in late 1994. As a result, the mid-way crisis was over. 4.5 The training goal surpassed expectations within the budget estimated in the contract; The original training plan in the contract were satisfactorily completed. The remaining surplus supported a GMP training assessment by extemal assistance and a new training program based on the assessment report. There have been technical and management training activities abroad in total amount of more than 550 persons/months in the framework of the contract and equipment training was provided 200 times. Efficient and effective training was highly beneficial to the implementation of the contract. The trainees have now formed the backbone for proper operation of the facilities in the future. 4.6 As PO's representative agency, the Project office with only six permanent staff members takes overall responsibility for communication, management coordination and monitoring. The manpower inadequacy, particularly lack of senior legal, financial and engineering professionals has made the Office face extreme difficulties in addressing such a large and complicated project. It is important to structure a proper project organization. It is recognized that well organized and staffed project management organizations at all operational levels are key to smooth project implementation. The staff at the managerial level should be relatively stable to keep confsistency of policy and management during the project implementation. 4.7 Inadequate communication between parties especially in the civil construction, poor performnance of EF's civil works' subcontractor, multi-party involvement and simultaneous operation in three different sites all blocked the smooth implementation and had impacted on the final end of the project. 4.8 The PO utilized both foreign and domestic technical assistance during project preparation and implementation, which have been greatly beneficial for the project. The technical assistance helped the PO especially to assess the process development and make decisions. After termination of EF's DPT contractual obligations, external assistance has been sought for DPT process development in the new facilities.  35 ANNEX A 5. Evaluation of World Bank Performance 5.1 The vaccine component of the World Bank's Rural Health and Preventive Medicine Project is the largest project of biological product innovation in China, even in the world. The Bank officials have contributed to the project both in its course of preparation and evaluation as well as in the aspects of giving guidance and supervision in its implementation. 5.2 The Bank officials provided financial support by securing a grant in the amount of USD 15 million and administered the grant from the Rotary International. In addition, the Bank agree to shift the remaining surplus from other components of the Health Project 1I to the vaccine component. 5.3 Since the implementation of the project, ten missions were dispatched by the World Bank including the representative from Rotary intemational for investigation and supervision. The missions provided important technical assistance in relation with GMP, technical assistance management and equipment procurement. 5.4 Underestimation of the cost needed to build three vaccine facilities in China caused enormous difficulties in the preparation and the implementation. 6. Evaluation of Borrower Performance 6.1 The Chinese Government has listed the vaccine component as one of national key construction projects and paid serious attention to its implementation. The Leading Group, MOH, chaired by Minister Chen convened from time to time to provide supervision and coordination. It should be noted that as approved by the Chinese Government additional fund of tJSD 22 million was appropriated to help solve the mid-term financial shortage. 6.2 Facing great challenges in terms of project complexity, multi-involvement, manpower shortage and lack of experiences and adequate skills, Project Office together with the project institutes exerted every effort to absorb the heavy workload, to overcome difficulties and solve the problems in the project implementation. As a headquarters of the project, the Project Office has contributed greatly towards efficient and effective project communication, coordination, management and supervision. It has also played important roles in organizing project activities such as facility construction, process development, equipment procurement, training, etc. Especially in the later period of the project, Project Office organized the exchange of technical and managerial experiences among the institutes and assiste hem to reinforce their advantages and to avoid the weak points during the commissioning and validation. 7. Status of Cost Study 7.1 Proposed by the World Bank in the mid-term implementation, a cost study of three production lines has been conducted by the Evaluation Centre of Medical Technology, Shanghai Medical University, assisted by international experts. The value of the study can orient the staff of the new vaccine facilities to cost and cost effectiveness analysis. If the cost of the new vaccine is much higher, alternative approaches might be considered to reduce the costs and planning must begin to mitigate the problem now. The study is also expected to be the basis for the Government  36 ANNEX A to deterrmine the price of the products of the new facilities. The report on cost study of OPV production line has been completed. The dose cost of OPV is increased. Suggestions have been made by the evaluation team. 8. Status of Domestic Review Process Plan of WHO Certification 8.1 The availability of modem facilities for vaccine production, including the improved quality control mechanisms, especially improved GMP conditions seems encouraging for product export in the fUture. The first step is to follow the national procedures to obtain GMP certificate for the facilities and pre-production license for trial production. 9. Project Sustainability 9.1 Sufficient recurrent budget for the new facilities must be provided to keep the facilities in operation in accordance with GWM standards. To gradually reduce the burden of operational costs, the project institutes should analyze the costs and try to replace the imported production materials by domestic ones. 9.2 Rational prices for vaccines under EPI should be based on standardized cost, takdng governments economic affordability into account. In terms of manufacturers' benefits and the inadequacy of EPI's expense, a policy may be sought for to solve the problems. 9.3 With external technical assistance, continued training courses and upgrading of GMP knowledge will be conducted among vaccine production facilities.  37 ANNEX A Part HI: Borrower's Comments on Bank's ICR Introduction After a careful review of the Implementation Completion Report (ICR): Rural Health and Preventive Medicine Project (Health II Project), we wish to extend our nigh appreciation for the comprehensive coverage, objective analysis and constructive recommendations contained in the Report. Health II comprised of two different components in nature is indeeeS a unique project in terms of its complexity and unintended protraction in contrast with other World Bank projects established so far. We completely share with you the overall assessment of the achievements, outcomes issues and lessons learned as presented in the Report in which many of positive elements contained in our initial evaluation documents have been incorporated. We only would like to make some specific comments summanrzed as below. 1. Non-Vaccine Production Component 1.1 It should be noted that as a separate sub-component, funding support to the ten provincial and municipal EPSs in Gansu, Sichuan, Jilin, Hubei, Ningxia Hui, Heilongjiang, Shandong, Jiangxi, Harbin and Chongqing has also contributed to the strengthening of preventive services in respect to physical and training capability. Consequently the whole non-vaccine component may be grouped as rural health (56 counties in five Provinces), ten provincial and municipal EPSs, CAMs, drug control institutes and health insurance research. 1.2 The rural health insurance experiment is a pioneering one, which was conducted at a time when the cooperative medical system in most areas of China was being dismantled and efforts were made to explore possibility of restoring or replacing the system. The findings of the experiment published in a Rural Health Insurance Handbook were widely distributed among the health authorities and health policy researchers and the methodologies of scientific estimation of affordability of the peasants, ratios of premium, etc., were then actually adopted by many of researchers in their studies on rural health care financing. Therefore, the findings from the rural health insurance experiment are used not only a decade later but were also implicitly or explicitly used at the time of their conclusion. 2. Vaccine Production Component 2.1 Rather than combining both limited intemational bidding and competitive international bidding, the revised approach by using consultant service according to the World Bank guideline was to engage an engineering firm, which would be associated with a vaccine producer. 2.2 The lessons learned from the vaccine component as summarized in the Report are both costly and profound, which are related to how vaccine production will be properly upgraded or renovated in developing countries. The lessons will be useful not only for China and the World Bank, possibly also for other developing countries.  38 ANNEX A 2.3 Through implementation of the vaccine project, the project institutes own not only the advanced facilities but also a number of trained young people involved in different aspects of production process and management, which is intangible and valuable resources of the institutes. Making use of those resources will be beneficial to operation of the new facilities, motivation of those trained young people and avoidance of turnover from the institutes. 2.4 The three new vaccine plants even when they come into final operation will be faced with enormous problems relating to cost, price, maintenance, etc. Consideration on the aspects of cost, price, and assessment of economic feasibility of vaccine production mentioned in the report is most helpful for the Ministry of Health and the project institutes to take actions in order to keep the facilities operate and survive. While MOH will take relevant steps to support the new facilities without compromising the ongoing EPI the World Bank will also hopefully give assistance in this respect. However, it is debatable whether support for privatizing the vaccine plants is the best solution. 2.5 To help resolve one after another problems emerged in the process of the implementation the contributions made by all the World Bank staff who were involved in the project cannot be overemphasized, and our comments would not be complete without mentioning Mr. Herbert Boehm who among many has witnessed the darkest or most difficult time of the component and its break of dawn as well.  IBRD 29543 iO 0O' 90' 10