China - Key Studies Development Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 19473 IMPLEMENTATION COMPLETION REPORT CHIINA KEY STUDIES DEVELOPMENT PROJECT (CREDIT 2210-CHA) June 9, 1999 Human Development Sector Unit East Asia and Pacific Regional Office This document has a restricted distribution and may be used by recipients only in the perfonnance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENTS Currency Unit = Yuan (Y) At Appraisal $1 = Y 3.76 At Completion $1 = Y 8.26 FISCAL YEAR January 1 - December 31 WEIGHTS AND MEASURES Metric System ABBREVIATIONS AND ACRONYMS CAS Chinese Academy of Sciences CICET China International Center for Economic and Technical Exchanges FILO Foreign Investment and Loan Office ICB International Competitive Bidding ICR Implementation Completion Report IDA International Development Association KDSP Key Studies Development Project LAN Local Area Network MOE Ministry of Education NCFC National Computer and Network Facility NSFC National Natural Science Foundation of China NSF National Science Foundation PAG Programme Advisory Group PI Principal Investigator PLMP Pilot Laboratory Management Program SAR Staff Appraisal Report SDPC State Development and Planning Commission SEdC State Education Commission SKLs State Key Laboratories SLs Special Laboratories SPC State Planning Commission SSTC State Scientific and Technical Commission TA Technical Assistance UNDP United Nations Development Programme Vice President Jean-Michel Severino, EAP China Country Manager Yukon Huang, EAP Sector Manager Alan Ruby, EASED Staff Member Sandra Erb, Operations Analyst, EASED  FOR OFFICIAL USE ONLY CONTENTS PREFACE ................................................................... iii EVALUATION SUMMARY ............................................................V PART I: PROJECT IMPLEMENTATION ASSESSMENT ..........................................................1 A. PROJECT OBJECTIVES AND DESCRIPTION ...................................................................1 B. ACHIEVEMENT OF PROJECT OBJECTIVES ....................................................................2 C. IMPLEMENTATION RECORD AND MAJOR FACTORS AFFECTING THE PROJECT ... 12 D. PROJECT SUSTAINABILITY ................................................................... 13 E. BANK PERFORMANCE .................................................................. 14 F. BORROWER PERFORMANCE .................................................................. 15 G. ENVIRONMENT WITHIN WHICH PROJECT WAS APPRAISED AND APPROVED ....... 1 5 H. ASSESSMENT OF OUTCOME .................................................................. 15 I. FUTURE OPERATIONS .................................................................. 16 J. KEYLESSONSLEARNED .................................................................. 16 PART :I: STATISTICAL TABLES ................................................................... 17 TABLE 1: SUMMARY OF ASSESSMENTS .................................................................. 17 TABLE 2: RELATED BANK LOANS/CREDITS .................................................................. 18 TABLE 3: PROJECT TIMETABLE .................................................................. 20 TABLE 4: LOAN/CREDIT DISBURSEMENT: CUMULATIVE ESTIMATE AND ACTUAL ... 20 TABLE 5: KEY PERFORMANCE INDICATORS .................................................................. 21 TABLE 6A: PROJECT COSTS .................................................................. 22 TABLE 6B: PROJECT COSTS .................................................................. 22 TABLE 6C: PROJECT FINANCING .................................................................. 23 TABLE 7: STATUS OF LEGAL COVENANTS ............................................. 23 TABLE 8: COMPLIANCE WITH OPERATIONAL MANUAL STATEMENTS .. 24 TABLE 9: BANK RESOURCES: STAFF INPUTS ......................... ......................................... 24 TABLE 10: BANK RESOURCES: MISSIONS ............................................ 25 ANNEX A: BORROWER'S CONTRIBUTION TO THE ICR ................................................... 27 ANNEX B: ICR MISSION'S AIDE MEMOIRE .................................................................. 3 1 Ths document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without Wor]ld Bank authorization.  - iii - IMPLEMENTATION COMPLETION REPORT CHINA KEY STUDIES DEVELOPMENT PROJECT (CREDIT 2210-CHA) PREFACE This is the Implementation Completion Report (ICR) for the Key Studies Development Project in China, for which Credit 2210-CHA in the amount of SDR 92.9 million ($131.2 million equivalent) was approved on March 15, 1991 and made effective on June 4, 1991. The credit was closed on December 31, 1998. There were three extensions of the project for a total of two and a half years. The credit was fully disbursed, and the last disbursement took place on April 12, 1999. The ICR was prepared by Sandra Erb under the supervision of Halsey L. Beemer, Jr. in t:he Human Development Sector Unit of the East Asia and Pacific Region (EAS]HD), and reviewed by Alan Ruby, Sector Manager, EASHD. The borrower provided comments that are included as an annex to the ICR. Preparation of this ICR was begun during the Association's supervision mission in May 30-June 26, 1997 and continued through the final supervision in January 1999. The ICR is based on materials in the project file. The borrower contributed to preparation of the ICR by providing views reflected in the mission's aide-memoire, preparing its own evaluation of the project's execution and initial preparation, and commenting on the draft ICR.   - v - CHINA KEY STUDIES DEVELOPMENT PROJECT CREDIT 2210-CHA EVALUATION SUMMARY Introduction I . In the early 1 980s the then State Planning Commission (SPC) initiated the State Key Laboratories (SKL) program which concentrates scarce resources on existing laboratories affiliated with universities and Chinese Academy of Sciences (CAS) institutes that: (a) work on key specialties of economic and social relevance; (b) excel in research output, staff and facilities; (c) demonstrate links with industry; (d) play a significant role in educating graduate students; and (e) as open laboratories, provide opportunities for researchers from other labs as well as foreign researchers to work collaboratively on projects of national and international importance. While initiated in the institutes of the Chinese Academy of Sciences (CAS), the SKL program moved to the unive:rsity sector which had, after the end of the Cultural Revolution, taken on the responsibility of training students at the masters and doctoral levels and hence needed to be substantially upgraded to perform these tasks. 2. While progress had been made during the late 1970s and early 80s in restructuring and improving the quality of scientific research and training, many problems remained. These fell into four categories: (a) limited career prospects for those receiving advanced scientific training because of the small academic scientific infrastructure; (b) difficulties in ac(quiring the physical and intellectual inputs necessary for conducting research; (c) inappropriate methods of training, for both scientists and technicians; and (d) poor articulation between research institutes themselves and with end users of technology in industry. The Key Studies Development Project was developed with these issues as fundamental reference points and prepared to deal with many aspects of these problems. Project Objectives 3. Project Objectives and Scope. The project sought to support the Government's efforts to reform the Science and Technology (S&T) Management System by promoting technology development, increasing efforts in applied research, continuing progress in basic research and reforming the S&T personnel system. To achieve this goal the project set out to: (a) increase the production of scientific researchers trained to international standards through masters, doctoral and post-doctoral programs; (b) enhance the quality and productivity of research in strategic areas relevant to long-termn economic and social development; (c) strengthen the management of scientific research at both the national and laboratory level; (d) provide a modern scientific infrastructure for Chinese scientists trained at home and abroad; and (e) encourage scientific cooperation across institutional and niational boundaries.  - vi - 4. The Association and the Government developed a project that was to support: (a) research and training in SKLs and Special Laboratories (SLs) l; (b) a pilot laboratory management program; (c) a national computer and network facility; and (d) a program advisory mechanism. Implementation Experience and Results 5. Graduate Training. One of the objectives of the project was to enhance the training of graduate and post-graduate students in science and technology in selected laboratories, to the highest international levels. By the end of 1997 the combined MSc and Ph.D. enrollments surpassed the 1996 target by 6.5 percent.2 Similarly, the post- doctoral enrollments for the period 1988 to 1997 surpassed the target by 38 percent. Another positive trend for graduate education was the small but increasing number of students that went directly on for the Ph.D. degree without first acquiring a MSc degree. However, not all the numerical targets for graduate production were achieved. The goal of masters degree students trained in project-supported laboratories was missed by about 12 percent. This was due, in part, to strict quotas for graduate students imposed by the Ministry of Education (MOE). The ratio of MSc to Ph.D. enrollments decreased from 2.67 in 1992 to 1.61 in early 1997, a positive change, as Ph.D. students are normally trained to the state-of-the art in their respective fields. The MOE recently adopted a policy of expanding enrollments in higher education from the current 7 percent to 15 percent by 2010. Therefore, the increases in graduate enrollments are highly likely to be continued. 6. Women Students. At the time of project design approximately 25 percent of scientific and technical personnel were women, although participation of women in graduate programs was substantially less. The project set a target for female participation in MSc and Ph.D. graduate programs of approximately 22 percent of total enrollment. At the end of the project, overall figures showed modest increases in graduate training for women. MSc degrees increases from approximately 18 percent in 1992 to 24 percent in 1997, Ph.D. degrees 7 percent to 14 percent and post-doc fellows from 0 to 8 percent for the same period. (See Table 5) Anecdotal evidence indicates that the low representation of women is due at least in part to the perception that job availability for women graduates is increasingly affected by the freedom of managers to discriminate against women in employment. Job discrimination against women in high level scientific fields appears to be growing as the labor market forces appear to be strengthening. One observed result of this is discipline switching by female science graduate students from Special Laboratories (SLs) are distinguished from SKLs by their sponsorship and level of funding. Many of the SLs are not eligible to be SKLs because their work is in basic science rather than applied sciences. They remain under the jurisdiction of their representative host universities and do not receive additional special funding from the State. 2 In the Staff Appraisal Report (SAR) there were no development indicators. The list of key development indicators found in Table 5 resulted from efforts made in 1994 by the Bank and the Ministry of Education to produce indicators with baseline data from project preparation documents and the first set of laboratory annual reports, and targets for an expected closing date of June 30, 1996. The project was extended for the first time for one year during which the ICR mission collected 1997 indicator data.  - vii - the theoretical, technical and research aspects of a discipline to those of design and marketing of the final products of the research stream. 7. Curriculum Development. The curriculum has become broader as the research performed in the project supported laboratories became more modem and uses more sophisticated equipment. In addition, young faculty that were able to attend conferences and work abroad as a result of project-supported technical assistance activities have tended to modernize and widen the scope of the courses they teach. Interaction with other scientists abroad, through visits and the Internet, as well as academic committees of the laboratories, also contributed to curriculum improvement. New courses were also introduced as a result of the acquisition and capabilities of some of the new equipment. The net effect of all of these changes is that graduate curricula currently have stronger fundamental components than earlier, and the more specialized courses are not as narrow as they once were. This helps avoid rapid academic obsolescence of the graduate students trained under the previous narrower approach. 8. Equipment. A total of US$116 million, about 88 percent of the credit was used for the purchase of scientific equipment, primarily from foreign sources. Whereas most of the equipment in earlier projects was ordered as complete sets of integrated instrumentation, in the Key Studies Development Project (KDSP) individual pieces of equipment were more often ordered to support a research activity or a teaching program. This large injection of new equipment significantly improved the quality and quantity of the research performed and graduate training provided at the laboratories 9. The higher quality scientific equipment with the greater reliability of its computerized operation and output permitted use by larger numbers of students without requiring very detailed training in their operation. This allows for more time on research and thus increases the utilization rate of the equipment as well as a greater number of graduates that can be trained during a give period. The net effect has been, lab directors report, to attract more and better graduate students to laboratories funded by this project. Not only can they work on the most sophisticated instrumentation but they can complete their research in shorter periods of time. The large number of specialists trained during this project has provided a substantial part of the human resource foundation for China's current expansion of its high level science and technology infrastructure. Without this increase the large investments the Chinese government is currently making in high level science and technology would not be sustainable. 10. In some labs, the equipment acquired made possible research results which are up to or surpass world standards. In many others, the quality and quantity of research have both improved notably. The new equipment, especially the new mini-supercomputer, has enabled China to enter new fields of inquiry and to reconceptualize problems in innovative ways as a result of the gains and measurement and computation. 11. Equipment Procurement. Traditional World Bank procurement methods are inadequate for projects of higher education and scientific research. One of the results of  - viii - the use of traditional World Bank procurement methods - which encourages aggregated packages of identical equipment in order to obtain lower prices - is that all of the equipment, especially computational instrumentation such as sophisticated workstations, will become obsolete at the same time thus forcing a whole range of institutions to upgrade their equipment at the same time. Such large scale, simultaneous up-grading is expensive and can be avoided if procurement is staged. Optimal utilization of the equipment can also be ensured if the staging is done to coincide with the needs of the research agendas of the laboratories rather than to the dictates of the procurement process. Bank procurement methods also encourages bundling of similar commodities into one set of technical specifications to achieve a price advantage that puts pressure to compromise on the technical specification. This meant that some lab directors were not able to get the precise piece of equipment that was needed for their research. In addition, the procedures for buying computers is an overly lengthy process which means computers are often out of date by the time they arrive. The project preparation team worked out a modified ICB/Intemational Shopping procurement process which could have overcome some of these problems but the MOE and CAS, familiar with the traditional methods used in the two earlier Bank-supported higher education projects, chose not to use the proposed modified procurement process. 12. Research Funding. One of the assurances received during preparation was that as the project increased the quality of the research infrastructure, the government would increase the amount of research funding available to the labs. Although no project funds were used to directly support research, in is clear that the government provided an increasing level of research funding, both the individual project-supported labs and the research community more broadly. The available research funds in project-supported labs more than doubled during the three year period 1994 through 1997, growing from a laboratory average of Y 0.9 million for 1994 to Y 2.1 million for 1997. This increasing level of research funding contributes to the confidence that the MOE and CAS have regarding the sustainability of the project achievements. 13. Peer Review Research. Fundamental to the design of the project was the strengthening and broadening of the use of the model of principal scientist proposed, peer and panel reviewed research funding. The State Development and Planning Commission (SDPC), formerly the State Planning Commission, having decided on the key areas in which the project could consider applications, established a series of peer review panels which screened proposals, rejecting some and requesting substantial modifications in most of the remainder. Individual laboratory directors were responsible for writing the research and graduate training proposals which would be supported by this project. These proposals were then submitted for peer review and finally a sample of the proposals were reviewed and approved by the Bank appraisal team. The thrust of this design component was to try to enhance the role of the individual laboratory director and increase his or her responsibility for the research direction of the lab, the quality of the research and the development of the graduate programs that would support the research.  -ix - 14. This goal is in the process of being achieved. It is an observable fact that over the life of the project lab directors have increased their responsibilities, in the development of research agendas, seeking funding to support a sizable portion of the laboratory research, and, to an increasing extent, the content of course material offered in their labs. 15. Pilot Laboratory Management Program (PLMP). Improvement of laboratory manag,ement was an explicit goal of the project, with the seven lab Pilot Laboratory Management Program (PLMP) being the main vehicle to this end. UNDP contributed $253,000 to support this component of the KSDP. In broad terms, the objectives of the program were to provide a forum for lab directors from different fields to compare experiences, to provide a limited amount of management training to lab personnel, to encourage awareness of technology transfer possibilities, and to disseminate the lessons of the project to the other participants in the KSDP. Although improvements in the PLMP labs are visible, the overall impact of the PLMP program is harder to see outside the seven laboratories chosen for participation in PLMP. 16. It is clear that considerable efforts were made in the 1991-1995 period to make the project a success and to propagate the lessons learned but it was not evident that laboratories beyond the seven knew of PLMP lessons. Possible explanation for the disconnect between the efforts expended by those involved with PLMP and the other labs are: (a) the National Natural Science Foundation of China (NSFC) had limited resources for follow-up activities; (b) the special resources going to the seven PLMP labs for managerial improvements were not available to the other KSDP participants; and (c) there were a good number of changes in lab leadership during the course of the project. 17. The project did not produce any one model of good management, but it did call attention to the importance of: (a) having strong leaders as lab directors (establishing the "director responsibility system") and as chairmen of the academic committees; (b) using the a,cademic committees effectively to set research and teaching directions, and to make contacts with other parts of the technical community; (c) setting clear objectives for the lab's work; (d) developing cooperative relations with industry; (e) handling effectively the procurement of advanced imported equipment; (f) establishing effective rules and regulations for safe and efficient lab operations; and (g) following the main project guidelines - "opening, interchanging, collaborating and competing." One way to capitalize on this component of the project would be to produce a manual for laboratory management that incorporates the experiences of the PLMP labs and to widely disseminate the manual. 18. Joint Laboratories: One of the goals of the project was to encourage scientific cooperation across institutional boundaries and to this end the project supported joint laboratories. The joint laboratories were meant to bring together labs in different universities or from a university and a CAS institute to focus on one research topic through sharing instrumentation, faculty and graduate students and research funding, the objective was to support the notion that science is a collaborative venture. Five of the projiect's 133 labs were designated as joint laboratories. This effort was not completely  - x - successful because of the strong preference for academic compartmentalization and the pressure on lab directors to build fully integrated and complete labs so that their labs could be accredited. Accreditation meant increased government support for research. Since labs were accredited as individual and not joint labs, the incentives did not exist for true partnerships. 19. National Computer and Network Facility (NCFC). The project sought to support the development of a prototype scientific computing facility that would ultimately be expanded nationwide. The objective of the component was to facilitate resource sharing amongst institutions by establishing an enhanced computation facility linked by a demonstration network to the computer centers of Peking and Tsinghua Universities and the Computing Center of the CAS. The strategy for implementing the NCFC was to first construct the local area network (LAN) at each participating institution and then establish the NCFC backbone network connecting the three institutions. Procurement of the mini-supercomputer was to be contingent on the successful completion of the network sometime between 1991 and 1993. 20. The LANs were in place by 1993, the fiber optic network linking Peking and Tsinghau Universities and some of the Beijing Institutes of the Chinese Academy of Sciences was put into place by March 1994 but procurement delays prevented the mini- supercomputer from being installed until June 1996. Despite the delays, substantial progress has been made at the NCFC. There are nearly 200 science and technology institutions connecting with NCFC including 123 institutes under CAS and several dozen institutes of state ministries and commissions. The center currently has a CPU utilization rate of 70 percent with parallel computing in full operation. There are several scientific computer projects on going with support from the NSFC. 21. A User's Committee of representatives from nine different scientific disciplines was established in 1997. The committee has the responsibility for formulating policies on the allocation and use of the supercomputer resources, allocation of time, examining and approving users' applications for computing on the supercomputer, monitoring the operation and services of the supercomputer and organizing academic exchanges for users. 22. One the major successes of the KSDP was the substantial increase of Internet access to China. The establishment of the Internet link in April 1994 was the first multi- institutional Internet link between the Beijing scientific community and the international academic community. Solving the early developmental problems of establishing the Internet link has allowed for easier startup for the number of links which have been established since that time. 23. Technical Assistance. The Programme Advisory Group (PAG), consisting of five senior Chinese scientists and four scientists from abroad, was established in 1992 with support from UNDP to provide continuing high quality consultation and advice to the MOE and CAS on the overall direction and progress of implementation in the various  - xi - componaents of the KSDP. The PAG played an important role in helping to integrate the diverse components of the KSDP and evaluating the results of the project, both at the mid-term and at the final stages. The PAG was particularly important in the establishment of the NCFC, providing excellent advice on the operation of a national computing center, the establishment of the users committee, and a review of the technical specifications of the mini-supercomputer which become the heart of the NCFC. However, as an advisory committee, it did not have the executive power and organizational resources to push project implementation. This was particularly apparent when the PAG tried to establish horizontal lines of communications between labs to share research methods, results and international specialists. 24. Domestic and International TA Programs. Domestic and international technical assistance programs were designed to promote collaborative research with foreign researchers. This included attendance at international meetings, inviting foreign experts to China to give lectures and provide advice on research work, short overseas visits to research laboratories and personnel training. Data received from both the MOE and CAS show that, in most cases, the overall numerical targets were reached. However, the quality of these programs could have been better if the foreign specialist program had organized tours for the world renowned specialists to several universities or institutes so that they could deliver several keynote lectures with broad integrative themes in science or laboratory management. Summary of Findings, Future Operations, and Key Lessons Learned 25. Project Outcomes. The project outcome is rated as satisfactory. There are increases in graduate students trained at high levels of quality, enhanced capacity for such training in the future, and impressive research results emanating from the project. Laboratory management has improved and more effective technology transfer strategies are beginning. Overall, the project made a significant contribution to Chinese research and advanced education capacities and to the modernization of thinking about research, education, and laboratory management. 26. Future Operations. Experience derived from the design and preparation of Key Studies Development Project had a direct impact on the Bank's Technology Development Project (Ln. 3847-CHA) and, to a less degree, the Higher Education Reform Project (Ln. 4474/Cr. 3213-CHA). The former used many of the targeting and review mechanisms used in the Key Studies Project while the vast bulk of the universities supported in the latter project were also in the Key Studies Project as well as one of the two earlier University Development Projects. Future lending in the education sector is now being discussed with the government and if such new lending develops the area of science and technology is a possible area for further investment.  - xli - Lessons Learned. The main lessons learned in this project are that: (a) with appropriate resources and careful planning it is possible to accelerate China's scientific and technological development by training its graduate students at international levels of proficiency and in increased numbers; (b) it is difficult to transfer managerial experience acquired in other countries to China; (c) the scarcity of equipment has made the community maximize the amount of new equipment purchased, at the expense of acquiring adequate supplies of spare parts and consumables for the proper operation of that equipment. This occurred in spite of built-in assurances to the contrary; (d) the sharing of major facilities, such as the mini-supercomputer at NCFC or the joint laboratories, among a large variety of research and teaching institutions, is difficult due to the strong compartmentalized organizational structure within the S&T community; (e) the dissemination of managerial know-how within and among laboratories is slow and more efficient means should have been adopted from the very beginning; and (f) the PAG might have adopted a more pro-active role, akin to that of a managerial Principal Investigator (PI) for the Project.  - 1 - CHINA KEY STUDIES DEVELOPMENT PROJECT CREDIT 2210-CHA PART I: PROJECT IMPLEMENTATION ASSESSMENT A. PROJECT OBJECTIVES AND DESCRIPTION 1. Project Objectives and Scope. The project sought to support the Government's efforts to reform the Science and Technology (S&T) Management System by promoting technology development, increasing efforts in applied research, continuing progress in basic research and reforming S&T personnel. To achieve this goal the project set out to: (a) increase the production of scientific researchers trained to international standards through masters, doctoral and post-doctoral programs; (b) enhance the quality and productivity of research in strategic areas relevant to long-term economic and social development (these were: energy, transportation and communication, raw and processed materials, machinery and electronics, agriculture, biotechnology, new materials, health and pharmaceuticals, environmental protection and natural geo-engineering and basic science); (c) strengthen the management of scientific research at both the national and laboratory level; (d) provide a modern scientific infrastructure for Chinese scientists trained at home and abroad; and (e) encourage scientific cooperation across institutional and national boundaries. 2. Policy Context. In the early 1980s the then SDPC initiated the SKL program which concentrates scarce resources on existing laboratories affiliated with universities and CAS institutes that: (a) work on key specialties of economic and social relevance; (b) excel in research output, staff and facilities; (c) demonstrate links with industry; (d) play a significant role in educating graduate students; and (e) as open laboratories, provide opportunities for researchers from other labs as well as foreign researchers to work collaboratively on projects of national and international importance. While initiated in the institutes of the Chinese Academy of Sciences, the SKL program moved to the university sector which had, after the end of the Cultural Revolution, taken on the responsibility of training students at the masters and doctoral levels and hence needed to be substantially upgraded to perform these tasks. 3. While progress had been made during the late 1970s and early 80s in restructuring and improving the quality of scientific research and training, many problems remained. Major issues fell into four categories: (a) limited career prospects for those receiving advanced scientific training because of the small academic scientific infrastructure; (b) difficulties in acquiring the physical and intellectual inputs necessary for conducting research; (c) inappropriate methods of training, for both scientists and technicians; and (d) poor articulation between research institutes themselves and with end users of technology in industry.  - 2 - 4. Linkages between Project, Sector and Policy Objectives. In order to address both the sectoral and policy consideration outlined above, the Association and the Government developed a project that was to support: (a) research and training in SKLs and Special Laboratories (SLs), (b) a pilot laboratory management program; (c) a national computer and network facility; and (d) a program advisory mechanism. 5. The major risks perceived at the time of appraisal were: (a) difficulties in managing the project given its complexity. The project included a large number of laboratories which conduct research in many diverse fields; and (b) the competitive mechanism that was to allocate research funds might not function as expected so as to provide sufficient research funds to project laboratories. The risks were seen to be manageable due to the strong leadership and coordination that the government had agreed to furnish as well as proposed improvements in the competitive mechanism for monitoring the availability of research funds, and by the agreement of the Government to take appropriate action when necessary to see that mechanism remained competitive. B. ACHIEVEMENT OF PROJECT OBJECTIVES 6. Graduate Training. One of the objectives of the project was to enhance the training of graduate and post-graduate students in science and technology in selected laboratories, to the highest international levels. A major increase not only in the quality of the training was envisaged, but also in the numbers of students trained at the MSc, Ph.D. and post-doctoral levels. By the end of the project the trends in graduate enrollments at the SKLs and SLs were very positive. The sum of the MSc and Ph.D. enrollments in early 1997 was 8042, which exceeded the project total target of 7545. Similarly, the post-doctoral enrollments for the period 1988 to 1997 increased from 49 to 346 which exceeded the target of 250. The modern and efficient research equipment acquired by the laboratories with project funds permitted more research to be done in a given time then had been possible earlier and therefore, more students to be trained. However, not all the numerical targets for graduate production were achieved. The goal of 5659 masters degree students trained in project-supported labs was missed by about 12 percent. This is due in part to the strict enrollment quotas set by the MOE. 7. The ratio of M.Sc. to Ph.D. enrollments decreased from 2.67 in 1992 to 1.61 in early 1997, a positive change, as Ph.D. students are normally trained to the state-of-the- art in their respective fields, whereas MSc ones are not. The MOE has approved a policy of direct enrollment of some graduate students in Ph.D. programs without requiring them to first obtain MSc degrees. This is an important step because it affords a graduate student the opportunity to work on a longer and more substantial doctoral research project which is more effective in producing highly trained research scientists and scholars. The fraction of the total number of Ph.D. students admitted into such programs is relatively small, of the order of 10 percent or less, and many of the laboratories have not begun this approach yet. It would be desirable to increase this fraction to 15 percent or 20 percent and to extend it to all project laboratories. Appropriate new screening procedures (such as qualifying examinations) during the training of such direct Ph.D. students should be  -3 - introduced in order to weed out those that are not able to perform at an acceptable level and to direct them to other programs. In addition, the number of faculty approved for supervising Ph.D. research should be further increased and the MSc degree further de- emphasized. The post-doc program is still quite small with only about three post-doc fellows per project-supported lab in average over the life of the project. Post-docs are a very important component in the most advanced modem science and technology research activities, and their numbers should be substantially increased in the future. 8. Women Students. At the time of project design approximately 25 percent of scientific and technical personnel were women, although participation of women in graduate programs was substantially less. The project set a target for female participation in graduate programs of approximately 22 percent of total enrollment. At the end of the project, there were modest increases in graduate training for women. MSc degrees increased from approximately 18 percent in 1988 to 24 percent in 1997, Ph.D. degrees 7 percent to 14 percent and post-doc fellows from 0 to 8 percent for the same period. (See Table 5) Anecdotal evidence indicates that the low representation of women is due in part to the perception that job availability for women graduates is increasingly affected by the freedom of managers to discriminate against women in employment. Job discrimnination against women in high level scientific fields appears to be growing as the labor mnarket forces appear to be strengthening and the job assignment system which was a product of the planned economy is fading. One observed result of this is discipline switching by female science graduate students from the theoretical, technical and research aspects of a discipline to those of design and marketing of the final products of the research stream. 9. Curriculum Development. The project has had a positive influence on the curricula at the project laboratories. As the research performed in these laboratories became more modem and uses more sophisticated equipment, the nature of the curriclulum became broader. Young faculty having done work or attended conferences abroad have tended to modernize and widen the scope of the courses they teach. Interaction with other scientists abroad, through visits from and to laboratories abroad and through the Internet, as well as suggestions from members of the academic committees of the laboratories also contributed to curriculum improvement. New courses were introduced as a result of the acquisition and capabilities of some of the new equipment. The net effect of all of these changes is that graduate curricula currently have stronger fundamental components than earlier, and the more specialized courses are not as narrow as they once were. This helps avoid rapid obsolescence of the graduate students trained under the previous narrower approach. 10. One of the problems identified in the two earlier Bank-supported higher education science and engineering projects is largely absent in the KSDP, and that was the development of new narrow courses based strictly on the introduction of new pieces of scientific or computational instrumentation. In the earlier two University Development Project curriculum development and the introduction of new courses was most often the result of individual new pieces of physical infrastructure rather than a more holistic  -4 - approach which took the overall research goals of a lab or the institute as the guiding model for developing the new courses. The lab directors in the KSDP who wrote the project design proposals had to lay out a broad set of research objectives and graduate training goals, and these design proposals drove curriculum development, with the new instrumentation supporting these goals rather than new equipment driving curriculum development. 11. Equipment. A total of US$116 million, about 88 percent of the credit was used to purchase scientific equipment, primarily from foreign sources. Whereas most of the equipment in earlier projects was ordered as complete sets of integrated instrumentation, in the KSDP individual pieces of equipment was more often ordered to support a research activity or a teaching program. Some was designed by the laboratories and built in China with some imported components. The equipment ranged from small instruments costing a few thousand dollars or less to others in excess of $1.8 million. Some of it was state-of- the-art when ordered and continued to be so after arriving and being put into operation. Some of it became obsolete between the time of selection and delivery. This was the case for most of the computer workstations, the introduction of which into the labs was plagued by modifications in the procurement technical specifications, export license delays, and initial hesitation to buy appropriate application software. Nevertheless, this large injection of new equipment significantly improved the quality and quantity of the research performed graduate students trained in the laboratories. 12. Many of the lab directors who presented proposals for review to be funded by the KSDP had had previous experience in the earlier World Bank projects. They realized that the importation of expensive, state-of-the-art and totally integrated sets of research equipment increased their research capacity but often locked them into a narrow spectrum of scientific work which did not allow for the disaggregation of equipment sets. Based on this experience, directors realized that disaggrated equipment sets were necessary to move on to additional and newer topics with their concomitant new areas for graduate training. It was evident to the ICR mission that most, if not all, the Key Studies- supported research labs had been able to move beyond instrument driven research agenda of the earlier period to more project and goal driven research and had thus been able to achieve what might be considered a "second generation" of research direction and capacity. Many of these university labs had been recipients of World Bank funds in the early and mid 1980s and it is evident that the way they now do science differs from when the World Bank first invested in them. Much of this can be attributed to the international perspective that the technical assistance portions of the earlier projects brought to the labs. Chinese scientists going abroad and international scientists coming to China as well as Chinese graduate students returning to work in these upgraded labs will continue to ensure high quality research. 13. Equipment Procurement. One of the results of the use of traditional World Bank procurement methods - which encourages aggregated packages of identical equipment in order to obtain lower prices - is that all of the equipment, especially computational instrumentation such as sophisticated workstations, becomes obsolete at  -5 - the same time thus forcing a whole range of institutions to upgrade their equipment at the samne time. Such large scale, simultaneous up-grading is expensive and can be avoided if procurement was staged throughout the project. Utilization of the equipment can be also assured if the staging is done to coincide with the needs of the research agendas of the laboratories. 14. During preparation the Association team worked out a modified ICB procurement process which as tailored to the special needs of buying scientific instrumentation and computational equipment. This process was based on prequalifying suppliers and then asking for price quotes on specific bids and was meant to both speed up the process of buying the equipment and develop a database of equipment producers for the Chinese purchasers. However, the MOE, accustomed to the traditional ICB procedures used in two previous university projects, decided not to use the modified ICB procurement process. The CAS, with less experience in Bank procedures, chose to follow the lead of the MOE. Had the MOE and CAS chosen the modified procurement method some of the problems experienced could have been avoided. 15. Student Access to Project Supplied Equipment. An additional important improvement in the training of the graduate students has been their hands-on use of the new scientific equipment. The higher quality and reliability of such equipment and its coriputerized operation and output permits the efficient use by larger numbers of students without requiring very detailed training in their operation. There are still some very expensive and complex pieces of equipment which are operated only by very well trained technicians, but even these are frequently used while the graduate students whose sarmples are being analyzed or tested are present. The net effect has been to attract more and better graduate students funded by this project. For the continued health of the laboratories and of the quality of the graduate training they provide, it is vitally important that this equipment be well maintained and periodically updated so that a state-of-the-art operation can be sustained. 16. Research Results of Project Laboratories. In some labs the equipment acquired in the project has made possible research results which are up to or surpass world standards. In many others, the quality and quantity of research have both improved notably. In considering the impact on research it is helpful to understand these in terms of gains in accuracy and precision of measurements and marked increases in computational power and speeds. New equipment has enabled China to enter new fields of inquiry and to reconceptualize problems in innovative ways as a result of the gains in measurement and computation. The benefits are in both fundamental science and in areas of potential and actual applications. An important measure of the adequacy of this research is the number of papers published in peer-reviewed domestic and international journals and as the indicators show, these numbers have increased substantially. 17. The most important component of the training of graduate students in modern science and technology is the original research they perform for their graduate degree theses and dissertations , under the direction of qualified research advisers. The  -6 - equipment acquired by the laboratories with the funds supplied by the project has been instrumental in perrnitting such research to be conducted at an improved level of quality and quantity. 18. Research Funding. The opportunities for research funding in China usually come from three types of sources: (a) first is the "vertical" dimension which includes various sources of funds from the central government; (b) second are "horizontal" sources which includes projects from other ministries, contract research, the sale of technology and products from the labs and from local governments. These sources have been increasing over time and now constitute an increasing portion of the research funding. Although such activities may be beneficial in the short run, they can jeopardize the long range soundness of laboratory and the quality of the training of its students, if it becomes dependent on this type of funds; and (c) third, are fumds that come from the international environment. In some cases, this involves foreign companies working with Chinese labs, or with the larger institutions of which the labs are a part (or with the case of the Ford Motor Company, with the NSFC). International sources of funds - from foreign governments or international organizations - are also becoming quite important for labs working on environmental problems. 19. One of the assurances received during preparation was that as the project increased the quality of the research infrastructure, the government would increase the amount of research funding available to the labs. Although no project funds were used to directly support research, in is clear that the government provided an increasing level of research funding, both the individual project-supported labs and the research community more broadly. The available research funds in project-supported labs more than doubled during the three year period 1994 through 1997, growing from a laboratory average of Y 0.9 million for 1994 to Y 2.1 million for 1997. Despite the increases for research funding in the KSDP labs, the national expenditures for R&D increased only slightly, in real terms after correction for the relatively large scientific inflation rate. As privileged institutions, it is likely that SKLs and SL did better than average in staying even with this inflation. 20. China's research policy in recent years has sought to incorporate meritocratic criteria, usually applied through peer review, in its funding decisions. In the SKL program, this has led to a hierarchical ranking of research centers (and projects) with different amounts of resources flowing to different levels in the hierarchy. Accordingly, Chinese laboratories have become very conscious of their rankings and very concerned about maintaining or improving them. The accreditation process plays a key role in this process. 21. Peer Reviewed Research. Fundamental to the design of the project was the strengthening and broadening of the model of principal scientist proposed, peer and panel reviewed research funding. This model was introduced was introduced into China by the U.S. National Science Foundation (NSF) through the NSFC and was used as the primary modality for deciding on how project funds would be distributed. The SDPC, having  -7 - decided on the key areas3 in which the project could consider applications, established a series of peer review panels which screened proposals, rejecting some and requesting substantial modifications in most of the remaining. The thrust of this design component was to try to enhance the role of the individual laboratory director and increase his or her responsibility for the research direction of the lab, the quality of the research and the development of the graduate programs. 22. This goal is in the process of being achieved. It is an observable fact that over the life of the project lab directors have increased their responsibilities, in the development of research agendas, seeking funding to support a sizable portion of the laboratory research, and, to an increasing extent, the content of course material offered in their labs. 23. Accreditation. Although the World Bank did not play a role in the process, accreditation of SKLs that was conducted by the NSFC on behalf of the SPDC, (which considered the overall value of the labs to society) and the SSTC (whose primary interest is in operational capabilities to conduct research). This accreditation process followed an ear.ier review of the KSDP labs which determined that they had met the criteria for initial designation as SKLs. Once accepted as a SKL, the laboratory was then eligible to be cornsidered for accreditation. Once accreditation was achieved the SKLs were eligible for subostantially increased core funding from the government and became favored recipients of bloc funding for research projects. The accreditation review for all 75 SKLs in KSDP has now been completed with all passing the accreditation process. 24. The situation for the SLs is different. The Special Labs (SLs) were from the start seen as a way to support basic rather than applied research in universities and therefore had a less direct connection to the near-term economic development goals of the country and did not receive additional foundational government funding as had the SKLs. These wi l not be reviewed using the SKL procedures described above. Instead, they will be subject to review as "open labs" by the MOE which, if they pass, will provide them with a research support budget which will allow them to both carry on their own work and to invite scientists from other research institutions to participate in research work. It is reported that the SLs will be eligible for "211 "4, but this decision has yet to be made by the Ministry of Education. 25. Academic Committees. Academic Committees were established in all of the laboratories. These Committees meet about once a year mainly to approve the research projects of visiting scientists from other Chinese laboratories. In some cases they also examine research proposals to be submitted to NSFC and other funding agencies and suggest new areas of research. The central in-house research activities are determined by These were in the following fields: energy, raw and processed materials, transportation and communications, machinery and electronics, agriculture, bio-engineering, new materials, medicine and health care; and the environment and earth science engineering. The "21 1" Project refers to the selection of 100 leading universities that will be built up in order to reach international standards in the 2 1" century.  -8 - the professors, the ideas they have, and the research support they can generate from NSFC and other agencies through competitive proposals. 26. "Progress in Natural Sciences" began publication as a result of project support. It started in 1991 with an initial budget of RMB 400,000, the journal combined two previous publications which reported on the activities of the SKLs. It is published in Chinese and English by the NSFC and reports research findings from the SKLs. The peer-reviewed articles which appear in the journal cover the spectrum of science done in the SKLs and therefore rivals the Chinese Academy of Sciences (CAS) own "Science in China" in scope, coverage and, reportedly, importance. The current rejection rate of articles by its international editorial board is from 70 percent to 80 percent of submissions. Of China's 3000 academic journals, "Progress" is one of 28 which is listed in the Scientific Citation Index. With a domestic subscription of 3000 and international subscription of 30-40, the journal is likely to play an increasingly important role in disseminating the scientific results of China's strongest laboratories both in the CAS and the universities of both the line ministries and the MOE. 27. In 1997 the journal won a Third Prize for scientific journals given out by the Chinese government. The pressure to increase distribution from the current once every two months to monthly is great as the volume of good quality submissions is increasing. The NSFC is continuing to support the publication of the journal with a subsidy of RMB 500,000/ year. Over the life of the project the international distribution of the journal was assisted by the British publisher Francis and Taylor, a connection that was facilitated from the start by the World Bank. Francis and Taylor continues to support the international distribution since the closing of the project. This jury-reviewed journal with large domestic and growing international distribution is one indicator of the strength the research produced in the project-supported labs. Likewise the NSFC's decision to continue its support to the journal is an indicator of the government's commitment to sustain the momentum of the labs. 28. Joint Laboratories. One of the goals of the project was to encourage scientific cooperation across institutional boundaries and to this end the project supported joint laboratories. The joint laboratories were meant to bring together labs in different universities or from a university and a CAS institute to focus on one research topic through sharing instrumentation, faculty and graduate students and research funding, the objective was to support the notion that science is a collaborative venture. Five of the project's 133 labs were designated as joint laboratories. This effort was not completely successful because of the strong preference for academic compartmentalization and the pressure on lab directors to build fully integrated and complete labs so that their labs could be accredited. Accreditation meant increased government support for research. Since labs were accredited as individual and not joint labs, the incentives did not exist for true partnerships. 29. Pilot Laboratory Management Program (PLMP). Improvement of laboratory management was an explicit goal of the project, with the seven lab Pilot Laboratory  -9 - Management Program (PLMP) being the main vehicle to this end. UNDP contributed $253,000 to support this component of the KSDP. In broad terms, the objectives of the program were to provide a forum for lab directors from different fields to compare experiences, to provide a limited amount of management training to lab personnel, to encourage awareness of technology transfer possibilities, and to disseminate the lessons of the project to the other participants in the KSDP. Although improvements in the PLM4P labs are visible, the overall impact of the PLMP program is harder to see outside the seven laboratories chosen for participation in PLMP. 30. It is clear that considerable efforts were made in the 1991-1995 period to make the project a success and to propagate the lessons learned but it was not evident that laboratories beyond the seven knew of PLMP lessons. Possible explanations for the disconnect between the efforts expended by those involved with PLMP and the other labs are: (a) the NSFC had limited resources to expend on follow-up activities; (b) the special resources going to the seven PLMP labs for managerial improvements were not available to the other KSDP participants; and (c) there were a good number of changes in lab leadership during the course of the project. 31. The project did not produce any one model of good management, but it did call attention to the importance of: (a) having strong leaders as lab directors (establishing the "director responsibility system") and as chairmen of the academic committees; (b) using the academic committees effectively to set research and teaching directions, and to make contacts with other parts of the technical community; (c) setting clear objectives for the lab's work; (d) developing cooperative relations with industry; (e) handling effectively the procurement of advanced imported equipment; (f) establishing effective rules and regulations for safe and efficient lab operations; and (g) following the main project guidlelines - "opening, interchanging, collaborating and competing." One way to capitalize on this component of the project would be to produce a manual for laboratory management that incorporates the experiences of the PLMP labs and to widely diss,eminate the manual. Naltional Computer And Network Facility (NCFC) 32. Implementation of the Demonstration Computing Network. The project sought to support the development of a prototype scientific computing facility that would ultimately be expanded nationwide. The objective of the component was to facilitate resource sharing amongst institutions by establishing an enhanced computation facility linked by a demonstration network to the computer centers of Peking and Tsinghua Universities and the Computing Center of the CAS. The strategy for implementing the NC'FC was to first construct the local area network (LAN) at each participating institution and then establish the NCFC backbone network connecting the three institutions. Procurement of the computer was to be contingent on the successful completion of the network sometime between 1991 and 1993.  -10- 33. The LANs were in place by 1993, the fiber optic network linking Peking and Tsinghua Universities and some of the Beijing Institutes of the Chinese Academy of Sciences was put into place by March 1994 but the mini-supercomputer was not installed until June 1996. The delays can be attributed to several problems: the limited budget available for the purchase of the mini-supercomputer and the export license restrictions placed on high speed computers purchased by China which remained in place until the end of 1993 are chief among these reasons although the cumbersome Bank's two stage ICB process also added to the delays. 34. Despite the delays, substantial progress has been made at the NCFC. There are nearly 200 science and technology institutions connecting with NCFC including 123 institutes under CAS and several dozen institutes of state ministries and commissions. The center currently has a CPU utilitization rate of 70 percent with parallel computing in full operation. There are several scientific computer projects on going with support from the NSFC. 35. One of the side effects of the KDSP points to the possibility of the emergence of indigenous parallel processing supercomputers which may serve regional needs in ways which the NCFC is unlikely to do. The development of the "Dawn 1000" at the University of Science and Technology of China in Hefei is a case in point. Reportedly, the Dawn machine, designed using architecture similar to the NCFC mini supercomputer, is being made available to scientists in Nanjing and Shanghai as well as to those in Anhui, and a second generation version is now under development. The current one has a peak speed of 2.5 Gflops and an actual parallel Linpack performance of 1.6 Gflops which puts it in a range of performance only a factor of about 2 below that of the SGI Power Challenge at NCFC or of the IBM S-2 at Tsinghua University. 36. Users/Allocations Committees. A User's Committee of representatives from nine different scientific disciplines was established in 1997. The committee has the responsibility for formulating policies on the allocation and use of the supercomputer resources, allocation of time, examining and approving users' applications for computing on the supercomputer, monitoring the operation and services of the supercomputer and organizing academic exchanges for users. 37. Internet Connectivity. One the major successes of the KSDP was the substantial increase of Internet access to China. The establishment of the Internet link in April 1994 was the first multi-institutional Internet link between the Beijing scientific community and the international academic community. The early developmental problems of establishing the internet link experienced by the NCFC has allowed for easier startup for the number of links which have been established since that time. One and a half years after the operation of the NCFC network, the MOE network (CERNET), a spin-off of the NCFC with its center at Tsinghua University linking many universities all over China, was put into operation. Users at Peking and Tsinghua Universities have transferred off the CASNET which was set up by NCFC and now have moved to the CERNET thus providing better service to both universities and institutes of the CAS.  38. Based on NCFC, CSTnet (originally CASnet) has evolved to cover the provincial capitals all over China. At present, CSTnet has a 4Mbps international channel with the United States Internet and a 128 Kbps channel with Japan Internet. There are approximately 500 access networks with 50,000 computers directly linked to the network ancl approximately 100,000 enduers. CSTnet has interconnected with CERNet, ChinaNet ancd GBNet. CSTNet has networked with over 300 research institutes which are spread across 30 provinces or cities nationwide in various sectors. (i.e. the Chinese Academy of Sciences, National Natural Science Foundation, and Ministry of Science and Technology) Technical Assistance 39 Programme Advisory Group. The PAG, consisting of five senior Chinese scientists and four scientists from abroad, was established in 1992 with support from UNDP to provide continuing high quality consultation and advice to the MOE and CAS on the overall direction and progress of implementation in the various components of the K'SDP. It met at least once each year during the course of the project, and was supported by a secretariat established in the MOE's Foreign Investment and Loan Office (FILO). The PAG was expected to support the SDPC's role in providing guidance to the Project at the policy level as well. During project implementation the PAG was especially attentive to the SKLs, the PLMP, and to the NCFC; its agenda also came to include the promotion of the use of TA, making suggestions about the repayment of the loan, technology transfer issues, and helping to speed up China's connection to the Internet. It was active in reviewing the annual indicators of progress for each lab and made site visits to most of the labs during the course of the project. In addition, it made study tours to North Amnerica and to Japan. These roles proved to be invaluable for the supervision of the project. Bank missions used the PAG reports to augment its supervisions, sought advise from PAG members and used the group to push project goals forward. 4CI. The PAG played an important role in helping to integrate the diverse components of the KSDP. However, as an advisory committee, it did not have the executive power and organizational resources to push project implementation. Laboratories visited during the final evaluation were often unclear about the PAG's existence. Stronger integration of the program components would have added additional value to the overall KSDP. This is especially true with regard to a stronger central role in coordinating solutions to pr ocurement problems and the setting aside of monies for spare parts, and in wiser and more imaginative uses of TA funds. Although it is not clear that PAG was equipped for these roles, the absence of a stronger central coordinating body meant that the promise of the KSDP as an integrated national program, as opposed to an equipment acquisition effort, was not fully met. 41. Domestic and International TA Programs. Domestic and international technical assistance programs were designed to promote collaborative research with foreign researchers. This included attendance at international meetings, inviting foreign experts to China to give lectures and provide advice on research work, short overseas visits to research laboratories and personnel training. Data received from both the MOE  - 12 - and CAS show that the overall numerical targets were reached. However, the quality of these programs could have been better if the foreign specialist program had organized tours for the world renowned specialists to several universities or institutes so that they could deliver several keynote lectures with broad integrative themes in science or laboratory management. C. IMPLEMENTATION RECORD AND MAJOR FACTORS AFFECTING THE PROJECT 42. Project Concept and Design. The basic concept of the project, of providing IDA assistance to qualified labs to help bring them up to international standards, was sound. It accorded with Chinese national policies and dovetailed with national programs to raise the levels of Chinese advanced education and research. The attention given to the creation of a national supercomputer center and to upgrading the quality of lab management were also sensible objectives which addressed Chinese needs. The idea of having an overall steering or advisory committee also was sound given the scope and complexity of the project, as was the idea of adding value to the project by having a lab management development component. 43. Degree of Achievement of Project Objectives. The objectives of the project were unevenly met. Achievements in graduate education and in research performance are quite impressive and are consistent with the logic of the project design. Progress on the development of a national computer center was initially slow, but, since 1996 large progress has been made in connectivity and organizational structure of the NCFC. 44. In the area of laboratory management the progress made appears to be a product of the confluence of the project's initiation and changes in the overall economic and social environment in China. The PLMP program was a useful part of this process of upgrading laboratory management, but its potential contributions to the project as a whole were not fully realized. More active central direction and leadership could have strengthened the component. The PAG, while performing many important roles envisioned for it, could not function as an executive body that could pull the diverse elements of the project together. 45. The Role of China International Center for Economic and Technical Exchanges (CICET). CICET is, formally, the executing agency for the UNDP supported components of the project. Although the ICR mission did not have meetings with CICET, it was discussed with some of the participants of the KSDP who voiced considerable dissatisfaction with the ways in which funds were dispersed by CICET, especially with regard to the payment of consultants. Payments were slow and the bureaucracy involved was tiresome and burdensome. Much of the technical assistance portion of the project would not have been possible without the funding provided by the UNDP. But the necessity of using CICET to administer the funding became an impediment to using this technical assistance efficiently.  - 13 - D. PROJECT SUSTAINABILITY 46. Prospects for Project Sustainability. The strong likelihood of project sustainability can be seen in a number of areas: (a) key to the success of the project from the point of view of the Chinese government has been the accreditation process that the labs have undergone in order to be targeted recipients of increased funding. This accreditation process, that project-support labs passed, is now complete and the additional funds are coming to the labs, from the NSFC, the Ministry of Science and Technology and line ministries with specific research projects. The yh Five Year Plan targeted S&T funding for the accredited SKLs and the current plans for the I O' Five Year Plan is reportedly going to target even more funds for the nation's SKLs; (b) the number of graduate students produced by the project-supported labs, approximately 8000 at the MSc and Ph.D. levels, have been trained to world standards and many are placed in labs all over China teaching new graduate students and carrying on research projects; (c) the NCFC as a national model for computing and networking has spawned computer networks in other major cities and is the heart of the MOE's own Chinese Education and Research Network (CERN); (d) the NSFC continues to use panel and peer review for the distribution of the large bulk of its research fumding. Although this constitutes only a small percentage of the overall Chinese research funding, it does indicate that the process used by the SDPC to support labs under the Key Studies Project was seen as a success by the Chinese government; and (e) the new equipment brought a stronger fundamental comnponent to the graduate curricula and the more specialized courses are not as narrowas they were at the start of the project. This trend helps to avoid rapid obsolescence of the graduate students trained under the previous more narrow approach. 47. There are, a couple of issues which may impede sustainability. First, the meager faculty salaries and graduate student stipends are a problem for the project's sustainability. The prospects of working in well equipped laboratories and doing world class science in China - made possible by the project - is a powerful attraction for young scientists. However, the gap between income levels in the Chinese academic world and other career opportunities (e.g., academic opportunities abroad, careers in business) continues to widen drawing good graduate students away. Until this gap is narrowed many of the post-doc students that go abroad for training will not return to Chinese laboratories in large numbers. Second, although there are to be increases in government support for basic research, the growing importance of contracted applied research and development for the participating labs, and the pressures to transfer technology, could distort primary mission of promoting advanced graduate training in basic science. The long standing problem in China of relating research to production remains. 48. Overall, the prospects for sustainability of the project is likely. The government of 'China has made a strong financial commitment to SKLs and the "211" project will provide resources to many of the SLs. The laboratories and universities have developed special incentives to keep good Ph.D. graduates i.e. improved housing, larger research budgets, accelerated promotions and good graduate students. The government has identified science and education as the key to achieving economic development in the 21l'  - 14- Century and is targeting university labs, including the SKLs, to play a key role in this effort. These targets are being supported by a number of priority programs which have brought increased funding in the 90 Five Year Plan and can be expected to continue in the I Oth Five Year Plan. E. BANK PERFORMANCE 49. Bank, UNDP, and Borrower Performance. The project was designed with close cooperation between IDA and the Chinese Government. The Bank Group mobilized a team of qualified experts to work with MOE and CAS so that the laboratories could participate on a large scale. IDA supervision missions combined with the PAG supervisions allowed each laboratory to be visited as many a two or three times. This provided continuous monitoring of the implementation of the project. The Bank and UNDP both conducted a joint mid-term review of the project in January 1994, using a team of qualified scientists to review the progress of the graduate studies and research projects in the laboratories. The final ICR mission was also a joint Bank-UNDP effort. Financial assistance from UNDP made it possible to continue the support of the PAG which proved to be a excellent mechanism to assist IDA monitoring implementation of the project. 50. One of the main problems with the project was that of procuring the desired equipment in a timely fashion. This problem is a function of both the Association's performance and that of the borrower. The employment of ICB procedures by the Bank for a project of this kind was inappropriate, as noted in the mid-term review. The delays which are a part of ICB procurement, plus the pressures to bundle contracts to achieve economies of scale often mean that instrumentation is often not the latest model or is not exactly what is needed because of the homogenization of technical specification. On the other hand, Chinese procedures also created more difficulties than were necessary, especially procedures which required middle-men as procurement agents who increased the distance between end users and equipment vendors. Likewise, the MOE decided not to use the modified ICB process proposed during project preparation. 51. The Role of the United Nations Development Programme (UNDP). The UNDP was instrumental in providing the financial support necessary to support most of the non-equipment value-added components of the project, (e.g., the PAG, the PLMP and the TA components). During the project design period it was not possible to convince the SPC, and subsequently the MOE and CAS to use of IDA funds for these value-added portions of the project. Hence the support provided by the UNDP was critical in trying to introduce new lab management methods, in supporting the PAG' s activities, and the technical assistance to introduce new ideas into the Chinese scientific system. Cooperation between the UNDP and the World Bank was close and productive, especially during the mid-term review and ICR mission when the UNDP participated in both exercises. The UJNDP's role in actual project implementation was minimal. Evolving country program objectives and changes in personnel contributed to this  - 15 - problem. A more active interest on the part of UNDP in its share of the project - including improving procedures with CICET - would have been desirable. F. BORROWER PERFORMANCE 52. In line with a good record in designing, preparing and implementing the IDA assisted education projects, the Chinese team demonstrated satisfactory performance in preparing and implementing this project. A problem with the project was the failure to achieve a higher level of coordination and integration of its many parts. This problem was a function of the lack of a strong executive project coordinating body in China. It was expected that the SPC would perform this function when the project was initiated, ultimately the SPC did not play this role. Instead, it ceded it to MOE which was organizationally ill-equipped for this task. These arrangements kept the project from achieving the kinds of benefits from synergistic interactions which were envisioned at the outset of the project. Difficulties in managing a project with such complexity was identified in the SAR as a risk. Assurances by the government at the time of project preparation that strong project coordination would be provided proved to be more difficult to carry out during the implementation stage. G. ENVIRONMENT WITHIN WHICH PROJECT WAS APPRAISED AND APPROVED 53. The project was successfully appraised on June 3, 1989, presented to the Board fo:r approval on January 29, 1991, signed March 15, 1991 and made effective on June 4, 1991. The lengthy time between appraisal, signing and effectiveness was the result of decisions taken within the Bank based on realities in China especially within the academic community, and had an impact on the project in several ways. Equipment necessary to support new research and graduate training programs was delayed for some two to three years. Equipment lists drawn up during project preparation in 1988 were three years old by the time procurement started and had to be modified causing delays. Some laboratory research agendas had, inevitably, changed over the period and no longer focused on the targets agreed at appraisal. Graduate enrollments in major universities dipped substantially in the 1989, 1990 and 1991 academic years and this reportedly affected the quality of the students taken into the project-supported labs. It is important in the review of this project to factor in the full impact of these delays, and the reasons which caused them, when reviewing project indicators, duration of project implementation, dates of closing, and so on. H. ASSESSMENT OF OUTCOME 54. Project Outcomes. The project outcome is rated as satisfactory. There are increases in graduate students trained at high levels of quality, enhanced capacity for such training in the future, and impressive research results emanating from the project. Laboratory management has improved and more effective technology transfer strategies are beginning. Overall, the project made a significant contribution to Chinese research  - 16 - and advanced education capacities and to the modernization of thinking about research, education, and laboratory management. I. FUTURE OPERATIONS 55. Experience derived from the design and preparation of Key Studies Development Project had a direct impact on the Technology Development Project (Ln. 3847-CHA) and, to a less degree, the Higher Education Reform Project (Ln. 4474/Cr. 3213-CHA). The former used many of the targeting and review mechanisms used in the Key Studies Project while the vast bulk of the universities supported in the latter project where also in the Key Studies Project as well as one of the two earlier University Development Projects. Future lending in the education sector is now being discussed with the government and if such new lending develops the area of science and technology is a possible area for further investment. J. KEY LESSONS LEARNED 56. Lessons Learned. The main lessons learned in this project are that: (a) with appropriate resources and careful planning it is possible to accelerate China's scientific and technological development by training its graduate students at international levels of proficiency and in increased numbers; (b) it is difficult to transfer managerial experience acquired in other countries to China; (c) the scarcity of equipment has made the community maximize the amount of new equipment purchased, at the expense of acquiring adequate supplies of spare parts and consumables for the proper operation of that equipment. This occurred in spite of built-in assurances to the contrary; (d) the sharing of major facilities, such as the mini-supercomputer at NCFC or the joint laboratories, among a large variety of research and teaching institutions, is difficult due to the strong compartmentalized organizational structure within the S&T community; (e) the dissemination of managerial know-how within and among laboratories is slow and more efficient means should have been adopted from the very beginning; and (f) the PAG might have adopted a more pro-active role, akin to that of a managerial Principal Investigator (PI) for the Project.  - 17- PART II: STATISTICAL TABLES TABLE 1: SUMMARY OF ASSESSMENTS A. Achievement of Objectives Substantial Partial Negligible Not Applicable Macroeconomic policies x Sector policies X Financial objectives x Institutional development X Physical objectives X Poverty reduction X Gender issues x Olher social objectives x Environmental objectives X Public sector management x Private sector development X [, Project Sustainability Likely Unlikely Uncertain Li ~~~~~~x C. Bank Performance Highly Satisfactory Satisfactory Deficient Identification P-reparation assistance -Appraisal __ _ _ _ _ _ _ _ _ _ _ -Saipervision I _ _ _ _ __ _ _ _ _ _ D. Borrower Performance Highly Satisfactory Satisfactory Deficient Preparation X Inplementation x C;ovenant compliance x Operation (if applicable) E. Assessment of Outcome Highly Satisfactory Unsatisfactory Highly Satisfactory Unsatisfactory x  - 18 - TABLE 2: RELATED BANK LOANS/CREDITS Year of Loan/Credit Title Purpose Approval Status Preceding Operations University Development Project- (LN. 2021-CHA To strengthen science & engineering in 28 1981 Completed 1986 Cr. 1167-CHA) leading Chinese Universities. Agriculture Education and Research Project - (Cr. To assist agriculture higher education and 1982 Completed 1989 1297-CHA) research in 11 agricultural colleges and 7 institutions. Polytechnic/Television University Project - (Cr. To increase the number of students and to 1983 Completed 1992 141 1-CHA) improve the quality of instruction in China's 17 polytechnic institutions and 28 television universities. Rural Health Medical Education Project (Cr. 1472- The educational component of this project 1984 Completed 1991 CHA) was designed to enhance the quality of education, training & curriculum. Second Agricultural Education Project (LN. 2444- To improve and assist agriculture education 1984 Completed 1992 ClLA) in 23 agricultural colleges, 12 agricultural technical schools. Second University Development Project (Cr. 1551- To increase technical & managerial 1985 Completed 1992 CHA) personnel through policy & institutional changes related to the engineering and economic/finance subsector. Provincial University Project- (Cr. 1671-CHA) To provide direct assistance to 60 provincial 1986 Completed 1992 university in 28 provinces and planning assistance for the higher educationsubsector. Gansu Provincial Development Project - (LN. Part of comprehensive plan to attain universal 1987 Completed 1995 2812) Education Component (Cr. 1793-ClIA) nine-year basic education and to improve the quality of instruction in primary and lower secondary schools in Gansu Province. Teacher Training Project (Cr. 1908-CHA) To support the Govemment's efforts to 1988 Completed 1993 achieve UBE by expanding and upgrading in- service training of lower secondary school teachers with an emphasis on improvement in quality of teachers. Textbook Development Project-(Cr. 2006-CHA) To support the textbooks upgrading program 1989 Completed 1995 by the Chinese govemment. Objectives are to improve the education quality of textbooks at all levels of schooling, to increase the variety of available textbooks and to improve .__________________________________________ the efficiency of textbook production. Vocational & Technical Education Project (Cr. To improve the quality of technical & 1990 Completed 1995 2114-CHA) vocational training in China through supporting 59 secondary vocational/technical schools in 10 provinces and 3 municipalities Medium-Sized Cities Development Project (LN. To assist Changzhou, Luoyang and Shashi 1991 Completed 1997 3286-CHA) cities to improve overall sector planning and l___________________________________________ - management including education.  - 19 - Year of Loan/Credit Title Purpose Approval Status Following Operations: 'Education Development in Poor Provinces (Cr. To support three main goals for the 1992 To be completed 2339-CHA) development of education in relatively poor 1998 provinces-Shanxi, Shaanxi, Hubei, Hunan, Guizhou and Yunnan. Effective Teaching Services Project (Cr. 2471- To contribute to improved leaming at the 1993 To be completed CHA) lower middle level by providing teaching 1998 services, including instructional methodology, teacher training and management of teachers training and assignment. Basic Education in Poor and Minority Areas (Cr. To support the attainment of universal 1994 To be completed 2651-CHA) primary education and expansion of coverage 2000 of lower secondary education in poor and minority areas in 6 provinces: Xinjiang, Sichuan, Inner Mongolia,Ningxia, Guangxi, and Jiangxi. Third Basic Education Project (Cr. 2831 -CHA) To support the attainment of universal 1995 To be completed primary education and expansion of coverage 2001 of lower secondary education in poor and minority areas in seven provinces: Anhui, Fujian, Gansu, Hebei, Henan, Jilin and Qinghai Vocational Education Reform Project (Cr. To improve and increase the supply of skilled 1996 To be completed 2063/LN. 2898-CHA labor, raise the quality and efficiency of the 2002 VTE system and build up capacity for monitoring and evaluation in four high growth provinces andTianjin municipality. Fourth Basic Education Project (Cr. 2954-CHA) To increase access/equity for the absolute 1997 To be completed poor in primary and junior secondary 2002 education; to enhance the quality of primary and junior secondary schooling, as well as teacher training education and training programs; to improve efficiency in the delivery of education through improving educational management at all levels Southwest Poverty Alleviation Project (Cr. 3906- The education component of the project will 1995 To be completed CHA) address low educational attainment among 2001 those in project areas by enhancing the delivery of primary education to the poor through primary school renovation and construction, tuition assistance and nutritional supplements for poor students, provision of textbooks, instructional equipment and fumiture and teacher and management training. Higher Education Reform Project Ln. 4474-CHA The objective is to improve the quality and 1999 To be completed Cr. 3213-CHA) relevance of undergraduate basic science and 2005 engineering programs through integrated reform activities in curriculum and teaching methodology.  - 20 - TABLE 3: PROJECT TIMETABLE Steps in project cycle Date planned Date actual Identification 12/88 12/88 Preappraisal 5/89 6/3/89 Appraisal 9/07/89 9/07/89 Negotiations 4/16/90 4/09/90 Board presentation 2/20/90 2/26/91 Signing 3/15/91 Effectiveness 6/4/91 Project completion 12/31/95 6/30/98 Loan closing 6/30/96 12/31/98 TABLE 4: LOAN/CREDIT DISBURSEMENT: CUMULATIVE ESTIMATE AND ACTUAL ($ million) FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 Appraisal estimate 10.0 65.0 121.2 129.2 131.2 131.25 131.2 131.2 131.2 Actual 0.0 10.5 35.9 81.3 107.1 119.2 128.2 131.9 132.86 Actual as%of 0.0 16.1 29.6 62.9 81.6 90.8 97.7 100.5 101.2 adjusted estimate Date of final disbursement 4/12/99 5 Project was extended for the first time during FY97 6 Difference in Appraisal disbursements and actual disbursements is due to change in US dollar value of SDR.  - 21 - TABLE 5: KEY PERFORMANCE INDICATORS Estimated Actual Key implementation indicators in SAR Base Line-I Base Line-2 Target Actuals 1988 1992 1966 11/97 GRADUATE STUDENT ENROLLMENTS Enrollment-MSC 3197 3635 5659 4961 Enrollment-Ph.D. 788 1361 1886 3081 Enrollment-Post Doc 49 169 250 346 SUPPORT OF SCIENTIFC RESEARCH Numbers of Researchers in Laboratory 2818 3717 7510 3708 Availability of Research Funds per lab average annual fanding (RMB Million) 0.65 0.9 1.0 2.1 DEVELOP SCIENTIFIC INFRASTRUCTURE Percentage of Lab Equipment Purchased & Installed 3.17 15.7 100 96.7 INTERNATIONALIZATION OF SCIENCE Overseas Studies/Research & Intemational Conference Total for Laboratory 327 1649 4785 Total Supported by Project 11 306 780+1386 2016 Foreign Specialists Visiting Labs: Total forLaboratory 330 1697 5079 Total Supported by Project 2 82 305 573 GRADUATE STUDENT ENROLLMENTS ShiftinRatioofM.Sc.toPh.D.Degrees 5.4 3.7 <3.1 2.3 Percentage of Female Graduates to Total -M.Sc. Program 17.8 20.5 <22 23.8 -Ph.D. Program 7.0 9.4 <22 14.9 -Post Doc. Program 0.09 2.7 8.1 SUPPORT OF SCIENTIFIC RESEARCH Research Publications -Domestic Publications 27 89 40 236 -Intemational Publications 5 20 10 53 -International Conferences 5 19 12 53 BUILDING LABORATORY MANAGEMENT Establishment of Lab Academic Committee 137/142= 96% Open Laboratory 130/142= 92% Percentage of Labs using Peer Review 115/121= 95% DEVELOP SCIENTIFIC INFRASTRUCTURE Accreditation of Laboratory 100/142= 70% NOTE: In the original project documentation there are no development indicators. The list presented above was the result of efforts made by the Association and the Ministry of Education in 1994 to produce development indicators. The two different baseline data figures are for the pre-appraisal year of 1988 and the post signing date of 1992.  - 22 - TABLE 6A: PROJECT COSTS (Yuan million) Appraisal estimate Actual/latest estimate Item Local Foreign Total Local Foreign Total Laboratory Component 427.0 555.6 982.6 452.9 630.2 1,083.1 Pilot Laboratory Management Program 0.3 4.3 4.5 0.0 2.2 2.2 National Computer and Network Facility 5.3 22.4 27.8 12.8 21.6 34.4 Program Advisory Mechanism 0.1 3.3 3.4 0.0 2.8 2.8 Total Base Cost 432.7 585.6 1,018.3 465.7 656.8 1,122.5 Physical Contingencies 35.5 48.0 83.5 Price Contingencies 83.5 59.5 143.0 Total Project Costs 551.7 693.1 1,244.8 TABLE 6B: PROJECT COSTS ($ million) Appraisal estimate Actual/latest estimate Item Local Foreign Total Local Foreign Total Laboratory Component 81.8 106.4 188.2 86.8 120.7 207.5 Pilot Laboratory Management Program 0.1 0.8 0.9 0.0 0.4 0.4 National Computer and Network Facility 1.0 4.3 5.3 2.5 4.1 6.6 Program Advisory Mechanism 0.0 0.6 0.6 0.0 0.5 0.5 Total Base Cost 82.9 112.2 195.1 89.3 125.7 215.0 Physical Contingencies 6.8 9.2 16.0 Price Contingencies 16.0 11.4 27.4 Total Project Cost 105.7 132.8 238.5  - 23 - TABLE 6c: PROJECT FINANCING ($ million) Appraisal estimate Actual/late estimate Source |Government I IDA | UNDP Total Government IDA UNDP Total Civil Works 21.7 0.0 0.0 21.7 29.8 0.0 0.0 29.8 Equipment 42.8 115.9 0.2 158.9 57.0 120.7 0.0 177.7 Books & Educational 1.8 2.8 0.0 4.6 M[aterials 0.1 1.2 0.0 1.3 Trmining and Specialist 5.8 11.8 1.4 19.0 Services 0.5 10.5 0.5 11.5 Graduate Education and Laboratory Operations 33.7 0.7 0.0 34.4 9.0 0.0 0.4 9.4 Total Project Costs 105.8 131.2 1.6 238.6 96.4 132.4 0.9 229.7 TABLE 7: STATUS OF LEGAL COVENANTS Cove- Original Revised Agreement nant Present fulfillment fulfillment type Status date date Description of covenant Comments 3.0.1 .b 09 C 06/30/93 Adhere to the Implementation Complied with Program as in Schedule 4 of 01/04/94 Credit Agreement: (a) conduct mid-term review 3.0l.b 10 C 06/01/91 (b)- Maintain FILO in SEdC and Complied with OEC in CAS 3.0l.b 03,10 C 06/01/91 (c)- monitor availability of funds Complied with to SKL's and SL's 3.0l.b 03, 10 C (d)- identify causes of shortages of Complied funds and if happening address causes 3.03 12,10 C 08/01/91 Establish and Maintain Program PAG duties have Advisoiy Group (PAG) been completed with submission of final report to CAS and MOE 4.01.b (ii) 01 C 08/31/92 Fumish IDA certified Audit Last Audit report Report was for FY 97 and was received 08/25/98 4.01.c (iv) 09 C 08/31/92 Furnish IDA with opinion on Complied with Statement of Expenditures (SOE) 4.02 09 C 06/01/91 Monitor and evaluate project Complied with progress according to IDA accepted indicators and targets. Covenant Class: Status: I = Accounts/audits 8 = Indigenous people C = covenant complied with 2 = Financial performance/revenue 9 = Monitoring, review, and reporting CD = complied with after delay generation from beneficiaries 10 = Project implementation not CP complied with partially 3 = Flow and utilization of project covered by categories 1-9 funds 11 = Sectoral or cross-sectoral 4 = Counterpart funding budgetary or other resources 5 = Management aspects of the allocation. project or executing agency 12= Sectoral or cross-sectoral policy/ 6 = Environmental covenants regulatory/institutional action 7 = Involuntary resettlement 13 = Other  - 24 - TABLE 8: COMPLIANCE WITH OPERATIONAL MANUAL STATEMENTS There was no significant lack of compliance with an applicable Bank Operational Manual Statement (OD or OP/BP) TABLE 9: BANK RESOURCES: STAFF INPUTS FY 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 Total Preappraisal 4.8 46.7 160.6 212.1 Appraisal 124.4 124.4 Negotiations 24.2 24.2 Supervision 18.9 24.1 55.9 31.7 30.3 23.7 44.6 5.4 25.3 259.9 Completion 35.4 35.4 Total 48 46.7 60 24.4 18.9 24.1 55.9 31.7 30.3 23.7 80.0 5.4 25.3 631.8  - 25 - TABLE 10: BANK RESOURCES: MISSIONS Performance rating ______________________ ________ __________ _______ Specialized staff Imple- Devel- Stage of project cycle Month/ Number of Days skills represented mentation opment Type of year persons in field /a status /b objectives problems /c Preidentification Identification/Preparation 12/88 5 15 ED, 4 Cons. NR NR Preappraisal 6/89 7 20 ED,OA,5 Cons. NR NR Appraisal 9/89 5 16 ED, OA 3 Cons. NR NR Negotiation 4/90 4 0 ED, Eco.,Leg.DO NR NR Posltappraisal . . Postappraisal . . Board approval 2/91 0 1 yr. board delay of project Sigiing 3/91 0 NR NR Effective 4/91 0 NR NR Supervision I/Initial 4/91 1 7 ED NR NR Supervision 2 5/91 2 16 ED,TE 1 1 Supervision 3 10/91 1 14 ED 2 1 Supervision 4 9/92 1 14 ED I I Supervision 5 7/93 2 14 ED,ED 1 1 Supervision 6 10/94 2 21 ED,OA S S Supervision 7 7/95 2 10 ED,OA S S Supervision 8 3/96 2 14 ED,OA S HS Supervision 9 2/97 1 2 ED s s ,Completion 10 7/97 4 27 ED,OA,DO,OO S s Supervision 11 1/99 2 2 ED,OA S HS /a DO: Disbursement Officer; EC: Economist; EN: Engineer; FA: Financial Analyst; LC: Legal Counsel; 00: Operation Officer; RA: Research Analyst; RS: Resettlement Specialist; TE: Transport Economist; TS: Transport Specialist. /b 1: Highly satisfactory; 2: Satisfactory. /c Typical problems included: implementation delays in technical assistance and studies.   -27 - ANNEX A ANNEX A: BORROWER'S CONTRIBUTION TO THE ICR PREFACE 1. Due to the cooperation among the departments related and the united efforts made by the project institutions, the project has realized its planned objectives and accomplished the missions with striking success. The borrower has no objections to the project description and comment made by the Bank. For the project impact and project sustainability the borrower will account as follows. PROJECT IMPACT Laboratory Establishment and Development 2. With different groundwork and investment scale, SKLs had been intended to set up in line with the national accreditation criteria and SLs had been built with reference to SKL criteria. During the project implementation, operational mechanisms as "opening, association, mobilization and competition" have been put into practice, and meanwhile, a set of bylaws and regulations have been established and perfected. For increasing the management and operation efficiency, experience for pilot programs of laboratory management has been summarized and disseminated. Academic committees and laboratory management committees with scientific research authorities and pundits have been set up. Up to now, all the project SKLs have passed the State Key Laboratory accreditation and obtained the stable funding sources for operation. Thus the capability for getting national project has been strengthened, and the academic paper quality and quantity have been considerably improved. Meanwhile, the scientific research has boosted the teaching development. Specialized Laboratories have also developed in respects of the scientific research and teaching. The Bank-funded NCFC has set up the basic structure for Chinese science and technology going onto the international leading- edge track and provided the economic and efficient tool for international communications and resource sharing. Staff Development 3. Since the beginning of the project implementation, the basic infrastructure of the laboratories has been improved owing to the newly procured equipment and established management system, thus attracting and stabilizing a number of outstanding teaching and research staff. With years of efforts made, a first rank, steady-going teaching and research contingent has been developed with the well-known pundits as the head, middle- aged staff as the mainstay and young scientists as the principal part. These staff come from different scientific fields with optimized knowledge structure, reasonable age  - 28 - ANNEX A distribution, competence for shouldering high-level project and powerful potentiality for development. The project institutions have placed significant emphasis on nurturing young academic leaders, providing necessary conditions and opportunities for turning them into creative, active and adaptive teaching and research leaders. For ensuring the regular operation of the laboratories and equipment operational efficiency, all the project laboratories have brought up a group of independent and stable technical management staff. Disciplinary Development 4. The improvement of the laboratory conditions has greatly boosted the development of the scientific disciplines. Project laboratories use the newly procured equipment to develop new experiment courses, further strengthening the student adaptability for the scientific development. The opening of the laboratories attract experts and scholars with different academic background to engage in collaborative research and exchange leading to the overlapping and inter-inclusion of the scientific disciplines. In that case, the research has brought along the development of the teaching and the disciplines have become, as designed and expected, the basis both for teaching and for scientific research. Scientific Research 5. With the implementation of the project, project laboratories have obtained much competitive capability for getting national key projects of basic science and applied science. For the past several years, funding sources for these laboratories mainly come from various state funding projects and local cooperative projects. These projects have provided laboratories with research opportunities and necessary funds, enhanced the development of the teaching and scientific research, strengthened the connection with industrial enterprises, pushed the scientific research achievements into application, and obtained satisfactory social benefits and economic returns. Cooperation with domestic and foreign enterprises in different approaches as collaborative research and development, technology transferring and cooperative production, has increased the pertinence and relevance of the scientific research. Postgraduate Training 6. The leading researchers with high professional skills and profound academic attainment and the advanced research facilities attract more and more applicants for postgraduate education. Therefore, postgraduate enrollment, with the level of postgraduate training improved, and has been keeping a steady growth with striking increases of Ph.D enrollment. Domestic and International Academic Exchanges  - 29 - ANNEX A 7. The implementation of the technical assistance activities provided project institutions with more opportunities for international conference, study tours and fellowship, allowing project institutions to be well informed of the latest academic development and trends. Project laboratories have set great store by the regular academic activities. Besides usual thesis selection, public answering for postgraduate dissertation and. research subject identification, distinguished experts and scholars have often been invited for lectures and extensive collaborative research activities have been engaged, with academic ideology activated, academic vision expanded, new technology and ideas informed and project laboratories influence spread. PROJECT SUSTAINABILITY 8. Recently, the Chinese government has devoted much attention to the reforms and development of higher education. From 1995, the "Project 211 " was launched for strengthening 100 higher educational institutions and disciplines. In 1998, the State Council set up a Science-Technology Leading Group with decision-makers from related ministries as the members and the premier as the head, bringing forward the "Action Plan of Educational Revival for the 21lt Century". The government has also increased investment in education, allocating a one percent increase for education from central the budget, that is, from 5.2 percent in 1997 to 8 percent in 2000. At the same time, provincial financing is encouraged to increase the allocation for education. The government awareness and input increase for education afford guarantee of the project sustainability. For this, the project sustainability plan and related measures have been made as follows: (a) Establishing the relevant follow-up management organizations, keeping project laboratories in constant contact and making regular exchanges among project laboratories. Drawing upon the experience and lessons of the laboratory management to consummate the established laboratory management system and mechanism in order to adapt to the modem science and technological development; (b) Expanding the opening of the laboratories and improving the connection and exchanges among the research staff, trying to catalyzing the operational efficiency of the existing equipment and pushing equipment utilization rate with annual increasing rate by 5 percent; (c) Increasing the number of the staff for training and conference attendance abroad with annual increasing rate by 5 percent to 10 percent; (d) Setting down relevant preferential policies to increase the female postgraduate enrollment; (e) Taking specific measures to encourage academic paper publications on domestic and international journals and using awarding mechanisms to push quality paper and treatise publications;  - 30 - ANNEX A (f) Increasing the number of laboratory staff for domestic and foreign training to improve the academic level and professional competence of the laboratory staff; (g) Enhancing the funding management for ensuring the laboratory operational funding requirements; listing laboratory operation expenditure in the institutional annual budget plan as special funds; and (h) Strengthening the partnerships between project laboratories and industrial enterprises; making use of laboratory preponderance to develop social services and to explore the funding channels.  -31- ANNEX B ANNEX B: ICR MISSION'S AIDE MEMOIRE CHINA KEY STUDIES DEVELOPMENT PROJECT (Cr.2110-CHA) Aide-Memoire May 30-June 26,1997 1. A World Bank mission comprising Mr. Halsey Beemer (mission leader), Ms. Sandra Erb (Operations Analyst, Consultant) and Aron Kuppermann (Senior Scientist, Consultant) and visited Shanghai, Nanjing, Hefei, Harbin, Shenyang and Beijing June 3- 26, 1997 to review progress on the implementation of the Key Studies Development Project (KDSP). The mission was augmented by a member of the Programme Advisory Group (PAG), Mr. Stewart McIntyre, and by a project evaluation mission from the United Nations Development Programme (UNDP) which consisted of Masatsugu Kimura, Programme Officer, UNDP/Beijing, Liu Shenggang (Senior Scientist, Consultant), Ma Weixiang, (Senior Scientist, Consultant), Richard Suttmeier (Senior Policy Specialist, Consultant) and an Environmental Studies Team supported by UNDP and led by Jarad Cohon (Senior Scientist, Consultant). While in Shanghai the joint World Bank/UNDP team took part in the project-supported Technology Transfer Workshop at Fudan University. Mr. Hu Wenbin (Education Specialist) and Ms. Yang Xiaohong, (Disbursement Specialist) both of the Resident Mission Beijing participated in various Beijing mission meetings. The mission expresses its appreciation to the State Education Commission (SEdC), the Chinese Academy of Sciences (CAS), and the National Natural Sciences Foundation of China (NSFC) as well as educational and scientific authorities in Shanghai, and Beijing municipalities and Jiangsu, Anhui, Heilongjiang, Liaoning Provinces for the arrangements made and the hospitality extended the mission. Unless otherwise indicated in the follow-up letter from the World Bank to the government, the agreements recorded in the Aide-Memoire are final with the signature of the Task Manager. 2. The task manager would also like to recognize and thank the UNDP for its continuing involvement in the project and their substantial support of the final project evaluation. The UJNDP's funding of the technical assistance (TA) portion of the project allowed for considerable value added to project components which had impact on all project laboratories. 3. Attached to this aide-memoire is the "World Bank/UNDP Mission Report". (See project files.) This "Mission Report" provides additional details, explanations and  - 32 - ANNEX B discussion to points made in the aide-memoire and therefore paragraph numbers are provided to refer the reader to the specific expanded sections in the mission report. I. Summary. 4. Project Outcomes. All parties associated with the project can take considerable pride in its achievements. The most important of these are the increases in graduate students trained at high levels of quality, the enhanced capacity for such training in the future, and the impressive research results which emanated from the project. At a second order of significance, the improvement of laboratory management and the beginnings of more effective technology transfer strategies can be celebrated. Additionally, the project's support of the peer review process for distribution of scientific funding reinforced the governments policies of making such decisions based on the quality of labs and scientific personnel rather than by administrative decision. Overall, the project made a significant contribution to Chinese research and advanced education capacities and to the modernization of thinking about research, education, and laboratory management. (see Annex 1 for overall Project Development Indicators.) 5. Project Concept and Design. The basic concept of the project, of providing IDA assistance to qualified labs in order to help bring them up to international standards, was sound. It accorded with Chinese national policies and dovetailed with national programs to raise the levels of Chinese advanced education and research. The attention given to the creation of a national supercomputer center and to upgrading the quality of lab management were also sensible objectives which addressed Chinese needs. The idea of having an overall steering or advisory committee also was sound given the scope and complexity of the project, as was the idea of adding value to the project by having a lab management development component. The addition of technology transfer objectives to the project complicated the project design, but on balance was a useful addition. 6. Degree of Achievement of Project Objectives. As the discussion which follows indicates, the objectives of the project were met unevenly. Achievements in graduate education and in research performance as a result of the project are quite impressive and are consistent with the logic of the project design. Progress on the development of a national computer center has been disappointing to all involved. The Chinese side was slow in operationalizing their objectives for the center, specifying the type of equipment they wanted, and in organizing users in order to develop sensible policies for the center's utilization. 7. In the area of laboratory management, the considerable progress that has been made appears to be the result of both the project's initiation and changes in the overall economic and social environment in China which were supportive of managerial improvements. The PLMP program was a useful part of this process of upgrading laboratory management, but its potential contributions to the project as a whole were not as fully realized as they might have been had there been more active central direction and  - 33 - ANNEX B leadership. The PAG, while performing many important roles envisioned for it, could not function as an executive body that could pull the diverse elements of the project together. 8. Opportunities Missed. It seems clear to the mission that the project could have achieved even more if there had been an entity which could have played the role of Chief Scientist or Principal Investigator (PI). Missing from much of the project was the overall coordination of those components which would have genuinely contributed to the comrnon needs of the Chinese scientific community. Such elements as an integrated technical assistance program, a widely disseminated set of improved laboratory management practices, or a national computing and network facility which served the scientific communities of both the State Education Commission (SEdC) and Chinese Academy of Sciences (CAS) labs. The SEdC and CAS implementation units were more focused on individual lab equipment procurement issues than overall programmatic strengthening. The Programme Advisory Group (PAG) and the National Natural Sciences Foundation of China (NSFC) lacked the authority and mandate to play such a role and the State Planning Commission (SPC), who initiated the project, turned over implementation to the SEdC and CAS at an early stage. All this contributed to what the mission considers to be opportunities missed for the project as a whole. 1I. Implementation Issues 9. Project Extension. During meetings with the CAS and SEdC, as well as during the site visits, it has been reported that upwards of US$ 800,000 of ordered and contracted for scientific and computational instrumentation (evenly divided between the CAS and SEdC) would not arrive in China until after the project closes June 30, 1997. Over 80 percent of the equipment in question is spare parts and auxiliary equipment specifically ordered by lab directors in order to carry out research programs underway or envisioned in the near future. The CAS has requested that the project be extend in order to be able to take delivery of this equipment and have its costs covered by project funds. SEdC is willing to forgo the delivery of the US$ 400,000 of equipment and has not asked for an extension. 10. The mission strongly recommends that the SEdC reconsider their position and join with the CAS in requesting the Ministry of Finance (MOF) in making a request for an extension in order to allow for the full and complete delivery of project supplied equipment 11. Statement of Expenses (SOE) Review. Statement of Expenses (SOEs) for the project were reviewed and the found that the procedures followed by the SEdC were con-ect and in good order. Documentation has been maintained in good order, supporting documentation is adequate and expediters were found to be eligible for Bank disbursement. However, the mission was unable to cross check data gather from site visits with supporting documentation in SEdC files and therefore the mission was not able to carry out its planned full sample review. The mission is concerned that the SEdC Finance Department may well be understaffed with seven staff for seven full Bank  - 34 - ANNEX B projects. The over-stretched, because of the heavy work load in the Beijing, are not able to upgrade their own computer skills or make site visits to project universities to review the procedures being used at the university level. The review also found that communications between FILO divisions needs to be improved. For instance, it is reported that the TA Division is unaware of total level of project expenditure in this category which must make it difficult for the TA Division to adequately develop its annual program. FILO management might well consider increasing the training given to the Financial Division and seeks ways to increase useful communications between different divisions in FILO. 12. Implementation Completion Review (ICR) Process. The mission asks the SEdC and the CAS to complete its portion of the ICR report with special attention to the development of an operation plan for the next stages of the project-supported State Key Laboratories (SKLs) and Special laboratories (SLs) and the National Computing and Network Center (NCFC). Now that the investment phase of the project is complete, both SEdC and CAS should adopt an operational plan which will ensure that investments will be sustained. As outlined in the Bank's guidelines on ICR preparation, the operation plan should include indicators to monitor both performance and development impact. This operational plan should be sent to the Bank along with the Part Two of the ICR report and should arrive at the Bank by September 1, 1997. III. Mission Findings 13. Impact of the Project on Graduate Training - Trends in Graduate Enrollment. One of the most important objectives of the project was to enhance the training of graduate and post-graduate students in science and technology in selected laboratories, to the highest international levels. The trends in graduate enrollments at the State Key Laboratories (SKLs) and Special Laboratories (SLs) have been very positive. The overall project indicators for MSc, Ph.D. and post doc degrees have been exceed.(see attached indicators for overall achievement). However, the mission believes that even more students can be trained with the additional modern research equipment. What seems to be impeding the attainment of these higher goals are the strict quotas for graduate students imposed on the laboratories by State Education Commission (SEdC) and the excessively severe entrance examinations for MSc candidates which screened out well qualified students. 14. Another goal of the project was to decrease the ratio of MSc to Ph.D. enrollments. The indicators from early 1997 numbers shows that the MSc to Ph.D. enrollments have decreased from 2.67:1 in 1992 to 1.61:1 in 1996. The mission viewed this as a positive change, since Ph.D. students are normally trained to the state-of-the-art in their respective fields and MSc students are not. 15. The mission learned that the SEdC recently approved a policy of direct enrollment of some graduate students in Ph.D. programs, without requiring them to first obtain MSc degrees. The mission feels that this is a positive step, but the fraction of the total number  - 35 - ANNEX B of Ph.D. students admitted into such programs is relatively small, of the order of 10 percent or less. It would be desirable to increase this fraction to 15 percent or 20 percent and to extend it to all project laboratories. 16. The post-doc program is still quite small with only about three post-doc fellows per lab on average. Post-docs are a very important component in modem science and technology research activities, and their numbers should be substantially increased in the future. 17. Women Students. Overall figures show modest increases in graduate training for women. MSc degrees increased from approximately 21 percent to 26 percent, Ph.D. degrees 7 percent to 14 percent and post doc fellows from 0 to 4 percent. The reasons for the srnall representation of women at all levels are manifold. Efforts need to be made to increase the numbers of women as full participants in the science and technology mainstream of the workplace as well as to increase women's prominence in science and technology. In order to achieve this goal it is recommended that a special scholarship program be designed to attract women to careers in science and technology at the highest levels. It is important to increase in this way the pool of highly trained personnel needed for the country's development. 18. Impact on Curriculum Development. The project has had a positive influence on the curricula at the project laboratories. As the research performed in these laboratories became more modem and used more sophisticated equipment, the nature of the curriculum became broader. New courses were introduced as a result of the acquisition and capabilities of some of the new equipment. The net effect of all of these changes is that graduate curricula currently have a stronger fundamental component than earlier, and the more specialized courses are not as narrow as they once were. This trend is very positive, since it helps avoid rapid obsolescence of the graduate students trained under this new approach. 19. Student Access to Project Supplied Equipment. An additional important improvement in the training of the graduate students has been their hands-on utilization of the new scientific equipment. The higher quality and reliability of such equipment and its computerized operation and output permits their efficient use by larger numbers of students without requiring very detailed training in their operation. For the continued health of the laboratories and of the quality of the graduate training they provide, it is vitally important that this equipment be well maintained and periodically updated so that a state-of-the-art operation can be sustained. 20. Impact of Project on Research -- Research Results of Project Laboratories. In some labs, there is no doubt that the equipment acquired in the project has made possible research results which are up to or surpass world standards. In many others, the quality and quantity of research have both improved notably. New equipment has enabled China to enter new fields of inquiry and to reconceptualize problems in innovative ways as a result of the gains in measurement and computation. The benefits  - 36 - ANNEX B are in both fundamental science and in areas of potential and actual applications. An important measure of the adequacy of this research is the number of papers published in peer-reviewed domestic and international journals and, as the indicators show, these numbers have increased substantially. 21. The most important component of the training of graduate students in modern science and technology is the original research they perform for their graduate degree theses, under the orientation of qualified research advisers. The equipment acquired by the laboratories with the funds supplied by the project has been instrumental in permitting such research to be conducted at an improved level of quality and quantity 22. Research Funding. Being participants in the KSDP, and having the status and material benefits which came with participation, have clearly enhanced the prospects of the labs in the competition for future funding. These benefits, though, vary considerably from lab to lab depending on the field or discipline, the quality of the work, and the quality of laboratory leadership. The opportunities for research funding in China at the end of the project usually come from three types of sources: (a) first is the "vertical" dimension which includes various sources of funds from the central government; (b) second are "horizontal" sources which includes projects from other ministries, contract research, the sale of technology and products from the labs and from local governments. These sources have been increasing over time and now constitute an increasing portion of the research funding. Although such activities may be beneficial in the short run, they can jeopardize the long range soundness of the laboratory and the quality of the training of its students. Great care must be taken not to allow the laboratories to become excessively dependent on this type of funding; and (c) third are source of funds that come from the international environment. 23. The available research funds increased by a factor of 1.74 over the five year period 1992 through 1996, growing from a laboratory average of 1.21 million RMB for 1992 to 2.1 million RMB for 1996. This is more than double the SAR target for 1996 of 1.0 million RMB. Despite the increases for research funding in the KSDP labs, the national expenditures for R&D increased only slightly, in real terms after correction for the relatively large scientific inflation rate. As privileged institutions, it is likely that SKLs and SLs did better than average in staying even with this inflation. 24. China's research policy in recent years has sought to incorporate meritocratic criteria, usually applied through peer review, in its funding decisions. The mission applauds this trend and encourages its further extension. As represented by the SKL program itself, this has led to a hierarchical ranking of research centers (and projects) with different amounts of resources flowing to different levels in the hierarchy. Accordingly, Chinese laboratories have become very conscious of their rankings and very concerned about maintaining or improving them. The accreditation process plays a key role in this process.  - 37 - ANNEX B 25. Accreditation. An important element in the future life of the SKLs is the prospect of an accreditation conducted by the NSFC on behalf of the State Planning Commission (SPC) (which considers the overall value of the labs to society) and the State Science and Technology Commission (SSTC) (whose primary interest is in operational capabilities to conduct research). This accreditation process follows the earlier review of the Key Studies Development Project (KSDP) labs which determined that they had met the criteria for initial designation as SKLs. Once accepted as a SKL, a laboratory is then eligible to be considered for accreditation. The first round of accreditation review, which was focused on the 80 SKLs which were not part of the KSDP, has now been completed. The practices used for it will, in the main, be replicated for the 75 SKLs in the KSDP. 26. The situation for the SLs is different. These will not be reviewed using the SKL procedures described above. Instead, they will be subject to review as "open labs" by the SEdC. The SLs will be eligible for "211" funding, but this decision will be made by corresponding university authorities. 27. Impact of Project on Laboratory Management - Pilot Laboratory Management Program (PLMP). Improvement of laboratory management was an explicit goal of the project. (See paragraph 26 of Mission Report for detailed PLMP objectives). Although improvements in the PLMP labs are visible, the overall impact of the PILMP program is harder to see outside of the seven labs chosen for participation in PLMP. 28. While it is clear that considerable efforts were made in the 1991-95 period to make the project a success and to propagate the lessons learned, the mission was struck in its site visits by the consistency with which labs responded to questions about PLMP with puzzLed silence. Possible explanations for the curious disconnection between the efforts expended by those involved with PLMP and responses to PLMP-related questions during site visits are further discussed in paragraph 30 of the Mission Report and are briefly restated here: (a) the NSFC had limited resources to expend on follow-up activities; (b) the special resources going to the seven PLMP labs for managerial improvements were not available to the KSDP participants; and (c) there were a good number of changes in lab leadership during the course of the project. 29. Perhaps the biggest factor now operating among the KSDP labs for improved managerial effectiveness is the review and accreditation processes being carried out by the NqSFC on behalf of the SPC and the SSTC. Management practices are one criterion in these, but more generally, overall lab performance - in a sense, the sum of management practices - is being tested by authoritative outsiders wielding significant rewards and punishments. Assuming that the integrity of this evaluation system can be maintained and protected from politicking and the influence of personal relationships, lab management should continue to improve. The mission recommends that the experiences  - 38 - ANNEX B of the PLMP labs be incorporated into a manual for laboratory management to be widely disseminated. 30. Technology Transfer - Shanghai Exhibition. In the SAR, the importance of technology transfer to the success of the KSDP was not emphasized. However, the direction of China's policies for S&T in the 1 990s, and the rapid development and growing technological complexity of China's industrial economy combined to insert the issue of technology transfer as a far more central role than had been envisaged. Accordingly, the importance of technology transfer issues was considered in the PLMP activities and the PAG suggested that an exhibition be held at the end of the KSDP where many of the participating labs could promote their research results, inventions, and capabilities to industry. At the same time, a seminar was proposed wherein the present- day understandings of technology transfer processes could be discussed by Chinese and foreign experts. 31. The exhibition was held in Shanghai at Fudan University from June 5-13, 1997, with over 90 KSDP labs represented and over 1000 invitations issued to representatives of industry. During the period, over 800 industrial visitors attended over the period. (See Mission Report paragraph 38 for details of some immediate results of the exhibition.) 32. The inclusion of technology transfer concerns in the KSDP is an understandable and, in principle, useful element of the project. The mission did, however, have two concerns about this addition. The first is that much of the discussion of technology transfer seems to address only one part of what is the real objective of technology transfer, viz., technological innovations in the economy and in the delivery of "public goods" (e.g., environmental protection). A large literature and much experience suggests that views of innovation which see technology transfer as being primarily a "supply driven," "linear" process from research to production are quite inadequate. The second concern follows from the first, and has to do with the suitability of academic organizations as the primary agents for supplying technology. The mission therefore recommends that the promotion of technology transfer objectives among the participants in the KSDP be done with care, with due attention to current models of innovation and with due consideration for the special strengths, and "comparative advantage," of the laboratories. 33. The overall record of technology transfers from the participating laboratories is modest. This should not be surprising. The KSDP labs include a mixture of those whose primary mission is basic research and those with more applied programs. In both cases, the primary objectives of the project were related to graduate education and research. While technology transfer can - and perhaps should - play a role in these activities, fully developed attention to technology transfer would have involved a significantly different project design, one which should have made innovation the focus of attention. 34. National Computing and Network Facility (NCFC) -- Networking/Internet. The rapid growth of computer networking in China has been quite impressive and was  - 39 - ANNEX B not ant'icipated at the beginning of the project when the NCFC facility was first proposed. There are now four networks operating in China with Internet connections. These include the China Science and Technology Network (CSTnet), operated out of the NCFC, the China E]ducational and Research Network (CERNet), which is sponsored by the SEdC and links together 200 of China's institutions of higher education, CHINAnet, which is operated by the Ministry of Posts and Telecommunications, and the GBnet, operated by Liantong (UNICOM) which serves the "three goldens" project. The NCFC facility is the center of the CSTnet, which includes the NCFC backbone network, completed in 1993 made up of the local networks of CAS (CASnet), Tsinghua University (TUnet), and Beijing University (BUnet). 35. Mini-supercomputer. In addition to serving as the center of CSTnet, NCFC also serves as a scientific database center and, of course, as a center for supercomputing. There have been an inordinate number of delays in the selection and procurement of the mini-supercomputer some but by no means all caused by export license difficulties. Finally, the SGI Power Challenge System, arrived in June 1996. In August 1996 it was put inlo trial operation and in January 1997 into formal operation. It is being used by researchers at seven CAS laboratories and at Beijing University. Currently, its rate of utilization is about 70%. The availability of this facility to a wider user community is yet to be announced. 36. Due to the delays in the acquisition by the NCFC of the SGI Power Challenge, researchers at Tsinghua University obtained from the government non-World Bank funds with which they acquired an IBM SP-2 parallel computer. Its performance is comparable to that of the SGI Power Challenge. It currently has about 160 users, of which 130 are from 'rsinghua University, 10 from Beijing University and 20 from other universities in and out of Beijing. At any one time, 30 to 40 different jobs are running on this facility. All users have remote access to it and it is being used at about 70% capacity. Thus, Tsinghua University has shown its ability to rapidly acquire and put into operation a major computational facility and make it available to a relatively large number of users from diverse institutions. It is the opinion of this mission that these efforts deserve further and substantial financial support which will permit an additional upgrade as the Tsinghua computer becomes saturated and, of equal importance, to acquire the comrmercial software packages which will significantly augment the usefulness of this facility to the general scientific community. 37. Users/Allocations Committees. In order to maximize the scientific output of these computers, a users conimittee should be established at each, which should make recornmendations about which hardware upgrades and software packages to acquire, and on improvements on the computer's operation. In addition, allocation committees made up of computational science experts should be organized for each of these computers to review the proposals of prospective users and allocate time on them based on scientific merit and appropriateness of the computational methods proposed. This is a more effective way of ascertaining productive use of the computers than adopting charging  - 40 - ANNEX B algorithms for this purpose. It is the opinion of the mission that the use of these facilities should remain free to the end users, as is currently the case. 38. It is also the opinion of the mission that utilization of the networks, including the links to other countries, should be free of charge to the end users. The cost of utilizing these links should be borne by the government. Users requiring the transfer of extremely large data bases which may strongly impact the available bandwidth of the network should be required to submit proposals for such activities which would be judged by rigorous peer-review before this kind of utilization is undertaken. 39. While the supercomputer is now coming into more general use, a series of important policy issues have yet to be settled. The first of these is whether non-CAS users will have the same opportunities on the machine as CAS researchers (who thus far have had the overwhelming majority of the access). This problem may become more severe with the introduction of the satellite telecommunications link which will greatly facilitate remote access to the machine but which will not be available to the university system. It should be noted that even in Beijing, the NCFC is already perceived by one university SKL as belonging to the CAS and as being too expensive to use. 40. The question of who kas access to the machine is closely related to the development of policies and an organizational framework for users, and policies for financing the Center's operations. Concerning the latter, the NCFC has proposed that NSFC (or the SPC) support the Center at the annual rate of about RMB 2 million. In return, it would offer access to users free of charge. The NSFC is more inclined to support supercomputing as a cost included in individual projects, but also believes that an appropriate solution to the operating cost problem would be to have a cost sharing arrangement involving the SSTC, CAS, SEdC, as well as itself. It is not clear when a decision will be made on this matter, but continued delay compromises the achievement of this important objective of the KSDP. 41. It is clear that the impact of NCFC is considerably less than had been expected. Scientists outside of the Beijing region know little about NCFC and those who do know about it do not consider using it. Especially disappointing is the fact that at least some scientists from Beijing University with large scale computational needs do not find the existence of NCFC as suitable to their needs. The experience of NCFC contrasts sharply with Tsinghua's leadership in the development of its center and of CERNet which has been made operational in a fraction of the time taken by NCFC which is not yet fully operational. 42. Inputs/Infrastructure -- Equipment. A total of $116 million, about 88% of the funding from the World Bank, was used for the purchase of scientific equipment, primarily from foreign sources. Most of it was ordered as complete pieces of equipment. The acquired equipment ranged from small instruments costing a few thousand dollars or less to others in excess of $1.8 million. Some of it was state-of-the-art when ordered and continued to be so after arriving and being put into operation. Some of it became  -41- ANNEX B obsolete between the time of selection and delivery. This was the case for most of the workstations. Nevertheless, the impact of this large injection of new equipment on the quality and quantity of the research performed at the laboratories and on the quality of the training of graduate students and the number of graduate students trained was major, as detailed in earlier sections of this report. 43. "Progress in Natural Sciences" is published in Chinese and English by the National Natural Science Foundation of China (NSFC) and reports research findings from the SKLs. Started in 1991 with an initial budget of RMB 400,000, the journal combined two previous publications which reported on the activities of the SKLs. The peer- reviewed articles which appear in the journal cover the spectrum of science done in the SKLs and therefore rivals the Chinese Academy of Sciences (CAS's) own "Science in China" in scope, coverage and, reportedly, importance. However, the mission found that there was relatively limited recognition of the journal among visited labs. The NSFC is continuing to support the publication of the journal with a subsidy of RMB 500,000/ year. The international distribution of the journal has been assisted for the last three years by the British publisher Francis and Taylor, although it is not clear as to how much longer such a relationship will continue. 44. Technical Assistance -- Programme Advisory Group (PAG). The Programme Advisory Group (PAG) was established to support the SPC's role in providing guidance to the Project at the policy level. PAG was especially attentive to the SKLs, the PLMP, and to the NCFC; its agenda also came to include the promotion of the use of TA, making suggestions about the repayment of the loan, technology transfer issues, and helping to speed up China's connection to the Internet. It was active in revieviing the annual indicators of progress for each lab and made site visits to most of the labs during the course of the project. 45. The PAG, as an advisory committee, did not have the executive power and organizational resources to push project implementation as far as it might have gone. A stronger integration of the program components would have added additional value to the overall KSDP. This is especially true with regard to a wider dissemination of PLMP experiences and a broader involvement of ordinary researchers in the PLMP, and to a stronger central role in coordinating solutions to procurement problems and the setting aside of monies for spare parts, and in wiser and more imaginative uses of TA funds. Although it is not clear that PAG was equipped for these roles, the absence of a stronger central coordinating body meant that the promise of the KSDP as an integrated national program, as opposed to an equipment acquisition effort, was not fully met. 46. Domestic and International TA Programs. Domestic and international technical assistance programs were designed to promote collaborative research with forei,gn researchers. Data received from both the SEdC and CAS show that, in most cases, the overall numerical targets were reached. However, implicit in any program to which the World Bank adds its name is an opportunity to reach beyond the original goals and achieve new ones. The foreign specialist program could have been organized into  - 42 - ANNEX B tours of China by world renowned specialists who could have prepared and delivered several keynote lectures at each university or institute. The prestige of the Bank- supported project could have been used to ensure that these were the best possible speakers and that their talks were well prepared, dealt with broad integrative themes in science or laboratory management, and interpreted. IV. Issues Affecting Project Implementation 47. Repayment of the Loan has reportedly been resolved with the government's recognition that the importance of the project investments into the labs and the national significance of the contributions of the labs was such that the government, and to a lesser extent the universities or institutes, rather than the individual labs should bear the repayment burden of the project credit. Therefore, the repayment ratio has been decided by the MOF and the SPC to be 60% by the Ministry of Finance (MOF) with 40% from the CAS, the SEdC or the appropriate line ministry. Of the 40%, the university or institute itself has been asked to assume 20% or a total of 8% of the total repayment burden. 48. Resolution of this issue has been slow in coming and affected the implementation of the project. Especially in the university-based labs, directors were initially hesitant to use the technical assistance portion of project funds, thus depriving their institutions of the benefits of attendance at internal conferences, participation in joint research activities, or inviting foreign specialists to their labs. 49. Equipment procurement, for the most part, went well with scientific and computation instrumentation arriving at the project labs on time and being tested and accepted. However, as with other Bank-supported higher education projects, the Bank's procurement procedures often stood in the way of achieving the maximum scientific impact from the expensive equipment. Chinese procurement procedures necessitated the use of government-approved tendering companies which meant that the laboratory end- user and equipment vendor had little or no chance to interact and provide the best equipment appropriate for the scientific task which the lab was pursuing. Some labs complained that the SEdC and CAS's practice of packaging individual requests for sophisticated instrumentation into groups of similar types of equipment provided some labs with less or more than they had requested or needed. 50. Buying computing equipment using the Bank procurement procedure is especially cumbersome and in the end, expensive. With the fast pace of development of computational equipment and the slow pace of ICB (in this case further slowed by export license issues and the delay in project effectiveness), the computer often arrived two to three years after they were chosen by the lab directors. These procedures should be changed in future projects with higher education components. 51. The mission feels that the decision of the lab directors on the appropriate instrumentation should be paramount and more freedom should be introduced into the  - 43 - ANNEX B equipment procurement process in the support of science projects such as the Key Studies Development Project. Increasing the authority and responsibility of the individual scientists and lab directors was one of the fundamental reform design elements of this project, and it is the mission's view that the equipment procurement methods necessitated by Bark and Chinese government procedures and practices does not encourage this reform in the role of Chinese lab directors. 52. The roles of the CAS, SEdC and SPC were all important in achieving the level of success evident in the final stages of the project. The SEdC (and more particularly its Foreign Investment and Loan Office (FILO) and the CAS (and more particularly its Office of External Financing (OEF), were instrumental in ensuring that the equipment procurement procedures were followed, that technical assistance moved forward, and that the progress of the project was monitored and supervised. The SPC played an initial and vital role in setting the project on its course and then turning over to the SEdC and the CAS the responsibility of implementing the project. 53. SEdC and the CAS could have provided the substantive direction of a traditional Chief Scientist or Principle Investigator of large and complex scientific undertakings in other countries. Equipment procurement and TA administration are critical to the success of such an investment-focused project as this one was, but the mission feels that aspects of the project would have benefited from a scientific and managerial leader who could have worked to resolve such issues as the slow development of the NCFC, the repayment of the: credit by the lab directors, dissemination of the best practices from the PLMP activities or the maximum beneficial use of technical assistance. Without such an entity, project supervision by the SEdC and CAS tended to focus on issues such as equipment procurement for individual labs and not the more programmatic issues which affected all of the labs. Without such an entity as a Chief Scientist the mission feels that the project lost the opportunity to develop some of the broad integrative practices that the PLMP or the 'NCFC could offer in the way of improved lab management or increased computer end-user utilization. 54. Prospects for Project Sustainability. Coming as it did at a time of rapid econDmic development and internationalization in China, the project can be thought of as riding a wave of change which is likely to enhance its sustainability. China has more wealth than it had at the outset of the project, and more resources are available for advanced education and research. The Chinese government has indicated its intention to increase support for education and research during the 9th Five Year Plan, and has launched a series of national programs to support these objectives. This augers well for the project's sustainability. 55. There are, however, a number of issues which must be confronted if the promises of sustainability are to be realized: (a) there is the problem of the future of the SLs. Although some of these will qualify for "211 " program funds, their exclusion from the SKL  - 44 - ANNEX B accreditation review means they will be ineligible in the near term for possible elevation to SKL status and will thus lose the opportunity to secure increased operating funds; (b) the meager faculty salaries and graduate student stipends are a problem for the project's sustainability; (c) although there are to be increases in government support for basic research, the growing importance of contracted applied research and development for the participating labs, and the pressures to transfer technology, hold the potential for distorting the primary mission - that of promoting advanced graduate training in basic science - of the KSDP; and (d) the prospects for sustainability would be enhanced were there more commitment among labs to cooperate and share equipment. (see paragraphs 74-78 of the World Bank/UNDP Mission Report for further details on each point.) V. Lessons Learned and Recommendations: 56. Lessons Learned. The main lesson learned in this project is that with the investment of appropriate resources and careful planning it is indeed possible to accelerate China's scientific and technological development through the training of its graduate students at international levels of proficiency and in increased numbers. In addition, the following new understanding of the process involved includes the following elements: (further details can be found in paragraph 79 of the Mission Report.) (a) It is difficult to transfer managerial experience acquired in other countries to China; (b) The scarcity of equipment has made the community maximize the amount of new equipment purchased, at the expense of acquiring adequate supplies of spare parts and consumable for the proper operation of that equipment; (c) The sharing of major facilities, such as the mini-supercomputer at NCFC, among a large variety of research and teaching institutions, is a new experience for China's S&T community, to which they are not accustomed and which is not facilitated by their strongly compartmentalized organizational structure; (d) The dissemination of managerial know-how within and among laboratories is slow and more efficient means for its acceleration should have been adopted from the very beginning;  - 45 - ANNEX B (e) The PAG might have adopted a more pro-active role, akin to that of a managerial Principal Investigator (PI) for the Project; and, (f) The World Bank ICB procurement procedures are not adequate for projects in higher education and scientific research. 57. Recommendations. As a result of the lessons learned, the mission makes the following recommendations for the near future. 58. Regarding training: (a) The number of students accepted directly into Ph.D. programs should be increased to between 15% and 20% at least at all 211 universities; (b) The ratio of Ph.D. to MSc degrees granted should be further increased; (c) An affirmative action program to substantially increase the fraction of Ph.D. degrees given to women should be adopted; and (d) The number of faculty permitted to advise Ph.D. students should be substantially increased, especially for 21 1 universities. 59. Regarding research: (a) The SKLs should issue yearly reports summarizing their research activities, the number of degrees granted, graduate enrollment, and a comprehensive list of papers published; (b) The SL laboratories should be awarded enough financial support by that University to ensure that the equipment acquired using the World Bank funds be maintained in operational condition and utilized; (c) The sources of funds for SKLs and SLs for fundamental research should be sufficient for them not to become so dependent on short range service work so as to distort their main mission; and (d) The review of competitive project proposals to ministries and other sources of governmental funding should be done according to the same rigorous peer review methodology used by NSFC. 60. Regarding management: (a) A management manual should be produced which incorporates the best practices of the PLMP laboratories and distributed to all SKLs and SLs;  - 46 - ANNEX B (b) An overall administrator with a strong scientific background and of high standing in the scientific community should be appointed for such future projects as the KSDP; and (c) In future equipment acquisitions it should be required for approval that appropriate funding for spare parts, supplies, and other maintenance costs for four years of operation be set aside and not be available for other expenditures until the last year of this period. 61. Regarding the NCFC: (a) A fully operational user committee and under its aegis, an allocations committee should be set up as soon as possible; (b) The utilization of the computer should be cost-free for the end user and the operation costs should be borne by the government in order to ensure the maximum scientific impact of that important facility be achieved; (c) Future investments in this kind of national facility should go to an institution which has demonstrated the greatest competence in managing such a facility and in making it effectively and extensively available to users outside their university or academy; and (d) The use of the national network, including its links to other countries, should be free of charge to the end user. 62. Regarding technical assistance: (a) For the acquisition of major new pieces of equipment, costing in excess of $500,000, experienced and unbiased technical assistance should be required to ascertain that the equipment being acquired is appropriate for the utilization intended; and, (b) Visits by foreign experts in the areas of interest for SKL and SL laboratories should continue to be encouraged and funds appropriate for paying for such visits should be made available. 63. Regarding procurement: (a) The current World Bank ICB procurement procedures have been found to be inappropriate for projects of higher education and scientific training and research and should be modified. Halsey L. Beemer, Jr. June 26, 1997