China - Second Agricultural Education Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 14622 PERFORMANCE AUDIT REPORT CHINA SECOND AGRICULTURAL EDUCATION PROJECT (LOAN 2444-CHA/CREDIT 1500-CHA) JUNE 19, 1995 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Abbreviations BAU Beijing Agricultural University IAP International Advisory Panel IBRD International Bank for Reconstruction and Development ICB international competitive bidding IDA International Development Association MAAF Ministry of Agriculture, Animal Husbandry and Fisheries NEFU Northeast Forestry University O&M operations and maintenance OECD Organization for Economic Cooperation and Development OED Operations Evaluation Department PAR Performance Audit Report PCR Project Completion Report SAEP Second Agricultural Education Project FOR OFFICIAL USE ONLY THE WORLD BANK Washington, D.C. 20433 U.S.A. Offles of Direator-Geneal Operatden Evekuation June 19, 1995 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT Subject: Performance Audit Report on China Second Agricultural Education Project (Ln. 2444-CHA/Cr. 1500-CHA) Attached is the Performance Audit Report on the China-Second Agricultural Education project (Ln. 2444/Cr. 1500-CHA, approved in FY84) prepared by the Operations Evaluation Department (OED). The project was the fourth in a series of Bank-supported operations, which aimed to strengthen and diversify education at the secondary and post-secondary levels. It supported and built upon the experience of the Agricultural Education and Research project (Cr. 1297-CHA) which ran concurrently. The project was implemented largely as designed and met most of its main objectives. Training was expanded in 23 agricultural colleges; eight extension support centers were established; 12 agricultural technical schools were strengthened; two model facilities-an agricultural library and a food science department-were built and equipped; and distance education production facilities were improved. But the high cost of operating and maintaining sophisticated buildings and equipment procured under the project limited their use and put their sustainability at risk. In staff development and training, performance was also weak. Large numbers of students sent abroad for higher education failed to return and the management of education and research was not much improved. Project preparation failed to anticipate the strongly entrenched attitudes of educators who managed their institutions according to traditional models, which segregated education from research. Failure to establish the International Advisory Panel, agreed at appraisal, and weak supervision during the early years of implementation, allowed weaknesses in preparation to go uncorrected. In consequence, while most of the key educational performance indicators set at appraisal have been met, beneficial institutional development has been limited. Weak current funding for higher education may prevent this situation from being corrected in other than a few selected institutions. Current Bank support for doctoral and post-doctoral training in China through the Key Studies Development project (Cr. 2210-CHA) has, however, taken on board the experience of Second Agricultural Research and other earlier projects. The audit confirmed the ratings given to the project when the Project Completion Report was reviewed. Outcome is rated as satisfactory, with modest institutional development and uncertain sustainability. The findings of the audit will contribute to OED's forthcoming study of agricultural lending to China. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  FOR OFFICIAL USE ONLY Contents Preface ............. 3 Basic Data Sheet................................................. 5 Evaluation Summary ................................................ 7 1. Background .................................................. 11 2. The Project .................................................. 13 3. Implementation Experience ...................................... 17 4. Project Outcomes ............................................. 25 5. Findings and Issues ............................................ 29 Annex A: Supporting Tables ........................................ 35 Annex B: Borrower's Response ...................................... 41 Tables in Text Table 2.1: Summary of Project Costs .................................. 15 Table 2.2: Summary of Bank-financed Project Components ................... 15 Table 3.1: Compliance with Project-specific Covenants ...................... 23 Table 4.1: Return Rate of Overseas Fellows .............................. 27 Table 5.1: A Comparison of Performance Indicators in Five Early Bank-Supported Projects in China ........................... 30 Boxes in Text Box 2.1: Highlighted Lessons of Agricultural Research and Education (Credit 1297-CHA)................................. 14 Figures in Text Figure 3.1: SAR Estimate Disbursements ................................ ... 17 Figure 3.2: Actual Disbursements ........................................ 17 Figure 3.3: Time Spent on Preparation and Supervision, days in field .......... .. 22 Tables in Annex Table Al: Bank Support to China for Education and Human Resource Development . 35 Table A2: Project Implementation - Appraisal Estimates and Actual Completion ..... 36 Table A3: Project Results - Appraisal Estimates and Actual Completion .......... 37 Table A4: Project Financing Appraisal Estimates and Actual Costs .............. 39 Table A5: Project Costs - Appraisal Estimate and Actual (US$m) ............... 40 This report was prepared by Christopher Gibbs (Task Manager), who audited the project in June 1994. Carla Sarmiento and Elaine Wylie provided administrative support. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  3 Preface Higher education was adopted as the first target for Bank lending when the People's Republic of China claimed China's seat at the World Bank in 1980. The Second Agricultural Education Development Project was the fourth such project to be approved. Loan 2444-CHA in the amount of US$45.3 million and Credit 1500-CHA in the amount of SDR 22.0 million (US$23.5 million equivalent), were approved on June 14, 1984. The credit was fully disbursed by February 5, 1988 while the loan closed on June 30, 1992, one year behind schedule, and was fully disbursed by August 20, 1992. This Performance Audit Report is based on the Project Completion Report, Staff Appraisal Report, loan documents, project files, discussions with relevant Bank staff, and an audit mission to China in May-June 1994. The audit mission held discussions with the Ministry of Agriculture in Beijing and visited five institutions which benefitted under the project in Beijing, Harbin, Chengdu, and Guangzhou. Their full and kind cooperation is gratefully acknowledged. The draft of this report was sent to the Borrower for comments. The response is incorporated as Annex B.  5 Basic Data Sheet SECOND AGRICULTURAL EDUCATION PROJECT (LOAN 2444-CHA/CREDIT 1500-CHA) Key Project Data (in US$ million) Appraisal Actual Actual as % of Estimate Appraisal Estimate Project Costs 175.0 126.1 72.0% Loan/Credit Amount 68.8 70.5 103.7% Disbursed --- 70.5 Canceled --- --- Economic Rate of Return Institutional Development Modest Cumulative Loan/Credit Disbursements (in US$ millions) FY85 FY86 FY87 FY88 FY89 FY90 FY91 FY92 FY93 Appraisal 0.7 7.0 17.0 41.0 62.0 68.8 Actual 1.5 12.0 24.4 25.2 57.2 63.4 65.5 68.8 70.5 Actual as of % 214.3 171.4 143.5 61.5 92.2 92.1 95.2 100.0 102.5 Estimate Final Disbursement August 20, 1992 Project Timetable Item Date Planned Date Revised Date Actual Identification 1/83 1/83 Preparation 6/83 Appraisal 7/83 7/83 Loan/Credit Negotiations 5/84 Board Approval 6/14/84 Credit Signature 7/9/84 Credit Effectiveness 10/17/84 Credit Closing 6/30/91 12/31/91 6/30/92 6/30/92 Credit Completion 12/89 6/30/92 6/30/92 6 Staff Inputs (in staff weeks) FY83 FY84 FY85 FY86 FY87 FY88 FY89 FY90 FY91 FY92 FY93 Total Preappraisal 14.6 14.6 Appraisal 2.3 60.3 62.6 Negotiations 9.8 9.8 Supervision 0.6 12.2 8.5 3.6 7.1 10.9 9.9 3.4 4.5 7.4 68.1 Other 0.6 10.2 10.8 TOTAL 17.5 80.9 12.2 8.5 3.6 7.1 10.9 9.9 3.4 4.5 7.4 165.9 Mission Data Project Cycle Month/ No. of Days in Specialization Performance Types of Year Persons Field Representeaf Rating Problems' StatuSb. Identification 1/83 3 5 AEdu, Eco Appraisal 7/83 5 20 Edu, Eco, AEdu, EduP Postappraisal 10/83 3 15 Edu, AEdu, Arch Supervision 1 9/84 3 21 Edu, Eco I M Supervision 2 2/85 2 15 Eco, Edu I M Supervision 3 10/85 2 15 Eco, Arch I M Supervision 4 6/86 1 7 Eco I M Supervision 5 11/86 1 5 TS n.a Supervision 6 10/87 1 15 TS 1 M Supervision 7 11/87 1 15 Arch n.a Supervision 8 5/88 2 12 TS, Agr n.a Supervision 9 11/88 2 15 TS, Agr I M Supervision 10 10/89 2 12 TS, Arch I M Supervision 11 5/90 1 8 TS I M Supervision 12 3/91 1 8 TS 1 M Supervision 13 11/91 1 14 TS 2 M Supervision 14 6/92 1 21 TS I M a. Specialization: AEdu = Agricultural Educator, Agr = Agriculturalist, Arch = Architect, Eco = Economist, Edu = Educator, EduP = Educational Planner, TS = Training Specialist. b. Performance Status: I = Minor or no problem, 2 = Moderate problems, 3 = Major problems. c. Types of Problems: M = Management 7 Evaluation Summary Introduction 1. The People's Republic of China (PRC) claimed China's seat at the World Bank in 1980 and Bank support for higher education was quickly identified as a priority for devel- opment. China's educational system had been badly disrupted by the Cultural Revolution between 1966 and 1976. Higher education had suffered acutely and there was need for both restoration of educational facilities and resources and forward progress. The first Bank- supported project in China for University Development (Credit 1167-CHA/Ln. 2021-CHA) was prepared quickly and approved in 1981. This was followed by a steady stream of II projects for higher education, training and human resource development, approved between 1982 and 1994, with Bank funding of more than US$1.3 billion (listed in Annex A). Table A.1, the Second Agricultural Education Project (SAEP) (Cr. 1500-CHA/Ln. 2444-CHA) was the fourth of these projects. 2. SAEP followed the Agricultural Education and Research Project (Cr. 1297-CHA). This project had been approved in 1982 to provide assistance to colleges in a wide range of agroclimatic zones. SAEP placed greater emphasis than its predecessor on agricultural educa- tion and technical training in colleges as regional resources. In order to focus SAEP on education, further Bank support for agricultural research was channeled through a separate Second Agricultural Research Project (Credit 1516-CHA). SAEP was approved in June 1984 with an International Development Association (IDA) Credit of US$23.5 million equivalent and an International Bank for Reconstruction and Development (IBRD) loan of US$45.3 million. Objectives 3. The objectives of SAEP were to assist Government upgrade and expand professional and middle-level agricultural manpower (sic) by providing assistance to colleges, training and support centers, technical schools, and distance learning centers. Funds were provided for civil works, furniture and equipment, technical assistance, and overseas fellowships in four components: expansion of training in 23 agricultural colleges (including II colleges already supported under Credit 1297-CH4A); establishment of eight extension support centers; support for 12 agricultural technical schools; development of two model facilities-a library at Beijing Agricultural University and a food science department at Central China Agricultural University, Wuhan; and improvement of instructional materials for distance learning. Implementation Experience 4. Quality of the project at entry was mixed. Much of the strength that existed at entry stemmed from the careful preparation of the Agricultural Education and Research Project (Credit 1297-CHA), which preceded SAEP by two years. Weaknesses stemmed from the relatively rapid pace of project preparation and the speed with which it followed comparable, earlier projects. Bank efforts to slow down the pace of preparation paid dividends but these were insufficient. 8 5. SAEP was implemented largely as designed, but with a number of important changes. Notable early success was achieved with equipment procurement and placement of overseas fellows. Early preparation for international competitive bidding (ICB) took place with the result that bids for US$36.0 million were agreed by the Bank within one month of project approval. Overseas fellows for Master's and PhD degrees, and candidates for nondegree studies were also identified and sent abroad quickly. The quality of the candidates for degree studies was so high that they were frequently able to earn scholarships and financial support in their overseas institutions. This relieved the project of financial responsibility for their support and freed up funds for additional fellows to use, raising the total number of students sent to earn graduate degrees from 144 estimated at appraisal to 317. Educational performance of students, faculty members, and overseas fellows was enhanced by extensive use of the language training facilities supported by SAEP. 6. Overall strengths in implementation included good preparation of project performance indicators, which were well chosen and routinely monitored throughout implementation. The Project Office learned complex Bank procedures quickly and applied them competently. ICB generally produced a wide response and costs 10-15 percent below original estimates. Most equipment procurement was efficiently managed. Micro-computers, performed poorly because of erratic power supplies, unreliable air conditioning and back-up power, and sporadic use. A second major procurement effort was made toward the end of project implementation. Uncommitted funds were reallocated to procure additional equipment, spare parts and accessories. Maintenance contracts were purchased with project funds to help ensure adequate support for equipment already procured. 7. Technical assistance was inadequately prepared, with little explanation of its rationale or means of implementation. Details were left to be developed after approval by the project's International Advisory Panel (IAP). An IAP was, apparently, working effectively at the time in the University Development Project (Cr. 1167-CHA/Ln. 2021 -CHA), but the Borrower was not wholly committed to the replication of this concept under SAEP and its IAP was never constituted. Borrower commitment to technical assistance was also limited because of its high cost and the scarcity of specialists who had proven experience of China. 8. Detailed designs of the model facilities were unavailable at project approval, contrary to Bank practice that requires project preparation be sufficiently advanced that procurement and other important phases of implementation should begin shortly after loan approval. Disagreements over design issues among Borrower institutions lasted for several years, and were not resolved until year four of implementation. The Bank provided architectural specialist support through supervision missions but the resulting facilities were compromises, not the models the Bank had hoped for. Bank supervision during the first two years of implementation was limited to Beijing and Wuhan. Supervision coverage improved in year three, but most project institutions were never directly supervised during implementation. Results 9. SAEP has contributed to institutional strengthening in the colleges, schools and centers it supported. Most of the key educational performance indicators agreed at appraisal have been achieved. For example, numbers of students, facility development, student: teacher 9 ratios, variety of course offerings and areas of specialization, numbers of hands-on experiments conducted by students, use of audio-visual equipment, library opening hours, frequency of open library stacks, number of class periods per day-all have improved significantly. Overall, the project outcome is considered to be satisfactory with modest institutional development. 10. Institutional development is rated as modest for two reasons. First, the project increased the number and sophistication of educational facilities and equipment, including computers, scientific laboratory equipment, library equipment, and studio production equipment for distance learning through radio and television. But, electronic equipment is likely to be obsolete in 3 to 6 years, and other types of equipment (electrical, thermal, mechanical, or optical) to be obsolete in 6 to 12 years. Unfortunately, most project institutions do not know where the next generation of equipment will come from. Furthermore, (O&M) funds have been so limited that much of the equipment procured has been underutilized or become unserviceable because funds for repairs and spare parts were unavailable. 11. Second, nearly half of the overseas fellows supported by SAEP have not yet returned to China, and it is apparent that many of them will never return. This loss of outstanding talent has affected project implementation by delaying realization of the expected benefits returning fellows would have brought. It will also reduce the long-term impact of SAEP by slowing growth in faculty quality and reducing Government's willingness to sponsor long-term degree studies overseas in the future. Government has taken a pragmatic approach to attract overseas fellows home with positive incentives-personal and professional-and freedom of travel, but the impact has so far been modest. 12. Technical assistance was successful only in part. As indicated in paragraph 6, preparation was inadequate and Borrower commitment was weak. But, under the circumstances, the Government was right to be cautious. Too few technical specialists had proven experience of China and the services of those who were in demand could be bought using bilateral grant funds. Two important covenants, concerning establishment of an International Advisory Panel and use of tracer studies for project-supported graduates, were not complied with. Sustainability 13. Project sustainability is considered to be uncertain. Scarcity of new capital funds for replacement equipment and of recurrent funds for equipment O&M puts the sustainability of the project-supported equipment and facilities at risk. SAEP institutions must struggle creatively to find new sources of funding if they are to have the equipment necessary successfully to climb the technological ladder that faces educators in the agricultural sciences. Funding scarcity is compounded by the nonreturn of overseas fellows, who would have utilized SAEP's sophisticated equipment more fully, had they returned. Findings and Issues 14. Inadequacy of recurrent funding which reduced SAEP's institutional development and lowers its prospects of sustainability, is a common finding in all of the first four Bank- 10 supported projects for higher education as well as SAEP. Borrower assurances that adequate recurrent funding would be provided, proved inadequate. Bank support for capital investments continued, however, when domestic supplies of operating funds were not sufficient to operate and maintain the capital already in place. This experience suggests that new mechanisms are needed to ensure the adequacy of O&M funds, that the actual provision of O&M support needs to be carefully monitored by the Bank, and that covenants be required to safeguard these requirements. 15. The nonreturn of overseas fellows funded by SAEP poses substantial costs on the Borrower: loss of investment in higher education and outright loss of valuable human capital if fellows remain permanently outside China. Low return rates for overseas fellows, however, are associated with all four early Bank-financed projects that supported overseas fellowships for university development and agricultural education and research. China's policy of encouraging overseas fellows to find alternative sources of finance in host institutions and to extend the duration of their studies backfired. Government's reluctance to support long-term studies overseas limits development of post-graduate technical and professional staff and the acquisition of skills and techniques developed outside China. Short-term, nondegree studies are only a partial substitute. The Bank learned these lessons slowly, but they are reflected in the Key Studies Development Project (Cr. 2210-CHA), approved in 1991, which reinforces China's capacity for PhD and post-doctoral training. 16. Technical assistance under SAEP was poorly prepared and received inadequate commitment. Government's reluctance to use borrowed funds for technical assistance was evident in four of the first five Bank-supported projects for higher agricultural education and research, and the majority of completed agriculture projects in China: preparation of technical assistance in Bank-supported agriculture projects had become a formality. Experience gained under SAEP suggests that China and the Bank must reconsider the customary approach to justification and provision of technical assistance where short- or long-term expatriate advisors are injected into a project, at the lender's urging and with the Borrower's concurrence, but without confidence in the likely outcome. 17. Overall, this audit reconfirms the findings of the PCR. The audit was undertaken as a contribution to OED's forthcoming study of Bank support for agriculture in China. 11 1. Background 1.1 The People's Republic of China (PRC) claimed China's seat at the World Bank in 1980 and the first Bank Group support for China was approved in 1981 for University Development (Credit 1167-CHA/Ln. 2021-CHA). Following that, 11 projects aimed at higher education and human resource development were approved between 1982 and 1994, providing combined support approaching US$1.2 billion. The Second Agricultural Education Project (Credit 1500-CHA and Loan 2444-CHA) approved in 1984 was the fourth of these projects (see Table A.1, Annex A). 1.2 Following the revolution in 1949, China's agricultural development was impressive. Grain yields doubled between 1952 and 1978, and China became able to feed 22 percent of the world's population from less than 11 percent of the world's arable land. This was achieved by intensification of agriculture that employed high-yielding varieties of wheat and rice, chemical fertilizers and mechanization, irrigation, and organized rural populations. Higher education supported agricultural development but suffered serious setbacks during the Great Leap Forward (1958-60) and the Cultural Revolution (1966-76) when the balance between basic education, that emphasized political education, and advanced training was abandoned. 1.3 China's educational system was totally disrupted during the Cultural Revolution. Higher education suffered the most when colleges and universities were closed, equipment and books were destroyed, and vocational, technical and post-graduate education were largely abolished. But new development strategies and reforms began to be adopted after 1977, starting in the rural sector. Lower level units were given more freedom to exploit local comparative advantage, decide production strategies, seek out efficiencies and consumer preferences, and respond to market signals. The importance of technical and intellectual expertise resurfaced. 1.4 By 1979, higher education was growing again but the enrollment rate of 1.2 percent was only a quarter of the average for all developing countries. Scientific equipment, books and journals were in short supply: teachers were numerous but underqualified, out of touch with advances made outside China. Even in key universities, which were designated as centers of excellence, only 8 percent of the staff had completed postgraduate studies, and 16 percent had three years or less of higher education. Curricula were unbalanced, in favor of science and engineering, and costs per student were high-equivalent to US$1,000 per year-because of very low student/teacher ratios and the high proportion of students who were boarders. 1.5 In 1980, in the Bank's earliest formal policy dialogue with China, both parties agreed that Bank support should help rebuild China's higher education system and promote agricultural development. A three-tier system supported agricultural education: universities and colleges at the top, agricultural secondary schools in the center, and agricultural curricula in middle schools at the base. The first Bank-supported project in China was for University Development (Credit 1167-CHA/Loan 2021-CHA)' as a whole, in the amount of US$200.0 1. OED Report Number 7561, December 1988. 12 million. This project was formulated quickly, in under a year, following the PRC's membership of the Bank and approved in June 1981. Agricultural Education and Research (Credit 1297-CHA)2 was approved the following year, and aimed at selected universities and colleges in the top level of agricultural education. The Second Agricultural Education Project (Credit 1500-CHA), which followed quickly at China's urging, focused on education alone and extended Bank support to the level of agricultural secondary schools. 1.6 In 1981, China's State Agricultural Commission established a Project Leadership Team, a Project Committee, and a Specialist Expert Committee to formulate the Agricultural Education and Research Project Credit 1297-CHA). Numerous individuals were involved in its formulation, including six Bank staff in preparation, nine in appraisal, and five in post- appraisal. The project aimed to improve teaching and research quality; expand enrollment and research capacity; and improve educational organization and management. Eighteen top-flight educational and research institutions were to be assisted. But before the project was approved in November 1982, a Bank mission on education and manpower planning recognized the opportunity to broaden further Bank support for agricultural education, and preparation of the Second Agricultural Education Project was initiated. Continued support for agricultural research was to be provided by the Second Agricultural Research Project (Credit 1516-CHA), for which preparation began at the same time. 2. OED Report Number 9708, June 1991. 13 2. The Project 2.1 The objectives of the Second Agricultural Education Project (SAEP) were to expand and upgrade professional and middle-level agricultural manpower (sic) by providing assistance to four types of institutions: colleges, training and extension support centers, technical schools, and distance learning support facilities. SAEP extended Agricultural Education and Research (Credit 1297-CHA) by providing assistance to colleges in a wider range of agroclimatic zones and emphasizing undergraduate education and technician training over postgraduate education and research. 2.2 SAEP was identified in January 1983, only three months after the Agricultural Education and Research (Credit 1297-CHA) was approved and one month before that credit became effective. The project was prepared in June 1983, pre-appraised in July 1983, appraised in January 1984, approved in June 1984, and became effective in October 1984. When SAEP became effective, disbursements under Credit 1297-CHA still totalled less than US$700,000. SAEP followed hard on the heals of its forerunner. The speed and timing of preparation of SAEP allowed little time for the experience of its forerunner to accumulate or be assimilated. Highlights of lessons learned in Credit 1297-CHA were first summarized in the Project Completion Report (PCR), prepared in 1988, and were not available to the designers of SAEP. They are repeated here in Box 2.1 in order to highlight issues which also became apparent in SAEP, especially points 5, 6 and 8. 2.3 A summary of SAEP's targets and achievements is provided below and in Annex Tables A2 and A3, and a summary of project costs in Annex Tables A4 and A5. SAEP supported expansion and improvement of pre-and in-service training for 23 key agricultural schools, including 11 schools assisted through Credit 1297-CHA, and model library and food science facilities; establishment of eight training and extension support centers to train staff for the agricultural extension system; assistance for a pilot program to strengthen agricultural technical schools; expansion and improvement of the supply of instructional materials used by three distance learning centers; and support for planning and evaluation. Appraised and actual project costs are summarized below in Table 2.1. 2.4 Bank support was for equipment, civil works, specialist services, overseas fellowships, and books and instructional materials is summarized in Table 2.2.' 2.5 Except for construction of a library at Beijing Agricultural University (BAU) and a food science department at Central China Agricultural University in Wuhan, all project civil works were financed and executed by the Government. The two exceptions were agreed on the grounds that they were to be "model" facilities, i.e. new and complex facilities, likely to be replicated elsewhere in China. Bank participation in their development would promote technology transfer and provide an opportunity to study China's approach to bidding and contracting procedures. Equipment procured under the project included computers, scientific equipment, and language laboratory equipment. Overseas fellowships were for Masters and PhD degrees, advanced nondegree studies and short-term courses. Local and foreign technical 3. For convenience, the IBRD and IDA are jointly referred to as "the Bank" throughout this audit. 14 assistance was to be provided to previously and newly assisted colleges, for development of the model facilities, extension centers, language training, technical schools, distance learning, and evaluation, planning and new project preparation. Box 2.1: Highlighted Lessons of Agricultural Research and Education (Credit 1297-CHA) 1. The agricultural education subsector was carefully studied prior to project preparation. 2. Agricultural research and education were successfully combined. 3. Realistic and attainable indicators of progress and achievement were set. 4. Successful language training was critical to overall project performance. 5. Specialist technical assistance did not receive adequate oversight from IDA or the Borrower. 6. Delayed return of fellows from overseas training reduced project benefits. 7. The results of manpower and research studies require dissemination. 8. Overly rapid education system expansion can lead to inattention to recurrent cost provision. 9. Project O&M systems need to be institutionalized. 10. Opportunities to replace elements of the command economy with market-oriented approaches should be seized. 11. Opportunities for inter-institutional linkages were missed. 12. Educational institutions should be treated as part of the system they serve, not separately. 13. Institutions of higher education need to find mechanisms for self-financing. 2.6 Evaluation would use standardized instruments to monitor progress in implementation by project units, reporting ultimately to an International Advisory Panel. Tracer studies would be used to monitor progress in the careers of agricultural professionals and technicians supported by SAEP. Finally, the project would support preparation of new investments in agricultural education, in particular for agricultural technical schools, training of extension personnel, and improvement of teachers in agricultural schools. 15 Table 2.1: Summary of Project Costs Summary of Project Components Appraised Cost Actual Cost US$m US$m Agricultural Colleges and Model Facilities 121.7 (69.5%) 101.0 (80.1%) Training and Extension Support 8.2 (4.7%) 9.7 (7.7%) Agricultural Secondary Schools 9.8 (5.6%) 9.8 (7.8%) Distance Learning and Support Facilities 2.5 (1.5%) 4.9 (3.9%) Planning, Evaluation and Future Project Preparation 0.3 (0.2%) 0.7 (0.6%) Contingencies 30.8 (17.6%) --- --- Total 175.0 (100.0%) 126.1 (100.0%) Table 2.2: Summary of Bank-financed Project Components. Summary of IBRD-and IDA-financed Components Appraised Cost Actual Cost US$m US$m Equipment 46.0 (66.9%) 52.7 (74.8%) Civil Works 1.0 (1.5%0 4.3 (6.0%) Specialist services 5.5 (8.0%) 3.0 (4.3%) Overseas Fellowships 9.4 (13.7%) 9.5 (13.5%) Books and Instructional Materials 1.4 (2.0%) 0.3 (0.4%) Fees, deposits and unallocated costs 4.6 (6.7%) 0.7 (1.0%) Total 68.8 (100.0%) 70.5 (100.0%) 2.7 Project preparation was led by a Project Office in the Former Ministry of Agriculture, Animal Husbandry and Fisheries (MAAF). Problem analysis during project formulation, however, was less thorough than for Credit 1297-CHA, where very detailed preparation was undertaken. SAEP was relevant to China's educational policies at the time and consistent with the Bank's strategy for agricultural education. But as a successor operation to Credit 1297- CHA, fewer resources were invested in project justification, despite the fact that it broke new ground by emphasizing undergraduate education and technical training over postgraduate education and research. 2.8 The project responded to the needs and circumstances of benefitting institutions. Each institution requesting assistance was required to prepare a development plan and these were screened in light of agreements reached during the Bank's identification and preparation missions. A Review Committee comprising members of project institutions was established to provide technical expertise to the Project Office. This approach sought to promote a sense of ownership among the participating institutions. 16 2.9 Two related pressures became evident during project formulation. First, pressure was apparent from the participating colleges to shift the intended focus of the project from teaching improvement to research. The leaders of key universities argued that SAEP should strengthen their rehabilitation as leading research institutions, despite prior agreement that teaching provided this project's main focus. Second, Borrower emphasis was placed on the purchase of sophisticated equipment. Technical assistance and project software was de-emphasized. When faced with a choice between a piece of expensive laboratory hardware and a piece of expensive technical assistance, Borrower institutions preferred the equipment. Preliminary equipment lists also identified items which would further the universities' desire to promote their research rather than their teaching function. 2.10 The Bank responded in three ways. First, it reconfirmed that support for agricultural research was to be provided by the Second Agricultural Research Project (Credit 1516-CHA), which was in preparation: support for agricultural research and education was to be separated.' Second, by slowing down the process of project preparation to allow Bank pre- appraisal, appraisal and post-appraisal missions to provide additional opportunities for advice and technical assistance. Bank management's concern for lending and staffing coefficients were not to be allowed to push the project too fast.' Third, the Bank made a specific requirement for a post-appraisal mission to re-evaluate all requests for advanced equipment to ensure that they contributed to the universities' education and teaching functions. 2.11 Bank management expressed a number of concerns during preparation. First, that the implementation experience of the first agricultural education project was so limited, especially since SAEP was proposing additional funding for 11 colleges already supported by the first project. Second, that discussion of China's agricultural education budget was minimal, and the project would raise substantially the level of recurrent costs. Would future budgets be able to sustain the investments made under SAEP, and their replacement when it was due? Third, how would design and quality of construction funded mainly by the Chinese be assured? Fourth, how will composition of equipment lists, fellowship nominations and technical assistance be judged? And fifth, how will teaching and learning be measured. 2.12 The responses of staff reported in project files indicate that additional funding for the II colleges supported under Credit 1297-CHA was required if they were to become centers of excellence in teaching as well as research. China's commitment to future recurrent funding and to quality control in building design and construction were to be discussed at negotiations. Equipment lists were to be carefully reevaluated by the proposed post-appraisal mission, and an international advisory panel (lAP) would assist the project office to develop evaluative instruments to measure the project's qualitative objectives. However, implementation experience demonstrated that management's concerns about the absence of lessons of experience and the availability of adequate recurrent finance were justified. 4. Bank support for Second Agricultural Research Project (Credit 1516-CHA) was approved on September I, 1984, approximately three months after approval of Second Agricultural Education (Credit 1500-CHA/Loan 2444-CHA). 5. World Bank Office Memorandum, August 3, 1983. 17 3. Implementation Experience 3.1 The project was implemented largely as designed, but implementation of most components was slower than expected at appraisal, with one exception-equipment procurement. Because detailed preparation for procurement paralleled project preparation, SAEP was able to commit and disburse funds quickly. Three-quarters of Bank financial support (IBRD and IDA) was for procurement of laboratory, computer and library equipment, i.e. US$52.7 million out of a total of US$70.5 million. International competitive bidding (ICB) preparations for the first US$36.0 million of equipment acquisition were completed in May 1984 with assistance from a Bank procurement mission. Bidding opened in June 1984 and closed that September. As a consequence, the Bank was able to gives its "no objection" to the bids in November 1984, accounting for 50 percent of Bank support for SAEP within one month of loan effectiveness. The unusual pattern of disbursements is shown clearly in Figure 3.1. Figure 3.1: SAR Estimate Disbursements Figure 3.2: Actual Disbursements 40 40 35 35 30 30 25 25 20 20 is ) 15K,1 10 10 0 JZIL E:_________ 85 BB 87 88 89 90 85 B6 87 B6 89 90 91 92 93 Fiscal Year Fiscal Year 3.2 Actual disbursement of SAEP funds followed quickly. More than 54 percent of the Bank's support for SAEP was disbursed within 20 months of loan effectiveness. Upstream inputs in bid preparation and use of consultants for bid evaluation paid early dividends. Institutions benefitting from equipment procurement expressed great satisfaction with the equipment they received on its arrival. But lack of spare parts and consumables for procured equipment was noted within three years of loan effectiveness. Institutions complained to Bank supervision missions of lack of operating funds and Bank staff observed unqualified technicians using (and being unable to use) the new equipment, and of poor facilities to house it.6 3.3 Microcomputers performed poorly early on. Within four years of approval, project files record the first complaints of difficulties with microcomputers which were "often out of 6. Bank Supervision Report, October 23, 1987. 18 order".' Only two bids were made to supply mainframe computers under SAEP, one of which was strongly preferred by the Borrower. This preference was reflected in a complex procedure adopted by the Borrower to evaluate the bids.' The Bank considered this evaluation procedure defensible but not to be repeated. The computers chosen, however, proved highly sensitive to the environment and workload, and did not perform reliably under Chinese conditions, where power and airconditioning were unreliable, backup power supplies were too costly to maintain, and workload was erratic. Formal requests followed from the Borrower to purchase maintenance contracts from the supplier with funds from the loan/credit. The Bank had no objection to this. But it was apparent early that domestic funds for recurrent costs for operation and maintenance (O&M) of procured equipment were insufficient. In particular, hard currency for purchase of spare parts and supplier services beyond the warranty period was extremely scarce.' 3.4 The Bank-supported University Development Project (Credit 1167-CHA/Loan 2021- CHA), recognized a guideline that annual equipment repair and maintenance costs in OECD countries averaged 10 percent of capital costs. Allowing for differences in personnel costs, the Bank agreed that the guideline might be slightly lower for China, unless foreign personnel were used.'o A Chinese study prepared by the Ministry of Education for eight of China's top universities reached the same conclusion. The MAAF, however, estimated that agricultural universities needed about half the guideline amount, or 4.7 percent of capital costs for O&M. Actual allocations, however, for all agricultural university O&M were less than 1 percent." 3.5 Because O&M funds were limited, institutions benefitting from SAEP quickly became creative in tackling repair and service problems.12 Some institutions established routine tracking systems to monitor equipment use and repair needs. Others set up dedicated service departments to ensure that scheduled maintenance was undertaken using in-house resources to the greatest possible extent. In some cases these departments became very skilled in repairing equipment by making replacement parts they could not afford to buy, or buy cannibalizing broken equipment for useful parts to keep other equipment running. Some laboratory equipment was used only sparingly because the most likely users were among the overseas fellows, and they were out of the country. 7. Bank Supervision Report, September 28, 1988. 8. World Bank Office Memorandum, November 16, 1984. 9. The annual cost of a maintenance contract for an US$850,000 microcomputer was US$35,000, in hard currency. In 1988, this was equivalent to more than Y180,000. For several of the SAEP universities visited by the audit mission, the total O&M budget was less than this. 10. The guideline was based on a 1963 OECD study. 11. Northeast Forestry University (NEFU) in Harbin, Heilongjiang provides a example of O&M funding restrictions. While NEFU received US$3.047 m in Bank support for equipment procurement, its 1994 O&M budget for the university as a whole was Y20,000 (or about US$2,300). 12. The example of a repair to an imported spectrophotometer estimated to cost YI0,000 by the manufacturer's agent, that was completed locally at the Northeast Forestry University at a cost of Y150 is far from isolated. 19 3.6 The project's international advisory panel (IAP) was never constituted. An IAP, assisted by an Executive Director and a secretariat located in Washington had been a part of project management for University Development (Credit l167-CHA/Loan 2021-CHA) and three other higher education projects in China. But the cost-effectiveness of IAPs was questioned by Government and its support for them fell from one project to the next, and the decision to establish one for SAEP was consistently deferred. 3.7 The loan/credit for SAEP stipulated that the IAP be formed by December 31, 1984, with terms of reference acceptable to the Bank, but this did not take place. Project files indicate that by October 1984 there were neither preparations nor a plan for the IAP. After that date, the files are silent until July 1986, when they indicate that a technical assistance contract was being negotiated with a US-based company which had assisted in recruitment under Agricultural Education and Research (Credit 1297-CHA). The implication was that this arrangement, not discussed in the SAR, was an alternative to the IAP. But it is unclear what functions the IAP was intended to perform since terms of reference were not discussed or agreed, as required. 3.8 Project files do not indicate clearly when the change from an IAP to a firm-based project advisory system took place, but by September 1986 the company selected to lead this activity had prepared five study tours. These addressed library modernization and management, computer centers, agricultural mechanization, processing and storage of fruits and vegetables, and agricultural electrification, architecture and ecology. All the study tours offered were to institutions in the United States and Canada, a very narrow interpretation of "international." 3.9 Application of technical assistance planned under SAEP was limited and slow. After two years of implementation only 30 months of foreign specialist input had been used out of a proposed total of 740 person months. As a result, the budget allocated to technical assistance was eventually revised to allow purchases of equipment and books, and civil works to increase. The Bank formally agreed that technical assistance requirements had been overestimated at appraisal and acknowledged that Government was able to recruit foreign specialists at a low cost, and pay them in local currency. Foreign interest in China was so high at the time that specialists were willing to travel to China to provide expertise, provided their travel costs were covered. Nevertheless, project files indicate that Government sought actively to minimize expenditures on foreign specialists. Only in the field of language training was use of foreign specialists higher than expected at appraisal. 3.10 Leadership in language training was provided through contracts with a specialized, foreign-based company which had established two language training centers under the first Agricultural Education and Research project to train Chinese language teachers in China. Trained language teachers returned to their campuses to run language laboratories constructed with SAEP funds. The original contracts for foreign language specialists expired in May and October 1986. Project files indicate that concern for continuity was raised in October 1985 and restated in July 1986. Despite their importance, these contracts were not renewed without interruptions as the Borrower sought ways to maintain the language program and reduce its hard currency cost. But foreign language consultants were eventually seen to be indispensable 20 and 78 percent of the 665 person months of SAEP's total foreign specialist time was applied to language training, a sharp increase over the 160 person months indicated at appraisal. 3.11 Many more overseas fellowships were approved under SAEP than expected at appraisal. Nearly three times as many PhD candidates (56, rather than 19), and more than twice as many Masters candidates (261, rather than 125), were sent overseas. The number of candidates for advanced study and short-term fellowships was close to the planned target (459, rather than 473). This expansion was made possible without expanding the costs of this component by the success of the overseas fellows at finding new, non-project sources of financial support after they arrived at their destinations. Universities and colleges which hosted the Chinese students found domestic sources of funding for them, relieving SAEP of the full financial responsibility. It was China's policy at the time to encourage these moves. The Chinese students chosen for foreign fellowships were of such high calibre, and the host institutions so satisfied with their performance, that they were able successfully to find new sources of financial support. But this apparent success had high costs. 3.12 Chinese students on overseas fellowships not only found new sources of funding, they were able to prolong their stays outside China considerably. In October 1987, project files record the Bank's concern that overseas fellows were staying away much longer than planned by advancing from Master's degree programs to PhD programs. Periods away from China grew typically from two years to five years. This proved disadvantageous in three ways: first, the institutions in China to which the overseas fellows were due to return had to wait, delaying the initiation of new course offerings and other contributions the overseas fellows were expected to provide; second, the scientific equipment which had been procured quickly under SAEP in the expectation that the overseas fellows would return as primary users, remained underutilized or idle; and third, and most critically, when the events of Tianamen Square took place in June 1989, a large number of overseas fellows were still outside the country. After Tianamen Square, many decided not to return. 3.13 By June 1992, project files indicate that of 754 overseas fellows sent out from China, only 374 had returned. The earlier euphoria that overseas fellows were able to win financial support from foreign-based institutions and relieve SAEP of the bill, turned sour. The Project Office at that time reported that overseas training was considered to be excessive: more training should have been local. The PCR, reviewed by Operations Evaluation Department (OED) in October 1993, indicated that of 776 overseas fellows sent out 392 had returned, summarized below. By the time of audit the number of returnees had risen to 416, or 54 percent of the total. The highest return rate of 79 percent was among the 459 faculty and staff sent for non-degree courses if advanced study: the lowest return rate of 15 percent was among the 261 Master's degree candidates. 3.14 Despite their relative smallness in terms of cost,' the Bank-financed model facilities absorbed a high proportion of Bank effort in project implementation, especially for the first three years, 1984-87. The Bank's rationale for supporting these civil works rested on the 13. Civil works funded by the Borrower exceeded 523,000 sq. m., at a cost of more than Y175 million. This represented almost 88 percent of the SAR target, at a cost in terms of local currency of 209 percent of the appraisal estimate. 21 desire to influence their design and construction in order to incorporate technical and managerial advances made outside China. For these reasons, the Bank invested heavily in supporting the architectural and engineering design of the model facilities. But progress was slow because of disagreements between the MAAF and the two benefitting institutions over design issues and the MAAF's desire to control costs. Three years into the project, progress on design and approval of the model facilities had advanced very little. Chinese experts were still visiting library facilities in the United States in 1987 to study relevant building designs and functions. 3.15 Progress of the BAU library was instructive. Immediately following project approval, an agreement was drawn up for provision of foreign assistance by specialists in library design and management. This assistance was efficiently provided and the foreign specialists who suggested library design concepts and principles with which the Chinese were unfamiliar nursed the hope that they would be retained to design and supervise construction of the library. For their part, the Chinese valued the advice they had received but intended to design and construct the library themselves. Unfortunately, the design concepts and principles suggested by the foreign specialists were too complex and costly for the Chinese. In consequence, design and design approval languished. More than three years after loan effectiveness a simplified design was approved by the MAAF, construction began in year four, and commissioning took place in year six. But, as with Wuhan's food science laboratory, the BAU library building-while practical and efficient-was not an innovative model. 3.16 Civil works carried out by the Borrower were, in aggregate, largely on target: out of 598,000 sq.m. of construction expected at appraisal, 523,000 sq. m. (87 percent) were actually completed. However, there was enormous variation between targets and achievements for individual sites. These variances resulted mainly from overly ambitious targets and local shortages of counterpart funds. The audit mission visited five universities supported by SAEP, two in Beijing and three in the provinces. Buildings constructed with counterpart funds under SAEP were generally satisfactory, although scarcity of funds for building O&M made the buildings age prematurely. Unreliable electric power supplies were noted in several locations, lowering the usefulness of some of the equipment procured under SAEP which needed dependable power for continuous operation or air conditioning. Provision of back-up power facilities was a common problem, and the supply of emergency power for sensitive equipment, such as microcomputers, became too costly for most institutions.'4 3.17 The Project Office monitored progress under SAEP with an agreed set of indicators, summarized in Annex Table A3. Annual tracer studies of graduates, also required under SAEP, were not organized. Because all graduates at that time were provided jobs in government agencies, tracer studies were seen by the Borrower as unnecessary. Nor were plans for future investments in agricultural higher education drawn up. Suggestions that the Bank support a major initiative to create institutions comparable to the U.S. land grant system were made and withdrawn when the state planning commission (SPC) indicated that it had more urgent demands for Bank support in other sectors. 14. Microcomputer back-up power required 60 batteries which needed to be replaced every four years at a cost of US$16,000. Name of the institutions visited by the audit could afford this. 22 3.18 The management performance of the Borrower recorded in SAEP's supervision reports was not problem free. Of 14 supervision mission reports, implementation summaries (Form 590) are missing from the files for four missions. Where records are complete, moderate problems (a rating of 2) were reported for Borrower management performance by ten consecutive supervision missions. Similar ratings were applied to procurement and technical assistance following most supervision missions. 3.19 Project supervision was generally adequate, with one notable weakness: for the first two years of implementation supervision missions failed to visit any of the institutions supported by SAEP outside Beijing, with the exception of the model facility at Wuhan. While the average amount of time devoted to supervision from year three of implementation onward declined (see Figure 3.3), a systematic effort was made by the Bank's task manager then to visit many of the newly-supported institutions. The model facilities attracted a disproportionate share of the total supervision time invested. Given the Borrower's reluctance to buy technical assistance, Bank architectural expertise as applied to the model facilities was important and appreciated, but it was not sufficient to ensure a high level of innovation. Figure 3.3: Second Agricultural Education: Time Spent on Preparation and Supervision, days in field 180 150 - ~120- c 90 - t6060 3 0 - r 0 -.= 83 84 85 86 87 88 89 90 91 92 Fiscal Year Preparation = Supervision M PCR Mission 3.20 Two conditions agreed in the SAR were not complied with: the IAP was not established and project monitoring through tracer studies was not carried out. Project files suggest that Borrower support for these conditions was weak during the design stage and, during implementation (see Table 3.1). 23 Table 3.1: Compliance with Project-specific Covenants Covenant Compliance 1. Maintain the project office established under Cr. Complied with 1297-CHA 2. Appoint on International Advisory Panel (IAP) by Not complied with December 31, 1984 3. Monitor and evaluate annually the progress made Complied with by Project Institutions 4. Select fellowship candidates in accordance with Complied with criteria acceptable to the Bank 5. Survey annually the relationship between academic Not complied with preparation and actual employment, between 1984 and 1992  25 4. Project Outcomes 4.1 Project outcomes are well described in the PCR. The audit reconfirms that SAEP has made a useful contribution to improved teaching and learning in a sample of 23 China's institutions of agricultural education at several levels. Performance at the 11 institutions which received Bank support for the second time tended to be higher than for the 12 institutions supported for the first time. Bank support helped to provide essential equipment for science laboratories, computer centers, language laboratories, distance learning production facilities, and books and periodicals. Overseas fellowships for graduate degree and nondegree programs was provided to 776 specialists, of whom just over half (416) have returned to China, summarized below. Some institutions fared better than others." In addition, two model facilities-an agricultural university library and a food science laboratory-were designed and constructed. Together, these inputs comprise a major investment in agricultural higher education. 4.2 In terms of educational performance outcomes, SAEP proved largely satisfactory. Indicators of performance, summarized in Table A3, show improvements in student enrollment and graduation in participating institutions at all levels except the graduate level, well beyond appraisal estimates. The proportion of female undergraduate and graduate students also improved dramatically. Despite the nonretum of many overseas fellows, the proportion of faculty with overseas experience improved beyond appraisal estimates, as did the proportion of faculty with training from institutions other than the one where they were teaching. 4.3 Indicators of improvement in curriculum and teaching methods were generally good. Notably, the number of new areas of specialization offered by universities at the undergraduate and graduate levels increased by 435 and 295 percent respectively, over the levels expected at appraisal. Students were enabled to perform many more scientific experiments themselves and able to complete the number of experiments required by the curriculum. Use of audiovisual equipment by teachers also improved, but less than expected at appraisal. 4.4 While equipment utilization records appear spotty, it is apparent that personal computers and language laboratories have been very heavily used and to good effect. Availability of basic scientific equipment-such as microscopes and standard laboratory equipment for undergraduate science courses-has also enabled very large numbers of students to improve their proficiency through hands-on experience. Efficient management of the procurement process enabled participating institutions to receive their equipment early in the period of implementation. Utilization of more sophisticated equipment is uneven, but generally on the low side because of high costs of operation, the low return rate of overseas fellows, and the high frequency of equipment being unserviceable for lack of spare parts or consumables, such as reagents. 4.5 Improvements in indicators of university management were also positive. Libraries supported by SAEP instituted open stacks policies for the first time, giving students direct I5. Beijing Agricultural Engineering University sent 26 fellows overseas. At the time of audit only one had returned. 26 access to the books on the shelves. Library opening hours were increased by an average of 54 percent over the levels prevailing before the project. Teacher:student ratios improved over the low aggregate levels prevailing before SAEP, i.e. from 1:3.2 to 1:4.8, but by less than forecast at appraisal. 4.6 SAEP's aim to raise classroom and laboratory utilization rates dramatically were met only in part. While academic space per student remained high, utilization of classroom space improved through large increases in the number of classes offered per day. Overall, indicators of educational performance-quantity and quality-were satisfactory. Some performance levels might have been higher if the IAP had been established and technical assistance been more in line with the SAR proposals. Participating institutions showed considerable willingness to innovate but lacked direction on where to focus and how, especially with regard to software for educational systems management. 4.7 Three-quarters of the Bank support for SAEP was for equipment. Of this, the majority was for sophisticated laboratory equipment necessary for training a new generation of students, supporting graduate student research, and available for faculty research. SAEP enabled the participating institutions to make a quantum leap in their instructional and research capabilities. Unfortunately, some of the equipment, the computers for example, is already outdated. Even if the computers procured had been adequately housed, operated and maintained, they would still need to be replaced by new machines. Other equipment, while not outdated, such as growth chambers, spectrophotometers and chromatographs, is often in need of major repairs or replacement because it is unserviceable. The MAAF and participating institutions are unable to update their equipment for lack of capital funds, and unable to utilize it optimally for lack of current operating funds, making sustainability uncertain. 4.8 The most heavily utilized equipment supported by SAEP appears to be in language laboratories and computer centers. Language training appears to be highly valued by students and staff alike, at institutions of all levels. The demand from students and faculty members for foreign language skills and computer numeric is very high. Personal computers procured by the benefitting institutions with Bank support, despite their age, are in almost constant use. In contrast, the microcomputers procured under SAEP have proved a poor investment. The microcomputers became outdated quickly, proving unreliable under Chinese conditions, and their O&M costs were too high for the benefitting institutions to bear. 4.9 About 14 percent of Bank support for SAEP was for overseas fellowships, but many overseas fellows on degree programs have failed to return to China (See Table 4.1). China has taken a pragmatic approach to this difficulty by staying in touch through correspondence and personal contacts, offering new incentives to attract them back, and providing opportunities for short-term visits to China without commitments from the fellows for their permanent return. But the cost of the high drop-out rate is significant. While some attrition should have been expected, SAEP suffered significantly. About two-thirds of the Masters and PhD candidates have not returned, and many of these were the best and the brightest of the mi4-1980s graduate students in agricultural science. This outcome will have a negative effect on project sustainability. 27 Table 4.1: Return Rate of Overseas Fellows Overseas fellows Overseas fellows Percentage Number returned returned Number % PhD 56 17 47 Masters 261 38 15 Advanced study 459 361 79 Total 776 416 54 4.10 The so-called model facilities at BAU and in Wuhan will have a longer lasting impact, although this will be less than expected when Bank support for this component was justified. The high cost of translating "state of the art" principles in library and food science laboratory design and management into reality proved unacceptable to MAAF. Satisfactory solutions were achieved, but they made only small contributions to advance Chinese building design practice. Given the scarcity of O&M funds for these institutions, the balance struck between investment in civil works, management systems, and books and periodicals was probably acceptable. If the original concepts had been built, the results might have been technologically excellent, but the cost of civil works, relative to O&M and recurrent book and periodical budgets, would have been excessive. 4.11 SAEP's contribution to strengthening administration of the participating institutions was beneficial, if uneven. The ability of the benefitting institutions to manage space and specialized facilities was enhanced, but their ability to plan educational programs to meet labor market and training needs, manage university staff, and execute tracer studies on graduates was not enhanced. Bank supervision was too thinly applied to enhance this capacity beyond the so-called model facilities. Technical assistance which was intended to have strengthened this role was not invoked. The reluctance to apply tracer studies agreed in the SAR reduced the ability of Borrower institutions to measure long-term project pay-offs. Given the dynamism of China's economy, disinterest in tracer studies as evaluation tools can no longer be justified. 4.12 The contribution of technical assistance to SAEP was less than expected because the Borrower was never fully committed. While a substantial program of technical assistance may have been justified, the program actually proposed was not based on the specific needs of the participating institutions but on a predetermined share of the total investment. On the demand side, the Borrower was not convinced of the value of technical assistance and reluctant to incur the cost. This position was evident in other Bank-supported projects in China and Bank staff should have been better prepared to counter it if they believed the component could make the contribution that was expected and help the Borrower to "buy in". In addition, participating institutions were numerous and widely scattered and Bank supervision was unable to make its own contribution to technical assistance except in limited doses to specific institutions. Low utilization of technical assistance, however, did allow for a higher than planned allocation of SAEP funds to acquisition of textbooks and teaching materials. 28 4.13 Repayment difficulties, which stem from the devaluation of the yuan against the US dollar and China's onlending policies, and which are affecting several early Bank-supported projects in China, are less of a problem in SAEP. Sixty-five percent of the credit/loan was used by institutions controlled from China's center, and for which the Ministry of Finance is directly responsible. In these cases, the institutions will be unaffected by devaluation and the allocation of foreign exchange risk to local government. However, thirty-five percent of the credit/loan was used by provincial institutions for which local governments are responsible. The effect of devaluation of the yuan (from US$1.00: Yuan 2.00 at appraisal, to US$1.00: Yuan 8.7 at audit), combined with China's internal onlending policy (which requires repayment within 20 years, following a five-year grace period) results in an implicit interest rate for SAEP of 14.1 percent. At audit, MAAF indicated that some local authorities have expressed concern about the higher than expected cost of repayment. But the question of whether local authorities would have borrowed SAEP funds in 1984 had they known the interest rate would have been above 14 percent, remains unanswered. 29 5. Findings and Issues 5.1 The overall performance of SAEP is considered to be satisfactory. The project contributed somewhat to the restoration of China's educational capacity in three ways: physical restoration of buildings and equipment, faculty restoration through overseas fellowships, and partial restoration of confidence within university faculties as useful contributors to China's new progress. In addition, new progress took the form of new buildings, sophisticated equipment and strengthened management of educational institutions. Implementation benefitted from a good understanding of the setting and the problems of educational development learned during the preparation and implementation of three preceding projects. Nevertheless, the timing and speed of SAEP's preparation reduced the potential for learning from the earlier projects. Significant Bank support for higher education in China was justified, but the pace and timing of the operation increased the likelihood that earlier problems of implementation would be repeated-which is what experience shows has happened. Significant highlights of PCR and PAR findings for the first five Bank-supported projects for agricultural research and higher education approved between 1981 and 1985 are contained in Table 5.1. 5.2 Attention to the quality of teaching and instruction in SAEP was relevant and justified, and the performance indicators set were realistic. Indicators of educational and management performance show generally satisfactory results, but the lack of tracer studies precludes a full assessment of intermediate project impacts. Some participating institutions sought to channel SAEP resources to support their research functions more directly, and the Bank was right to resist attempts to dilute the project's focus on teaching. Inclusion of agricultural secondary schools has been beneficial. The experience of the Beijing Agricultural School, visited by the audit mission, is exemplary but not unique. Agricultural secondary schools are the prime sources of field staff for China's municipal and county agricultural extension services, and Beijing Agricultural School has become a model for the type. 5.3 Implementation performance by the Borrower was competent. The Project Office functioned satisfactorily, although the quality of management by participating institutions was variable. Moderate problems of project management by the Borrower were reported by Bank supervision missions throughout seven years of implementation. The lack of agreement among Borrower institutions and the low level of preparation at approval lowered the value of the model facilities. While the outcome was satisfactory, the amount of new technology incorporated in the model facilities was limited. 5.4 The wide scattering of participating institutions, inadequate supervision during the first two years of implementation, and the limited use of technical assistance for other than language training, allowed low-grade, project management problems to persist. In a large country, such as China, where projects can be widely dispersed, it must be accepted that Bank supervision will take more time and longer missions to be fully effective. 30 Table 5.1: A Comparison of Performance Indicators in Five Early Bank-Supported Projects in China based on PCR and Audit Findings University Agricultural Education Polytechnic and TV Second Agricultural Second Agricultural Development: and Research: University: Research: Education: Cr. 1167-CHA/ Cr. 1297-CHA Cr. 1347-CHA Cr. 1516-CHA Cr. 1500-CHA/ Ln. Ln. 2021-CHA 2444-CHA 1981 1982 1983 1984 1985 Overall performance: Overall Performance: Overall Overall Overall performance: In most respects, Overall, highly performance: performance: Most project outcomes have successful. Educational capacity Most components performance indicators surpassed expectations. significantly increased were successfully were satisfied. despite a range of implemented. constraints. O&M budget: O&M budget: O&M budgets: O&M budgets: O&M budgets: Budget for O&M was O&M needs to be Maintenance funds Operational budgets Local budgets were "austere but adequate". institutionalized and deficient. A are modest and PCR inadequate to operate adequately financed, maintenance account is concerned about and maintain established in 1990 institutional ability to improvements to with unutilized maintain and repair facilities and equipment. project funds to sophisticated support 28 equipment. institutions. Overseas fellows: Overseas fellows: Overseas fellows: Overseas fellows: Overseas fellows: Return of overseas Mixed success because Not applicable. Nonreturn of many Nonretur of overseas fellows much slower of delayed return of research staff is of fellows a significant than expected-less overseas fellows, great concern. 50% loss. Only 54% of than 5% of 906 of trainees are not fellows returned by returned. expected to return. 1994. Technical assistance: Technical assistance: Technical assistance: Technical assistance: Technical assistance: No comment. T/A did not receive Difficulties in Slow utilization of T/A not well designed. good oversight from launching T/A. T/A. Great need to Weak preparation and the Borrower or IDA. Arrangements for identify and recruit weak Borrower implementation were relevant T/A commitment. not specific. specialists. Recruitment of nonChinese speakers and specialists w/o experience of China is unproductive. IAP: 1AP: lAP: lAP: IAP: Chinese Review Not applicable. Chinese Review Not applicable lAP not constituted as Committee and IAP Committee and lAP required. Some were critical factors in functioned, but less functions replaced by a successful effective than consulting firm. implementation. foreseen. Sust-inability: Sustainability: Sustainability: Sustainability: Sustainability: Electronic equipment Overly rapid expansion Original concept of Planned institutional Lack of O&M funds obsolete in 3-5 years, of the system, will be locally managed improvements have and nonretur of and other equipment difficult to sustain, institutions been put in place, but overseas fellows reduces obsolete in 6-12 years. functioning according concern that institutes institutional to central policies will cannot repair and development and puts be hard to sustain, maintain sophisticated sustainability at risk. equipment procured. 31 5.5 Sophisticated laboratory equipment procured under SAEP enabled participating institutions to fill important gaps in their hardware, widen the scope of their course offerings, give advanced students the opportunity to conduct hands-on experiments, and satisfy faculty needs to engage in research. The early preparation of procurement documents for ICB, however, contributed to a too-rapid build-up of new equipment, and the cost of operating and maintaining much of this equipment exceeded the budgets available almost everywhere. While the efficient acquisition of new equipment was viewed as an ICB coup, it created an O&M hazard. Assurances that sufficient budgets would be made available for O&M were not fulfilled. The result has been general underutilization of equipment procured and premature retirement of equipment for which repairs could not be afforded. Equipment utilization rates were further depressed by the nonreturn of overseas fellows. The Bank should, therefore, require implementing agencies formally to commit adequate annual O&M budgets for new facilities and equipment, and these commitments should be covenanted. Loan and credit disbursements should be tied to adequate provision of recurrent budgets to participating institutions. 5.6 Participating institutions have been creative in their search for low-cost means to operate and maintain equipment procured under SAEP. In many cases, equipment which would ideally be retired from service is still in operation. But equipment for which spare parts must be purchased with hard currency frequently stands idle. Most importantly, financial support for equipment O&M is insufficient. Funds are not available to replace redundant equipment with new technology. SAEP enabled a number of institutions to take a step up the technological ladder; lack of recurrent funds for O&M allowed some of that progress to be eroded; and, lack of capital for new generation equipment holds the institutions back from making necessary new advances. Institutions which centralized their equipment repair and maintenance facilities performed their essential maintenance functions more effectively than those that did not. 5.7 Robust and unsophisticated laboratory equipment, such as balances and optical microscopes, has been well used by teachers and students. Language training and computer education both built upon the success of the first Agricultural Education and Research Project. While the language training facilities provided under SAEP were among the most intensively utilized, the microcomputers were among the least successful. Good computer education performance, however, was based on desk-top computers, many of which were provided by other projects and by the microcomputer supplier as compensation for poorly performing equipment. 5.8 The low return rate of overseas fellows is disappointing but not entirely unexpected. Some attrition rate should have been expected because it is a characteristic of overseas fellowship programs generally. The experience of the very first Bank-supported project in China for University Development is instructive. This project was approved in November 1981, became effective in February 1982, and closed in June 1986. By December 1986, only five percent of the 904 overseas fellow sponsored by the project had returned to China, and by early 1988 when the PCR was finalized, these numbers had not changed significantly. Without the Tianamen Square crisis, the return rate of overseas fellows was extremely low, low enough to raise questions about the efficacy of the approach and safeguards. However, the PCR for University Development, which makes this point for the first time, was not 32 completed until January 1989, i.e. formal recognition of the performance of overseas fellowships came too slowly. But task managers should have been aware of low return rates and the increased risk of nonreturn when overseas fellows prolonged their stays outside China in relatively well-endowed academic environments. China's policies that encouraged extended periods overseas have been superseded, but Bank acceptance of this policy at the time contributed to a costly brain-drain. 5.9 Between 1981 and 1986, the Bank approved one project for support of higher education in China per year, with loans and credits for this purpose totalling nearly US$700.0 million over six years. This high rate of capacity expansion exceeded the Borrower's ability to sustain the investments made in buildings and equipment with recurrent expenditures for O&M. Limited O&M funds makes the sustainability of SAEP's investments uncertain. The risk of unsustainability is heightened by the low rate of return of overseas fellows who would be primary users of the new facilities and equipment. The low rate of return of overseas fellows, in turn, limits institutional development achieved under SAEP to modest levels. 5.10 With the exception of foreign language training, technical assistance under SAEP was poorly utilized. Poor utilization stemmed from inadequate preparation and weak commitment. The text of the SAR makes no serious attempt to justify the US$6.0 million of Bank support for technical assistance or to identify specifically how technical assistance would be employed. Inclusion of the phrase "specialist assistance" was deemed sufficient to justify the component. However, from the date of loan effectiveness, the Borrower was reluctant to draw on the resources allocated for specialist assistance. Specialist assistance may have oeen a routine component of Bank projects, but it was judged by the Borrower to be ineffective for local institutional development and costly. Language training specialists were the significant exceptions to this rule. 5.11 Underutilization of technical assistance is characteristic of most completed Bank- supported agriculture projects in China." The main causes of underutilization appear to be largely the same in all cases: weak design and limited Borrower commitment. Technical assistance components were generally not "designed", simply "identified" by the donor, using the traditional model of short-or long-term expatriate advisors. The Bank perpetuated this approach when it should have been aware early on of the Borrower's reluctance to accept it uncritically. But project files reveal little about the nature of the problem or possible solutions, only the reluctance to commit borrowed funds for this purpose. The Bank needs to reconsider its approach to technical assistance in China. Where a traditional approach to technical assistance is agreed, its use and effectiveness should be carefully monitored to ensure compliance with agreements made. Technical specialists who are required to provide short- term assistance must be carefully selected and prepared, and given practical guidance on how to adapt their expertise to Chinese conditions. 5.12 The IAP was never convened and some of its likely functions performed through a contract with a consulting firm. An IAP had performed effectively in the University Development Project (Credit l l67-CHA/Loan 2021 -CHA) but, as with technical assistance, 16. This is a preliminary finding from OED analysis of Bank lending for agriculture in China, a study due to be completed in 1996. 33 China believed that the IAP mechanism was not cost-effective and chose not to invoke its use in SAEP. Since China had access to technical assistance from grant-funded sources, low rates of utilization of Bank support for technical assistance tends to understate the actual amounts applied. Nevertheless, less technical assistance was applied in fewer areas of specialization than intended at appraisal.  35 Annex A Supporting Tables Table Al: Bank Support to China for Education and Human Resource Developmentr Credit/Loan Project Name Approval US$ million Number Date Credit 1167/ University Development 1981 100.0 Loan 2021 University Development 1981 100.0 Credit 1297 Agricultural Education and Research 1982 75.4 Credit 1347 Polytechnic and TV University 1983 85.0 Credit 1500/ Second Agricultural Education 1984 23.5 Loan 2444 Second Agricultural Education 1984 45.3 Credit 1551 Second University Development 1985 145.0 Credit 1671 Provincial Universities 1986 120.0 Credit 1908 Teacher Training 1988 50.0 Credit 2006 Textbook Development 1989 57.0 Credit 2114 Vocational and Technical Education 1990 50.0 Credit 2210 Key Studies Development 1991 131.2 Credit 2339 Educational Development in Poor Provinces 1992 130.0 Credit 2471 Effective Teaching Services 1993 100.0 Credit 2651 Basic Education in Poor and Minority Areas 1994 100.0 Total 1,312.4 a. This list includes only projects whose primary purpose is education. Educational components exist in several other projects whose inclusion would expand the list further. Examples include Ganzu Provincial Development (Credit 1793-CHA) which includes an education component appraised at US$20.0 million. Annex A 36 Table A2: Project Implementation - Appraisal Estimates and Actual Completion Appraisal Actual Actual as Estimate Completion % of Appraisal Civil Works - construction (sq m) newly assisted colleges 274,800 182,744 66.5 previously assisted colleges 121,617 143,162 117.7 model facilities 17,500 17,500 100.0 training and extension centers 23,730 41,900 176.6 agricultural technical schools 154,220 131,684 85.4 distance learning and support facilities 5,703 6,066 106.4 total new construction 597,570 523,056 87.5 - renovation (sq.m.) 16,489 32,601 197.7 Technical Assistance - local specialists (person months) 1,534 690 45.0 - foreign specialists (person months) 737 665 90.2 - foreign specialists, excluding language training (person months) 577 149 25.8 Overseas Fellowships - PhD (no.) 19 56 294.7 - Masters (no.) 125 261 208.8 - Advanced study and short term (no.) 473 459 97.0 37 Annex A Table A3: Project Results - Appraisal Estimates and Actual Completion Indicators Status in Appraisal Actual as % 1983 Estimate Actual of Appraisal Enrollment at New colleges Undergraduates (increased) 19,500 32,000 31,849 99.5 Graduates (increased) 500 2,300 1,477 64.2 Extension centers 4,800 9,000 63,582 706.5 Agricultural technical schools 5,000 9,000 9,422 104.6 Agricultural broadcasting schools 400,000 600,000 1,300,000 216.7 Improve ratio of undergraduate: graduate students 40:1 14:1 21.6:1 154.3 Increase annul intake of new colleges 5,500 8,000 9,129 114.1 Output from New colleges Undergraduates 5,000 7,500 8,976 119.7 Graduates 200 800 523 65.4 Extension centers 2,000 7,000 63,582 908.3 Agricultural technical schools 1,500 2,500 2,434 97.3 Agricultural broadcasting school -- 150,000 150,000 100.00 Agricultural Film Studio (hours of film/video production) 13 96 99 103.1 Improving Sex Ratio Enrollment: % females Undergraduates 17 25 28 112.0 Graduates 6 12 18 150.0 College Academic Staff Development Reducing % of staff with less than 4 years of higher education 8 2 6 33.3 Increasing % of academic staff with 5 or more years of higher education 24 45 n.a.' -- Improve ration of lecturers: associate professors 24:1 10:1 1.4:1 -- Increasing % of academic staff who would: be trained locally at other institution -- 23 26.70 116.1 be trained overseas -- 10 12.13 121.3 participate in seminar -- 20 41.18 205.9 College Curriculum & Pedagogical Improvement Increasing % of experiments performed out of total required by curriculum 60 90 93.5 103.9 (continued) Annex A 38 Indicators Status in Appraisal Actual as % 1983 Estimate Actual of.Appraisal Increasing % of teachers using audiovisual equipment 5 70 36.4 52.0 Increasing subject matter areas for which audio- visual teaching material are avail. (no.) few (5) 30 -- -- Add new specialties at graduate level (no.) -- 31 91 293.5 Add new specialties at undergraduate level (no.) -- 20 87 435.0 College Management Improvement Opening hours for libraries/week 48 80 73.8 92.2 Broadening access to library materials no open many many -- stacks Improving teacher: student ratio 1:3.2 1:6 1:4.8 57.1 Increasing number of trained Technicians -- 300 exceededb Administrators -- 100 exceededb Improve space standards and utilization by: Reducing area of academic facilities per student (sq m) 30-35 18-20 Not -- Increasing number of class periods/day 4 8 6.2 77.5 Increasing utilization of classroom/space (%) 40 70 76.5 109.3 Source: Project Office, MAAF a. Survey of 12 colleges shows teachers with 6 years of higher education increased from 8 percent to 26 percent during the project period. b. Exact numbers not available. 39 Annex A Table A4: Project Financing Appraisal Estimates and Actual Costs Appraisal Actual Actual as Estimate Cost % of Appraisal IBRD (US$m) - equipment 30.6 34.9 114.1 - civil works 0.5 4.3 860.0 - specialist services 3.7 1.4 37.8 - overseas fellowships 6.2 4.3 69.4 - books and instructional materials 1.0 0.3 30.0 - fee 0.1 0.1 100.0 - unallocated 3.2 0 0 Total 45.3 45.3 100.0 IDA (US$m) - equipment 16.0 17.8 111.3 - civil works 0.5 0 0 - specialist services 1.8 1.6 88.9 - overseas fellowships 3.2 5.2 166.4 - books and instructional materials 0.4 0 0 - initial deposit in special account 1.6 0.6 37.5 Total 23.5 25.2 106.4 Government (US$m) 106.2 55.6 52.4 Total 175.0 126.1 72.1 Annex A 40 Table AS: Project Costs - Appraisal Estimate and Actual (US$m) Appraisal Estimate Actual Actual total Item as % of Local Foreign Total Local Foreign Total Appraisal Estimate A. Agricultural Colleges 12 Newly Asst. Colleges 39.4 27.9 67.3 29.5 37.0' 66.5 98.8 11 Previously Asst. Colleges 22.9 12.4 35.3 11.9 14.0b 25.9 73.4 Food Science Dept. 0.9 1.2 2.1 0.8 1.4 2.2 104.8 Library 2.8 2.4 5.2 2.0 2.5 4.5 86.5 Tech. Asst. 2.6 9.2 11.8 0.2 1.7 1.9 16.1 Subtotal 68.7 53.0 121.7 44.4 56.6 101.0 83.0 B. Training & Extension Support 8 Centers 3.0 4.0 7.0 3.9 5.5 9.4 134.3 Tech. Asst. 0.2 1.0 1.2 0 0.3 0.3 25.0 Subtotal 3.2 5.0 8.2 3.9 5.8 9.7 118.3 C. Agr. Sec. Specialized Schools 12 Schools 4.6 4.0 8.7 4.5 4.8 9.3 114.9 Tech. Asst. 0.3 0.8 1.1 0 0.5 0.5 45.5 Subtotal 4.9 4.9 9.8 4.5 5.3 9.8 100.0 D. Distance Learning & Supporting Facilities 1.2 2.1 3.3 2.5 2.4 4.9 148.5 E. Evaluation, Planning & Preparation of Future Projects 0.3 1.0 1.3 0.3 0.4 0.7 53.8 Base Cost 78.2 65.9 144.2 55.6 70.5 126.1 87.4 Physical Contingencies 3.9 3.3 7.2 -- -- -- Price Contingencies 12.9 10.7 23.6 -- -- -- Total Project Costs 95.0 79.9 175.0 55.6 70.5 126.1 72.1 a. Including US$I million for teaching materials. b. Including US$0.8 million for Foreign Language Training Centers. 41 Annex B FAX NO: 202-5zz-u123 arch Z, 1985 TO: MR.GRAHAM DONALDSON CBIEF AGRICULTURE AND UNAN DEVELOPMENT DIVISION OPERATIONS -EVALUATION DEPARTNT- .THE WORLD BAIN FRON: ZEU ZI DONG DEPUTY DIRECTOR PROJECT OFFICE DEPARTMENT OF EDUCATION MINISTRY OF-AGRICULURE BEIJING CEINA FAX:5002448 Re: China-Second AOricultural Educatin Project (La.2444-CIA/Cr,1500-CIA) Draft Audit Report Dear Mr.donaldson: Wc have received your ikaft Audit Report on SAEP. We basically agree with the audit conclusions on the SAEP by Lhe OED. We think we will have a pleasant cooperation with the Bank in the future. Sincerely ZhuZi Dons Deputy Director ra -. Project Office Education Department F  IMAGING Report No: 14622 Type: PPAR