China - Rural Health Manpower 4

 ICRR 11172 Report Number : ICRR11172 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 09/11/2002 PROJ ID : P003502 Appraisal Actual Project Name : Cn-rural Health Manpower Project Costs 186.10 99.8 (hlth4) US$M ) (US$M) Country : China Loan/ US$M ) 110.14 Loan /Credit (US$M) 109.14 Sector (s): Board: HE - Health (100%) Cofinancing US$M ) (US$M) L/C Number : C2539 Board Approval 93 FY ) (FY) Partners involved : Closing Date 12/31/1999 03/31/2001 Prepared by : Reviewed by : Group Manager : Group : Laura Raney Ronald S. Parker Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The goal of the project was to improve the quality of rural health workers in China, thereby contributing to better quality health services and an improved health status of the rural population in the project area. The specific objectives were to: Strengthen health personnel planning in the six provinces and at the national level; Retrain and train better qualified teachers, health workers, and managers by reorienting the training process through curriculum reform, a wider variety of teaching and learning methods, and better and more supervised practicums; Develop alternative means for mobilizing financial resources to support rural health care delivery, and compensate rural health workers and provide incentives to emphasize preventive care; Upgrade the physical conditions of the training institutions and of health service facilities, improve inter-institutional coordination, and develop training networks with defined roles for each training institution; and Strengthen the national capability for carrying out work force planning and manpower policy formulation, for providing assistance to the project provinces and for evaluating the project activities and results in order to disseminate the experience gained to the rest of the country. b. Components Health Workforce Planning (US$ 3.3 million, 3 % of costs), including (a) development of a health planning capacity in the provinces; (b) training the planning staff; (c) improving the planning data base; (d) implementing health manpower planning; (e) establishing a qualitative basis for manpower planning; and (f) monitoring and evaluating the manpower planning function. Health Workforce Training (US$ 85.1 million, 77 % of costs), (a) establishing provincial training networks and plans; (b) training teachers and administrators for health training schools; (c) improving the skills of the existing health workers; (d) revising pre-service education and training new health workers; (e) upgrading infrastructure of training institutions. Rural Health Services Management (US$ 18.9 million, 17 % of costs), including (a) developing health services management and supervision systems; (b) providing regular on-the-job training; (c) developing financing, compensation and incentive systems; and (d) upgrading infrastructure of health service facilities. Project Management (US$ 1.81 million, 2% of costs) and & Central Support (US$ 1.1 million, 1% of costs) to augment the institutional capacity of MOPH and especially of three line departments for coordinating and supporting project implementation activities and disseminating project results. c. Comments on Project Cost, Financing and Dates The project was designed at the beginning of a period of rapid social and economic change, to which health schools and health facilities reacted without a clear policy or planning framework. One unintended consequence was a rapid increase in intakes of trainees that was unrelated to demand. As a result, at the mid-term review there was reallocation of funds from training and technical assistance (15%) to equipment and civil works. After a 15-month extension all credit funds were utilized. 3. Achievement of Relevant Objectives: The project achieved most of its numeric targets per its process and revised output indicators established during the midterm review (see ICR Annex 1), but because of difficulties in data collection, evidence is lacking regarding progress toward many of the outcome indicators established in the project design document. Over a quarter of a million rural health workers at the village, county and prefecture levels received training under the project, along with three thousand health managers, trainers and planners. Construction targets were met or exceeded. Through infrastructure and equipment provision, working conditions of rural health workers improved. A health workforce planning system was established, along with databases and manuals in the six project provinces. Various teaching reform pilots were conducted, and new teaching methods were implemented, which were reported to have improved the problem-solving abilities of trainees. The quantity, structure and distribution of the health workforce have been rationalized in the project areas. There is limited evidence, however, regarding the impact of training and reforms on the knowledge and teaching skills of teachers, the overall quality of the rural health workforce, or the quality of service delivery. 4. Significant Outcomes/Impacts: The project brought increased attention to health workforce planning, health manpower quality, and services management in the project provinces. The health workforce planning concepts, methods, manuals and experience from the project have contributed to some personnel reforms. The rural health care financing and rural doctors’ payment studies contributed to on-going policy development of the subject. The project completed its physical objectives, rehabilitating and expanding infrastructure at township health centers and village clinics as well as health training facilities at the provincial, prefecture and county levels. Outcomes of the service management and training components have been adopted beyond the project areas, namely the mechanisms of clinical supervision and quality monitoring, and the pilots of teaching reform that developed and tested training methodology, teaching material, and curriculum. 5. Significant Shortcomings (including non-compliance with safeguard policies): The appraisal document highlighted the importance of focusing attention on the replacement of income foregone during in-service training (main component). This did not happen. The project was affected by the financial burden that government requirements (governing on-lending and cost recovery based on use) imposed on needy communities, which may have reduced the benefits of the project in the very areas it was designed to serve. During implementation, the project encoutered difficulties responding to changes in government training policies, including significant increases in outputs from private training schools -- which resulted in a shift from a shortfall in trained health workers to an oversupply. Multiple layers of authority were involved -- central (three), provincial (six) and prefectural governments (36) as well as county health bureaus (374) -- contributing to difficulties in communication and coordination. Although target indicators and an evaluation plan and were developed and presented in the project design document, evaluation did not begin until after the mid-term review, with redefined, less comprehensive indicators. Thus, there is little evidence regarding the impact of project investments on service quality, utilization or health outcomes. The lack of effective maintenance for civil works contributes to questions regarding sustainability of the investment. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory Though the project completed virtually all planned activities and met most of its output targets, the effectiveness of the health worker training was suboptimal, the role of training facilities given changing policy is unclear, and there is limited evidence regarding the impact of training on service quality. Institutional Dev .: Substantial Modest The project made substantial investments in training and developing health manpower planning systems, but the latter have not yet been adequately internalized. Sustainability : Likely Non-evaluable Sustainability of project activities is contingent on adaptation to the changing policy environment. Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. NOTE: 7. Lessons of Broad Applicability: 1. Project objectives need to be realistic, and to reflect the demonstrated abilities of the implementing agency. 2. Financial requirements need to be realistic, reflecting actual capacity to repay. 3. Policy changes need to be reflected in training efforts as soon as practicable. 4. Lack of effective maintenance arrangements places physical investments in the health sector at risk.  5. Project indicators for output and coverage need to be defined at the outset with particular attention given to requirements for data collection, its management, feedback from monitoring and evaluation and institutional arrangements for capacity building. 8. Assessment Recommended? Yes No Why? The project has already been assessed to provide a building block for the CAE, as part of a cluster of eight projects. 9. Comments on Quality of ICR: The ICR provides a good description of the project's design and its overall strengths and weaknesses, but there is limited evidence regarding the actual impact of training. The borrower comments are generally positive regarding the project, but additional pertinent information might have been included in the main ICR text.