Turkey - Second Health Project

Document of The World Bank Report No: 31604-TU IMPLEMENTATION COMPLETION REPORT (CPL-38020 SCL-3802A SCPD-3802S) ON A LOAN IN THE AMOUNT OF US$ 150 MILLION EQUIVALENT TO THE REPUBLIC OF TURKEY FOR A SECOND HEALTH PROJECT October 27, 2005 Human Development Sector Unit Turkey Country Unit Europe and Central Asia Region  CURRENCY EQUIVALENTS (Exchange Rate Effective 05 June 2005) Currency Unit = New Turkish Lira I New Turkish Lira = US$ 0.74 US$ 1 = 1.35 New Turkish Lira FISCAL YEAR January 1- December 31 ABBREVIATIONS AND ACRONYMS ANC Antenatal Care CDC Center for Disease Control CHRMS Computerized Human Resource Management System CMSS Computerized Management Support System CPR Contraceptive Prevalence Rate DALY Disability Adjusted Life Years DHS Demographic and Health Survey GOT Government of Turkey HHDR Home/ Hospital Delivery Ratio HOIS Hospital Information System HTP Health Transition Project IMR Infant Mortality Rate M&E Monitoring and Evaluation MEER Marmara Emergency Reconstruction Project MMR Maternal Mortality Rate MOH Ministry of Health MSA Management Service Agreement MTR Mid Term Review NBD National Burden of Disease NHA National Health Accounts OPCS Operations Policy and Country Services PCU/ PIU Project Coordination Unit/ Project Implementation Unit PHCP Primary Health Care Project PSR Project Supervision Report SAR Staff Appraisal Report SIS State Institute of Statistics SHP Second Health Project SPH School for Public Health TFR Total Fertility Rate U5M Under 5 Mortality UNDP United Nations Development Program UNOPS United Nations Office for Project Services WDR World Development Report Vice President: Shigeo Katsu Country Director Andrew N. Vorkink Sector Director Charles C. Griffin Sector Manager Armin H. Fidler Task Team Leader: Ibrahim Akcayoglu  TURKEY Second Health Project CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 12 5. Major Factors Affecting Implementation and Outcome 24 6. Sustainability 26 7. Bank and Borrower Performance 27 8. Lessons Learned 31 9. Partner Comments 33 10. Additional Information 34 Annex 1. Key Performance Indicators/Log Frame Matrix 35 Annex 2. Project Costs and Financing 38 Annex 3. Economic Costs and Benefits 40 Annex 4. Bank Inputs 41 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 44 Annex 6. Ratings of Bank and Borrower Performance 45 Annex 7. List of Supporting Documents 46 Annex 8. The Borrower's Contribution to the ICR 48 Annex 9. List of Equipment purchased and status of THC construction in 23 Project 61 Provinces Annex 10. Analysis of Trends for Selected Health Outcomes 1993-2003. 65 Annex 11. Comments Received from MOH on August 15, 2005 70  Project ID: P009076 Project Name: Second Health Project Team Leader: Ibrahim Akcayoglu TL Unit: ECSHD ICR Type: Core ICR Report Date: October 27, 2005 1. Project Data Name: Second Health Project L/C/TF Number: CPL-38020; SCL-3802A; SCPD-3802S Country/Department: TURKEY Region: Europe and Central Asia Region Sector/subsector: Health (86%); Central government administration (14%) Theme: Other communicable diseases (P); Rural services and infrastructure (P); Health system performance (S); Population and reproductive health (S); Decentralization (S) KEY DATES Original Revised/Actual PCD: 11/05/1988 Effective: 01/31/1995 01/31/1995 Appraisal: 08/15/1994 MTR: 07/14/1998 07/14/1998 Approval: 09/22/1994 Closing: 12/31/2001 12/31/2004 Borrower/Implementing Agency: GOVERNMENT/MINISTRY OF HEALTH Other Partners: STAFF Current At Appraisal Vice President: Shigeo Katsu Wilfred Thalwitz Country Director: Andrew N. Vorkink Michael Wiehen Sector Manager: Armin H. Fidler Ralph Harbison Team Leader at ICR: Ibrahim Akcayoglu Alexander Preker ICR Primary Author: Anne Bakilana 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: UN Institutional Development Impact: M Bank Performance: U Borrower Performance: U QAG (if available) ICR Quality at Entry: U Project at Risk at Any Time: Yes  3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The Second Health Project (SHP) should be seen as part of a broader Bank supported program in Turkey after the adoption of the 1987 Basic Law on Health Services, which opened the door for new efforts by the Government of Turkey (GoT) to improve the performance of its health sector. The Second Health Project (SHP) aimed to expand on the activities of the First Health Project (FHP) to the country's 23 most deprived provinces. At the time of its preparation, overall health indicators showed that Turkey had seen major improvements in health indicators since the 1960s. Life expectancy at birth was around 67 years in 1991, up from 47 years in 1960; child mortality was around 94 per 1000 live births, down from 217 per 1000 in 1960; and Infant Mortality Rate (IMR) was about 58 per 1000 live births, down from 208 per 1000 in 1960. Indicators for the country's most deprived eastern provinces were worse. IMR in the 23 project provinces stood at 78.6 per 1000 live births and under-5 mortality stood at 104.0 per 1000. While these deprived areas had more or less the same average for the population served by health posts (4, 961 for Turkey vs. 4, 652 project provinces' average) and almost the same number of health centers (14, 672 for Turkey vs. 12, 764 for the 23 project provinces), there were large inequalities in other health indicators. For example, the population per midwife or population per nurse ratios were quite uneven (668 Turkey average vs. 4, 328 for the 23 project provinces). For the 23 project provinces, Total Fertility Rate (TFR) was about 8.7 children per woman compared to 3.4 Turkey average; and the literacy rate was about 50% for women in the eastern provinces compared to the Turkey average of 81.9%. The Loan Agreement states that the objectives of the Second Health Project were to: a) upgrade and expand the provision of health care in the project provinces; and b) improve health care management. These objectives were stated slightly differently in the project's Staff Appraisal Report (SAR) as follows: The primary goal of the SHP was to reduce the burden of disease due to avoidable illness and disability in Turkey. To achieve this goal, the specific development objectives (DOs) were to: a) improve equity of access to essential health services in 23 Eastern, low income priority provinces; and b) improve the quality of health care management in selected institutions. The project aimed to improve health status, reduce infant mortality rates, decrease fertility rates and improve life expectancy in the participating provinces. The objectives of the project as stated in the SAR have formed the basis of this project's implementation and supervision. The Bank's Implementation Completion Report (ICR) writing guidelines specify that this ICR must assess the degree to which the project's objective was clear, realistic and important for the sector and the Bank's country assistance strategy. This ICR finds that the objectives of the SHP as stated in the SAR were in line with health sector priorities in the country and they were consistent with the Government's desire to improve health status in the 23 poorest provinces. The objectives responded to the Bank's overall strategy for health and development as expressed in the Bank's 1993 World Development Report (WDR). Objectives were also consistent with the 1993 Country Assistance Strategy (CAS) and are still relevant to the 2003 CAS objectives. This ICR found that though the aim and objectives of the project were very clear, they were also too broad and over ambitious. The breadth of the project's objectives, the untenable links between its many activities and project objectives, as well as various issues of project design (as discussed under Section 3.5), meant that project activities were not well focused and were not designed to meet or to measure the expected outcomes of the project. - 2 -  In addition to the SHP, the Bank has assisted in the Government of Turkey's efforts to improve health outcomes in the country through its support for three other health sector projects as follows: l The First Health Project (FHP) was the first Bank financed project in Turkey. It was approved in May 1989 with objectives to: i) improve access to basic health services in 8 underprivileged provinces; ii) enhance the efficiency of delivery of service; iii) improve financing of the public health system; and iv) strengthen the management capacity of the Ministry of Health (MOH). The FHP closed in December 1998, with approximately US$60 Million over the appraised budget and about US$31 Million in outstanding liabilities. The FHP received an overall outcome rating of satisfactory at ICR but was rated unsatisfactory by the Bank's Operations Evaluation Department (OED); l The third health project in Turkey funded by the Bank was the Primary Health Care Project (PHCP). It was signed in June 1997 with objectives to improve access and quality of primary health care in 2 provinces; and to serve as the basis of extending the tested health care primary reforms nationwide. The PHCP faltered because implementation depended on passage of three laws, none of which received the necessary support to be enacted. After 21 months of stagnated implementation and almost no disbursement, most of its proceeds were reallocated to the Marmara Earthquake Emergency (MEER) project in 1999. The PHPC closed in June 2001, having supported only a small fraction of planned activities and without achieving its intended development objectives; and l The fourth health project supported by the Bank became effective in July 2004. The overall objective of the Health Transition Project (HTP) is to support health sector reforms that aim to improve governance, efficiency, user and provider satisfaction and long term fiscal sustainability of the Turkish health care system. This project responds to the current Government's Urgent Action Plan and the Program for Transition in Health, which aims to institute health sector reforms to address health inequalities and inefficient use of resources which contribute to poor health outcomes in Turkey. 3.2 Revised Objective: The SHP's development objectives were revised in 1999 to address significant delays in implementation during its first five years and to address financial shortages and health sector challenges raised by the 1999 earthquakes. The project's overall aim remained the same but an extension was made to the objectives so as to address the needs of five provinces affected by earthquakes. The objectives of the project were restated in the Project Implementation Report (PIR) as follows: The project's primary goal is to reduce the burden of disease due to avoidable illness and disability. The specific objectives of the revised project were: (i) to improve equity of access to essential health services in 23 eastern low-income, priority provinces, and 5 provinces affected by the August and November 1999 earthquakes; and (ii) to improve the quality of health care management in the Ministry of Health (MOH) and selected institutions. The project aimed at improving health status, reducing infant mortality rates, decreasing fertility rates, and improving life expectancy in the participating provinces. Reasons for Project Revisions: By November 30, 1999, about two years before the project was expected to close, project output was noticeably behind schedule and only about 25.6% of the loan had been disbursed against the expected 90%. There were three main reasons that led to delays and the revision of the project: 1. Significantly delayed implementation caused by diverse factors as follows: l Borrower's decision to postpone implementation of the Infrastructure Development sub-component (under Primary Health Care component) to Project Year 3. In 1995/1996 Turkey's economy went through a period of macroeconomic turmoil. As a result, tight fiscal controls - 3 -  were adopted by the Government and this led to a decision to concentrate its efforts and finances to the completion of the FHP which was still being implemented at that time. This had a serious impact on the implementation of the Infrastructure Development sub-component (which was to involve the largest portion of activities supported by the project - almost 70% of the SHP) which was then postponed to Project year 3; l Serious shortages of Government contributions. Another direct result of the tight fiscal measures adopted by the Government after the mid 1990s was that by Mid Term Review (MTR) the Government's share of total disbursement amounted to about US$1.2 million; the expected figure by this milestone was US$ 34.9 million, i.e. the Government's contribution amounted to 3.4% of what it should have been. This was critical to implementation of all activities that required the Government's contribution. l Key legislation was not tabled in parliament. Implementation of Hospital Management Restructuring sub-component stalled and were later dropped when the Law on Primary Health Care Services and Family Physicians failed to get the needed support to pass into law. This legislation, which was key for the implementation of most activities under the Health Policy and Management component, did not receive the needed support from the various Governments that led the country in the 1990s (SHP's implementation probably spanned 11 different Turkish Governments). l Ineffective structure and functioning of the PCU. By 1999, it had become apparent to the Bank that the structure of the Project Coordination Unit (PCU) and its working relations with the Ministry of Health (MOH) were strained and ineffective. The PCU was originally convened to manage implementation of the FHP and was originally acknowledged as a key player in the successful implementation of Bank supported health projects, which at one point included all three health sector projects (FHP, PHCP, and SHP). Until around 1996 the PCU worked in close partnership with the MOH. At MTR, its staffing of mostly well paid consultants and some MOH civil servants, and its management which was independent of the MOH were considered ideal by the Bank. The status of the PCU was that of a dynamic center of excellence in supporting project implementation, in piloting activities, conducting academic studies and in formulating health policy. This was a model PCU, showcased across the region as an example to emulate. According to the Bank's internal documents, relations between the PCU and MOH began to deteriorate around 1996, mostly due to changes in governments and political affiliations. The management of Bank supported health sector projects began to be questioned by new and subsequent administrations, and tensions between the PCU and MOH grew to the extent that from 1996/ 1997 onwards, the PCU was seen to be working in competition with the MOH, basically seen as trying to implement activities that were the responsibilities of the MOH. This resulted in reduced cooperation between the MOH and PCU, making the PCU ineffective which considerably slowed down implementation especially since the project design had not assigned implementation tasks to MOH or its divisions. l Lengthy judicial processes concerning Bank supported Health Projects.The implementation of the SHP was severely affected by lengthy and extended inspections and investigations on project related activities. At the core of these investigations was the alleged misuse of loan proceeds from Bank supported projects. As was extensively raised by those interviewed during the ICR Mission, these investigations impacted negatively on the performance of the PCU. Not only did these continuous inspections require that documents be produced whenever requested (meaning that time performing normal duties was reduced); presence of inspectors from different Government agencies in the same building and same floors as the offices of the PCU created a high degree of suspicion, a climate of fear and generally an unhealthy working environment. The real result of all these events created a strong - 4 -  reluctance on the part of PCU Coordinators and staff to take any decisions that could potentially lead to future inspections, for example for many months in 1999 contract signing for project activities slowed down significantly. It is the Bank's understanding that all allegations of fund misuse have so far been unproven. 2. Earthquakes that shook Turkey in August and November 1999: These earthquakes caused the Government of Turkey an estimated US$1.8-2.2 billion fiscal burden. As requested by the Government, the Bank agreed to reallocate US$22.5 million from the SHP (and US$ 14.5 million from the flailing PHCP) towards earthquake emergency reconstruction activities which formed a package of some US$270 million quickly put together from the proceeds of 8 loans from the Bank's various sectors and presented to the Board within 3 weeks of the earthquakes. These were additional resources to those of the Marmara Earthquake Emergency Recovery Project (MEER) (US$ 505 million) whose main objective was to help restore living conditions in regions affected by the earthquakes, to support economic recovery and resumption of growth and develop an institutional framework for disaster risk management and mitigation. In the health sector, MEER together with the portion of funds from SHP were to support the repair of damaged healthcare facilities and strengthen community mental health services to better prepare the country for disaster. MEER is expected to close in May 2006. 3. The need to rectify a faulty Management Service Agreement (MSA) between MOH and UNDP/UNOPS: According to the Bank's internal documents and interviews held with the project's Task Team Leader at Mid Term Review (MTR), it became apparent that the MSA contracted between the Government of Turkey and UNDP/ UNOPS in 1995 did not meet the Bank's fiduciary and financial management standards or the various clauses of the Loan Agreement. This was not the first time that an MSA had been used in Bank operations, among the forerunners in the region was the MSA signed between the Government of Turkey/MOH and UNDP/UNOPS to provide management of activities relating to activities of the FHP. The MSA worked in the following way: after advice from the MOH/Government on items that needed funding, UNOPS forwarded a Blanket Application for Withdrawal of project proceeds to the Bank after which deposits to UNDP's account in US$5 million lots would be made. In return UNDP would undertake procurement, enter contracts with individuals, engage consultants and firms; and administer fellowships and study tours on behalf of the Borrower for a management fee of about US$2.8 million, which is about 5% of the total cost for activities to be managed (US$54.7 million). Both the first MSA under the FHP and the one under the SHP were hailed as great successes in speeding the flow of funds by circumventing cumbersome Bank and Government of Turkey bureaucracies while the PCU stayed on course with management of the project. However, the MTR raised concerns about the lack of an audit clause and lack of sufficient oversight of project activities by the Bank. It was also a concern that by relegating management of such a large portion of the project to the UNOPS, the PCU (which already had close to 200 staff) and the MOH were not gaining valuable experience in managing the project. In fact, a large proportion of total disbursements at MTR was of funds that had been disbursed to a Management Service Agreement (MSA) account and did not yet represent delivery of goods and services. 3.3 Original Components: The total project cost was estimated at US$200 million (US$150 million Bank loan and a Government's contribution of US$50 million). The project had two components with seven sub-components. Component 1: Primary Health Care (US$155.6 million total cost). Aimed to improve equity of access to essential health services in 23 low income priority provinces through: a) upgrading the training of primary health care personnel; b) expanding basic health care interventions known to be cost effective; and c) strengthening the network of health care facilities to support the above activities. This was the centerpiece of the Second Health Project. This component had three sub-components. as follows: - 5 -  a. Primary Health Care Training (US$16.2 million): The aim of this sub-component was to improve the quality of essential health services by upgrading the training of primary health care personnel in the 23 project provinces. The project was to support: a) curriculum development, practice-related learning materials and assistance in revising program evaluation criteria; b) training of trainers; c) strengthening of in-service training programs; d) support in developing follow-up and supervision mechanisms; and e) opportunities for further training in primary health care related areas; b. Basic Health Care Interventions (US$2.9 million): The aim of this sub-component was to improve the effectiveness of essential health services through: a) extension of successful interventions to 23 project provinces (UNICEF-inspired Safe Motherhood and Midwife Training Program, the Acute Respiratory Infection Program, and Child Nutrition and Growth Monitoring Programs); c) support for public education campaigns (TV and advertising campaigns); c) development/ implementation of referral system; d) support for community participation and involvement by Non Government Organizations NGO) in the health sector; and e) Evaluation of cost-effective interventions through operational research; and c. Infrastructure Development (US$136.5 million): The aim of this sub-component was to improve access to essential health services in the 23 project provinces through: a) the refurbishment and upgrading of existing facilities; b) the construction of a limited number (49) of new facilities; and c) the procurement of basic diagnostic equipment, furniture and vehicles. The civil works focus was on the construction, furnishing and equipping of 29 urban in-service training centers, 14 urban health centers, 3 public health laboratories; and 3 cold storages for vaccines. The project was also to support renovation, furnishing and equipping of 4 mid-size hospitals with 200-300 bed capacity, 110 urban health centers, 115 rural health centers and 4 public health laboratories. In addition the project was also expected to equip health posts with midwife kits and refills, equip the largest state hospitals in each province, 4 obstetrics hospitals and 7 other large non-state hospitals in project area with basic diagnostic, acute care and obstetric equipment; and provide 244 four wheel drive vehicles. Component II: Health Policy and Management (US$44.4 million total cost). Aimed to improve the quality of health care management in selected institutions by: a) Upgrading the training of staff in policy making, planning, management, monitoring and evaluation; b) decentralizing and restructuring management responsibility in six pilot hospitals; c) introducing basic management support systems in Provincial Health Directorates and selected hospitals in the project provinces as well as evaluating a more comprehensive management information system in one urban pilot hospital in Western Turkey; and d) conducting pre-investment studies to support broader reform of the health sector. This component had four sub-components. a. Health Services Management Training (US$10.7 million total cost): The aim of this sub-component was to strengthen management capabilities in health care system through: a) Training a critical mass of Turkish trainers in concepts of modern health care management; b) development of continuing education program to upgrade managerial skills of hospital managers, health center staff, health directors in project provinces and Ankara MOH staff; c) providing a rapid management development program tailored for pilot hospital managers; and d) development of a strategy to extend this training to the entire country. It was expected that Turkish and Foreign institutions would train a total of 46 trainers who will be responsible for training 192 hospital staff, 288 people from Provincial Health Directorates, 120 people from the central MOH and 60 people from pilot hospitals. In addition 140 health center doctors and 100 specialists from project provinces were expected to be trained annually for 7.5 years duration of the project. The project was also expected to support 63 staff-months for management support and 524 staff-months for institutional capacity building, 58 staff years in fellowships as well as study tours, teaching materials, - 6 -  audio visual equipment, hardware and software and library resources to establish and strengthen the programs described; b. Hospital Management Restructuring (US$4.5 million total cost): The aim of the sub-component was to improve cost effectiveness, efficiency and quality of hospital services through a) introduction of new organizational structure in six pilot hospitals; b) redefinition of senior management functions in pilot hospitals; c) increasing the responsibility, accountability and autonomy (including financial) of managers in the pilot hospitals. The project was to support the recruitment, selection and development of hospital management teams; establishment of departments responsible for business planning, general management, finance, personnel, clinical affairs and in-service training; the development and introduction of hospital policies and procedures for increased hospital autonomy; the development of accounting systems and pricing mechanisms to improve efficiency of resource use; and development of quality standards for improvement of patient care and the environment; c. Computerized Management Support Systems (US$12.4 million total cost): The aim of the sub-component was to improve cost effectiveness and efficiency of health services through a computerized management information system. The sub component would support: a) introduction of network-based management support systems for hospital management, personnel, payroll, accounting, billing, general ledger, pharmacy, admission discharge and transfer functions to 34 hospitals in the project provinces; b) setting up of comprehensive network-based management support system to integrate financial and administrative management with clinical functions (radiology, hematology, obstetrics, etc.) including providing access to clinical records in one urban hospital in Western Turkey. In addition, the project was to support the extension of the 15 module basic Health Information System (HIS) developed for MOH and provincial Health Directorates under the FHP to the Provincial Directorates of the 23 project provinces. This HIS was expected to enhance overall health care resource management, service delivery and statistics collection. Also to be supported was hospital manager and staff training in concepts and functions of the computerized support systems and also to support development and implementation of application software; and d. Pre-investment Studies and Project Management (US$16.8 million total cost): The aim of this sub-component was to ensure that future reforms in the health sector focus on reducing the burden of ill health due to preventable and treatable diseases and disability in the country. The project was to support a National Burden of Disease (BOD) study using the methodology used in the Bank's 1993 WDR so that the total BOD in the country could be disaggregated by age, sex and disease category according to ICD-10 (International Classification of Diseases) diagnostic codes. In conducting this study, the project would assist the MOH build capacity to undertake such a study and use the obtained data to set health priorities and predict trends in key indicators. The sub-component was also to support cost-utility studies that would help identify affordable cost-effective packages of interventions for preventable and treatable diseases in the country. To strengthen PCU Management capability to expedite project implementation and conduct pre-investment studies activities were planned that would support the provision of staff training, materials and equipment to PCU to support project administration and procurement. 3.4 Revised Components: The Loan Agreement was amended in September 1999 to include support for five additional provinces affected by earthquakes. Given the urgency with which the support for these additional provinces was put together, not much detail on activities to be supported appears in the Project Implementation Report. Project revisions were considered minor and were therefore not presented to the Bank's Board of Executive Directors. Revisions to the project included: - 7 -  l A reduction in the size of the project so that there were fewer activities and reduced total cost, with some of the proceeds going towards the support of earthquake related activities; l Rightsizing the PCU by substantially reducing the number of staff from about 190 at the end of 1999 to less than half that by mid 2000; and adopting an appropriate skills mix by keeping a smaller number of qualified staff that would work in more consolidated teams. The Bank team that came on board in 1999, ensured that the function of the PCU was redirected towards coordination and away from implementation; and most of the tasks of implementation were moved out of the control of the PCU to the MOH and its departments. As the PCU reverted to its original purpose of coordination, only the Infrastructure Development sub-component and project management tasks remained under the PCU. Major improvements in working relations between the MOH and PCU were seen after these major changes within the PCU, it is fair to say that these changes considerably speeded implementation; and l Substantive corrections to the MOH-UNDP/UNOPS Management Service Agreement were made so that the agreement included tighter financial management, a restriction such that only management of procurement of training services was authorized and not of goods as was the case before. For the first time, an audit clause for loan proceeds managed under the MSA was added. These revisions took into account the little progress that had been achieved by the project, the reduction in funds due to the reallocation of US$22.5 million to earthquake related health-care needs, other evolving sector priorities and the feasibility of completing implementation in a reasonable time frame. Project closure date was kept as December 2001, though it was understood that implementation would be completed in three years and that an extension (to December 2002) would be granted closer to this closing date. The project finally closed in December 2004, after three requests for extensions were granted to enable the Government to complete implementation. The first extension was to June 2003, the second to December 2003, and the third and final, to December 2004. According to the Project Implementation Report (PIR) produced for the 1999 revisions, the total cost of the project was reduced from US$200 million to about US$185 million (US$150 million Bank loan remained the same but the Government's contribution was reduced to US$35 million). The revised total cost included US$25.4 million that had been allocated for earthquake rehabilitation activities. After considering various pieces of information about the revised project, one could say that the total cost of SHP components (without earthquake related activities) could be expected to range from a minimum of approximately US$ 125 million to a maximum of approximately US$ 158.8 million. Unfortunately, the PIR only presented the cost of components to be covered by the loan proceeds that had yet to be disbursed; and so for the purpose of this ICR the following revised total costs were obtained by adding up loan proceeds that had been disbursed, loan proceeds yet to be disbursed, Government contributions that had been disbursed as well as the expected Government contributions. Revisions to the Primary Health Care Component. The aims of this component remained basically the same but after revisions the total cost of the component was re-estimated at US$80.6 million, a substantial reduction from US$155.6 million originally estimated, mostly because of reduced activities under the Infrastructure Development sub-component. Reasons and nature of revisions are discussed under the relevant sub-components. a. The Primary Health Care Training sub-component: The revised total cost was estimated at $15.9 million, not too different from the original amount of US$16.2 million. This sub-component was to continue supporting original project activities as described above; b. The Basic Health Care Interventions sub-component: The revised total cost was estimated at US$8.2 million, a substantial increase from the originally estimated US$2.9 million total cost. Though the aim of the sub-component remained the same, activities on community participation and involvement of NGOs in - 8 -  health sector and those related to the development of a referral system were dropped after failure to operationalize community participation, which would have included the establishment of community health committees, training of community health workers; and training of school teachers in selected public health issues. In their place interventions and programs such as Safe Motherhood and Family planning were scaled up, and later, support for immunization campaigns, which were not in the original design, was added. This significant upward revision in the budget for this sub-component is a reflection of a relative success of activities that had been achieved before revisions especially because other sub-components. had yet to see any output; and c. The Infrastructure Development sub-component: The revised total cost of US$56.5 million instead of US$136.5 million originally estimated caused the most drastic change in the estimated total cost of the project. Instead of all the activities that were originally planned, the focus of this sub-component became to provide a locus of training in the 23 poorest provinces through well-equipped functioning Training Health Centers (THCs). These are facilities that would offer accommodation of trainees and provide primary health care services under one roof. The project would also enhance diagnostic capacity in selected hospitals by purchasing selected medical equipment for 23 state hospitals, and 7 maternity non-large hospitals. The large downward revision of the budget reflects the decision that activities in the sub-component had already been severely delayed as was presented under Section 3.2, and so fewer activities were to be supported. Revisions to the Health Policy and Management Component. The revised total cost of US$43.4 million was very close to the original estimate of US$44.4 million. However, though the aims of the component also remained the same as in the original design, there were important changes within sub-components. some of which were rearranged by dropping some activities, merging some and separating others as described below: a. The Health Service and Hospital Management sub-component: This was a new sub-component aimed at consolidating activities of the Health Services Management Training and those of the Hospital Management Restructuring sub-components. Since all the activities of the Hospital Management Restructuring sub-component were dropped (after the Law on Primary Health Care Services and Family Physicians failed to materialize), activities to be supported were those related to the Hospital Services Management Training only which meant training of Provincial Health Directors and Hospital Managers as was described under Section 3.3. A proposal to establish a Flagship Center that would act as a center of excellence for health sector management and training to support health sector reform key studies in Turkey was added. This new sub-component had an estimated total cost of US$7.97 million; b. The Computerized Management and Support System sub-component: The revised total cost of US$14.4 million was an increase from the original US$12.4 million, though the sub-component was to support fewer activities than was in the original design. The sub-component was still to support the development of the CHRMS system, but the extension of Hospital Information System (HOIS) for MOH and 23 project provinces was scaled down in order to streamline the already challenging task of developing CHRMS (HOIS=HIS discussed under Section 3.3c above). Support for HOIS mainly involved purchase of computers and other equipment on which to run CHRMS, this support was extended to all provinces of the country; c. The Studies sub-component: In the original project design, undertaking of at least two key studies as well as project management activities were to be jointly supported under one sub-component titled `Project Management and Studies'. After revisions, the budget for Studies was separated from that of project management mainly to transfer the bulk of implementation tasks out of the PCU and into the MOH. The Studies sub-component's total cost was estimated at US$4 million, this was to finance the BOD study; NHA-CE studies plus any other studies identified during the course of implementation; and - 9 -  d. The Project Management sub-component: The original total cost for the Project Management and Studies component was US$16.8 million, after revisions, the total cost for Project Management (now separate from the Studies sub-component) was estimated at US$16.8 million meaning that the cost of Project Management was effectively revised upward, although the PIR does not provide sufficient information to justify this increase. These funds were to support all the functions of the PCU as described in previous sections; as well as to cover the total fee of US$2.7 million for management functions performed under the MSA between the MOH and UNDP/ UNOPS. Lost opportunity. Project revisions in 1999 provided an opportunity to re-evaluate the added value of Bank's financing for the reduced project, given that expected policy reform activities had failed to move forward. It was also an opportunity to realign activities to realistic development objectives that could be achieved within a reasonable time frame and to bring to life a dormant Monitoring and Evaluation (M&E) system that had so far not been made use of. However, this opportunity was not taken, and the revisions to the project primarily consisted of dropping some activities while keeping some of the most difficult to implement given the lack of capacity in the country and the degree of coordination needed to implement them (such as Infrastructure Development and Computerized Management Support System). In addition, insufficient time was allowed for the completion of the revised components. 3.5 Quality at Entry: The Bank's OPCS guidelines for ICRs specify that this ICR must assess the quality of the design of the project as well as the reasonableness of assumptions about external factors that are likely to have affected implementation and achievement of the project objectives. The Bank's Quality Assurance Group (QAG)'s Quality at Entry (QER) rating for the SHP is not available (because the effectiveness of the SHP predates QAGs QER). However, this ICR rates the Quality at entry as Unsatisfactory for the following reasons: l Poor link between project components and its development objectives (DOs). The aim and objectives of the SHP were clearly defined in the SAR. They responded to poor health in the country and were well aligned with the Government's efforts to improve health status in the country which were consistent with the Bank's 1993 Country Assistance Strategy (CAS) and remain relevant according to the 2003 CAS objectives. The SHP was an ambitious project that aimed to change major health outcomes in 7.5 years, an unrealistic objective. Moreover, the proposed project activities were not only too numerous and unfocused, they were also not well designed to meet the expected outcomes of the project. For example, the link between large investments in hospital and health care infrastructure (70% of the project) and the expected project goals were not sufficiently addressed in the SAR. The existence of low hospital bed occupancy (42%) among populations with extensive health needs among the poorest inhabitants of the country and the proposed further investments in infrastructure was raised in the Staff Appraisal Report (SAR) and by Bank staff who were the peer reviewers of the project. Despite this knowledge, the issue of policy that would adequately address factors hindering equity of access received little mention in project design. On the other side, the project did include important elements of policy reform and of capacity building, demonstrating elements of conflict on how the specified objectives of the project could be reached. Unfortunately, the project was not sufficiently detailed in the purpose of the large amount of training that was proposed or how the risks of high rates of turnover for health sector staff in the 23 project provinces would be mitigated. l The long period between project concept and approval did not improve the final design of the project. Deficiencies in design are probably partly explained by the fact that the SHP's Project Concept Document was produced in 1988 and the project, as conceived then remained basically the - 10 -  same as it was approved in 1994. What this means is that developments in the country and in the health sector in general had little impact on the approved project 6 years later. l The design was not sufficiently informed by health sector research on major factors hindering access to health care in the Eastern Provinces of Turkey. The SHP could have greatly benefitted from analysis on whether access in the poorest 23 provinces was limited by poorly developed infrastructure, distance, quality of existing services, political, socio-cultural factors or cost. Basically the project could have been better designed if analysis on why, despite the availability of hospital beds and other health care facilities in the project provinces, the population still had very poor health outcomes. Instead the design relied heavily on the concept that health can be improved by more investments in the number of health care facilities. l The design was complex and overestimated the capacity of implementing agencies. Though the SHP had only two components, the number of activities was large and included a large infrastructure component aiming to expand access to health care; and a large number of training activities aiming to improve management and delivery of service. All these activities required close MOH-PCU coordination (which at most times was ineffective as was described before), as well as coordination of various key players (eg., MOF, Ministry dealing with land issues) at central and provincial directorates. The project design did not adequately factor in the need to build the capacity of the implementation agencies themselves, which had already been identified during implementation of the FHP and in the SAR as very limited. The design particularly failed to adequately address lack of capacity in procurement of Training and of IT goods and services (infact in the early years of implementation, the Bank itself did not seem to have sufficient capacity in this area). In addition, capacity to monitor and evaluate progress and outcomes was not adequately addressed. l Inadequate consideration of risks and reality. The design did not factor in potential risks of having a large PCU that operated as a separate entity from the MOH, the risks of political instability and many changes in Governments, the reality of bureaucratic processes in the country, limited capacity of local construction firms and the reality of shorter construction work calendar in the Eastern regions of the country; and the risks and consequences of delays in implementation of the IT component. The design was also rather vague on how community participation could be mobilized, probably due to the lack of analysis on how community participation and involvement of NGOs in the health sector would be operationalized (essential to meeting the objectives of the Basic Health Care sub-component). In fact, the project's operational manual gives limited information on how this was to be put in action. In general, there was little participation of stakeholders in the design of this project probably reflecting the fact that the importance of stakeholder involvement in project design was not as keenly appreciated then, as it has become in more recent years. l Lessons from the FHP were noted but recommended solutions were inadequate. The Staff Appraisal Report (SAR) clearly notes reasons for FHP's slow start (for example, the PCU's unfamiliarity with procurement, poor PCU/ MOH capacity to develop and implement large infrastructure and training programs and frequent changes in management of MOH). Solutions identified and expected to make implementation of the SPH smoother, such as continuation of the MOH-UNDP MSA and reliance on a large PCU, did not reflect these lessons. l Monitoring and Evaluation framework described some of the expected outcomes and impact; but it was inadequate in how M&E would be managed. According to the SAR, there were several project outcomes and results that the project aimed to achieve; and for some of these, thirteen 13 very specific indicators, targets as well as adequate baselines were defined as presented in Annex 1 on Key Performance Indicators. Those targets which were specified can be evaluated as very specific and measurable, but they were very ambitious given the expected length of the project. Unfortunately, no - 11 -  indicators were specified for other expected outcomes such as those related to equity of access and improved health care management. The SAR acknowledged the challenges involved in measuring some of these expected outcomes that depended on training and Information Technology Investments; and as a result, these were basically left unspecified, with no baseline and therefore not measurable. In addition to lack of targets for some outcomes, for a number of the indicators, baseline data were not available and the methods for data collection, the scheduling and frequency of data collection were not specified. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The Bank's OPCS guidelines specify that this section must assess the degree to which the project in its entirety achieved its development objectives as specified in the SAR. In undertaking this assessment, particular attention has been paid to make a distinction between the project's performance in using project inputs (for example, specified budgets for the different project components), from project output (training programs, curricula, infrastructure, and all other activities and purchases that were supported by the project); expected project outcomes (better equity of access and health care management); and the expected project impact (lowering the burden of disease, lowering IMR etc). The aim of this assessment is not to evaluate the extent to which the project's implementation met the procurement plan that was agreed to between the Bank and the implementing agency in order to deliver the expected outputs of the project. It is important to note that inputs do not always lead to outputs and that outputs may not necessarily lead to expected outcomes. Moreover, in some cases, even when expected outcomes are achieved, the expected impact might not be forthcoming especially when the impact is dependent on factors beyond the scope of a particular sector-specific project such as the passing of a particular legislation. In the case of the SHP, the design of the project was based on a number of assumptions including the existence of a causal link between project outputs (such as construction of Training Health Centers, medical equipment, training, investments in IT, basic health care interventions, research) and project outcomes (such as improved equity of access to health care and improved health care management) which were then expected to have an impact on IMR, child mortality, life expectancy etc. Based on this project's design, implementation of project activities was carried out on the assumption that project funds had been rationally allocated across project components (for example, that it was rational to allocate 70% of the project to support infrastructure development activities); and that if all activities were completed as designed and in accordance with procurement plans agreed with the Bank, then this would lead to successful outputs, which would mean that outcomes are also successful and that the expected impact would be forthcoming. It is quite grave, that the design of the SHP did not appropriately link the project's expected outcomes and impact from the proposed inputs and outputs. It is also unfortunate that the project did not undergo major revisions or restructuring to correct this deficiency, i.e. the original and revised objectives remained more or less the same; and so this assessment still will rate the achievement of outcomes given the original objectives. While the project did succeed in producing a sizeable proportion of expected outputs, the overall achievement of the project's objectives is rated Unsatisfactory for the following reasons: l The SHP was not designed to meet the expected project outcomes because of the very weak link between project objectives and the bulk of planned activities. In the end, despite the large amount of output from this project, it is likely that their contribution to expected outcomes is, at best, very modest. In addition, it is unfortunate that those important and much needed policy reforms that would have been key in changing the management of health care in the country were later dropped from the project and therefore most of the capacity building activities are unlikely to have added to expected - 12 -  outcomes; l Despite this project's implementation being longer than originally planned, project activities were still not fully completed at project close and so the expected outputs were not achieved. Moreover, because of the length of time that passed between project design and implementation of some sub-components some outputs were no longer very useful by the time they were delivered; and l Significant deficiencies in the original design of the M&E framework, as well as monitoring and evaluation of project outputs and outcomes created a real challenge in the rating of achievement of project objectives, in some cases, even of expected outputs; and of outcomes. This was especially challenging for measuring the outcome and impact of training activities and of Information Technology investments. The achievement of objectives is therefore rated against the degree to which outputs that should have contributed to goals are seen to have done that. Rating of project aims and goals was considered using available data that was amenable to evaluation of trends in selected health indicators for the period 1993-2003, which covers most of the duration of implementation. Objective I: Improvement in equity of access to essential health services in 23 Eastern, low income priority provinces (and 5 provinces affected by the August and November 1999 earthquakes)*. The achievement of this objective is rated Unsatisfactory for the following reasons: l The improvement of access to health services in the 23 project provinces was given a very high priority in the project's SAR. After the revisions of 1999, this was expected to be achieved partly through the construction of Training Health Centers which were designed to provide training facilities to provincial health sector staff and also to provide accommodation for those attending training. In addition these buildings were designed to also offer primary health care services to the population living in the neighborhood of the THCs. At project close, the construction of 5 of the 23 THCs was incomplete and the use of those completed THCs as centers of learning or as health care centers had yet to begin. Given the faults in how this project was designed, it is debatable whether activities that were expected to contribute to better equity of access are really going to have the desired effect despite the hard work that went into completing them. On the positive side, improved access to faster diagnosis through the purchase of diagnostic and pediatric equipment is likely to have contributed to improved access to timely diagnosis of major health concerns in the 23 project provinces. Visits to 5 state hospitals during the ICR mission noted that purchased equipment was being used and staff appreciated the benefits of having modern diagnostic equipment within proximity; and l One of the main factors hindering equitable access to health care in the project provinces was the unavailability of health care providers in these areas (See section 3.1 above). Training supported by this project was expected to alleviate this shortage. Unfortunately, a large number of staff that received training did not stay in the project provinces and therefore their contribution to this particular objective was probably minimal. The fact that staff that had received training moved and that benefits might have accrued to provinces other than the 23 that the project aimed to support, might not necessarily be considered a bad outcome or fault of the project, although it cannot be stated that this was positive for the project provinces. On a positive note, reports of field visits to the provinces confirm a great appreciation for this training on individual staffs' development. *Though a thorough analysis of earthquake related activities supported through the MEER Project will be undertaken when the project closes in 2006, it is likely that upon assessment of the MEER project, some positive contributions in improving access to essential health services in the 5 provinces affected by the earthquakes are likely to be attributed to the SHP. The project has so far supported drafting of the National Mental Health Policy; strengthening of emergency health stations; rehabilitation of - 13 -  hospitals, construction of health centers; and purchase of medical equipment and furniture for damaged health care facilities. Objective II: Improvement in quality of health care management in selected institutions. The achievement of this objective is rated Unsatisfactory for the following reasons: l Health sector management reforms did not materialize. The project supported significant quantities of training of managers but the benefits might not have been directly beneficial to improving the quality of health care management within the central MOH or in the 23 Provincial Health Directorates. Indirectly, these trainings could have acted as non-monetary forms of compensation to health sector workers in the project provinces, but in absence of the necessary health sector reforms, beneficiaries continued to work in the way they had always worked as was learnt during the ICR Mission; and l Expected improvements in competence and cost effectiveness of health services expected from Computerized Management Support System have yet to be realized. For reasons that will be discussed in the following sections, a large part of the CMSS is not operational and though the project did fund training on general computer literacy and to a smaller extent training of development, support and implementation of application software, a high proportion of IT staff did not stay long in the employment of the MOH. Other project goals and aims: To reduce the burden of disease due to avoidable illness and disability in Turkey; and to improve health status, reducing infant mortality rates, decreasing fertility rates and improving life expectancy in the participating provinces. It is not possible to rate the achievement of the project's primary goals and aims because: l No baseline Burden of Disease (BOD) data exists for Turkey. The 1993 WDR which was extensively used in project design, presented the BOD for broad regions of the world and not for specific countries. Data from the BOD study, conducted through the support of this project, shows that Turkey disease burden is approximately 10, 802, 494 DALYs. However, one can neither say how the BOD has changed in the 10 years of implementation or quantify how the project's activities have played a role in whatever changes there might have been during this time; and l Though the SAR provided a baseline of various health outcomes for provinces, there was no evaluation strategy for data collection for the 23 provinces during the course of implementation meaning that no direct attribution to project activities can be made. Moreover, for other outcomes evaluation design was weak in that monitoring and evaluation questions were vague and not well articulated. Without suggesting attribution to the SHP, especially since the contribution of this project to the overall yearly MOH budget was very small, trends in various outcomes for 5 regions (grouped provinces) of Turkey between 1993 and 2003 were compared. These trends were not drawn from data collected under the project's M&E system; they are only meant to demonstrate what has happened in the country in the 10 years when the project was implemented. Demographic and Health Survey (DHS) data, not surprisingly, show change in some of the health indicators with specified targets in the SAR over the course of the last 10 years. The East (with 22 of 23 SHP provinces as well as 3/8 of FHP provinces) and North regions are some of most deprived in the country; these and others, were compared to the better-off West region. All other things being equal, it is reasonable to expect that those regions comprising of project provinces should, at the minimum, not have experienced a worsening of health outcomes in 10 years. More information on DHS data, the methods used in analysis and the targets that were specified in the SAR is presented in Annex 10. - 14 -  1. IMR has dropped from about 53 per 1000 in 1993 to about 29 per 1000 in 2003; and under-5 mortality has dropped from 61 per 1000 in 1993 to about 37 per 1000 in 2003 but regional differences have worsened. Though IMR has declined, for the poorest provinces, IMR is almost at levels experienced by better-off regions a decade ago; and the West-East, West-North regional gap in IMR might have increased between 1993 and 2003. Figure 1: Regional Trends in Infant Mortality Rates 1993-2003. 7 0 1 9 9 3 1 9 9 8 2 0 0 3 6 0 000 5 0 per Rate 4 0 3 0 Mortality Infant 2 0 1 0 0 W e s t S o u t h C e n t r a l N o r t h E a s t T u r k e y Target in SAR was to lower the 23 project provinces' IMR by 40% and U5 mortality by about 23%. The largest drop in IMR has been experienced by the West, South and Central regions. After 10 years, not only are these declines short of the SAR target, actually when East and North are compared to the West region, the difference in child deaths as measured by IMR and U5 mortality seems to have grown. 2. TFR has declined marginally from about 2.7 to about 2.2 children per woman; the largest drop was in the North, Central and East regions. Contraceptive prevalence rate increased from about 63% in 1993 to 71% in 2003, the increase was quite large in the East region where it increased by about 37% (to reach 58% in 2003) compared to 3.8% for the West (to reach 74% from an already high 72% in 1993). The increase in CPR for the East region was above the SAR target which was to increase CPR by 30% and lower the 23 project provinces' TFR by 30%. After a decade, the gap in CPR between the West and the East region remains about 16 percentage points. Figure 2: Comparison of the Gap in IMR between the West and other regions. - 15 -  3 0 2 5 region 1 9 9 3 1 9 9 8 2 0 0 3 West 2 0 the to 1 5 1 0 compared when 5 IMR 0 S o u t h C e n t r a l N o r t h E a s t Excess - 5 Figure 3: Percentage change in IMR, U5M, TFR, Full vaccination rates, HHDR, CPR and ANC rates between 1993 and 2003. 9 0 W e s t S o u t h C e n t r a l N o r t h E a s t T u r k e y 8 0 7 0 A N C 6 0 5 0 1993-2003 C P R 4 0 3 0 outcomes 2 0 1 0 I M R U 5 M T F R F u l l v a c . H H D R selected 0 in - 1 0 - 2 0 change - 3 0 - 4 0 Percentage - 5 0 - 6 0 - 7 0 - 8 0 3. All regions saw improvements in the proportion of births that are delivered in health care facilities. Nationally, the ratio of percentage of births delivered at home (HHDR) compared to those at a health care facility dropped from about 0.67 to 0.27 between 1993 and 2003. Target in SAR was to lower the 23 project provinces' Home Hospital Delivery Ratio (HHDR) by 10%. The smallest change has been in the West region, where the proportion born at health care centers was already highest in 1993; and largest in the Central and North regions, but the ratio in the East still remains far below the national average even after a 64% improvement in 10 years. 4. The percent of women who received antenatal care also increased from 63% to 81% nationwide. Target in SAR was to increase antenatal and postnatal contact rates to three contacts. DHS data did not allow for analysis of number of ANC contacts, available data allowed for analysis of general contact rates, - 16 -  these show that the most significant change in ANC coverage has been in the East region where the proportion of women who received ANC from a low level of 34% in 1993 (compared to 86% in the West). Despite this improvement, the gap compared to the West region remains 30 percentage points (lower from the 52 percentage points gap in 1993). 5. The percent of children fully vaccinated decreased from 65% to 54%. Target in SAR was to increase immunization rates for tetanus, polio and measles to 95%. This analysis could only obtain comparable data on full immunization and these data show that there has been a worsening of full coverage of children against these major diseases. All regions experienced declines in proportion of children that were fully vaccinated but the most severe decline occurred in the South, followed by the West regions. In light of vaccination campaigns that are discussed in the output section of this ICR, it is very difficult to judge whether or not those campaigns had any effect in minimizing the size of the decline in the East and North region. 4.2 Outputs by components: This section will provide a component by component evaluation of the achievement of their specific objectives as stated in the SAR, paying particular attention to whether the project was appropriately designed to be able to achieve its objectives and whether there was sufficient criteria set for measuring the achievement of objectives. Component I. Primary Health Care (Original total cost: US$155.6 million, Revised total cost: US$80.6 million and Actual total cost: US$80.58 million). ICR Rating: Unsatisfactory. The objective of the Primary Health Care component was to improve equity of access to essential health services in the 23 low income priority provinces. The achievement of this objective is rated as unsatisfactory partly because the link between activities that the project supported and the expected outcomes was very weak to begin with. In other words, the project as designed is unlikely to have led to the expected outcomes. In some cases, the achievement of the objective is hard to evaluate because no indicators for how outcomes should be evaluated exist. Of the three sub-components under this component, only the Basic Health Care Interventions sub-component is evaluated as a successful contribution to the objectives of the component and of the project, as is discussed below. The project supported an impressive quantity of training activities under the Primary Health Care Training sub-component, but its contribution to improving equitable access to essential health services and improving capacity at selected health care facilities is very difficult to judge. Part of the Infrastructure Development sub-component remained incomplete at project close, nevertheless, given the weak link that can be made between the need to construct the THC as a way of improving access to essential health services in the provinces, it is very probable that the contribution of this sub-component is very modest at best. Analysis by sub-component a. Primary Health Care Training (Original total cost: US$16.2 million, Revised total cost: US$15.9 million and Actual total cost: US$ 12.7 million): Training output from this sub-component is considered partially successful in contributing to project objectives of improving access to health care because training is likely to have better prepared new health care workers for the new working conditions in the Eastern provinces. The PCU/ MOH estimates that as many as 27, 000 people might have been trained by this project under this sub-component. The project is also likely to have built capacity, especially at the School of Public Health (SPH), in preparation and coordination of these types of training. However, given the high staff turnover in the project provinces, the contribution towards better equity of access from those who received training is hard to measure. Difficulties in measurement of achievements from this sub-component - 17 -  are exacerbated by the fact that while post-training questionnaires were filled by course participants, these largely focused on participants' satisfaction, and not on working skills or practices making it impossible to comment on the impact that such training would have had. The project supported two types of basic health services instruction to health personnel: l Adaptation Training: This was targeted at first level health personnel new in their posts, newly appointed to the province or appointed to MOH from other public institutions, usually on the agreement to be employed in first level health institutions in deprived provinces. Training ran for 10 working days and included the following topics: characteristics of the province regarding health services; legal arrangements regarding health services and workers; functions and management of village clinics; basic principles and techniques of health service planning; introduction to epidemiology in regional health management; participation of public in health services; and health programs. According to the PCU/ MOH, there were 12, 749 health personnel who received this type of training. l Career Development Training: This was provided to ensure personnel had the knowledge and capabilities to improve efficiency in provision of health services. These were organized by the Provincial Health Directors and Provincial Training Teams, training was targeted at first level health centers. Training included topics such as: environmental health; family health; mother health and planning; child health; mental health; training on health issues encountered frequently in the provinces; general principles in control of contagious diseases; diseases preventable by vaccination; respiratory system infections; diseases transmitting via water and foods; sexually transmitted diseases; zoonotic diseases; tropical diseases; chronic diseases frequently encountered in the first level of healthcare; first aid services and emergency interventions; preventive eye health; mouth and teeth health; and nutrition. According to the PCU/ MOH, there were 14, 349 people who received this type of training. Training was a result of collaboration between MOH Headquarters, Provincial Health Directorates (PHDs) and Universities (including a critical role played by the SPH) to develop training materials. The project supported the creation of Provincial Training Teams and Central Training Teams at the MOH; and, eventually these acted as Trainers of first level health personnel in the provinces. Training Teams received "Train the trainers" instruction once every two years aimed at improving their training capabilities. b. Basic Health Care Interventions (Original total cost: US$2.9 million, Revised total cost: US$8.2 million and Actual total cost: US$7.4 million): This sub-component is evaluated as having made an important contribution to better basic health care interventions by supporting key immunization campaigns, family planning efforts; and by supporting the drafting of various drug protocols, in addition to providing training to MOH staff and supporting important public information campaigns. Not only are all these output likely to have contributed to the status of public health in the project provinces, through this sub-component, the project also supported production of protocols that will have lasting utility in the health sector for years to come. Specifically the following activities were supported by the project: l Rational Use of Drugs. The project supported studies and training in rational drug use concepts. Approximately 160 persons participated in "train the Pharmacotherapy Trainers" sessions and approximately 14 publications were produced. In addition, the project supported Rational Drug Use studies which produced important findings on how significant savings for health care budgets can be obtained through more economic use of pharmaceuticals; prescription assessment studies and some research using patient registry books as well as direct recording of prescriptions in pharmacies was undertaken in two project provinces (Bayburt and Bolu); Training for Provincial Training Teams who were expected to train Rational Drug Use concepts to new personnel as part of routine training sessions; and collaboration of academicians in ensuring that the Rational Drug Use concept is a part of - 18 -  medical school training and was supported by the Higher Education Board recommendations on the use of the designed training program in universities medical faculties. l Development of "Diagnosis and Treatment Guidelines for First Level Health Institutions". These provide guidelines to be used in identification and solution of frequently observed health problems at first level health institutions. Fifty-thousand (50, 000) copies of the "Diagnosis and Treatment Guidelines for First Level Health Institutions (2003)" have been printed and distributed to doctors providing first level health services. l Vaccination campaigns to eradicate measles. Though this activity was not originally included in the original project design, it was probably one of the few successful outputs of the project. Working with the MOH, and other partner organizations (WHO, UNICEF, CDC), the Bank supported purchases through UNICEF of 28 million doses of measles vaccines, 34 million doses of combined diphtheria-pertussis-tetanus (DPT) vaccines; Equipment and materials (vaccine carriers, disposable safety boxes, refrigerator monitors, temperature data loggers, syringes, used-syringe safety boxes, ice boxes, refrigerators, freezers, cold chain monitors and laboratory kits); Development and organization of formal training seminars, educational meetings and through the provision of guidelines, action plans, training material on vaccine management, adverse event management, vaccine logistics, cold chain management, and injection safety; and part of MOH's public campaigns on immunization. l Public education and health promotion programs: The project supported studies on maternal and child health, malaria, tuberculosis and cancer; procurement of family planning methods (26.4 million condoms; 396, 000 IUDs); and material development and printing, and training of health workers and the public. It is unfortunately, not possible to link the outputs from health promotion programs or vaccination campaigns to the large increase in contraceptive use or in the decline in full immunization that were presented in the previous section. It is understood that various attempts have been undertaken to compile and analyze data linking project activities under this sub-component with the expected health outcomes. It is also the case that the Government, the World Health Organization (WHO) and UNICEF have conducted analyses on immunization coverage and associated issues such as injection safety. It is unfortunate that all such data have not been properly brought together so as to ascertain the impact of investments made under this sub-component with, for example, fewer disease outbreaks, or with program design and delivery. There has also been no published reports on the impact of health promotion exercises such as those on safe motherhood and family planning. Despite this absence of concrete evidence on impact, activities delivered under this sub-component required a real uptake of new ways of delivering basic health programs in the supported provinces, it is for this reason that the sub-component is seen as a successful contribution to objectives of the project. c. Infrastructure Development (Original total cost: US$136.5 million, Revised total cost: US$56.5 million and Actual total cost: US$60.48 million): At project close, the project had supported the construction of Training Health Centers (THCs) and the equipping of project provinces' state hospitals, as well as maternity hospitals with diagnosis and pediatric equipment and vehicles. Overall, however, because of the poor link between the objectives of the component and the bulk of the output that were planned (largely the construction and equipping of THCs), the contribution of outputs from this sub-component to the objectives of the project (i.e. to improve equity of access to health care) is considered marginal. The output supported by this project included: l The construction and equipping of 18 out of the expected 23 THCs . The ICR Mission observed that these buildings are spacious (3, 391 m2) four floor buildings, they include a basement, two floors - 19 -  intended for clinical health care purposes and two additional floors intended for training and accommodation facilities for trainees. All the buildings are of identical proportions though they will serve very different catchment areas, ranging from populations of about 35, 000 in Bayburt to 130, 000 in Erzurum. Some of these buildings are in provinces with high seasonal migration and some are in provinces which already have a large stock of unused health centers, such as Bayburt. The contribution of additional facilities in some of these provinces to the improvement of equity in access to health care is hard to evaluate. l The project supported purchase of equipment for 23 existing State Hospitals and 7 maternity hospitals including 795 pieces of medical equipment including Anesthesia units and ventilators; Equipment for to support surgery and electro medicine procedures; 100 Different pieces of laboratory equipment; Computed tomography equipment; Magnetic resonance imaging; Diagnostic ultrasound equipment; Pieces of Neonatal therapy and care equipment; Monitoring and emergency equipment; 34 X-ray imaging equipment; 23 cross country vehicles; and 23 minibuses. A full list is available in Annex 9. Several reasons led to the less than successful implementation and output from this sub-component. These include the initial delay in starting construction caused by the Government's decision to postpone activities until Project Year 3; delayed bidding of construction work to June 2002 meaning that construction began in September 2002 (i.e. 8 years after effectiveness); the dispute between the MOH and the construction firm supervisor causing a year's delay after MOH suspended construction while it undertook investigations for irregularities in contracts; poor capacity of some construction firms which resulted in a very slow pace of working and work of unacceptable standard and the fact that some firms went bankrupt midway through construction activities. Part of the reasons for incompletion of the THCs is also due to the unrealistic time frame for completion that was set after the 1999 revisions, the timetable did not take into account the shorter working calendar caused by harsh winters in the Eastern provinces. The new project closing date basically meant that construction was expected to be completed within 12 months of work, since work could only be done in 6 months of the year. Component II. Health Policy and Management (Original total cost: US$44.4 million, Revised total cost: US$43.4 million and Actual total cost: US$42.2 million). ICR Rating: Unsatisfactory. The aim of this component was to improve the quality of health care management in the MOH and selected institutions. Output from this component is considered to have had limited success in contributing to the improvement of quality of health care management in selected institutions because: i) legislation that would have changed the system within which managers work and hence made the training output beneficial to health sector management did not materialize; and ii) an important sub-component which was expected to improve the management of health care (CMSS), though considered complete by MOH, still has major shortcomings as will be described below. It is unfortunate that for all the impressive quantity of training supported by the project, this might not necessarily have improved management of health care in the country. On the positive side, the project successfully supported the Studies sub-component as described below. As with previous subcomponents, neither the SAR nor the PIR provided yardsticks for measuring output from training or IT investments, it is only possible to measure achievement against what was the expected contribution to the objective. Analysis by sub-component a. Health Services and Hospital Management (This sub-component was created after the 1999 revisions with an estimated Total cost of US$7.96 million. Its Actual total cost was US$8.22 million). - 20 -  This sub-component is considered unsuccessful in improving the management capacities of provincial health managers and hospital managers because: i) key legislation necessary for changes in hospital management systems was not enacted; and ii) high staff turnover in the project provinces. The project supported Health Services Management Training for 148 Provincial Health Directors, Deputy Provincial Health Directors and MOH staff who participated in the Health Services Management Training. Health Services Management Training was also provided to 372 Chief Hospital Physicians, Hospital Managers and Chief Nurses of hospitals with 50 bed capacity. ICR Mission interviews with beneficiaries of training revealed a mixed view of the relevance of the training received, in terms of content and relevance to the Turkey situation, and also that in absence of policy reforms the training received did not change the way they worked. But the training was described as a useful incentive to workers in deprived provinces. b. Computerized Management and Support System (Original Total cost: US$12.4 million, Revised total cost: US$14.4 million and Actual total cost: US$11.3 million). This sub-component was not as successful in improving cost effectiveness and efficiency of health services as would have been expected. Implementation of this sub-component suffered significant initial delays caused by inadequate initial systems analysis, insufficient consultation with would be users of the proposed new system, lack of internal support for the Human Resource Management (HRM) cluster; and also due to poor performance from a subcontractor. As a result of significant delays in completing software related activities and the deployment of the Computerized Human Resource Management System (CHRMS), the system's usefulness was bypassed by other IT developments within the Government. The project supported: l Development of a Computerized Human Resource Management System (CHRMS) which involved four sub-clusters described below: Financial Management Cluster was designed to enable development of plans and to enable effective allocation and control of financial resources. By project close the software application had been developed but the application has not been used as was envisaged in the original design. One of the reasons is that the Ministry of Finance has adopted its own budget software which is used by all Government agencies including the MOH. It is not clear whether this software application will ever be used in the future. Materials Management Cluster was designed to centrally keep track of material movements, stocks, fixtures, medical equipment, vehicles, general resource management and maintenance processes. By project close the application software had been developed but the application is not being used. The main reason for this is that all public health institutions including the Provincial Health Directors can now procure their own requirements without the need to go through the MOH, meaning that there is no longer a necessity for the MOH to centrally keep track of materials and stocks movement, a development that was not foreseen at the systems requirement stage. Human Resources Management Cluster was designed to keep track of personnel movements, payroll and accruement related transactions. This sub cluster initially received little support because of the fear that personnel decisions will no longer be influenced by individuals and eventually required personal intervention of a high level MOH official to become operational. By project close, this was the only cluster that is at least partially useful to the MOH though the system still needs further customization to meet new and additional HR and IT requirements of the MOH. Drugs and Pharmacy (Regulatory) Information Cluster was designed to perform functions of the General Directorate of Drug and Pharmacy at MOH, and that of the Division of Pharmacy at PHD's. By project closure the software application had been developed but was not being used, and at the time of ICR preparation the software did not contain any useful data that could assist in performing functions of the relevant directorate. l Purchase of Hardware: The project supported the purchase of equipment for operations in the central - 21 -  MOH and in the provincial health directorates. This included: 523 workstations, 80 printers, 11 workstations and 3 printers for each Provincial Health Directorate (PHD) as well as equipment to support computer networks including power supplies, cables etc. l Training in Information Technology: The sub-component also supported training of 5, 230 MOH staff in IT areas of general computer literacy; office automation; system operation; network management & maintenance; system management & maintenance; human resource management system; finance resource management system; material resource management system; and drug & pharmacy information system. Part of this training was on the 4 sub-clusters of the CHRMS, of which only 25% is now used by the human resources department of the MOH. Interviews with MOH's Department of Data Processing indicated that despite the large amount of training it is hard to gauge the impact of these training activities on management capacity at MOH since many of those trained in IT support, development and implementation of application software had left the employment of the MOH. At the time of the ICR, the DIP which is responsible for implementation of the CMSS is strongly supported by IT consultants, a situation that raised serious concerns as to how CHRMS revisions will be undertaken and how investments in IT are in general supported within the MOH. c. Studies (This sub-component was created after the 1999 revisions with an estimated Total cost of US$4.2 million, the Actual total cost was US$5.5 million). This is by far the most important contribution of the project. The project successfully supported two key studies that are already providing useful information to the Government, MOH and other researchers. Though attempts to prepare for implementing activities in this sub-component are mentioned early in project implementation (1996), the work really started after the project was revised in 1999 and it was a major feat that was accomplished in the last 2-3 years of the project and required commendable effort from the School of Public Health which oversaw the whole exercise of coordinating data collection and processing. These studies are making, and will continue to make, extremely useful contributions in the decision making process by the Government and MOH. In most measures, activities of this sub-component has built capacity to undertake such a study, to use the obtained data to set health priorities and predict trends in key indicators and also to identify affordable cost-effective packages of interventions for preventable and treatable diseases in the country. l National Burden of Disease Study. The National BOD-CE Study is already providing YLL, YLD and DALY values calculated using GBD 2000 standard, differentiated by age, gender, region, rural-urban differences and cause of death, as well as projected values for 2010 to 2030. HALE values by age group and gender were also calculated. A household survey was conducted (11, 481 households) to record incidences of deaths (ICD 10 system) in the past one year which identified preventable deaths and the preventable burden of disease. Data for the calculation of Cost-effectiveness of selected health care interventions from 29 health institutions. Estimates were made for 38 interventions and using these, 9 basic service packages were developed. l National Health Accounts. The project supported the collection of the first NHA data base for 1999 and 2000. A two-phase representative household survey that identifies household health expenditures was conducted; questionnaires and surveys were designed and applied in order to obtain data on the health accounts of 152 institutions of those with 742- bed capacity. The data was collected using the recommended OECD approach and meet policy requirements of the country. The data is a commendable effort of collaboration between the School of Health Administration, Hacettepe University, Bigiktas, The Ministry of Health, and the Harvard School of Public Health. l Capacity Building. In order to build capacity outside the School of Public Health, the State Institute of Statistics (SIS) has been entrusted with the tools for future NHA data collection tasks. This will also enable future efforts to conduct NHA data will receive financial support through the national budgetary - 22 -  process. In addition, the following capacity building activities were undertaken. Two training sessions (2 weeks and 1 week long) were held for Committee Members of all Health Organizations; 60 participants from various units of the Ministry of Health and from other sectors ( Ministry of Internal Affairs, State Statistics Institute, State Planning Organization; Treasury, Pension Fund, Social Security Agency for the Self-Employed, Social Security Agency, Ministry of Finance, Universities) attended a 2-week training session on BOD-CE which also included the use of DISMOD taught by WHO officials; 176 voluntary personnel from Provincial Health Directorates received "Remote Training Program" on BOD-CE; and 100 people, including Provincial Health Managers from 81 provinces, attended a 1 week workshop on the methodology and results of the BOD-CE study. d. Project Management (This sub-component was created after the 1999 revisions with an estimated Total cost of US$16.8 million and the Actual total cost was US$17.2 million). The evaluation of the outputs from this sub-component is based on the overall management of the project from its effectiveness in 1995 to its closing in 2004. The Bank acknowledges that management of the SHP can be clearly distinguished between a pre-1999 phase when coordinating and management of project implementation was highly ineffective; and a more effective post-1999 period where there were significant improvements, especially in the last few years of implementation. l Up until the 1999 revisions, there is little demonstrable output from the SHP. Primarily due to the ineffective functioning of the PCU that had been entrusted with the responsibility of coordinating project activities, as well as financial management and procurement. Despite having close to 200 staff (this included consultants paid under the MSA and civil servants), most of the few activities that the project was supporting were being undertaken under the MSA because the PCU had actual little capacity in planning activities, procurement, accounting or in financial management. Consultants employed at the PCU were implementing project activities including engagement in academic studies and health policy formulation, functions that were more under the realm of the MOH. Over the years, a rift between the MOH and PCU had developed to a level that significantly slowed down implementation when cooperation with MOH dwindled with frequent changes of Government and the resulting frequent PCU management turnover (the SHP probably saw 11 PCU coordinators in 10 years of implementation, 9 of these in the first 5 years). l When a new Bank team took project leadership in 1999, they found an overstaffed PCU with an inconsistent quality of skilled staff. Project accounting was poor, there was inadequate preparation of annual work programs and preparation of bidding documents and procurement took too long. After the 1999 revisions, implementation responsibilities were transferred to MOH units, keeping only the Infrastructure Development sub-component under the PCU, the function of the PCU was redirected back to coordination, the number of staff was reduced and as a result the performance of the PCU significantly improved. Management became more effective, especially in functions such as project reporting which became adequate, but audit reporting and financial management generally remained problematic mostly due to high turnover of financial management staff. The post-1999 management also coincided with more stable Governments and, as a result, better continuity of MOH and PCU management. Despite these major improvements, the PCU's performance in monitoring and evaluation of outputs and outcomes, i.e. in defining and identification of baseline data, scheduling of data collection and in identifying responsibilities for ensuring data availability of measurable indicators) remained weak as it had been throughout implementation. 4.3 Net Present Value/Economic rate of return: N/A 4.4 Financial rate of return: - 23 -  N/A 4.5 Institutional development impact: It is very likely that the health sector in general benefited from the large volume of training supported by this project as was presented above. In addition, the project's support of the Studies component did build the School of Public Health's capacity to coordinate the design and data collection of two important areas of health management. This capacity is now being transferred to the State Institute of Statistics (SIS) which has been entrusted with the tools for NHA-CE data collection. Though the project supported a significant quantity of training, its institutional development impact is rated as Modest for a number of reasons: l Training was not supported by relevant policy reforms and was diluted by high staff turnover. There are various estimates as to how high the staff turnover rates are for the 23 project provinces. The Bank's documents puts this rate at about 50%, ascertained at the time of a 2002 Bank mission which aimed to explore M&E opportunities. This ICR's preparation interviews (March 2005) in the 5 visited provinces estimated that of those that received training about 70% had left the province (it could not be estimated how long the staff that left had stayed after they had received training, this might have given a better perspective on this issue), a primary health care service survey in 2002 estimated that staff turnover rates were about 25.4 % for physicians, 16.1% for midwives, 18.7% for nurses, 22.8% for health officers and 14.0% for environmental officers. Similar concerns are relevant for the CMSS sub-component which trained a large number of staff in Information Technology topics. l Variations in quality of training received. Training beneficiaries interviewed during the ICR Mission had very different views on the quality of training received; many considered training as a perk for a challenging job rather than a way of improving the way they have always worked. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: l Earthquakes: The 1999 Marmara Earthquake caused the Government an estimated US$1.8-2.2 billion fiscal burden. This diverted government resources and efforts to emergency reconstruction resulting in sharp decline in government contribution to SHP by about US$15 Million. l Poor capacity of contractors: Part of the delayed implementation of construction activities can be explained by poor capacity of construction firms, a situation that delayed construction and in some cases necessitated the rebuilding of some Training Health Centers. Implementation was also delayed when some construction firms went bankrupt causing major delays as strategies to bail them out were figured out. l Weather: In the crucial last stages of implementation, the construction of THCs was also delayed by the uncompromising winter weather of the Eastern regions where construction firms' capacity meant that construction could take place for a maximum of 6 months of the year. l Security: The war in Iraq during 2003 negatively affected the Turkish economy and diverted the Government's attention. Actually, the war with Iraq was given as a valid reason to extend the project for the second time, since it was envisaged that an inflow of refugees into the Southeastern parts of the country could stress health care needs in the region. In addition, the special security situation in the eastern part of the country might have hindered implementation and as a result could have affected the final achievement of outcomes. 5.2 Factors generally subject to government control: l Financial Crises: Turkey went through several financial crises in the life of this project. Not only did the resulting tight fiscal measures cause constant delays in counterpart funding but economic crises - 24 -  severely affected personal incomes of the population which are likely to have contributed to worsened access to health care. l Cumbersome government procedures: Project implementation was considerably delayed, in some cases months were added at crucial stages of implementation because of lengthy Government procedures. For example, the 2002/ 2003 directive mandated that all payments, even those from the Loan Account, be approved by the MOF in addition to all approvals required by the MOH. This significantly delayed approval of Letters of Credit for procurement of vaccines, and actually delayed delivery of vaccines by about 16 weeks. Another bureaucratic hurdle was caused by differences between Turkish and the Bank's procurement procedures which led to additional lengthy reviews and internal approval procedures. l Management Service Agreement: The UNDP/UNOPS-MOH MSA was an expensive arrangement; and while it was an innovation and among the pioneers in using alternative project delivery systems in order to speed implementation, the lack of transparency, its cost and the fact that it did not build capacity, significantly diminished its value. l Political instability: Frequent change of power and weak coalitions led by minority parties in the 1990s was not a conducive environment for public policy reform. Consequently, in this uncertain environment, bureaucratic caution and controls increased, and political leadership changes resulted in long periods of inaction, policy reversals and highly disruptive personnel changes at the PCU and MOH. Political instability also partly contributed to the lengthy judicial processes, a situation that was extensively discussed by all past and current staff at the PCU as having negatively affected their ability to make decisions. l Wavering government commitment and shortage of champions: While the Government showed strong will to continuously extend the project and to keep the Bank engaged, its commitment to meeting the project aims and goals in a reasonable time frame was severely lacking. This project significantly suffered from lack champions for reforms necessary to implement key components such as the Hospital Management Restructuring and initially the computerization of the management of human resources. 5.3 Factors generally subject to implementing agency control: l Size and structure of the PCU. The PCU became a large, overstaffed implementation agency with poor capacity to actually implement project activities despite its size. There was large overlap in functions performed by highly paid consultants and staff who performed duplicated tasks, a situation that was very cost ineffective. l Failure to avail qualified Financial Management, Procurement and Management Information Systems staff. There were long periods where the PCU did not have qualified financial management staff resulting in poor internal financial control. Part of the reasons for poor output from the CMSS subcomponent can be explained by lack of IT staff qualified in MIS within the PCU, a shortage that was exacerbated by high rate of staff change which the PCU was not able to keep up with. This situation precluded continuity and contributed to delays in implementation. l Monitoring and Evaluation. The PCU did not ensure availability of staff that could monitor and evaluate project output and outcomes. Despite the availability of a project baseline and the multitude of data sources that could have been used to track progress, this remained one of the weakest components of the project. 5.4 Costs and financing: - 25 -  The 1999 revisions reduced the total project cost from US$200 million to US$185.00 million (US$ 150.00 million Bank loan remained the same, but the Government contribution was reduced to US$ 35.00 million). The total project cost was further reduced in August 2003 when US$ 22.15 million of the loan was cancelled following a request from the Government citing country reasons which had led it to believe that this portion of the loan would not be exhausted by project close. With this cancellation, the Bank's loan was reduced to US$127.85 million, meaning that the total expected project cost (inclusive of support for earthquake rehabilitation) was effectively reduced to US$162.9 million, i.e. 81.5% of the original amount. At project closure, the actual total project cost amounted to US$131.72 million (this being 66% of the original estimated cost of US$200 Million); about US$112.48 million of the loan amount had been disbursed (about 75% of the original loan amount) about US$9 million remained undisbursed and 55% of the original estimated Government contribution had been disbursed by project close. As will be seen in Appendix 2, the Government's total contribution was 66% of the revised expected contributions and for various components its contribution was actually much higher than was estimated at the revisions 'reappraisal', though the dollar amounts of the contributions were indeed very low. The discrepancy between the original loan amount of US$150 million against the disbursed (US$112.4 million) plus cancelled (US$22.15 million) plus undisbursed (US$9 million) amounts to about US$6 million. The Bank understands that this discrepancy could be explained by the various currency changes that the Loan accounts were subjected to in the early years of the project. The project's loan accounts were first held in US dollars, then changed to Deutsche Marks and later changed to the Euro. A number of reasons led to this large difference between the originally estimated total project cost and the actual project cost. This includes: the very slow start up that necessitated revisions where a number of activities were dropped from the project, macroeconomic turmoil that affected Government contributions during the early years of implementation and the earthquakes that led to reduction of the SHP budget to fund earthquake reconstruction. In addition, the actual project cost is less than was expected because of lack of capacity in procurement of certain medical facilities which led to them not being procured (this was the case for Magnetic Resonance Imaging equipment which was expected to be procured under the Infrastructure Development subcomponent). At the time of ICR preparation, Turkey has already repaid the Bank US$ 63.8 million (more than 50% of the loan amount). The repayment started in February 2000, about the time of project revisions when little of the project's output could be shown; and in this year the Borrower paid the Bank a significant proportion (73%) of the total amount that was disbursed that year. 6. Sustainability 6.1 Rationale for sustainability rating: ICR Rating: Unlikely. The current Government has stated its strong commitment to extend health coverage to the entire population of Turkey; and to pass laws that will restructure the social security system, establish universal health insurance; and restructure and reorganize the MOH. It is conceivable that activities and output initiated through the support of the SHP might have some use in reforms which are crucial for the implementation of HTP. However, the sustainability of both the activities and achievement of the SHP is rated as unlikely, first, due to overall lack of successful outcomes from the project, meaning that there is little to sustain at this point; and second because the Bank has not seen concrete plans or evidence concerning some activities that were supported by the project. For example: l The sustainability of activities related to the CMSS remains a concern. At ICR preparation, only 25% of the CHRMS system is operating at an acceptable level and until the ICR Mission arrangements had yet to be confirmed for the systems' revisions. Unless the non-operational sub-clusters are revised - 26 -  and made operational, the investment in the CHRMS system will have a negative economic rate of return. Another concern with the CMSS is one that probably applies to the general sustainability of Information Technology investments, that is, how best to finance the high cost of IT consultants who largely manage the system for MOH's Department of Information Processing (DIP). It is the Bank's understanding that for now continued support for IT consultants manning CMSS will be provided through the Bank supported Health Transition Project, however in the long run the MOH needs to find a sustainable way to finance and support IT investments. l The sustainability of Training Health Centers and their future role as locus of training in the provinces is not very clear. While it is encouraging that an agreement has been reached to complete the construction and equipping of Training Health Centers under the support of the ongoing HTP, it is still not clear how training needs such as those supported by the SHP will be sustained in the future. This is especially a concern since the poorest regions of Turkey that were targeted by the project, suffer from very high staff turnover. 6.2 Transition arrangement to regular operations: There has been no need for formal arrangements to transfer implementation to regular operations because after 1999, project implementation (apart from Infrastructure Development subcomponent) largely became the responsibility of the MOH: i) basic health care interventions were implemented by different units within the MOH; ii) primary health care training and activities related to the studies subcomponent were undertaken by the School of Public Health in collaboration with universities inside and outside Turkey; and iii) implementation of the CMSS was the responsibility of the MOH's DIP, strongly supported by a group of IT consultants. The Bank-supported HTP provides an opportunity to continue engaging in activities around the completion and sustainability of infrastructure development activities. Future evaluations of this project should not only look at completion of construction but also their staffing, their usage and whether provincial staff receive necessary training. Attempts should be made to adequately assess: i) the rate of staff turnover in the project provinces and how recent Government decisions to employ contractual health sector staff might alleviate the shortage of staff in the provinces; and ii) how activities under the IT subcomponent and others within the MOH will continue to be supported. Though the SHP did not collect sufficient information to allow for monitoring or evaluation of outcomes, it might be possible to: i) track health outcomes for the provinces through further disaggregated analysis of DHS data (for example to look at immunization rates), data from the BOD, the NHA and Cost Effectiveness studies supported by the SHP; and other studies that are readily available; ii) conduct a thorough evaluation of the Bank's focus on health and poverty and how much the Bank's involvement in the health sector has contributed to human development in Turkey. 7. Bank and Borrower Performance Bank 7.1 Lending: ICR Rating: Unsatisfactory. l In identification, the Bank appropriately outlined major shortfalls in the country's health indicators and identified priorities according to both the government's development strategy and the Bank's assistance strategy. However, the Bank did not make use of the best global practices to address identified needs and did not invest in pertinent sector work on the main drivers of health status in Turkey. l During preparation the Bank could have further assisted the Borrower in putting in place adequate project management structures. This is especially true for the procurement of training - 27 -  services, information technology as well as the procurement of sophisticated medical equipment. In addition, the Bank could have further assisted in arrangements for sound financial management structures and in providing more useful suggestions to compensate for insufficient project management capacity. l At appraisal, the Bank's provided sufficient quantity of staff for the appraisal mission. The mission included five health specialists, three IT specialists, two operations specialists, one economist and three team assistants. However, continuity to supervision was inadequate as the project's leadership changed a few months after the project was appraised though a significant proportion of other team members stayed on even after the project became effective. The lending instrument, a Sector Investment Loan (SIL) was appropriate, given more limited choice of instruments a decade ago when compared to recent years. l The Bank's performance in appraisal was unsatisfactory for uneven evaluation of commitment. The commitment of the Government of Turkey at the time of project preparation was appropriately appraised as very strong, but there was little appraisal of support from other stakeholders and beneficiaries especially those in the project provinces. The importance of involving all stakeholders in project design was probably not as well appreciated then as it has become in recent years. Given prior experience from the First Health Project, the appraisal of MOH and the PCU's capacity was highly unsatisfactory; and appraisal of the PCU's financial management capacity and of political risks was deficient. 7.2 Supervision: ICR Rating: Unsatisfactory. The SHP was implemented over 10 years during which time quantity and quality of supervision significantly varied. The following factors were considered in this rating: l Sporadic quantity and quality of supervision: SHP conducted 19 documented supervision missions over 10 years (3 missions short of the average 2 missions a year), though according to discussions with individual team members there might have been more missions than official records have managed to capture. Missions were an average of 2 weeks long and involved approximately 4 people. However, prior to 1999, Missions that involved field visits to the provinces were few and on-site supervision of the infrastructure development component happened rarely if at all. Annex 4 shows that there was a long gap in supervision between March 1996 and December 1997, this is explained by team leadership changeover that made mission travel impossible to arrange. Documentation shows that periods around revision and near project closure received more intensive supervision and management oversight than did other periods. The supervision of SHP was indeed exemplary for some years after the 1999 revisions as was appropriately concluded by the QAG quality of supervision review of 2001. However, ICR mission interviewees acknowledged the superior quality of advice given for some subcomponents, but also some advice of marginal quality in other areas, a fact that can be explained by different teams that supervised the project since its approval. Quality of supervision was also marred by the slow reaction to challenges posed by the evolution of tensions between the PCU and MOH over a period of many years. Challenges posed by these situations could have been assessed and addressed earlier than they were; l Uneven management oversight and supervision: Especially in the early years of the project (pre 1998), the role of management in supervision is not clearly visible (explored in Bank documentation and ICR preparation interviews with former team members). All in all, the project did receive focused management attention around crises (e.g. at MTR when deficiencies in the MSA became apparent); and when major decisions such as whether to close the project or not had to be made. The earlier years of - 28 -  implementation, the signing of the MSA and the poor use of monitoring indicators could have received more attention than was the case. The supervisory role of management was clearly deficient in failing to provide sufficient oversight of legal covenants. Clearly the Bank failed to supervise the signing of the MSA between the MOH and UNDP/UNOPS by giving a `no objection' to an arrangement that contravened Bank regulations; l High project leadership turnover in the early years of project supervision: The supervision of the SPH was led by 6 different Task Team Leaders (TTLs) who led their team for periods ranging from less than 6 months to periods of approximately 4 years (the first two crucial years of project supervision saw 3 different TTLs). ICR preparation interviews with MOH/PCU counterparts acknowledged the very different personalities, commitment, hard work and supervision `styles' that characterized the different TTLs and the relationships that they had with them. But they also commented on the disruptions and adjustments that had to be made in working relations when these changes occurred; l Poor articulation and use of indicators: The lack of Bank capacity in supervising monitoring outputs and outcomes was very noticeable. The use of M&E tools was completely non existent in the first 5-6 years of implementation. The MTR did not sufficiently address this deficiency and no formal evaluation of the project progress was ever conducted. The 1999 revisions redefined some of the M&E indicators but articulation of how outputs, outcomes and impacts could be measured still remained weak. There was marginal attention to progress towards development objectives and more serious attention paid to disbursement and component by component activities. Efforts to strengthen M&E in 2002 clearly identified ways to revive monitoring of output and outcomes but these probably came too late and did not really take off; l Serious lack of realism in progress reporting: Overall, the rating of project progress was seriously disconnected from reality, in many cases, being more favorable Satisfactory than the justified Unsatisfactory (apart from the final years of the project when the rating was the appropriate U). Even after the project was revised and implementation had improved considerably, it is still difficult to justify that progress towards meeting objectives for this project could have been rated S even during periods of major improvements in supervision, partly because the original design of the project is unlikely to have led to a Satisfactory outcome. Interviews with various members of teams that worked on this project acknowledged the missing project Development Objectives -project activities link, though this was never acknowledged during supervision. l Project extensions were not effective: The Second Health Project was extended three times on trust that the Borrower was committed to seeing project activities completed. It is debatable whether the decision to revise the project in 1999 (which effectively guaranteed an extension) could have been handled differently. Recall that in 1999, two years before the original project closure date, little output had been achieved and a number of factors were hindering implementation and supervision. With the benefit of hindsight, it is clear that it would have been advisable to fully restructure the project in 1999 by redefining the DOs, significantly reduce the size of the project for activities that had yet to start but keeping activities that were absolutely necessary for achievement of goals and then allow the project to close in 2001 as was originally planned. On the other hand, once the decision to revise and later the project were made it is clear that a realistic project closing date should have been set to enable completed construction of the Training Health Centers. In hindsight, it is clear that despite the perceived strong Borrower desire to keep the project open, there were sufficient factors and reasons that should have led to a decision to close the project even earlier. l At times ineffective engagement with the Borrower: Prior to 1999, the working relation between the Bank and the MOH/ PCU was not an effective one. The Bank lacked independence in providing a true - 29 -  picture of project status, and, at MTR the relationship between the Bank and the Borrower was characterized by strong control of information by the PCU/ MOH resulting in a lack of transparency in reporting of implementation progress. After a new Bank team came on board in 1999, the Bank and the MOH/ PCU worked harder to establish an effective working relationship that significantly speeded implementation in the 3 of the 4 final years of the project. During this time the Bank maintained and nurtured dialogue with the Borrower leading to the signing of the HTP which deals with some of the systemic issues that remain a challenge to the Turkish health sector. l Deficient documentation and record keeping: The quality of documentation of implementation progress varied significantly from one team to another, being exemplary in some cases but thin in others. Overall, at the end of 10 years of implementation, the Bank's performance at record keeping and storage is highly unsatisfactory. For the pre-1999 years, there are missing Aide Memoires, Aide Memoires with no annexes and incomplete Project Supervision Reports. Different locations of project team leaders at different times (from Washington to Budapest, to Ankara, to Washington and finally Ankara), the Bank's reorganization and the introduction of IRIS all contribute to what might be a skewed and incomplete history of project implementation and supervision. A concrete example of this incomplete picture was presented under point 1 of this section. 7.3 Overall Bank performance: The Bank's overall performance is rated Unsatisfactory. Despite consistency with the Bank's assistance strategies, the project's identification was deficient and the design was overly complex for an insufficiently prepared PCU and MOH. Despite providing ample quantity of supervision, the quality of supervision during crucial years of implementation was deficient. The Bank was also deficient in supervision of the Borrower's monitoring and evaluation of project outputs and outcomes, making it a real challenge to reach a conclusion on the impact of this project. However, despite the lack of success of the SHP and all other previous Bank supported projects in the country, the Bank's team maintained dialogue with the Borrower and engaged the Borrower in a new project that will focus on very much needed reforms. Borrower 7.4 Preparation: ICR Rating: Unsatisfactory. The Government's commitment to the goals of this project can be clearly seen in its eagerness to finance the infrastructure development activities of the SHP. However, the quality of preparation of the project could have been strengthened by a strong health sector strategy that clearly defined the Government's priorities and commitment. Such a strategy should have been informed by more rigorous analysis of health sector issues. The lack of success for most activities of the SHP proves that the Borrower was not sufficiently prepared for the operational aspects of the project. Functions such as procurement and financial management were not sufficiently covered by a solution such as the MSA with UNDP/UNOPS. The Borrower did not involve other stakeholders in preparation and was therefore not prepared for the lack of support for subcomponents that required reforms. 7.5 Government implementation performance: ICR Rating: Unsatisfactory. The unstable macroeconomic environment, that persisted for almost as long as implementation lasted, severely affected the Government's ability to provide timely and sufficient counterpart funds, a factor that adversely contributed to delays in implementation. While the Government remained committed to keeping the project active and to the completion of activities, political instability of the 1990s and the ensuing lack of support for important health sector policy reforms made the implementation of activities of the Health Policy and Management component impossible. Insufficient support for a sub cluster of the Computerized - 30 -  Human Resources Management System also caused significant delays in implementation and final outcomes of the Health Policy and Management component. In addition, implementation was severely slowed down and made ineffective by the Government's inability to ensure consistent appointment of PCU staff and managers with sufficient technical and management skills. The high rate of changes in MOH Ministers and other high level counterparts also contributed to lack of continuity in the Bank's engagement with the sector. 7.6 Implementing Agency: ICR Rating: Unsatisfactory. The performance of the MOH was severely affected by inadequate capacity to not only manage implementation but to also operationalize the Government's intentions on improving health outcomes in the most deprived provinces. In some ways, the MOH was affected by political instability that resulted in many changes in the leadership and direction of the MOH as well as high staff turnover in general. These changes also caused a general lack of focus on outcomes, instead attention was more on delivering on activities of the project which were unfortunately not well linked to project objectives. Though the PCU's performance saw very significant improvement in management of the project after the 1999 revisions, overall, the PCU's management over 10 years was ineffective. A highly ineffective PCU structure, which was not only overstaffed but also cost ineffective, eventually marred working relations with the MOH whom they had to closely collaborate with. Shortage of skilled financial management staff at the PCU resulted in marginally adequate accounts and audits, an ailment that was exacerbated by the high rates of turnover partly caused by political instability and the negative working environment created by long judicial processes around the work of the PCU. It is due to the limited capacity of the MOH and the difficult working relationship between the MOH and PCU that shocks mentioned under Section 5 above had such a negative effect in the implementation of this project. Portfolio review of projects which were being implemented in the same environment show that implementation of these other projects did not suffer as negative an outcome as did the SHP. 7.7 Overall Borrower performance: ICR Rating: Unsatisfactory. The unsatisfactory overall Borrower performance is partly explained by the lack of a strong cohesive and long term strategy to improve poor health outcomes in the poorest provinces of the country. A strong strategy that clearly addressed issues around equity of access would have withstood the various changes in political leadership and would have guided the various Governments that took leadership of the country at different stages of implementation. Despite its strong desire to complete activities planned, the Borrower was not able to effectively support the project with timely counterpart funds or with adequate staffing of the PCU. Most importantly, deficient monitoring of outputs and outcomes of the project meant that valuable lessons that could have been gained in factors that affect project performance have been lost. 8. Lessons Learned l Deficiencies in initial project design can not be easily remedied midway through implementation unless effective restructuring is undertaken: The most fundamental lesson from implementation of the SHP is one that is very basic and one that is applicable to all projects irrespective of sector or region. The importance of project design can never be underestimated. The initial design of the SHP missed almost all the elements of good project design. A key deficiency was that the project design was not based on sufficient knowledge of the Turkish health sector. In addition, the project was too complex, it was ambitious and unfocused and inflexible to changing political and economic conditions. - 31 -  Analytic sector work would have highlighted many of the deficiencies in project design, especially in identifying priority challenges that needed to be tackled first in the worst off regions. Another more specific lesson that arises from this project is that efforts to revise a project that had significantly flawed original design requires more drastic action than minor revisions and intensive supervision. Despite some exemplary supervision and hard work injected into the project after 1999, it was disappointing but probably not surprising to see that project performance did not improve sufficiently to lead to a satisfactory outcome. Another basic lesson is that, once a project is revised then sufficient time must be allocated to completion of activities. l The Bank needs to adopt a culture where statements of expected project impact are more realistic: The SHP's development objective very clearly responded to the needs of the country at that time. However, part of the failure of the SHP to have any measurable impact rests squarely on the fact that the project over promised in its statement of expected impact on major health indicators by the end of the project's implementation. The expected impact of the project should have been stated more modestly, especially when ones considers that there are other important determinants of health outcomes which were not being addressed by this particular project. l Pay attention to recommendations from reviews of design quality: Though this project became effective before the inception of Quality at Entry (QEA) reviews by the Quality Assurance Group (QAG), it was comprehensively peer reviewed at the time of its preparation. Documentation shows that a number of design deficiencies were raised in early stages of preparation and sound recommendations were brought to the attention of the project preparation team. These were not incorporated in the final design; it is hence not a surprise that the project did not meet the anticipated outcomes. l It is possible to maintain dialogue on global good practices in the midst of unsatisfactory implementation: Preparation and signing of the fourth Bank supported HNP project occurred towards the final and difficult years of implementation of the SHP. By that time it had become clear that the final rating of the SHP would likely be unsatisfactory. It is therefore commendable that the Bank team continued to engage the Government in dialogue on the major factors affecting health outcomes in the country. In many aspects, the design of the HTP is very far removed from that of its predecessors in terms of areas of focus, the choice of lending instrument, and economic sector work that informed the project, all of which show that the Bank and the Borrower have taken seriously some of the lessons of previous projects' implementation. l The role of project coordination units in assisting Borrowers to produce sustainable and appropriate outputs needs careful evaluation: The risk of a PCU becoming a parallel unit, implementing rather than coordinating the project is a real one. Such PCU's invariably undermine the Borrower's capacity to undertake procurement and financial management, they deprive the Borrower of the best local staff and it can become very difficult to collapse such units back to the relevant Ministry when a project closes. Implementation of the SHP underscored the importance of carefully considering the added value that PCUs might bring. l While training activities might not guarantee efficiencies and performance improvements by the end of the project, but for poorly paid employees training might maintain morale and facilitate future policy changes: The SHP supported an impressive quantity of training activities for which measurable benefits are hard to find especially since training was not supported by policy reforms and because there was a high degree of staff turn over. But such training was mentioned by beneficiaries to be an important compensation for working in more remote less advantaged provinces. It has also been suggested that training in health sector management planted seeds of change that might benefit implementation of proposed health sector reforms, some of which will be supported under the HTP. - 32 -  l Bank supervision tasks need to be more focused on achieving measurable outcomes rather than outputs alone: In addition to ensuring that supervision teams are focused on providing both quality and continuity in supervision, (in which case the team must have the right skills mix, with members that understand local ways of doing business, preferably led by a locally based experienced health expert), it is critical that quality supervision is provided through closer collaboration between the country and sector teams which in the case of the SHP might have ensured that the project was more firmly rooted in the reality of the country and would have benefited more from experiences of other projects. The importance of closer links between country and sector teams could have provided checks and balances in the enforcement of the articles of the loan agreement and bank procedures, for example. l M&E must become a focus of Bank work and teams have to be accountable for the formulation and supervision of realistic M&E tasks: The Bank needs to be more accountable in designing and supervision of M&E even when projects lacked this focus at design. The Bank must lead by example by training its own staff on the importance of M&E, and gain sufficient knowledge on tools to measure outcomes and the impact of outputs that are not easily measured such those from training and IT investments. Unless Bank teams are accountable for supervision of M&E, it will be hard to convince Borrowers to build capacity in this area. It is no longer defendable that the Bank and Borrower are still unable to evaluate outcomes of loans. l Arrangements such as the MSA might be beneficial in select circumstances, but the Bank must assure at all times that a project is audited by an independent auditor who meets the Bank's accounting and auditing standards: By serving the original purpose, arrangements such as the MSA between MOH and UNDP/UNOPS can be extremely useful in speeding implementation in countries where bureaucratic processes might be a hindrance. However, it is imperative that the Bank continues to assure that resources from supported projects are properly accounted for by including auditing clauses that satisfy the Bank's standards and guidelines. In the case of the SHP, standards of auditing provided by the Borrower's Treasury internal auditors did not meet standards of auditing acceptable to the Bank and though this deficiency was later remedied, it was at the cost of the project in terms of time that was spent on internal Bank inquiries and investigations by the Borrower. l The Bank's position against alleged fund misuse must be more forcefully presented: In the past decade or so, the Bank has taken a very clear public stance against misuse of funds in projects it supports. One of the key messages received during the ICR interviews was that the Bank needs to work harder at delivering its message of zero tolerance on corruption and misuse of funds; and to further explain that the Bank has an internal processes of dealing with such allegations. In its supervision of the SHP, Aide Memoires show that the Bank did raise its concerns regarding the impact that lengthy judicial processes were having on PCU performance, though to the public, the Bank remained silent on specific allegations of misuse of funds from the SHP. The Bank did conduct its own internal review of the MSA, civil works and awards of architectural design contracts and found the allegations to be without evidence. Though it is appropriate that the Bank should not interfere in a country's internal investigative/ judicial processes, when allegations of Bank project funds misuse are raised, the Bank should make further efforts to reassure the beneficiaries of Bank loans that the Bank does not tolerate funds misuse and that there are Bank procedures in place to address any such allegations. 9. Partner Comments (a) Borrower/implementing agency: The Government's contribution to this ICR is included in Annex 8. - 33 -  (b) Cofinanciers: (c) Other partners (NGOs/private sector): 10. Additional Information The Government provided the ICR team with comments on the draft version of the ICR which was sent to the Government for review on June 22, 2005. These comments were received on August 15, 2005, and are included under an Additional Annex 11 of this ICR. The ICR team carefully reviewed these comments and made changes where appropriate, as well as clarified sections where additional information was needed. The revised, current version of the ICR was shared with the Government again based on which the Government agreed that the ICR could be distributed to the Bank Board of Executive Directors, with the Government's original set of comments attached as an Annex to the ICR. - 34 -  Annex 1. Key Performance Indicators/Log Frame Matrix Indicator/Matrix Target in SAR/ PIR Actual/latest Estimate I. SAR original Improve the following outcomes Outcome goal: in the 23 provinces: Improvement in i) Reduce IMR by 40%; IMR: Turkey: 52 in 1993; 29/ 1000 in 2003: a 45% health indicators reduction. Project provinces: reduction by 32% (IMR, MMR, TFR, and Life ii) Reduce U5M to 80%; U5M: 61 in 1993; 37/ 1000 in 2003: a 39% reduction. Expectancy). Project Provinces: a 30.4% reduction. iii) Improve MMR by 30%; MMR: Turkey: estimates range from 52 per 100 000 (1999) and 130 per 100 000 live births (1997). No data were available to measure this outcome for project provinces. iv) Reduce TFR by 30%; TFR: Turkey: 2.7 in 1993; 2.23 in 2003: a 17% reduction. Project provinces: a 17% reduction. v) Reduce Infections due to No data were available to measure this outcome. Tetanus, Polio and Measles by 95%; Intermediate Process Indicators: vi) Increase ANC to 3 counts; ANC: Turkey: 63% in 1993; 81% in 2003: 29% increase. Project provinces: 79% increase. No data were available to measure the number of ANC counts. Data is on any ANC. PIR restated the vii) Decrease HHDR by 10%; HHDR: Turkey: 0.67 in 1993; 0.27 in 2003: 60% increase. Outcome goal as Project provinces: a 64% increase. follows: Overall health indicators viii) Increase CPR by 30%; CPR: Turkey: 63% in 1993; 71% in 2003: 13% increase. should be Project provinces: a 37% increase. commensurate with health ix) Increase immunization rates by Full vaccination: Turkey: 65% in 1993; 54% in 2003: a 16% Turkey's income 95%. decline. Project provinces: a 14% reduction. level. (No numerical x) Improve life expectancy. Life Expectancy in Turkey rose from 68.9 years in 2000 to 70 targets were set). in 2003. There was no numerical target for this indicator. xi) Reduce Child deaths due to No data were available to measure this outcome. ARD by 30%; xii) Improve ambulatory health No data were available to measure this outcome. care staffing rates to 95%; xiii) Improve availability of No data were available to measure this outcome. antibiotics by 100%. II. Improvement SAR did not define outcome or impact indicators to measure output from certain public health in health care interventions (TV ads for e.g.); training; or IT investments. Their contribution was to be `measured' by management. improvement in health outcomes. PIR did not define indicators for this outcome. III. PIR set goal: To Reduce Burden of Disease No baseline (pre project) BOD data exists for Turkey and BOD by reduce regional discrepancy by region. provinces data were not available. Turkey's DALY in 2002/ 2003 - 35 -  health disparities. amounts to 10, 802, 494 (2, 730, 794 from Group I diseases, 6, 905, 893 from Group II disease, 1, 165, 807 from Group III diseases). It is therefore impossible to say how the BOD has changed in the decade of implementation. Using other readily available data such as DHS and without suggesting attribution to project activities: Regional disparities in some outcomes have improved: The gap in CPR between the West and the East region has narrowed but remains about 16%; The West-East difference in TFR has narrowed from 2.4 in 1993 to 1.8 children in 2003; The West-North-East gap in HHDR has narrowed; and The West-East gap in ANC has narrowed by 73%. Regional disparities in some outcomes have worsened: In 1993, difference in IMR between the West region and the East region was about 17 percentage points this has grown to 19 in 2003. The West-North gap in U5 Mortality has grown from 1.5 %age points in 1993 to 18% age points in 2003. Output Indicators PIR Indicator End of Project Target in PIR Actual Output 1. More equitable and improved access to essential health services in 28 provinces. i. Continued training of health