Turkey - Basic Education Program Project

Document of The World Bank Report No: 27696 - TU IMPLEMENTATION COMPLETION REPORT (SCL-43550) ON A LOAN IN THE AMOUNT OF US$300 MILLION EQUIVALENT TO THE REPUBLIC OF TURKEY FOR A BASIC EDUCATION PROJECT IN SUPPORT OF THE FIRST PHASE OF THE BASIC EDUCATION PROGRAM JUNE 24, 2004 Human Development Sector Unit Europe and Central Asia Region  CURRENCY EQUIVALENTS (Exchange Rate Effective June 9, 2004) Currency Unit = Turkish Lira Turkish Lira 1 = US$ 0.000001 US$ 1 = Turkish Lira 1,466,472 FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS APK Justice and Development Party APL Adaptable Program Lending BEP Basic Education Project CAS Country Assistance Strategy EGITEK General Directorate for Educational Technologies ERDD Education Research and Development Directorate EU European Union GoT Government of Turkey IBRD International Bank of Reconstruction and Development ICT Information and Communication Technology MoF Ministry of Finance MONE Ministry of National Education NGO Non-governmental Organization OECD Organization of Economic and Co-operation and Development PCC Project Coordination Center PISA Program for International Student Assessment TOR Terms of Reference Vice President: Shigeo Katsu Country Director Andrew N. Vorkink Sector Director Charles C. Griffin Sector Manager Maureen McLaughlin Task Manager: Ferda Sahmali  TURKEY BASIC EDUCATION PROJECT APL I CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 6 5. Major Factors Affecting Implementation and Outcome 13 6. Sustainability 15 7. Bank and Borrower Performance 16 8. Lessons Learned 20 9. Partner Comments 21 10. Additional Information 21 Annex 1. Key Performance Indicators/Log Frame Matrix 23 Annex 2. Project Costs and Financing 25 Annex 3. Economic Costs and Benefits 28 Annex 4. Bank Inputs 29 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 32 Annex 6. Ratings of Bank and Borrower Performance 33 Annex 7. List of Supporting Documents 34 Annex 8. Beneficiary Survey Results 35 Annex 9. Stakeholder Workshop Results 41 Annex 10. Borrower's Evaluation Report 43  Project ID: P009089 Project Name: BASIC ED Team Leader: Ferda Sahmali TL Unit: ECSHD ICR Type: Intensive Learning Model (ILM) of ICR Report Date: June 25, 2004 1. Project Data Name: BASIC ED L/C/TF Number: SCL-43550 Country/Department: TURKEY Region: Europe and Central Asia Region Sector/subsector: Primary education (83%); Tertiary education (11%); Central government administration (6%) Theme: Education for all (P); Rural services and infrastructure (P); Social analysis and monitoring (S); Gender (S) KEY DATES Original Revised/Actual PCD: 02/04/1998 Effective: 08/12/1998 08/12/1998 Appraisal: 05/05/1998 MTR: 05/31/2000 04/12/2000 Approval: 06/23/1998 Closing: 06/30/2001 12/31/2003 Borrower/Implementing Agency: REPUBLIC OF TURKEY/MINISTRY OF NATIONAL EDUCATION Other Partners: STAFF Current At Appraisal Vice President: Shigeo Katsu Johannes Linn Country Director: Andrew N. Vorkink Ajay Chhibber Sector Manager: Maureen McLaughlin James Socknat Team Leader at ICR: Ferda Sahmali Michael T. Mertaugh ICR Primary Author: Hong Chen 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: L Institutional Development Impact: N Bank Performance: U Borrower Performance: U QAG (if available) ICR Quality at Entry: S U Project at Risk at Any Time: Yes  3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: In 1973, the Basic Law for National Education (Law 1739) redefined basic education as comprising eight years of compulsory schooling. However, the Government was unable to enforce this mandate for compulsory education mainly because of the lack of facilities and teachers. In 1983, Law 1739 was amended requiring enrollment wherever the physical infrastructure was in place. Throughout the 1990s, there was a growing public interest in expanded basic education. In 1995, the Government requested the Bank's assistance in preparing a project to support universal basic education. In August 1997, Parliament approved a new Basic Education Law (No. 4306) which established the timetable for implementing universal eight-year basic education. Most importantly, this law provided major budget resources in the form of taxes earmarked specifically for the investments necessary to achieve that goal. The new Government's expanded basic education strategy was articulated in its action program, which received a high level of private and voluntary contributions. A major expansion of school capacity was needed to reach the goal of expanded universal basic education and reduce crowding in existing schools. The constraints included inadequate classroom capacity and shortage of teachers. Parental demand for education in rural areas in the East and the Southeast appeared to be weak due to the low quality of teachers in their villages and the cost of sending their children to school. The Government's basic education program was to address the issues of raising quality, provide material incentives to children from poor households, improve incentives for teachers in rural schools, promote school-based activities to make basic education schools a resource of the community, and expand preschool education. The Government's program was to be implemented in three phases: launch, consolidation, and subsequent phase. In the policy letter sent to the Bank in 1998, the Government stated that the expansion of the five-year compulsory cycle to eight years was a centerpiece of its 7th Five-Year Development Plan (1996-2000). Government also reiterated its strategy to achieve universal attendance in eight-year basic education and improve significantly the quality of basic education. There was substantial confusion over the project development objectives (PDOs) as the Loan Agreement had a set of objectives for the basic education program and the project. However, various sections within the Project Appraisal Document (PAD) had inconsistent sets of goals for the project. For the purpose of this ICR, the original PDOs are those from the original Legal Agreement: (a) the expansion of the capacity of basic education schools; (b) training of teachers, school principals and inspectors; (c) development of central and provincial implementation capacity to carry out the program; and (d) creation of a mechanism to monitor and evaluate the outcomes of the program. The original project objectives (DOs) reflected the priorities of the Government at the time of project preparation. They were consistent with the Bank's Country Assistance Strategy (CAS) (September 1997) which called for increased administrative resources aimed at social development and poverty alleviation. The CAS advocated shifting the focus of investment activities to poorer regions. It stated clearly: "Our highest priority in education is to help the Government achieve its goal of universal schooling through the eighth grade". The Letter of Sector Policy sent to the Bank in April 2002 indicated the following goals of the Ministry of National Education (MONE) in basic education: increase school enrollments to 100%; improve the quality of education; reduce the number of students per classroom to 30; eliminate gradually the practice of combining classes in rural schools; and end double-shift teaching. The Bank's current CAS for Turkey (October 2003) for FY04-06 included key education outcome indicators: net enrollment rate in basic education maintained at not less than 99%; increased girls' enrollment, - 2 -  especially in secondary education; and male/female ratio improved from 1.20 to 1.15 by 2006. The development objectives of the Government and the APL1 were neither clearly nor consistently presented in the main text of the PAD and the Annex 1 of the PAD. There were also inconsistencies when the PAD was compared to the Legal Agreement. There was a lack of clarity on the DOs for the APL1. The PAD outlined the Development Objective for the Phase I of the Basic Education Program (establish capacity to implement, monitor and evaluate the Program, initiate key investments to achieve Program objectives), but did not indicate the DOs for APL1. Even if these were clarified as the DOs for APL1, they seemed inappropriate as they tended to be more of an input than outcome nature. The logframe gave the key performance indicators for the program objectives, but only the trigger actions were listed for the key performance indicators for the project objectives. There were no clear measurable outcomes and related indicators for APL1. The implementation arrangements of the project required well-coordinated and trained MONE officials, both at the central, provincial, subprovincial and school levels. School construction, renovation, and upgrading under the project were spread over the whole country. The accountability and the competence of the implementation agencies were high risk areas for the project. The project design had appropriately put in place a set of risk minimization measures. The most significant risks were: cooperation among various elements of the MONE; lack of complementary inputs (such as training or educational software); excessively high expectations established by the Government's Basic Education Program that exceed the Ministry's implementation capacity; and delay in procurement of IT hardware and software that could delay the delivery of computer instructions. 3.2 Revised Objective: Due to the needs of the emergency created by the Marmara earthquake of August 1999, the Legal Agreement was amended in December 1999. While maintaining the broad objectives of the first phase to support the borrower's program, the second paragraph of Schedule 2 to the Loan Agreement was amended as: (a) the expansion of enrollment in basic education schools; (b) training of teachers, school principals, and inspectors; (c) development of central and provincial implementation capacity to carry out the program; (d) creation of a mechanism to monitor and evaluate the outcomes of the program; (e) reconstruction and rehabilitation of educational facilities for preschool, basic education, and secondary education which were destroyed or damaged in the Marmara earthquake of August 1999; and (f) resumption of preschool education, basic education, and secondary education in the area affected by the Marmara earthquake of August 1999. Since this was a formal amendment to the Legal Agreement, it was reported to the Executive Directors. This amendment reallocated the Proceeds of the Loan to make resources available for Marmara earthquake needs. It also changed the first objective from "expansion of the capacity" to "the expansion of enrollments". The amendment of November 24, 2000, which included the changes to some of the components and the reallocation of loan proceeds, was reported to the Executive Directors. Although it was not the appropriate document to indicate a formal change in the project development objectives, the Project Status Report (PSR) of April 2003 changed the project development objectives to read: a) the expansion of enrollments in basic education schools; b) training of teachers, school principals, and inspectors; c) improving basic education quality by providing information, communication and technology (ICT) classrooms, educational materials to schools; d) development of central and provincial implementation capacity to carry out the program; and e) creation of a mechanism to monitor and evaluate - 3 -  the outcomes of the program. 3.3 Original Components: The Loan contained four components: (1) Expanded Basic Education Coverage (cost: $48.4 million): This component was to carry out: a) teacher training, recruitment, and deployment; and b) school construction, extensions, and upgrading. There was no loan financing used under the first APL for teacher training, recruitment and deployment, although an extensive program of in-service training was provided under the second component. The first APL would finance the rehabilitation and related architectural and engineering design work of about 200 village schools, 100 central village schools, and 40 urban schools in the provinces of Sirnak, Van, Mardin, Hakkari, and Bitlis. (2) Improved Basic Education Quality (cost $233.5 million): This component was to finance the upgrading of information technology (IT) classrooms and provide hardware and software for 2,828 IT classrooms throughout the country. The first APL would support in-service training for teachers, inspectors, and school principals. It would provide essential teaching and learning materials to rural schools. The Loan would also support the development and implementation of assessments of learning outcomes. (3) Program Implementation Support (cost $4.8 million): This component would provide support to the Program Coordination Team (PCT) with consulting services, study tours, and office equipment. It also aimed to strengthen educational management information systems, including school mapping. (4) Monitoring and Evaluation (cost $13.2 million): This component would assess the outcomes of the program against several categories of performance indicators, and provide continuous feedback to the Program Coordination Team (PCT) and the related line units of the Ministry to improve the effectiveness of the program in meeting its objectives as implementation was underway. This component also included support for a Monitoring Response Facility (MRF) to support school-based initiatives, and public media efforts promoting the Basic Education Program. The components were clearly linked to the PDOs. The project supported the top priorities outlined in the Government's program. It targeted the activities to assist the rural and poor urban schools. The component activities were the necessary actions to achieve the development objectives. The lessons learned from computer laboratory schools components under the National Education Development Project (NEDP) were taken into account during the design of APL1. The project also incorporated experiences from UNICEF's program on rural education. It introduced a school improvement program MRF, which was considered innovative at the time. The design of monitoring and evaluation was extensive. Despite all of these positive steps, however, the four project components posed a strong challenge to the Government's ability to implement. 3.4 Revised Components: To address the emergency needs created by the Marmara Earthquake, the Loan Agreement was amended in December 1999 to add a component, Part F: Marmara Earthquake Relief Component. This new component was mainly to: 1) reconstruct and rehabilitate preschool, basic education and secondary education facilities which were damaged by the earthquake; 2) provide educational materials and student support in the form of textbooks, educational materials, - 4 -  school uniforms, basic clothing and nutritional supplements to student affected by the earthquake; 3) support homeless teachers by providing basic clothing, furniture and basic furnishings for temporary accommodations; 4) contract transport services to transport students and teachers to temporary schools; 5) provide technical assistance and consultant services to develop and provide psychological counseling and training to affected students and teachers; 6) provide consultant services and studies to monitor and evaluate the emergency recovery actions and their consequences; and 7) provide consultant services for the design and supervision of construction, rehabilitation, and retrofitting of damaged schools. The activities for extensions to about 40 basic education schools in Ankara province were deleted from the project. In the November 2000 Loan Amendment, Part F component was revised and reduced to only three types of activities: 1) contracted transport services to transport students and teachers to temporary schools; 2) provision of educational materials and student support in the form of textbooks, educational materials, schools uniforms, basic schooling, and nutritional supplements to students affected by the earthquake; and 3) consultant services and studies to monitor and evaluate the emergency recovery actions and their consequences. The amendment also reinstated the 40 Ankara basic education school extensions, which were deleted from the project in December 1999 in order to make resources available for earthquake recovery activities. The amendment added the rehabilitation of about 1,300 formerly closed rural basic education schools in provinces throughout Turkey and 11 basic education boarding schools in provinces throughout Turkey. The Loan was able to finance these reinstated and new activities because some of the earthquake recovery actions planned for Loan financing were carried out with grant funds or other assistance. In addition, the amendment added a subcomponent under the Improved Basic Education Quality Component: Expansion of the Mother/Child Preschool Education Program and strengthening of the MONE's capacity to implement the program through training of MONE staff and provision of instructional materials. 3.5 Quality at Entry: The ICR performance rating for quality at entry is unsatisfactory. The strength of the project design was that it reflected Government's priorities in the sector. There was also a high degree of ownership by the counterpart at the time of project launch. The Bank incorporated what were believed to be successful lessons from the previous National Education Development Project (NEDP), i.e. introduction of ICT into the curriculum. It also built on the available experiences from the UNICEF work in Turkey. Nevertheless, the PAD did not define clearly the development objectives. There were no measurable outcomes stipulated for APL1. This caused difficulties in monitoring activities. It also did not allow meaningful supervision by the implementation agency and the Bank. This was partly due to the fact that Turkey APL1 was one of the first APLs ever prepared in the Eastern Europe and Central Asia (ECA) region. The APL approach had just been introduced in the Bank in FY98. The logframe approach was still at the early stage of development. A Quality at Entry Assessment was conducted for the project in 1998. The overall Assessment of Quality was rated as marginally satisfactory. The report rated the project satisfactory for the project's concept, objectives and approach, environmental aspects, and poverty and social aspects. The report confirmed the national ownership for the project. It stated that the pilot activities under the NEDP and the - 5 -  involvement of UNICEF were helpful to the design of APL1. However, the Assessment expressed the concern that "the Bank's financial activities focused on underserved regions, but almost 50% of Bank finances was for computers which were more likely to make economic sense in better served areas." The ratings were marginally satisfactory for technical and economic aspects, institutional capacity analysis, risk assessment and sustainability, and Bank inputs and processes. The Assessment stated that the project used a relatively simple analysis, even for the computer component of the project, to justify the technical soundness of the project/program. Options were not examined carefully regarding boarding, bussing, and double-shifts schools. There were no alternative designs seriously considered by the project. The preparation team disagreed with this finding and indicated they had urged the government explore options. On the quality and coherence of the economic rationale underpinning the project, the Assessment found that the micro justification was satisfactory, but that it was not helpful in choosing among options. The macro analysis was found to be lacking. On the institutional capacity aspects, the Assessment commented that the project built on the knowledge of the institutional framework in the past, but did not develop the framework for the implementation of the construction work. There were high expectations in the project design, but the triggers assumed low performance. In addition, the project did not attempt to strengthen performance capacity. On the risk assessment and sustainability, the Assessment indicated that there were no actions included in the design to minimize construction and especially recurrent cost risks. It mentioned further that the arrangements for monitoring to ensure timely implementation were still to be defined. Bank's inputs and processes was assessed as marginal satisfactory, adding that disbursements may be slow given the status and the preparation of the project. The Assessment rated the APL1 unsatisfactory for financial management aspects and readiness for implementation. It commented that project financial management concerns were not addressed in accordance with OP/BP 10.02 during project preparation. As a result, the limited financial management information available would hinder efficient and effective project management. Prompt attention to improve financial management arrangements was needed, particularly under the decentralized operations of this project. It stated that only strong political will would offset the large financial risk. There was no major cost recovery feature incorporated into the project design. On the readiness for implementation, the Assessment commented that the project attempted to improve the quality of the government program, but it had no project implementation plan and no planned procurement packages. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The assessment of the outcomes is based on the revised development objectives in the Amendment to the Legal Agreement dated December 1999, as stated in paragraph 3.2 of the previous section. Assessing these objectives also makes sense because the Government expressed in its Letter of Sector Policy in 1998 the objectives of universal attendance for an eight-year education and significant improvement of the quality of basic education. The overall outcome of the project is rated unsatisfactory. The first project development objective, the expansion of enrollments in basic education schools, is the only development objective this ICR rates satisfactory. The expanded capacity of basic education schools contributed significantly to the increase of enrollment. Government built 103,983 new primary education classrooms during 1997-2001, bringing to a total of 265,000 classrooms. This created additional capacity for more than three million basic education students. The number of teachers recruited for primary education schools since 1997 amounted to 103,692. The teacher-student ratio dropped to 26 in 2003 from 33 in 1996. The net enrollment rate for the eight-year primary education program was raised - 6 -  from 85.63% in 1997 to 96.3% in 2003. The number of female students was 4,085,452 in 1998 and increased to 4,828,276 in 2002-2003 academic year. The transition rates from 5th to 6th grade reached 100% from 1998/99 to 2003/04 school year, except the rate was 96% in 2002/03 academic year. The schooling rate for pre-school education was 13.1% according to the 2004 data from MONE. However, these achievements were mostly attributed to the Government's effort, and the Bank's APL was only a small part of the larger program (see details below in Outputs by Components). A qualitative study conducted by a local consultant indicated shortcomings associated with this development objective. Absenteeism, varied between 1% to 10% throughout the provinces and it was about 10% higher among female students than that of male students. The bussing program introduced by the Government's Basic Education Program actually contributed to the increased class size in larger villages and city centers, and double-shift education increased from 16% of schools in 1996 to 22% of the country's 36,064 schools in 2000. In 2003, there were 8,532 schools practicing double shifts - 4,410 in urban areas and 4,122 in large villages. The bussing program had adverse effects such as female students attending schools further from home and parents being isolated from their children's' schools. On the second project development objective, training of teachers, school principals, and inspectors, the rating is unsatisfactory. The potential, critical in-service training originally planned for APL1 did not take place under the project. Major training activities were carried out under the Government's budget. The Bank did review the training course materials at the beginning of the project, but APL1 did not attempt to review the effectiveness of the training conducted by the Government after the project failed to spend the allocated $31.2 million in the first year of implementation. Out of the $31.2 million planned for the in-service training programs, only $3.7 million were spent. See details of in-service training carried out under APL1 in section 4.2, component 2, subcomponent 2, below. The third project development objective, development of central and provincial implementation capacity to carry out the program, is rated unsatisfactory. At the end of APL1, MONE's capacity to implement the project, as well as the program remained weak. Program coordination continued to be inadequate toward the end of the project. Communication and collaboration within the units of MONE had been problematic. There was no effective information flow within the Ministry regarding project, and also the program activities. The planned education management information system was not implemented under APL1. At the provincial level, there was limited capacity building in terms of promoting and coordinating education work. Schools and communities were not able to actively participate in the plans for them under the project, except for the MRF activities. The fourth project development objective, creation of a mechanism to monitor and evaluate the outcomes of the program, is rated unsatisfactory. The achievements of this objective were extremely limited. The evaluation to provide essential feedback to make course corrections and respond to changing circumstances did not take place under APL1. There were many assessments and studies included in the original project design (about 11 studies). At the request of the Government, the number of studies was later reduced to four, with Child Labor in Turkey already completed at the time. Out of all the assessments and studies, the project financed only three studies, a) cost analysis, b) social impact, and c) institutional impact of bussing schools, YIBO/PIO schools, and traditional schools; these studies were conducted after long delays. Under the APL1, MONE failed to set up a mechanism to monitor and evaluate the outcomes of the project and the program. The Ministry did not coordinate the monitoring and evaluation activities within the relevant units in MONE. The Research, Planning and Coordination Council (APK) was not involved in the project in a substantive manner. The Education Research and Development Department (ERDD) was the main unit to prepare and implement student achievement assessment tests throughout the country, but it had little cooperation with APK. Limited monitoring and evaluation of project activities was - 7 -  undertaken by a small unit within the PCC, and there was no collaboration with APK nor ERDD. This lack of overall coordination undermined an effective monitoring and evaluation mechanism. As part of the monitoring and evaluation activity, the Monitoring and Response Facility was to provide the flexibility to respond to development needs at the school level. Although positive outputs were generated from the MRF projects, the subcomponent activities took a long time to launch. In February 1999, the guidelines for the MRF were finalized. Only in December 2001, six months after the original closing date, did the PCC and the Commission established at the MONE approve 81 projects for implementation. Failure to develop further the idea underlying this subcomponent through effective communication between the Bank and the MONE and capacity building at the local and school levels severely delayed implementation and limited use of allocated budget. This negatively affected the Government's view towards similar activities in subsequent cooperation with the Bank. The Government requested the elimination of such activities in a subsequent APL2 Loan, stating that it was not worth the additional effort. After many follow up discussions, the Government expressed the interest of adding school improvement activities to APL2. The project objectives of (e) and (f), which were related to the Marmara earthquake, are not being rated as some of the earthquake recovery actions planned under the APL1 were carried out with grant funds or other assistance. Concerning the achievement of the development objectives for the basic education program, the ICR provides the following assessments. The assessment took into account that these objectives were actually set for the whole APL period and the 5 1/2 years of implementation of APL, which was more than half of the planned APL program. The overall outcome of the program is rated unsatisfactory. The three program development objectives for the basic education program were: (a) to achieve universal coverage in an expanded, eight-year basic education cycle; (b) to improve the quality and relevance of basic education; and (c) to make basic education schools learning resources for the communities. On the first program development objective, the assessment is similar to the assessment of the project development objective (the expansion of enrollments in basic education schools) and is rated satisfactory. On the second program development objective, there was not enough evidence to substantiate the achievement and is rated unsatisfactory. In addition to the points made in the previous assessment of the project objectives, particularly the assessment of the second, third, and fourth project development objectives, the following can be added. As shown from recent international assessments, such as TIMSS and PIRLS, Turkish students were still well behind most of their counterparts in student learning. Computer equipment was provided under the project to 2,802 ICT classrooms and ICT formators and coordinators were trained, but insufficient computer software was provided to the schools. The actual IT courses taught by the IT formators were limited to IT laboratories that focused on basic computer skills. Subject teachers were not trained in how to integrate computers into their subject teaching. The Bank asked the Government to prepare and update the IT policy paper and formulate a strategy to integrate IT into the curriculum and teacher training. After 5 1/2 years of project implementation of APL1, including three extensions of the closing date, the IT policy paper was not completed before the project closing date. The ICT impact study was also not completed during project implementation. While computer training may have accomplished the Government's goal of achieving computer literacy (sufficient - 8 -  measuring of computer literacy had not been done under the project), there was not enough evidence to show that the APL1 inputs contributed significantly to the project's development objectives of improving quality of basic education. The potential uses of computers in schools are many and implementation processes are long-term. The ICT investments that were made probably set the preconditions to achieve more ambitious goals for the future, which includes radical change of school culture. However, it is too early to draw final conclusions in this ICR, except to present a snap shot of an ongoing process. The much-needed educational materials were purchased under the project, but only after long administrative and procurement delays, and three extensions of closing date. Of all the planned social assessments, only three studies were completed at the later stage of project period, but again the MONE did not disseminate the study results to the broader audience, such as stakeholders and academics. The Monitoring and Response Facility (MRF) yielded good results in terms of increased parental and community involvement in schools, but again, this activity was seriously delayed. The approval of the first group of projects under the MRF came only after May 2001, one month before the original closing date. The actual activities implemented were far less than the planned. There were $8.4 million planned for MRF, only $0.7 million was actually spent. On the third program development objective, there were few facts to show that significant progress had been made in this regard and is rated unsatisfactory. There was evidence that some schools offered the communities the ICT laboratories for after school use. There were good examples that MRF activities also reinforced the link between communities and schools. However, for the majority of schools that APL1 supported, it was difficult to conclude that they had become the learning resources for the communities. In fact, the bussing program introduced by the Government had some adverse effects. In some teachers' views, there were no longer educational and multi-activity cultural institutions left in the villages. The Government later amended the bussing program, reopening many of the village schools. 4.2 Outputs by components: Component 1: Expanded Basic Education Coverage (US$48.4 million in PAD, US$94.4 million actual). This component is rated satisfactory. The objectives of this component were to support the training and recruitment of additional teachers and the design, construction, expansion and upgrading of basic education schools. There was no loan financing used under the project for the first subcomponent which covered teacher training, recruitment, and deployment. This ICR is not rating this subcomponent, because the project did not supervise the activities that were not loan-financed. The second subcomponent, financed by the APL1, was School Construction, Extensions, and Upgrading (US$48.4 million in PAD, US$94.4 million actual). This subcomponent is rated satisfactory. The planned outputs established in the PAD for the Phase I of the project were to construct 450,000 new student places, and rehabilitate 300 existing schools in rural areas. The project financed the renovation and extension of 199 large rural schools and 123 central village schools. It financed the infrastructure work for 3,188 ICT classrooms at 2,802 basic education schools. Also, under APL1, there were 44 urban schools in Ankara Province renovated and extended, 1,287 rural schools and 42 YIBOs and PIOs renovated all over the country. The design of this component met the urgent needs of expanding school capacity at the time of project start. The focus of construction and rehabilitation were rural and poor urban areas. The major weaknesses of the subcomponent were that the needs assessment and construction work were carried out at central and provincial/subprovincial levels, and the schools and communities had limited input regarding the design and the specifications of the civil works. The technical staff of MONE at the central and - 9 -  provincial levels did not have the capacity and resources to prepare, supervise, and monitor the construction and renovation. The Bank supervision teams observed many defects in the completed works and brought them to the attention of the MONE on several occasions. Component 2: Improved Basic Education Quality (US$233.5 million in PAD, US$190.4 million actual). This component is rated unsatisfactory. The objectives of the component were to improve basic education learning outcomes and increase the relevance of basic education. It would support in-service training for teachers, inspectors, and school principals; provide textbooks and educational materials for basic education schools; provide hardware, software, and training for information technology in basic education schools; and carry out a number of other initiatives designed to improve the quality of basic education. The outputs for this component were outlined within the following five subcomponents, one of which was financed under the Government's Basic Education Program. Subcomponent 1: Information Technology (US$154.0 million in PAD, US$105.9 million actual). The subcomponent is rated unsatisfactory. The objectives of the subcomponent were related to the objectives of Component 2. The outputs set for this subcomponent were: 2800 IT rooms prepared and equipped; the number of trained IT coordinators is doubled. These outputs were achieved under APL1. The APL1 procured computer hardware/software and peripheral equipment for 22,854 rural schools, overhead projectors for 18,517 rural schools, and 6,255 data projectors for 2,802 basic education schools installed with IT classes. The project also purchased 1500 notebooks for master formators, primary education inspectors, and 130 notebooks for the Ministry's inspectors. The APL1 financed the training of 2,058 IT formators, 250 master formators training. Since there were no IT coordinators at the beginning of the project, it was not possible to conclude the number of trained IT coordinators was doubled. According to the PAD, the design of the IT activities of this project was built upon the successful Computer Experimental School component under the National Education Development Project. Under the NEDP project, computer laboratories were established and 235 primary and secondary teachers were trained. Computer-aided education programs for the vocational high schools were also developed. The Computer Experimental School component was evaluated by an international group of experts in 1996, and was found successful in integrating IT into learning and engaging community involvement in the initiative. After 5 1/2 years of APL1 implementation, it was not evident that the outputs accomplished under this subcomponent contributed significantly to the achievement of objectives under this component. Subcomponent 2: In-service Training (US$31.2 million in PAD, US$3.7 million actual). This subcomponent is rated unsatisfactory. The objective of this subcomponent was to support the Government's program on providing two weeks of in-service teacher training each year for approximately 305,000 basic education teachers and school principals. The outputs outlined in the PAD were: by the end of Phase I of the Project, there would be 300,000 existing teachers and 30,000 new teachers who had participated in in-service training programs, and 3,000 school inspectors trained. In-service training was to carry out the training based on modules designed by the MONE, covering topic areas including pedagogical methods, learning assessment, community participation, and classroom management and communication. By the end of the project, APL1 financed only the following: training for 405 teachers on active learning and special learning methods; training for 3,781 principal candidates; training for 861 inspectors on active learning and special education methods; training for 2,956 inspectors on active learning and teaching strategies and use of materials; and English language training for 150 MONE staff. Clearly, the substantial planned in-service training was not carried out under APL1, and the project had little information on Government's training activities. - 10 -  Subcomponent 3: Provision of Essential Teaching and Learning Materials to Rural Schools (US$47.6 million in PAD, US$76.5 million actual). This subcomponent is rated unsatisfactory. The objective of this subcomponent was to improve learning achievement in poorly equipped rural schools. The subcomponent output was to finance the provision and distribution of a package of teaching and learning materials to 25,900 schools in rural areas. The APL1 did distribute educational materials to 22,287 rural schools and 2,993,692 students throughout the country. However, the procurement of these educational materials took an unacceptably long time to complete. The project closing date was extended three times, to a large extent, to allow the completion of procurement of these materials. Subcomponent 4: Student Social Aid (US$45 million through grant funds not financed by the project). The rating for this subcomponent is not applicable as it was not financed by the APL1. The activities were planned under the Government's budget, and the APL1 did not supervise this activity. Subcomponent 5: Assessment of Learning Achievements (US$0.7 million in PAD, US$0.1 million actual). This subcomponent is rated satisfactory. The objective of this subcomponent was to develop and implement assessments of learning outcomes. There were no specific output indicators included in the logframe of the PAD under this subcomponent. The APL1 was to support the representative surveys of approximately 70,000 students in 500 basic education schools to assess learning achievements in four subjects: Turkish, science, math and social studies. The survey was to expand to include computer literacy and foreign language proficiency in 1999 and subsequent years. The results of the assessments would be shared with the provinces to improve educational performance. The Department of Educational Research and Development (ERDD) of MONE completed and analyzed the results of the student achievement tests in Turkish, mathematics, science, and social science. An Interim Report for the Student Achievement Assessment Tests was finalized in April 2003. ERDD also prepared the first drafts of the National Reports for TIMSS 1999 and PIRLS 2001 in April 2003. The results of these tests were put on MONE's website, and the test results, both national and international, were presented to the senior level management at MONE. The results of the tests provided valuable comparative information to MONE. They were also disseminated to universities which had education faculities, provincial and subprovincial directorates of education and the relevant schools. Component 3: Program Implementation Support (US$4.8 million in PAD, US$0.6 million actual). The component is rated unsatisfactory. The component had two subcomponents, which separately explained the objectives of this component. The supporting argument for the rating, the output, and the design of this component is discussed in subcomponent 1. Subcomponent 1: Support for the Program Coordination Team (US$1.8 million in PAD, US$0.6 million actual). This subcomponent is rated unsatisfactory. This subcomponent was to provide implementation support to the Program Coordination Team (PCT). The role of the PCT was not clearly spelled out at the early stage of project implementation, and it caused confusion within the MONE. The name of PCT was later changed to Project Coordination Center (PCC). The project financed the consultants employed in the PCC. However, there was no evidence indicating that implementation was improved after the recruitment of consultants. However, program coordination remained weak throughout project implementation. The implementation capacity in the PCC continued to be limited. Subcomponent 2: Educational Management Information Systems (US$3.0 million in PAD, US$0.0 million actual). The rating for this subcomponent is not applicable as both the Bank and the borrower agreed to delete this subcomponent from the project. This subcomponent was to support the Ministry's Council for Research, Planning (APK) to improve the Ministry's educational information management - 11 -  capacity, train and strengthen APK technical staff in the use of state-of-the art education management information systems, and acquire up-to-date computer hardware and software. In 2000, the Ministry expressed the interest to complete the MIS upgrades, which it initiated under the National Educational Development Project before developing an electronic school mapping system, which was planned under APL1. Component 4: Monitoring and Evaluation (US$13.3 million in PAD, US$1.0 million actual). This component is rated unsatisfactory. The objective of this component was to assess the outcomes of the program and provide continuous feedback to the PCT and the related line units of the Ministry to improve the effectiveness of the program. The supporting argument for the rating, and the discussions on the output as well as the design are included in the subcomponent sections. Subcomponent 1: Social Assessment and Monitoring (US$3.0 million in PAD, US$0.3 million actual). This subcomponent is rated unsatisfactory. The objectives of this subcomponent were to use the on-going evaluation to provide essential feedback for gauging progress, making course corrections, and responding to changing circumstances. There were no output indicators set for this subcomponent. The plan was that the results of monitoring efforts would be evaluated in a participatory manner at several levels including: PCT, relevant departments of the MONE, provincial, and subprovincial administrators. There would also be national seminars and joint evaluations with the Bank, international and national experts. Out of the many activities, the numerous planned assessments, and studies (about 11), the APL1 was able to finance only three studies: (a) cost analysis; (b) social impact; and (c) institutional impact of bussing schools, YIBO/PIO schools, and traditional schools. These studies were conducted after long delays. Details findings of these studies are available in project files. Subcomponent 2: Monitoring Response Facility (US$8.4 million in PAD, US$0.7 million actual). This subcomponent is rated unsatisfactory. The objective of this subcomponent was to provide the Basic Education Program with the flexibility to respond to developments arising during implementation, and to support the development of pilot projects and initiatives, consistent with program objectives, at the school level. The output indicators for this activity were not included in the PAD. During the supervision mission in January 2002, the Bank and the PCC agreed on the output of the "School improvement program implemented in eight provinces". By the closing of the project, APL1 financed 85 MRF projects. Although there had been no impact analysis conducted, field visits carried out by the MONE and the Bank, as well as the monthly Project Monitoring and Evaluation reports PCC received from project schools showed that there were great demands and interest at the school level for the MRF. The MRF projects took an unacceptably long time to implement. Out of the $8.4 million planned for this subcomponent, only $700,000 were spent. Subcomponent 3: Basic Education Promotion (US$1.9 million in PAD, US$0.0 million actual). The rating for this subcomponent is not applicable since the activities under this subcomponent were cancelled at the request of the MONE. The original objective was to provide specific information to the public on the development of the Basic Education Program. During implementation, MONE stated that public support for the Basic Education Program was strong, and it preferred to finance some limited promotion activities including publication of bulletins, under its own budget. The Bank agreed to this proposal. 4.3 Net Present Value/Economic rate of return: Not applicable. 4.4 Financial rate of return: - 12 -  Not applicable. 4.5 Institutional development impact: Overall, the institutional impact of the project has been negligible. During this first phase of APL, the Bank project focused on providing inputs, such as improving the school infrastructure, providing educational materials, purchasing IT equipment, and training of IT coordinators/formators. This was to set the stage for future development of the system. Most of the inputs were provided to the project during the latter part of the project (after 3 extensions). As a result, there was little time for impact to develop. The substantive part of teacher training was carried out by the Government and the Bank's role in this aspect was limited to ICT training and active learning. As pointed out above (section 4.2, Components 3 and 4) and below (section 7.6), the MONE research functions were not fully coordinated among the different departments. The monitoring and evaluation unit established within the PCC under the project was only reporting the activities under the project and had little collaboration with other departments in the MONE. The supervision of project activities was not fully integrated in the MONE. For example, supervision of civil works was coordinated by the Primary Education Department of the MONE, however, MONE suggested that the Investment and Facilities Department of the MONE could have offered more expertise in this aspect. The decisions on the needs of each school were largely made from the central ministry level. MONE staff working at the provincial level were not always capable of preparing subcomponent activities and supervising them. There were many aspects of ongoing education work under the Government's Basic Education Program, but the APL1 did not contribute to streamlining the functions of all parties in the overall education system. Follow-up activities will need to address this need for further streamlining. The design of setting up the PCC away from the MONE was subject to debate. The Bank resorted often to consultants to assist with project management, procurement and financial management. The Government had expressed the need to utilize the line units instead of recruiting additional consultants for the PCC. The lack of communication and collaboration among the line units of the MONE and the PCC affected project implementation. The physical separation of the PCC from the MONE exacerbated this problem. More importantly, this project did not effectively contribute to the institutional capacity building of the Ministry. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: As mentioned earlier, the Marmara Earthquake in August 1999 significantly affected project activities. The Government had to reconstruct and rehabilitate the educational facilities for preschools, basic and secondary education schools, which were destroyed or damaged by the earthquake. Consequently, rehabilitation and upgrading of about 40 urban basic education schools in low income areas of Ankara province were dropped from the project. Although there were further changes later to delete some of the activities related to the Marmara earthquake and to add the activities planned originally, this held up the implementation of some activities. 5.2 Factors generally subject to government control: The financial crisis of late 2000 resulted in a 9.4 percent reduction in GNP, which reduced the Government's ability to finance expenditures. Although the Government was committed to maintain the share of education at 4.25 percent of GNP and counterpart funding for the project was never a problem, the total expenditure on education decreased. The Government's requirements and procedures for the approval of procurement documents caused serious delays. To obtain visas from the Ministry of Finance and later the Courts of Accounts always took a long time. Towards the end of the project, the Treasury Department - 13 -  implemented a new Debt Management Law. This Law and the budget law for 2003 posed additional approval steps for procurement and the payment process. The change of government after the election of November 2002 resulted in the change of senior MONE officials, including the change of the PCC Directors. The changes caused delays since new officials had to familiarize themselves with the project. 5.3 Factors generally subject to implementing agency control: MONE's structure was very fragmented, with departments that had overlapping functions. The collaboration and coordination among the line units and the PCC had always been problematic. Information flow within the Ministry was not effective even towards the end of the project. Communication within the PCC was not effective. The training provided to the PCC staff did not meet expectations in the short-term. Procurement and financial problems were severe. Management issues also caused problems at the provincial and local levels regarding civil works. Provinces were provided with World Bank guidelines and introductory training. The provincial staff went through a lengthy and difficult process since they did not have previous experiences. However, an effective system to monitor the performance of the civil works contractors, supervising consultants and provincial staff was not put in place. The project had to extend the closing date largely due to the delay in procurement of various goods. A total of US$6 million was cancelled by the Bank during the third closing date extension. The reason was that the Bank did not expect that the project would be able to complete all the procurement activities. On the assessment studies, MONE found it hard to agree to have third parties, Ankara and Bashkent Universities, carry out the studies while the Ministry had three of its own research departments: APK, ERDD and Vocational and Technical Education Research and Development Center (METARGEM). There were also disagreements between the Bank and the MONE regarding the terms of reference. 5.4 Costs and financing: The total project cost was estimated to be US$2,825 million, out of which US$300 million was to be Bank financed, and US$2,515 million was to be Government financed. At the time of this ICR, the total disbursed Bank loan was US$286.2 million and the total amount spent from Government's budget was US$2,281.4 million. A total of US$6 million was cancelled from the Loan in June 2003. As of June 24, 2004, the government is requesting that the Bank cancel the additional undisbursed $7.81 million from the Loan. Major changes on costs include: PAD estimate Actual US$ US$ Increase in construction and upgrading of basic education schools 48.4 94.4 Reduction in in-service training programs 31.2 3.7 Increase in educational materials 47.6 76.5 Reduction in information technology 154 105.9 Reduction in assessment of learning outcomes 0.7 0.1 - 14 -  Increase in bussing facility 0.0 2.3 Increase in free lunch for students 0.0 2.0 Reduction in program management team 1.8 0.6 Reduction in social assessment, monitoring and studies 3.0 0.3 Reduction in Monitoring Response Facility 8.4 0.7 Significant delays in procurement led to three extensions of the Loan closing date. Reasons for the persistent procurement problems during project implementation include: the lack of capacity in the PCC although efforts were made to recruit consultants for the procurement unit; the political instability and frequent changes of government's internal rules and regulations prevented the PCC from handling procurement matters efficiently; the insistence of MONE to be involved in the detailed review of procurement documents prior to approvals; and the lack of planning at the MONE for procurement activities. As mentioned in the previous section, the Bank decided to cancel US$6 million in 2003 because it did not feel confident that the government would be able to complete all the planned activities. 6. Sustainability 6.1 Rationale for sustainability rating: The project sustainability is rated likely. Given the limited institutional development impact the project has had, and the uncertainty of project benefits, it is difficult to argue that the benefit of the project would be sustainable. However, from the social and financial point of view, the agenda for education reform and continuous improvement is sustainable. The new Government of Justice and Development Party was committed to the Rapid Coverage for Compulsory Education Program. It has continued the implementation of the program as planned. The Government has provided free sets of textbooks to all primary school children and has launched a "Hundred-Percent Support for Education" campaign. It introduced the Law No. 4842, which allows tax deductions for donations to education, this helped to increase private donations to education five fold. The Government's 2004 budget places special emphasis on education. For the first time, education spending will exceed the defense budget. If education budgets of other public institutions and higher education were excluded, MONE's budget share represents 3% of GNP. The current Government's commitment to the Program was illustrated in details of its budget to the Planning and Budgeting Commission of the Parliament. The Government is planning to review and upgrade the primary education curriculum in 2004. A protocol between the Ministry of Transport and the MONE was signed in December 2003 to provide high speed internet connectivity to approximately 40,000 schools including most of the basic education schools and all secondary schools. According to the protocol, high speed internet connectivity will be provided to these schools for a 3-year period between 2004 and 2006. The cost of internet connectivity will be covered under the MONE's budget. The State Planning Organization has been eager to see the results of the ICT Impact Study. They would like to be informed of the effectiveness of ICT investment carried out so far, and follow up with more informed decisions. The maintenance of schools has been assigned to the municipalities and the provincial governments. - 15 -  Turkey's interest in joining the European Union (EU) will be a major stimulus for further development of the basic education program. UNICEF has been engaged even more actively since 2002 in attracting girls in the poor provinces in Eastern Anatolia. It initiated a pilot project in 2002 a program called "Come on, girls, let's go to school!" which has been expanded to more provinces and is being jointly monitored by the Government and the UNICEF. Other NGOs and private sectors have launched new programs to improve education quality. It should be noted that in order to achieve the maximum benefit, an effective monitoring and evaluation system needs to be strengthened within the MONE and at the provincial level. 6.2 Transition arrangement to regular operations: In July 2002, the Bank approved the Second Basic Education Project (APL2). The follow up tasks for APL1 are included in the implementation of APL2. The Government and the Bank have also been preparing the Secondary Education Project for sometime, and it is presently scheduled for Board presentation in the next fiscal year. Both projects will reinforce the activities under APL1. Bank has also launched a Turkey Education Sector Study: Sustainable Pathways to an Equitable, Effective, and Efficient Education System. This study is to build a technical foundation for policy analysis. It will also support Turkey's goal of revitalizing its education system to deliver quality educational services both equitably and efficiently and assure that Turkey's citizens will be better prepared to compete in global labor markets and participate actively in the knowledge society. This study also aims to enhance the capacity in the relevant government agencies through data collection, workshops, international seminars, or joint preparation of studies, etc., to build up the information base on Turkish education, enhance the institutional base for carrying out related analytic work in the future, and help Turkish authorities replace the outdated paradigm of educational reform. 7. Bank and Borrower Performance Bank 7.1 Lending: The Bank's lending performance was unsatisfactory. The Bank and UNICEF were instrumental in raising the awareness of universal education in Turkey. The Government was clearly committed to the Basic Education Program and was eager to have the Bank's assistance. The actual preparation of APL1 took a long time mainly because: 1) the Government wished to prepare the project itself (demonstrating government ownership, but requiring a longer preparation period); 2) The Government and the Bank agreed that a basic education project pilot be set up within the now closed National Education Development Project that would experiment with some of the concepts that would help in the design of APL1. The Bank addressed the most urgent needs in the sector at the time and focused on equity issues, directing attention to the poor and the rural population. As pointed out earlier in the Quality at Entry section, the project design was rated unsatisfactory by QAG for financial management aspects and readiness for implementation. NEDP created 53 experimental schools on Computer Aided Education and it was confirmed by the Bank in 1996 that this project component was successful. This became the basis for a much larger scale of IT investment under the APL1. The Bank also asked the Government to prepare an IT policy paper and outline a strategy on how to integrate IT into curricula and teacher training. The Bank recognized the needs of carrying out various assessments and monitoring and evaluation during project implementation. However, the social assessments listed under Annex 1, Attachment 1 of the PAD were too fragmented. The most significant weakness to this design was that it did not focus on the outcomes of the activities. In particular, it should have concentrated on institutional capacity building at the central and local level. On the management of the program and the project, efforts to streamline the MONE during preparation were - 16 -  resisted by some of the government officials. Problems on implementation were predicted by the Bank at preparation, but were not fully addressed. An implementation plan was not prepared for the project. Setting up the PCC and recruiting consultants may have helped in the short run to move the project ahead, but they did not contribute to institutional capacity building of the Ministry of Education as well as the provincial government. 7.2 Supervision: The Bank's overall supervision performance is unsatisfactory. At the beginning of project implementation, the Bank and Government focused on the whole Basic Education Program, in particular, the increase in enrollment especially for girls, the number of teachers recruited and trained, and the schools constructed or renovated. This was necessary since APL1's contribution amounted to only 11.9% of the total government's commitment for Phase I of the program. The Bank asked the Borrower to emphasize monitoring and evaluation. Major indicators, particular the indicators for IT component were to be defined by the Ministry after the project took off. It was essential for the Bank to review the project in the context of the whole program particularly since APL1 had only input and output indicators, not the impact or outcome indicators, for the project specifically. Bank supervision missions had been conscientious about ensuring that the project was performing in compliance with the legal covenants, procurement and financial policies. The Bank advocated strongly on the adherence of financial management procedures and procurement rules. Despite the above, Bank did not consistently evaluate its value-added role nor continuously assess its options to make a larger impact on the program, taking into account the changed circumstances, progress made and new challenges. In implementation, especially during the first year or two, the Bank should have focused more on correcting the deficiencies and improving the design. The Government added new courses to the curriculum and launched a massive effort in teacher training. The review of the curriculum was not carried out under the project. This was partly due to Government's unwillingness to have the Bank involved in this area and this was not part of the project. The Bank could have eventually looked for other entry points during the implementation to engage the Government in a dialogue in this very important area, particularly as the Loan was financing civil works, computer equipment and education materials. APLs are designed to be more flexible compared to the ordinary investment loans, but the inflexibilities of the triggers constrained the teams. When reviewing the Bank's input, extensive emphasis was placed on resolving bottlenecks related to procurement and financial issues instead of focusing on outcomes. The IT training for formators and IT coordinators was conducted under the project, but the IT equipment could not be procured for a long time. Refresher training courses had to be conducted for the formators. The Bank repeatedly tried to convince the Government to complete the ICT impact study before loan closing, but failed. While APL1 experienced periodically interrupted progress, the Government's program moved ahead. Frequent changes of Bank task team leaders disrupted continuity of supervision. There was also no implementation plan for a long time after the project started. The ratings in the earlier Project Status Report (PSRs) were mostly satisfactory for project implementation reflecting Government's progress, but in a few cases, the PSRs failed to weigh the potential APL1 problems. Aide-Memoires and Project Status Reports were structured around procurement activities. Measurement of progress by the Bank, followed by the Borrowers, was on completing procurement activities. Partnership with EU, other international agencies, and civil society could also have been a more concerted and effective effort in working with the Government. The sometimes weak communication between the Bank and the Government delayed project implementation. There were issues regarding program and project management and coordination, which - 17 -  the Bank and MONE were not clear on at various times of implementation. The role of the line units in the MONE and the PCC caused confusion among all parties throughout the project. It was noted that the MONE did not understand the concept and ideas associated with the Monitoring Response Facility and this led to a slow start in the submission of appropriate proposals. At the very early stage of project implementation, the Bank had suggested that the PCC select facilitators, or that the Mother and Child Foundation or the Middle East Technical University assist with the preparation of MRF proposals. However, according to the Bank, the Government initially took a different approach by asking provincial education authorities to generate proposals for the schools, without the involvement of schools, teachers, students, and community members. The Government stated in its own assessment (see Annex 10) that the Bank did not provide enough guidance on this subcomponent. The MRF staff claimed that they had requested training and documents on the subject but couldn't get any response from the Bank. The goal and the mechanism of this subcomponent were not clearly defined at the start of the project. Towards the second half of the original implementation period, the Bank made efforts to assist the PCC with organizing the provincial staff and helping the schools prepare proposals with the participation of students, parents, school staff, and local leaders. Eventually, all the 85 MRF projects were prepared in collaboration with the school communities, parents, teachers, students. The Bank's field office did play a critical role in communicating with the PCC and in ensuring the coordination and collaboration within the MONE. Given years of poor project performance and a huge backlog, during the last year of implementation, the Bank's supervision concentrated nearly exclusively on procurement. The Bank intensified supervision efforts after February 2001, following the January supervision of APL1 and preappraisal of APL2. The Bank worked closely with the Borrower in detailed implementation plans for every activity, and instituted very close monitoring. This close monitoring included weekly detailed meetings between the Bank and the Borrower, following every procurement and activity on a step by step basis. It was this effort that made it possible to close the project with most of the basic activities completed, at least in quantitative terms. Nevertheless, there was the need to enlist the Government's effort more rigorously. Government had staff in the MONE to carry out the activities, what the Bank could have assisted was that to effectively utilize MONE's manpower as well as efficiently organize the various departments within MONE to carry out a national program. 7.3 Overall Bank performance: The overall rating for Bank performance is unsatisfactory. The Bank did not continuously seek opportunities, or entry points, to engage in dialogue with the Government on other aspects of education quality (although they were not necessarily in the original project design), such as review of the curriculum and teacher training content. In the 1997 CAS, it was mentioned that the previous National Education Development Project had a difficult experience politically to put national level quality improvement into place, thereby the Bank resorted to focus on a pilot operation to lay the basis for what would be a sizeable investment to meet the acute shortages in school capacity. Nevertheless, continuous Bank involvement, particularly the substantial physical investment planned for APL1, justified the need for much more substance in the dialogue between the Bank and the borrower. In addition, the Bank proceeded with the preparation of APL2 without seriously looking at the critical lessons learned from APL1, such as outcome focused implementation, capacity building of the MONE, and a sustainable monitoring and evaluation system. Borrower 7.4 Preparation: The Borrower's overall performance in preparation was satisfactory. Government expressed strong commitment to the Basic Education Program throughout the preparation process. Major efforts were made by the Government in all aspects of education to realize the expansion of basic education. At the time of - 18 -  project launch in 1998, Government had already committed $1.5 million for the new program. Government established the Basic Education Coordination Council in June 1998. The Council consisted of 9 units and over 40 members. It was envisaged that the line units of the MONE would implement the program and the project with the Council playing the coordination role. Government placed extreme importance to the preparation work and appointed key staff to work with the Bank to formulate the APL1. However, the project design did not take into account the complexity of the project, and the government's capacity to implement the project. Appropriate project management and implementation structure were not established. 7.5 Government implementation performance: The overall rating for Government implementation performance is unsatisfactory. The Government was very committed to the program and the project throughout implementation. There had always been good relations between the Government and the Bank. Frequent meetings were held with senior Bank officials in the Bank's Ankara office. Even during the financial crisis, the Government never failed to provide the counterpart funding. However, the extensive and complicated Government approval procedures delayed project implementation. The visas required by the Ministry of Finance and later the Court of Accounts proved to be very time consuming. Often times, officials in these organizations had to review the documentation themselves, including procurement documents. The clearance from the Ministry of Finance budget personnel who was stationed in the MONE also posed additional bureaucratic hurdles. 7.6 Implementing Agency: The overall rating for implementing agency performance is unsatisfactory. At the beginning of the project, Government established the Basic Education Coordination Council. In December 1999, Government set up the Basic Education Steering Committee to formulate policies for the program, and to plan, direct, and implement project related activities. The agreement had always been that project activities would be carried out by the units of the MONE in coordination with the Basic Education Coordination Council. MONE and the Council did carry out studies of their programs, which led to constructive proposals to improve the efficiency, quality, coverage, and responsiveness of the Program. The PCC had consistently prepared comprehensive progress reports, covering the overall view of the Government's program, although the reports towards the later stage of the project were more oriented to procurement activities. Despite the above-mentioned efforts, MONE continued to be as fragmented in its roles and functions as it had been when the project started. The units (ERDD, APK, and METAGEM) were responsible for various aspects of research. ERDD and the Board of Education did not always work closely together. Monitoring and evaluation of the project and the program was not sufficient within the MONE, including the PCC. The approval process within the MONE continued to be lengthy. The assessments and studies were reduced to four during implementation, with the Child Labor in Turkey study already completed. Nevertheless, MONE still took a long time to actually launch the three remaining studies. This was partly due to MONE's concern that the studies would be carried out by universities rather than the three research units of MONE. The Bank had urged the MONE to update the IT issue paper as well as prepare an E-learning strategy paper. These were sent to the Bank for comments by late 2002. However, after the Bank commented on the papers, the MONE senior management had not taken positions on them. The civil works implementation in the more remote provinces suffered from lack of timely and frequent supervision visits by the MONE. The provinces lacked the capacity to fully assume the responsibility for construction and rehabilitation. There were concerns that teacher training was not synchronized with the educational materials. This matter requires further examination. On the procurement of computer equipment, there was also a delay when the Government insisted on having a particular brand of computers. The Bank had to reaffirm its position related to its procurement principles. - 19 -  7.7 Overall Borrower performance: The overall rating for Borrower performance for the project is unsatisfactory. The Government achieved great progress under the Basic Education Program. As pointed out earlier, there had been major increase in enrollment rates, in the number of teachers recruited and trained and in the availability of schools. The Government always regarded basic education as a high priority. Funding for the sector and the project was emphasized and protected. The Government complied with the loan covenants, as well as the financial and procurement procedures. However, the implementation of the APL1 had been sluggish. The previous sections provide elaborations on the many reasons for the slow implementation. 8. Lessons Learned The Bank needs to improve its monitoring and course correcting during implementation and this process begins with a clear and unambiguous statement of project objectives. Project objectives need to be clearly defined to enable the borrower and the Bank to supervise the project. There is need to focus on the "big picture" in the sector, and assess the effects of Bank's contribution on a regular basis, especially in the evolving country situation. Supervisions, especially at the initial project implementation stage should correct any inadequacies from project preparation. Supervision Aide Memoires and PSRs need to be structured to reflect progress on output and outcomes and not only on activities. Concrete follow up actions on the recommendations from the "Quality at Entry" or "Quality Enhancement Review" need to be ensured. Bank teams should address all major issues highlighted in these reports and justify to the management one way or another on all major concerns. The QER panel provided many valuable comments for APL1, but serious, concrete actions were never taken. Focus on institutional capacity building in monitoring and evaluation of projects is critical. It is important to set up a good system of M&E for a project. It is even more essential to help a client build an M&E system which can be sustainable after the project closes. To explore the potential of government capacity and build upon it often yields better ownership and sustainability. Failure to do this in APL1 led to a breakdown in M&E. The Bank needs to review and assess all of the components which contribute to development objectives, including the ones that are not financed by the Bank. In APL1, there were activities that were not financed by Bank funds, such as student social aid and teacher training (APL1 was only to finance in-service teacher training). These activities had significant impact on the overall program, but since they were not financed by the Bank, they were not reviewed by the Bank. Triggers put into APLs should not be created to be rigid, constraining the government or the Bank team to fully to carry on the project agendas. Triggers should be policy-oriented and results-focused. In APL1, inappropriate triggers that focused on procurement actions were put in which later led the task team concentrate solely on procurement activities. The Bank and the borrower should agree on the outcomes and the monitorable performance measures for ICT investments. ICT investments are complicated and there is ongoing experience in this area of work all over the world. For large investments, the Bank needs to conduct serious cost effectiveness analysis. The evaluation of these kinds of large investments should be constantly assessed. Establishment of a project management structure is a pre-requisite for implementation. It is important to assess the institutional capacity of the implementation agency during preparation of the - 20 -  project, and make arrangements to ensure the management structure would meet the demand of the project implementation. Milestones and targets for implementation need to be agreed and strictly followed by the borrower and the Bank. Each quarterly report should report on the achievement or reasons why the targets were not obtainable. These milestones and targets could be made public to all stakeholders as well as beneficiaries. Bank management would also need to monitor the progress of these milestones/targets and review them in light of the outputs the project achieved. Realistic risk assessment in project design and measures to mitigate them are essential. Projects need to work out the worst case scenarios to ensure adequate implementation plans are in place to avoid delays and setbacks. Lessons from APL1 were not assessed in time in order to incorporate them during the preparation of the APL2. This has led to deficiencies in the design of APL2. Important assessments and evaluations under APL1 were not completed, and therefore could not contribute to the design of APL2. For the Bank to prepare subsequent APLs, apart from meeting the triggers, there should be reviews of the lessons learned from the ongoing APL. 9. Partner Comments (a) Borrower/implementing agency: Detailed evaluation from the borrower is included in Annex 10. (b) Cofinanciers: There were no co-financiers in APL1. (c) Other partners (NGOs/private sector): The NGOs participated in the MRF activities under the project. Private sector provided substantial support to the education sector, but the project did not work with them directly. 10. Additional Information The Government prepared its contribution to this ICR. Government is generally very positive about the APL1. This obviously is a big contrast to the Bank's evaluation of the project. The explanation for the differences of opinion includes: 1) The APL1 was supervised mainly based on the inputs. Government's evaluation is a reflection of how both the borrower and the Bank review the project. The Bank's supervisions, which were also followed by the borrower, were inputs- and activities-focused. Project development objectives and outcomes were not consistently discussed. The inputs the Borrower referred to in its report do not necessarily translate to the achievement of outcomes. For example, under the increasing of basic education quality, the report stressed the various inputs the project was able to provide, but did not comment on the linkage of these inputs to outcomes and objectives. 2) Government is reviewing the APL1 in light of the whole basic education program and what APL1 was able to contribute. Whereas the Bank's evaluation concentrated on the possible impact the project could have or would have on the sector in Turkey. A beneficiary assessment was conducted for this ICR that was based on school visits, and interviews with students, parents, teachers, and schools administrators. The main findings from the assessment were consistent with those of this completion report. Administrators, teachers and parents indicated that they were not consulted on the project activities, however, they stated that what the project provided was actually what the schools needed. The assessment also pointed out that the poor construction - 21 -  of the school buildings was a serious problem. It also indicated that the project activities did contribute to the increase of student enrollment. One of the major concerns the assessment discovered was the lack of maintenance for the school buildings, facilities, and equipment. Detailed report of the beneficiary assessment is available in Annex 8 and in the project file. A stakeholder workshop was conducted to discuss the draft ICR. Participants shared the concerns that the APL1 was not implemented as well as it could have been. Some people expressed the need to draw lessons from this experience in order to improve implementation for the on-going APL2. Other representatives stressed that the government's basic education program achieved a lot, and did not understand the reason for some of the unsatisfactory ratings given by this ICR. The Bank team explained that this evaluation was only for the APL1, not for the whole government's program. An important point was made during this workshop which was: under APL1, exchange of international experiences did not take place. A teacher from an Ankara school expressed the concern that the schools had many other problems, such as lack of water and electricity, and that the computers were purchased for the schools, and the project expected the schools to solve the related problems. They received some substandard videotapes, but the TVs provided to the school could not be fully utilized for educational purposes. In general, participants were eager to learn the lessons and improve the effectiveness of any follow up projects. A summary of the stakeholder workshop is included in annex 9. - 22 -  Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: Indicator/Matrix Projected in last PSR1 Actual/Latest Estimate 450,000 new student places No specific baseline numbers indicated in Under the Government's Basic Education the Loan documents. Program, 12,186 new classrooms and 6,072 additional classrooms were constructed. Rehabilitation 300 existing schools in No specific baseline numbers indicated in Under the APL1, 199 large rural schools rural areas the Loan documents. and 124 central village schools were renovated and extended. Works for 3,188 ICT classrooms at 2,802 schools completed. 1,287 rural schools and 42 YIBOs and PIOs were renovated. 44 Ankara urban schools renovated and extension works completed. 300,000 existing teachers plus 30,000 221,000 teachers received training on ICT Under the Government;s Basic Education new teachers participate in the in-service literacy; 250 master ICT formators Program, 221,000 teachers were trained training programs trained; 405 teachers received training in on the most effective and efficient use of Active Learning and Teaching Method; IT rooms, hardware and software 100 MONE staff received English products. training. Under the APL I, 405 teachers were trained on Active Learning and Special Learning Methods; 3,781 received training for candidate principals; 150 MONE staff received English language course training. 3,000 school inspectors trained 3,000 inspectors trained. 2,956 inspectors received training on Active Learning and Teaching Strategies and Use of Material; 861 inspectors received training on Active Learning and Special Education Methods. Educational materials package distributed Globes, world maps, atlas, math sets, These educational materials were to 25,900 rural schools science materials, story books, children distributed to 22, 287 rural schools and literature distributed. 2,993,692 students throughout the country. Social assistance program implemented in Students bussed. This was part of the government's village schools as well as centralized program. It is implemented, but was not bussing schools financed by the Bank. 2,800 IT rooms prepared and equipped 2,802 ICT rooms equipped, 1,630 Computer hardware/software and notebooks, 22,854 PCs, 6,255 projectors peripheral equipment have been procured provided; overhead projectors to 18,517 for 22,854 rural schools and overhead schools. projectors were provided for 18,517 rural schools and 6,255 data projectors for 2,802 basic education schools installed with IT classes. 1,500 Notebooks for Master Formators, primary education inspectors and 130 notebooks plus 1 server for MONE's inspectors. - 23 -  The number of trained IT coordinators is There were no IT coordinators in the basic Under the Government's Basic Education doubled education schools before the project Program, 221,000 teachers were trained started. on the most effective and efficient use of IT rooms, hardware and software products. Under the APL I, 2,058 IT formators were trained on IT coordination; 250 master formators trained on master IT formator training; 250 master formators again received refresher courses. Output Indicators: Indicator/Matrix Projected in last PSR1 Actual/Latest Estimate Monitoring and Evaluation capacity for Small but weak M & E Unit established The project originally aimed at a the program established at PCC; M & E consultant not hired; M & mechanism that would be created in E Unit at the PCC did not work with MONE to monitor and evaluate the ERDD nor with APK. outcomes of the Project. M & E Unit was set up at the PCC but the Unit has not worked closely with other research and development units of the Ministry, i.e., Research Planning and Coordination Council (APK) and Education Research and Development Department (ERDD), and capacity of MONE has not increased significantly. There is still not clear and mutual understanding at the Ministry regarding the definition of roles and responsibilities of these units and the ways of collaboration among them. APK's MIS system for school planning n/a This activity was not financed under the and management updated and staff Loan. trained 1End of project - 24 -  Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million 1) Expanded Basic Education Coverage Construction and upgrading of basic education schools 48.40 94.40 195.04 Teacher training and recruitment 2) Improve Basic Education Quality In-service training program 31.20 3.70 11.86 Educational materials 47.60 76.50 140.76 Information technology 154.00 105.90 63.57 Assessment of learning outcomes 0.70 0.10 14.29 Student social aid 3) Program Implementation 0.00 4.30 Program management team 1.80 0.60 33.33 EMIS/school mapping 3.00 0.00 0 4) Monitoring and Evaluation Social assessment, monitoring and studies 3.00 0.30 10 Monitoring response facility 8.40 0.70 8.33 Basic Education Promotion 1.90 0.00 0 Total Baseline Cost 300.00 286.50 Total Project Costs 300.00 286.50 Total Financing Required 300.00 286.50 Amounts may not add up to exact totals due to rounding. Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 25.10 62.60 1238.70 1326.40 (0.00) (21.10) (52.80) (0.00) (73.90) 2. Goods 204.30 5.80 2.00 188.30 400.40 (172.20) (4.90) (1.60) (0.00) (178.70) 3. Services 0.00 0.00 56.40 42.00 98.40 (0.00) (0.00) (47.40) (0.00) (47.40) 4. Incremental recurrent 0.00 0.00 0.00 690.00 690.00 costs (0.00) (0.00) (0.00) (0.00) (0.00) Total 204.30 30.90 121.00 2159.00 2515.20 (172.20) (26.00) (101.80) (0.00) (300.00) Amounts may not add up to exact totals due to rounding. Figures in parentheses are the respective amounts financed by the World Bank. - 25 -  Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 23.11 0.53 0.00 23.64 (0.00) (19.57) (74.87) (0.00) (94.44) 2. Goods 181.23 0.56 0.00 0.00 181.78 (179.35) (1.02) (0.67) (0.00) (181.04) 3. Services 0.00 0.00 1.00 0.00 1.00 (0.00) (0.00) (4.60) (0.00) (4.60) 4. Incremental recurrent 0.00 0.00 0.00 0.00 0.00 costs (0.00) (0.00) (0.48) (0.00) (0.48) Total 181.23 23.67 1.52 0.00 206.42 (179.35) (20.59) (80.62) (0.00) (280.56) Services included the training costs. Goods, works and services for subprojects under Part E of the project (MRF) amounted to $0.70 million. Goods under Part F.2 of the project (school lunches in earthquake zone) amounted to $2.29 million. Contracted transport services (student bussing in earthquake zone) amounted to $1.98 million. 1/Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2/Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) Percentage of Appraisal Component Appraisal Estimate Actual/Latest Estimate Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF. 1) Expanded Basic Education Coverage Construction and 48.40 1897.40 94.40 1624.20 195.0 85.6 upgrading of basic education schools Teacher training and recruitment 2) Improve Basic Education Quality In-service training 31.20 5.80 3.70 0.10 11.9 1.7 program Educational materials 47.60 44.10 76.50 267.90 160.7 607.5 Information technology 154.00 28.80 105.90 6.70 68.8 23.3 Assessment of learning 0.70 0.10 0.10 0.00 14.3 0.0 outcomes 3) Program Implementation Program management 1.80 42.40 0.60 0.00 33.3 0.0 team EMIS/school mapping 3.00 0.60 0.00 0.00 0.0 0.0 - 26 -  4) Monitoring and Evaluation Social assessment, 3.00 0.60 0.30 0.10 10.0 16.7 monitoring and studies Monitoring response 8.40 1.60 0.70 0.10 8.3 6.3 facility Basic Education 1.90 0.30 0.00 0.00 0.0 0.0 Promotion Additional cost include: Student Social Aid: Appraisal estimate: Bank - $0.0 million; Government - $0.0 million Actual: Bank $4.3 million; Government $382.3 million - 27 -  Annex 3. Economic Costs and Benefits No economic costs and benefits analysis was carried out at the time of the ICR. As stated in the Quality at Entry section, the Quality at Entry Assessment rated marginal satisfactory for technical and economic aspects. The Assessment stated that used a relatively simple system, even for the computer component of the project, to justify the technical soundness of the project/program. Options were not examined carefully regarding boarding, bussing, and double-shifts schools. There were no alternative designs seriously considered by the project. The preparation team disagreed with this statement and indicated they had urged the government explore options. On the quality and coherence of economic rationale underpinning the project, the Assessment pointed out that micro justification was okay, but it was not helpful in choosing among options and that macro analysis was lacking. - 28 -  Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 02/04/98 10 Principal Education Economist (1); Counsel (1); Disbursement Officer (1); Social Development Sector Leader (1); Senior Implementation Specialist (1); Senior Informatics Specialist (1); Senior Country Officer (1); Human Development Specialist (1); Policy Analyst (1); Senior Textbook Specialist (1) Appraisal/Negotiation 05/05/98 9 Principal Education Economist (1); Counsel (1); Disbursement Officer (1); Social Development Sector Leader (1); Senior Implementation Specialist (1); Senior Informatics Specialist (1); Senior Country Officer (1); Human Development Specialist (1); Policy Analyst (1) 05/18/98 9 Principal Education Economist (1); Counsel (1); Disbursement Officer (1); Social Development Sector Leader (1); Senior Implementation Specialist (1); Senior Informatics Specialist (1); Senior Country Officer (1); Human Development Specialist (1); Policy Analyst (1) Supervision 12/18/1998 9 MISSION LEADER, ECONOM S S (1); HUMAN DEV. SPECIALIST (1); SR. IMPLEMENT. SPEC. (1); SR. EDUCATION SPEC. (1); FINANCIAL MNGMT. SPEC. (2); SOC.DEV. SECTOR LEADER (1); SOCIAL SCIENTIST (1); COMM. DEVEL. SPEC. (1) - 29 -  07/16/1999 9 MISSION LEADER, PR ECO S HS (1); HUMAN DEVELOPMENT SPEC (1); SR IMPLEMENTATION SPEC (1); SR EDUCATION SPEC (1); SR INFORMATICS SPEC (1); LEAD INFO SPECIALIST (1); IT SPECIALIST (2); PROCUREMENT SPECIALIST (1) 12/08/2000 8 PROGRAM TEAM LEADER S HS (1); SR. EDUCATION SPEC. (1); OPERATIONS ANALYST (1); OPERATIONS OFFICER (2); PROCUREMENT SPECIALIST (1); SR. MGT. FINANC. SPEC. (1); SR. INFORMATICS SPEC. (1) 02/28/2001 9 TTL, LEAD EDU. SPEC. (1); S S PTL, CO-TTL (1); SR. OPERATIONS OFFICER (1); PROCUREMENT SPECIALIST (1); FINANCIAL MAN. SPEC. (1); OPERATIONS ANALYST (1); WORLDLINKS COORD. (1); CONSULTANT IT (1); CONSULTANT ARCHITECT (1) 09/20/2002 10 TTL, LEAD EDU. SPEC. (1); S S SR. EDUCATION ECON. (1); SR. OPERATIONS OFF. (1); SR. EDUCATION SPEC. (1); LEAD INFORMATICS SPEC. (1); OPERATIONS ANALYST (1); FINANCIAL MANA. SPEC. (1); PROCUREMENT SPEC. (1); CONSULTANT ICT (1); CONSULTANT ARCHITECT (1) 10/24/2003 6 TTL & SR. OPERAT. OFF. (1); S S LEAD EDCUATION SPEC. (1); LEAD INFORMATICS SPEC. (1); OPERATIONS OFFICER (1); PROCUREMENT SPECIALIST (1); FMS SPECIALIST (1) ICR (b) Staff: - 30 -  Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation Appraisal/Negotiation 158 395 Supervision 462 1155 ICR 16 44 Total - 31 -  Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 32 -  Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 33 -  Annex 7. List of Supporting Documents The Institutional Evaluation Research on Primary Education Schools. Ministry of Education, and Baskent University, Ankara August 2002. Research on Social Impact Assessment of Basic Education Schools. Ministry of Education Project Coordination Center, and Ankara University, Ankara July 2002. The Cost Analysis Research on Primary Education Schools. Ministry of Education, and Baskent University, Ankara August 2002. Turkey - Education Sector Study: Sustainability Pathways to an Equitable, Effective, and Efficient Education System. A Programmatic Sector Study Concept Note. World Bank, Washington DC. July 2003. Turkey: Rapid Coverage for Compulsory Education Program. Case Study. Ankara January 2004. Beneficiary Assessment for Implementation Completion Report. Ankara May 2004. - 34 -  Annex 8. Beneficiary Survey Results BENEFICIARY ASSESSMENT FOR IMPLEMENTATION COMPLETION REPORT Project Background The Basic Education Project under that World Bank financed Project was to support the first phase of the Turkish Government's Basic Education Program which aims to: (a) achieve universal coverage in an expanded, eight-year Basic Education cycle; (b) improve the quality and relevance of Basic Education; and (c) make Basic Education schools learning resources for the community. The World Bank financed Project planned to achieve these objectives by: (a) the expansion of the capacity of Basic Education schools; (b) training of teachers, school principals and inspectors; (c) development of central and provincial implementation capacity to carry out the Program; and (d) creation of a mechanism to monitor and evaluate the outcome of the program. Specifically, the Project was to: a) expanding the capacity of basic education schools throughout the country, (b) facilitating school attendance of children who are at least likely to attend school, c) reducing classroom overcrowding and double shifting, d) improving training and incentives for teachers, e) improving the supply of educational materials to basic schools, f) introducing computer-aid learning, and increasing parental and community involvement in schools. The World Bank financed Project consisted of four components of which particular interest to this assessment are the following activities: 1) Expanded Basic Education Coverage. The component financed the rehabilitation of about 200 village schools, 100 central village schools and 40 urban schools. The project also financed the rehabilitation of about 1300 formerly closed rural basic education schools in provinces and 11 basic education boarding schools in provinces throughout Turkey. 2) Improved Basic Education Quality. This component financed the upgrading of computer rooms and for the provisions of computer equipment and software. It also provided in-service teacher training of computer teachers (formators). The component provided essential teaching and learning materials to rural schools. The Purpose of the Beneficiary Assessment The Project activities has been closed end December 2003 and an Implementation Completion Report is being prepared by the World Bank. An Intensive Learning Completion procedure is being followed to ensure that lessons learned from the Project are fully explored and incorporated into the World Bank's future work. The present beneficiary assessment is part of that process of discussion with the beneficiaries on the benefits of the project. The purpose of this beneficiary assessment is to discuss with the participants in the Project, and those affected by it, the impact of the project activities on the educational system, and its relevance to the stated Project goals. Beneficiary Assessment Methodology Qualitative research method was used for this assessment study. Data were gathered through conventional interviews, focus group discussions, and observations. Provincial officials, sub-provincial officials, community leaders, school principals and assistant principals, parents, and students in the benefited schools were used as sources of data. A semi-structured interview form was developed for interview. During the interviews the - 35 -  interviewers took notes or used the tape recorder with the consent of the related person. Content analysis was used to describe the data gathered with open ended questions directed to the participants. Selection of Provinces and Schools The provinces have been classified into two categories as developed and undeveloped according to their level of development determined by State Planning Organization (SPO or DPT). Two provinces from each category were taken. As developed provinces Ankara and Antalya were selected, while Mardin and Diyarbak