Turkey - National Education Development

 ICRR 10738 Report Number : ICRR10738 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 08/17/2000 PROJ ID : P009029 Appraisal Actual Project Name : National Education Project Costs 177.2 149.1 Development US$M ) (US$M) Country : Turkey Loan/ US$M ) 90.2 Loan /Credit (US$M) 69.1 Sector (s): Tertiary Education Cofinancing US$M ) (US$M) L/C Number : L3192 Board Approval 90 FY ) (FY) Partners involved : None Closing Date 06/30/1998 06/30/1999 Prepared by : Reviewed by : Group Manager : Group : 2. Project Objectives and Components a. Objectives The project's development objective was to modernize the education and training system to meet OECD standards . The project supported the Government's efforts to improve student learning and achievement throughout the country through policy and operational measures for improving the quality of primary and secondary education and teacher education, and the effectiveness of management and administration (as below under components). An amendment to the Loan Agreement was approved in 1996 which added the objective to expand the coverage of basic education (grades 1--8) and early childhood education (ECE). b. Components a) primary & secondary education was to be improved by : establishing a method for reviewing, upgrading, and updating curricula and instructional materials on a continual basis; improving the system for textbook and materials distribution; and strengthening the Ministry of National Education's (MONE) testing and evaluation capabilities; b ) teacher education was to be improved by: revising and upgrading preservice teacher training curricula, textbooks, and instructional materials, and establishing a system of assessing development needs; c ) effectiveness of management and administration in the MONE was to be improved by establishing a mechanism for strengthening internal communications within MONE, improving the MONE's management information system (MIS) capabilities and by moving selected aspects of basic planning and decision making to the provincial education directorates . In addition, the project was to include a number of policy initiatives for strengthening the education system . It also provided for minor renovation, furnishing and equipping of 200 curriculum laboratory schools; 53 experimental schools for the pilot computer education program; and finally construction and renovation of 4 education resource centers (ERCs). c. Comments on Project Cost, Financing and Dates The ICR version available to OED incorrectly shows the loan amount as $ 10 million on the title page. The restructured project reallocated funds amounting to $ 18.5 m. Of this sum, $8.1 m. was finally canceled from the Basic Education Pilot component. The project was implemented through two coordinating units, the Council of Higher Education (YOK) and MONE, which maintained the two accounts, though this complicated administration of the special account which was not kept separately for each unit . 3. Achievement of Relevant Objectives: Primary and secondary education . The original objective to improve quality was partially achieved through multiple physical inputs and staff training . Outcomes are not fully recorded in the ICR and have been supplemented by communication with the Region. Capacity to conduct student tests and assessments was developed and regular achievement tests were mounted in about 400,000 schools, though the ICR and the borrower's report do not report results for student learning and achievement actually improved . The Educational Research and Development Center also developed and applied diagnostic tests in 204 selected schools, including under the Third International Mathematics and Science Study Repeat Study (results due in September 2000). However, the ERDDC was not established as a legal entity and suffered from resource constraints . Six research studies on important topics were done, but not well, and the MONE did not respond to them . A system for improving curricula and instructional materials was partially achieved but only 3 out of 21  curricula developed under a new model were approved by the Board of Education (BOE). Over 200 Curriculum Laboratory Schools were provided with computer and communications equipment and nearly 2,400 teachers were trained. The schools attracted high demand from teachers, students and communities. Although the evaluation was not completed by completion, quality was apparently maintained by restricting enrollment. After completion, the Government planned to expand the CLSs country -wide. The Region reports that analysis of the assessment results showed that learning achievement in the project schools was significantly superior to that in non -project schools. A pilot to test Computer Experimental/Laboratory Schools was similarly successful based on an interim evaluation and an additional unplanned expansion of the pilot resulted in an additional 182 CEE/LS schools being designated. Quality improvement in textbook publication was not achieved while targets for using private printers were partially achieved--the shortcomings due mainly to lack of cooperation between key MONE departments and exacerbated by the BOE. A major training program was conducted by the Film, Radio and TV Center for developing non -textbook materials. Failure to procure the main equipment and to maintain plant meant that improvements in production capacity and FRTVC capability did not occur . Teacher Training . Input targets were mainly achieved and exceeded but the ICR gives little indication whether the quality of training actually improved, except that FoEs used new books and equipment . Given the lack of curriculum change and institutional supports, it appears unlikely that training programs alone would have led to any more than modest improvements. The component under YOK was expanded to accommodate 8 new Faculties of Education. And so 42 FoEs in total received equipment and books, which were widely used by staff and students . After major delays and problems with technical assistance (TA) and fellowships (4 and 3 years respectively), the MA and Ph.D. programs were substantially successful, a system of faculty -school partnerships was set up and 2,000 partnerships received training; and an accreditation program was launched through a standards reference publication, dissemination to 366 people and training for 36 assessors. The component under the MONE expanded and upgraded its in -service-training department and the development of training materials and programs . Through central and local courses, nearly 600,000 participants received in-service training, 300 % of the original target. Three ERCs were established, one more than originally planned and all were equipped . But they were not successfully used by administrators or teachers due to unclear mandates and MONE's lack of commitment and failure to provide qualified staff . Improved MONE Management Effectiveness . The component was not fully implemented, largely because of the poor institutional environment and did not achieve its objective . Two units were merged into a Management Assessment and Development Unit which developed MONE job descriptions and staffing regulations, but it was not integrated into the ministry . Instead, it was given separate status through the Minister and still has uncertain legal status and institutional supports . Similarly, inspectors training was undertaken but not supported by the Board of Inspectors . Capacities in planning and budgeting were successfully strengthened through studies, training and some budget processing changes . The MIS infrastructure, however, was only partially established . It did not accomplish linkages and networking between central and provincial agencies and thus did not achieve efficiency gains planned . The additional components under the revised components were implemented successfully and made substantial inputs into the preparation of follow -on operations. The ECE program, ongoing with NGO/UNICEF assistance since the early-1980s, has been expanded and is expected to lead to gains in educational performance in basic education in the long run. The basic education pilot component laid a basis for the ongoing Basic Education Project and an unprecedented expansion (900,000 students) in basic education since 1997. 4. Significant Outcomes/Impacts: The additional components have been useful in identifying and preparing Bank -assisted projects in basic, secondary and higher education. 5. Significant Shortcomings (including non-compliance with safeguard policies): The major shortcomings were: over-complex design; inadequate preparation (steered by the Bank team and a committed undersecretary who left office too soon ); and weak ownership and commitment by some project -related units and sector agencies . This situation resulted in large investments in facilities, equipment and training, as in previous projects, with uneven developmental outcome in terms of the quality of primary and secondary education and the management framework. The ICR (p. 11) states that the project design had focused on improved inputs to learning and educational management, and assumed that the improved inputs would bring about expected improvements in educational outcomes . In a number of cases, this assumption proved incorrect . Inertia and rivalry within MONE and inadequate incentives to attract qualified staff meant that the project fell short of achieving the full benefit of the improved inputs. Curriculum development, a key input, is provided as an example of uneven achievement. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory Management efficiency of MONE did not become more effective as planned, impeding sector progress towards original objectives. Institutional Dev .: Modest Modest Institutional inputs and training targets achieved but legal framework and reform not supported by line ministry. Sustainability : Uncertain * Unlikely Except for Bank lending pipeline, risks are high for ownership, commitment, budget availability, inter-agency cooperation and availability of technical skills in MONE . Bank Performance : Satisfactory Unsatisfactory A marginally unsatisfactory performance . Appraisal and support to preparation weak, as ICR states, and supervision not proactive and lacking in appropriate skills-mix. Restructured only in 1996, five plus years after loan approval . Indicators not retrofitted, as required. Bank performance improved after 1995 and borrower expressed satisfaction with advice. Borrower Perf .: Satisfactory Satisfactory Preparation did not ensure feasibility, commitment, appropriate management and budget for the MONE component. MONE units performed unevenly, though the Bank shares responsibility for this . YOK had high institutional capacity from the start and performed satisfactorily, achieving targets. Quality of ICR : Satisfactory NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. NOTE: 7. Lessons of Broad Applicability: ICR. ICR Project design needs to be based on realistic assessment of constraints and a recognition of which constraints can be mitigated by project design and which cannot be . A project aiming to reform and restructure basic and secondary education should have been launched after the legal base was obtained and this base should be compatible with the agreed project goals and objectives . If it is not possible to establish the legal basis early, then the Bank should be proactive in seeking strong remedies . Leadership is important in launching new programs but is not sufficient for successful project implementation . As change permeates all levels of organizations, the assignment of qualified staff who are involved in implementation, is highly important also, including all the affected sector agencies and units and the project coordinating units . Borrower's reports include many practical lessons and advice . YOK: Faculty-school partnerships and standards and accreditation enable nation -wide contacts to be maintained, YOK: fellowship participants to be utilized and mounting motivation and commitment to be harnessed for further progress . MONE : It is necessary: (a) to establish a legal project implementation unit accountable for the MONE: government-supported project budget; and (b) to identify the terms of reference of the unit and its legally recruited staff and other units involved in implementation . 8. Assessment Recommended? Yes No Why? Ratings require validation. The problems reported for this project are repetitive in the sector lending program for the country and call for an in -depth sector/country review by the Region and /or OED. 9. Comments on Quality of ICR: The ICR is marginally satisfactory . It merits a satisfactory rating because it is candid, includes full description of implementation progress and provides convincing evaluative comment . However, the ratings are not fully supported, in part because important information on outcomes is not provided . This was subsequently provided by the Region . The ICR could be improved by distinguishing between implementation performance and achievements /outcomes; and between outcomes directly attributable the project and those due to follow -on projects or extraneous factors .