Document of The World Bank FOR OFFICIAL USE ONLY Report No. 15386 IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY INDUSTRIAL SCHOOLS PROJECT (LOAN 2536-TU) FEBRUARY 21, 1996 EC1/2HR Human Resources Sector Operating Division Central and Sourthern Europe Departments Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS At Appraisal: 100TL = US$0.238 US$1 = 420TL At Completion 10,OOOTL = US$0.337 US$1 = 29,609TL WEIGHTS AND MEASURES (Metric System) FISCAL YEAR OF BORROWER January 1 - December 31 ABBREVIATIONS AND ACRONYMS CORD Center for Occupational Research and Development CNCM Computerized Numerically Controlled Machines DTVE Directorate of Technical and Vocational Education ECI Europe and Central Asia Region, Department I EC1/2HR Human Resources Division for EC Departments land 2 GDVTE General Directorate of Vocational and Technical Education GOVT Republic of Turkey IBRD The World Bank ICR Implementation Completion Report ILO International Labour Organization ISP Industrial Schools Project MONE Ministry of National Education METARGEM Vocational Education Research and Development Center PCU Project Coordination Unit PIU Project Implementation Unit PRITT Project Research and In-service Teacher Training RMT World Bank Resident Mission Turkey SAR Staff Appraisal Report USATDP United States Technical Development Program VTET Vocational and Technical Education FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY INDUSTRIAL SCHOOLS PROJECT [LOAN 2536-TU] CONTENTS PREFACE EVALUATION SUMMARY PART I. PERFORMANCE ASSESSMENT FROM IBRD'S PERSPECTIVE A. Statement and Evaluation of Project Objectives .......................... 1 B. Achievement ofObjectives ....................................... 2 C. Major Factors Affecting Project Implementation ......................... 2 D . Project Sustainability ..........................................7 E. Bank Performance ............................................ 7 F. Borrower Perform ance ......................................... 8 G. Assessment of Outcome ........................................ 8 H . Future Operations ............................................ 9 I. Key Lessons Learned .......................................... 9 PART II. STATISTICAL ANNEXES Table 1: Summary of Assessments Table 2: Related Bank Loans Table 3: Project Timetable Table 4: Loan Disbursements: Cumulative Appraised and Actual Table 5: Key Indicators for Project Implementation Table 6: Studies Included in the Project Table 7A: Project Costs Table 7B: Project Financing Table 8: Status of Legal Covenants Table 9: Bank Resources: Staff Inputs Table 10: Bank Resources: Missions APPENDIX A Borrower's Evaluation Report: Conclusions and Recommendations MAP: IBRD#27630 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed wiLhout World Bank authorization. IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY INDUSTRIAL SCHOOLS PROJECT (LOAN 2536-TU) Preface This is the Implementation Completion Report (ICR) for the Industrial Schools Project in Turkey, for which Loan 2536-TU in the amount of US$57.7 Million equivalent was approved on May 29, 1985, and made effective on July 19, 1985. The Loan was closed on December 31, 1994, after two one-year extensions from the original closing date of December 31, 1992. Final disbursement took place on April 25, 1995, at which time a balance of US$1.98 Million was cancelled. This ICR was prepared by Patricia Kleysteuber, Task Manager (EC1/2HR), with the support of Judith De Costa (EC1I/2HR). Ferda Sahmali (RMT) and Darrell Hull, (Curriculum Development Consultant) prepared background reports which are available in the ECA Files. The ICR was reviewed by Ralph W. Harbison, Division Chief, EC1/2HR, and Spyros Margetis, Acting Projects Adviser, EC1. This ICR is also based on the results of a Completion Mission in February 1995, visits to a representative sample of schools by Ms. Sahmali and during the completion mission, and materials available in ECA files. The Borrower (Ministry of National Education) prepared its own evaluation report, and the report's conclusions are included verbatim as Appendix A of this report. The Borrower also contributed to the preparation of this report by providing data, contributing to discussions, and commenting on the draft ICR. IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY INDUSTRIAL SCHOOLS PROJECT (LOAN 2536-TU) Evaluation Summary Introduction. i. This project was one of a series of projects in secondary and post secondary education in Turkey designed in the early 1980's to develop suitably trained technical manpower to promote growth in export-oriented industries. This strategy conformed with the Bank's dialogue with the government of Turkey as outlined in the "Industrialization and Trade Strategy Report". This Loan was preceded by Loan 748-TU (closed 1981), also supporting industrial schools, and was followed by three projects appraised in the mid 1980's -- two post-secondary technician training projects (Loan 2399-TU and 2922-TU) and a non-formal vocational education project (Ln.2776-TU). In particular, these projects addressed the lack of technology programs and equipment in technical and vocational schools, and reflected the importance given to vocational education. Project Objectives ii. The three objectives of the project were to improve the quality of programs, increase the quantity of trained industrial manpower, and improve the planning and management capacity of the Directorate of Technical and Vocational Education (DTVE) in the Ministry of National Education (MONE). These objectives reflected the Government's priority to promote industrial productivity, expand the manufacturing sectors, increase exports, and maintain global competitiveness while, concurrently, promoting economic growth. Provision of adequate training equipment for selected trade specializations was essential to increase labor productivity and the Bank would support the Government's goal to increase enrollment in Industrial Schools from 17% to 24% of secondary education. These objectives were relevant to Turkey's strategy to achieve market competitiveness and economic growth; these objectives are relevant today as Turkey works at maintaining its market share of exports in an increasingly global economy, and joins the European Customs Union. iii. Evaluation of Objectives. The two project objectives to improve the quality and quantity of vocational education were achievable, and the project did improve the quality of vocational education and delivered better technology equipment and programs, books, and teacher training. However, in retrospect, it is recognized that the specializations were too narrowly defined and focused excessively on vocational skills training which today is inappropriate at the secondary level. The third objective to achieve institutional development was rather unrealistic due to the limited technical assistance resources provided under the project in proportion to the magnitude of the needs in the MONE for efficient management and for effective formulation of sector policies, and also due to the low status and inadequate pedagogical and technical knowledge of teachers and schools managers. An ability to plan for evolving needs in vocational education and an ability to manage a quality education system (quality teachers, quality programs, in a quality environment) was not reached. Implementation Experience and Results 6 . iv. Project Outcome. The project's outcome is rated as "satisfactory " since it was successful in achieving its appraised objectives to a substantial degree. Physical targets were surpassed, but quality was sometimes deficient mainly due to poor monitoring and support, and poor teacher qualifications; these efficiency factors should have been more carefully assessed during preparation and remedied during implementation. The project equipped 79 Industrial Schools in 13 program specializations. In addition, computers and software were provided for 200 Computer Classrooms in Industrial Schools. Technical assistance and fellowship training was provided, as appraised, for curriculum development and teacher/staff training. Also, the production of 42 technical textbooks was completed against 50 expected at appraisal. In terms of enrollment, it was anticipated that 3,573 students would graduate annually; in 1994, there were 4,728 graduates, an increase of 32%; enrollment demand for these schools still far exceeds capacity due to shortage of secondary schools. During the two-year project extension, a comprehensive study on vocational education and the strengthening of teacher training were added and successfully completed. Most important, the project is expected to improve its quality shortcomings during its operational phase, as outlined in the borrower's completion report and project sustainability plans. v. Project Costs and Implementation Table. The project had significant cost savings, mainly due to a decrease in the price of technology equipment, but also due to the MONE's responsible choice of least-cost solutions and filling essential needs. The savings in the equipment category allowed for a reallocation to other expenditure categories, mainly technical assistance. The appraised project cost was US$72.7 Million (Government: US$15.0M and Bank: US$57.7M); actual costs were US$68.8 Million, or 95% of appraised costs (Government: US$13.2M and Bank: US$55.63M). A remaining Loan balance of US$1.98 Million was cancelled. The main difference in expenditures between appraised and actual costs was due to a reallocation to the technical assistance and fellowships category which was raised from US$3. IM to US$5.9M. In terms of disbursements, the project generally lagged far behind appraised estimates due to the fact that expenditures were clustered around large procurement packages. By the original Loan closing date, only 63% of the Loan had been disbursed. At that time, the MONE requested to expand the project significantly; the Bank, however, declined to extend the project programs to additional schools as there was already recognition of the need for less specialized vocational skills than the project provided, and for increased flexible vocational cluster programs. The Bank agreed instead to support activities that would improve education quality in vocational schools, and financed: computer literacy programs to improve the general content of vocational education; a study in vocational education; and, to strengthen the weakest part of the project, provision of additional in- service teacher training. These expanded activities amounted to about US$10.0 Million, or 18% of the total actual Loan of US$55.63M. vi. Bank Performance. Although the Borrower appeared satisfied with the degree of project preparation, in retrospect, there was need for greater detail, in particular regarding project management requirements, scheduling L:" project activities, terms of reference for consultancies, budget monitoring and related tools to oversee a national project that is centrally managed, and, most important, involvement of the schools in selecting their programs and the corresponding equipment and training. Today, the Bank gives greater priority to the issue of ownership by the stakeholders and the need for detailed description of implementation arrangements. vii. Borrower Performance. The Borrower maintained its full commitment to supporting project objectives for the duration of the project and worked closely with the Bank. The Borrower's shortcomings were mainly institutional and operational -- institutionally with severe constraints in obtaining qualified staff and reluctance to use technical assistance in project management and implementation and, operationally, with limited implementation ability in a highly centralized yet weak system. Both the Borrower's and the Bank's performance were on average "satisfactory " for the Project Preparation stage. Performance by the Borrower in Project Implementation is rated "deficient " due to overall inefficiencies but recognizing this is a heavy burden on the implementing agency, while the Bank's performance is rated "satisfactory " but considering its limited responsibility for on-the-ground implementation. Summary of Key Lessons Learned for Future Operations viii. There are several lessons learned from the implementation of this project. Project Management Professionals Although the MONE objected to using project management consultants, evidence of the benefits of contracting the needed expertise has been clearly demonstrated. In the Bank's view, the financial losses from poor project implementation in terms of delays and inefficiencies far outweigh the cost of contracting the needed management and administrative services to deliver inputs on time to beneficiaries. Implementation of Technical Assistance (TA) It is essential that a detailed implementation plan for technical assistance be completed during project preparation and include the identification of qualified national counterparts for consultants. Terms of reference should be detailed and understood by both the client and the contractor. Curriculum Reform Curriculum reform is a difficult task, especially in vocational education. To introduce new curricula, appropriate materials, and essential equipment at a national scale is a challenge. It is also difficult to include local industry in a centrally managed project, although industry does not always appropriately influence training as the focus is on narrow specializations for the short term; however, in the medium and long term both workers and industry benefit from broader and flexible skills to be able to adapt to future markets and new technologies. Lack of detail in design of an implementation plan can be detrimental, in particular, definition of the role of local schools in program development and changes in curricula and pedagogy. The importance of these factors is often underestimated at appraisal and results in unrealistic expectations of the project's impact and benefits. In-Service-Teacher Training and Staff Training The importance of in-service training is often underestimated with inadequate provision in terms of numbers of participants and the types of training needed to derive full benefit of project inputs such as new curricula and materials. During project preparation the detailed arrangements for the implementation of an in-service training program need to be carefully designed, and financing should preferably be made available through the Loan. Fellowship Training Objectives Fellowship training programs are costly and their objectives need to be carefully appraised. This includes an assessment of the availability of qualified candidates and their respective suitability for positions they would occupy upon their return. The experience in this and other projects in Turkey indicate a retention rate of about 30%. This low retention rate is further affected by lack of follow-up of candidates to ensure that they apply their newly-acquired skills in targeted key positions. Adjustment in Project Design Because of the fast pace of development in science and technology, and the changing demands in the work environment, it is essential in the field of vocational education to offer up-to-date and flexible programs. Unfortunately, the project remained as designed, focusing on teaching old technologies rather than emerging technologies, experimentation, and the need to become life-long learners. The need for job-specific skills is being replaced by the need to apply academic skills in the workplace. Curriculum reforms now support an integrated coursework program combining academics with vocational learning. This stronger academic foundation is essential for future learning and development of life-long learning skills. It is important that projects and loan agreements incorporate tools for corrections in project design and content and encourage a periodic reassessment of needs. Procurement Procedures In the case of technical and vocational education, where the needs and interests of a community are of essence, it is not desirable to impose at the individual school level standardized programs and equipment which have been centrally selected. The interest, need, and motivation of each local school are essential to the success of a vocational or technical program. An assessment of both the benefits and risks of centralized or locally- planned participatory procurement is essential. Sometimes losses from lack of ownership override any possible cost savings derived from economies of scale through bulk procurement. Plans for Future Project Operations and Sustainability ix. The MONE has made a serious commitment to continuing monitoring the operation of project- supported schools, and continues to have a Project Coordinator. The operational phase of the project, including performance indicators to be monitored, were included in the borrower's evaluation report. This provides assurances that the project's benefits will continue to increase. The Bank has highlighted to the MONE management that many of the benefits from the project are yet to materialize as a substantive amount of equipment has yet to be efficiently used due to lack of teacher training. Follow-up by MONE will be a key factor to improve the project's impact and its cost-effectiveness. 1 Part I. PERFORMANCE ASSESSMENT FROM IBRD'S PERSPECTIVE A. Statement and Evaluation of Project Objectives 1. Statement of Objectives The Industrial School Project (ISP) had three objectives as stated in the Staff Appraisal Report (SAR) dated April 18, 1985: "(a) to improve the quality and increase the quantity of trained industrial manpower by equipping and re-equipping seven existing trade specializations and equipping four new trade specializations in 39 existing industrial schools; (b) to improve the quality of existing curricula and training materials, and to assist the development of programs for new trade specializations; and (c) to improve the planning and management capacity of the DTVE by providing technical assistance. " These objectives clearly reflected the Government's priority to improve industrial productivity, expand the manufacturing sectors, increase exports and maintain global competitiveness and, concurrently, promote economic growth. Through sector work, inadequate training capacity was identified as one of the causes of low industrial productivity. The ISP was to address the particular issue of "inadequate industrial and vocational training facilities at the pre-service secondary school level", as identified in the Bank's "Industrialization and Trade Strategy" (Rpt. 3641-TU, Feb. 1982). Provision of adequate training equipment was essential to improve the quality of training to be able to increase labor productivity and support an advanced technological environment. The Bank would assist the Government to reach its goal of expanding enrollment in Industrial Schools from 17% to 24% of secondary education by 1991. Since these schools feed the manufacturing export sector, their success was critical to increased productivity. 2. Evaluation of Objectives Certainly the project objectives were relevant and directly responded to the Country's strategy to achieve economic development and stability. Many of these concerns continue to be valid today as Turkey moves towards joining the European Union, and make its manufactured goods competitive in an increasingly global economy. The areas selected for development were those where technology changes are occurring rapidly. Letter training programs, better access and opportunity for quality training, especially for students from low- income families, higher participation for women in technical training, and better quality graduates were to be the outcomes from achievement of the objectives of the project. There first two project objectives were realistic and achievable and were a direct outcome of the project's physical investments which developed new curricula and equipped project schools. However, the third objective, to achieve institutional development proved to be unrealistic mainly due to: (i) the lack of qualified counterpart staff at the central level to forge partnerships with consultants and foreign institutions; (ii) the low status and inadequate pedagogical and technical knowledge of teachers and school managers; and (iii) the very limited technical assistance resources provided under the project in proportion to the magnitude of the needs in the MONE for efficient management and for effective formulation of sector policies. 2 B. Achievement of Objectives 3. Based on the standards prevailing at the time of loan approval, the project did achieve its first two objectives to increase the quantity and the quality of programs in Industrial Schools. However, as explained in "(b)" below, standards have changed during the course of implementation, and the institutional capacity to address such change was not developed to the degree needed. (a) "Improve the Quality and Quantity of Trained Industrial Manpower" The project did deliver technology equipment, books, and teacher training for the appraised program specializations, and resulted in an increase in the number of trainees, far exceeding the appraised targets (4,728 graduates in 1994, or 32% above appraised estimates). However, linkages between equipment and curricula were weak. The project was successful in greatly improving learning conditions in project schools which previously lacked any practical skills training and knowledge of technology. In particular, the project was highly successful in introducing a computer literacy course in secondary. technical education and provided computers for over 250 schools nationwide. (b) "Improve Curricula for new Trade Specializations" The project did complete development of new and revised curricula for 13 trades, or two more than appraised. However, there were long delays in approval of the curricula which did not dovetail with the equipment- based training for teachers nor the appropriate pedagogy for better student learning. More importantly, labor markets today require flexible mobile workers not with specialized trade skills but rather with broader knowledge including computer literacy and stronger math, science, communications, and general work skills (team, analytical, and problem solving skills) (c) "Improve the Planning and Management Capacity of DTVE" The project provided limited technical assistance to the Directorate of Technical and Vocational Education for Boys (DTVE) mainly for project implementation, and limited use of experts to conduct five studies (3-person-months per study). Instead of the five small studies, the project financed during the project extension, a study to improve awareness of global issues and trends in vocational education and its relation to the "world of work". One must note that the limited technical assistance envisaged under the Project did not relate to the appraised development objective to improve planning and management. Currently, the planning and management capacity of the DTVE and MONE in general is constrained by several factors, mainly a rigid and obsolete vocational education system, and resistance to innovation and change to meet the requirements of today's globally competitive labor force and adapt to a changing work place that needs more flexible workers and less narrowly specialized labor. Today, the DTVE pursues the same narrow specializations it planned over 10 years ago when this Project was conceived. An ability to plan for evolving needs and manage a quality education system was not reached. C. Major Factors Affecting Project Implementation 4. The project was successful in achieving its appraised objectives, in particular, delivering goods and services as envisaged in the project, which generated significant benefits given poor pre-project 3 conditions in schools with unmet basic needs in technical and vocational education programs; however, the project did not achieve all its expected quality improvements, and the benefits derived were short of their potential to deliver better training programs. There were several factors discussed below which affected the implementation and quality performance of the project 5. Project Management Project management was repeatedly highlighted as one of the main constraints in project implementation throughout the project years. The Bank and the MONE seemed to differ on their understanding of effective project management. MONE preferred to staff the management unit with MONE teachers who generally did not have the needed language proficiency nor the technical knowledge to work effectively with consultants and the Bank. MONE was also hesitant to entrust outsiders, fearing external criticism and trying to protect internal staff. The Bank insisted, to no avail, on the need for professional management, neutral from political influences, technically competent, in touch with the local school level, and proficient in English and/or other foreign languages. The main lesson learned is that it is difficult to implement a project that is national in scope, through a highly centralized agency with limited ability to maintain contact with the stakeholders, and suffering from significant staffing constraints as mentioned. The central control of physical, financial, and human resources promoted dependency and lack of initiative or autonomy at the local level which did not help implementation. 6. Performance of Consultants The MONE (the client) appeared to be dissatisfied with the performance of two of the consulting firms under the project, the ILO and the British Council, as noted in their evaluation report. Although their report does not delve into the reasons for their dissatisfaction, one of these reasons may have been the lack of detail in the terms of reference for the assignments which prevented a clear understanding of the scope of the consultants' work, their deliverables, and the linkages with MONE counterpart staff in the context of their responsibilities. In addition, weakness in project management by the client rendered the work of the consultants less efficient and less task oriented. Further, a well recognized factor that causes friction between MONE staff and consultants is the resentment vis-a-vis higher salaries earned by consultants. However, it is understandable that consultants would be resented when there is little understanding by MONE staff of the outputs that should come forth from individual consultancies or of the need for consultants to have qualified counterpart staff. These are serious impediments that continue to affect the provision of technical assistance in MONE, but they are surmountable with proper planning and goodwill on both sides. Consultants did provide the expertise they were hired for, mainly curriculum development, overseas training, and procurement services, although there were inefficiencies in the way the consultancies were managed without appropriately qualified counterparts, and in processing the resulting products. The consultants hired during the project extension, the Center for Occupational Research and Development (CORD), were recognized by the client as having performed to satisfaction, and substantive benefits were derived from their work which was closely intertwined with contributions by national staff. Several lessons have been learned regarding the effective use of technical assistance and are already being applied in other projects, mainly: (i) objectives and outputs of each consultancy must be clearly detailed in the terms of reference; (ii) consultancy objectives cannot be disproportionate to the tasks and timeframe of the consultancy, e.g. institutional development objectives are hard to achieve with a limited technical assistance input; (iii) the client needs to have a full understanding and realistic expectations of the objectives, terms of reference, and expected outcome of the consultancy; (iv) large, long-term consultancies should be avoided as the scope of the work and objectives are often not clearly defined without an evident impact or outcome; (v) qualified national counterparts, in 4 particular with language skills, need to be identified as partners in consultancy tasks. 7. Centralization The Government has made decentralization one of the top priorities in its Seventh Five-Year National Development Plan (1996-2000). The high degree of centralization of physical and financial management in the public sector generally, and in the MONE specifically, had a significant negative impact in the project, diminishing the expected benefits derived from the project investments. For example, the project management staff seldom was able to visit schools to assess the use of new curricula and equipment for the project-supported programs and, consequently, was also not able to diagnose and respond to problems and needs with the introduction of sophisticated technologies to teachers and the classroom environment. In addition, little was offered in teacher training to upgrade the pedagogical skills of teachers in new effective and more progressive teaching methods. Lessons were learned regarding the difficulty of implementing projects which are national in scope under a highly centralized but weak system with little autonomy and initiative at the local level. 8. Procurement With centralized procurement the project experienced superb results in the purchase of large numbers of standardized equipment, to be distributed nationally. Workstations for computer literacy classrooms were purchased at a very low cost, and resulted in high benefits to the students and schools. However, this same centralized procurement method has disadvantages in the more technically sophisticated equipment where schools did not have the ability to choose the equipment and the school manager and teachers may not have had the interest or ability to implement a new program. In this case, neither the procurement system nor the project management served the best interest of the stakeholders, since provision was not individually tailored to each school, and resulted in decreased returns on investment when a significant percentage of the equipment is yet to be used effectively. Due to delays, the project resulted being end-loaded, with much of the equipment delivered in the last project year. However, the MONE has maintained follow-up of project schools as agreed for the project's operational phase (see Appendix A); in addition, improved usage of the equipment and further development of the curricula has occurred in recent months and within a year after project completion. This highlights the importance of the project's operational phase to which the MONE has shown its full commitment. 9. Sector Policies Although the project was to provide some support to develop MONE's capacity to adjust its programs according to labor market needs in industry, the programs that are offered today in Industrial Schools are not directly linked to emerging needs in industry. The inefficiencies of the system are evident from the low employment rates of graduates in their fields of study, based on survey information (about 5%). The provincial Vocational Training Councils are also not functioning, and there has been minimal feedback from industry to vocational schools. One of the main reasons for the lack of linkages between vocational schools, industry, and job markets is that Industrial Schools are part of the system of formal secondary education for which the MONE is responsible. In Industrial Schools, students are enrolled in the three-year vocational programs or four to five-year technical programs. The preferred path for technical students remains a university education, while vocational students have little chance to enter higher education or to be employed in their fields of study. Furthermore, since job flexibility is important in today's fluid labor markets, providing narrowly focused skills training is detrimental as students are given the false impression of being skilled workers but in reality and stigmatized for life with poor knowledge in the core areas of math, science, communications, and technology that are being demanded by 5 prospective employers and will be essential for job mobility. A rationalization of the transition from school to work or from school to higher or technical education is yet to be articulated by the MONE and the Council of Higher Education. 10. Government Commitment The Government remained strongly committed to the project throughout implementation in expectation of improvements in the quality of workers available for industry, and considering the high demand for enrollment in these schools. These were positive aspects for the project and contributed to positive relations with the Bank and counterbalanced the project's shortcomings. Industrial Schools are a priority for the MONE and the Government, and the project was viewed positively by the Borrower. 11. Curriculum Development and Board of Education Curricula were developed and upgraded in the various project-supported trades, by consultants and MONE counterpart staff, including teacher commissions. There were some deficiencies in the organization and development of the curricula which caused delays and technical problems, and this was mainly due to consultants working independently from MONE staff. The curricula were not approved until 1992, in the eighth project year. For curricula which should be quickly updated to respond to changes in technology and industry, such a long drawn out process with no sense of urgency seems counterproductive. Since Industrial Schools are secondary schools, all curricula have to have Board of Education approval and meet certain requirements in form and content. The methodology for design, quality control, and approval of curricula remains a controversial issue because of the many stakeholders' interests in participation in the process including, among others, university faculty, school teachers and managers, the religious establishment, industry, and vocational trainers. Because of the multiplicity of actors and the magnitude of needs to maintain updated curricula, textbooks, and teaching materials, the MONE should direct more efforts into improving the efficiency of the process. 12. Textbooks The project successfully provided forty-two urgently needed technical textbooks in Turkish. Again, delays in implementation reduced the potential benefit as technology advances quickly and the validity of such materials is short-lived. Lessons in the need for expediency in adapting and developing textbooks and teaching materials applies equally as it does for curriculum development and the need for continuing education of teachers to keep up with technology and developments. All these are areas that need focus for improvement. 13. Foreign Fellowships The project provided a large number of fellowships (about 805 person-months), mainly in the vocational program areas developed. In all probability, the fellowships component was the least successful and least cost-effective activity under the project, mainly due to the small pool of qualified applicants available in public schools and in management, as well as poor oversight by the MONE of the objectives and expected outputs from fellowships. Unfortunately, even when lessons were known regarding the need for careful targeting and planning of expensive overseas training programs, there was carelessness in the selection of candidates who were: not qualified with sufficient language or technical skills for the intended program; not informed of the objectives or expectations of the programs; and not required to apply in their jobs the knowledge gained in the training. The weaknesses in the design, management, and implementation of the fellowships program had a negative impact on the quality of teaching and the utilization rates of new equipment provided under the project. One of the consequences of this experience has been the restriction on the number of fellowships in subsequent projects to a very 6 limited and selective set of candidates, and much tighter monitoring of program content. 14. Monitoring and Evaluation It was agreed as part of the project that the MONE would monitor certain key school and student performance indicators that would allow for program evaluation. However, in practice, there was little performance monitoring of the project schools. As mentioned under Project Management lessons, the PCU had almost no interaction at the school level, and was unable to perform the appropriate monitoring and evaluation function to allow the project to be responsive to needs at the school level. In retrospect, monitoring closely the proportion and performance of in-service teacher training, and ensuring that the In-service Training Department of the Ministry offered the programs needed under the project would have had a positive impact in increasing the project benefits; at completion, one of the critical weaknesses of the project was the lack of effort made at in-service teacher training, which had been affected by austerity measures. A lesson for future projects is that a financial provision must be made under the project to supplement MONE resources for field monitoring rather than letting this be entirely government-financed. 15. Beneficiary Participation One of the disadvantages of a centrally managed project or agency is the lack of ownership by the beneficiaries of project inputs, i.e., for this project, teachers, managers, and students at the individual school level. Each school is generally an individualistic entity reflecting the spirit of the managers, teachers, and students. There are certain types of regulatory (quality control), research, and policy guidance activities that are appropriately and cost- effectively managed centrally; other activities like program or textbook selection are more appropriately overseen by the stakeholders, as this immediately creates ownership and cost- efficiency. Designing projects with community and beneficiary involvement requires specific planning, making provisions for greater autonomy in design and implementation by the beneficiaries. This is important particularly in the education sector, where communities, teachers, and school managers are better motivated through ownership and accountability. As this project was centrally managed and implemented, both the positive and negative aspects were manifested at the school level -- either schools and students were grateful for receiving goods, materials, and training which would have otherwise been out of their reach, or they were indifferent as to why a piece of equipment or training had been provided to them and took little initiative to use it. As communications and access to goods and services are becoming more universal, projects that are less centrally managed and more community based are possible and should be considered. 16. Austerity Measures The project was affected in some ways by economic austerity measures which limited both national and Loan funding for greatly needed in-service teacher training and English language training. It is recognized that the benefits of the project's investments would be far larger had more emphasis been given to teacher training. Again, the MONE now understands that there are shortcomings in the use of equipment which was the major project investment, and that utilization rates can be improved during the operational phase of the project which could further increase project benefits. 17. Project Cost Changes The project had significant cost savings, mainly due to a decrease in the price of technology equipment from the early 1980's when the project was appraised. The MONE was also generally conservative in its spending and deserves credit for due attention to economy in the purchase of equipment. Although the MONE wished to expand the number of schools which were provided technical training equipment, the Bank agreed only to an increase 7 from 39 to 79 schools; the Bank believed that by 1990, there was already a well established trend moving towards increased general education at the secondary level and cluster vocational programs to allow graduates greater flexibility to change jobs in an increasingly changing work environment. Therefore, the Bank agreed to extend the Loan with the condition that financing would be provided for computer literacy programs for 200 schools to improve the general content of vocational education, and to finance a study to asses the needs for improvements in vocational education in Turkey; in addition, to strengthen the weakest part of the project, teacher training, a component was added to develop 17 model schools for In-service Teacher Training and Program Development Centers. These expanded activities amounted to about US$10.0 Million, or 18% of the total Loan of US$55.63 Million. 18. Implementation Delays The project suffered from long implementation delays, in particular at project start-up, then during project management changes in its first half, and finally with protracted revisions of the new curricula as they were finalized for the approval by the Board of Education. Project delays were not due to any major fault in project design or in unforeseen events but rather to slow implementation and the lack of adherence to schedules and forward planning. The borrower should calculate savings generated from implementing a project on time versus the losses to students who do not receive the benefits (in this case many project inputs to schools came in the tenth year), as well as higher costs over time. Project management was highlighted repeatedly by the Bank as an area to strengthen, noting the need for time management to enforce scheduling and accountability, as well as improvements in management of financial, physical, and staff resources. In the last project year, however, significant improvements were made to speed up project implementation. D. Project Sustainability 19. The MONE has made a serious commitment to continued monitoring in the operation of project-supported schools. Plans for the operational phase of the project, including indicators to be monitored were included in the borrower's completion report (see Appendix A). Consequently, the project is positively assessed as "likely to be sustainable ". However, the impediments that prevent appropriate follow-up of the operation of schools at the local level are still present, in particular, the fact that almost negligible funds are made available for MONE staff to visit schools. Also, because of the centralized nature of the MONE, school visits appear not to be given priority at any level. As follow-up was recognized to have been weak during the project, even up to completion, the PCU established a Follow-up Unit, for which it requested funding for travel and subsistence from other Loans. Furthermore, the Bank has highlighted to MONE management that some of the benefits from the project are yet to materialize as a substantive amount of equipment was yet to be efficiently used due to lack of teacher training. Follow-up will be a key factor to improve the project's impact and its cost-effectiveness. E. Bank Performance 20. Although the borrower appears satisfied with the degree of project preparation, in retrospect, there was need for greater detail, in particular as to project management requirements and scheduling of project activities, terms of reference for consultancies, monitoring at the school level, and most important, involvement of the school in selecting the programs offered and the corresponding equipment. Today, the Bank gives greater priority to the issue of ownership by 8 stakeholders and the need for detailed description of implementation arrangements. Also, in retrospect, given the limited implementation capacity in the MONE, one can note that during preparation the Bank underestimated the difficulties of implementing the project and achieving institutional development objectives and that limited technical assistance provided under any project would rarely allow for achievement of broad institutional development. Regarding the project's responsiveness to the country's development objectives, one can rate the project as "highly satisfactory " as it directly supported the goals of improving industrial productivity and competitiveness in global markets. Regarding supervision, the Bank provided the average supervision amount, with relatively good staff continuity (see table 10). Based on the overall performance of the project, one concludes it has a "satisfactory " rating for preparation and supervision, but considering that the Bank has no responsibility in direct project implementation F. Borrower Performance. 21. The Borrower maintained full commitment to supporting project objectives for the duration of the project. In this sense, the Borrower's performance was, on balance, "satisfactory" since management and teachers generally endorsed the project's objectives and cooperated fully with the Bank during preparation, implementation, and completion. The borrower's shortcomings or deficiencies can be classified mainly as institutional and operational: (a) as mentioned earlier, institutional development was negligibly impacted by the project due to severe constraints at the central level to attract qualified individuals to oversee a policy dialogue and provide intellectual leadership in developing the necessary dynamic secondary vocational education needed for today's changing work environment; the borrower was reluctant to use technical assistance, in particular, national experts from the private sector or universities, while preferring to tap its resources of teachers; however, this proved to be counterproductive since teachers in Turkey generally do not have the necessary competitive business, technical, and language skills needed for this project which was to bring the cutting edge of knowledge in secondary vocational education; and (b) operationally, the MONE managed project implementation by recruiting teachers, mainly from vocational secondary schools, to perform managerial, administrative, and technical work which required fluency in English due to the international nature of the work. Unfortunately, poor management and negligible follow-up impacted negatively on the project. Regarding Loan covenants, the borrower complied with requirements which were less on the technical side and more on maintaining a project management unit and the financial accounting, auditing, and reporting requirements. G. Assessment of Outcome 22. Although the project was extended for two years, and there were shortcomings in the expected quality and flexibility of the programs supported under the project, the project still was implemented in conformity with original project designs which are supported by the MONE, and is considered "satisfactory ". Physical targets were fully achieved or surpassed, albeit quality was sometimes deficient mainly due to poor monitoring and poor teacher qualifications which should have received closer scrutiny during preparation and implementation. In terms of student enrollment, it was anticipated that 3,573 students would graduate annually, but by 1994, there were 4,728 graduates, an increase of 32%. Most important, the project is expected to improve and overcome its deficiencies during its operational phase. 9 23. The institutional development aspects of the project were less tangibly defined in the project, and were also more difficult to achieve given the poor resources available in the Ministry (considering that teachers receive low professional recognition and financial rewards and the sector attracts many not committed to the teaching profession). One element in the project not supported by MONE were five studies which were broad and ambitious in scope if they were to make meaningful recommendations (five studies using 20 person-months). Not surprisingly, the borrower was not willing to use technical assistance to undertake these studies (see table 6); instead, for the extension of the project, a large new study to assess vocational education needs was agreed and successfully completed. H. Future Operations 24. The MONE has prepared a plan for the operational phase of the project, aimed mainly at improving the equipment utilization rate in project schools, as well as the knowledge of teachers in the use of the equipment and to improve the availability of teaching materials and student workbooks. Appendix A of this report includes MONE's proposal to monitor the operational phase of the project (paras. 241-264). The Bank will also continue to monitor improvements in the performance of project schools by including site visits while visiting schools for other projects, and while maintaining a dialogue with the MONE and the PCU. After project completion, MONE continued to submit progress reports regarding the monitoring and teacher training activities during the project's operational phase. I. Key Lessons Learned 25. There are many lessons learned from the implementation of this project, some of which have already been applied in this project, or are being applied in newer projects under implementation or preparation. Project Management Professionals Although the MONE objected to using project management consultants, evidence of the benefits of contracting the needed expertise has been clearly demonstrated. In the Bank's view, the financial losses from poor project implementation in terms of delays and inefficiencies far outweigh the cost of contracting the needed management and administrative services to deliver inputs on time to beneficiaries. Implementation of Technical Assistance (TA) It is essential that a detailed implementation plan for technical assistance be completed during project preparation and include the identification of qualified national counterparts for consultants. Terms of reference should be detailed and understood by both the client and the contractor. Curriculum Reform Curriculum reform is a difficult task, especially in vocational education. To introduce new curricula, appropriate materials, and essential equipment at a national scale is a challenge. It is also difficult to include local industry in a centrally managed project, although industry does not always appropriately influence training as the focus is on narrow specializations for the short term; however, in the medium and long term both workers and industry benefit from broader and flexible skills to be able to adapt to future markets and new technologies. Lack of detail in design of an implementation plan can be detrimental, in particular, definition of the role of local schools in program development and changes in 10 curricula and pedagogy. The importance of these factors is often underestimated at appraisal and results in unrealistic expectations of the project's impact and benefits. In-Service-Teacher Training and Staff Training The importance of in-service training is often underestimated with inadequate provision in terms of numbers of participants and the types of training needed to derive full benefit of project inputs such as new curricula and materials. During project preparation the detailed arrangements for the implementation of an in-service training program need to be carefully designed, and financing should preferably be made available through the Loan. Fellowship Training Objectives Fellowship training programs are costly and their objectives need to be carefully appraised. This includes an assessment of the availability of qualified candidates and their respective suitability for positions they would occupy upon their return. The experience in this and other projects in Turkey indicate a retention rate of about 30%. This low retention rate is further affected by lack of follow-up of candidates to ensure that they apply their newly-acquired skills in targeted key positions. Adjustment in Project Design Because of the fast pace of development in science and technology, and the changing demands in the work environment, it is essential in the field of vocational education to offer up-to-date and flexible programs. Unfortunately, the project remained as designed, focusing on teaching old technologies rather than emerging technologies, experimentation, and the need to become life-long learners. The need for job-specific skills is being replaced by the need to apply academic skills in the workplace. Curriculum reforms now support an integrated coursework program combining academics with vocational learning. This stronger academic foundation is essential for future learning and development of life-long learning skills. It is important that projects and loan agreements incorporate tools for corrections in project design and content and encourage a periodic reassessment of needs. Procurement Procedures In the case of technical and vocational education, where the needs and interests of a community are of essence, it is not desirable to impose at the individual school level standardized programs and equipment which have been centrally selected. The interest, need, and motivation of each local school are essential to the success of a vocational or technical program. An assessment of both the benefits and risks of centralized or locally- planned participatory procurement is essential. Sometimes losses from lack of ownership override any possible cost savings derived from economies of scale through bulk procurement. Table 1: Summary of Assessments A. Achievement of objectives Substantial Partial Negligible Not applicable Macro policies X Sector policies X Financial objectives X Institutional development X Physical objectives X Poverty reduction X Gender issues X Other social objectives X Environmental objectives X Public sector management X Private sector development X Other (specify) B. Proiect sustainability LtiUn]ely Uncertain x C. Bank performance* Highly satisfactory Satisfactory Deficient Identification x Preparation assistance x Appraisal x Supervision X [*Note: It would be preferable if "Bank Performance" was evaluated by an independent outsider.] D. Borrower performance Highly satisfactory Satisfactory Deficient Preparation X Implementation X Covenant compliance X Operation X E. Assessment of outcome Highly satisfactory Satisfactory Deficient x Table 2: Related Bank Loans Loan/Credit title Purpose Year of approval Status Preceding operations 1. Education Project Development of secondary technical 1971 Completed 1981 (Ln.748-TU) and vocational education 2. Industrial Training I Development of post-secondary, two- 1984 Completed 1990 (Ln. 2399-TU) year technician training programs. Subsequent Operations 1. Non-Formal Voc. Training Provision of vocational training for 1985 Active (Ln.2776-TU) youths and adults. 2. Industrial Training II Expansion of post-secondary, two-year 1988 Active (Ln.2922-TU) technician training programs. 3. National Education To focus on quality enhancement and 1990 Active Development Project management improvement aspects of (Ln.3192-TU) the Government's broad sectoral development programs through improved student achievement, educational efficiency, teacher education, and organizational management. Table 3: Project Timetable Steps in project cycle Date planned Actual Identification (Initial Project Brief) August 1984 August 14, 1984 Preparation September 1984 September, 1984 Appraisal December 1984 November 26, 1984 Negotiations April 1985 April 8, 1985 Board Presentation May 9, 1985 May 9, 1985 Signing May 29, 1985 May 29, 1985 Effectiveness August 27, 1985 July 19, 1985 Project Completion December 31, 1992 December 31, 1994 Loan Closing December 31, 1992 December 31, 1994 Table 4: Loan Disbursements: Cumulative and Actual (US$ thousands) FY86 FY87 FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 Appraisal estimate 2.5 4.0 8.2 15.2 26.2 40.2 52.7 57.7 Actual 2.03 2.03 4.23 19.97 21.54 28.65 28.65 36.36 42.61 55.72 Actual as % of estimate 81.2 51 52 131 82 71 54 63 74 96 Date of Final Disbursement: April 25, 1995 Undisbursed Balance Cancelled: US$ 1.98 Million Table 5: Key Indicators for Project Implementation Key Implementation Indicators in SAR Unit Estimated Revised Actual Schools Equipped Development of Program Specializations programs 11 13 14 Schools schools 48 79 74 Computer Training Classrooms schools 200 200 200 Technical Assistance: Curriculum Development months 86 67 62.61 Translation of Textbooks months 6 6 3.26 Project Management & Procurement months 27 27 19.61 Feasibility Studies months 20 7.5 3.90 VTET Study & 17 PRITT Schools (Project Extension) months 84 52 41.00 Teacher Training in Turkey (Project Extension) months 100 37.5 128.91 Fellowships: Curriculum Development months 686 714 803 Textbook Design months 6 4 3 Project Management months 24 65 2.8 Study Tours (Extension) months 100 10.5 20.3 Translation & Printing of Technical Textbooks books 50 Copyrights 42 obtained for 42 titles Student Enrollment in Project Schools by Program Specialization: Electronics Electrical students 5360 8,498 Pneumatics & Hydraulics students 6480 9,644 Foundry students 600 149 Machine Tools & CNCM students 1200 1007 Telecommunications students 1200 281 Dyeing & Textiles students 240 544 Printing students 120 909 Control & Instruments students 240 434 Marine Electronics students 120 73 Computer Science students 120 199 Medical Electronics students 480 823 Tool & Die students - 119 students 1,036 Table 6: Studies Included in the Project Purpose as defined at Status and Study appraisal or as redefined Impact of Study 1. Computer aided learning To benefit from work completed by Used USA-TDP computer applications specialists and funding to complete focus on improving education. this work. 2. New program development To analyze industrial requirements for MONE used its own in the Industrial Schools preservice industrial training, and resources, although evaluate the capacity of current and limited. planned programs within the industrial schools to meet these needs. 3. Development of training programs for To review the existing Cancelled. Some Commerce and Tourism Schools. and planned training programs to meet work financed under the requirements of the modern Ln.2776-TU. commercial, public sector, hotels, and tourism sector. 4. Non-formal adult training To assess community Cancelled. Adult needs for non-formal adult Education is part of education across a representative the Non-Formal cross section of Turkish regions and Department, and evaluate the effectiveness of present studies were done program in meeting these needs. independently by the MONE. 5. Cost control and recovery To examine ways and means in which Cancelled. MONE the MONE can: (a) control and reduce did not wish to recurrent costs; (b) raise additional undertake this study financial resources to support the using Loan funds nor industrial school system. technical assistance. 6. METE - Study on Voc. and Tech. (This study was not included at A comprehensive Education in Turkey appraisal.) To assess and make study and an recommendations for improving the International effectiveness and flexibility of Vocational Symposium were and Technical Education and Training in successfully Turkey. completed. Table 7A: Project Costs Appraisal Estimate Actual (US$M) (US$M) Item Local Foreign Total Local Foreign Total costs costs costs* costs 1. Refurbishment & Furniture 1.8 0.0 1.8 1.8 0.0 1.8 2. Equipment, Materials, 8.9 50.0 58.9 8.9 49.8 58.7 Books 3. Technical Assistance. Consultants 0.2 1.1 1.3 0.2 3.7 3.9 Fellowships 0.0 2.0 2.0 0.0 2.2 2.2 4. Future project development 0.0 0.3 0.3 0.0 0.0 0.0 5. Incremental recurrent costs 3.0 0.0 3.0 2.3 0.0 2.3 6. Unallocated 1.1 4.3 5.4 0.0 .025 .025 TOTAL 15.0 57.7 72.7 13.2 55.72 68.92 *MONE Estimates: March 15, 1995 Table 7B: Project Financing Appraisal estimate (US$M) Actuallestimate (US$M) Local Foreign Local Foreign Source costs costs Total costs costs Total IBRD -- 57.70 57.70 -- 55.72 55.72 Govt 15.00 -- 15.00 13.20 13.20 TOTAL 15.00 57.70 72.70 13.20 55.72 68.92 Table 8: Status of Legal Covenants Agreements Section Covenant Status Original Actual Description of covenant Comments Class fulfillment Fulfillment date Date Loan 2536- Section 1 C No Date Open and maintain in dollars a Always in compliance TU 2.02 (b) special account in its Central Bank on terms and conditions satisfactory to the Bank. Section 10 C No Date The Borrower shall at all times The Borrower always 3.01 (b) maintain a Project maintained project Implementation Unit (PIU) management staff and acceptable to the Bank. The PIU was committed to was renamed PCU (Project implementation; Coordination Unit) in May 1990 however, the Bank when the National Education found the PCU was Dev. Project (Ln.3192-TU) weak in managerial, became effective. technical, administrative, communication, and/or foreign language skills. Section 10 CD 03/31/86 08/06/86 Employ Consultants and make Consultants (British 3.03 (British arrangements for fellowship Council, ILO, and Council) training for the Project, in CORD) were contracted 07/13/87 accordance with a program and for all the programmed (ILO) timetable acceptable to the Bank. work; however, there 11/19/93 were inefficiencies in the (CORD) implementation of TA and fellowship training which are outlined in the report. Section 12 CP 09/30/86 Started in In order to ensure that the Although some schools 3.04 1986, but Industrial Schools system operated advisory carried out adequately responds to the needs committees there was no only in a of industry, the Borrower shall, monitoring or follow-up few no later than Sept. 30, 1986: to oversee their schools out (a) establish schools advisory operation due to lack of of own committees at all industrial resources & low initiative schools included in the Project; priority. Trade (b) form trade advisory committees also operated committees for each of the new in some provinces, at the trade specializations included in provincial level. As the Project. Each such expected, the success of committee shall include such committees depends representatives from industry in on the initiative of the its membership and shall meet at school director. regular intervals and in any event at least once a year. Section 12 CP 12/31/89 08/1994 For purposes of identifying the Follow-up of graduates 3.05 longer-term development was conducted using requirements for industrial surveys done through schools, the Borrower shall carry METARGEM. A out a study under terms of methodology was reference acceptable to the Bank developed for follow-up to collect information on the of graduates, and employment of graduates from METARGEM will use a four industrial schools and shall, more systematic no later than Dec. 31, 1989, approach to improve furnish to the Bank the findings what school directors do and conclusions of said study. independently. Agreements Section. Covenant Status Original Actual Description of covenants Comments Class fulfillment fulfillment date date Section 10 C Not Dated Project schools are to be Adequate physical and 3.06 equipped and refurbished and staff resources were are to be operated in accordance provided for the schools, with procedures as to promote and all Project schools the objectives of the Project, and are functioning well and provide resources and qualified are fully- or over- instructors and administrators for enrolled. However, effective utilization and operation poor knowledge of of such schools. instructors and the lack of in-service training and teaching materials decreased the utilization of equipment and the quality of instruction. Section 1 C No Date The Borrower shall maintain Always in compliance, 4.01 (a) records and accounts adequate to although there were reflect in accordance with sound inefficiencies in record accounting practices the keeping and reluctance operations, resources, and to use professional expenditures in respect of the consultants and to Project of the departments or improving skills to agencies responsible for carrying automate records. out the Project. Section 1 C Annual The Borrower shall have project Always in compliance, 4.01 (b) accounts and the Special account although the quality of for each fiscal year audited, in audit reports was poor accordance with appropriate due to the limited scope auditing principles consistently of the audits. applied by independent auditors acceptable to the Bank, and furnish a certified copy of the report no later than six months after the end of each such year. Covenant Class: .Accounts/Audit 2. Financial Performance/generate revenues for beneficiaries 3. Flow and utilization of Project Funds 4. Counterpart Funding 5. Management aspects of the Project or of its executing agency 6. Environmental covenants 7. Involuntary Resettlement 8. Indigenous People 9. Monitoring, review, and reporting 10. Implementation 11. Sectoral or cross-sectoral budgetary or other resource allocation 12. Sectoral or cross-sectoral regulatory/institutional action 13. Other Status: C: Complied with CD: Compliance after Delay NC: Not Complied with SOON: Compliance Expected in Reasonably Short Time CP: Complied with Partially Table 9: Bank Resources:Staff Inputs Stage of project Planned Revised Actual cycle Staffweeks Staffweeks Staffweeks Through Appraisal No data available No data available 46.7 Appraisal-Board No data available No data available 2.7 Supervision 80.9 127.3 133.9 Completion 12.0 12.0 14.2 TOTAL 197.5 Table 10: Bank Resources: Missions Performance rating Days Stage of project Month/ No. of in Specialized staff Types of cycle year persons field skills represented Implementation Development problems Status objectives Through July- 5 16 EDC, ECN, ARC appraisal Dec. 84 Appraisal Jan. - 5 - EDC, ECN, ARC through Board May 85 approval Supervision Oct. 85 3 2 TM, ECN, ARC. 1 1 Apr.86 2 2 EDC,ECN 1 1 Nov. 86 2 3 TM, PRC 1 1 Feb, 87 3 5 TM,EDC,ARC 1 1 Jan. 88 2 5 EDC, TM 1 1 Jul. 88 1 5 EDC 1 1 Spt. 88 1 5 EMT I I Dec. 88 2 6 TM I 1 June 89 2 6 TM, EMT 1 2 Oct. 89 2 6 TM, EMT 2 2 May 90 3 8 TM,EDC,ARC 2 2 Nov.90 1 4 EDC 3 2 May 91 2 3 EDC, TM 2 2 Feb. 92 2 5 TM, EDC 2 2 Oct. 92 3 4 TM, EDC 2 2 Feb. 93 3 5 TM,EDC,OPR 2 2 Jun. 93 3 6 TM, OPR, EDC 2 2 May 94 3 8 TM, EDC, COM 2 2 Nov. 94 3 6 TM,COM,EDC 2 2 Completion Feb. 95 2 10 TM, EDC, HR 2 2 EDC = Technical Educator COM = Computer Technology Specialist ARC = Architect HR = Human Resources Specialist ECN = Economist EMT = Employment & Training Specialist TM = Task Manager PRC = Procurement Specialist File: m:\patricia\icr2536\tables APPENDIX A BORROWER'S EVALUATION REPORT: CONCLUSIONS AND RECOMMENDATIONS I1. EVALUATION OF PERFORMANCE A. KEY LESSONS LEARNED 178. Industrial Schools Project is the first comprehensive project that the Ministry of National Education has applied. The Ministry personnel who will be responsible for the implementation and itemizing the management and implementation planning of such comprehensive project are to be determined previously. Additionally; this personnel should be provided training on project management, implementation and evaluation prior to their designation for the project-related activities. The continuity of the project activities should be provided and frequent changes of personnel should be avoided. 179. It has been revealed that, with regard to the project planning, itemizing the responsibility of the project-related units has a remarkable importance taking into consideration the fulfilment of the responsibilities in direction of the project objectives. Additionally, in order to benefit from " Technical Assistance" services planned for efficient implementation, terms of reference should be prepared also for local counterparts. For the employment of the specialists, these terms of reference should be considered. Accordingly, the studies of local and foreign specialists should be evaluated coincidentally. Periodical informative meetings shall be held in favor of the establishment of interdepartmental group working in project planning and embodiment of the project objectives by relevant units. 180. The consultant firms and local/foreign consultants designated to the Ministry units should be carefully supervised and controlled so that maximum educational benefit will be received. Monthly reports of these consultants should be reviewed by the relevant department and the deficiencies be rectified. Furthermore, a payment structure should be envisaged in which the specialists will be paid according to their work and work quality instead of their assignment duration. 181. Regarding the realization of the activities determined in project implementation plan, the last optional date needs to be stated in time schedule. In case the activity cannot be implemented within the determined period, a possible sanction including a review of the activity should be applied. Project Implementation Plan should be classified according to the priorities. In view of the economical status of the project, it is important to make a planning which prevents the implementation of one activity without completing the previous one. 182. On-the-site determination of the needs of the project-related units by the officials of these units has a remarkable significance, in view of the actuality of the needs and efficient use of the equipment. This will also show the units the responsibilities regarding the completion of necessary infrastructure preparations. After determining the actual needs, technical specifications of the equipment to be procured should be prepared by specialists who have been trained abroad and who have acquired highly technical qualifications and excellent language skills. Similarly, procurement and follow-up procedures should be handled by the same committees. 183. The candidates for fellowship training should be selected based upon their language levels and be provided pre-training in local universities in their respective areas. Accordingly, the successful ones should be sent for fellowship programs abroad. In order to receive maximum benefit from the training, particular attention should be paid to the quality of the training environment and trainees. Similarly, certain criteria (quality of teachers, equipment, social conditions of the region, etc.) should be determined and followed for the schools abroad where training is held. 184. Particular attention should be paid to teacher needs and in order to improve quality in education, assignments should be determined according to these needs. Fellows returning from their training abroad under the project should be assigned to the project schools. Assignment procedures should be handled with the coordination of the respective General Directorates prior to fellows' leaving for training abroad. Furthermore, periodical in-service training for administrators and teachers in the project schools should be included in the initial project planning. 185. The value of the equipment procured within the scope of the Industrial Schools Project (Phases 1, 11, 111 and IV) is 50.3 million US Dollars. For the effective use of these equipment of such a big investment, regular maintenance and repair process should be ensured. To this purpose, particular attention should be paid to technical personnel training. Also, operational costs of the production equipment, technical personnel training and employment should be planned in the project preparation. The education programs should be reviewed periodically, with the participation of the industrial organizations, against the changing technology. B. FUTURE APPLICATION OF LESSONS LEARNED 186. With the Industrial Schools Project, 274 Secondary Vocational Schools in Turkey have been equipped. In order to ensure equal opportunity for everybody, this type of physical improvement should be expanded to include other secondary vocational institutions. However, full benefits cannot yet reflect on secondary vocational education as the existing physical capacity is not fully utilized and graduates are not properly monitored. Therefore, actual cost of the vocational education in secondary level is increasing more arid more. For future investments in this area, financial aspect of the vocational education should be considered as follows: * It would be more realistic to establish laboratories for technology training instead of procuring equipment for production workshops. * For the financing of vocational training in specific areas, a system should be established to which organizations benefiting from this training can contribute. Increasing the variety of financial sources would have positive effect on education quality and continuity. 187. In regard to the International Symposium held in June 1994, to analyze vocational education in Turkey and in the world in the year 2000's, most of the reports, as well as the final report, stress the fact that production and distribution methods for goods and services are rapidly changing in parallel to the rapid developments in technology and globalization, while approaching to the year 2000's. The companies that can keep up with these changes have a say in international markets. Having realized that hierarchical organizations are slow in meeting these changes, these types of companies are transferring responsibilities from middle-level personnel to front-line personnel. This has resulted in drastic changes in the skills and qualifications that are now demanded. With the introduction of high technology, organizations will soon be in high competition to survive in international markets. Thus in order to meet their needs, a well trained work force should be produced with high performance, technological understanding and application ability, effective communication and problem-solving skills, team work spirit and initiative. Producing such a high quality work force will not be possible with classic vocational education methods which mainly focus on handicraft. To have a successful economic future in Turkey, vocational education system should be restructured to develop a broad-based vocational education at secondary education level and to produce high performance work force. 188. The personnel policy has a crucial role in adapting the above- mentioned changes in the structure. To ensure continuity in the employment of well trained personnel within the Ministry and in schools is the most effective way to benefit from the investments made in this field. Thus; it is important to develop a personnel promotion and social & financial benefits system which focuses on performance and productivity in vocational and technical education. 189. To achieve the desired quality in vocational and technical education, it would also be beneficial to restructure the system as to give financing and program design responsibility to the organizations employing the well trained work force. C. COMPLIANCE WITH LOAN COVENANTS 190. In accordance with Industrial Schools Project Loan Agreement dated 29 May 1985, the Government and Ministry of National Education made the following commitments to the World Bank: 191. Section 2.02. (b) of the Loan Agreement reads as follows: (b) "The Borrower Shall, for the purposes of the project, open and maintain in dollars a special account in its Central Bank on terms and conditions satisfactory. Deposits into and payments out of, the special account shall be made in accordance with the provisions of Schedule 5 to this agreement" Based on this subject, a special account was opened in Central Bank and commitments were executed by obeying to payments according to categories.. 192. Section 2.03 of the Loan Agreement reads as follows: "The closing date shall be December 31, 1992 or such later date as the Bank establish. The Bank shall promptly notify the Borrower of such later date". Due to the fact that the project activities could not be not completed by 31 December 1992, first extension was made until 31 December 1993. Upon the second extension, the project was closed on 31 December 1994. 193. Section 3.01 (b) of the Loan Agreement reads as follows: "In order to ensure coordination amongst the directorates or agencies involved in the Project and effective monitoring of progress under the project, the Borrower shall at all times maintain a Project Implementation Unit with staffing and terms of reference acceptable to the Bank". Above-mentioned commitment was executed by the Government. However, pointed out PCU carried out their activities according to the job definitions only after 1992. Accordingly, following activities were planned and commited in Annex Schedule-2: (a) Equipment and Renovation of the Industrial Schools (b) Inservice Training Programs on Selected Fields and Program Development (c) Translating and Publication of the Technical Textbooks (d) Other Activities. Commitments related to sections (a), (b), (c) and (d) were executed except timing. 194. Section 3.03 of Loan Agreement reads as follows " or such later date as the Bank Shall agree, in accordance with Section II of schedule to this agreement, employ consultants and make arrangements for fellowship training for the project, all to be provided in accordance with a program and timetable acceptable to the Bank." Activities in relation to this section had been performed in considerable delay. Consultancy Services were procured by contracting British Council (2 extensions ), ILO and CORD. 195. Section 3.04 (a-b) of Loan Agrement reads as follows: "In order to ensure that the Industrial school system adequately responds to the needs of industry, the Borrower shall, not later than September 30, 1986 (a) establish schools advisory committees at all Industrial Schools included in the project and (b) form trade advisory committees for each of the newtrade specialisation's included in the project. Each such committee shall include representatives form industry in its membership and shall meet at regular intervals and in any event ant least once a year." With the law no. 3308; " Apprenticeship and Vocational Training Commissions" were formed at central and local level. However, it could not be said that these commission conducted their works effectively. 196. Section 3.05 of Loan Agreement reads as follows: "For purposes of identifying the long-term development requirements for industrial schools, the Borrower shall carry out a study under terms of reference acceptable to the Bank to collect information on the employment of graduates from four industrial schools and stall, not later than December 31, 1989 furnish to the Bank the findings and conclusions of said study." Some project schools have got employment information but activities are not conducted within a system. Schools which could determine employment information presented their findings in Annex-20. 197. Section 3.06 of Loan Agreement reads as follows: "The Borrower shall cause the schools to be equipped and refurbished under the Project to be operated in accordance with such procedures as shall promote the objectives of the Project and shall provide or cause to be provided, promptly as needed, the funds, facilities, services and other resources as well as qualified instructors and administrators in adequate numbers reguired for the effective utilisation, staffing and operation of such schools". Schools have been equipped within the scope of the project but some of the equipment could not be effectively used. Teachers in adequate numbers are present in such fields as electrical; however some fields such as CNC Machines and Computer do not have enough teachers. 198. Section 4.01 (a.b.c.) of Loan Agreement reads as follows: " (a) The Borrower shall maintain or cause to be maintained records and accounts adequate to reflect in accordance with sound accountting practices the operations, resources and expenditures in respect of the Project of the departments or agencies of the Borrower responsible for carrying out the Project or any part thereof. (b) The Borrower shall (i) have the accounts referred to in paragraph a) of this section and the special account, for each fiscal year audited, in accordance with appropriate auditing principles consistently applied, by in de pentend auditors acceptable tothe Bank; (ii) furnish to the Bank as soon as a valuable, but in any case not later than six months after the and of each such year, a certified copy of the report of such audit by the said auditors, of such scope and in such detail as the Bank shall have reasonably requested; and (iii) furnish to the Bank such other information concerning said accounts and the audit there of and said reports as the Bank Shall from time to time reasonably regrets. (c) For all expenditures with respect to which withdrawals are request from the Loan Account on the basis of statements of expenditure the Borrower Shall; (i) maintain or cause to be maintained, in accordance with paragraph (a) of this section, sparate records and accounts referring such expenditures; (ii) retain, until one year after the closing Date all records ( contract orders, invoices, bills, receipts and other documents ) evidencing such expenditures; (iii) enable the Bank's representatives to examine such records; and (iv) ensure that such separate accounts are included in the annual audit referred to in paragraph (b) of this section and the report thereof contains, in respect of such separate accounts, a separate opinion by the said auditors as to whether the proceeds of the Loan withdrawn in respect of such expenditures have been used for the purpose for which they were provided." 199. Above-mentioned committments were mostly executed. Financial Records had been regularly maintained by PCU until the end of 1989 ( same records are present in GDVTE ). As of 1989, due to the changes in the budget expenditure categories, these records have been maintained along with the local allocations that were made to all schools under GDVTE. Although financial project resources allocated in the government budget for each year were obvious, dispersion of the allocations based on schools could not be determined within this period. Therefore, financial records could not be regularly maintained. D. EVALUATION OF MONE PERFORMANCE 200. Due to the fact that high level managers such as Minister, Undersecretary, Deputy Undersecretary are continuously changing, explaining the project requires time process. This situation created difficulties in implementation of the project and procedures were delayed. 201. Lack of a project observation commission in GDVTE lengthened the duration of the solving problem. Also, due to a temporary lack of coordination between PCU and GDVTE, the problems of the schools could not be solved. 202. Industrial Schools Project is the first project that was implemented by the Projects Coordinaton Unit which was known as Project Implementation Unit at the beginning. Therefore, there have been several delays in activities time to time in view of insufficient experience. 203. Due to the previous donation-based project implemented by the Ministry, especially at the end of first phase procurement, it was observed that equipment delivered to the schools was accepted as donation by the several schools. Moreover, some school managers were not well-informed about the project scope. 204. PCU have continuously assigned their personnel on temporary base and the staff were not given a determined responsibility. However, the Unit established an organisation within their body and performed terms of reference after 1992. 205. An annual working calendar must be prepared in accordance with the project objectives and the project plan; and this calendar must be carried out in due time, giving priority to legal arrangements which will reduce the unnecessary bureaucracy. 206. Project applications and outputs ( without project practices ) must be periodically and realistically evaluated at the end of each year. Project Closing Report must be prepared based on these evaluations. 207. It was observed that Project Implementation Unit did not efficiently follow-up, nor realise selection of the suitable foreign centers where education would be performed. 208. It was observed that, instructors charged by GDVTE on several equipment purchases could not select suitable equipment due to lack of vocational and technological knowledge. 209. Fellowship trainings did not provide maximum benefit for the fellows as they did not receive an orientation training and were sent to the foreign countries without a proper orientation and program schedule. 210. It was observed that unspecialized persons were also participated in overseas training provided by the firms ( Dyne & OMC ). Furthermore, the fields of the fellows did not respond to the training fields. Thus, these trainings remained only as an excursion, experience and research activity. E. EVALUATION OF BANK PERFORMANCE 211. According to the meetings made with competent persons who are in the project preparation phase, everything has been realised based on rules. The Bank has paid great attention to the activities related to preparation phase which was conducted carefully. Working papers confirms this thesis. 212. It was observed that the Bank could not advise and guide effectively as far as proper implementation concerned. Thus, a careful monitoring is necessary. 213. Frequent additions and changes led to several delays in the project. Attention must be paid to such situations in implementation phase. 214. Strong corporation with the project implementation unit must be realised in the project preparation phase. While school visits have been made, one person also from the project implementation unit must definitely participate in these visits. Also, the importance of the SAR must be clarified in the Loan Agreement phase. Under the light of these facts; the Bank executed almost all commitments. F. ASSESSMENT OF OUTCOME 215. Within the Industrial School Project; technical teachers were trained both at home and abroad, and they reviewed and applied technological novelties. School managers and Ministry staff went to the foreign countries, reviewed knowledge and experiments related to education and transferred their findings to the their associations. Moreover, the lack of equipment in the selected vocational areas of the project schools was responded. Thus, this project has contributed to the Turkish economy by providing trained manpower to the work force. The project, however, has not completely achieved their objectives in this respect. 216. If the following subjects are performed, project will achieve their objectives: Equipment provided from the project will be appraised one by one; inactive ones will be made into usable state and maintenance of the other active ones will breezed and make into more active state. Programs that were provided from the project, inclusive of the Saturday and evening education, will keep pace technological novelties and moreover, teacher training will be completed with the inservice training. Ill. ADOPTION OF PROJECT OPERATIONAL PHASE A. DESCRIPTION OF OPERATIONAL PHASE 217. Within the scope of the Industrial School Project, a file system was started within GDVTE but due to the fact that project implementation unit has been separated from their body, weaknesses have taken form in said system. Project was examined in every side by the General Directorate and related assessment documents were sent to the project schools. Responses are being evaluated as well as technical deficiencies. Monetary transitions and arrangements will be planned in accordance with the said evaluation results. 218. Research and Development Unit has initiated their activities related to the associational regulation under the General Directorate. This situation was explained in NUMBER TWO OBJECTIVE - Curriculum Development. Also, Projects Section under the General Directorate will search solutions for the problems by working with PCU. B. PERFORMANCE INDICATORS FOR MONITORING OPERATION AND DEVELOPMENT IMPACT Following situations which were also mentioned in several sections of this report prevented to be attained required results: 219. Personnel assigned in project management were not received a detailed training program about the project management subject. 220. Due to the lack of similar project implementation previously, experience was insufficient. 221. Changes on the project management 222. Teachers who were selected to be charged in project schools, were assigned to non-project schools. 223. The firms presented inadequate training for equipment which were purchased in project scope to the education and training services. 224. The overseas training provided for the teachers was insufficient in usage of the products such as high tech machine and equipment. 225. Short term foreign assignments that were performed within the scope of the project was not appropriate. 226. Technical consultancy services were insufficient. 227. The staff trained in foreign countries and foreign consultants did not work in collaboration in project development process. 228. Knowledge accumulation of the personnel charged in the project management was insufficient, which concerns what the project evaluation criteria are and how they will be actualized, indicate that the project implementation could not be attained. 229. In order to solve the problems related to instructor training, machine and equipment and to reflect project implementations onto the industrial technical education; 230. Evaluation forms that include several subjects such as updating the education programs, educational needs of the teachers, productive usage of the machine and equipment in the project schools were sent to the project schools. Evaluating activities of the attained dates by the commissions are in process. As a result of above-mentioned activities 231. Determination of educational needs of the technical teachers in the project schools on usage of the machine and equipment purchased within the scope of the project and to meet this need, the following matters will be realised within the scope of a plan: 232. Preparation of the in-service training programs; 233. Determination of the place and time of the in-service teacher training; 234. In-service teacher training provision of the faculty members will be ensured from at home and abroad; 235. Effective in-service training activities 236. Proper usage of machine and equipment puchased by MONE; 237. Development and modernization of the education programs; 238. Also, in respect to the cooperation and coordination between PCU and Vocational and Technical Education Research and Development Unit, the following activities were planned to be completed until the end of May 1995. 239. Establishment of a system to observe graduates from the technical education and project schools including project; 240. Operation of the Vocational Guidance Councils with the participation of the technical deputy school manager, instructors of the project fields, representatives from the related industrial firms and vocational foundations, under the school manager within the scope of the project schools. � А � и т л � о г ° иvгroеаэzч. Q \.�а J ' ., � У' � �'1J./ 1ЙQ / � J о \ � � . а .! �,� !'" � _ �VL, / � gOa rцr�. � .i' •� � '\ а tПш� -�'.� � ' д' s � ` ш I ` �.1 �~'.�„{ -•,/ %� Q � � V � г ( � �� � Q � � с � I у i О � �' ° о z � О ` � ш � r�•�. .i.� I �. > 1 � и а� Z� •1 � ' . I J 1.^� 1 1 ���i � � s а ь { . ��ц � 1. � � F о � а � О t i j• � n�а 1 1 i Z а г !J • � ��� г g � s � g � � � wn ' ��� а � z _ 1 r' , � ' t •� г' i- � Р7 ® i - � r %Q � � . I If. .\ .\ � � ���� ��� ; � � � ��, � 1• . ��, \.� У в а _ �, �� ' ` ь �, - � � � _ s� � � � � 1, Q ; °1 Е, � �F � f� ц� �� fi �� �. � " � 1 I. Q 1 \ 1 ',{ �� °�.�, aS w 01' � � уг � U � � �• • \ � �v � .�� § tr ' г • 1 , - \• � �й о �Е п �Ё � � ` � г � � � � 3 > ? v � х �� � 3 � "в � w � �� о' • t - 1 •. ° ,� � , г � � � � д ' г- � �� ��� �о 1 г? � � / � 1 � Г иГ•J i •� � 1�.J' � i� �� „ во 1 \ r ar . ` • � г� •\ � �� � � __ ..� ` ZW • � � ) - Ё J`-�� � �, .� а т ' - п �) , ° � о • � > � - f� �О� ' 1 И Q • � i . , i У� �� , }! � �� � . � • i� � �, г1 ._ 1.� , . _ � 1 � � - '1 .1._ ��� i I .ij / , 1 1 �� � • 1 �9I у1 ' , _ � _ `` __ "In ` ` , 7! . г� � � � •� � 1 �� ,� oi� g� � ( Ё . ; " � л � Г� � 1с < а г '.� а , � ,'i _"л - �. ,�ч _ • i ' �i ' �":; а в�,J � о д> � � � � - Е � � , � � , � � 1 '� ���' � �, 1 i . V „i ь �� - r� и � � l г� '�• ,_ z�;i ! о 1- i' � � ° 1 �, г , /к' 1' `_ Z _!,, �� ' '(1 Ё � _ i �г�� 1 ` � 1_� '1 �е I `7, I� й 1• . �г `� 'у' 1 у� 11- 1 1 � Гс� " .г., г у -- � ` � . 6 ` S � - ,� � r. � `� . 1 _ � , 1 а � j , ��, v ��ti.,�- �. = - - , о I �i 1 Т � 1 � �iЭ„ • а � -� � ар � � � У i - . � � " � \ � ��' . � Т_ w о � а �i �� гг � , � � �, � � � v `��� � ? � , � , ! • , � ,>\ f � i 1 и + г 1�" 1 � ��� '\ \ ' �� 1 � �• 1 �1 (Т ' � � д , ._ i / 1 �,��\ ° 1 I� �, г'� г� • , ,��:�i h/1 с _$ - т �'� ( / � ��-� 7I! ��i. ' УΡI� � � I - , ' � 1 : 1� I � S � ��и''` I � С I �� L� r � ° i1 _��°S ' /_ ...-f гг� о� `О '•Х3,� .�,}�:�� ��� ('' • :n__ _ � - _nr� иа` ш k � •Ё�� л 1� � �У • , ,•� � ('� � �• . • < , - � У_+ �,�т ll .- ����_,� о b Л � Ri Ф l'.�оИ�+� �I �� Г�r� У �� ��� ii�lm �" �-I в�п- � � {д 1 {о~ 1J: I i '�� �- � � ,�� - r _� � _ v_-� �,г _ �?� 1 * i-� � � � �.т �� � `. Г,j� 1 � •, \ --''}' а li��,' �� l . ��� ``�" !�г� � �г „ �г� � '�. г- � - � ь�[ / � € г . г � ; ` �j -� 3-� .�^ в , _ � _Г� �- ��, !� �� ljT } ° _ W"' �^в4у� � ��[ 1 � ' ��.� г ) а� � з о j� _ � вТ� �i.i�r ��g �`., * •.� . /�����` ` � ��.;�?r.)� ', Я ;�� _ 1 j - ё � �� ' в�� `ж �_ • г"• � i ,а �� l ( ,� \г ��^J а 8$ i S �_. �-- , r�r.n � • 1• �t_s С{ ti � i i� U 8 of п t - - '� Т �_ -г+,�т^� `��'-1 ,�-�^ * ,��. ,�� - ьа \'" ш ��? � �• ; ,I' � \' � 9 �-} ГЕ''�� ,,�; " . .. Ё Ё � 5 r� i. 4 �n/� i� ��� � - Q -"�.• �� 7 -' _ ,'� • t�• г� � � �.'.�.'1 ? � г ' . • , . - о _ - � у " ! \ ' iI г• ; `._ i,/I ^ �г°. � t � . \�� oaS ипзбау l J г� _ ., , , IMAGING Report No: 153$C Type: ICR
Группа Всемирного банка · Implementation Completion and Results Report
Turkey - Industrial Schools Project
Открыть оригинал документа
Полный текст размещён на сайте публикующей организации. lawenc.com индексирует метаданные и ведёт на официальный источник.
Полный текст
Основные сведения
Организация
Группа Всемирного банка
Тип документа
Implementation Completion and Results Report
Страна
Турция
Источник
Всемирный банк