Turkey - Non-Formal Vocational Training Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 16528 IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY NON-FORMAL VOCATIONAL TRAINING PROJECT [LOAN 2776-TUJ APRIL 30, 1997 EC1/2HR Human Resources Sector Operating Division Central and Southern Europe Departments Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVAI ENTS At Appraisal: 100 TL = USS .14984 US$1 = 667.35TI, At Completion: 10,000 TL = US$ .13437 US$1 = 74.422TI, WEIGHTS AND MEASURES Metric System FISCAL YEAR OF BORROWER January I - December 31 ABBREVIATIONS AND ACRONYMS DGTVE - Directorate of Girls' Tclinical and Vocational Education EA3PH - East Asia Population and Human Resources Operations Division ECI - Europe and Central Asia Region, Department I ECl/2HR - Human Resources Division for EC Departments I and 2 ECA - Europe and Central Asia Region EMTPM - Public Sector Management & Information Technology Team GOVT - Republic of Turkey HDDED - Human Development Department, Education IBRD - International Bank for Reconstruction and Development ICR - Implementation Completion Report IDA - International Development Association METARGEM - Vocational Education Research Institute for MONE MOF - Ministry of Finance MONE - Ministry of National Education NFEI - Non-Formal Education Institute NFTI - Non-Formal Training Institute NFVT - Non-Formal Vocational Training PCU - Project Coordination Unit PIU - Project Implementation Unit PIU - Project Implementation Unit RMT - World Bank Resident Mission Turkey SAR - Staff Appraisal Report SERCO - {FullName} SOE - Statement of Expenditures TA - Technical Assistance TOR - Terms of Reference Vice President: Johannes Linn, ECAVP Director: Kenneth G. Lay, ECIDR Division Chief: Ralph W. Harbison, ECI/2HR FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY NON-FORMAL VOCATIONAL TRAINING PROJECT (LOAN 2776-TU) Contents PREFACE EVALUATION SUMMARY PART 1. PERFORMANCE ASSESSMENT FROM IBRD'S PERSPECTIVE A . Statement and Evaluation of Project Objectives .... ........................................... .........1 B . A chievem ent of O bjectives ................................... ........................................... .........2 C. M ajor Factors Affecting Project Im plementation .. .....................................................3 D . P roject C ost C hanges............................................. .....................................................6 E . P roject S ustainab ility ....................................................................................................6 F . B ank P erfo rm ance .................................................. .....................................................7 G . B orrow er Perform ance............................................ .....................................................7 H . A ssessm ent of O utcom e ......................................... .....................................................8 I. F uture O peratio n s ................................................... .....................................................8 J. K ey L essons L earned .............................................. .....................................................8 PART II. STATISTICAL ANNEXES Table 1: Summary of Assessments Table 2: Related Bank Loans Table 3: Project Timetable Table 4: Loan Disbursements: Cumulative Appraised and Actual Table 5: Key Indicators for Project Implementation Table 6: Key Indicators for Project Operation Table 7A: Project Costs Table 7B: Project Financing Table 8: Status of Legal Covenants Table 9- Bank Resou.-Lc.s, Staff It puts Table 10: Bank Resources, M4issior s Axpendix A: Borrower contribution to the ICR This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY NON-FORMAL VOCATIONAL TRAINING PROJECT (LN 2776-TU) Preface This is the Implementation Completion Report (ICR) for the Non Formal Vocational Training Project in Turkey, for which Loan 2776-TU in the amount of US$58.5 Million equivalent was approved on January 6, 1987, and made effective on April 2, 1987. The Loan was closed on December 31, 1995, its original closing date. Final disbursement took place on May 9, 1996, at which time a balance of US$0.5 Million was canceled. Earlier, an amount of US$3.6 Million was canceled since this was not expected to be utilized. A total of US$54.4 Million of the Loan was disbursed, or 93 %. This ICR was prepared by Patricia Kleysteuber, Task Manager (EC1/2HR), with the support of Judith De Costa and Danelle Howell (EC1/2HR). Mr. Friedrich Konigshofer (Sr. Operations Officer, EMTPM), and Mr. Ihsan Sezal, (Education Development Consultant) prepared background reports which are available in ECA Files. The ICR was reviewed by George Zaidan, Projects Adviser, ECI. Peer reviewers whose comments were incorporated in the report were Bruno Laporte, (Principal Education Specialist, HDDED), and Michael Mertaugh, Neeta Sirur. and Brad Herbert (ECl/2HR). This ICR is based on the results of a Completion Mission in April 1996, which visited a representative sample of schools, and on materials available in ECA files. Mr. Konigshofer and Ms. Sahmali (RMT) participated in the Completion mission and provided comments on the draft report. The Borrower (Ministry of National Education) prepared its own evaluation report, and the report's conclusions are included verbatim as Appendix A of this report. The Borrower also contributed to the preparation of this report by providing data and information for the report. IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY NON-FORMAL VOCATIONAL TRAINING PROJECT (LOAN 2536-TU) Summary i. Introduction This project was one of a series of projects in secondary and post secondary education for the development of vocational education in Turkey. This project was preceded by Loan 2536-TU also supporting secondary vocational education (Industrial Schools Project, closed Dec. 1994), and Loan 2399-TU supporting post-secondary technical training (Industrial Training Project closed Dec. 1990). It was followed by two projects appraised in the late 1980's and currently under implementation -- Ln. 2922-TU supporting the further expansion of post-secondary technician training, and Ln. 3192-TU supporting quality improvements in primary and secondary education. ii. Project Objectives. The project had two objectives: (a) to increase and improve the supply of well- trained labor for manufacturing industries, through (i) improved quality of non-formal vocational training programs and enhanced relevance to the labor market needs, and (ii) expansion of enrollment in vocational education and training programs with an increase of about 110,000 trained workers per annum; and (b) to improve employment and income generating opportunities for disadvantaged groups with little education and training, through (i) employment of a significantly higher percentage of graduates of non-formal vocational training institutes, and (ii) creation of about 16,000 self-employment opportunities, mainly for women engaged in income generating activities in the home. iii. Evaluation of Objectives. The project's first objective supported the long-term strategy of supplying better trained manpower for manufacturing industries in order to raise productivity and growth of manufacturing output. The Government gave priority to increasing manufactured exports, which depends on the availability of better-educated and skilled manpower. The proposed project also responded to demands from industrialists for skilled technicians and craftsmen. This translated into the need for an effective formal and non-formal vocational education system to improve the knowledge, skills, and attitudes of an under- educated population. The project's second objective supported the development of increased earning potential and employability for the less advantaged socio-economic groups. This objective responded well to the Bank's poverty alleviation mandate addressing the needs of the disadvantaged -- the unskilled unemployed, adult women trying to enter the labor market, or the long-term unemployed who are also poor and unskilled. The project's objectives were appropriate and supported the Government's development strategy for the sector. iv. Project Outcome. The project is rated overall as satisfactory. It substantially achieved its physical objectives and partly achieved its institutional development objectives at completion. The quality of non- formal vocational education and training and its relevance to labor market needs improved significantly, in particular, compared to the pre-project status of schools. The project supported 607 schools, more than twice the 268 schools expected at appraisal. Furthermore, computer classrooms were provided for an additional 341 schools under a fourth phase using cost savings. Technical assistance services and fellowship training also exceeded appraised targets. The production of new technology textbooks fell slightly short of target but nevertheless made a significant contribution to the acute shortage of adequate textbooks. The final phase of the project included a highly successful integrated pre-school education program in conjunction with adult education for mothers, targeting low-income families; this component fits well with the project objectives as the program focused on building self-confidence in women and their role in supporting their children's education. Finally, in terms of employment of graduates, all students enrolled in Apprenticeship Training Centers are employed, and graduates from the Commerce and Tourism Schools are the most successful in obtaining employment in the growing service sector in Turkey -- and these two groups make up over 50% of the project supported schools; other more narrow vocational specializations for manufacturing trades have had low employment rates in a job related to the student's respective program specialization. Regarding institutional development impact, the Ministry of National Education (MONE) did acquire expertise to respond better to training needs for emerging labor markets, in particular. under the Directorate of Girls' Technical and Vocational Education (DGTVE). The project did not achieve its institutional development objective to specifically strengthen the MONE's Non-Formal Education Institute (NFEI) to be able to update vocational programs; however, the staff training and materials development that took place under this component has benefited a new entity of the MONE now responsible for this work. v. Project Costs. Actual project costs totaled US$72.16 Million. There was a marginal overrun of 1.5% in local costs due to a greater amount of school renovation undertaken by the MONE. Loan expenditures had an underrun of US$4.1 Million, or 7% of the Loan which was canceled. Although the MONE requested informally an extension of the Loan to use the remaining funds, the Bank recommended against an extension due to changes in strategy to strengthen and reform vocational education in Turkey. A reallocation of funds was made under a Loan amendment in July 1994 to increase support for improved school performance and project implementation and to include financing for expansion of the Mother/Child Development Program, a program targeted to low-income families. vi. Bank Performance. The Bank was responsive and sensitive to national development priorities in the design of project objectives, and the project effectively supported the country development strategy. The project inputs were required to modernize the knowledge and skills of the labor force and significantly improve the quality of vocational education and responsiveness to emerging job markets. The project is recognized to have contributed to achieving these goals by project beneficiaries. The project was adequately prepared although it was ambitious in expecting institutional change to take place in a very weak agency and overestimated the implementation capacity of the MONE and the local school level. Although the project suffered from inefficiencies during implementation. the Government and the Bank were cognizant of these problems and repeatedly addressed these shortcomings. As the project was fully implemented and is sustainable, it is considered to have been on balance satisfactory in preparation and implementation. vii. Borrower Performance. The MONE and government maintained their full commitment to project objectives throughout project implementation and remain committed to project sustainability. The MONE implemented the project as appraised which led to the achievement of objectives of the project. The MONE and the Government continue to be committed to improving the employability of their secondary school graduates and to providing job related skills to the unemployed. Where the MONE has remained weak is in responding to the need for swift project implementation in the use of Bank funds, and being more open to contracting services external to the Ministry. On balance, the borrower's performance is rated as satisfactory based on its successful completion of the project and its commitment to project sustainability. Summary of key lessons learned for future operations: viii. There are several lessons learned from the implementation of this project. School Management The importance of effective school management is often overlooked or underestimated. School managers are not selected for their leadership or managerial skills, and tend to stagnate in their positions while not being required to update their managerial and pedagogical knowledge and skills. thus, many school managers have little initiative, they do not provide incentives to teachers, and they also lack outreach skills to interact with the community and local businesses. In the Project, school managers often did not have the ability to select and run appropriate programs as was envisaged. Lessons: Education projects should include substantive training for school managers and encourage periodic assessments and rotation of school managers on a competitive basis. - iii - Adjustment in Project Design. Education projects need to adjust to emerging needs in society, discoveries in science, technology, and pedagogy, and changes in labor markets; they can not be static in design over a long implementation period. Although adjustments were made to the project during implementation, it was difficult to incorporate timely change and encourage innovativeness. partly because of the rigidity of the public sector and partly due to the perceived requirement to adhere to an agreed project on both the side of the Borrower and the Bank. Lesson: The borrower and the Bank need to apply more flexibility in adjusting the project design and simplifying proceduresfor approval of changes consistent with the project's broad objectives and scope. Apprenticeship Program. In retrospect, it is recognized that the apprenticeship program should have evolved with greater general education content, which is what apprentices are lacking, rather than strengthening hands- on skills. Although the Project did not aim to reform the apprenticeship program, the MONE is aware of the serious need to do so. Apprentices are almost exclusively primary school leavers with five years of education or less, joining the program at age 14. Between the ages of 11 and 14, primary school leavers cannot work and cannot join the apprenticeship program; this gap for primary school leavers needs to be addressed since it is a serious problem for children who are put to work illegally at a very young age and have little chance of returning to formal schooling in later years. The apprenticeship system encourages employers to hire youth with little education at low wages, and their current training inhibits career growth and job mobility. Apprentices should receive more education on core subjects like math, and science, and language arts, preferably through applied learning and other teaching methods designed for academically weaker students who are often from low socio-economic backgrounds. Lesson: Periodic assessments, more frequent than a mid-term review, should take place to assess the sustained validity of the project design and to make the necessary adjustments. Girls Vocational Schools. It is recognized that there are merits to offering single-gender education due to proven benefits in higher academic performance and higher enrollment of girls. In Turkey, however, single- gender schools are justified on an occupational basis, reinforcing traditional gender stereotypes. Field visits show that girls schools are generally well run and have motivated teachers, and that girls are benefiting from completing secondary vocational and technical education; furthermore, there is a demand for the more culturally appropriate single-gender schools. Girls vocational schools have also responded to increased demand for job training by women who are being forced into the labor market due to difficult economic times, in particular in rural areas; given their gender focus, these schools have been successful in attracting women into skills training in a non-threatening environment. Lesson: Overall, girls vocational schools have very positive results in encouraging school attendance by girls and encouraging girl's and women's employment; however, their programs should not be justified on the basis of gender-specific occupations. Decentralization Although the project was designed to conform with the centralized structure of the MONE, in retrospect, the project may have been more successful if it had been managed and administered at the regional and local levels. The project supported reliance and ownership at the central level of the MONE. Much of the daily decision making continues to be made at the central level and this overburdens the central level with administrative tasks, takes away from policy and quality control functions of the central level, and weakens the identity, ownership, and self-reliance of the school and its staff. Lesson: To succeed in strengthening individual schools, ownership of the project must be at the school level to be able to empower managers and teachers, while leaving central authorities to enforce quality control and conduct policy planning and research. Project Management The Bank repeatedly expressed concerns regarding the need for.effective project management and implementation. Finding the appropriate mix of autonomy and integration for project implementation management requires an informed choice about what is the best project management option to achieve outcomes for the beneficiaries, the project, and the implementing agency. Institutional development objectives are sometimes confused with the need for effective project implementation management, leading to a weak project management structure with insufficiently qualified staff and inadequate resources. Lesson: The borrower and the Bank must undertake an analysis - iv - of options for project management during project preparation, and make an informed choice of the correct project implementation management structure given constraints to achieve the project outcomes in a timely fashion. ix. Plans for Future Project Operations and Sustainability. The MONE is committed to project sustainability as evidenced by follow-up and project school surveys after Loan closing and its agreement to maintain a project coordinator in the PCU. The MONE has agreed to allocate a budget for overseeing the project's operational phase. Since project schools are expected to be continually over-enrolled to meet demands for secondary education and adult education, the project benefits are expected to be sustained and to increase greatly for several years to come. There are some risks that certain of the more sophisticated project equipment may not be repaired if needed due to lack of knowledge and resources at the local school level. PART I. PERFORMANCE ASSESSMENT FROM IBRD'S PERSPECTIVE A. STATEMENT AND EVALUATION OF PROJECT OBJECTIVES 1. Statement of Objectives The Non-Formal Vocational Training (NFVT) Project had two objectives as stated in the Staff Appraisal Report (SAR) dated November 12, 1986: (a) to increase and improve the supply of well-trained labor for manufacturing industries, through (i) improved quality of non-formal vocational training programs and enhanced relevance to the labor market needs; and (ii) expansion of enrollment in vocational education and training programs with an increase of about 110,000 trained workers per annum (b) to improve employment and income generating opportunities for disadvantaged groups with little education and training, through (i) employment of a significantly high percentage of graduates of non-formal vocational training institutes; and (ii) creation of about 16,000 self-employment opportunities, mainly for women engaged in income generating activities in the home. 2. Evaluation of Objectives. The project's first objective fit well into a long-term strategy to supply better trained manpower for manufacturing industries in order to increase productivity and growth of manufacturing output. Government gave priority to increasing manufactured exports, which would depend on the availability of better-educated and skilled manpower. The proposed project also responded to demands from industrialists for skilled technicians and craftsmen. This translated into the need for an effective formal and non-formal vocational education system to improve the knowledge, skills, and attitudes of an under-educated population. Vocational education would also compensate for a low level of compulsory education which remains at only five years -- in 1984 when the project was prepared statistics showed that 50% of the registered urban unemployed had primary education or less. Considering also that expansion of the secondary education system has not kept up with population growth, school to work linkages were, and continue to be, of top importance in the government's agenda. Thus, it was appropriate for the project to support the Government's focus on provision of quality training linked to employment and increased productivity. 3. The project's second objective supported the development of increased earnings potential and employability for the less advantaged socio-economic groups. This objective responded well to the Bank's poverty alleviation mandate addressing the needs of the disadvantaged -- the unskilled unemployed, adult women trying to enter the labor market, or the long-term unemployed who are also poor and un-skilled. The non-formal vocational training institutes are well placed within the public sector to assist those who are trying to enter the formal labor market and lack skills and attitudes to make that transition. It is difficult to address the needs of the unskilled unemployed, which usually includes migrants, housewives, early school drop outs, and seasonal workers; the network of public training facilities of the MONE responds to these needs, including training in entrepreneurship and self employment opportunities. Economic reforms have for some time been having a negative impact on income distribution leaving the unskilled unemployed behind, many of which are migrants from rural to urban areas with expectations for better employment opportunities; therefore, it was appropriate to recognize the importance of skills training for increasing employment opportunities, especially for women who were in transition from a traditional agricultural life or home life to a more modern urban lifestyle. Although this project was not appraised to include a poverty alleviation and gender focus for the development of women, in retrospect, the project visibly had such a focus and merits recognition in this regard. B. ACHIEVEMENT OF OBJECTIVES 4. Project Outcomes: The project is rated to have substantially achieved its physical objectives and partially achieved its institutional development objectives at completion. The quality of non-formal vocational education and training and its relevance to labor market needs improved significantly, in particular, compared to the pre-project status of these schools. Many vocational schools should be commended for they have made impressive use of resources to provide a solid vocational education, and one is left with the impression that education is taking place. and that values and skills are being acquired in preparation for the world of work- other schools are less dynamic and may leave an impression of need for improvement, initiative, and leadership which is sometimes lacking due to certain complacency at the local level that comes from a centrally regulated and hierarchical system and to lack of community advocacy. The project did complete and exceed appraised targets for support to schools, and all schools are successfully fully-enrolled or over-enrolled. The project supported 607 schools, more than twice the 268 schools expected at appraisal. Expansion of enrollment in the original project schools also exceeded appraisal targets by 20% with 130,967 students enrolled in project-supported programs. Furthermore, computer classrooms were provided for an additional 341 schools under a fourth phase using cost savings. In terms of technical assistance for program development, a total of 214 person-months were provided under the project, or 69% over appraisal estimates. Fellowship training for teacher and MONE managers and school inspectors added up to 801 months, slightly exceeding the appraised 790 months. New technology textbooks fell slightly short of target, with 32 out of 44 technical textbooks published under the project, but nevertheless this component made a significant contribution to the acute shortage of adequate textbooks. The final phase of the project included a highly successful integrated pre-school education program in conjunction with adult education for mothers, targeting low-income families and offered through adult training centers of the MONE. This Mother/Child education program trained 1,751 mother-child pairs, as well as well as 143 teachers for the delivery of this 6-month program. Finally, in terms of employment of graduates of project-supported programs, it was not possible to calculate employment rates as over 50% of project expenditures were incurred in the last project year. with most programs being 3-year programs: however, all students enrolled in Apprenticeship Training Centers are employed, and graduates from the Commerce and Tourism Schools are the most successful in obtaining employment in the growing service sector in Turkey -- these two groups making over 50% of the project supported schools. Other more narrow vocational specializations for manufacturing trades have had low employment rates in a job related to the student's respective program specialization. One concludes that the project achieved its most important development objective to increase employment opportunities for the most socially disadvantaged youths and unskilled adults as well as promoting employment opportunities for women to join the labor market. 5. Institutional Development: The Project achieved some of its institutional development objectives improving the MONE capacity to better respond to training needs from changing labor markets. In particular, institutional development did take place in the Directorate of Girls' Technical and Vocational Education (DGTVE); quality improvements for girls' vocational schools included: school management workshops for 78 school directors and assistant directors and 8-month job shadowing experiences in the UK for two central managers, technology education workshops for 66 teachers; modular curriculum development, including in-service teacher training programs; promotion of project objectives through lectures on quality improvements, including advanced teaching/learning methods; and establishment of networks between teachers, industry, and the community. Institutional development was successful under the DGTVE because of the direct involvement of the stakeholder who had full responsibility for implementation. Visits to selective schools during the completion mission verified the success of the programs of the DGTVE in outreach efforts to assist women in the community to consider job-related training and employment. Overall. developments in modular curricula in the MONE and in skills training for women are very positive results from the project and have opened schools to the community where they were closed before. In particular. these programs are at their best to help women gain confidence to enter the formal labor market. 6. The project did not achieve its institutional development objective to directly strengthen the Non- Formal Education Institute (NFED to be able to respond proactively to emerging labor market needs by updating vocational programs and through continued in-service teacher training. However, indirectly, the Project benefited a new vocational education research center, METARGEM, which was created as the project started. METARGEM quickly overshadowed the weak NFEI with greater autonomy and resources, and with the full support of the MONE. METARGEM also took much of the trained staff from the NFEI, and benefited from modular curricula and materials made available through the Project. Furthermore, some of the NFEI's responsibilities, mainly the maintenance of occupational standards and testing which were to be developed under the project, have also been officially assigned to an independent commission associated with the Ministry of Labor, which is supported by the Bank under the Employment and Training Project (Ln.3571- TU). The Project, however, was successful in supporting the development of new programs, with support from technical assistance. As the NFEl continues to be a weak agency, with limited staff resources, and with little support from MONE management, the MONE is reviewing the roles of these two agencies and would consider a possible merge which could result in efficiency gains and a well-recognized stronger entity. In conclusion on the achievement of development objectives, we can say the project achieved its most important objective to increase employment opportunities and thus improve the socio-economic status of the most socially disadvantaged youths and unskilled adult as well as promoting employment opportunities for women to join the labor market 7. Regarding institutional development in the area of project management, it is noteworthy that the PCU became much more effective in its last project year, with expertise developed in project management, procurement, and accounting, better integrating consultants with PCU staff. An important lesson for the Ministry is learning how to better use and work with contracted staff, a lesson that is important for the future development of the sector; regrettably, these realizations came a bit late for the PCU when many inefficiencies of working with the schools could have been averted. The last year of the project was overall the most successful of the project in all aspects -- greater efficiency in planning and organization of computer programs for 341 schools nationally with fully successful use of equipment by students and teachers, positive results from institutional development in the DGTVE and the PCU, and a successful early child development program, all having high educational and developmental returns. C. MAJOR FACTORS AFFECTING PROJECT IMPLEMENTATION 8. The major factors or constraints affecting implementation are explained below. We note, however, that these factors became clear only in retrospect, and that they should be taken into consideration by both the borrower and the Bank in the design and implementation of future projects. Also, this section should be read recognizing that the project was implemented substantively as appraised and completed within its projected implementation period of nine years; the comments do not mean to take away credit due to the Borrower who has ownership of the project and remained committed to its implementation. 9. Sector Policies. Part of the difficulty of the project was the fragmentation in vocational education, which mirrored the structure of the MONE. In retrospect, the project probably should not have supported the existing vocational education system and structure which has much duplication and inefficiencies. The project was not able to bring improvement to the overall management of vocational education, but the project was not specifically designed to make structural changes for improved efficiency; however, the system's inefficiencies become more evident as the system expands in a rapidly changing work environment. Many of the policies which drive the current vocational education system date back to the 1940's and are thus in need of --4- modernization, in particular, program specialization and pedagogy. The MONE strongly believes in a vocational focus for its secondary age students in response to a need for a smooth school to work transition, and due to restrictions in access to higher education. Some inefficiencies which prevent better use of resources include: (i) fragmentation with separate departments being responsible for vocational education and training for the 14-19 age cohort which makes up over 50% of post-primary school enrollment in the public school system; (ii) poor school management with almost no accountability for quality education, and few incentives for motivating teachers, compounded by weak capacity at the central level for quality sector management and development; (iii) an oversupply of narrow vocational specializations relevant to the job market, with fragmented management, teachers, and students, supported by an entrenched vocational higher education system which trains teachers for narrow specializations: (iv) recognizing that girls vocational schools are successful for promoting women's employment and increase enrollment of girls schooling in a culturally appropriate environment, the specific gender specialization they offer in traditional domestic occupations reinforces gender stereotypes; and (v) outdated pedagogy not responsive to the skills needed for today's labor markets requiring critical thinking, analytical, and problem solving skills, higher order language skills, math and science knowledge, computer technology skills, and communication skills. These inefficiencies affect the operation and quality of schooling. 10. Local School Participation and Local Capacity. The project was designed in phases where schools would be selected to participate in the project through needs surveys. Although surveys were conducted using agreed criteria, based on a representative sample of schools visited, it appears that the selection was done more on the basis of supply rather than demand to introduce a program at a school, and more influenced by politics in a highly centralized Ministry which allocates resources rather than by individual planning at the school and community levels. Surveys were often done through the mail as funds for local travel were unavailable to either central MONE staff or PCU staff. Consequently, some programs were launched in schools where there was little interest or need. In several schools there was lack of ownership often compounded by lack of technical knowledge and of modern pedagogy. A key weakness affects school planning and quality is the lack of school management training and accountability for headmasters. The MONE should embark on a serious effort of assessment of the knowledge and performance of school managers, including efforts to empower and appoint qualified managers with updated knowledge, skills and attitudes to operate better quality schools and to optimize the use of resources. 11. Weak Institutional Capacity and Centralization One of the main factors affecting project implementation negatively was the managerially and technically weak nature of the centralized Ministry administration. The centralized power in the Ministry hinders initiative at the provincial and school level -- while the central level can do little to help the local level, in turn local schools often expect guidelines and support from their Directors at the central level. This inaction on both ends is detrimental to the education system, and it is hoped that through the Government's new decentralization focus a change will take place in the education sector from a generally passive to a more proactive and dynamic system. Action at the local school level is also hindered by lack of technical knowledge both in subject areas and cutting-edge pedagogy of teaching and of managerial staff. Like any organization or entity, a school can be as much as its leadership sets out to achieve; thus, the introduction of a system of incentives to strengthen school management and develop leadership, initiative, and accountability at the local school level would greatly benefit the school system, while at the same time strengthening the central supervisory and advisory level. 12. Project Complexity and Implementation Although the project was well prepared and project management and implementation arrangements were appraised carefully, in reality the mechanisms for project implementation did not work as envisaged given the difficult working environment in a large centralized yet weak bureaucracy of the public sector. Project implementation was to be headed by a Deputy Undersecretary, a Management Committee, and a PIU (renamed PCU), with the project being national in scope and including a variety of schools and programs for both formal secondary vocational education as well as adult non-formal training. During implementation there were several deputy undersecretaries responsible for the different departments implementing the project each claiming ownership over the various schools they oversee. Having too many players stalled the project often, and the large number of programs and schools nationally made it difficult to implement centrally, especially given the lack of supervisory ability of the MONE. Although great efforts were made to provide technical assistance in project implementation to the general directorates, this seemed to have been successful only in the Directorate for Girls' Vocational Education (DGVTE). In theory the project should have been implemented more quickly and smoothly because there was great need for the project inputs and constant support of the borrower throughout implementation, but in practice implementation reverted to the pace of the local environment which owned and implemented the project. 13. Instructor Training A critical component of the project was the ability to provide ongoing in-service training to teachers, instructors, and school managers. As this important component was placed with the NFEI, one of the weakest arms of the MONE, very little was achieved through the Institute. In retrospect, the target of 30% retraining of instructors was overambitious given capacity constraints, and it may have been more appropriate to strengthen and encourage cooperation with the MONE's In-Service Training Department responsible for this task. Economic austerity measures also affected negatively in-service training as this is the first activity which suffers budgetary cuts. Focus on the importance of provision and promotion of in-service training should continue as it is critical to improving the quality of education. The current perception that in- service training is not necessary, or that it is an admission of incompetence needs to be changed through special campaigns aimed at changes in attitude, in particular, for school managers and central managerial staff. 14. Implementation Delays. The project suffered a two year delay in start-up mainly due to uncertainty regarding the establishment of a joint Project Implementation Unit for the two Bank- financed projects, including the Industrial Schools Project (Ln. 2536-TU), and also due to delays in awarding the contract for the technical assistance services envisaged under the project. The quality of project management also caused delays due to lack of adequate resources for project implementation and shortage of qualified staff in the MONE. Unfortunately, the follow-up project with the MONE (National Education Development Project, Ln. 3192-TU), also suffered a similar two-year delay in start-up due to complications in the same areas. Indecision on the part of the MONE management compounded by political influences and internal inefficiencies worked to delay most contract awards and, consequently, delayed the delivery of project inputs to beneficiaries. It is now recognized that to counter these inefficiencies, projects need to be shorter, with specifically focused interventions, detailed implementation plans, and short-term targets or milestones. It is also important to note that commitment to implementation on the part of the borrower works better for short-term activities providing a good incentive politically and good motivation to staff who are commended for visible results. Also, long- term institutional development objectives can be achieved through short term success which raise awareness and expectations. Despite these delays and inefficiencies project implementation is always an enriching experience to the client considered to be a supplemental benefit and a catalyst for change. Effective project implementation in the MONE continues to be a serious concern for both the Government and the Bank. 15. Proiect Management. The Bank repeatedly expressed serious concerns in the area of project management, and this applied equally to the three projects under implementation by the MONE. The project had a complex implementation scheme because of the involvement of several MONE departments and a larger variety of schools; therefore, it was difficult for the PCU to keep the various directorates and departments equally involved at all times. In addition to the problems mentioned earlier on management indecision and political influences, there are other issues that require attention, including: assignment of better qualified managerial, technical and administrative staff to the PCU with fluent foreign language skills; lack of a budgetary allocation for field supervision by PCU staff to allow for regular travel to visit school sites: and reluctance to contracting national fixed-term or consultant staff to PCU positions as needed, preferring to assign teachers with lower qualifications and poor language skills. Unfortunately, use of Loan funds for efficient project management and implementation still remains controversial for both the Treasury and the MONE because of extremely low salaries and internal travel budgets in the public sector. Despite all -6- constraints, it is noteworthy to recognize that in the last two years of implementation the PCU was much more effective in carrying out its responsibilities 16. Use of Consultants. The use by the MONE of both local and international consultants continues to be a controversial issue despite the fact that such services are in great need. In some areas the use of consultants was quite successful while in others it proved unproductive because of a mismatch between recipients and providers of the services. Some of the factors preventing the effective use of consultants continues to be: (i) difficulty in forming partnerships between MONE officials and with national/international consultant counterparts because of the large gaps between the qualifications and incomes of civil servants and consultants; (ii) the foreign language working knowledge of MONE staff is limited to a handful of individuals, which also hampers their participation in foreign training; (iii) lack of precisely defined and appropriate TORs, compounded by lack of understanding of the objectives of technical assistance by much of the MONE staff and/or consultants, with consequent unrealistic expectations, (iv) lack of leadership by the Client in monitoring the inputs/outputs of consultant's work and ensuring that TA is well targeted at appropriate tasks that will yield results and show evidence of the benefits of TA; (v) keen assessment and understanding by the contractor of the TORs ensuring that the Client also has realistic expectations of consultancy outputs and gives importance to the Client's involvement when appropriate. D. PROJECT COST CHANGES 17. Actual total project costs are calculated at US$72.16 Million, a marginal overrun of 1.5% due to greater school renovation undertaken by the MONE. On the other hand, Loan expenditures had an underrun of US$4.1 Million, or 7% of the Loan which was canceled. This cost savings was mainly due to decreases in the price of technology equipment from the appraised estimates in the mid 1980's and to the MONE's conscious attention to economy in procurement. Although the MONE requested informally an extension of the Loan to use the remaining funds, the Bank recommended against an extension due to changes in strategy to strengthen and reform vocational education in Turkey. The main differences in project expenditures were: (i) higher expenditures in technical assistance (50% above appraised estimates) due to the addition of the Mother/Child Development Program and additional technical assistance obtained for improved project school performance and project implementation; (ii) lower expenditures for books and educational materials (58% below appraised estimates) due to reluctance by MONE to purchase educational materials which are considered a luxury; and (iii) higher expenditures allocated by the MONE for school refurbishment (120% above appraised estimates). A reallocation of funds was made under a Loan amendment in July 1994 to: (a) increase support for improved school performance and project implementation (US$1.0 M); (b) finance the expansion of the Mother/Child Development Program (US$0.9 M); (c) finance computer classrooms to improve computer literacy for today's job market (US$9.0M); and (d) finance additional development of girls secondary vocational schools. E. PROJECT SUSTAINABILITY 18. The MONE is fully committed to project sustainability as evidenced by follow-up and project school surveys after Loan closing and its agreement to maintain a project coordinator in the PCU through end-1997. The MONE has agreed to allocate a budget for overseeing the project's operational phase. As mentioned earlier, project schools are expected to be continually over-enrolled to meet demands for secondary education and adult education, and project benefits will continue to increase for several years to come. There are some risks that certain of the more sophisticated project equipment may not be repaired if needed due to lack of knowledge and resources at the local school level. However, with greater efforts to decentralize responsibility and budgets to the local provincial and local school levels, it is expected that local school managers and teachers will be able to take more initiative and act directly when services, repairs, or training is needed. The Bank hopes to support in future the government to implement its decentralization plans for the education sector. F. BANK PERFORMANCE 19. The Bank was responsive and sensitive to national development priorities in the design of project objectives, and the project fitted well with the country development strategy. Project inputs were needed to modernize the knowledge and skills of the labor force and significantly improve the quality of vocational education and responsiveness to emerging job markets. The project is recognized to have contributed to achieving these goals by project beneficiaries-- students, teachers, and employers. The project was adequately prepared although it was ambitious in expecting institutional change to take place in a weak agency and overestimated the implementation capacity of the MONE as well as capacity at the local school level. Although the project suffered from inefficiencies during implementation in project management and program quality at the school level, the Government and the Bank were cognizant of these problems and repeatedly addressed these shortcomings which were discussed with Government officials. Pointing out problems repeatedly in Aide Memoires was frustrating for the Bank having no responsibility for implementation, while the borrower recognized these inefficiencies but was unable or unwilling to take action. Lack of initiative and managerial leadership are problems inherent in a weak and underpaid public sector. As the project was fully implemented and received consistent government support throughout implementation and is a sustainable project in an area which continues to be of top priority to the government, the project should be considered to have been on balance satisfactory at preparation and implementation. G. BORROWER PERFORMANCE. 20. The MONE and Government maintained full commitment to project objectives throughout project implementation and remain committed to project sustainability. The MONE complied in implementing the project as appraised and achieved most of both the physical and social objectives of the project. The MONE and the government continue to be committed to improving the employability of their secondary school graduates and to providing job related skills to the unemployed. In this sense, support for vocational education remains strong since a large segment of the population has been left behind without the knowledge, skills, and attitudes needed for today's quickly evolving service sector jobs and other competitive job markets in Turkey. Resources for the education sector, however, have remained limited and thus the project's resources were highly valuable. The MONE should also be commended for being economical in its purchases, proposing least cost solutions. Where the MONE has remained weak is in responding to the need for swift project implementation in the use of Bank funds, and being more open to contracting qualified outside staff to complement its weaknesses in management and administration and the lack of technical capacity at both the central level and local school level to modernize pedagogy and the vocational education system. However, recognizing that institutional development and evolution of the public sector is a longer-term goal, the MONE has remained committed to improving its recognized weaknesses. In balance, the borrower's performance is rated as satisfactory based on its successful completion of the project and its commitment to project sustainability. This rating also takes into consideration the fact that the MONE improved significantly project implementation in its last year and was able to monitor efficiently the last project phase. In particular, the MONE/PCU is commended for an excellent job in the implementation of the computer literacy classrooms component in the last project year. H. ASSESSMENT OF OUTCOME 21. Based on substantive achievement of project objectives (see section B above), which targeted the lower-income and more disadvantaged students and adults and women emerging from a home life into the labor market, the project is rated as having achieved a "satisfactory" outcome. Notable project benefits, among others, included: (i) strengthened the weakest segment of the public education system which lags most in effectiveness and continues to be in need of reform: (ii) upgraded the status of many schools through provision of essential equipment, materials, and refurbishment, for which there have always been funding shortages; (iii) substantively increased satisfaction for students, teachers, schools, and the community which benefited from new job skills and from the benefits of new technology, teacher development, and more books and materials which have been scarce in the Turkish language; (iv) increased efficiency in use of the sector's resurces by opening up schools to the community through the development of many adult courses; traditionally schools operated only with formally enrolled students, but the project opened doors to the community and helped meet demand for job skills, in particular, for women; (v) increased awareness for the need for life- long-learning, making schools a learning resource for the community; (vi) raised awareness of the importance of modular adult education for the Adult/Public Training Centers to meet the needs for continuing education and skills updating; and (vii) last but not least, supported the development of schools which are overenrolled since students exceed placement availability and, given current population growth, demand for schooling will continue to exceed placement over the next several years; pressures from immigration from the Balkans and Turkic Republics is an additional stress to the school system. I. FUTURE OPERATIONS 22. Further expansion and improvements are still greatly needed in the education sector in Turkey to achieve quality and efficiency in the public education and training system. A public sector that is characterized as overly centralized and bureaucratic yet weak and inefficient is in need of remedy to meet the population's expectations for improvement in their socio-economic status as they join the European Union and compete in global markets in the twenty-first century. Unfortunately, much of the labor force in Turkey remains unskilled or semi-skilled in most sectors and this is a major constraint to improving living standards and economic stability. There is need and opportunity for the Bank to support the growth and development of the public education sector in all areas of the system while strongly promoting needed efficiency gains and reforms. Also important for the Bank is to foster the development of the private sector in the provision of education in an effort to increase contributions from families which can afford private education alternatives while allowing the public sector to expand coverage to much of the underserved population. Although it is recognized that expansion of basic and secondary education is fundamental, including secondary vocational and technical education, affordability of such an expansion and efficiency in the allocation of resources and sources of funding need thorough review and policy orientation to maximize the benefits from resources potentially available to the sector. Expansion of compulsory basic education and restructuring and expansion of secondary education are the two main priorities in the Government's national development plan for the education sector; pre-school education is also being recognized as essential, but it is questionable how affordable it is for the public sector. The Bank is now financing studies under amended Loan 3192-TU on the feasibility and costs of such expansions, while also identifying greater efficiency measures for the allocation of sector resources. J. KEY LESSONS LEARNED 23. There were several lessons learned from this project, many of which were applied during project implementation. The lessons outlined in the recent ICR for the Industrial Schools Project completed one year earlier (SecM96-275) are all applicable to this project as well since these projects were implemented simultaneously by the same staff. Although these lessons are recognized by the borrower, it is often difficult to take action upon this knowledge due to political and capacity constraints. Selective main project lessons are summarized below. Poor School Management. The importance of effective school management is often overlooked or underestimated. School managers are not selected for their leadership or managerial skills, and tend to stagnate in their positions while not being required to update their managerial and pedagogical knowledge and skills; thus, many school managers have little initiative, they do not provide incentives to teachers, and they also lack outreach skills to interact with the community and local businesses. In the Project, school managers often did not have the ability to select and run appropriate programs as was envisaged. Lessons: Education projects should include substantive training for school managers and encourage periodic assessments and rotation of school managers on a competitive basis. -9- Adjustment in Project Design. Education projects need to adjust to emerging needs in society, discoveries in science, technology, and pedagogy, and changes in labor markets; they can not be static in design over a long implementation period. Although adjustments were made to the project during implementation, it was difficult to incorporate timely change and encourage innovativeness, partly because of the rigidity of the public sector and partly due to the perceived requirement to adhere to an agreed project on both the side of the Borrower and the Bank. Lesson: The borrower and the Bank need to apply much more flexibility in adjusting the project design and simplifying procedures without the need for formal amendments for changes that are consistent with the project's broad objective and scope. Apprenticeship Program. In retrospect, it is recognized that the apprenticeship program should have evolved with greater general education content, which is what apprentices are lacking, rather than strengthening hands- on skills. Although the Project did not aim to reform the apprenticeship program, the MONE is aware of the serious need to do so. Apprentices are almost exclusively primary school leavers with five years of education or less, joining the program at age 14. Between the ages of 11 and 14, primary school leavers cannot work and cannot join the apprenticeship program; this gap for primary school leavers needs to be addressed since it is a serious problem for children who are put to work illegally at a very young age and have little chance of returning to formal schooling in later years. The apprenticeship system encourages employers to hire youth with little education at low wages, and their current training inhibits career growth and job mobility. Apprentices should receive more education on core subjects like math, and science, and language arts, preferably through applied learning and other teaching methods designed for academically weaker students who are often from low socio-economic backgrounds Lesson: Periodic assessments, more frequent than a mid-term review, should take place to assess the sustained validity of the project design and to make the necessary adjustments. Girls Vocational Schools. It is recognized that there are merits to offering single-gender education due to proven benefits in higher academic performance and higher enrollment of girls. In Turkey, however, single- gender schools are justified on an occupational basis -- domestic occupations for girls and industrial or technical occupations for boys, reinforcing traditional gender stereotypes. Field visits show that girls schools are generally well run and have motivated teachers, and that girls are benefiting from completing secondary vocational and technical education; furthermore, there is a demand for the more culturally appropriate single- gender schools. Girls vocational schools have also responded to increased demand for job training by women who are being forced out into the labor market due to difficult economic times, in particular in rural areas; given their gender focus, these schools have been successful in attracting women into skills training in a non- threatening environment. Lesson: Overall, girls vocational schools have very positive results in encouraging school attendance by girls and encouraging girl's and women's employment Decentralization Although the project was designed to conform with the centralized structure of the MONE, in retrospect, we realize the project could have been more successful if it had been managed and administered at the regional and local levels. The project supported reliance and ownership at the central level of the MONE. Much of the daily decision making continues to be made at the central level and this overburdens the central level with administrative tasks, takes away from policy and quality control functions of the central level, and weakens the identity, ownership, and self-reliance of the school and its staff. Lesson: To succeed in strengthening individual schools, ownership of the project must be at the school level to be able to empower managers and teachers, while leaving central authorities to enforce only quality control and conduct policy planning and research. Project Management The Bank repeatedly expressed concerns regarding the need for effective project management and implementation. Finding the appropriate mix of autonomy and integration for project implementation management requires an informed choice about what is the best project management option to achieve outcomes for the beneficiaries, the project, and the implementing agency. Institutional development - 10- objectives are sometimes confused with the need for effective project implementation management, leading to a weak project management structure with insufficiently qualified staff and inadequate resources. Lesson: The borrower and the Bank must undertake an analysis of options for project management during project preparation, and make an informed choice of the correct project implementation management structure given constraints to achieve the project outcomes in a timelYfashion. PART 2. STATISTICAL ANNEXES Table 1: Summary of Assessments Table 2: Related Bank Loans Table 3: Project Timetable Table 4: Loan Disbursements: Cumulative Appraised and Actual Table 5: Key indicators for Project Implementation Table 6: Key Indicators for Project Operation Table 7A: Project Costs Table 7B: Project Financing Table 8: Status of Legal Covenants Table 9: Bank Resources. Staff Inputs Table 10: Bank Resources. Missions Table 1: Summary of Assessments A. Achievement of objectives Substantial Partial Neglizible Not applicable Macro policies X Sector policies X Financial objectives X Institutional development X Physical objectives X Poverty reduction X Gender issues X Other social objectives Environmental objectives X Public sector management X Private sector development X Other (specify) B. Project sustamability LkE!y Unlikely Uncertain x C. Bank performance Highly satisfactory Satisfactory Deficient Identification X Preparation assistance X Appraisal X Supervision X D. Borrower performance Highly satisfactory Satisfactory Deficient Preparation X Implementation X Covenant compliance X Operation X * Note: The satisfactory implementation rating is based on improved performance in the last two project years, and particularly in the final year E. Assessment of outcome Highly satisfactory Satisfactorv Deficient x Table 2: Related Bank Loans Loan/Credit title Purpose Year of approval Status Preceding operations 1. Education Project Development of secondary technical 1971 Completed 1981 (Ln.748-TU) and vocational education 2. Industrial Training I Development of post-secondary, two- 1984 Completed 1990 (Ln. 2399-TU) year technician training programs. 3. Industrial Schools Upgrading of Industrial Schools 1987 Completed 1994 Subsequent operations 1. Industrial Training II Expansion of post-secondary, two-year 1988 Active (Ln.2922-TU) technician training programs. 2. National Education To focus on quality enhancement and 1990 Active Development Project management improvement aspects of (Ln.3192-TU) the Government's broad sectoral development programs through improved student achievement, educational efficiency, teacher education, and organizational management. Table 3: Project Timetable Steps in project cycle Date planned Actual Identification (Initial Project Brief) September 11, 1985 Preparation October, 1985 Appraisal May, 1986 April 1, 1986 Negotiations November 3, 1986 Board Presentation February, 1987 January 6, 1987 Signing January 12, 1987 Effectiveness April 2, 1987 Project Completion June 30, 1995 December 31, 1995 Loan Closing December 31, 1995 December 31, 1995 Table 4: Loan Disbursements: Cumulative Estimatd aWd Actual (US$ thousands) FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95 FY96 Appraisal estimate 3.50 6.50 14.50 27.00 41.50 52.00 57.00 58.50 Actual 2.99 3.75 4.54 5.40 12.68 22.49 24.10 26.00 54.90 Actual as % of estimate 85 58 31 20 31 43 42 44 93] Date of Final Disbursement: Aprsil, 1996 (?) Undisbursed Balance Cancelled: US$ 3.6 Million Table 5: Studies Included in the Project Purpose as defined at Status and Study appraisal or as redefined Impact of Study 1. Evaluation Study to make a thorough evaluation of the following aspects: An evaluation study was completed Study (a) improvement of training needs assessment survey and their impact according to revised TORs as there were no on the relevance of training programs to the labor market need; graduates yet from project-supported (b) ability of graduates to find employment, to include a follow-up of schools. Linking success of programs with the employment record of graduates (first graduates of the project will employment of graduates is a controversial enter employment in 1998); issue in the MONE because these linkages (c) performance of the graduates on the job through interviews with are weak. Assessment of the effectiveness of employers; programs has also been carried out by (d) success in adapting existing curricula and developing new one; METARGEM; however, studies and (e) effectiveness of new in-service training for instructors and recommendations have had little impact on perfomance of instructors; program content and restructuring of MONE (f) development of the entrepreneurship skill training and information management. system on financial and advisory services. Table 6: Key Indicators for lmno umtation Key Implementation Indicators in SAR Unit Quanuaty I II III IV Total ShosEauioned & Refurbished Public Training Centers centers 159 40 120 159 9 3 Apprenticeship Training Centers centers 56 55 37 56 0 1 Adult Technical Training Centers centers 10 9 5 10 2 26 Commer. & Tourism Training Centers centers 30 30 3 30 272 335 Womens Vocational Education & Trg. centers 15 3 12 15 58 Apprenticeship Centers (added in '93) centers 23 0 0 23 0 foTAL94 Proaram-SunoortedSoecialzation Printing centers 2 2 Sewing centers 35 85 14 1 Garment and Leather Garment centers 39 39 29 107 Typing centers 46 26 72 Food Technology centers 2 2 Trico Knitting centers 21 75 45 141 Machine Embroidery centers 26 112 45 183 Office Equipment centers 10 10 Language Laboratories centers 90 90 Refrigeration & Air Conditioning centers 8 8 Electricity centers 41 79 39 159 Electronic centers 27 61 88 Ceramics centers 7 7 Shoe Making centers 5 8 13 Metal Works centers 20 60 8C Wood Working centers 55 55 Automotive centers 63 40 1 Computer Technology centers 42 293 3 Machine Shop centers 25 42 6 General Education Equipment centers 98 184 282 Comptuer Literary Classrooms TOTAL 2,281 Technical Assistance In-Service Training Curr. Design mm 54 62 Trainign Needs Assessment mm 12 6 Skill Testing and Certification mm 12 10 Entrepreneurship Courses mm 12 12 Support to PCU (Management & Proc.) mm 18 37 Evaluation Study mm 6 6 Instructional Mat. mm 12 9 Tourism mm 0 5 Natural Gas mm 0 5 Employment Promo. mm 0 1 Subtotal 126 153 echnical Assistance (2nd Contract 1995) Financial mm 13 9 Procurement mm 12 10 Project Implementation mm 19 8 Community Development mm 6 6 Leaming Resource Development mm 6 6 Office Manager mm 13 10 Institutional Development mm 12 1 Subtotal 81 60 TOTAL 207 213 Table 6: Key Indicators for Implementation Acual Key Implementation Indicators in SAR Unit Quantity to date (mm) FellowshiDB In-Service Teacher Training mm 450 39 Curiculum & Program Design mm 84 84 Instruct. Mat. & Leam. Res. Dev. mm 60 55 Skill Testing and Certification mm 36 55 Entrepreneurship Courses mm 45 56 Community Development mm 16 16 Support to PILU mm 12 12 METGE Teachers & Appren. Persons. mm 54 29 School Inspectors (1993) mm 60 V TOTAL 817 738 Instructional Materdas .SM 2 Translation ad Printing of Textbooks each 44 32 Production Revolving Funds funds 200 200 Enrollment in ProjeWt Specializations Electical students 670 4,820 Electronics students 390 2,193 Computers/Accounting students 520 40,427 Metalworking students 420 11,662 Weding students 100 2,84 Woodworking students 340 13,090 Automotive students 540 17,353 Machine Shop students 420 Plumbing students 100 1,160 Bricklaying students 60 12 Sewing students 700 11,575 Garment Making students 680 799 Leather Garment Making students 120 23 Trico students 440 871 M/C Embroidery students 560 11,858 Carpet Weaving students 320 1,718 Killim students 160 70C Blanket Making students 40 44C Home Economics students 480 80C Book Binding students 120 38 Shoe Making students 100 114 Language Training students 240 2,118 Typing students 540 1,662 Eng. Drawing students 80 24 Handicrafts students 640 3,801 Ceramics students 20 § TOTAL 8,800 130,964 ^ontractina Incremental Instructors (1994-95) Public Training Centers Instructor 1,080 5,163 Apprenticeship Training Centers Instructor 140 1,118 Women's Training Centers Instructor 90 8 Commerce & Tourism Training Centers Instructor 60 729 Adult Technical Centers Instructor 27 4@ OTAL 1,397 8,245 Integrated Pre-School Proaram Mother-Child Couples Trained Mother/Child 0 1,751 New Teachers Trained Instructor 0 109 Teacher In-Service Training Instructor 0 34 Province Coordinators Trained Managers 0 20 Symposium Earty Conference 0 1 Table 7A: Project Costs Appraisal Estimate (USSM) Actual (US$M) Item Loal Foreign Total Local Foreign Total costs costs cost* costs 1. Refurbishing 1.4 .4 1 8 4.0 0.0 4.0 2. Equipment 6.9 53.5 60.4 11.7 48.0 59.7 3. Instructional Materials 1.1 0.6 1 7 0.0 0.7 0.7 4. Consumable Materials 2.8 - 2 8 1.2 0.0 1.2 5. Technical Assistance 0.4 4.0 4 4 0.9 5.7 6.6 a. Consultants 0.3 1.5 1.8 b. Fellowships 0.1 2.5 2.6 TOTAL 12.6 g 711 17.8 544 72.2 Table 7B: Project FiNancing Appraisal estimate (US$M) Actual/estimate (US$M) Local Foreign Local Foreign Source costs costs Total costs costs Total IBRD - 58.5 58.5 - 54.4 54.4 Govt 12.6 - 12.6 17.8 - 17.8 TOTAL 12.6 58.5 71.1 17.8 54.4 72.2 Table 8: Status of Legal Covenants Agreements Section Covenant Status Original Actual Description of covenant Comments Class fulfillment Fulfillment date Date Loan 2776- Section 3 C The Borrower shall open and In compliance TU 2.02 (b) maintain in Dollars a Special Account in its Central Bank. Section 10 C The Borrower shall carry out the In compliance 3.01 (b) Project in accordance with the Implementation program set forth in Schedule 5 to this agreement. Schedule 5 C The Ministry shall have the In compliance. 5.A general responsibility for the implementation of the Project. The Project Coordination Unit (PCU) shall operate as the technical and administrative secretariat of the Project Management Committee. Schedule 10 C The responsibility for the In compliance 5.B implementation of the program provided under Part A of the Project shall be assigned to the relevant directorates and coordinated by the Deputy Undersecretary for Tech. and Voc. Training. Schedule 3 C Each NFVT institution shall In compliance 5.B.2(a) prepare a separate application for assistance. Applications will be forwarded from the provincial level to the management committees for approval, in accordance with criteria satisfactory to the Bank. Schedule 3 C The project will be implemented In compliance 5.B.2 (b) in three phases. Phases II and III will be prepared in accordance with the same selection criteria as Phase I, unless the Borrower and Bank shall otherwise agree. Schedule 5 C The Borrower shall ensure the In compliance 5.B.3 (a) availability of staff and facilities for the PCU until the completion of all activities under the project. Schedule 5 C Thc Borrower shall assign to the In compliance 5.B.3 (b) PCU the responsibility for the coordination of activities under the Project and monitoring their progress; the maintenance of the Project Accounts; and the procurement of goods and services under the Project. Agreements Section Covenant Status Original Actual Description of covenant Comments Class fulfillment Fulfillment date Date Schedule 5 C The relevant directorates shall be In compliance 5.C.1 responsible for collecting and furnishing the PCU with information on the status of their activities. The PCU will use this information to compile quarterly reports to the Management Committee and the Bank. Schedule 5 C 06/30/91 The progress reports shall form In compliance 5.C.2 the basis for the evaluation referred to in Part A(2) which shall be furnished to the Bank. Section 1 C The Borrower shall maintain, or In compliance 4.01 (a) cause to be maintained, records and accounts adequate to reflect the operations, resources and expenditures of the Borrower necessary for carrying out the Project or any part thereof. Schedule 1 C The Borrower shall ensure that 4.01 (b) (i- each fiscal year the Project iii) accounts and Special accounts shall be audited in accordance with auditing principles consistently applied, by independent auditors acceptable to the Bank. The Borrower shall furnished the Bank, as soon as possible, with a certified copy of the audit report. Covenant Class: . Accounts/Audit 2. Financial Performance/generate revenues for beneficiaries 3. Flow and utilization of Project Funds 4. Counterpart Funding 5. Management aspects of the Project or of its executing agency 6. Environmental covenants 7. Involuntary Resettlement 8. Indigenous People 9. Monitoring, review, and reporting 10. Implementation 11. Sectoral or cross-sectoral budgetary or other resource allocation 12. Sectoral or cross-sectoral regulatory/institutional action 13. Other Status: C: Complied with CD: Compliance after Delay NC: Not Complied with SOON: Compliance Expected in Reasonably Short Time CP: Complied with Partially Table 9: Bank Resources:Staff Inputs Stage of project Planned Revised Actual cycle Staffweeks Staffweeks Through Appraisal No data available No data available 65.2 Appraisal-Board No data available No data available 2.9 Supervision No data available No data available 108.4 Completion 11.0 11.0 TOTAL Table 10: Bank Resources: Missions Performance rating Days Stage of project Month/ No. of in Specialized staff Types of cycle year persons field skills represented Implementation Development problems Status objectives Through Oct. 85 Not Not Not Available appraisal Dec. 85 avail. Avail Appraisal Apr. 86 3 17 ECN,PRC,EDC through Board May 86 1 2 LEG approval I_I Supervision Mar. 87 3 6 ARC, EDC, EDC 1 1 Jan. 88 2 6 TM, EDC 1 1 Jun. 88 1 8 EDC I 1 Dec. 88 2 3 TM, EMT I 1 Jun. 89 2 TM. EMT 2 1 Nov. 89 2 5 TM, EMT 2 2 May 90 3 8 TM, ARC, EDC 2 2 Dec. 90 1 5 EDC 2 2 May 91 2 7 TM, EDC 2 2 Feb. 92 3 5 TM, EDC, EMT 2 3 Sept.92 3 5 TM, EDC, HR 2 3 Jan. 93 3 5 TM, EDC, HR 3 3 Apr. 94 2 4 TM, COM S U Oct. 94 3 8 TM, COM, EDC S U Jul. 95 3 12 TM, COM, HR S U Dec. 95 1 7 TM S U Completion Apr. 96 3 10 TM, COM, HR S U EDC = Technical Educator COM Computer Technology Specialist ARC = Architect HR = Human Resources Specialist ECN = Economist EMT Employment & Training Specialist TM = Task Manager PRC = Procurement Specialist File: m:\patricia\icr2776\tables APPENDIX A EVALUATION OF PERFORMANCE A. KEY LESSONS LEARNED 1, Additional to the experience gained from the implementation of the project, it will be useful to take into consideration these lessons for future planning. 2. Proiect Management: In order to realize the objectives of such a comprehensive project, the responsibilities of the project management unit and other Ministry units should be determined in detail and the terms of reference for the respective staff, as stated in the organization chart, should be included in the planning. It is necessary to allow the unit responsible for the project implementation to have the expenditure authority on the budget foreseen by the project activities. It is considered impractical to have an efficient and productive output with a project management unit lacking the above-mentioned characteristics. 3. Technical Assistance: The efforts of planned consultants who will work in the areas determined as essential for the development of the sector, should be prepared in detail in the project design. The Ministry personnel to be assigned as counterparts of these consultants should be determined according to the terms of reference. This point is essential in terms of productiveness for the unit which provide services and for the Ministry to obtain efficient outputs from these services. Unless these points are provided, it is not possible to obtain any positive result. The result will be a loss of time and money. 4. Overseas Fellowship Traininq: Since the cost of fellowship training is quite high, the goal should be determined more carefully within the project. Especially, a provision for overseas training of those having a relatively low level foreign language proficiency will not be productive. A planning which includes an initial in-country training of the selected candidates will lead to a more useful output both in terms of project cost and of the benefit to be obtained from the overseas training. The scope of the overseas training and the qualifications required should be detailed in light of the project objectives. 5. Instructional Materials and Program Development: Curriculum development is a difficult and important issue in vocational and technical education. Under the project, this issue is planned to be under the responsibility of the Non-Formal Training Institute (NFTI). The organization of the NFTI, affiliated to the General Directorate of Apprenticeship and Non-Formal Training, lacks the participation of the work/business sector. On the other hand, the necessary professional working environment could not be created due to the qualifications of its staff being far from those required in an institute. Regarding these type of studies, a structure containing the participation of the business community should be established and such an organization should have its own budget, along with a comprehensive knowledge about the related fields. The efficiency and continuity of the activities can only be provided with a structure which includes a strong participation from the business community. Since this could not be provided under the project, the activities remained as theories and no application opportunity could be found for the prepared documents, trained staff and selected fields. 6. Inservice Teacher Training: The MONE has an Inservice Training Unit to organize inservice training activities. The budget with regard to the inservice training activities is allocated directly for this Unit. All training institutions inform this Unit about their inservice training needs and the In-service Training Unit makes the necessary planning according to budget-related facilities. At the macro level, though this kind of structuring is considered to have economic aspects, the determination of inservice training needs, timely and efficient response to this need, coordination of this training by responsible units and even locally held in-service training seminars would be productive. 7. Procurement Procedures: In vocational and technical education, the application of a program sensitive to environmental needs requires the determination of needs according to this objective. Although specialization has been achieved on training needs assessment, the activities as to a large-scale training needs assessment could not be expanded. Though the central level planning have been useful in terms of timelines and quality, some procurement held for the project schools have led to unprofitable use and ownership due to incomplete knowledge of physical infrastructure and environmental needs. Hence, adequate resources and time should be allocated for needs assessment during the project design and at the commencement of the implementation process. B. FUTURE APPLICATION OF LESSONS LEARNED 8. Under the project, a total of 341 schools and some departments of 293 non- formal training institutions were equipped with computer hardware and software. Providing the industry sector with a more qualified work force through all non-formal vocational training institutions can only be possible with full-capacity utilization of this equipment and with the expansion of project benefits to the other schools. Since the expansion of such a physical capacity is hard work in financial terms, it will not be possible to be a profitable one. Therefore, initially, the following should be provided: (a) Beyond its contribution to the country's economy, the workforce trained through non-formal training also contribute to the trained individuals and the employers. The cost of this training is quite high. While only the Government is paying for this cost, it is not realistic to anticipate a continuity through expansion of this training. Hence, a structural planning should be done where the beneficiaries of education provided in the non-formal training institutions will contribute to the expenditures of the training. (b) A legal arrangement should be considered with regard to the accessibility of the workshops and laboratories of the equipped project schools to the individuals being trained and even to the industry. Furthermore, it is essential to have rapid changes in technology injected into the business community and naturally to amend the educational programs to respond to these changes. Through regular follow-up studies and developments, vocational and technical research and 2 development centers such as METARGEM and NFTI should be provided with a legal structure that will enable a preferred implementation in terms of finance and personnel policy. (c) Despite these deficiencies, due to the delays in the project activities, the General Directorate of Apprenticeship and Non-Formal Training and other General Directorates should strengthen their project follow-up sections with qualified personnel and focus on follow-up, orientation and inservice training activities. This point will have positive impact on the project outputs. C. COMPLIANCE WITH LOAN COVENANTS 9. Under the Non-Formal Vocational Training Project Loan Agreement published in the Official Gazette dated 21 March 1987, the Government and the Ministry of National Education committed to the followings: 10. Section 2.02 (a) of the Loan Agreement reads as follows: "The amount of the Loan may be withdrawn, from the Loan Account in accordance with the provisions of Schedule 1 to this Agreement, as such Schedule may be amended from time to time by agreement between the Borrower and the Bank, for expenditures made (or, if the Bank shall so agree, to be made) in respect of the reasonable cost of goods and services required for the Project described in Schedule 2 to this Agreement and to be financed out of the proceeds of the Loan." In accordance with the above paragraph, amendments have been made within the allocated budget in the categories stated in Amendments in Project Loan (line 23), as a result of the mutual agreement held on 18 August 1995, in view of the changing needs and deviations in the project implementation schedule. 11. Section 2.02 (b) of the Loan Agreement reads as follows: "The Borrower shall, for the purposes of the Project, open and maintain in dollars a special account in its Central Bank on terms and conditions satisfactory to the Bank. Deposits into, and payments out of, the Special Account shall be made in accordance with the provisions of Schedule 6 to this Agreement." According to this paragraph, a Special Account in the Central Bank was opened and, as stated above, this commitment was fulfilled by making the payments in conformity with the categories. 12. Section 2.03 of the Loan Agreement reads as follows: "The Closing Date shall be December 31, 1995 or such later date as the Bank shall establish. The Bank shall promptly notify the Borrower of such late date." The project was closed on December 31, 1995. 13. Section 3.01 (a) of the Loan Agreement reads as follows: "The Borrower declares its commitment to the objectives of the Project as set forth in Schedule 2 to this Agreement, and, to this end, shall carry out the Project through the Ministry with due diligence and efficiency and in conformity with appropriate educational, financial, economic and administrative practices, and shall provide, promptly as needed, the funds, facilities, services and other resources required for the Project." The Ministry, through the use of existing resources and facilities, has attempted to provide the provisions stated in this paragraph. The Turkish Government contribution to this project was realized as US$ 17.7 million, though planned in SAR as US$ 12.6 million. A detailed-information is presented in Annex-8. 14. Section 3.01 (b) of the Loan Agreement reads as follows: "Without limitation upon the provisions of paragraph (a) of this Section, the Borrower shall carry out the Project in accordance with the Implementation Program set forth in Schedule 5 to this Agreement, as such Schedule may be amended from time to time by agreement between the Borrower and the Bank." The provisions set forth in Schedule 5 have been, partly, provided. Quarterly testing and evaluation reports to be presented to the Executive Committee and to the Bank were prepared twice yearly. 15. Section 3.02 of the Loan Agreement reads as follows: "Except as the Bank shall otherwise agree, procurement of the goods and consultants' services required for the Project and to be financed out of the proceeds of the Loan shall be governed by the provisions of Schedule 4 to this Agreement." The activities with regard to this paragraph could only be realized with delays. Consultancy services were procured via contract-signing with SERCO Int. (UK- extended once), British Council, Mother-Child Education Foundation, International Language Center (UDM) and English Fast (Yeni Ufuklar). 16. ' Section 4.01 (a.b.c) of the Loan Agreement reads as follows: "(a) The Borrower shall maintain or cause to be maintained records and accounts adequate to reflect in accordance with sound accounting practices the operations, resources and expenditures in respect of the Project of the departments or agencies of the Borrower responsible for carrying out the Project or any part thereof. (b) The Borrower shall; (i) have the accounts referred to in paragraph (a) of this Section and the Specials Account for each fiscal year audited, in accordance with appropriate auditing principles consistently applied, by independent auditors acceptable to the Bank; (ii) furnish to the Bank as soon as available, but in any case not later than six months after the end of each such year, a certified copy of the report of such audit by said authors, of such scope and in such detail as the Bank shall have reasonably requested; and (iii) furnish to the Bank such other information concerning said accounts and the audit thereof and said records as the Bank shall from time to time reasonably request. (c) For all expenditures with respect to which withdrawals from the Loan Account were made on the basis of statements of expenditure, the Borrower shall; (i) maintain or cause to be maintained, in accordance with paragraph (a) of this Section, separate records and accounts reflecting such expenditures; (ii) retain, until at least one year after the Bank has received the audit report for the fiscal year in which the last withdrawal from the Loan Account was made, all records (contracts, orders, invoices, bills, receipts and other documents) evidencing such expenditures; (iii) enable the Bank's representatives to examine such records; and (iv) ensure that such separate accounts are included in the annual audit referred to in paragraph (b) of this Section and that the report thereof contains, in respect of such separate accounts, a separate opinion by said auditors as to whether the proceeds of the Loan withdrawn in respect of such expenditures were used for the purposes for which they were provided." 17. The commitments stated above were, mostly, fulfilled. Contributions from the local budget and subsidies to the schools, along with the amendments to the budget expenditure categories, since 1987, of the General Directorate of Apprenticeship and Non-Formal Training, were kept within the relevant General Directorate. Although, in fact, the yearly resources allocated from the budget is determined, the measurement of how much forwarded to which school could not be settled. The project expenditures were regularly audited by Prime Ministry Treasury Controllers' Council at the end of each fiscal year. and a copy of the prepared report was forwarded to the Bank. The EPIC procurement program was procured to enable the PCU Budget Section to keep the accounts more regularly and as required. As payments were made both through the Special Account in the Central Bank and through the World Bank directly, both account summaries were kept separately. D. EVALUATION OF MONE PERFORMANCE 18. The Government has accelerated the Turkish economy's opening to international competition by applying structural change programs in the economy. The priority given to improvement in new technological capacity and to the development of small scale industry to encourage export comes along with the investments concerning the provision of skillful, trained and a qualified workforce. Within this framework, the Ministry of National Education started the implementation of Industrial Schools Project in 1985 to increase the qualified workforce demand needed by the developing industrial sector through formal education. Similarly, with the preparatory activities of Non-Formal Vocational Training Project, an aim was to increase the qualified workforce supply by developing vocational education for adults through non-formal education. From this perspective, it can be considered the project has been designed in accordance with parallel to the development policy of the country. However, there has been an optimistic, but not realistic, viewpoint which reveals a moderate adaptation of the existing capacity and resources (personnel and physical) to the project objectives. 19. As foreseen in the project planning, a Project Management Council was established consisting of four General Directorates affiliated to the Deputy Undersecretary responsible for vocational and technical education for an efficient implementation of the project. A great effort was made to allow project implementation under the administration of this Council. However, the changes in the management of the Ministry have affected the structure and implementation of the Council and consequently several delays have been experienced in the project related activities. 20. On the other hand, within the framework of the Ministry's personnel policy, assignment of the PCU personnel through Ministry Approvals on a temporary basis have prevented these individuals taking certain responsibilities. However, the Unit established an organization within its own, determined the terms of reference and designated its staff according to these terms of reference after 1992. 21. To ensure efficient use of the equipment purchased under the project, a Project Follow-up Section was established however its studies could not be productive. The basic reason for this is the PCU does not have its own budget which creates several problems in the designation of its staff to visit the project schools. The insufficient numbers of project follow-up staff of the four General Directorates lengthened the period of solving problems directed by the project schools. 22. During the project implementation, it was foreseen that the implementation capacity of the Ministry would be supported by consultancy services and fellowship training under Technical Assistance Services. However, the Ministry could not effectively benefit from these services due to the international features of the provided consultants lacking Turkish language and insufficient foreign language proficiency of the Ministry staff. It has also been determined that overseas fellowship training could not be profitable as anticipated by reason of the unsatisfactory foreign language level and employment of trained personnel in inappropriate positions. 23. Few qualified personnel employed by the PCU made great effort and accomplished the timely realization of procurement activities which has constituted the major part of the project. Inservice training was provided to ensure an efficient use of the purchased equipment for training purposes. However, it is necessary to conduct more comprehensive inservice training activities for increasing positive project outputs. It can be stated that the MONE performed "satisfactorily" within the existing structure. E. EVALUATION OF BANK PERFORMANCE 24. The major objective of the Non-Formal Vocational Training Project (NVTP) is, supporting the Apprenticeship and Vocational Training Law passed in 1986, to advance the employment of the youth and adults out-of-school and/or establishing their own business. Education has been the most efficient process in social development due to its impact on the individuals future life and on the social structure. The proposed project has aimed at contributing to the provision of a workforce need which prevents the economic development of the industry, to improving employment opportunities and to disadvantaged individuals' (unemployed adults and drop-outs) generating their incomes. Furthermore, the project includes such moderate objectives as training of qualified manpower for increasing the profitability of the industry and to fill the gap experienced in non-formal vocational training field. 25. Augmentation of new functions and institutions to a system has always been a painful process. Even to reach an agreement on the amendments in the system is a great step. Despite, regarding the system development aspect of the project (especially in Skill Testing and Certification, Enterpreneurship and Learning Resources Development fields), the resources and the general strategy foreseen by the project to change the structure of non-formal training institutions has been unsatisfactory. The Bank has performed a satisfactory monitoring and counselling task, as well as a fair staff continuity, for the effective implementation of the project. In view of the overall performance of the project, preparation and supervision activities can be considered as "satisfactory". F. ASSESSMENT OF OUTCOME 26. The Non-Formal Vocational Training Project have achieved the objective regarding the augmentation and improvement of physical capacity, but not the objective related to the institutional development, As of 1994-1995 school year, a total of 2,100 trainees from the centers equipped under the project were provided with a qualified educational environment in, nearly, 20 different fields. The project was implemented in accordance with its original design. The Mother-Child Education Program foreseen in 1994, resulted in a total of 1751 mother-child pairs were trained in 18 provinces. Thus, another non-formal training activity with concentration in early child training was included in the project, one which also has a positive impact on child development aside from vocational training. Furthermore, with the METGE project of the General Directorate of Girls' Technical Education supported under the project in 1995, the quality and scope of the non-formal training activities conducted by this Directorate were achieved. On the other hand, the computer labs equipped under the project enabled the quality improvement of the vocational courses provided in PTCs and Commerce VHSs. The overseas and in-country training provided the Ministry staff, teachers and school administrators with knowledge and skills on various applications in their specialization areas. It has been observed that some of the trained individuals transfered their knowledge and skills to their units. 27. According to the first TA with SERCO, only 74 of the 143 teachers who has taken fellowship training abroad are asigned in the project schools. Some teachers have been resigned due to economical reasons whereas some have been transferred to YOK (Higher Education Council). The circumstances drawn above has decrease the project productivity. Related situation is given in table 12. TABLE : 12 POSITION NUMBER OF % . .TEACHER . .. 5 Workinq at the Project Schools 74 52 No okn at the Project Schools 23 16 Working at YOK 13 9 Working at Varios Ministry Unit 31 22 Resigded 2 1 Total 143 100 28. Within the scope of the project student follow-up programme has not been applied. However in the pilot researches of METARGEM applied in 1995 the related situation is seen: %47 of the graduates of CTTC are occupied, % 53 of the graduates are not occupied in any kind of occupation whereas %17 of them are preparing for University Entering Exam, % 37 of the graduates of Industrial Vocational Schools occupied, % 27 are studying in universities and %17 of the graduate are occupied in a family business whereas %16 of them are preparing for University Entering Exam (METARGEM 1995, Publication No 2,3). 29. The reaserch made under METGE Project has shown that; %41 of the graduates of Girls Vocational Schools are occupied in a job, % 49 of the graduates are using their skills at home for income generationare and %10 of them are preparing for University Entering Exam,(METGE,1995). In the apprenticeship system, apprenticeships have to be employed by employers, so they can be able to follow aprenticeships training Center, apart from special conditions of the graduates (illness, occupied in another work in better conditions ect.) %96 of them are occupied either in orjinal job or in tehey ar settig up their business. In public training systems % 50 of the fellowships are occupied in any kind of jobs and are successful and they aim to develop their skills attending to special programs like computer courses ect. (GDANE, 1995) This situation shows that though the numoer of the graduates from Non-Formal Training System is more than the number of the graduates from Formal Vocational Education Systems the desired goals are not achieved. Despite the fact that this situation is caused by several factors, one of the important reason of it is that the school-industry-suitable programs triangle has not been successfully built The increasing occupation in Non-Formal Education System is due to the best application of this triangle. 30. A major part of the project-related procurement activities were executed in the final years of the project. Therefore, full-capacity utilization of the purchased equipment and more beneficiaries' making use of the equipment (e.g. weekends and evening training) will influence the project outputs positively. To enable the MONE to continue the systematic upgrading of programs in accordance with the needs of industry, a better coordination and task share between NFTI and METARGEM, along with periodical and regular inservice training activities, will contribute to the success of the project. Taking into consideration of the above mentioned activities, the project outputs can be considered as "satisfactory" ADOPTION OF PROJECT OPERATIONAL PHASE A. DESCRIPTION OF OPERATIONAL PHASE 31. The Projects Coordination Unit of the MONE continues to perform its tasks. Project Coordinator and Follow-Up Section, in coordination with the relevant General Directorates, will ensure the arrangements for inservice training activities and the effective use of the procured equipment and materials for training purposes. For a more consistent implementation of these activities, the General Directorates will, annually, allocate an adequate portion from their budget. 32. On the other hand, the activities with regard to the task share and coordination between NFTI and METARGEM are on-going, which will enable the upgrading of the programs and assessment of project efficiency. These units will effectively work on these activities. B. PERFORMANCE INDICATORS TO MONITOR OPERATION AND DEVELOPMENT IMPACT 33. As stated in previous sections of this report. to increase the project outputs positively, the PCU and the Project Sections of the relevant General Directorates will work together to remedy the insufficiency of the number of teachers in project- related vocational fields. 34. Within the above-mentioned coordination, directly or in collaboration with the Inservice Training Department, inservice training needs will be responded through the use of project schools equipped under the project as inservice training centers. The utilization of the teachers trained in and out of the country as monitor teachers will increase the efficiency of the inservice training activities. 35. It is necessary for the relevant General Directorates to allocate the necessary budget for the realization of above-mentioned activities and for the provision of additional instructional materials that may be needed by the schools. 36. METARGEM has started a research on training needs assessment and evaluation of the efficient use of computer labs equipped under the project. Similarly, for an objective evaluation of the other project outputs, NFTI and METARGEM will, in coordination, monitor the graduates and evaluate the observations of the employers. The Project Sections of the relevant General Directorates should be strengthened both for the realization of these activities and for a regular monitoring of the other activities. IMAGING Report No.: 16528 Type: ICR