Turkey - Employment and Training Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No: 22427 IMPLEMENTATION COMPLETION REPORT (CPL-35410; SCL-3541A; SCPD-3541S) ONA LOAN IN THE AMOUNT OF US$ 67.0 MILLION EQUIVALENT TO THE REPUBLIC OF TURKEY FOR AN EMPLOYMENT AND TRAINING PROJECT June 27, 2001 Human Development Unit Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENTS (Exchange Rate Effective 06/20/2001) Currency Unit = Turkish Lira (TL) 1 Lira = US$ 0.00000081 US$ 1 = 1,240,000 Lira FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS ACEV - Mother and Child Education Foundation JIT - Job Insertion Training ALP - Active Labor Program LMI - Labor Market Information AR - Accounts Receivables MLSS - Ministry of Labor and Social Security ATHB - Research and Technical Services Unit MONE - Ministry of National Education BIBB - Bundesinstitut ftir Berufsbildung MSA - Management Services Agreement CEM - Country Economic Memorandum MSKIATHB - Occupational Standards CIAC - Career Information Advisory Committee Commission/Research and Technical cIC Career Information Center Services Unit CIDS - Career Information Delivery System NOSI - National Occupational Standards Institute CIU - Career Information Unit PDO - Project Development Objectives DGSPW - Directorate on the Status and PCU - Project Coordination Unit Problems of Women QSCT - Quick Start Customized Training DTA - Document Tracking and Archiving SAR - Staff Appraisal Report ETP - Employment and Training Project SBS - Siemens Business Systems FA - Fixed Assets SIS - State Institute of Statistics HLS - Household Labor Force Survey SSK - Social Security Agency GATB - General Aptitude Test Battery TA - Technical Assistance HR - Human Resources TCEP - Temporary Community Employment IIBK - Turkish Employment Agency Program ILO - International Labor Organization UBC - Unemployment Benefit Calculation ISCO - International Standards of and Compensation Occupational Classification System UNDP/UNOPS - United Nations Development Program ISKUR - Turkish Employment Organization Office for Project Service JAR - Job Application and Referral US DOL - United States Department of Labor JCC - Job and Career Counseling WALD - World Academy of Local Democracy Vice President: Johannes F. Linn Country Director: Ajay Chhibber Sector Manager: Michal J. Rutkowski Task Manager: Ferda Sahmali  FOR OFFICIAL USE ONLY TURKEY Employment and Training Project TABLE OF CONTENTS Page No. 1. Project Data I 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 3 5. Major Factors Affecting Implementation and Outcome 14 6. Sustainability 15 7. Bank and Borrower Performance 16 8. Lessons Learned 17 9. Partner Comments 19 10. Additional Information 28 Annex 1. Key Performance Indicators/Log Frame Matrix 29 Annex 2. Project Costs and Financing 35 Annex 3. Economic Costs and Benefits 37 Annex 4. Bank Inputs 38 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 39 Annex 6. Ratings of Bank and Borrower Performance 40 Annex 7. List of Supporting Documents 41 Map This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.   Project ID: P009064 Project Name: Employment & Training Team Leader: Ferda Sahmali TL Unit: ECSHD ICR Type: Core ICR Report Date: June 22, 2001 1. Project Data Name: Employment & Training Project L/C/ITF CPL-35410; SCL- Number: 3541A; SCPD-3541S Country/Department: TURKEY/Human Development Unit Region: Europe and Central Asia Region Sector/subsector: SE - Labor Markets & Employment KEY DATES Original Revised/Actual PCD: 01/11/1991 Effective: 05/26/1993 05/26/1993 Appraisal: 11/25/1991 MTR: 03/28/1996 03/28/1996 Approval: 12/10/1992 Closing: 12/31/1998 i2/31/2000 Borrwerfrimpletnewing Agency: REPUBLIC OF TURKEY/MLSS/IIBK (fSKUTR) Other Partners: MSK/ATHB, SIS, DGSPW STAFF Current At Appraisal Vice President: Johannes F. Linn Wilfried Thalwitz Country Manager: Ajay Chhibber Michael Wiehen Sector Manager: Michal J. Rutkowski Ralph Harbison Team Leader at ICR: Ferda Sahmali Michael Mertaugh ICR Primary Author: Arvo Kuddo 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainabilitv: L Institutional Development Intpact: M Bank Performance: S Borrower Performance: S QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: Yes  3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The Employment and Training Project (ETP) has four development objectives: 1) to improve allocative efficiency in the labor market through the diversification of employment services. This would be achieved by the development and phased implementation of a strategy for provision of enhanced public employment services and facilitating development of private providers; 2) to improve the absorption of unemployed, unskilled workers in productive employment. The project would finance: a) the generalization of the pilot program of job-insertion training developed by IIBK, and b) the strengthening of IIBK's employment services, including the introduction of employment counseling for job seekers and screening of job seekers, the modernization of job placement practices, and improved management of employment services; 3) to improve the efficiency of labor market decisions by providing better information on employment prospects and training requirements by occupation, and on the skills of job seekers. This development objective would be achieved by: a) developing and introducing career counseling services in labor offices and schools for students and members of the labor force; b) developing employer-relevant occupational standards and certification; and c) producing and disseminating information from various sources on labor market trends. 4) to promote women's productive employment. This objective would be achieved by supporting: a) deliberate actions to meet the needs of women's employment under the other project activities; and b) a program of research under the Directorate on the Status and Problems of Women (DSPW) to clarify the nature of the constraints on women's employment. 3.2 Revised Objective: Project objectives were not revised. 3.3 Original Conmponenits: ETP comprises 6 components: (i) Job-Insertion Training: to support a generalization of the pilot program in job-insertion training. The component consists of two major subcomponents: (a) the financing of a Training Fund to support job-insertion and self-employment training; and (b) the provision of technical assistance to support development of the Fund and related project activities. (ii) Diversification of Employment Services: to provide for the development and phased implementation of a plan to diversify employment services through the encouragement of appropriately regulated private employment services; (iii) Employment and Counseling Services: to improve IIBK's basic job placement services, and to incorporate employment counseling and applicant screening as an integral part of IIBK employment services. The component comprised the following subcomponents: (a) development to improved employment services; (b) development of counseling services; (c) implementation of improved employment and counseling services; (d) automation of employment services; and (e) Project coordination. -2-  (iv) Labor-Market Information: to help inform key decisions involved in employment and training - decisions by Government and by the private sector about appropriate training offerings, decisions by secondary students among alternative secondary school specializations, occupational decisions by members of the labor force wishing to change jobs or obtain new jobs, and decisions by collective bargaining agents as to appropriate levels of staffing and remuneration. The Component has three subcomponents: (a) improved coverage and reporting of SIS employment statistics, (b) creation of a labor market information database, and (c) labor market research and dissemination. (v) Occupational Standards and Certification: to improve labor force mobility, recruitment efficiency, and training program relevance by developing employer-relevant occupational skill certification. (vi) Women's Employment Promotion: to expand women's access to jobs offering better prospects for career growth, including jobs in traditionally male-dominated occupations. 3.4 Revised Comzpontents: Components were not revised except, due to the major earthquake of August 17, 1999, the Loan was amended so as to allocate uncommitted funds (US$ 10.760 million) to be used in the earthquake area for the Temporary Community Employment Program (TCEP) and Job Insertion Training (JIT) courses. During the project implementation, sub-components Automation of Employment Services and Project Coordination Unit of Component III were specified as separate components. 3.5 Quality at Entry: QAG review not available at the time of this project preparation. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Overall achievement of project objectives is judged to be satisfactory. The project was designed as a response to, and in support of a major economic reform program initiated by the Government of Turkey during the 1980's. The project was directed at refining Employment Services in Turkey to better address labor market policies and serve the labor force, and to finance development and broadening of employment services at IIBK. In general, most development objectives were successfully accomplished within the Project framework, and especially components on job insertion training, on employment and counseling, on labor market information, on occupational standards and certification, and on women's employment promotion achieved all or most of its goals. In particular, the job insertion program substantially exceeded its goals in terms of clients served; unit costs were about one-third of expenditures initially projected, and job placement rates were also very high. With regard to the Employment Services, there was a major upgrading of business plan, software, and staff of the IEBK (ISKUR). New operational manuals were prepared and extensive staff training provided. However, although necessary steps to diversify employment services (i.e. alter legislation to allow private employment services) were taken, the approval of legislation is pending in Parliament. However, the Government has conmnitted itself to completion of this activity. By Project completion, the achievement of the Project Development Objectives (PDO) is assessed as follows: First objective - to improve allocative efficiency by diversifying employment services. This was designed to be achieved through the encouragement of appropriately regulated private employment - 3 -  services. According to the Staff Appraisal Report (SAR), the Government had to "develop a plan for phased diversification of employment services in Turkey, and subsequently implement the plan.. .beginning not later than December 31, 1994." The Government issued a Decree in the Official Gazette on October 4, 2000, providing a regulatory framework for ISKUR, as well as for private employment agencies, paving the way for their licensure. The Decree, however, does not authorize private employment agencies to begin operations until September 1, 2002 and, in addition, supplementary legislation to address such areas, as penalties and financing, still needs to be enacted before licensing of private employment agencies can begin. Moreover, in 2000 the Constitutional Court annulled all decrees enacted by the Government and requested that such decrees be submitted to the Parliament and enacted within the nine month period. Therefore, although necessary regulatory acts were submitted, albeit later than initially envisag2d, this development objective to legally allow private employment services to operate was not achieved. However, the Government has committed itself to completion of the activity. (For details, see description of Component II). Second objective - improved absorption of unemployed, unskilled workers in productive employment. This development objective was met. (For details, see description of Components I and III). The number of people who have received training exceeded the target, and a new subcomponent on Temporary Community Employment Program was also completed. Third objective - improved efficiency of laboi-market decisions to be achieved by: a) developing and introducing career counseling services in labor offices and around 300 secondary schools; b) developing and adopting employer-relevant occupational standards and certification, and c) producing and disserninating information from various sources on labor-market trends. The development objective was partly achieved. The labor market information component and development of occupational standards and certification were completed. However, the Career Information Delivery System (CIDS), as initially envisaged, was not established since the proposals regarding CIDS were not in compliance with the technical specifications. Also the legislation for the establishment of the National Occupational Standards Institute (NOSI) has been submitted but was not enacted within the project period. The Government has proposed the draft law and committed itself to passage of the legislation. (For details, see description of Components III, VI and VII). The Ministry of Labor and Social Security has proposed that a department at the Ministry would take over the responsibility to supervise and carry out respective activities after the Project completion, although the Project anticipated the establishment of an autonomous institution NOSI under the auspices of the Ministry. The lack of legislation may adversely affect the sustainability of the work carried out under this Component. Fourth objective - women's productive employment that would be promoted by: a) deliberate actions to meet the needs of women's employment under the other project activities, and b) conduct of research and dissemination of research findings on how to alleviate the constraints on women's employment. This objective was achieved. (For details, see Component VIII). The loan had a closing date of December 31, 2000, compared to the original closing date of December 31, 1998, after two extensions requested by the Client so as to attain the Project goals and objectives within the extension. Despite these extensions, the borrower requested and the Bank agreed to keep a category open beyond the closing date in order to make final payments on one particular contract. Accordingly, the final disbursement - 4 -  under the loan was made on May 2, 2001 and the remaining undisbursed balance of USD 2,035,224.49 under the Loan 3541A was cancelled effective May 2, 200L" 4.2 Outpiuts by comnponents: Component 1: JOB INSERTION TRAINING The objective of this component is to improve the absorption of unemployed people into productive employment. The component was implemented by the ISKUR Manpower Training and Rehabilitation Department. The TA contractor for this Component was SERCO Education Ltd. The principal outcomes of the Component are as follows: Job Insertion Training: The objective for the number of people to be trained indicated in the SAR was 73,200 for the five-year duration of the project. Since 1993, actually almost 84,600 people have received training in 4,574 training courses (including 77,200 people as of 31 December 1998), and 70 percent of the participants of the employment guaranteed vocational training courses were females. Relevant training support was provided by the experts from the Directorate General on Status and Problems of Women under the long term training program on the issues of promotion of women's employment in nontraditional occupations and prevention of gender discrimination. Training classes were targeted at the unskilled unemployed population within the age range of 15 and 35 years old. Local and National Employment Councils were established in order to arrange vocational training courses on demand basis throughout the country. The program provided training for participants who were guaranteed a job at the end of the training program. The work places were monitored three months after the completion of training. Although the target for placement rates was 60 percent, the average job placement rate (those successfully completing training courses) was much higher: 94 percent in enterprise on-the-job training. 89 percent in public training institutions, and 90 percent in private training institutions. The training program did not require that participants remain in their jobs for any specified length of time, and training vendors were not motivated to place participants in long-lasting, sustainable jobs. The ISKUR did not operate the training courses itself but collaborated with employers, private training organizations and semi-public training centers in organizing training courses. Around 37 percent of trainees received training at enterprises, 31 percent in private training institutions, and 14 percent in public training institutions. Almost 16,000 individuals participated in Self-Employment Training Program (with no guarantees of employment after training). In collaboration with training institutions and employers, ISKUR reached and even exceeded target objectives on training. Considering very high placement rates of trainees and much lower costs than initially anticipated, job insertion training proved to be cost effective. However, due to cost limits established by IIBK, at the initial stage of project implementation, for some courses there was acute shortage of equipment and materials, and courses were too short in length to optimize the training effectiveness. As a supplementary activity, in 1998 the Turkish component of a quasi experimental Cross- Country Evaluation of the Active Labor Programs, including job-insertion training, was completed. The evaluation confirmed that the Bank funded training program had a positive impact on the number of jobs in 1996 (survey year), on the likelihood of self-employment and on the current earnings at the time of the survey. The training program was most effective for females, for those who were previously unemployed, and for the long-term unemployed. -5-  Temporary Community Employment Program (supplementary activity): Due to the major earthquake of August 17, 1999, the Loan was amended so as to allocate uncommitted funds (US$10.760 million) to be used in the earthquake area for the Temporary Community Employment Program (TCEP) and for Job Insertion Training (JIT) courses. By the end of 2000, 142 TCEP projects were approved. Out of total, 139 TCEP projects are located in five provinces in the earthquake zone (Bolu, Diizce, Kocaeli, Sakarya and Yalova) and three in Ordu, Mardin and Erzurum. A total of 4,582 people have participated in TCEP projects. As of April 3, 2001, US$ 4.942 million out of US$ 7.0 million allocated for TCEP projects was spent in the earthquake region, and US$ 32,400 out of US$ 1 million allocated was spent for 3 TCEP projects in other regions. An average cost equaled US$ 1,079 per participant of TCEP program which is a very high cost for public works programs. Overall, albeit the training funds were significantly reduced, the training objectives were exceeded; the scope of the component was also expanded, and the outcome is therefore assessed as satisfactory. Component 2: DIVERSIFICATION OF EMPLOYMENT SERVICES The objective of this component is to improve the allocative efficiency of the labor market through the diversification of employment services. The project financed the development and initial implementation of a strategy for the provision of employment services through a range of public and private providers. The TA contractor for this Component was ABT Associates Inc. A comprehensive study was conducted on Regulations of the Private Employment Agency, and a National Seminar on private employment services was held in 1994. The draft legislation on the Establishment and Duties of the Turkish Employment Agency (ISKUR) was forwarded to the Ministry of Labor on April 1998 and the Ministry forwarded it to the Prime Ministry to be submitted to the Council of Ministers. The draft legislation on the Establishment of the Turkish Employment Agency including the Regulations on the Private Employment Agencies was subsequently revised taking into consideration the Unemployment Insurance Law which was recently passed through the Parliament. The Government adopted to legalize the ISKUR Law via Governmental Decree which was published in the Official Gazette dated 4 October 2000 and permits licensing of private employment services. The Decree, however, does not authorize private employment agencies to begin their activities until September 1, 2002 and additional legislation to address such areas as penalties and financing must also be enacted before licensing of private employment agencies can begin. Recently, the Constitutional Court annulled all the decrees enacted by the Government and requested that all such decrees be submitted to the Parliament and enacted within nine month period. The objective to allow private services to operate was not accomplished within the Project framework although the Government has made necessary steps in submitting legislation to the Parliament. On May 10, 2001, the Bank has received a letter from the Deputy Director General of ISKUR confirming that the draft law was submitted to the Parliament and is expected to be enacted as of September 1, 2001. (See Borrower's Contribution). Enactment of principal labor legislation on the issue was too ambitious task to be achieved even within such a long period of project implementation. Therefore, the outcome of this component is assessed as unsatisfactory. -6 -  Component 3: EMPLOYMENT AND COUNSELING SERVICES The objective of this component is to expand the breadth of labor exchange services at local employment offices and to improve the efficiency and cost-effectiveness of employment services. In particular, the component objectives envisaged the provision of a) short-term, in-service training of up to eight week each on selected topics for 2,000 staff in local employment offices; b) long-term (total eight months), pre-service training for up to 1,000 IIBK staff; and c) development of training materials required for the training programs. This component is implemented by the ISKUR Manpower Training and Rehabilitation Department, the Department of Employment Services and the Personnel and Training Department. The TA contractor for this Component, until the termination of the contract in October 1997, was the US Department of Labor. Subsequently, services were provided by individual consultants. Long-term training and One-Year Master Program contracts were implemented by Ankara University. The following summarizes the outcomes of major sub-components. Development of Improved Employment Services: Unemployment insurance scheme was introduced in the context of the Government initiated 1999 social insurance reform. The Law authorizing Unemployment Insurance came into effect on August 25, 1999, and unemployment insurance (unemployment benefits) will be implemented in Turkey for the first time. Unemployment insurance premiums are being collected by the Social Security Agency (SSK). Unemployed people who qualify for benefits will begin to receive them in March 2002. The project assisted with refining the plan for implementing an unemployment benefit and financing system, integrated with the existing unemployment legislative studies in Turkey. ETP was thus an important element in a structural reform program on which the Bank advised through the CEM and supported with the Economic Reform Loan. The transition from IIBK to ISKUR is in progress. With the exception of the senior staff, employees from IIBK were transferred directly over to ISKUR. Operations Manual and Legislative Manual were prepared, and these manuals are now being updated to take into account all the legislative changes made to date as well as the changes imposed by the transition from IIBK to ISKUR. In order to produce short term labor projections through the data collection by ISKUR, Labor Market Information Questionnaire, Establishment Information Form, Labor Force Schedule and Establishment Feedback Form have been developed. In addition, labor market bulletin focused on the promotion of the activities of ISKUR was developed. Development of Counseling Services: The principal objective of this subcomponent was to assist employment service clients and secondary school students (in selected pilot schools) to make a choice between different types of professions and choose related training, and to provide counseling services to job seekers. The Career Information Unit (CIU) and the Career Information Advisory Committee (CIAC) functioned well during the project implementation period. A total of 40 ISKUR Career Information Centers (CICs) were established and they are actively being used. In 1993-2000, more than 329,000 people have used the job and career counseling services and visited Career Counseling Centers. ISKUR's web site is also being updated and information on occupations, education and training being added. A total of 404 Occupational Files have been approved by the CIAC. Adaptation of the General Aptitude Test Battery (GATB) is in the process and significant work has been done in this area (e.g., GATB tests were applied to 2,997 people). Aptitude test batteries have been completed for 18 professions in 7 different categories, and test guidebooks were published. Career information sumrnaries of 155 professions requiring higher education and 100 intermediary professions have been prepared. After the establishment of Career Counseling Units, respective services were set up in 15 local offices which had the necessary facilities and trained staff. In-service training has been provided to the staff of - 7 -  Career Information Centers. Research on 51 education and training institutions was completed and forwarded to the Centers for use. A report on the adaptation of the Turkish Dictionary of Occupations to ISCO'88 (International Standards of Occupational Classification System) was prepared in cooperation with MSK/ATHB. The technical advisory committee was established and the technical specifications for the Career Information Delivery System (CIDS) were prepared. However, the procurement of the automated CIDS, which is the key aspect of the counseling component, was not realized since the proposals regarding CIDS were not in compliance with the technical specifications. Implementation of Improved Employment and Counseling Services: As envisaged under the Project, the refurbishment of ISKUR Soke Training Center with required supplies and equipment was accomplished. More than two thousand ISKUR employees have received Orientation Training and 962 selected staff received Foundation Training. Additional specialized training was provided to 360 selected employees of ISKUR. Long-term training was provided to 300 ISKUR employees with university degree through the contract with Ankara University. The ISKUR staff trainers participated in study tours to Ireland and Germany. On September 5, 2000, a protocol on the development of One-Year Master Program without thesis and a four-month certificate program concerning employment and counseling services, labor market information, human resources development and management was signed with Ankara University, Faculty of Political Science. These university programs are expected to start in September 2001. A training program on Job and Careex Counseling Services for 16 employees of ISKUR was carried out a. Ankara Career Information Center between September 25 and October 18, 2000. Computer training (one- to two-week courses) were provided to 145 employees focusing on system administration by the consultants recruited under the Automation Component. The implementation of this component has helped built national capacity to develop employment services' institutions, and to provide improved services to the unemployed. The outcome of this component is assessed as satisfactory. Component 4: AUTOMATION OF EMPLOYMENT SERVICES The objective of this component is to refine and implement automated management and service delivery throughout ISKUR in order to improve service delivery and cost effectiveness, and to support labor market, education and training program efficiency. Implementation of the Component is carried out by the Data Processing Department of ISKUR. Three consultants financed by the project have been contracted by the ISKUR to augment the agency expertise during the turnkey contract implementation phase. The main contractor is Siemens Business Systems (SBS), responsible for the turnkey development, implementation, and training the staff in the use of eleven application systems (Document Tracking and Archiving (DTA), Accounts Receivable (AR), Fixed Assets (FA), General Ledger (GL), Human Resources (HR), Payroll (PAS), Job and Career Counseling (JCC), Active Labor Program (ALP), Labor Market Information (LMI), Job Application and Referral (JAR), and Unemployment Benefit Calculation and Compensation (UBC)), and the delivery and installation of the hardware and telecommunications infrastructure to support countrywide use of the systems. Other TA contractor for this Component was Deloitte & Touche Tohmatsu LA Group Ltd. which completed very successfully the contract for detailed preparation of the ISKUR automation strategy and related activities for the project. The Contract with SBS was signed on 26 February 1998 for a duration of 24 months. However, due to the fact that the activities under this contract could not be completed, the contract was extended until 31 December 2000. The initial delays in the Automation Component were caused by the delays in Component 3. Employment and Counseling Services, where the US DOL was not able to deliver the input needed by Deloitte & Touche in the preparation of the General Requirements document. - 8 -  The delivery and installation of the hardware and telecommunications infrastructure to support countrywide use of the systems was successfully deployed. However, due to considerable delays in development of application software, deployment, end-user training and data migration, this turnkey contract was not completed in the project framework. With respect to software developments, 11 application systems were conditionally accepted by the end of December 2000. As confirmed in the letter signed by the Deputy Director General of ISKUR dated May 10, 2001, SBS and the ISKUR will continue to work diligently and allocate the necessary human resources to ensure that the system will be ready for use in September 2001. The deployment of automation component will be done along with the upgrading of the system to Windows 2000. In order to enable ISKUR and SBS to provide application software training, the Bank agreed to leave the relevant disbursement category of the Loan (Category 2 of Schedule 1 of the Loan Agreement as amended on September 28, 2000) open for a three month period. Training program at the cost of US$ 210,000 for 300 ISKUR staff was completed by March 31, 2001, which allows the Project to benefit from the US$5.8 million worth of investment made to hardware and software infrastructure. In summary, the outcome of the component is assessed as satisfactory. Component 5: PROJECT COORDINATION UNIT The objective of this component is to ensure timely implementation, coordination and monitoring of scheduled project activities across the public and private agencies involved in the project, as well as within IIBK (ISKUR). The Project Coordination Unit (PCU) played an important role in facilitating communications between the IEBK, numerous TA contractors and other implementing agencies of the project, and provided assistance in administrative matters, procurement and financial issues as laid down in the project documents. Information dissemination was done at all levels when required and timely progress reports were prepared. In 1998, the PCU organized a very successful international conference on "Active Labor Programs - Design and Impact". The PCU was staffed through a United Nations Development Program Office for Project Services (UNDP/UNOPS) and Management Services Agreement (MSA). The TA contract was signed with the UNDP/UNOPS on 16 July 1993 for a duration of 60 months. Parallel to Loan Amendments, which extended the project activities until the end of December 2000, the Bank reviewed the extension request and approved the MSA extension until December 31, 2000. The implementation of this Component has helped build institutional capacity of IEBK to provide improved services to the unemployed. Overall, the outcome of this Component is assessed as satisfactory. Component 6: LABOR MARKET INFORMATION The objective of the component is to create improved labor market information database and enhance labor market research and dissemination in order to strengthen the capacity of public and private institutions, and individual citizens to make informed decisions about appropriate employment and training offerings. The State Institute of Statistics (SIS) was responsible for the implementation of this component of the project, and International Labor Organization (ILO) provided technical assistance. -9-  The Component accomplished the following objectives: Labor Statistics: Surveys conducted within this Component filled the gap in labor and wage statistics, and their results have been used in preparing the 8th Five Year Development Plan. The implementation of the Employment and Earnings Surveys, including quarterly data collection and yearly dissemination, has been fully institutionalized. The first round of the Employment and Earnings Survey, designed on a biannual basis, was launched in October 1996. The Manufacturing Productivity Survey was launched in 1998, and the Agricultural Wage Survey was first time commenced in 1994. The Household Labor Force Survey (HLFS) has been expanded, and quarterly surveys have become operational as of February 2000. Also a supplementary module on education and training was added to the HLFS and conducted first time in October 1997. In 2000, the first full-scale Informal Sector Survey was launched. The speed of data collection and dissemnination is improving continuously as the new labor surveys become routine in the SIS. The production cycle for the 1998 survey was over two years, for the 1999 survey about one year, and for the 2000 survey about 8 months. It is expected that from now on these surveys would achieve the agreed target of six-month cycle. All employment related publications in the context of the project were published and are available for users. Labor market statistics are also available to researchers and to the general public via SIS web-site at http://www.die.gov.tr. Disseminated database is updated timely with the publication. Labor Market Information Database (LMI): The preparation of the LMI database is fully completed and data are being disseminated to the public and to the research commnunity. In addition to the Household Labor Force Survey data, the timely statistics are being made available within the agreed objective of three months after survey completion, with a possibility of further reduction to 45 days. Labor Market Informnation Research: The SIS has fully institutionalized the publication of labor research information. Four issues of the Annual Labor Market Bulletins were published: in 1995, 1996, 1997 and a combined volume for 1998 and 1999. The 2000/2001 issue is being prepared. Seven issues of the Labor Market Analysis were prepared. Twelve semninars were held and more than 60 research papers were prepared and presented by labor market experts. The SIS interaction and cooperation with the country's research community is very good. The project assisted SIS to gain from the experience of other countries on different aspects of data collection and analysis of labor market information. More than one hundred SIS staff members participated in one or another way in the activities of the project. Altogether 45 fellowships were organized for the staff to obtain external training. Additional Activities: (a) On-line Data Dissemination System project: The dissemination system analysis and database system design activities of this supplementary sub-component were completed. However, signing of the contract for the procurement of hardware and software infrastructure, and training took longer than anticipated. The SIS, after the World Bank's no-objection, signed the contract on October 18, 2000. However, Ashford International, the US company was not able to deliver the hardware and software to the SIS within the project framework; (b) Improvement in the Survey Address Frame: The SIS has made significant progress in achieving the objective of using GPS technology to improve the address frame used for households and establishments of the Great Bursa Municipality. The GPS data, based on photogrammetric maps, have been acquired and are now ready for use. The implementation of this component exceeded the scope of the original agreement with the World Bank, and its outcomes are assessed as highly satisfactory. - 10 -  Component 7: OCCUPATIONAL STANDARDS AND CERTIFICATION The objective of this component is to improve labor force mobility, recruitment efficiency, and training program relevance by developing employer-relevant occupational skill certification. This component was implemented by the tripartite Occupational Standards Commission (MSK) with the support of the Research and Technical Services Unit (ATHB). The technical assistance (TA) contract was signed with the Bundesinstitut fir Berufsbildung (BIBB) on December 5, 1994, for a duration of 60 months. The duration of the TA contract was extended on September 28, 1998 to increase the scope of services. The activities foreseen under this component were successfully completed by the TA contractor by June 30, 2000. Although the core objectives of the Occupational Standards and Certification Component were successfully completed, submission and approval of the legislation for the establishment of the National Occupational Standards Institute (NOSI) was not accomplished within the project framework. The SAR indicates that "the MSK will review the legal status of MSK and ATHB by September 1, 1994, and if MSK members concur the Government will support development of plans to formalize MSK as a fully or partially self-financing foundation or association by December 1, 1995." Although the draft law for the establishment of NOSI was sent to the MSK member organizations already in 1 997, and the draft law was submitted by the Ministry of Labor and Social Security to the Council of Ministers in April 1998, it has not been enacted yet. The lack of legislation may adversely affect sustainability of the work carried out under this component. The Component has the following main sub-components. 1) Occupational Standards: 250 occupational standards were developed and approved by the Occupational Standards Commission (MSK), and thus the TA contractor have accomplished the targeted 250 occupational standards as stipulated in the SAR. The ATHB has translated all 250 occupational standards into English. The updating of the Turkish Dictionary of Occupations for 250 occupations was completed by the MSK/BIBB within a framework of a format agreed between the ISKUR and MSK/BIBB. 2) Testing and Certification: Test item development for 250 occupations was completed. Test items (written and practical tests) were edited and entered into the database. 3) Occupational Standards Commission (MSK): ISKUR has forwarded the draft law on the establishment of the National Occupational Standards Institute to the Ministry of Labor and Social Security who has forwarded it to the Prime Ministry. Enactment of the Law is expected to be realized in 2001. The Ministry of Labor and Social Security has proposed that a department at the Ministry would take over the responsibility to supervise and carry out respective activities after the Project completion while the Project anticipated the establishment of an autonomous institution NOSI under the auspices of the Ministry. Delays in the establishment of a specialized occupational standards institution have already had a negative impact on the demand for respective services. 4) Documentation Center and PR Activities: The occupational standards database is being used to store, retrieve, process and disseminate occupational standards. Data entry for occupational standards, and data entry and editing activities for test items have been completed. All documents, materials and equipment of the Documentation Center were also handed over to ISKUR in July 2000. Overall, implementation of this Component provided a valuable contribution to the establishment of, for the first time in Turkey, occupational standards and occupational skill certification according to - 11 -  international standards. The Government has committed itself to completion of this activity, and the outcome is assessed as satisfactory. Component 8: WOMEN'S EMPLOYMENT PROMOTION The objective of this component is to expand women's access to jobs and better prospects for career growth - including jobs in traditionally male-dominated occupations. This Component was implemented by the Directorate General on Status and Problems of Women (DGSPW). The DGSPW is an affiliated body of the Prime Ministry through the State Ministry responsible for women's affairs. The Technical Assistance (TA) contract was signed with the British Council on February 28, 1994 for the duration of 48 months. The TA contract has been extended for one year under the new terms of reference agreed by the DGSPW, the British Council and the Bank. The scope of work under the new terms of reference anticipated to design a gender awareness training program and its implementation. The TA contract was completed in May 1999 and the final report was submitted and revised by the DGSPW. The following summarizes the status of the Component activities. 1) Dissemination and Promotion of Research Studies: The Project supported action-oriented research on female employment issues in Turkey. Sixteen research studies were planned and conducted. The research included topics, such as gender discrimination in the work place; urban unemployed women; evaluation of vocational training courses as contribution to women's employment; women in domestic services, and so on. The Policy Paper which was prepared based on the outcomes of research studies, and contained recommendations on Employment, Education, Institutionalization of Women's Organizations and on Public Opinion Building, provided input to the preparation of the 8th Five Year Development Plan. The Policy Paper was discussed in two Special Ad Hoc Parliamentary Committees on Labor Market and on Women Participation in the Society, and Policy recommendations were noted in the reports of these committees. Research reports have been published as books (1200 copies), and books were sent to the target groups. A documentary film was produced and is ready to be used at the moment. rn order to disseminate research findings, a two-day conference was conducted. 2) Documentation Center: The Documentation Center is continuing to expand, and at present contains 1,801 books, 172 reference books, 243 articles and around 15,000 newspaper clippings and 762 "gray materials" on women's issues. The Center has also a 1000-slide visual archive which is also open to the public. Two librarians were recruited to work in the Documentation Center through the funds provided by the UNDP (as the DGSPW could not hire librarians due to legal limitations and lack of funds). The Center is extensively used: 554 clients visited it between 1 June- 30 September 2000 alone. 3) The component was further extended and the Gender Awareness Training Program was added: 17he DGSPW in cooperation with the World Academy of Local Democracy (WALD) conducted gender awareness training programs with Great Municipality of Bursa, Osmangazi, Yyldyrym, Nilufer Municipalities and Antakya Municipality. Reports on the related training were submitted to the DGSPW for their review and comment. The training program was conducted by the Turkey Family Planning Fouxiaation. The DGSPW experts also carried out a training program in cooperation with Mother and Childi Education Foundation (ACEV). The DGSPW is also working to update Gender Training Materials with the Middle East Technical University and Women's Studies, Research and Implementation Center. Genider awareness programs were piloted in 3 agencies. The DGSPW has now its own web-site (http://www.kssgm.gov.tr). Aithough this component was successfully completed, the legal status of the DGSPW still creates nurner-jLs limitations in coordinating, financing and implementing various gender related activities. Despite its 10th anniversary on October 2000, the law establishing the DGSPW as an autonomous legal - 12 -  entity has not been passed yet. The DGSPW does not have enough personnel (the staff includes only 41 individuals) and has limited funds for their activities. This may have an impact on the sustainability of the Component. Overall, outcomes of the Component are assessed by the Bank as highly satisfactory. 4.3 Net Present Value/Economnic rate of return: Not applicable. 4.4 Finanzcial rate of retini: Not applicable. 4.5 Institutional development impact: Overall, the project was highly relevant to the prevailing labor market conditions, and within the project framework, helped to resolve some institutional problems in the Turkish labor market. Nevertheless, the institutional development impact of the programs carried out under ETP is assessed as modest. The ETP promoted the integration of the unskilled unemployed into productive employment, improved the efficiency of labor-market choices through improved information on employment prospects and training requirements, and on job-seekers' skills. Among the mrost principal institutional outcomes of the project was the establishment, for the firs! time in Turkey, of an unemployment insurance system although this was accomplished as a part of a broader social insurance reform strategy and was supported by other Bank activities in Turkey. Also development of 250 occupational standards and related activities is a significant accomplishment which would lead to improved productivity by promoting the movement of skills to their best uses in the economy. By reinforcing job placement capacity and providing job-insertion training, the project, through the absorption of unskilled workers in productive employment, helped to minimize the effects of worker dislocation. The project assisted in capacity building with respect to the Ministry of Labor and Social Security, IIBK (ISKUR) and other institutions dealing with labor market. The project provided an extensive training for the staff of national and provincial labor offices that has helped to improve the quality of service although numbers of IIBK staff trained and hired were not at the level envisioned at the project start. There was a major upgrading of IIBK business plan, services, and software. This included also development of wide-range career counseling services. In addition, IIIBK completed a special quasi-experimental design evaluation of active labor programs to determine the net impact of training programs, an activity that was not envisioned at the outset. The improved monitoring of labor market developments and labor-market information is providing a more objective basis for labor-market decisions by providing better information on employment prospects and training requirements by occupation, and on the skills of job seekers. Another significant output of this project, not envisaged at the outset, is the recognition of Turkey as a leader in the development of informal sector statistics in the world. Turkey is a founding member and an active participant of the Delhi Group on Informal Sector Statistics, a gathering of experts from national and international agencies. - 13 -  5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of governlment or implementing agency: The project was designed in early 1990s. Since then, the frequent changes in Government and management at IIBK (ISKUR) affected the project. And, although the six major components were maintained during implementation, there was some reallocation of project funds which the Government re-directed to the Temporary Community Employment Program, as a response to the earthquake in 1999. 5.2 Factors generally siubject to government control: The Project fell behind the original appraisal schedule due to the following factors: (a) difficult economic and political climate which lead to frequent changes of ruling Government parties and ministries, and the associated learning and adjustment periods for key policy makers both at the Ministry and IIBK (ISKUR); (b) overly optimistic appraisal schedule, especially with respect to enactment of the law on private employment services; (c) the need for cooperative action among numerous public-sector and private-sector agencies in order to ensure coherence of implementation among the various elements of the project; 'd) due to changes in project management and complex procurement, the information technology and automation component was significantly affected by late start. There were .six management changes in IIBK during the project life which interrupted project activities. However, the stability in the last 3 years of the project activities has meant quick recovery. Also restrictions were put in place in hiring new staff as compared to an agreement at the start of the project in the SAR. To address the risks, several advisory and policy-making bodies were established under the project, including the National Training Review Board and the Local Training Advisory Committees for the Job- Insertion Training Component; the Career Information Advisory Committee for the Employment and Counseling Services Component; the Labor-Market Information Advisory Board for the Labor Market Information Component, the Occupational Standards Commission for the Occupational Standards and Certification Component, and the Research Advisory Committee for the Women's Employment Promotion Component. There were delays in implementing sub-components under Component III due to slower reformation of the employment service, as envisioned in the SAR, and changes in ISKUR management. Also technical assistance contract with USA DOL was terminated due to a payment dispute (now resolved) which hindered the improvement of the efficiency and expansion of the services of I113K appropriate to Turkey's changing needs. 5.3 Factors generally subject to implementinzg agency control: The ETP was the first World Bank project that the IIBK would implement and coordinate. Therefore, the IIBK with its limited initial implementation capacity did not have any experience in project management which lead to early staffing and implementation delays, cost changes and inefficiencies in the usage of technical assistance. Some other activities, such as counseling services subcomponent under Component III, were negatively affected by the suspension and follow-up cancellation of the technical assistance contract by US DOL in 1997. Also the IIBK rnanagement was not sufficiently involved with the project design at earlier stage that had impact on the ownership of the project. Nevertheless, most activities which have been assigned to ISKUR in the project documents, were realized, and a very close working partnership - 14 -  was created with all the implementing agencies. Information dissemination was done when required and timely progress reports were prepared. The ISKUR Administrative Department was provided with training on World Bank procurement rules and regulations. 5.4 Costs and financing: Total project cost was estimated at US$107.4 million equivalent at appraisal, including the Bank loan at US$67.0 million equivalent. As of December 21, 2000, the amount of US$14.557 million equivalent from the Loan was canceled. Savings were generated mostly in Job Insertion Training Component. Training courses were shorter than initially planned and were focused on practical skills. Average unit costs per placement per trainee in enterprise on-job-training were US$158, in public training institutions, US$166, and in private training institutions, US$136, which is much below the target cost anticipated in the SAR (US$520). As a result, instead of US$41.6 million anticipated for the Job- Insertion Training Fund in the SAR, US$12.5 million were actually spent. However, the actual number of clients served exceeded initial objectives. Due to Government austere fiscal policies, actual costs of several other activities were also lower than initially anticipated. 6. Sustainability 6.1 Rationale for sustainability r ating: 0veral1 the sustainability of most project components is considered as likely, and the project will have a long-run impact on the development of labor market institutions in the country. The Ministry of Labor and Social Security is committed to project sustainability as evidenced by its agreement to a follow-up Privatization and Social Support Project which will include a labor redeployment program aimed at cushioning the social impact of labor displacement during privatization and economic reform to be implemented by ISKUR and other agencies. Political commitment in support of the reform of labor market institutions is likely to remain high at all levels of labor program administration. The continuity of the ISKUR staff and training are of the greatest importance for the project sustainability. Although the project has assisted to establish an infrastructure for job-insertion training and respective manpower training activities, austere fiscal policies, including inadequate local funding of active labor measures and limits to expand ISKUR staff may hamper sustainability of active labor market programs envisaged in Component I. ISKUR regional branch offices are heavily overloaded with work. During the implementation period of the project, ISKUR aimed to hire 500 new staff members but only 45 were actually hired. After the completion of the project, most branch offices would be providing mainly job placement services. Since the job and career counseling services and Career Counseling Centers are actively used by the general public, ISKUR is planning to increase the number of supporting staff and to provide them with academic training to ensure the active involvement of provincial office managers in counseling services. With respect to Automation Component, the ISKUR is determined to have a sustainable information system which would provide quality services to users. The Information System will be carried out by the Data Processing Department which has a staff of 27 people. Although the Ministry of Labor and Social Security has proposed that a department at the Ministry would take over the responsibility to supervise and carry out activities on the Occupational Standards and Certification Component, absence of establishment law may affect sustainability of some activities successfully implemented under this component. The validity period for the National Protocol signed by the Ministry of Labor and Social Security, Occupational Standards Commission, MEKSA and ISKUR on September 17, 1992 to oversee the establishment of the NOSI expired on December 31, 2000. The draft law on the establishment of Turkish Occupational Standards Institute has been forwarded to the Prime Ministry, and the draft is expected to be submitted to the Parliament shortly. Also, as is stated in the letter from the Deputy General Director of ISKUR dated May 10, 2001, the agency is expecting that - 15 -  the ISKUR Act, which also contains articles on the establishment of private employment services, be enacted by the Parliament as of September 1, 2001. 6.2 Trantsition arrangemnent to regular operations: None. 7. Bank and Borrower Performance Bank 7.1 Lending: The Bank properly identified the key areas of policy development in the field of employment and training services, such as legal framework, institutional capacity building, labor standards, and human development. The project was designed to enhance labor market institutions and support labor market programs. The objectives were consistent with the Bank Country Assistance Strategy. In retrospect, it appears that the ETP was ambitious in its goals and objectives, as they envisage the development task of several years. The change process necessary for modernizing IIBK appears to have been underestimated during project preparation, especially in view of unstable political environment and continual changes in IIBK (ISKUR) management. While the ambitions of the project were perhaps too high, vast majority of activities were successfully completed and accomplishments should not be underestimated. 7.2 Stupervision: Bank supervision was professional providing skill mix and continuity of the staff attached. Since 1993, 16 Bank supervision missions were conducted. The Bank correctly highlighted sector policy issues, properly addressed the serious constraints and paid attention to the need for training of staff of implementing agencies. There were no noticeable deviations from Bank policies and procedures on financial management, procurement and disbursement. The project benefited from the fact that key team members, including the task manager, essentially remained intact during the project implementation. In 1999, the project management was handed over to the staff of Turkey Country Office. 7.3 Overall Bank performance: Overall, Bank performance is assessed as satisfactory. The project has assisted in establishing new institutional framework for the implementation of labor market policies in the country. Borrower 7.4 Preparatiol1: In 1989 and 1990, the State Planning Organization, with Bank assistance, initiated a series of studies within the framework of Human Resources Development Project which provided Policy Documents of Employment and Human Resources Development Systems - Strategic Analysis and Action Plan. Ten background papers were commissioned. In March 1990, a synthesis report titled "Republic of Turkey: Human Resources and Employment Development Strategy" was presented and discussed in a national seminar in May 1990 which included a detailed analysis of the status of employment programs in Turkey. On the basis of the above mentioned strategy, the Bank and the Government of Turkey developed Employment and Training Project to finance development and broadening of employment services at IIBK. These reports allowed to identify key areas for labor policy design. However, the Government was committed to supporting primarily public employment services. - 16 -  7.5 Govenmnent implementation perfornmance: Borrower performance in implementation was adversely affected by high turnover of managerial staff in IIBK and other implementing agencies. Also not all the contractors of technical assistance completed their tasks on a timely basis and with high quality (i.e. the US DOL contract was terminated early). The socio-political environment of 1990s in Turkey was not very enabling for such a long term commitment in reforming labor sphere, particularly regarding institutional strengthening activities, since austerity fiscal measures restricted government expenditure on labor programs. 7.6 Implementinig Agenzcy: The performance of IIBK over the project cycle has been uneven. There were significant delays in starting with components, such as on occupational standards and certification, and automation of employment services. During the project implementation, there were problems with reporting and delivery of products as defined in the TA contracts which has resulted in slower than anticipated implementation of some components. Due to Government austerity measures, IIBK has been having continuing problems in obtaining the local funds necessary to match Bank financing, such as on job insertion training. Nevertheless, adjustment of the level of financing for each component of the project was accomplished, including an increase in the aggregate level for local shopping and local competitive bidding. During the final years of the project, the IIBK as the lead implementing agency significantly improved administering of this multi-agency project. 7.7 Overall Borrower performance: Overall, Borrower performance can be assessed as satisfactory. Most project components were accomplished although with some delay. On several components, outcomes exceeded initial expectations, and several new activities were added. However, a few principal activities were not finalized during the Project framework, including the approval of the law authorizing private employment agencies to begin operations, and the law on the establishment of NOSI. 8. Lessons Learned The main lessons learned from the project to improve Bank operations in Turkey and in other countries, many of which are related to the fact that this was one of the first projects of this type in the Bank, are the following: 1. Policy implementation: Policy implementation in a time of unstable coalition governments is difficult, particularly if implementation may involve legislative action. Two project activities, the licensing of private employment services and institutionalization of national occupational standards, included related loan covenants regarding institutionalization. Unfortunately, while the legislation is being processed, it was not passed by the end of the project. The passing of a law is thought to be unrealistic as it involves the political authority not under the control of the implementing agency. Passing the legislation before the project started was not an option, because the issues needed considerable research before this option was selected and related laws were drafted. Perhaps the best lesson learned is to be more up-front about the risk at the outset, so there are no surprises at the end, particularly in unstable political environments. 2. Adequate staff training: The accomplishment of project objectives can be successfully materialized by the staff that has relevant knowledge, experience and high motivation. The training of the staff of IIBK/ISKUR and other implementing agencies has proven to be an effective investment which helped to improve the staff performance and motivation of the personnel. This was confirmed with an - 17 -  active participation in all training programs. On the other hand, training courses should be designed for particular institution's requirements and restrictions in mind. 3. Adequate monitoring of output efficiency: One of the principal activities of the project was the provision of job insertion training. Regarding training while results were promising, little is known of the long-term impact on employment. The training certificates provided to the unemployed who successfully completed the training courses for self-employed were not solely sufficient enough to establish their own businesses. Data on the relevance of the training vis-a-vis training needs of employers and the quality of the training providers are lacking. There is a need for the development of a follow-up longer-term monitoring system. 4. Coordination of labor project activities: In this project, tripartite coordination and coordination involving representatives of the interest groups was crucial in especially determining the needs for job insertion traiTning, in developing occupational skill certification and in enhancing labor market information system. For example, the Labor Market Information Advisory Board was set up at the very early stage of the project, with representatives of the social partners and main governmental institutions involved in the production and use of labor market data. The Board accomplished to come to an agreement on standard statistical definitions on labor market indicators, to review and assess progress made in respect to each project activities, and guidance on further actions to improve the performance of the Project. SIS envisages its continuation beyond the life of this project. 5. Prioritization of implementation activities; There is a need for prioritization of implementation activities and deliverables. For example, prior to enforcement of massive training or counseling programs, strengthening of local labor office institutional capacity is needed. Also the establishment of a few pilot offices to facilitate field test procedures, as it was done in several cases, and enabling of cross- coordination between project components should be advisable to help ensure the feasibility of recommendations. 6. Improve Bank own activities and assistance: The Borrower has indicated that inconsistencies between the operations staff and the procurement department of the Bank have caused a lot of duplications in the project which could have been avoided. The Borrower also indicated that the Bank's documents which are related to the procurement of goods and services are not translated into the language of the country that implements the project. The English proficiency level of the employees working at administrative and financial units is not good enough to comprehend the documents. The staff that will be required in the project administration, implementation, procurement, disbursement and in financial issues should attend the seminars organized by the World Bank. Also UNDP implementation contracts should be much more focused on capacity building. 7. Expect delays in implementation of autonation component: Delays in implementation of especially Automation Component indicates that planning of a time-table should be made in a more realistic way taking into account the complexity of tasks and institutional capacity. Automation projects may take about twice as long as initially expected at the outset. Although in this project, the hardware was installed on a timely basis, software developments lagged behind. - 18 -  9. Partner Comments (a) Borrower/implementing agency: SUMMARY OF BORROWER'S IMPLEMENTATION COMPLETION REPORT This summary was prepared by ECSHD and is based on a Project Implementation Completion Report drafted by the Turkish implementing agencies of the ETP Project April, 2001 A. ASSESSMENT OF DEVELOPMENT OBJECTIVES AND DESIGN Original obiective: The objective of the ETP is to support the Government's reform program, by (a) improving allocative efficiency in the labor market through liberalized employment services; (b) improving the absorption of unemployed, unskilled workers in productive employment; (c) improving the efficiency of labor market decisions by providing better information on employment prospects and training requirements by occupation, and on the skills of job seekeis,-including secondary students who must choose between alternative education and training options with different employment implications, employers wishing to recruit employees with specific job skills, training providers who must plan the offering and content of training courses, and collective bargaining agents who must reach decisions on appropriate levels of staffing and remuneration, and (d) promoting women's productive employment. Revised obiective: None. Original Components: Originally, as indicated in the SAR, ETP comprised six components: (i) Job insertion training: to improve the absorption of unemployed people into productive employment. (ii) Diversification of employment services: to improve the allocative efficiency of the labor market through the diversification of employment services. (iii) Employment and counseling services: to expand the breadth of labor exchange services at local employment offices and to improve the efficiency and cost-effectiveness of employment services. (iv) Labor market information: to create improved labor market information database and enhance labor market research and dissemination in order to strengthen the capacity of public and private institutions, and individual citizens to make informed decisions about appropriate employment and training offerings. (v) Occupational standards and certification: to improve labor force mobility, recruitment efficiency, and training program relevance by developing employer-relevant occupational skill certification. - 19 -  (vi) Women's employment promotion: to expand women's access to jobs and better prospects for career growth - including jobs in traditionally male-dominated occupations. In order to achieve Project objectives, during the ETP implementation, Automation sub- component and Project Coordination sub-component of Component III "Employment and Counseling Services" were specified as autonomous components and are therefore analyzed here as separate components. B. ACHIEVEMENT OF OBJECTIVE AND OUTPUT Assessment of output: The outcome of the project is assessed by the principal implementing agency ISKUR as satisfactory. Output bv Components (I) Job Insertion Training Principal objectives of the Component were as follows: (a) Improve the operation of Local and National Employment Councils; (b) Develop annual training plans financed under the ETP; (c) Improve regulations related to the job insertion training courses financed under the project, also including the procedures of contracting with training institutions; (d) Develop pilot programs in order to support creation of employment; (e) Support self-employment; (f) Promote women's employment. Achievement of Objective and Outputs: The Component includes the following two elements: (a) Establishment of a Fund to support the Job Insertion Training; and (b) Provision of the required technical assistance to facilitate development of the Fund and support the relevant project activities. The Project aimed to provide vocational training to 73,200 unemployed people. Throughout the project, 4,574 manpower training courses were organized for 84,571 people, and 70 % of the participants were females. In accordance with the amendments made under the Project, a Temporary Community Employment Program (TCEM) was successfully implemented in the earthquake region. A total of 140 projects with the participation of 4,582 people were implemented in Kocaeli, Yalova, Duzce, Bolu and Sakarya regions. Although the Quick Start Customized Training Program could not be promoted all over Turkey, the pilot applications (two in Eskisehir and one in Erzurum) were accomplished. Under the Job Insertion Training Component, Local Employment Councils were established with the participation of employee, employer and public representatives at the provincial level, and a National Employment Council with the same tripartite structure was established centrally. Annual Training Plans discussed and finalized by the Local Employment Councils were implemented following the approval of the National Employment Council. Relevant training support was provided by the experts from the General Directorate of Status and Problems of Women. - 20 -  (II) Diversification of Employment Services Principal objectives of this Component were as follows: (a) Assess the effectiveness of private employment services and related regulations; (b) Identify strategies for diversifying employment services; (c) Develop and implement a phased plan for diversification of employment services. Achievement of Objective and Outputs: Output is assessed as unsatisfactory due to the fact that the law to allow for the operation of private employment agencies has not been passed yet. Due to the fact that the time of implementation coincided with a period of political instability, the Law was not passed as it was foreseen in the project time schedule. The passing of a law is thought to be unrealistic as it involves the political authority more than the efforts of the Turkish Employment Organization. Nevertheless, study tours were organized for the Steering Committee members to Germany, Belgium, England and the Netherlands, and a report on private employment agencies was prepared. A national seminar was organized in November 1994 with the participation of representatives of the Government, employers, employees, and public and private agencies concerned with the Turkey's national development strategy. The decree with the force of Law related to the Establishment and Duties of the Turkish Employment Agency including the regulations on Private Employmeit 1Agencies became effective upon its publications in the Official Gazette on October 4, 2000 buL was annulled thereafter by the Constitutional Court. (III) Employment and Counseling Services Principal objectives of the Component were as follows: (a) Provide short-term training (8 weeks) to 2,000 ISKUR staff; (b) Provide eight-month long term training to 1,000 ISKUR staff (university graduates) (c) Develop and prepare training materials required for the training programs; (d) Establish a Career Information Unit within the ISKUR (e) Establish a Career Information Advisory Committee which will develop standardized career information resources to be used in developing career information; (f) Assist in selection, adaptation and development of a self assessment tool to be used at local offices as well as 300 selected pilot schools; (g) Develop and adapt printed materials, video materials and computer assisted information delivery systems to be used at local offices and 300 selected pilot schools; (h) Select, adapt or prepare interest and skills tests to be used at local offices and 300 selected pilot schools. Achievement of Objective and Outputs: As envisaged under the Project, the refurbishment of Soke Training Center with required supplies and equipment was accomplished. The following short-term training programs were implemented: a) two-week orientation training for 2,009 ISKUR staff; b) two-week foundation training for 962 ISKUR staff; c) two-week specialized training for 360 ISKUR staff. In addition, eight-month long term training was provided to 300 ISKUR staff - university graduates by Ankara University, Faculty of Political Sciences. Involvement in long-term training courses was lower than expected due to the heavy work load of the staff and insufficient number of university graduate personnel. Also the eight-week short term on the job training courses on specific subjects were delivered to the ISKUR staff by ISKUR staff trainers under the coordination of Personnel and Training Department. The protocol facilitating the establishment - 21 -  of a four-month certificate and a one-year master program was signed with Ankara University, Faculty of Political Sciences on September 5, 2000. The program will be accepting students during the 2001-2001 academic year. The Law on Unemployment Insurance was passed by the Parliament and became effective as of June 1, 2000. The payment of unemployment benefits will start as of March 2002. Operational Manual and Legislative Manual were prepared. Labor Market Bulletin focusing on the promotion of the activities of ISKUR was developed. With reference to the collection of labor market data, "Establishment Information Form", "Labor Force Schedule" and "Establishment Feedback Form" were developed and tested in two provinces. The Labor Market Information Questionnaire was tested in five provinces. An Allocation Manual was developed to assist implementation. With reference to the development of a model facilitating projections of unemployment at local levels, Household Labor Force Survey of the SIS and Job Request Forms were applied to 253 ISKUR applicants who are registered at the ISKUR. A training of 22 ISKUR staff was provided on transferring the data to the computer, statistical analysis of the data collected, preparation of graphics, and so on. A study was conducted in four provinces so as to calculate labor force and unemployment based on the data obtained from ISKUR. Under the Project, 34 Career Information Centers (CIS) have been established. CIAC has approved 404 career information files, and these files have been distributed to all local offices with Career Information Centers. In-service training has been provided to CIS staff. Research on 51 education and training institutions was completed and forwarded to the Centers for use. Career information summaries of 155 professions requiring higher education and 100 intermediary professions have been prepared. Career counseling services were set up in 15 local offices which had the necessary conditions and staff to provide the services concerned. In order to measure the skills required by certain professions in an objective manner, the adaptation of GATB, which was supplied by the US Department of Labor, was started in 1996, and a GATB Manual was prepared. During the selection of the Career Information Delivery System (CIDS), a technical advisory conmiittee was established and the technical specifications for the system were prepared. Also a respective study tour to the USA was organized. However, the proposals regarding CIDS presented by MDA in November 1998 and later on in February 2000 by SBS were found not to be complaint with the technical specifications and thus the system was rejected. The activities under counseling services sub-component were negatively affected for a year due to the suspension of the technical assistance contract on July 8, 1997 and unilateral cancellation of the contract by US DOL on October 8, 1997. GATB adaptation was especially affected and due to the closing down of the Measurement and Evaluation Center, the work done was stopped for a year. This has caused a delay of one year also in the adaptation of GATB process. (IV) Automation of Employment Services The objective of this component was to expand the automation of the employment and counseling services of ISKUR from the General Directorate to the local labor office level in order to improve service delivery, cost-effectiveness and to increase the efficiency of labor market education and training programs. Achievement of Objective and Outputs: Technical Contract which designates the activities, the products, materials and services provided under the project has been prepared by Deloitte & Touche. SBS, the project contractor, has presented the services to be provided in line with the technical contract including defining 11 required application systems and the functions of modules. The project starting point was to expand the systems set up beforehand in 5 Regional Offices and in General Directorate Data Processing Center to whole - 22 -  organization. In line with this, specific operational steps were developed considering the requirements in the technical contract. However, during the project, there have been increasing delays especially in software development and the component was not finalized even within the extension period of one year. This indicates that planning should be made in a more realistic way taking into consideration the current risks. Although there have been no deviations in hardware installation, due to the delay in suggested terms, the hardware itself was not sufficient to provide the anticipated help. (V) Project Coordination Unit The PCU component aimed to: a) Assist the implementing agencies in the operation procedures of the Project; b) Assist with the procurement activities to be conducted under the Project in accordance with the World Bank rules and regulations; c) Develop project reporting procedures and financial reports; and d) Disseminate project related information. Achtievement of Objective and Outputs: The PCU was established to ensure proper implementation of the project and provide administrative support to implementing agencies which were executing technical project components. The ETP was the first World Bank project that the Turkish Employment Organizations (ISKUTR) would implement and coordinate. Therefore, the ISKUR did not have any experience in World Bank project management. The PCU has provided assistance in administrative matters, procurement issues and financial issues as laid down in the Project documents. It has ensured the timely implementation of all activities within the limitations of the Project and has monitored the Project closely. The ISKUR Administrative Department was provided with training on World Bank procurement rules and regulations. The PCU worked very closely with the upper management of the ISKUR. The fact that there were 6 management changes during the life of the project has meant very hard work for the PCU and at points interruptions were lived. However, due to the stability in the last 3 years of the Project has meant quick recovery. In addition to its duties, the PCU has also taken additional responsibilities with the cancellation of the major US DOL technical assistance contract as well as other contracts, which have expired without the completion of all related activities. (VI) Labor Market Information The principal objective of this Component was to improve coverage and reporting of SIS employment statistics, create labor market research and dissemination in order to strengthen the ability and the capability of public and private institutions and individual citizens to make informed decisions about employment and training priorities. Achievement of Objective and Outputs: The following new surveys were conducted by the SIS: a) the first round of the Wage Structure Survey was launched in February 1995 (the next survey is scheduled for 2003 in line with the EU schedule); b) the first round of the Employment and Earnings survey was launched in October 1996. The survey has been applied twice a year since 1996. The second round of the 2000 application is currently in the field; c) the first survey referring to labor productivity in all manufacturing industrial establishments in the public sector and in the establishments in the private sector engaging 10 or more employees was launched in 1998, and the latest survey covered the third quarter of 2000 (results were published in December 2000); d) starting from 1994, agricultural wage survey in organized agricultural holdings in the - 23 -  public sector is now conducted annually; e) as a module of the household labor force survey, an education and training survey was conducted in October 1997; f) the full-scale Informal Sector Survey was launched in 2000. Also the household labor force survey was expanded, the sample size was increased from 15,000 households for the bi-annual survey to 23,000 households for the quarterly survey, and the quarterly design of the survey has become operational as of February 2000. As an extension of the project, a comprehensive address frame of households and establishments was prepared based on digital maps which are to be complemented with statistical information obtained from optical character reader forms filled as part of the preparatory work for the 2000 population census. The SIS has reduced the time lag between data collection and publication of final results of surveys from about 17 months to about 6 months. The results of the household labor force surveys have become available in public-use diskettes in Excel format. Four issues of the Annual Labour Market Bulletin have been published: in 1995, 1996, 1997 and a combined volume for 1998 and 1999. The 2000/2001 issue is being prepared. Seven issues of Labor Market Analyses were published. Moreover, since July 2000, SIS labour force statistics can be assessed through Internet via the LMI database at http://www.die.gov.tr. The Labor Market Information Advisory Board was set up at the very early stage of the Project, with representatives of the social partners and main government institutions. The main accomplishments of the Board have been: a) agreement on standard statistical definitions of wages, earnings and labour costs, employment, unemployment and underemployment; b) agreement on subject areas for labour market research and seminars; c) review and assessment of progress made in respect of each project activities, and guidance on further actions to be undertaken to improve the performance of the project. The SIS has appended several other activities to the project. In particular, an unemployment estimation model was developed to measure unemployment rates at the provincial level. The SIS on-line data dissemination system is being developed. Although Ashford International, the US firm failed to deliver computers on a timely basis, the SIS is planning to buy necessary hardware on their own. Supplementary activities also included the design of a geographically referenced address frame based on households and establishments covering Bursa Great Municipality. Respectively, 1:1000 scaled digital photogrammetric maps covering Bursa Metropolitan Area were provided. The SIS database is being gradually expanded to cover other statistical areas, in particular, wage statistics and consumer price indexes. A production database for Consumer Price Index is being developed covering data from 1994 onward, with 2000 as the base year for the index calculation. More than 100 staff members of SIS participated in one way or another in the activities of the project. In total 45 fellowships were organized for the staff to gain from the experience of other countries on different aspects of data collection and analysis of labor market information. Contributions of 85 national consultants and 55 international consultants were drawn to design and implement the activities of the project. Many SIS staff at regional offices have benefited from training. Between 1995 and 2000, twelve seminars on labor market research have been held and more than 60 papers prepared and presented. Another significant output of this project, not envisaged at the outset, is the recognition of Turkey as a leader in the development of informal sector statistics in the world. Turkey is a founding member and an active participant of the Delhi Group on Informal Sector Statistics, a gathering of experts from national and international agencies dedicated to the development of informal sector statistics. The Delhi Group is one of the so-called "city groups", established by and reporting to the United Nations Statistical Commission. SIS hosted the second meeting of the Group in 1998. - 24 -  (vii) Occupational Standards and Certification The major goal of this component is to increase the proportion of the labor force holding validated occupational skill certificates, thereby improving labor force mobility and productivity, and building safe bridges between employment and training. This component was developed to realize four basic objectives: (a) Develop occupational standards for 250 occupations determined by MSK; (b) Develop 250 test item banks for occupations, for which occupational standards were developed; (c) Define the general framework of Testing and Certification System adopted by business sector; (d) Establish National Occupational Standards Institution (NOSI). Achievement of Objective and Outputs: Technical activities involving the development of standards and test items banks were initiated in April 1995 and completed in June 2000. Activities for developing occupational standards for 250 occupations determined in line with the preferences of business sector were completed. For 250 occupations for which standards were developed, test item banks to be used in theoretical and practical skill testing were developed with the active support of teachers and specialists in the field concerned. test items were entered into database, thereby opportunity to implement future examinations with the support of electronic database was provided;. basic Principles Document concerning Testing and Certification System was prepared by ATHB, audited by MSK and finalized in accordance with evaluation and recommendations of the concerned parties. Research and Technical Services Unit (ATHB) was established for the implementation of technical operations. The draft Bill of NOSI, developed by MSK was submitted to the Prime Ministry but could not be submitted to the Turkish Grand National Assembly due to the interrupting general election and became out of date. Following the establishment of the new government, the draft Bill was reviewed and sent to the Ministry of Labor and Social Security. However, by the end of the project cycle, the draft has not been forwarded to the parliament. (viii) Women's Employment Promotion Women's Employment Project aims at expanding women's access to jobs offering, better prospects for career growth, including jobs in traditionally male-dominated occupations, and investigating obstacles to their participation in employment. Principal objectives of the Component were as follows: (a) Design the Research Fund; (b) Manage the Research Fund; (c) Support the Research Fund; (d) Develop awareness on gender concept and provide training programs to this end. Achievement of Objective and Outputs: At the early stage of the project implementation, a project group was established at the DGSPW with the participation of international and local consultants and counterparts. Also the Research Advisory Committee was established with the participation of public managers who concern themselves in female employment, of academicians whose study field is problems of women, and representatives of private sector and NGOs. The operations of the Research Fund were designed to accumulate concrete scientific data, which will provide support for the development of female employment policies, the basic fields for research were determined: a) discovering the unknown features of female employment; b) vocational training of women and the relation between training and employment; and c) discrimination in sectors against female employment and gender. Within this framework, 16 research projects were implemented, - 25 -  and 13 research reports have been published. Publishing process of the research papers has been carried out by the staff of DGSPW because of due date of the Technical Contract was May 1999. Documentation Center was established in 1996 so as to render service for research and studies on women and gender. The collection of the Center consists of 1801 books, 243 articles, 762 gray materials and 172 references. In order to disseminate research findings to the public opinion and relevant institutions, in 1997 a two-day conference was conducted. A documentary film called Working Women" was produced. A Visual Archive consisting of 1000 slides and a web-site of DGSPW (http://www.kssgm.gov.tr) have been created. A policy suggestions document was developed according to research findings which was put on the agenda of the Special Commissions of the 8th Five-Year Development Plan. The policy suggestions provided by the research findings has take a large place in the unpublished reports of Labor Market Special Commission and Special Commission for Woman Participation, which are among the special commissions of the 8th Five-Year Development Plan. Gender Training Program has been developed aimed at developing awareness to ensure the introduction of changes that provide gender equality in the plans and programs of public, private and non-governmental organizations. Gender Training Material used in this Program comprises of 55 interactive activities, and has been used by several institutions, such as the World Academy of Local Democracy, Turkish Family Planning Foundation, and so on. The managers of middle rank at various companies under Tepe Holding and Mother-Child Training Foundation were provided training by DGSPW specialists. -',. SUSTAINABILITY The Borrower confirmed that the manpower training activities conducted under the ETP will be carried out after the closure of the project. In line with this, the required infrastructure has been established. The Turkish Employment Organization is in the process of establishing a Private Employment Services Department which will assist in development of private employment services. To date, 329,389 people have used the job and career counseling services (1991-2000) and Career Counseling Centers (1998-2000). In order to maintain this success, the number of staff providing job and career counseling will be increased. The staff to be employed at these services will be provided with academic training using ISKUR resources as well as resources from outside. Also the staff working in Ankara and other provinces as job and career counselors will provide training on practical issues to newly recruited personnel. It is aimed to use GATB in local offices in an efficient manner. In this context, staff with Testing, Evaluation, Statistics and Psychology background will be used in these services. Seminars and in-service training programs will be organized to ensure active involvement of provincial office managers in counseling services. The activities conducted under the implementation of the labor market information component will be continued after the closure of the project. The training of the personnel working on this subject will also proceed. Furthermore, following the closure of activities related to the development of labor market information questionnaire, the coverage of the implementation will be extended to the whole country to produce short-term labor force projections. As confirmed in a letter signed by the ISKUR General Director dated February 22, 2001, the Turkish Employment Organization is going to finalize all the outstanding activities of the project in 2001. The draft legislation on the establishment of ISKUR including the Regulations on the Private Employment Agencies was submitted to the Parliament. The draft law is already being discussed in the Parliament and is expected to be enacted within this year. The same is true for the draft law regarding the establishment of the NOSI. The Ministry of Labor and the Ministry of National Education have reached an agreement on the subject, and the draft law is going to the Parliament and hopefully will be enacted in 2001. Institutionalization of NOSI would be the key to maintain the system, to make use of occupational - 26 -  standards in both formal and informal training programs and develop testing (measurement and evaluation) and certification system on the same basis. With respect to the Automation of Employmtnt Services Component, by March 31, 2001, the training of the relevant ISKUR staff was completed and acceptance of software applications with relevant testing and data migration is being expected shortly. D. LESSONS LEARNED The following points list the main conclusions and lessons learned: The fulfillment of the project objectives can be realized by the staff that has relevant knowledge, experience and high motivation. For the implementation of such a comprehensive project, the continuity of the managers and other employees is of great significance. Communication between institutions, units and services is very important. The guidance and support of national and international counselors is highly appreciated for technical studies of the counseling services. The objectives of the project with respect to training of the staff have been achieved with a notable performance and required motivation of the personnel. The investment made and the support given is evaluated as productive, and participation activity was very high. However, one of the deficiencies of the project would be the fact that many of the ISKUR staff members who will be working on the collection, evaluation, analysis and reporting of the employment data have not received sufficient training. It would be appropriate that the personnel training issue will be taken into consideratiort at the design phase of other projects. Due to high costs of Temporary Community Employment Program, the program will be repeated only if there is a necessity. One of the shortcomings of the Job Insertion Training Component was that manpower training certificate provided to the participants of self-employment courses was not solely sufficient to start business which raises the issue of the quality and extent of relevant training courses and confirms that cooperation of ISKUR with other institutions, especially with the Ministry of National Education is crucial. On Automation Component, the most outstanding mistake was that the time schedule was designed while not considering enough the risks. As the duration of the implementation has been longer than expected, needs have changed over time, the personnel in the working groups have changed and the confidence of the ISKUR personnel in the project and their motivation have diminished. In selection of working and test groups, the balance of specialists, related provincial personnel and personnel from the ISKUR General Directorate could not be established. Thus, there have been loss of time, effort and performance. Lack of trained and educated personnel caused deviations in the project outcomes. However, the quality of the outcome should not be sacrificed to provide the product on time. The project should be carried out in accordance with defined and foreseen standards. Also for the implementation of the project, it is necessary for the ISKUR managers at high level to assess, oversee and finalize the certification of software usage, integration of hardware and software, the training of users and the technical training of the current and newly recruited technical staff for data processing. In all phases of the project, including project design, implementation and assessment, a reconciliatory policy that involves all relevant parties has been pursued. The participation and contribution of all relevant parties have provided opportunity for the more proper and effective use of national resources. The factor that affected the Project implementation was the changes in World Bank regulations and procedures in the last quarter of the Project. The fact that the PCU was not informed of these changes - 27 -  beforehand has caused a lot of unnecessary work. Also, inconsistencies between the operations staff and the procurement department of the Bank caused a lot of repeats in the Project, which could have been avoided. The Turkish Employment Organization has assigned counterpart staff during the last 6 months of the project who have become accustomed to the work done in the PCU. This could have been more efficient had this been done from the start of the project. (b) Cofinanzciers: N/A (c) Othter partners (NGOs/private sector): N/A 10. Additional Information N/A - 28 -  Annex 1: Key Performance Indicators/Log Frame Matrix Outcome/Impact Indicators: Indicator/Matrix Projected in last PSR1 Actua1/Latest Estimate Component 1. Job-Insertion Training: (a) Develop Employment Councils and Tripartite Local and National annual training plans at provincial and Employment Councils were established local labor offices in order to arrange vocational training courses on demand basis. (b) Improve methods of contracting with Training classes were targeted at the local agencies who are going to provide unskilled unemployed youth, and 37% of job insertion training trainees received training at enterprises; 31% in private training institutions, and 14% in public training institutions. (c) Develop pilot programs for Three pilots for the Quick Customized Customized "Quick Start" contract Training Programs were implemented in training to support new investment Eskeshir and Erzurum; 48 people employment generation participated and 43 of them were employed. (d) Refine self employment program Almost 16,000 individuals (mostly from initiatives, with the emphasis in rural rural areas) participated in Self- areas Employment Training Program. (e) Operate Job Insertion Fund In total, 84,571 people participated in 4,574 training courses, and the average job placement rate was 94% in enterprise on-the-job training, 89% in public training institutions and 90% in private training institutions (compared to the target for placement rates of 60%). (f) Promote Women's employment 70% of the participants of the vocational training courses were females. Component II: Diversification of Employment Services: (a) Describe and assess the effectiveness A study on Regulations of the Private of Private Employment Services (PRES) Employment Agency was conducted and study tours to four OECD countries were organized. ISKUR activities were reviewed. b) Identify alternative strategies by which A National Seminar on PRES was held in employment services can be diversified 1994, and a comprehensive report was prepared. -29 -  Annex 1: Key Performance Indicators/Log Frame Matrix I tr t Proj: j Lttilt;;$00000t00 tti;00 0 00jected in