Mexico - Second Technical Training and Manpower Training Projects

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 16126 PERFORMANCE AUDIT REPORT MEXICO SECOND TECHNICAL TRAINING PROJECT (LOAN 2559-ME) AND MANPOWER TRAINING PROJECT (LOAN 2876-ME) November 15, 1996 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents, New Pesos per US$ (annual averages) 1988 2.28 1992 3.12 1989 2.64 1993 3.30 1990 2.95 1994 3.41 1991 3.07 1995 7.40 Abbreviations and Acronyms CAST Centros de Asistencia de Servicios Tecnol6gicos (Centers for Assistance in Technological Services) CECATI Centros de Capacitaci6n T6cnica (Centers of Technical Training) CIMO Programa de Capacitaci6n Industrial de Mano de Obra (Industrial Worker Training Program) CONALEP Colegio Nacional de Educaci6n Profesional (National College of Professional Education IBRD International Bank of Reconstruction and Development (World Bank) ICR Implementation Completion Report NAFIN Compaiia Nacional Financiera (National Financing Company) OED Operations Evaluation Department PAR Performance Audit Report PCR Project Completion Report PROBECAT Programa de Becas de Capacitaci6n para Trabajadores Desempleados Labor Retraining Program SAR Staff Appraisal Report STPS Secretaria de Trabajo y de Previsi6n Social (Secretariat of Labor and Social Welfare) WID Women in Development Fiscal Year Government: January 1 - December 31 Director-General, Operations Evaluation : Mr. Robert Picciotto Director, Operations Evaluation Dept. : Mr. Francisco Aguirre-Sacasa Division Chief, Agriculture and Human Development Mr. Roger Slade Task Manager : Ms. Helen Abadzi FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.SA Office of the Director-General Operations Evaluation MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Mexico Second Technical Training Project (Loan 2559-ME) Manpower Training Project (Loan 2876-ME) Attached is the Performance Audit Report prepared by the Operations Evaluation Department (OED) on the Mexico Second Technical Training project (Loan 2259-ME for US$81 million equivalent) approved in 1985 and the Mexico Manpower Training project (Loan 2876-ME for US$80 million) approved in 1987. Comments received from the borrower are reproduced as Annex 4. OED responses are contained in Annex 5. The projects were designed to improve the quality and increase the number of skilled workers and technicians at a time of severe economic crisis in Mexico. The projects financed institutional development, curriculum and materials development, and training or retraining scholarships for the unemployed. Scholarships were to be awarded in accordance with eligibility criteria set out in the Staff Appraisal Report. A focus on the lower-middle class of urban areas rather than the neglected rural populations was due to the government's reluctance to borrow for the social sectors. The outcome of both projects is rated as satisfactory, institutional development as substantial, and sustainability as likely. Nevertheless, Bank task management and supervision teams emphasized the economic aspects of the projects and paid less attention to instructional, poverty, women's and indigenous people's issues. As a result, the audit found that (a) in allocating scholarships financed under the project employers did not provide fair access to women; and (b) too little attention was given to the learning needs of the less educated adults selected for training. The Bank's insistence on social-sector interventions has paid off, and Mexico's social sector strategy, especially the extent to which it is targeted on the poor, has vastly improved in recent years. Nevertheless, to ensure that the full benefits flow from current and future vocational education projects in Mexico the following steps are needed: (a) Allocation criteria for training scholarships should provide equitable access to women. Bank expertise on gender should be used to help Mexico introduce innovative approaches that have worked elsewhere (such as setting aside scholarships for women in stereotypically male occupations). (b) The short-term training programs need expert review with the aim of maximizing the probability that adults with limited education will be able to learn the needed skills. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 (c) Specialized expertise is needed to develop action plans for the inclusion of indigenous groups, who are especially difficult to reach in the training programs. The audit also raises two broad and important lessons for the Bank: (a) Projects must be carefully screened and reviewed at entry and during supervision to minimize the possibility of inequitable provision of services to women and the poor. (b) Decisions to finance interventions that are targeted on the non-poor must be carefully weighed, especially if the borrower is unwilling to also undertake welfare enhancing measures targeted to the poor. Attachment FOR OFFICIAL USE ONLY Contents Preface .......................................................................................................................................... 3 Basic Data Sheet-Second Technical Training Project (Loan 2559-ME).........................5 Basic Data Sheet-Manpower Training Project (Loan 2876-ME)..... ..................9 EvaluationSummary ............................................................................................................... 13 1. Introduction ......................................................................................................................... 21 Background...... . ..............................................21 Previous and Subsequent Projects .. ..........................................................22 2. Second Technical Training Project (CONALEP II)........................24 Project Objectives and Components...................................................................... 24 Implementation Experience.................................................................................. 24 Physical Facilities.................................................................................................... 25 Curricula and Materials......................................................................................... 26 Staff Development and Instructional Delivery.......................................................27 Student Selection and Training-Social and Employment Issues.......................... 27 G ender .................................................. ................................................. 28 Benefits and Risks.............................................. ......................................... 29 Results ........................ ........................................... 29 Institutional Development ...... ....................................................... 30 Sustainability ....... ..... ...................................... ................................. 30 Recommendations ......................................... 30 3. Manpower Training Project ........................................................................................ 32 Objectives and Description ................................................................................... 32 Implementation .. .................................................................................... 33 Employment Services and Labor Retraining .........................................................33 The demonstration in-service teaching and productivity component (CIMO) ... 34 STPS Institutional Strengthening Component ...................................................... 35 Training Institutions Upgrading Component .........................................................36 Instructional Delivery............................................................................................ 36 Attention to Rural Areas.......................................................................................... 37 G ender Issues.............................................. ............................................38 Benefits and Risks................................................................................................. 39 Outcomes ......... ................................................................................40 Sustainability ............ ............................................................... 42 This report was prepared by Helen Abadzi (Evaluation Officer), who audited the project in March 1996. Pilar Barquero provided administrative assistance. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed widhout World Bank authorization. 2 Institutional Development..................................................................................... 42 Recommendations Specific to STPS Operations ................................................. 42 4. General Finding, Issues and Lessons .................................... 44 Borrower Performance......................................................................................... 44 Bank Performance................................................................................................. 44 Finding and Issues................................................................................................. 45 Women's Employment Difficulties Were Neglected .................................... 45 Technical Expertise Was Limited .................................................................. 46 The Social Sector Strategy Was Inadequate ...................................................46 Lessons ................ ........................................................................ 48 BOXES 1. How CONALEP Schools Function .................................................................................. 25 2. How PROBECAT Scholarships Are Used........................................................................34 3. Poverty Discrimination?................................................................................................... 38 ANNEXES 1. Manpower Training Project-Objectives and Design.......................................................49 2. Project Results: CONALEP II (Direct Benefits of Project for Total CONALEP System). 51 3. Project Results: Manpower Training Project ...................................................................52 4. Comments from the borrower .............. ............................................. 53 5. Responses to borrower comments . .................................................................... 61 3 Preface This is a Performance Audit Report (PAR) on two related education projects in Mexico supported by the World Bank during 1985-1992. The Second Technical Training project (Loan 2559-ME for US$81 million) was approved on May 28, 1985 and was completed on June 30, 1991; the funds were fully disbursed by July 26, 1991. The Manpower Training project (Loan 2876-ME for US$80 million) was approved in October 1987 and was substantially completed on December 31, 1992. The funds were fully disbursed, and the project closed on June 30, 1993, six months before the anticipated closing date. The PAR is based on the following sources: the Project Completion Reports (PCRs) issued as Report No. 10216 dated December 30, 1991 (Second Technical Training project) and Report No. 12562 dated November 30, 1993 (Manpower Training project); the Staff Appraisal Reports (SARs); the Loan Agreements for the projects; and the project files, in particular the supervision reports. An OED mission visited Mexico in March 1996 to collect other pertinent information. The size, complexity, and spread of project activities throughout the country made it impossible to verify all activities. For each subcomponent, the audit reports achievements as reported in the PCRs and complements them with interviews and observations of the audit mission. The extent to which the events witnessed by the audit mission were representative of what took place under the projects is, of course, unknown. Value added. The ICR focused on the achievement of project objectives. The audit mission focused on the social and instructional issues influencing project outcomes. Discussions were held with students, businessmen, and project administrators in order to understand the extent to which the training provided had helped improve productivity and incomes in Mexico. Following customary OED procedures, copies of the draft PAR were sent to the relevant government officials and agencies for their review and comments. A number of observations were made which have been incorporated into the PAR as Annex 4.  5 Basic Data Sheet SECOND TECHNICAL TRAINING PROJECT (LOAN 2559-ME) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 164.4 164.4 100 Loan amount 81 81 100 Date physical components completed 6/90/89 6/90/91 2-year delay Economic rate of return n.a. Cumulative Estimated and Actual Disbursements FY85 FY86 FY87 FY88 FY89 FY90 FY91a Appraisal estimate (US$M) 6.9 23.5 49.0 72.0 81.0 Actual (US$M) 11.5 30.7 54.0 65.4 75.2 79.3 Actual as % of estimate 49% 63% 75% 81% 92.3 97 Date of final disbursement: July 26, 1991 a. Actual disbursements as of May 23,1991; the Loan was fully disbursed by July 26,1991. Project Dates Original Revised Actual Identification mission 09/84 -- 08/84 Prep. appraisal mission 11/84 -- 11/84 Loan negotiations 04/85 -- 04/85 Board presentation 05/85 -- 05/85 Loan signature 07/85 -- 07/85 Loan effectiveness 10/85 01/86 12/85 Project completion 06/89 06/90 12/90 Loan closing 12/89 12/90 06/91 Staff Inputs (by Stage of Project Cycle) Stage of Project Cycle Actual Staffweeke Through appraisal 48.5 Appraisal through Board approval 52.9 Supervision 84.2 a. TRS data until 01/10/91. 6 Mission Data Date No. of Staff weeks Specializations (month/year) persons infield represented a Performance rating Identification 10/84 3 5 EP, TE, A Prep. appraisal 11/84 5 8 EP, FP, TE, E, EE, LO Total 8 13 G P M F Supervision 1 08/85 1 1 EP 1 1 2 1 Supervision 2 02/86 3 4 EP, 2TE 1 1 2 1 Supervision 3 05/86 3 3 EP, 2TE 1 1 2 1 Supervision 4 10/86 2 2 EP, TE 1 1 I I Supervision 5 10/87 1 2 TE 1 1 1 1 Supervision 6 05/88 2 3 2TE 1 1 2 1 Supervision 7 11/88 3 6 2TE, TM n.a. Supervision 8 01/89 2 6 2TE 1 1 1 1 0 D C M F Supervision 9 06/89 2 2 2TE 1 I 1 1 1 Supervision 10 11/89 3 3 2TE, E 1 1 1 1 1 Supervision 11 06/90 3 4 E, TE, IS 1 1 I I 1 Supervision 12 11/90 1 1 TE 1 1 1 1 1 Total 26 37 PCR 11/90 1 1 IS 1 1 1 1 1 Total 1 1 a. E = Economist; EE = Education Economist, A = Architect; EP = Education Planner, IS = Implementation Specialist; TE = Technical Educator; LO = Loan Officer; TM = Training Organization and Management SpecialisL b. G= General Status; P = Procurement; M = Management; F= Financial. c. 0 = Overall Status; D Project Development Objectives; C = Compliance with legal Covenants; M = Project Management Performance; F = Availability of Funds. I = Problem Free or Minor Problems; 2= Moderate Problems; 3 = Major Problems. 7 Other Project Data Borrower/Executing Agency: FOLLOW-ON OPERATIONS Amount Operation Credit No. (US$ million) Board date Third Technical Training Project 3358-MX 152 June 25, 1991  9 Basic Data Sheet MANPOWER TRAINING PROJECT (LOAN 2876-ME) Key Project Data (amounts in USS million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 156 156 100 Loan amount 80 80 100 Date physical components completed 12/31/93 6/30/93 6 months early Cumulative Estimated and Actual Disbursements FY88 FY89 FY90 FY91 FY92 FY93 Appraisal estimatea (US$M) 7.5 16.0 32.0 48.0 64.0 78.0 Actual (US$M) 6.0 21.7 38.2 59.7 75.0 80.0 Actual as % of estimate 80.0 135.6 119.4 124.4 117.2 102.6 Date of final disbursement (projected): June 30, 1993 a. Appraisal estimate adjusted to actual date of loan approval (October 6,1987). Project Dates Planned Actual Identification 05/86 05/86 Preparation I 10/86 10/86 Preparation II 02/87 02/87 Preappraisal 04/87 04/87 Appraisal 10/86 05/87 Negotiations 04/87 08/87 Board approval 05/87 10/87 Loan signature 10/87 10/87 Loan effectiveness 10/87 10/87 Loan closing 12/93 06/93 10 Staff Inputs (staff weeks) HQ Field Totala Through appraisal 42.5 33.5 76.0 Appraisal through 20.0 0.0 20.0 Board approval Supervision 22.5 47.8 70.3 PCR 0.8 0.5 1.3 Total 75.0 81.3 166.3 a. TRS data until 12/31/91. Mission Data Date No. of Staff weeks Specializations Performance Types of (month/year) persons infield represented' ratingb problems Identification 05/86 2 1.6 C, N n.a Preparation I 10/86 6 10.5 A, C, N. P, Q, R n.a Preparation II 02/87 5 12.0 C, F, L, M, 0 n.a Preappraisal 04/87 2 2.0 A, C n.a Appraisal 05/87 4 6.4 A, C, J, M n.a Supervision I 10/87c 2 2.0 C, F I n.a Supervision 2 05/88 5 7.0 C, F, G, I, K I n.a Supervision 3 11/88d 3 6.0 C, F, S 1 n.a Supervision 4 01/89 2 6.0 C, F 1 n.a Supervision 5 06/89 4 4.0 C, F, G, H 1 n.a Supervision 6 11/89 4 5.0 A, B, C, F 1 n.a Supervision 7 07/90 3 7.8 A, B, C 1 n.a Supervision 8 05/91c 5 10.0 A, B, C, D, E 1 n.a Completion 03/93 2 0.5 A,E 1 n.a a. A = economist; B = operations analyst; C = senior technical training specialist; D = labor statistics specialist; E = employment services specialist; F = technical education analyst; G = evaluation specialist; H = labor economist; I= education and training unit chief; J = institutional development specialist; K = vocational educator; L = management and in-service training specialist; M = project costing specialist; N = senior education planner; 0= loan officer; P = manpower information systems specialist; Q = industrial and management training specialist; R = technical educator, and S = training organization and management specialist. b. I = problem free or minor problems; 2 = moderate problems; 3/4 = major problems. c. Jointly with supervision mission of Loan 2559-ME. d. Jointly with supervision mission of Loan 2559-ME. e. Jointly with identification mission of Second Manpower Training Project. n.a. = not applicable. 11 Other Project Data Borrower/Executing Agency: FOLLOW-ON OPERATIONS Operation Amount Credit No. (US$ million) Board date Labor Market and Productivity Enhancement Project 3542-ME 174 02/15/92 Technical Education and Training 3805-MX 235 10/27/94 Modernization  13 Evaluation Summary 1. A performance audit of the Second Technical Training project and the Manpower Training project was conducted because the Project Completion Reports (PCRs) raised important questions regarding the effectiveness in providing technical training to large numbers of people and at the same time developing the institutions that would carry it out. The PCRs also raised questions regarding the benefits women and the poorer populations derived from these projects. 2. Both projects served large numbers of beneficiaries throughout the country through diverse means. The audit mission visited activities in Ciudad JuArez, Monterrey, Chiapas, and Mexico City. At each site, discussions were held with employers, students, graduates, and businessmen to assess the effects of training, scholarships, and technical assistance on the various beneficiaries. The concerned project agencies planned all visits extremely well. Background 3. To help spur economic growth after the economic crisis of the 1980s, the government sought to increase the number and skills of mid-level workers and technicians. Designed to meet the technological needs of the private sector, training efforts concentrated on younger people who had completed lower secondary education (grade 9). To prevent loss of trainees to higher education, the new pre-employment training programs were designed to be terminal. The most important training vehicle has been the National College of Technical Education (Colegio Nacional de Educaci6n Profesional; CONALEP), a semi-autonomous organization under the Secretaria de Educaci6n (Ministry of Education). CONALEP became the focus of three Bank- financed technical training projects, the second of them the subject of this audit (CONALEP II). 4. Substantial gains in productivity would require more in-service training of large numbers of employed workers, particularly for middle-level technicians and skilled workers in Mexico's 380,000 micro, small and medium-scale enterprises. To serve these workers and enterprises, the Bank financed a series of three manpower training projects that emphasized development of industrial and service skills in urban areas and were administered by the Secretaria de Trabajo y de Servicios Sociales (STPS; Ministry of Labor and Social Services). The first of these projects is also a subject of this audit. STPS contracted much of the training it financed to CONALEP, thus forming training linkages with this institution. Second Technical Training Project (CONALEP II) 5. Objectives and Design. The objectives of CONALEP II were to: (a) improve the operational efficiency and effectiveness of central and local administrations; (b) increase the supply of skilled workers and technicians; (c) improve the quality of training instruction; (d) pilot advanced maintenance systems for buildings and equipment and introduce significant cost- reduction measures; and (e) facilitate the development of strategic training options and institution building. To achieve these objectives, the project provided for: (a) technical assistance, fellowships, and management training; (b) construction, equipment, and furnishings for about 97 new training centers; (c) the upgrading and expansion of instructor training and 14 teaching materials; (d) the development of a building maintenance system; and (e) studies of efficiency measures and strategic training options. Implementation Experience 6. Implementation proceeded speedily and efficiently. A total of 130 vocational training centers were constructed and equipped, a substantial increase from the 97 planned during appraisal. The mission found the large amount of equipment financed by CONALEP II (some of it 8-10 years old) to be largely still useful and in good working order. A self-maintenance strategy was developed, which has been strengthened in recent years, to ensure the sustainability of the investment. 7. Individual schools offer various specialties as needed in the local markets, mainly in industrial, services, health, and agricultural areas. They have been located near industrial centers, and industry linkage committees have helped develop programs and curricula specific to the local economic needs. The project financed curricular improvements for about 40 percent of the courses and 1,111 publications. About 6.8 million copies were printed of textbooks, bulletins, syllabi, training materials, guidelines, handbooks, leaflets, extension bulletins, journals, and manuals. However, the mission could not determine whether the instructors were properly trained or how extensively they used the new texts. 8. The project started an innovative program to offer technological services to private industry (Programa de Asistencia de Servicios Tecnol6gicos-CAST). During CONALEP II, about 30 enterprises benefited through training of in-plant trainers. This program has greatly expanded under the follow-on project in about 30 areas of heavy industry, where just-in-time advance technology updating is important. Three industrial clients interviewed expressed great satisfaction with the services. Due to demand, CAST centers are able to become profitable relatively quickly despite the large investment in equipment. Results 9. The project clearly met its objectives and the outcome is rated satisfactory. By 1995 360,000 people had been trained. CONALEP offered courses in all specialties to 212,000 pre- employment students and 80,000 employed workers. This is a remarkable achievement, given the overall low status of vocational schools worldwide as well as in Mexico before CONALEP. 10. Nevertheless, the three-year CONALEP programs have considerable dropout. The completion rate was 52 percent at project closing, compared to 78 percent expected during appraisal. Dropout is particularly high during the first semester, when about 40 percent of students leave , partly due to a strong preference for academic secondary education. Interviews with students and graduates showed that many students initially lose their self-esteem when they leave the formal system and make efforts to return to secondary schools. The graduates interviewed by the mission reported much satisfaction with the program. However, they felt the need to re-enter the formal system for advanced studies after some years of work for further professional development. 11. Graduates have fared relatively well in the job market. The PCR states that at the end of the project implementation period, 62 percent of the three-year program graduates found employment within three months of completing their coursework, with 84 percent employed in 15 their area of training specialization. Almost all graduates were employed by the private sector and were earning on the average 170 percent of the minimum wage in 1991. A survey showed that 92 percent of interviewed employers in nine different economic sectors who hired CONALEP graduates expressed consistently for a four-year period that training was good or excellent. 12. Women's incomes have not benefited from CONALEP training as much as men's. The institution tries to improve the status of women's training, and there are no barriers to admitting them in any course. Nevertheless, women tend to congregate in the more stereotypically feminized professions, such as computer operation (informatics) in the three-year courses and industrial sewing in the shorter courses. The remuneration for feminized skills (e.g. seamstresses in textile factories) is quite low. As a result, tracer studies have found that female three-year graduates made up 44 percent of graduates placed in a job vs. 56 percent for men. Manpower Training Project 13. Objectives and Design. Broadly, the objectives of the Manpower Training project were to (a) strengthen employment services to place displaced workers in jobs and administer short- term PROBECAT scholarships; (b) assist private enterprises to upgrade the skills of employees; (c) upgrade STPS staff and strengthen STPS capability to manage information and carry out policy studies; and (d) increase the training capacity of selected institutions so that they can train the unemployed. The project financed scholarships for the unemployed, training and equipment for administering bodies (such as state employment services, training institutions), and assistance for small and medium-sized businesses. Implementation Experience 14. Implementation proceeded very well and met or surpassed expectations in almost all components. Disbursements were consistently ahead of forecast. 15. The employment services and labor retraining component received training and upgraded equipment. During the implementation period, state employment services served about 254,074 job seekers and 247,463 vacancies per year. By the end of 1992, over 293,217 unemployed workers had been retrained through PROBECAT scholarships (programa de becas de capacitaci6n en el trabajo) in over 19,939 courses, lasting 1-6 months. About 60 percent of men and 33 percent of women found jobs within three months following training. 16. The demonstration in-service teaching and productivity component (CIMO) was conceived as a pilot program. CIMO worked directly with over 32,304 separate enterprises and trained more than 124,182 workers. It proved very successful and was greatly expanded under the follow-on project. 17. The STPS institutional strengthening component: (a) expanded STPS's program of labor market surveys and studies, (b) trained 1041 STPS staff; (c) provided computers and software to state employment service offices in state capitals; and (d) supported the development of a computerized market information system that became operational during the follow-up project. 16 The training institutions upgrading component has been successful in increasing training capacity and in improving the quality of training and administrative efficiency of 445 participating training institutions. In these institutions, about 134,000 trainees were enrolled in preservice training courses by 1991/992; some 42,647 trainees hand participated in in-service courses, and 24,217 had participated in retraining program activities. Results 18. The project performed well and outcome is rated satisfactory. The project strengthened employment services at the state level, assisted small private enterprises to upgrade the skills of their workers, strengthened the market monitoring capability of STPS, and upgraded institutions that provided training in critically needed specialties. A strong monitoring plan made it possible to measure outcomes. 19. The CIMO program, which provided management help to enterprises, was quite successful. An evaluation study concluded that the program was able to build close and effective relations with enterprises and could provide effective management support in a short time with only a very light administrative structure. A case study of 30 enterprises participating in the program concluded that enterprises had indeed increased their productivity, gained better access to markets, and had improved organization and labor conditions. 20. The outcomes of the PROBECAT scholarship program are quite ambivalent. An impact study found a rather small effect on income and on the probability of finding a job after training, but the outcomes were mainly favorable for men; women did not benefit significantly from the program. To some extent, this may have happened because fewer women were eligible for retraining. However, it mainly seems due to women's exclusion from the more technical positions of many larger enterprises. Most employers interviewed by the mission expressed clear gender preferences and stereotypes for various occupations. Award of scholarships was market-driven, and about 80 percent of the scholarships went to men. 21. The benefits and sector strategy sections of the Staff Appraisal Report (SAR) stated that women and the poorest of the poor would benefit from this project. However, an overall poverty alleviation effect could not be verified. The analyses used to demonstrate it were probably influenced by a statistical artifact, regression towards the mean. 22. Related to the outcomes are the following issues: (a) Women's vocational choices are constricted. Scholarship recipients largely expressed the desire to study traditionally female areas. Some stated to the mission that they did so because employers would not hire them for more male-oriented positions. A great deal of women's training has been in sewing, but income-generation opportunities in sewing appear limited. No study has been done to find out how women use this skill after training, particularly when they lack sewing machines or a viable business plan. (b) Rural areas and poor and indigenous people need more training opportunities. The project explicitly targeted urban areas. Only 7 17 percent of the investment went to rural areas, despite an expressed need, largely because Bank-approved criteria limited scholarships to adult primary-school leavers. State STPS officials in areas like Chiapas had difficulty finding many such people and developing effective training solutions. The STPS also sought providers for indigenous people in remote rural areas and ways to deal with the environmental degradation created by some of their income-generation activities. To expand in rural areas, the program needs a knowledgeable anthropologist to conduct culturally appropriate social assessments. (c) Instructional delivery must improve. There was little or no monitoring of instruction, attendance, or learning outcomes by STPS, particularly for courses given outside the direct control of CONALEP. Some students stated that the training time (for instance, two months for carpentry) was too short for them to learn all they needed to know. The mission observed that shorter courses, which catered to less educated people, lacked manuals, and interviewed two instructors who were not clear about the curriculum they were teaching. However, the mission did not find conclusive evidence that the training program was deficient; insufficient teacher training and instructional delivery may account in part for the ambivalent outcomes of PROBECAT scholarships. PCR Ratings 23. The audit agrees with the implicit ratings of the Project Completion Reports (PCRs). For both projects, the outcome is rated as satisfactory. Institutional development impact is rated as substantial and sustainability as likely. The Bank's performance is rated as satisfactory. The borrower's performance is also rated as satisfactory. Institutional Development Impact 24. The audit agrees with the PCRs of both projects that institutional development impact has been substantial. CONALEP was able to execute a series of three complex projects that developed 252 well-run schools with a good reputation in the community. The schools visited had competent and educated staff, able on short notice to organize discussion sessions, invite graduates and industry representatives, and provide data regarding school operations. Several components of the Manpower Training project strengthened institutional development impact at the federal and state levels through staff training, labor market information systems, labor market studies, capability to search for jobs and place the unemployed, and the upgrading of training institutions. Sustainability 25. The audit agrees with the PCRs of both projects that sustainability is likely. 18 Borrower Performance 26. In both projects, borrower performance was satisfactory. The executing agencies proved proficient in project planning and implementation and showed competence in project management. The Manpower Training project, in particularly, enjoyed much government support and was implemented by a highly competent, enthusiastic, and hard-working management group. Bank Performance 27. For both projects, the Bank's performance was adequate during project appraisal and supervision. There was considerable staff continuity throughout the years, which facilitated implementation. However, supervision time was limited for projects of such magnitude. CONALEP II received only 12 supervision missions in six years, averaging only 6.2 staff weeks every year. For the Manpower Training project eight supervision missions were conducted, which consisted of 2-5 members for about two weeks in the field. 28. Government officials expressed satisfaction with the services the Bank had provided in supervising the two projects as well as with the quantity and quality of its staff. However, STPS expressed dissatisfaction with turnover of task managers and a strong desire to maintain continuity. 29. By traditional project criteria, the Bank's performance in the implementation of these two projects is rated satisfactory. However, Mexico's reletionship with the Bank in the 1980s created complexities in the Bank's country assistance strategy and its implementation. The government was extremely reluctant to borrow for the social sectors, and the Bank did its best to establish a working relationship with Mexico through interventions that the government preferred. As a result of these interactions, the mission identified the following important issues with respect to the Bank's strategy and implementation. (a) Lack of educational expertise. No attention was paid to the instructional processes and outcomes of the Manpower Training Project. Despite the difficulties in teaching persons of limited educational background, adult education specialists were not involved in the projects. Lack of instructional expertise may have contributed to a lower quality of instruction and adversely influenced project outcomes. The economists who managed the project studied external efficiency in detail but did not deal with internal efficiency. Inattentiveness to learning outcomes has made it impossible to link cause and effect in studying earnings outcomes. (b) Inaction on women's employment problems. Since 1990, studies have carefully monitored the effects of the Manpower Training project on women, but neither the Bank nor the borrower has taken any remedial action. The Staff Appraisal Report (SAR) of the subsequent project (Loan 3805-ME; 1992) mentioned the labor market barriers for women and proposed an action plan. However, the audit mission found that after four years of implementation, no substantive action had been taken on either developing an action plan or remedying the problem. This 19 inaction was particularly important in light of the fact that gender discrimination in most cases is illegal according to Mexico's labor laws. Since employers freely exclude women from potential employment through scholarships, it appears that Bank proceeds have been used to perpetuate gender discrimination practices for nearly a decade. The women's employment problems in the project highlight a dilemma. Is it acceptable from a development perspective to finance scholarship schemes that involve clear gender discrimination and practices that keep women at the bottom of the income scale? According to Operational Directive 4.20, the Bank is to assist its member countries to design gender-sensitive policies, modify legal frameworks to improve women's access to assets and services, and finance programs for women. Should the Bank also do the reverse, that is refuse to finance programs that involve gender discrimination? High-level decisions are needed. (c) Inadequate social sector strategy. The Bank emphasized the economic aspects of the projects and monitored productivity but did not deal with issues of the rural poor or the indigenous populations. The sector strategy seems to have suffered from the same problem. Five vocational-technical education projects were appraised in Mexico before any other social-sector projects were implemented. The SAR of the Manpower Training project promised complementary projects that would be better targeted to the poor, but the first these did not become effective until five years later. Clearly, Mexico did not want to borrow for the social sectors at the time. However, did the Bank follow its guidelines as a development institution when it agreed in the mid-1980s to dispense with projects targeted toward the poor? It seems to have acted towards Mexico more as a commercial bank than as a development institution at that time. 30. The Bank's insistence on social-sector interventions seems to have paid off. Mexico's social sector strategy has vastly improved in recent years. Nevertheless, the implementation of the two audited projects may have suffered because potential beneficiaries in the more deprived areas did not have the minimum qualifications to participate that they might have received through primary-education investments. Clear high-level guidelines are needed to determine how the Bank strategy should evolve when borrowers with large, underserved poor populations are reluctant to borrow for the social sectors. Lessons * Studies of women's earnings and calls for action plans may be insufficient to bring about improvement. Expertise, persistence, follow-through, and dedicated time during supervision missions are also needed. * Projects that require a minimum level of education may be implemented most effectively through the support of operations that provide the poor with the required education. Projects which have minimum educational requirements are not likely to reach the poorest of the poor, who lack the requisite level of education. 20 * The economic viability of vocational education projects is likely to suffer if they are developed and supervised with scant technical expertise. Economists must work closely with technical experts. * Terminal diploma courses in many countries suffer from low quality and low student demand. However, the success of CONALEP shows that it is possible to deliver quality education to large numbers of students. Implementing agencies must create a positive image for students and effective linkages with local enterprises. * Indigenous groups may be difficult to reach through projects designed for the general population. Anthropologically sensitive methodology and social assessments would be needed to maximize the benefit these groups can receive from Bank investments. 21 1. Introduction 1.1 A performance audit of the Second Technical Training project and the Manpower Training project was conducted because the Project Completion Reports (PCRs) described them as successful in providing large-scale technical training and developing at the same time the institutions that carried it out. At the same time, the PCRs raised important questions regarding the benefits women and the poorer populations derived from these projects. 1.2 Both projects served large numbers of beneficiaries throughout the country, and the audit mission was able to visit only a small sample of project activities. Sites visited were in Ciudad Juirez (where much activity was targeted towards the maquiladoraI industry), in Monterrey (where high-technology industry is concentrated), Chiapas (a rural area with limited industry and a large indigenous population), and in Mexico City (at a specific vocational school for service- oriented professions). In each site, discussions were held with employers, students, graduates, and businessmen aimed at assessing the effects that training, scholarships, and technical assistance had had on the various beneficiaries. Background 1.3 The projects grew out of the government's desire to alleviate the effects on unemployment of a severe economic crisis that enveloped Mexico in the early 1980s. Although by 1985 a stabilization program brought the situation under control, economic growth was still low. The country suffered from long-term problems, such as a high rate of population growth (about 2.5 percent per year), low levels of human capital formation, slow growth in agriculture, widespread poverty, highly skewed interpersonal and interregional income distributions, and an economy too dependent on oil revenues. To deal with the impact of these problems on growth and employment, the government sought to increase the number of skilled workers and technicians. A large and complex technical training system was developed that serve students with various levels of education. To make it possible for students to attain the levels of technological development needed by the industry, efforts were concentrated on training mid- level technicians, younger people who had completed lower secondary education. To prevent loss of trainees to higher education, terminal (essentially nonformal education) programs were designed that would meet the demand of the private sector. The most important training vehicle has been the Colegio Nacional de Educaci6n Profesional (CONALEP; National College of Technical Education), a semi-autonomous organization under the Ministry of Education (Secretaria de Educaci6n). 1.4 However, technological training by itself was considered insufficient to stimulate economic growth and productivity. Employment services and retraining programs were also needed to be strengthened and redirected to better serve the poorest among those displaced and unemployed. Particularly important was aid to the informal sector, which in the mid-1980s included one-third of the work force and was growing faster than the formal sector. Further gains in the productivity required more effective provision of in-service training for large numbers of workers, particularly for middle-level technicians and skilled employees in Mexico's 1. Industry assembling products specifically for use in other countries. 22 380,000 small and medium-scale enterprises (up to 250 workers). These enterprises, which in 1987 employed 51 percent of the industrial work force, were particularly affected by the economic adjustment measures but hey also played an increasingly important role under the liberalized trade system. To serve them, the Manpower Training project was developed. It emphasized development of industrial occupations in urban areas and was administered by the STEPS. Most training activities of the project were contracted to CONALEP. 1.5 Thus, the Bank financed two parallel series of manpower training projects in Mexico. The projects and their labor-market linkages mark a clearly enunciated, long-term policy of the government to support the type of training that will make Mexico a major producer and exporter of high-technology industrial products. Previous and Subsequent Projects 1.6 The Bank's presence in Mexico's education sector started in 1981. For the first 11 years it centered exclusively on vocational-technical education. No Bank-financed investments were made in any other education subsections until 1992, when a primary education project became effective (Loan 3407-ME). Also, no other social-sector investments were made until the first health project (Loan 3272-MX) became effective in 1991, partly because Mexico was not willing to borrow for the social sectors in the 1980s. (See section 4.) 1.7 The first Technical Training Project, (CONALEP I; Loan 2042-ME) became effective in 1981. The project initiated the development of CONALEP by providing civil works and equipment for 122 schools as well as institutional development. An audit conducted in March 1988 concluded that the project had a satisfactory outcome (see Report No. 7136, OED2). CONALEP I was followed by the Second Technical Training project (also known as CONALEP II), the subject of the current audit. Upon completion, the project was in turn followed by the Third Technical Training project, CONALEP III; Loan 3358-MIX; 1991). The other subject of the audit, the Manpower Training Project, started about two years after CONALEP II and was also followed by another project, Labor Market and Productivity Enhancement Project (Loan 3542-ME; 1992). Yet another technical-vocational education project became effective in August 1995 (Technical Education and Training Modernization project; Loan 3805-ME). Thus, a total of six projects have been financed in vocational-technical education (Table 1). 2. The problems identified in the audit centered on the high dropout rate, low financial sustainability, a shortage of instructional materials, deficient instructor training, poor building maintenance, and weak monitoring. These problems have been dealt with to a large extent, but high dropout rates continue to be a problem. 23 Table 1. Vocational-Technical Education Projects in Mexico Year of Colegio Nacional de Effectiveness Educaci6n Profesional Secretaria de Trabajo y Previsi6n Social 1981 CONALEP I (Ln. 2042-ME) 1985 CONALEP II (Ln. 2559-ME) 1987 Manpower Training (Ln. 2876-ME) 1989 1991 CONALEP III (Ln. 3358-ME) 1993 Labor Markets & Productivity (Ln. 3542-ME) 1995 CONALEP participates Technical Educ. & Training Modernization (Ln. 3805-ME) 1.8 Monitoring and Evaluation. Collection and analysis of monitoring data was an important feature of both series of projects, and the systems have improved with the years. Although the follow-up system is still imperfect, tracer data exist on graduates as well as dropouts, making it possible to study the effects of the projects. 1.9 To describe the two audited projects most clearly, the report presents separately the objectives, implementation experience, outcomes, and recommendations for each. These discussions by issues and general recommendations common to both projects. 24 2. Second Technical Training Project (CONALEP II) Project Objectives and Components 2.1 The objectives of CONALEP II were to: a. improve the operational efficiency and effectiveness of central and local administrations; b. increase the supply of skilled workers and technicians; c. improve the quality of training instruction; d. develop advanced maintenance systems for buildings and equipment on a pilot basis, and introduce significant cost-reduction measures; e. facilitate the development of strategic training options and institution building. 2.2 To achieve its objectives, the project included: (a) improvement of CONALEP's operational efficiency and effectiveness through technical assistance, fellowships, and management training; (b) increasing the supply of skilled workers and technicians through the construction, equipping, a and furnishing of about 97 new training schools; (c) upgrading and expansion of instructor training and teaching materials; (d) development of a maintenance system for buildings; and (e) studies of other efficiency measures and development of strategic training options. 2.3 Total costs for the Second Technical Training project were estimated at US$162.2 million, of which US$81 million was financed by an IBRD loan. The project was approved on May 28, 1985 and became effective in October 1985. It was completed in December 1990 and was closed after a delay of two years, on June 30, 1991. Implementation Experience 2.4 Despite some procurement delays, implementation proceeded speedily and efficiently. Project activities remained essentially as planned, but the scope widened. The number of schools targeted for construction increased by 25 percent. Construction savings, additional government funds, and exchange-rate changes provided the needed financing. 25 Box 1: How CONALEP Schools Function The CONALEP system offers about 120 courses of study, mainly in industrial, services, health, and agricultural fields. Different specialties are offered as needed in the local markets. Schools provide (a) short courses in specific subjects (including courses contracted by STPS), and (b) a terminal three-year diploma to students who have at least a lower-secondary education (9th grade). To ensure that technicians reach the labor market rather than the universities, CONALEP diplomas do not allow students to continue studies in higher education; though officially recognized, the education is in effect nonformal. This was a risky decision, because world-wide terminal-degree schools tend be of little demand and to attract the lowest-performing or most problematic students. However, the risk has not materialized in the case of CONALEP; the economic crisis of Mexico has made many students decide in favor of more immediate economic returns. CONALEP trainees are typically lower-secondary school leavers from a lower-middle class background who did not excel in school. (Some, however, are older and much better educated.) According to reports of administrators, the students like discipline and order, and CONALEP schools do not face major delinquency problems. A large variety of scholarships ensures that most people who want to study can indeed do so. According to the Implementation Completion Report (ICR), about 239,000 students were enrolled in project-financed CONALEP schools during the life of the project, surpassing the appraisal estimate of 200,000. Numbers have remained high in the years since project completion. CONALEP actively recruits students and advertises its training in the mass media. Its counselors systematically promote the training centers to the secondary schools that constitute their catchment areas; the mission observed groups of students visiting school grounds. However, schools have found it difficult to convince students to study some specialties for which there is a known market need. Students (particularly women) seem to be attracted to the "cleaner" specialties for which there may be less need (such as computer operation) than to "dirtier" specialties, such as machining and agricultural specialties. These choices create skewed supply of labor, but administrators have been reluctant to refuse enrollment to students in the more popular specialties. Physical Facilities 2.5 Civil works, furniture, and equipment were provided to a total of 130 vocational training schools, an increase from the 97 planned during appraisal. In addition to fixed schools, about 80 mobile training units (small trailers that carry equipment to remote sites) were constructed by students in automotive specialties of CONALEP schools, at considerable savings. According to criteria established in collaboration with the Bank, schools have generally been built near industrial or substantive labor-market activities (such as tourism), so that graduates would be absorbed. However, this meant that the less developed areas of the country would have fewer CONALEP schools. Mobile units have served to extend the reach of the schools to other areas. 2.6 The four buildings visited by the audit mission were well maintained, provided a pleasant environment to students, and were well equipped. The equipment financed by CONALEP II was largely found by the audit mission to be still useful and in good working order (with the exception of obsolete personal computers). It is maintained by students and teachers, who thus learn maintenance as well as equipment operation. The apparent good state of the civil works and equipment is important, given the large investment in this component (US$90 million). The self-maintenance strategy, which has been strengthened in recent years, is an important step towards ensuring the sustainability of the investment. 2.7 Linkage committees. To open specialties, schools must do an extensive market study and be able to show demand for at least five generations of graduates. Committees of local businessmen ("linkage committees") meet on a regular basis and advise individual schools on 26 what specialties to teach on the basis of local demand. The schools with the help of committees place graduates in jobs. In every school visited, the audit mission met with 3-6 members of linkage committees. They were representatives of businesses and administrators of organizations which employed graduates. Although they were all very complimentary of the work done by CONALEP, it was unclear to what extent they actually offered sustained leadership and guidance. In one school, for example, they said they met once a month for an hour. In another, the committee had been reconstituted only a month earlier, after a period of inactivity. (It would be helpful if the content of their specific suggestions were researched by CONALEP.) Although the substantive importance of their contribution may not be clear, their participation and support they generate may be to some extent responsible for the general high regard that communities seem to have for CONALEP schools. Curricula and Materials 2.8 In the context of improving instructional quality, the project financed curricular improvements in about 40 percent of the courses. The mission found that sophisticated curricular work had been undertaken. Curriculum guides, for example, gave clear breakdowns of behaviorally oriented instructional objectives and activities for the benefit of instructors, who are all part-time employees. 2.9 CONALEP has considerable technical expertise and develops its curricula and courses centrally. There is certainly a strong argument in favor of this method, since duplication of efforts is avoided. However, individual schools might benefit from more latitude in deciding which courses should be taught. For example, health assistants in the state of Chiapas needed to know indigenous languages, such as Tsotsil and Tseltal. The curriculum, however, specified that they should study English. On another occasion, however, the director of a school was able to substitute an obsolete computer language with a more updated and useful course. 2.10 About 1111 publications were developed, and 6.8 million documents printed, including textbooks, bulletins, syllabi, training materials, guidelines, handbooks, leaflets, extension bulletins, journals, and manuals. The audit mission found in the schools a sample of inexpensive but high-quality textbooks, supportive materials, and curricular guides. There were also many audiovisual materials and functioning equipment. 2.11 While the textbooks are eminently usable, to what extent do students actually use them? This was unclear. Although CONALEP instructors are not professional teachers, they are given latitude in how they teach, and some may choose not to use the textbooks extensively. Because textbooks are optional, students may not buy them, despite the low cost. Administrators said that many students study them in the library, but it is unclear to what extent their probability of passing a course is compromised by the lack of a concrete, readily available set of instructional material. The mission observed instruction in two classes. The textbook assigned to the course was not in use in either; the teacher dictated and students took notes, with considerable wastage of instructional time. Certainly the two observations cannot be generalized, but they are a cause of some concern. 2.12 Textbooks should certainly be set aside if they do not support the instructional objectives of courses, but then a set of notes, articles, or other material should be given to students. After the effort and the expense incurred in developing good-quality materials, it is unfortunate to leave decisions about their use to part-time teachers and students without a good reason. 27 Staff Development and Instructional Delivery 2.13 Under CONALEP II, a total of 17,695 instructors received pedagogical training. CONALEP instructors are professional technicians who can teach only part-time. Few receive teacher training before teaching in CONALEP schools. For this reason, CONALEP makes considerable investment in teacher training. Currently, training is (a) pedagogical and (b) subject-oriented. Discussions with instructors showed that all had received some training, and that some had received extensive training. However, it was unclear whether their teaching performance had improved. The instructors observed by the mission in classroom instruction were clearly in need of practically oriented training in instructional delivery and optimally using class time. They dictated material to students, did not use examples sufficiently meaningful to students' experience, did not ask questions that stimulated the thinking of all students, and tended to neglect the women, who were more reluctant to volunteer responses. Brief and practical teacher training is needed, possibly through videotapes that demonstrate effective and ineffective teaching. 2.14 Students undertake much practical work, which seems to be well targeted towards market needs. In the context of improving quality of instruction, the project also supported pilot production training programs in 70 schools, designed to improve student motivation and reduce training costs. The program produced 62,000 items of furniture, hand tools, and equipment. Because items were made more cheaply by students than by outside vendors, there was a net savings of about US$930,000 for CONALEP. The mission was not able to observe this activity in progress. But it found that students were satisfied with the amount and quality of practical training they were receiving. Student Selection and Training-Social and Employment Issues 2.15 CONALEP's three-year program is still not preferred by many entrants, and there is considerable dropout. The overall completion rate was 40 percent in 1979-86, and had risen to only 52 percent by the end of the project implementation period (as compared to 78 percent expected during appraisal). The dropout rate is particularly high during the first semester, when about 40 percent of students leave. As a result, outputs from the regular three-year program were 40,000 vs. an appraisal estimate of 48,000 although 173,682 students enrolled (Annex 2). 2.16 CONALEP studies show that dropout is due to internal reasons (academic failure, professor inadequacies, inadequate equipment) as well as external reasons: students' economic problems, jobs, inadequate academic and vocational preparation. The latter seems to be of considerable importance. Places in higher secondary schools are scarce, and many students who want to enter cannot. The mission found that many students find it hard at the end of the ninth grade to abandon any hope of going to the university. As a student in a focus-group discussion conducted in a Mexico City school stated, it is difficult to see oneself as someone who has no more hope for secondary education. Therefore, students may register in CONALEP only temporarily and leave when they find a suitably located secondary school that will admit them. This is partly responsible for the high dropout rate seen in all CONALEP schools. However, if students remain in CONALEP for a few semesters, they tend to get over this difficulty and do find professional fulfillment and self-esteem. 2.17 Students and graduates reported that CONALEP instructors and administrators do try to instill professional pride in them as technicians and help them overcome the status difficulty. 28 Training in professional behavior is also beneficial for students from less advantaged background who may lack the social skills required in the labor market. Program features in selected schools include the use of uniforms and instructions to dress up every day as if going to work. However, more systematic work seems needed in the area of professional pride and social skills, because the status problems are important yet frequently unexpressed. 2.18 One concern is that despite the efforts to make students feel better about their vocational choice, subtle contradictory cues are given. The instructors who are technicians are paid less than the instructors who are university graduates. Although many instructors are CONALEP graduates, school administration is in the hands of university graduates. Finally, high-level CONALEP staff are university graduates, although some positions could have been clearly filled by technicians, experienced in hands-on training. It might be useful to eliminate these inequities. 2.19 The mission interviewed groups of persons who had graduated during the CONALEP II period. The graduates interviewed included some who had achieved high-level positions, such as directing production in a large factory. Overall, they reported considerable satisfaction with the training program. They found that the CONALEP program prepared them well and were satisfied with their professional achievements. They rated most instructors as dedicated and knowledgeable in their subject area. One group of graduates, health assistants trained in Chiapas, did express displeasure with their employment prospects, because they had to work as temporary workers with no benefits in health campaigns of rural areas. The problems of these graduates underscored the difficulty of obtaining socially needed positions in public-sector agencies with severe budgetary constraints. 2.20 Many graduates who reach senior positions are unable to progress professionally. They find themselves ineligible to undertake higher studies that would improve their professional status, such as courses in institutes of technology. Therefore, they continue to earn less than less experienced persons with more advanced academic credentials. This is an important limitation that the terminal nature of CONALEP creates, and solutions must be found to deal with it. Gender 2.21 Tracer studies have found that female three-year graduates made up 44 percent of graduates placed in a job vs. 56 percent for men. All CONALEP courses are open to women, who according to the Staff Appraisal Report (SAR), were expected to occupy about 30 percent of training places. This expectation has been met, but women tend to congregate lower-paying professions traditionally filled by women: such as computer operation (informatics) in the three- year courses and industrial sewing in the shorter courses. There were women studying the less traditional and better paid trades, such as machining and electronics, but they were rare. The CONALEP administration is cognizant of the problems women face at work but has not developed in subsequent projects a coherent strategy to deal with them. Staff expressed the view that the position of women is gradually improving in a larger social context and that a wait-and- see rather than a corrective strategy is warranted. 2.22 Some CONALEP schools have been built near areas of indigenous populations. To what extent was CONALEP able to reach these people, who are often very poor, through its regular three-year courses, short courses, or through mobile units? The answer was unclear. Data collected under CONALEP II did not include ethnicity. Since few students from indigenous communities obtain relatively high levels of schooling, the numbers were probably small. 29 Benefits and Risks 2.23 The benefits of CONALEP II, as anticipated in the SAR were to strengthen CONALEP institutionally and to increase the numbers of skilled technicians in the country. The risks were that there would not be enough qualified instructors willing to work in CONALEP and that the benefits of production units might not materialize. 2.24 The expected benefits were obtained. However, the SAR only considered benefits accruing to CONALEP and not how the country would benefit from CONALEP's services. On the other hand, the expected risks did not materialize. Qualified instructors can be obtained in large numbers, in part because CONALEP was able to pay competitive wages, and several full- time technicians enjoy spending some time with students. Also, CONALEP was able to implement innovative components, and thus the expected benefits of the pilot training program production units were reportedly obtained. Results 2.25 The project clearly met its objectives. The neat and well-equipped physical facilities, the positive experiences of graduates and their success in professional development have apparently given CONALEP a good reputation among the general population. By 1995, 360,000 people had received training, and in that year, CONALEP offered courses in all specialties to 212,000 students and 80,000 on-the-job laborers. This achievement is remarkable, given the overall low status of vocational schools worldwide. 2.26 How well have graduates fared in the job market? The PCR states that at the end of the project implementation period, 62 percent of the three-year program graduates found employment within three months after course work completion, and about 84 percent were employed in their area of training specialization. Almost all graduates were employed by the private sector and were earning, on the average, 170 percent of the minimum wage in 1991. The follow-up studies did not provide information about employment stability. However, a survey showed that 92 percent of employers in nine different economic sectors who hired CONALEP graduates noted consistently over a four-year period that the training was good or excellent. At the time of CONALEP II, adequate control groups had not been developed to evaluate the impact of training relative to other training alternatives or to no training at all. This issue was addressed in the follow-up project. 2.27 CONALEP II findings were consistent with earlier findings during the CONALEP I period. The first round of the tracer study, encompassing more than 5,200 graduates who had gone to 18 training centers in three federal states during 1980-85 showed that 63 percent of them were employed. The second round, comprising 6,800 ex-trainees between 1982 and 1986, showed that 59 percent were employed. A simultaneous survey of 600 enterprises yielded a very positive evaluation of the CONALEP training program. 3. OED. United Mexican States: Technical Training Project (CONALEP I). Project Performance Audit Report. 1988. Report no. 7136, p. 7. 30 Institutional Development Impact 2.28 Institutional development impact has been substantial. With the civil works of the first two projects, CONALEP had 252 schools throughout the country in 1996. With the help of 654 staff years of specialist assistance and 76 staff months of short study tours abroad, administrative efficiency has improved. A graduate placement system is generally effective in obtaining jobs for graduates, and the tracer system is able to provide data regarding the work of graduates. The schools visited had competent and educated staff, able on a short notice to organize discussion sessions, invite graduates and industry representatives, and provide data regarding the operation of their schools. 2.29 As a result of its strengthened efficiency and institutional capacity, CONALEP has been able to implement strategic training programs. The PCR states that four innovative pilot programs not anticipated under the loan were directed at marginal populations, micro-industry, the self-employed, and the maquiladora industries. With mobile training units, 70 low-income groups were trained in home maintenance and actually repaired 212 houses. Through supplemental courses, about 9,100 third-year trainees received special training in entrepreneurship, and 103 small businesses had been established by the time the project closed. 2.30 One important and innovative program that started under CONALEP II was the provision on technological services to the private industry (Centros de Asistencia de Servicios Tecnol6gicos--CAST). During CONALEP II, about 30 maquiladora enterprises benefited from training of in-plant trainers. The private sector finances all direct costs and a small part of the overhead. This program has greatly expanded under the CONALEP III in about 30 areas of heavy industry, where just-in-time high-technology updating is important. The mission observed some current activities and interviewed three clients, who expressed great satisfaction with the services provided. An indicator of success and client satisfaction is that training centers which open, are able to become profitable relatively quickly and recover their equipment investment despite its large cost. Sustainability 2.31 Sustainability is likely. CONALEP enjoys strong support from politicians, businessmen, and the labor sector, and has been generally protected from budget cuts. It maintains close links and good standing with employers, providing a good guarantee that it will remain relevant to the training needs of enterprises. Recommendations 2.32 Suggestions specific to CONALEP's operations are: a. Address the reasons students leave the program, such as the low self-esteem that results when students leave formal education. Develop orientation sessions and seminars to discuss the issue early with students who register for the three-year courses. b. Although dropout largely takes place in the first semester, more advanced students leave for various reasons. Diminish the dropout rate by locating 31 them (though the tracer system) and encouraging them to complete their studies. c. Make graduates with considerable experience (for instance, five years in the workplace) eligible for admission to degree-oriented institutions, such as polytechnics. (A possible policy is already under review.) In addition to helping graduates, partly opening the door to the formal system may prevent some students from dropping out. d. Put more emphasis on brief and practically oriented teacher training. All instructors should learn specific techniques to assess student comprehension, encourage participation, optimize use of class time, and evaluate achievement. Teaching techniques are more important when persons with lower levels of education must be trained, as in the short courses that CONALEP has been contracted to teach for STPS. Instructional videotapes showing examples of good and poor teaching can be used to help teachers do their job. e. Develop a strategy to deal with the constricted vocational choices of women in an appropriate cultural context. Strong vocational counseling is needed to focus on women's decisions and reasons for doing so and to encourage them to experiment with professions of high remuneration and market demand. Experiments effective elsewhere, such as training and placing women in groups for non-traditional trades could be tried. f. Take steps to reduce the numbers of applicants to courses of study such as computer operation, for which the market is saturated. At the very least, if students prefer to study already saturated professions, their studies should not be subsidized. 32 3. Manpower Training Project Objectives and Description 3.1 The Manpower Training project was a complex and innovative project, which was prepared on the basis of a subsector study prepared jointly by several government institutions and by the Bank. The project financed scholarships for the unemployed and assistance for micro, small- and medium-sized businesses as well as training and equipment for administering bodies (state employment services, training institutions). In summary, its objectives were to (see detailed description in Annex 1): a. strengthen employment services to: (i) place displaced workers in jobs and (ii) administer short-term PROBECAT scholarships for them (programa de becas para la capacitaci6n en el trabajo); b. assist private enterprises to upgrade the skills of employed workers; c. upgrade STPS staff and strengthen STPS capability to manage information and carry out policy studies; d. increase training capacity of specific institutions so that they could provide training to the unemployed. 3.2 The above objectives were to be achieved through the coordinated implementation of four project components: a. The employment services and labor training component to prepare state employment services to place unemployed workers and administer short- term retraining scholarships for eligible persons. Selection criteria agreed gave preference to unemployed adults with at least primary education; b. The demonstration in-service training and productivity component provided technical assistance to micro small- and medium-size enterprises and enabled many to utilize unemployed scholarship recipients; c. The STPS institutional strengthening component upgraded the skills and logistical capabilities of STPS and its staff; d. The training institutions upgrading component provided equipment and training to selected vocational training institutions that served the unemployed scholarship recipients. 3.3 Total project costs for the Manpower Training Project were estimated at US$156 million equivalent, of which US$80 million was financed by an IBRD loan. The project was approved in October 1987 and the credit became effective immediately. It was completed in December 31, 33 1992 and thus required five years and three months for completion. It was scheduled to be closed by the end of June 1993, six months before the anticipated closing date. Implementation 3.4 The project was essentially carried out as planned. With the exception of the number of enterprises and training participating in mixed retraining programs (which involved preservice as well as in-service), all key implementation indicators were met or surpassed (Annex 3). Due to the acceleration in the implementation of the retraining program and the rapid growth of the CIMO program, disbursements were consistently ahead of forecast. Specifically, achievements were: Employment Services and Labor Retraining 3.5 The project had a positive impact on the state employment services. Staff were trained, and the offices acquired modern equipment and software. According to the PCR, staff became more motivated after training, and their turnover was reduced from 70 percent in 1989 to about 15 percent in 1991. During the implementation period, state employment services served about 254,074 job seekers and 247,463 vacancies per year mainly for skills-related rather than professional positions. The state employment agencies administered the PROBECAT scholarships and chose candidates, generally on the basis of criteria approved by the Bank. 3.6 The mission held discussions with the director and some staff of one state employment center. They confirmed that training had taken place in areas such as computer use, interview techniques, and knowledge of rules. It was reported that tangible increases in productivity had been achieved as a result of computerization of the employment data banks. Improved productivity meant improved services to employers as well as the unemployed. The employment service of Monterrey, for example, was able to place 25 percent of its applicants in jobs. 3.7 PROBECATscholarships. Recipients received the minimum daily salary for a period of one to six months. Training was offered in three modalities: (a) classroom-based training for unemployed people who hoped to get or to start a job, (b) mixed training, where businesses recruited apprentices without a commitment to hire and taught them in structured courses before making hiring decisions; and (c) less structured, on on-the-job training provided by employers who were committed to hiring 70 percent of the appendices. The scholarships were the core of the program and an issue of considerable political importance at a time of financial crisis and large-scale loss of income. Given the institutional capacity needed to carry it out, it was a very bold experiment, which has been honed and refined during the follow-on project. 3.8 By the end of 1992, over 293,217 unemployed workers had been retrained in over 19,939 courses Evaluation data did not include a control group, and it was difficult to assess whether trainees fared better than they would have without the program. About 60 percent of men and 33 percent of women found jobs within three months. Comparison with dropouts showed, however, that trainees not meeting the program's selection criteria resulted in very low rates or return, while targeted trainees obtained benefits far greater than the costs. (See Annex of the Project Completion Report-PCR.)4 4. World Bank. 1994. Mexico: Manpower Training Project (Ln. 2876-ME). Project Completion Report. 34 3.9 The mission held discussions with several groups of unemployed beneficiaries, small businessmen who hired them, as well as workers who received in-service training. (See Box 2 for examples). Box 2: How PROBECAT Scholarships Are Used * A glass company uses the scholarships to hire completely inexperienced male workers and train them. Earlier it had to take new workers as they came, assign them to a more experienced worker, and tolerate inefficiency as the workers learned on the job. With scholarship available, new students come in batches; an off-duty supervisor takes one oven out of service and trains them for specific periods and with a specific curriculum. During the training period the business screens apprentices for future employment. * A company that fabricates cables screens batches of male scholarship recipients who have lower or upper secondary education and trains them systematically. In addition to subject- matter knowledge, students learn about responsible behavior and build a relationship with the company. Both company officials and workers expressed a great deal of satisfaction with the training. * A maquiladora company of electronics parts trains batches of female scholarship recipients. Experienced supervisors train the students. Student progress is carefully monitored, and students are certified regarding the tasks they can perform. With certification, it is clear which workers are qualified to undertake various tasks, and this maximizes efficiency. * The social security hospital in a rural area with a large, poor, and non-Spanish-speaking population used the scholarships to train traditional midwives in safe motherhood. The women, who were illiterate, were brought into the hospital and trained on anatomical models through the help of an interpreter. They expressed much satisfaction with the training. * About 40 indigenous male students in a village of Chiapas received two-month training in carpentry through a mobile unit contracted from CONALEP. Several months later, the mission found that 11 of them had started a small workshop to process into simple furniture the wood that was cut down. They had obtained tools by petitioning for a local funding source and were already obtaining orders. They felt they needed more training, but they were not eligible for another scholarship. * The women of the above village were trained in sewing while men were trained in carpentry. But they have been unable to work because they did not have sewing machines nor readily available material. Some were of the opinion that they should have received sewing machines in lieu of a scholarship. The demonstration in-service teaching and productivity component (CIMO) 3.10 CIMO was a pilot program that was implemented in 30 regional units. It worked directly with over 32,304 separate enterprises and trained more than 124,182 workers through 10,644 courses. Appraisal targets were 20 regional units providing service to 5,000 enterprises and 75,500 workers through 8000 courses. The pilot proved very successful and was greatly expanded under the follow-on project. 35 3.11 CIMO is a broker and facilitator rather than a direct provider of services. It provides management consultant services and technical assistance in product development, marketing, business management, and accounting. Repeated visits by CIMO consultants help diagnose problems and implement plans. Training is a core activity of the beneficiary enterprises, many of which also hire students with PROBECAT scholarships. Courses are developed specifically for the needs of a business, and suitable instructors are located. Businesses finance 30 percent of the direct cost of training. (Small businesses of the same specialty may be grouped together if no conflicts of interests arise.) The mission saw several examples of this work, such as: * A restaurant owner who saw his clientele diminish with the economic crisis, got help from a consultant to restructure operation times, the menu, and obtain feedback from clients. He also got several PROBECAT scholarship recipients, whom he is training in an apprenticeship course to be waiters and bartenders. He found the advice satisfactory, and his business has improved. * A paint company, which hired employees with rather low levels of education and social skills, used training courses to teach them the basics of the work planning but also skills like conflict resolution. It observed a significant improvement in productivity and relations among employees. Discussions with employees indicated that they many remembered the course content and had found it useful. Others did not remember having taken a specific course. 3.12 CIMO has enjoyed support from micro, small, and medium-sized enterprises. An evaluation study concluded that: (a) CIMO's greatest strength is the close relation it has established with enterprises, and (b) the program can contribute substantially to fill some gaps in supporting enterprises. CIMO has demonstrated the ability to reach a large number of enterprises in a short time, using a very light administrative structure. STPS Institutional Strengthening Component 3.13 The institutional strengthening component has: (a) expanded STPS's program of labor market surveys and studies (7 completed surveys and 16 studies) and (b) trained 1041 STPS staff; (c) provided computers and software to the state employment service offices in state capitals; and (d) supported the development of a computerized market information system that became operational during the follow-up project. 3.14 Discussions with officials regarding this component confirmed that the training took place as planned and that the studies were conducted. The audit mission was able to review a number of the studies, which show considerable technical sophistication. However, the extent to which their findings were used to improve program delivery was not clear. 3.15 This component was also expected to develop a service to make information retrieval easy at various levels. An example would be an information request by a small company on how to encapsulate a vitamin. The information component has not worked well. There has been little demand for it, and updated information has been difficult to maintain. It seems like a very useful service, but it was probably prematurely provided. 5. Maza, Antonio. 1992. Evaluation ofSupport Mechanisms to Small and Medium-sized Enterprises in Mexico. World Bank. 36 Training Institutions Upgrading Component 3.16 This component has been successful in increasing training capacity in specialties of high demand and in new specialties and in improving the quality of training and administrative efficiency of 445 participating training institutions and 1,374 specialties. About 134,000 trainees were enrolled in preservice training courses, 42,647 trainees participated in in-service courses, and 24,217 participated in retraining program activities by 1991/92. The PCR states that the program has been received very well by the staff of participating institutions. Enrollment statistics showed that project-supported training institutions and programs had an 18.7 percent increase in enrollment in 1988-91, compared to an 8.4 percent overall increase in all comparable public-sector institutions. 3.17 The mission visited three institutions (Centros de Capacitaci6n de T6cnica Industrial- CECATI) which train students with primary-level education and which had received equipment under the this component. Equipment was found to be in working order and appropriate for the needs of courses. On one occasion only, one piece of automotive equipment was slightly outdated and unusable for recent models of cars. A number of instructors reported having received some in-service education. In interviews, course participants indicated satisfaction with the quality of training they were receiving. 3.18 Discussions with administrators brought to the fore problems regarding demand and supply of training that have been difficult to resolve. The demand for courses seems related to their perceived social status. Only the persons from the poorest families are eager to be trained in manual work, such as building trades. The ones who have more education or are from better- off families prefer courses that are related to services rather than to harder manual labor. Since students are free to choose what to study, an oversupply of service-related specialties is created, while the demand is in trades of less appeal. In effect, some PROBECAT scholarships are being used to create structural unemployment. 3.19 This opinion of administrators was disconcerting. A basic premise of the scholarship program was poverty alleviation. It appears, however, that scholarship recipients are making choices more closely related to an increase in social status than to maximization of income. Contrary to expected benefits as shown in the SAR, anecdotal evidence suggests that the STPS projects are not reaching the poorest of the unemployed. However, the magnitude of the problem has not been measured. Instructional Delivery 3.20 STPS, the ministry implementing the Manpower Training Project, did not directly teach any courses. It subcontracted training to CONALEP and to other private and public institutions. 3.21 Since the core of the STPS project consists of training, one would have expected to find monitoring of instruction and learning outcomes, particularly for courses given outside the direct control of CONALEP's in-house programs. However, the mission found that little or no attention was paid to this issue. In contrast to the attention paid to earning outcomes, the mission found no studies dealing with training processes and outcomes. 3.22 Systematic supervision of instruction is considered obligatory in other areas of nonformal education, such as literacy. Local STPS staff paid visits to employers' training sites 37 to ascertain that the scholarship recipients were indeed there, but the quality of instruction was not supervised. Lack of substantive supervision in the STPS projects raises questions regarding the curriculum and frequency with which classes actually took place in more isolated areas and informal environments. The mission could not verify to what extent classes took place as scheduled. All visits were announced, and it was not possible to find out what the training environment would have been on a normal day. 3.23 The mission did not conclude that training was deficient, but there was evidence of imperfection. The shorter courses observed by the mission, which catered to people less experienced in learning material, lacked manuals, and two instructors who were interviewed were not clear about the curriculum they were teaching. In some sites, the mission received assurances that technicians who taught trainees were trained in basic pedagogy, but direct questioning produced unclear results. Some students stated that the training time (e.g. carpentry in two months) was too short for them to learn all they needed to know. 3.24 The instructional issues certainly deserved some investigation because there are important problems in the training of less educated and less articulate individuals. (See recommendations section.) Studies on such subjects have been conducted elsewhere, and one would have expected the Bank supervision missions to request a literature review and provide it to the government. But no one knew of such efforts. Needless to say, insufficient training would be expected to influence subsequent earnings adversely. It seems risky to invest so much in training without a basic understanding of how well courses have worked and to what training influenced subsequent work behavior. Attention to Rural Areas 3.25 Though its objectives did not explicitly say so, the Manpower Training Project targeted urban areas. Only seven percent of the investment went to rural areas, although that is changing in the follow-up project; in 1996, 20 percent is expected to be directed to rural areas. One government official explained to the mission that implementation capacity of STPS in rural areas was extremely limited, and the ministry wanted to ensure that the funds were wisely spent. A question arises as to whether other agencies (such as the National Institute of Adult Education, an organization active in rural areas) could have taken on the responsibility, but apparently this alternative was not considered earlier. 3.26 Although a mode for training people in rural areas who have not completed primary education was not found, a need was clearly there. State STPS officials in Chiapas had difficulty finding primary-school graduates, as the Bank's criteria specified. To train people who did not have primary education under the follow-on project (Modernization of Labor Markets), the government had to open a special non-Bank financed category of trainees, with much more limited funds. During the time of the last economic crisis, the Bank did agree to finance training for this category. (It was, thus, possible to offer scholarships to participants like illiterate traditional midwives.) The Bank's criteria had been developed on the basis of general research findings, and it might have been wise to apply a more flexible local policy in certain areas. 3.27 Indigenous people. It has been difficult for STPS to find effective training solutions for indigenous people. Options seem limited. The audit mission found out that there were no STPS or CONALEP staff of indigenous background in Chiapas, where training of indigenous people 38 takes place in several areas. It might be useful if the services of a knowledgeable anthropologist were used and if culturally appropriate social assessments were conducted. Box 3: Poverty Discrimination? After careful screening, a sheet metal factory selects batches of young males for apprenticeship through STPS scholarships. By mid-1996, they had trained and hired 11 batches of employees; for the last batch they had received 230 applications, from which they selected 30. The selection criteria presented to the audit mission were as follows: a social worker visits the families and notes favorable and adverse factors: education level, personality characteristics, previous job performance, human relations, malnutrition, living in a cramped space, presence of a father, family illnesses (hypertension, heart disease, drinking, smoking), moral problems. Data obtained from the family are validated with neighbors. Factory employees emphasized that the presence of one or two adverse familial characteristics does not necessarily disqualify a candidate. Certainly, employers want workers who will give them the least trouble. However, the health and quality-of-life factors considered adverse by the factory are the exact problems of poverty that Bank strategy has been trying to alleviate worldwide. It would appear that Bank funds in this case have been used not to alleviate poverty but to bypass the poorest. Gender Issues 3.28 PROBECAT scholarships were given to the unemployed according to the wishes of employers. Gender was found to be inextricably linked to hiring decisions. Eight of ten employers interviewed by the mission expressed clearly stereotyped views regarding the work men and women are capable of doing and clear gender preferences. (Exceptions were a restaurant and a public hospital.) They hired women as sewing machine operators, assemblers of small electronic parts, secretaries, waitresses, quality-control inspectors. They hired men for most other areas, particularly work that involved manipulation and repair of machinery, exposure to heat, and lifting weights (even light weights). For example, a company that made cables obtained PROBECAT scholarships for groups of males who would operate equipment and lift small rolls of cables; they had never tried women for the job, but were of the opinion that women would not be as good. A glassmaking company only hired men for production and women for quality control. 3.29 Women's vocational choices were found to be very limited. Scholarship recipients largely expressed the desire to study traditionally female areas. Women interviewed by the mission stated lack of interest in the more male-oriented fields and inE ility to find a job in them. There have been several cases of women studying specialties such as refrigeration, lathes, and machine repair, but they were exceptions. For example, a widow whose husband had a workshop was trained to operate it. In areas like Oaxaca, where many men migrated to the US, women had to take over the more traditionally male jobs, and STPS staff reported several in carpentry and plumbing. 39 3.30 As a result of choices and hiring patterns, a great deal of women's training has been in sewing. In addition to garment industry jobs (which offer very low salaries), STPS and CONALEP staff felt that if poor women could sew, they could sew the clothes of their families as a cost-saving measure and also make items for sale to tourists. But is this what happens in fact? Most women do not have sewing machines, ready-made clothes are relatively cheap, and the cost of material may leave women little margin for savings or profit. Furthermore, it is unclear whether women learn to sew items that are salable. The mission observed training provided by CONALEP mobile training units in Chiapas to young, rural, indigenous women who wear traditional costumes. They were learning to sew western-style blouses. The teacher explained that they already knew how to make their indigenous clothes. But then what was purpose of the training? If it was income-generation, where was the relevant training? In a nearby market, many women were selling indigenous clothes to tourists, and the ones who knew better to approximate the tastes of the tourists could earn more. But trainees did not learn how to refine their traditional skills for market use, nor did they learn marketing strategies. Clearly, the income-generation potential of sewing needs much study, which has not been conducted. 3.31 Indigenous women living in their traditional environments were found to face particular difficulties in obtaining training, given: (a) their low levels of education, (b) limited income- generation options in their native environment, and (c) the rigid occupational gender segregation 6 that traditional societies tend to impose on their members. A large issue arises: should these women be trained for occupations of low earnings in a monetarized economy or should they be taught nutritional and productivity skills (such as home canning of vegetables) in order to feed their families more effectively? Answers are difficult to come. There are many in Mexico who lament the demise of indigenous societies and the conversion of Indian women into servants for the houses of the better-off. Neither the audited project nor its follow-up have yet dealt with these issues. Benefits and Risks 3.32 The benefits of the Manpower Training project, as anticipated in the SAR, were: (a) reduction of human resource constraints to growth through adaptation of trained manpower, so that supply can meet demand; (b) increased training opportunities and more equitable access to training and employment across regions and social strata through provision of expanded retraining for the poorest of the unemployed (including 27 percent of women); and (c) higher productivity and product quality in the formal economic sector through an increased supply of skilled workers. 3.33 Certainly, the project has done much to match supply of trained manpower with demand. However, the benefits expected for the poorest of the unemployed, particularly women, have not materialized. The urban focus of the project and its criterion of primary-school education meant that the project could not serve the poorest segments of Mexican society. 3.34 The risks of the Manpower Training project, as anticipated in the SAR, were: (a) government inability to finance counterpart funds; (b) change of government priorities; (c) 6. Work performed by the men of a community is often seen by women as undignified and unfeminine, and women will do it only as a case of last resort. Poor, less educated people are more likely to have rigid gender stereotypes and to discourage, ridicule, or harass non-conformists (Martin, M. Hard-Hatted Women. Seattle, Washington: Seat Press, 1988.) 40 failure of graduates to find productive employment, and (d) failure to reach an adequate number of employers. The risks did not materialize. The project and its successor have enjoyed broad government support, and employers have taken advantage of the benefits offered to them. Outcomes 3.35 On the basis of the PCR data and sample visits by the mission, the project is considered to have met its objectives, and its performance is considered satisfactory. It succeeded in strengthening employment services at the state level; assisted small private enterprises to upgrade the skills of their workers, although the extent to which this was accomplished was limited vis-i-vis existing needs; strengthened the market monitoring capability of STPS; and upgraded institutions that provided training in critically needed specialties. 3.36 Project outcomes, however, are quite ambivalent. The PCR provides an impact study of participation in the PROBECAT program,' which was the most important component of the project. The study found that: a. compared to a control group, trainees on average tended to find jobs more quickly; participation in PROBECAT reduced the mean duration of intervening unemployment spell by 2.5 months for males and by 1.9 months for females. The strongest effects are for trainees over the age of 25. (However earlier studies showed that only about 35 percent worked in occupations related to their training.) b. Male trainees are more likely than untrained males to be employed in the short term (i.e. three and six months after training than controls), but the effect does not persist after a year. c. Female trainees with work experience are more likely to be employed three, six, and twelve months after program completion as compared to controls. Those without prior experience are less likely to work, and the program appears to draw in a number of women who would have otherwise remained outside the labor force. d. Male trainees are more likely to find work in large firms than comparable controls. Larger firms are known to offer higher salaries and better benefits, and women are less likely to work for large firms. e. Training increases the monthly earnings of male trainees, but this effect is largest for males with 6-12 years of schooling. Training does not appear to raise the monthly earnings of women. f. The monetary benefits of training outweigh the costs of the PROBECAT program for certain groups of trainees. For males over age 25 with prior work experience, the benefits of participation outweigh costs within three 7. "The Impact of Mexico's Retraining Program on Employment and Wages." In Mexico: Manpower Training Project. Project Completion Report. no. SexM9418, January 5, 1994. 41 months of starting employment. For all other males, benefits outweigh costs within one year. For women younger than 25 and without prior work experience, the costs outweigh the benefits. 3.37 Earlier evaluations (e.g. Fields 1990) argued that the program does not produce occupational upgrading. Most people were found to work at the same level before and after training, but there seemed to be an overall increase in salaries. Self-employment was generated, but to a limited extent. The project may have indeed given many of its scholarships on people not ready to enter the job market in the short term; 68 percent of those who were not working were students or housewives. The recommendation from this finding was that such people should be prevented from getting scholarships. However, this policy may be short-sighted; women may enter the work force once their children are grown or when their financial conditions deteriorate. To prevent them from training at a time when they are not ready to enter the job market may prove detrimental to their well-being in the long run. 3.38 The benefits and sector strategy sections in the Manpower Training project raised expectations for attention to the very poor who, however, were not clearly targeted in the operation. Did the project alleviate poverty among its beneficiaries? Two Bank-financed studies conducted in 1990 and 1991 concluded that it did. The trainees earning the least made the most and largest income gains, while those earning the most actually suffered income losses.8 However, this finding appears to be an artifact of statistical analysis, called regression towards the mean,9 and it cannot be stated with confidence that the training had a poverty alleviation effect. If it is true, as Fields (1990) states, that "this is the strongest evidence available that the training program has had a beneficial effect in reducing poverty in Mexico", the evidence is very weak, indeed. 3.39 Ambivalent outcomes have been found in government-financed vocational training 10 projects of other countries, particularly when the trainees are unemployed. But it is premature to attribute the ambivalent findings to an ineffective methodology. It is possible that instruction for the short, less formal courses was deficient or even nonexistent. Since learning processes and outcomes were not monitored in Mexico, it is impossible to ascertain where the process went wrong. It was unfortunate that the Manpower Training project and its follow-on have not obtained the data that would help clarify this issue. 3.40 Outcomes of the CIMO program. A case study of 30 enterprises participating in the program concluded that enterprises had indeed increased productivity, better access to markets, and had better labor conditions and organization. An important lesson that CIMO learned was 8. Carlson, Samuel. 1991. Mexico Labor Retraining Program: Poverty Alleviation and Contribution to Growth. Latin America and the Caribbean Technical Department: Regional Studies Program, Report no. 6, World Bank, Washington, DC. Fields, Gary S. 1990. Mexico Manpower Training Project: Further Evaluation Findings. Internal Bank Document 1990. Vol. 4, Loan 2875 project file (p. 17). 9. Cook, T. and D. Campbell. 1979. Quasi-Experimentation. Rand-NcNally. Multiple measures of income would be needed to determine a true effect, given the tendency of employment instability among poorer Mexicans. 10. Zabalza, A. 1996. Is There a Case for Government Intervention in Training? Human Capital Development Working Papers no. 65, March 1996. Higher levels of education seem more useful in obtaining gainful employment. "Training and Jobs: What Works?" The Economist, April 6, 1996. 42 that training must be given exactly in the areas of need. Generic pamphlets and seminars seem ineffective in training people. 3.41 One accomplishment of this component is that it has helped change the attitude regarding training among its beneficiaries. Whereas many small businessmen found it of questionable use earlier, they were convinced of its utility when they saw results. One small businessman even said that the entire company strategy has changed since they realized the benefits of training. Much more investment is now directed in this area. 3.42 One drawback of this effective and innovative component is that it can only help relatively few businesses. Of the approximately 380,000 businesses in Mexico, the Manpower Training project could reach about 5000 and the follow-up project could reach another 10,000. The service provided to the country does not even reach five percent of the potential demand. The program was meant to demonstrate the utility of training and consultation rather than reach businesses nation-wide. However, it appears that much more needs to be done before a demonstration effect can be realized on a national scale. As it proves its effectiveness, it is unclear how to expand it without creating a large bureaucracy. Sustainability 3.43 Sustainability is likely. Through its institutional development components the Manpower Training project has touched hundreds of thousands of beneficiaries. It is probably too early to ascertain whether the achievements of the Manpower Training project and its follow- up project will be translated into sustained economic development. Institutional Development 3.44 The audit agrees with the PCR that institutional development impact has been substantial. Several project components strengthened institutional development at the federal and state levels through staff training, labor market information systems, 16 labor market studies, capability to search for jobs and place the unemployed, and upgrading of training institutions. 3.45 The Manpower Training project was quite complex and innovative. It built an institution and at the same time provided services to a large and disparate number of beneficiaries and businesses. The fact that this project was executed ahead of schedule and did reach a large number of beneficiaries is compelling evidence of the institutional development that some government agencies have undergone. Recommendations Specific to STPS Operations a. Given the ambivalent effects of the PROBECAT scholarships and the difficulties of government-financed training world-wide, the issue arises as to whether the large numbers of scholarships given to the unemployed should continue. In the follow-up projects, it might be more cost-effective if scholarships were given only to persons who found potential employers willing to train them. b. To safeguard the investment, the learning process and outcomes must be monitored. STPS should study and supervise instruction much more 43 closely and search for modes of delivery such that the less educated population can remember the material. Issues to study could be: * How best to impart psychomotor skills to people of relatively low education and low verbal ability? What percentage should be done through verbal means? * What training is found to actually change behavior? How effective are the frequently used verbal means of training? When people say they do not remember what they studied, does this mean that they did not change their behavior? * What behavioral change does an extra course (like the man financed by the project) produce among educated ministry staff, who have already taken many short courses? c. To increase the benefits that women obtain from project opportunities, STPS could: i. use scholarships as incentives to get groups of women into more professions and thus combat gender discrimination; ii. study the income-generation effects of the typically female occupations, and iii. study the international literature on non-traditional employment for women and experiment with successful examples. d. STPS should focus more on the needs of the rural poor, as already planned. To understand better the needs and vehicles for reaching the indigenous people, anthropological approaches should be used and social assessments should be conducted. 44 4. General Finding, Issues and Lessons Borrower Performance 4.1 In both projects, borrower performance was satisfactory. CONALEP proved proficient in project planning and implementation and showed competence in project management, despite a slow-down period at a time of large-scale government change. All components were implemented efficiently and effectively, and (according to the PCR) appropriate staff were deployed as necessary. The Manpower Training project was implemented by a highly competent, enthusiastic, and hard-working management group. Its components were executed and completed ahead of schedule, and outcomes were studied extensively. The excellent implementation performance of this complex project attests to the exceptional quality of the STPS management at all levels. It was significant that the project enjoyed strong support by the government, and its procurement needs were fulfilled on a priority basis, enabling staff to concentrate on the substance. Bank Performance 4.2 For both projects, the Bank's performance was adequate during project appraisal and supervision. Communications between Bank and counterpart staff were quite open and positive. There was considerable staff continuity throughout the years, which facilitated implementation of the difficult and complex Manpower Training project. However, supervision time was limited for projects of such magnitude. CONALEP II received only 12 supervision missions in six years, averaging only 6.2 staff weeks in the field every year. The Manpower Training project received more attention. Eight supervision missions were conducted, which consisted of 2-5 members for about two weeks in the field. 4.3 The PCRs describe the Bank as having played a facilitating rather than a leading role in supervising these projects, in part because the implementation units were very competent. During supervision, staff largely worked with counterpart officials in central offices and made relatively limited field trips. For example, no STPS or CONALEP official in San Crist6bal de las Casas in Chiapas recalled having received a supervision visit from a Bank mission. Bank Staff involved in the projects were of the opinion that the Bank's fiduciary role is often so time- consuming that it gets in the way ofsubstance. The concentration on procurement and research issues may have contributed to the lack of understanding regarding the inadequate effects of the projects on the poor and the women. 4.4 Government officials expressed satisfaction with the services the Bank had provided in the supervision of the two projects as well as with the quantity and quality of its staff. However, they expressed dissatisfaction with the recent rate of staff changes and a strong desire to maintain task manager continuity. It takes task managers a while to learn the complexities of the project and the country, and counterparts must make extra effort and time to debrief them. They find it counterproductive to start with a new task manager every three years or so. 4.5 Although, the Bank's performance in the implementation of these two projects is rated satisfactory, the mission identified important issues with respect to the Bank's sectoral strategy. These are discussed in the next section. 45 Findings and Issues Women's Employment Difficulties Were Neglected 4.6 As early as 1990, Bank studies showed that PROBECAT scholarships did not improve women's earnings and employment prospects, in part due to gender discrimination practiced by employers. This was particularly important in light of the fact that gender discrimination in most cases is illegal according to the labor law of Mexico.II Given the Bank's commitment to women's welfare, one would have expected Bank supervision missions or the follow-on project preparation to remedy the problem. In light of the operational directive on gender dimension of development (OP 4.20), the Bank had options, such as: * obtaining expertise by inviting women-in-development specialists on mission, who could advise on applying in Mexico innovative approaches that have worked elsewhere;12 * setting aside scholarships to hire women in jobs stereotyped for men and use them to encourage employers to find out women can do the work; * deciding not to finance scholarships for persons chosen by discriminating employers, just as the follow-on project decided not to finance scholarships for persons without primary school diplomas. The government would then finance gender-segregated scholarships from its own budget, if it chose, as it chose to finance them for persons without primary education. 4.7 Instead, no actions were taken. No lessons even appeared in the PCR about this problem. The SAR of the follow-on project included a reference to the need for removing labor market barriers for women (p. 47 para. 3.35) and planned an investigation into the issue (Annexes 1 and 5.4). Four years into implementation, however, no progress had been made on such an action plan. The audit mission discussed the problem with STPS management, and considerable interest was shown in finding solutions. Subsequently, STPS proposed a plan of action to an April 1996 supervision mission. By then, however, Bank proceeds had been used to perpetuate discriminatory and possibly illegal hiring practices for nearly a decade. 4.8 The women's employment problems in the project highlight a dilemma. Is it acceptable from a development perspective to finance scholarship schemes that involve clear gender discrimination and practices that keep women at the bottom of the income scale? One can argue that Mexico's financial crisis was so large, that discrimination against women has been a low- priority issue. Is this, however, a valid argument for a development institution that prides itself on its attention to women's issues? 11. "Queda prohibido a los patrones: Negarse a aceptar trabajadores por raz6n de edad o de su sexo." Article 133 (I), Ley Federal del Trabajo, I Ia. Edici6n Actualizada, Secretaria del Trabajo y de Previsi6n Social, November 1994. Exceptions are made for pregnant women in certain dangerous professions and nightly industrial work. 12. It has been found, for example, that women can be trained and work in traditionally male occupations if they are kept in groups and if they feel protected from male harassment. (Abadzi, H. Nonformal Education for Women in Latin America and the Caribbean: Solving the Mystery of the Unreported Trainees. ESP Discussion Paper Series no. 19, February 1994.) 46 4.9 According to OP4.20, the Bank is to assist its member countries to design gender- sensitive policies, modify legal frameworks to improve women's access to assets and services, and finance programs for women. Should the Bank also do the reverse, that is refuse to finance programs that involve gender discrimination? High-level decisions are needed. Technical Expertise Was Limited 4.10 CONALEP II was appraised and supervised by vocational educators. The Manpower Training Project and the follow-on projects in recent years, however, were mainly prepared and supervised by economists. Although the crux of the Manpower Training Project was transmission and elaboration of information (sometimes to people of limited education, who face special constraints) no specialist in adult education was involved in the projects. Lack of technical expertise in the Bank missions was matched by lack of expertise among STPS staff; The mission did not meet an STPS staff member with educational expertise. 4.11 Management by economists resulted in much emphasis on external efficiency issues and none on internal efficiency. The project was operated as if training was perfectly efficient and needed no monitoring. Inattentiveness to learning outcomes has made it impossible to link cause and effect in studying earnings outcomes. One wonders whether the Bank has financed projects in any other sector with so little sectoral expertise. The Social Sector Strategy Was Inadequate 4.12 The Bank appraised five vocational-technical education projects before any other social- sector projects were implemented. To justify its largely urban industrial and services oriented operations, the Man ower Training Project SAR promised in 1987 poverty-oriented action in separate operations. 4.13 There is certainly nothing wrong with a series of projects that catered to the lower- middle classes of urban areas in Mexico, if other interventions catered to the poor. But the promised complements did not materialize until 1992, when a primary education project became effective (Loan 3407-MX). This delay was not without consequences. People in the poorer areas, who lacked more effective access to education, did not have the minimum qualifications to benefit from the vocational projects; as mentioned earlier, STPS had difficulty finding adequately educated people for PROBECAT scholarships in states like Chiapas, the site of the 1994 Zapatista insurgence. One justification for the training provided to young people by CONALEP was the country's high population growth. It would seem, therefore, that a population project (or component in a health project) could have been prepared in conjunction with CONALEP projects. The paucity of needed social interventions calls into question the wisdom of implementing the vocational projects without support from other operations. 13. The section on Bank assistance strategy (Manpower Training project SAR, p. 8) states that: "Separate operations are anticipated to strengthen adult education and training in rural areas... The on-going and proposed operations described above, combined with anticipated studies of agricultural training and adult education (within a more general appreciation of the entire education sector) and substantial sectoral training components included in other operations (in agriculture, transpor, industry) would reduce human resource constraints in Mexico on a broad front and would be an essential comp!cmernt to al array of Bank-assisted projects aimed at economic restructuring and policy reform." 47 4.14 Why did the Bank invest so little in the social sectors in Mexico until the 1990s? The deficiency seems to have resulted from an interaction of government and Bank priorities at the time. Mexico, a large oil-producing country with a problematic economic history, preferred to borrow only for structural adjustment and infrastructure, and the priority of the Bank was to help '4 the Mexican economy recover and invest in sectors that produced growth. Poverty issues were not viewed as important at that time, a position that later engendered criticism against the Bank. Furthermore, like many other countries, Mexico did not want to borrow for primary education. One explanation given to Bank staff by some government officials was that their constitution prohibited borrowing for projects that were not directly productive. (A few years later, however, the constitution was reinterpreted.) The Bank wanted to develop a good working relationship with Mexico and respected the government's priorities. At the same time, Bank staff (whose numbers and specialties were sharply reduced after the 1987 reorganization) were concerned with some practical difficulties in investing in education, such as the special interests of the powerful teachers unions. 4.15 As a result of these factors, the Bank did not show as much leadership as it could have in the social sectors. It emphasized the economic aspects of the projects and monitored productivity and neglected issues of women, the rural poor, and the indigenous populations. Worse, some Bank financing has been used to bypass women and the poor for employment (examples in Boxes 2 and 3). It can be argued that Bank acted much like a commercial bank, with an interest in the economic viability of its investment, and without the poverty alleviation concerns that a development Bank should have. The "facilitator" role that the PCR ascribes to Bank performance can be interpreted as attention to short-term implementation events rather than the larger social issues of the projects. 4.16 Could the Bank have held a more focused dialogue on the social sectors circa 1987? Although the government independently decided to carry out structural adjustment, Mexico needed the Bank's help; dialogue has achieved much in other countries. India, for example, did not want to borrow for primary education until 1993 and had three vocational-technical education projects appraised in 1989, 1990, and 1991. At the same time, however, five IDA- financed projects focused on family welfare, nutrition, population, and indigenous populations. Although political and economic conditions may differ vastly, poverty-oriented strategies were being agreed with other large and reluctant borrowers the same time that the Mexico vocational lending program was developed. 4.17 The Bank's insistence on social-sector interventions seems to have paid off, and Mexico's social-sector strategy has become markedly reoriented in recent years. Both the government and the Bank have refocused on poverty and have set the stage for several innovative projects targeting the poor through education, health, and agriculture. But one cannot help but wonder whether earlier implementation would have brought greater cumulative gains to the population. This question brings some difficult issues to the fore that have not been resolved. It also brings to the fore unpublished OED research findings, which show that the Bank tends to have less influence in larger countries. How intensive should dialogue be with governments about social sectors? Should adjustment lending be conditional on a satisfactory social-sector policy? What combinations of lending instruments, staff specialties, and dialogue patterns must exist so as to maximize the poverty alleviation? These questions require high-level answers. 14. OED Study of Bank/Mexico Relations, 1948-1992. OED 1994. 48 Lessons * Studies of women's earnings and calls for action plans may be insufficient to bring about improvement. Expertise, persistence, follow-through, and dedicated time during supervision missions are also needed. * Projects that require a minimum level of education may be implemented most effectively through the support of operations that provide the poor with the required education. Projects which have minimum educational requirements are not likely to reach the poorest of the poor, who lack the requisite level of education. * The economic viability of vocational education projects is likely to suffer if they are developed and supervised with scant technical expertise. Economists must work closely with technical experts. * Terminal diploma courses in many countries suffer from low quality and low student demand. However, the success of CONALEP shows that it is possible to deliver quality education to large numbers of students. Implementing agencies must create a positive image for students and effective linkages with local enterprises. * Indigenous groups may be difficult to reach through projects designed for the general population. Anthropologically sensitive methodology and social assessments would be needed to maximize the benefit these groups can receive from Bank investments. 49 Annex 1 Manpower Training Project-Objectives and Design 1. The objectives of the project were to: (a) strengthen employment services provided at the state level and implement, as a transitional measure during the restructuring process, an expanded and improved retraining program for the displaced and unemployed workers, using existing training institutions; (b) assist private enterprises, especially small-medium enterprises in priority sectors and geographical areas to upgrade the skills of employed workers by introducing a pilot demonstration, promotion, and assistance program for in-service training; (c) strengthen the Secretariat of Labor and Social Welfare's (STPS) labor- market monitoring capabilities by upgrading of staff, improvement of its information system, and carrying out policy studies aimed at guiding the future development of the training system, and: (d) increase training capacity in critically needed specialties by upgrading selected public and private training institutions through the provision of complementary equipment linked to a commitment by participating institutions to improve training operations and facilities maintenance. 2. The above objectives were to be achieved through the coordinated implementation of four project components: (a) The employment services and labor training component was designed to (i) improve and expand employment services through the training of 400 staff and by financing data-processing equipment and related technical assistance; (ii) strengthen the planning, content, and implementation of STPS's retraining program (PROBECAT) as a transitional measure and establish systematic arrangements for monitoring and evaluation of the program's effectiveness; and (iii) provide improved retraining for about 160,000 economically disadvantaged and displaced or unemployed workers over 4.5 years, including financing of trainee stipends to be awarded in accordance with criteria agreed with the Bank (emphasis on unemployed adults with at least primary education); (b) The demonstration in-service training and productivity component was to introduce on a pilot basis a system for promoting and supporting in- service training and related services among enterprises, especially small- medium enterprises in priority economic sectors and regions. The component included measures to: (i) establish within or linked to 50 Annex 1 selected association of employers, 20 small regional in-service training promotion units. The 20 promotion units were to target specifically 5000 small-medium enterprises (out of a total of about 400,000 in the Mexican formal economic sector). A small central support group, called central in-service training promotion unit was to be established in STPS to coordinate the program. All units services were to be provided by consultants contracted for a fixed term. Loan funds would finance the program on a declining scale to encourage efforts to make the program self-financing through gradually increasing charges for advisory and training services; (ii) provide direct support for the organization and delivery of intensive, short-term training for about 75,500 employees in the 5000 targeted enterprises; and (iii) initiate promotion services for in- service training in these enterprises through dissemination of information on the labor market, productivity issues, and training opportunities to 20,000 additional small-medium enterprises employing about 800,000 workers; (c) The STPS institutional strengthening component was to strengthen STPS capabilities by: (i) upgrading about 100 staff of the Secretariat in areas of program design and evaluation, labor market analysis, and productivity issues; (ii) developing improved labor market information systems at the federal and state levels, including computerization and related training, and (iii) undertaking priority studies; and (d) The training institutions upgrading component was designed to enhance the capacity and utilization of existing training institutions by providing financing to some 500 training providers (later changed to 1000 training programs) for modernization or upgrading of equipment in existing facilities. Participating institutions were to be selected on the basis of agreed criteria, including a record of having provided efficient services in the past, and commitments to upgrading of curricula, implementing maintenance programs, and securing competent staff. 51 Annex 2 Project Results: CONALEP H (Direct Benefits ofProjectfor Total CONALEP System) Estimated at appraisal Actual benefits CONALEP I andfI CONALEP I and H CONALEP II combined CONALEP II combined Civil Works Training centers 97 219 130 252 Mobile training units 4 4 Units under construction 15 15 Furniture, equipment and materials Purchased (%) 100 100 100 115 Installed (%) 100 100 100 96 Students and Instructors Students Enrolled: Regular Programs 72,000 160,000 85,682 173,682 Up-grading Program 18,000 40,000 43,466 65,466 Totals 90,000 200,000 129,148 239,148 Instructors: No. Employed 7,500 16,700 6,050 12,770 No. of Hours/Week of (10) (12-14) Teaching Receiving pedagogical 10,000 27,695 training Efficiency and outputs Completion rate % 78 52 Annual outputs Technicians and skilled 21,600 48,000 13,982 40,382 workers (3-year programs) Worker upgrades 16,200 36,000 21,140 40,950 Totals 37,800 84,000 35,122 81,332 52 Annex 3 Project Results: Manpower Training Project Estimated at Actual Accomplishment actual Indicator appraisal (12/31/92) divided by estimate % A. Employment Services and Retraining 1. No. of State employment agency' 400 953 238.3 staff upgraded 2. No. of Trainees awarded stipends and trained 160,000 293,317 183.3 3. No. of trainees participating in follow up 8,000 7,739 96.7 study 4. No. of training centers used2 615 741 120.5 5. No. of courses conducted 8,000 19,939 249.2 6. No. of enterprises participating in "mixed"'3 2,160 1,078 49.9 retraining courses 7. No. of annual surveys of expenditures on 4 5 125.0 "mixed" retraining courses 8. No. of trainees awarded stipends for "mixed" 24,000 14,148 59.0 retraining courses B. In-service Training 1. No. of promotion unitS4 operating 20 30 150.0 2. No. of training courses conducted in 8,000 10,644 133.0 institutions 3. No. of workers trained by institutions and in- 75,000 212,380 281.3 plant 4. No. of enterprises participating in training 5,000a 77,668 1,553.4 5. Annual survey of expenditures on in-service 4 5 125.0 training under the project by participating enterprises 6. Enterprises reached through seminars and 20,000 31,211 156.1 other outreach services C STPS Institutional Strengthening 1. No. of STPS staff upgraded 100 1,041 104.1 2. Improved manpower information I 1 100.0 system operating 3. No. of studies 1 17 1,700.0 D. Upgrading of Training Programs 1. No. of traininf programs selected 1,000 1,374 137.4 for upgrading 2. No. of pro rams receiving supplemental 1,000 1,374 137.4 equipment 1. State Employment Service; 2. Annual average; 3. Retraining under cost-sharing arrangements with enterprises 4. Regional in-service training promotion units. a. Many enterprises and workers participated in more than one consecutive training activity. The number 212,380 represents the number of trainee courses; the actual number of individual workers who participated was 124,182. The number 77,668 is the total number of training activities with enterprises; the number 32,304 is the actual number of individual enterprises. b. Pursuant to an amendment to the Guarantee Agreement, dated April 24, 1990, these indicators were modified from a target of 500 training institutions to a more relevant target of 1,000 training programs (specialties). This became necessary because many training institutions were justifiably provided support for more than one program or specialty, in one or more locations, by participating in more than one annual phase of the program. 53 Annex 4 Unofficial translation General Directory of Employment Mexico, DF, June 13 1996 Mr. Roger Slade, Chief Agriculture and Human Development Division Operations Evaluation Department We thank you for the Mexico Performance Audit Report on Manpower Training Project, undertaken by Ms. Abadzi last March. We found it serious and a high quality evaluation showing a good comprehension of the Project implemented by the Ministry of Labor. In relation to the specific comments to the PCMO, I would like to say the following: a) After the beginning in 1995 of a new operating modality of the Training Fellowship Programs oriented towards the support of local initiatives of employment, we have been able to substantially increase the attention to groups of rural indigenous producers and of marginal urban zones. b) The Program of Integral Quality and Modernization (CIMO) is increasing its attention to microproducers of the rural sector, including some groups of indigenous producers. c) In relation to the lack of qualified technical personnel in educational aspects, it is important to point out that the training services contracted by the PROBECAT and by CIMO come from qualified training providers. Nonetheless, an effort will be made to improve the curricular content of the training. d) We found very interesting the proposals to promote the hiring of women in non- traditional positions through the training fellowships. e) During the current design process of the next operative phase of the Modernization of the Labor Market Project (PMMT), it is being considered the integration of CIMO and PROBECAT activities in support of the local employment initiatives in order to improve the quality of the support given to the population and producer groups that are more disadvantaged economically. Annex 4 54 Thanks for the opinion given about the capabilities of the staff of the Ministry of Labor to conduct the PCMO, currently the PMMT. I thank you again for this valuable document that will help us to improve the design and implementation of the PMMT. Director General 55 Annex 4 NATIONAL VOCATIONAL AND TECHNICAL EDUCATION SYSTEM - CONALEP Secretariat Administration of World Bank Credit Metepec, State of Mexico, June 19, 1996 Ref: ACBM-347-96 Roger Slade Chief, Agriculture and Human Resources Operations Evaluation Department World Bank Dear Sir: Following your letter received here on June I1, I have been instructed to attach our comments on the draft Performance Audit Report for the Second Technical Training Project (Loan 2559-ME) for consideration in the final version before it is transmitted to the Bank's Executive Directors. Yours truly, Is/ Ver6nica Galindo Galindo Administrator, World Bank Credit cc: [see original] 56 Annex 4 COMMENTS ON THE PERFORMANCE AUDIT REPORT FOR THE SECOND TECHNICAL TRAINING PROJECT (2559-ME) Para. 8 on p. 16 notes that the mission was not clear how the instructors were trained or why the students did not use the instructional materials. CONALEP has constantly striven to produce teaching aids for use by instructors and students; this task is expanding, to the extent that 105 disciplines are now covered, each with a dozen different courses, in which some 200,000 students are enrolled and slightly more than 14,000 teachers are employed. The quality of instruction depends largely on the proper preparation and skills of the teaching corps; as such, instructor training includes introductory courses to the practical aspects of teaching and systematization of those aspects for those instructors who are part of the system; courses in classroom technique, a creativity workshop, formulation of texts for home study, technology training, courses in new technology and operation and maintenance of machinery and equipment for all teachers who have worked steadily with CONALEP. The courses capitalize on written and audio-visual materials, designed by professionals (exprofeso), including educational software. The specialists who give the courses can be internal (teachers or academic coordinators from each school) or external (universities, production sector, etc) and/or equipment suppliers. The training provides the instructors with the necessary foundations for using the teaching materials designed to aid in the teaching/learning process. Efforts to foster use of the instructional materials in the classroom are an ongoing process. With regard to the statement that the instructional materials are not used by the students and that all instructors dictate material to the students, this is a very wide generalization, especially as it is based on observation of two teachers and 30 students (two classes, one at San Nicolds de los Garza [?s] 1 in Monterrey and the other in San Crist6bal de las Casas in Chiapas), in view of the numbers of students and teachers indicated above. Moreover, the instructors have the freedom to rely to varying extents on the plans and curricula designed for each course of studies and subject; there is no requirement to use the materials produced by CONALEP exclusively; they can be used as supplementary aids or as the primary source. On p. 17, para. 11 it is asked why graduates cannot go on to higher levels of study. In accordance with the decree establishing the National Vocational and Technical Education System, CONALEP offers post-secondary terminal studies. Its graduates are mid- 57 Annex 4 level technicians, in other words, skilled human resources who are somewhere between skilled laborers and university-trained professionals. Developing countries seek to make most efficient use of and maximize its resources in training skilled personnel; educating professionals requires a [illegible] investment. The difference between a mid-level technician and a professional, both seen as a finished product, represents a [illegible] margin of [illegible] years of post-secondary educational investment in training. A skilled technician has a more rapid investment cost recovery than a professional, as well as greater possibilities of being self-employed than a professional. Additionally, because technicians have to undergo continuous training and updating, CONALEP offers courses in new and specialized technology at its CAST centers, which provide a vertical option for graduates. Lastly, the students take a set of courses called the "basic system" which, like the open bachillerato system, can be revalidated as pre-university training. The following comments relate top. 17, para. 13, regarding the participation of women in job-oriented training programs: In traditional countries with deep-seated cultural roots and developing economies, women are economically dependent, because tradition has it that work related to maintaining the home is not considered productive and is not remunerated. It is considered a "simple activity" that "does not require much physical effort." Mexico is a multi-ethnic, tradition-bound developing country and, as such, women have made much more headway in the urbanized areas and cities than they have in small urban and rural communities. After considerable economic development, a middle class emerged in the cities and stimulated the demand for political and social changes that enable women to work with men on virtually an equal footing in the development of the country. However, this is a recent phenomenon; [women] have had the right to vote for less than 50 years. The first area targeted by women as part of equal rights was education, perhaps this is behind the emergence of the large numbers of mixed schools. But although co-ed schools predominate, this does not mean that there are not exceptions or that in all of the mixed schools the enrollments of boys and girls is balanced. The larger urban areas tend to be relatively balanced at given educational levels. However, the further the schools are from the cities, the greater the imbalance within educational levels. As education level rises, the gender imbalance in enrollments is reflected in the employment-oriented education choices and the vocational preferences or determinants in each case. Tradition and vocation within the family and community virtually oblige women to opt for 58 Annex 4 training that prepares them for their future social role and is more compatible with their physical strength. Until recently, women tended to study disciplines in the area of health, administration or services; it was rare to find a woman in the construction, extractive or related industries, or to be training for a career in those areas. It was unthinkable that a woman would study automotive mechanics, metal mechanics, forging or molding. Nevertheless, nowadays women are employed in all sectors. CONALEP does not plan its instruction in response to the gender of demand; it seeks to train human resources in accordance with the requirements of the regional and national production apparatus, providing a wide range of educational options with a certain degree of relevance to the labor market, which is open to all. It is not the schools, which train the pool of human capital, that have any regulations, laws or provisions whatsoever restricting the participation of women in their area of interest. Rather, these constraints are shaped by culture, vocation, tradition and social environment. They influence a person's decisionmaking, as distinct from objective reasoning that focuses on whether the intended area of study is compatible with self-employment, is over-saturated on the local labor market, etc. Significantly, only the Third Technical Training Project focuses on to the needs of women, but only in a very cursory manner, referring solely to technical education and not vocational training (para. 3.13 of the SAR for CONALEP III). In this connection CONALEP has a Marginal Areas Program that uses mobile training units to address the basic training needs of women, not necessarily resulting in higher income, but which nevertheless help improve their living conditions (for example, a short course on food canning in areas lacking refrigerators). The following comments refer top. 23, para. 23, regarding the importance of industrial advisory boards for each center: Each CONALEP school has an Advisory Committee, a legal entity by virtue of its establishment decree, that promotes education supply and [?the employment of] graduates in the area served by the school. It also analyzes the factors involved in decisionmaking, which range from changes in curricula to the establishment of a new school or the opening or elimination of courses of study. Both the production sector and the community in general, as evidenced by the patronatos at some schools, which help secure financing for special educational events. Para. 27 on p. 24 states that Mexican officials had expressed their dissatisfaction with the constant turnover in Bank staff assigned to the project. 59 Annex 4 In this case the Government officials referred solely to the representatives of NAFIN and the Secretariat of Finance and Public Credit. CONALEP has not had problems with the [illegible] officials of the Third Technical Education Project and enjoys good relations with them. The following comment refers top. 35, para. 2.5, where it is stated that there are 60 mobile training units serving remote sites: At present there are 80 mobile training units that were built at system centers, 78 of which using funding from the Third Project. The following comment refers to para 1.7 on p. 32, which mentions the six projects financed by the Bank to promote vocational and technical education: The Secretariat of Labor and Social Welfare is not the only executing agency of the Technical Education and Training Modernization Project (Loan 3805-ME). CONALEP is also involved in Component B - Modernization of Training Programs, which includes the initial pilot phase for acquiring technical experience in the development and operation of training modules and the preparation and development of training materials. Box 1 on p. 34 notes that the education represented by the diplomas granted by CONALEP is in effect nonformal. The diplomas awarded by CONALEP are subject to the General Law on Professions (for mid-level technician) and are recognized by the Secretariat of Public Education. As such, they are part of the formal education system; nonformal education in Mexico refers to that which is not officially accredited. On p. 35, para. 2.5, the report states that CONALEP has few schools in the less developed areas of the country. The criteria for opening a CONALEP training center are: (i) population of more than 60,000; (ii) existence of potential demand not adequately served by another educational institution of the same level; (iii) existence of a local economy with demand for skilled technical personnel; and (iv) express request by the social and production sectors for the opening of a center. In order to meet the low demand of areas that do not satisfy the foregoing criteria, CONALEP devised its strategy of extension units that offer vocational training on premises provided by the local governments (primary or secondary schools or offices) through the Marginal Areas Program, which uses the mobile training units to give vocational instruction. 60 Annex 4 Para. 2.16 on p. 39 speaks about dropout, the main causes of which are: (a) internal, where the system can carry out recovery programs, such as high levels of academic failure for certain subjects such as mathematics and physics, teacher inadequacies, equipment inadequacies, etc. and (b) external, such as the students' economic problems, inadequate academic preparation of new entrants, and lack of vocational preparation. 61 Annex 5 Responses to Borrower Comments The comments by the Secretaria de Trabajo y de Previsi6n Social did not contain any requests for changes. The comments by CONALEP did and have been taken into account as follows. 1. CONALEP provided details on the procedures for teachers' in-service training. However, they pertain to the most recent project, which was not being audited. Some of these details were inserted in para. 2.13. 2. According to the comment, the audit states that all teachers teach by dictation and do not use textbooks. Clearly, the pertinent paragraph, 2.12, was misread, and the discussion pertains to mission observations only. Additional clarification was provided. 3. Paragraph 11 appears to have been misread. The mission understands and supports the reasons why CONALEP has terminal programs, although many students would prefer access to higher education. 4. Borrower merely comments, no changes needed. 5. Observation taken into account, text corrected accordingly (Box 1, paras. 27, 2.5, 2.16).   IMAGING Report No: 16126 Type: PPAR