Colombia - Second Subsector Primary Education Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No. 18750 IMPLEMENTATION COMPLETION REPORT (ICR) COLOMBIA SECOND SUBSECTOR PROJECT FOR PRIMARY EDUCATION (Loan 3010-CO) December 28, 1998 Human Development Management Unit Country Department Unit for Colombia, Ecuador y Venezuela Latin American and the Caribbean Regional Office This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENCY UNITS Currency Unit = Colombian Peso (P) (July, 1989) (March 31, 1998) US$1 = P$305 US$1 = P$1,360 GOVERNMENT OF COLOMB3A FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS DNP National Planning Department FER Regional Education Fund FIS Social Investment Fund GDP Gross Domestic Product GOC Government of Colombia MEN National Ministry of Education NES National Executive Secretariat (for the project) OEI Organization of Ibero-American States OPSE Planning Office of MEN PCU MEN Project Coordinating Unit SECAB Secretariat of the Andres Bello Agreement SED Departmental Secretariat of Education SIED MEN Integrated Information System for Education UNDP United Nations Development Programme UPEP Universal Primary Education Program Vice-President: Shahid Javed Burki, LCR CMU Director: Andres Solimano, LCC4C Sector Leader: Constance A. Corbett, LCC4C/HD SMU Director: Xavier Coll, LCSHD Task Manager: Marth-a Laverde, LCCCO This document is for restricted distribution and may only be used by recipients in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  FOR OFFICIAL USE ONLY Table of Contents Page PREFACE EVALUATION SUMMARY ............................................ PART 1: PROJECT IMPLEMENTATION EVALUATION ...........................................I A. Project Objectives .I B. Achievement of Project Obectives .3 C. Main Factors Affecting Project Implementation .9 D. Project Sustainabilia 11.. E. Bank Performance ......................1.... . F. Borrower Performance .11 G. Assessment of Project Outcomes .12 H. Future Operations .12 . Key Lessons Learned .12 PART Il: STATISTICAL TABLES ................................... 15 PART III: APPENDIXES ................................... 25 A. Final Supervision Mission's Aide-Memoire ................................... 25 B. Borrower Contribution to the ICR ................................... 31 C. Government Commentary on the ICR ................................... 37 This document is for restricted distribution and may only be used by recipients in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  IMPLEMENTATION COMPLETION REPORT (ICR) COLOMBIA SECOND SUBSECTOR PROJECT FOR PRIMARY EDUCATION (Loan No. 3010-CO) PREFACE This is the Implementation Completion Report (ICR) of the Second Subsector Project for Primary Education in Colombia (Ln. 3010-CO), approved December 12, 1988. It became effective March 28, 1989. The loan amount was US$100 million but closed at US$95.8 million because of project cost savings in dollars resulting from exchange rate changes. The loan was closed March 31, 1998-27 months after the original closing date. The undisbursed balance of US$4.2 million was cancelled on August 26, 1998. The ICR was prepared by Martha Laverde (Task Manager), Carmen Elisa Fl6rez and Vicky Colbert (local consultants). Reviewers were Alberto Rodriguez, Ernesto Cuadra, and Jorge Barrientos. Preparation of this ICR is based on a completion mission in February 1998 and a review of project files, including the results of studies and surveys carried out under the Project. The Borrower's contribution appears in Appendix B; the Government's comments on the ICR are presented in Appendix C. In addition the Borrower's kind cooperation in providing the necessary information and participation in numerous discussions during preparation of the ICR are gratefully acknowledged.  IMPLEMENTATION COMPLETION REPORT (ICR) COLOMBIA SECOND SUBSECTOR PROJECT FOR PRIMARY EDUCATION (Loan No. 3010-CO) EVALUATION SUMMARY Introduction 1. The Second Subsector Project for Primary Education in Colombia (Ln. 3010-CO) was designed to support expansion and improvement of education formed as an integral part of the Government of Colombia's (GOC's) policy initiative to achieve greater economic and social development. This goal was articulated in the GOC National Rehabilitation and Poverty Alleviation Program, 1986-90, and accompanying six-year investment program, geared to increase the productivity of the poor by improving basic services. Improving primary education quality and improving access for all school-age children were explicit priorities for the education sector. The essential part of the investment program for education was the consolidation of the efforts underway to achieve universal coverage through the Universal Primary Education Program (UPEP). 2. The World Bank began to finance education projects in Colombia in 1968. The first three projects supported the development of diversified secondary education (Table 2). The fourth shifted the focus to integrated rural education development, with a primary education component. The fifth project (Loan 2192-CO) concentrated on rural primary education. It was conceived as the pilot first phase of a 1 0-year effort to provide high-quality primary education for all rural children and was implemented between 1982 and 1989. The Second Subsector Project for Primary Education (Ln. 30 10-CO), the second phase of the 10-year program, was designed to expand primary education coverage to marginal urban and rural areas, building on the lessons and experiences of the previous project. The UPEP, an integral part of the national investment program, was supported by Ln. 301 0-CO, and involved the joint efforts of the Ministries of Education and Finance, the National Planning Department, and the Bank. 3. The loan supported the Bank's country strategy oriented to assist the Government in increasing the efficiency of investments in the social sectors. Elements of the Bank's medium-term strategy in the education sector related to extension of basic education through the lower secondary cycle and selective improvements in upper secondary education. The Project contributed to implementing the Bank's country strategy by focusing on sector financing and allocation policies, resource mobilization, and the upgrading of primary education, particularly for rural and marginal urban populations. The project complemented other Bank-assisted operations in the areas of basic health, rural water supply, and municipal development. Project Objectives 4. The objectives of the Project were to strengthen resource mobilization at sub-national levels and support related policy development in the education sector, while raising the quality of primary education and expanding access to all school-age children. This was to be done by: (a) restricting the appointrnent of new teachers and strengthening incentives to relocate surplus teachers to rural areas and raising the share of non-personnel inputs in recurrent expenditures; (b) introducing  - ii - automatic promotion in an attempt to reduce repetition, dropout, and average cost per graduate; (c) increasing investments in primary education and directing the largest share to schools in disadvantaged rural areas; and (d) encouraging local contributions to finance civil works. Implementation and Results 5. Project implementation took over nine years-two years and three months beyond the scheduled closing date. Physical investments in the project exceeded original estimates by 30% owing to cost savings in dollars due to exchange rate changes and higher-than-programmed government counterpart contribution. The pace of investments exceeded the financial accounting capacity of the PCU within MEN to monitor national, departmental, or municipal efforts. A new financing mechanism, the Social Investment Fund (FIS), was established to handle investments, with a view to improving funds administration and obtaining a better balance between loan and counterpart funds. Although slow to get started, the FIS eventually became capable as an institution for handling investment responsibilities. 6. Several factors contributed to delays in implementation. These included decentralization initiatives and community participation priorities delineated in Colombia's Constitutional reform in 1991. The many and various jurisdictions executing investments lacked the technical capacity, coordination experience, and communications links (especially between municipalities and departments) to meet the new requirements. These problems had been identified as risks in the Staff Appraisal Report. 7. Legislative changes in the national educational framework (1994 General Law of Education) were in harmony with proposed educational and regulatory reforms. These included emphasis on community participation and the "active school" concept, which were built into Ln. 301 0-CO. But the lack of adequate institutional arrangements and capacity protracted implementation. Additionally, changes introduced in the education strategy and scope caused implementation setbacks. For the first four years, the project's focus was national in scope, covering about 1,000 municipalities, with emphasis on rural areas. In 1994, the focus was narrowed to target 325 municipalities with the worst education indicators. Since these municipalities were also among the poorest and had weaker-than-average institutional capacity, the pace of implementation slowed. In 1996, the scope was again expanded to cover all municipalities, while emphasis was shifted from infrastructure to learning materials (including textbooks) and furniture. 8. Given the foregoing, results were mixed but generally satisfactory, with some notable achievements but disappointments as well. The policy component was geared to promote expanded coverage and improved efficiency through increased primary education enrollment and automatic promotion, as well mobilization of local resources for educational purposes-key objectives of the GOC's strategy supported by the Bank. In addition, total government expenditures for primary education increased from 1.7% to 2.8% of GDP, reflecting commitment to these goals and bringing the country more in line with international averages. Enrollments at primary level grew by 71% compared with the target of 20%. Efficiency improved only slightly. 9. The management development component sought to improve the capacity for medium-term planning using the existing information system. In general, implementation for this component was weak, especially in the early stages, and results were below expectations. Management improvements were not fully implemented in MEN because of inadequate staff training and technical support at the central and departmental levels. Both project management and monitoring was difficult owing to lack of provision for such activities within component design. For example, project design did not include a training needs assessment or provision for evaluation  - iii - of training provided. Technical assistance was provided under the Project, resulting in development of a new education information system for decentralized entities. Unfortunately, problems with compatibility with other existing systems has impeded integration of these systems with those at national level. With respect to local capacity building, by 1996/97, some departments and municipalities, with the support of universities and NGOs, had begun to implement technical assistance and training strategies to strengthen knowledge of management principles generally and education management in particular. 10. The primary education subsector development component sought to improve the teaching- learning environment by supplying textbooks and libraries to virtually every primary school in the country; providing educational materials for students and teachers in 83% of rural schools and school furniture to 44% of rural students; and funding school rehabilitation and construction. Approximately 105,300 teachers were trained in both rural and urban areas. However, monitoring was insufficient to determine the effect of the training on improving teaching practices or student learning. Moreover, policy debates and disputes concerning aspects of the Escuela Nueva concept, such as its appropriateness for non-rural areas, prevented the spread of its curricular and pedagogical innovations to the extent anticipated. Design and application of an instrument to evaluate achievement (the National Quality Assessment System, SABER) and of an instrument (the Integrated Achievement and Value Assessment Survey, IAVAS) to assess a number of variables (including student achievement, social values, school management, and costs) were important results of the Project. The Government is currently applying the instruments in selected areas. 11. Bank Performance. Annual supervision missions and frequent meetings between local Bank staff and MEN and PCU staff facilitated strategic changes needed to meet overall objectives in the wake of legislative and institutional changes that affected implementation. As serious problems were encountered with respect to project management, accounting, and administration of funds, supervision focused more on financial, administrative, and management aspects than on educational improvement aspects. As a result, supervision missions' ability to assist the Borrower in redressing problems of inadequate technical and administrative capacity to address educational quality issues was limited. 12. Borrower Performance. The high turnover of managerial and technical personnel in MEN complicated execution of the Borrower Implementation Plan. Serious problems were encountered with respect to project management administration of funds. Once the problems were discovered, however, the Borrower took prompt action to address them. To support coordination, MEN created a special unit to coordinate external credit. Lack of clarity about the unit's role was complicated further by the many institutional and jurisdictional partners engaged in the financial execution of components. Borrower coordination focused almost exclusively on budget processes rather than monitoring of project execution in substantive educational areas. 13. Project Outcome. Outcomes of the project were generally satisfactory, with notable successes but also disappointments. The targets set for increasing coverage were exceeded, contributing to the overarching goals of attaining universal coverage while improving the internal efficiency of primary education. Primary school enrollments grew by 71% compared with the target of 20%. The goals of improving the quality of primary education by training teachers, strengthening school management, and providing textbooks and furniture were partially achieved. A significant achievement of the project was the development of evaluation instruments relating student achievement to school-related variables such as condition of physical facilities and availability of textbooks, libraries, and educational materials.  -iv - Future Operations and Lessons Learned 14. The Bank has initiated two new projects in the educational sector that address school quality improvement in a decentralized system, with project execution managed by the Department of Antioquia and the Municipality of Pasto, respectively. The current preparation of a project to improve the quality of rural education, based on alliances with the private sector and participation by civil society, is drawing on lessons learned from Ln. 301 0-CO. Lessons Learned * Major institutional and political reforms, such as decentralization efforts, that take place during project implementation often delay project implementation-even if, in principle, they support project objectives. If this occurs, the Bank and the Borrower will need time to analyze and reformulate project activities and revise implementation timetables. Project design should be sufficiently flexible to permit timely realignment of activities in line with reform priorities. * Projects that undertake comprehensive national educational reform programs-such as achieving universal basic education, improving quality, and establishing sector finances at the local level-should identify concrete and measurable objectives that can be attained within the project period * Monitoringperformance objectives in coordination with representatives of participating national and sub-national entities is crucialfor effective project implementation. In a decentralized context, a basic monitoring and auditing system is essential to provision of timely assistance to local authorities. - An integrated management information system, as well as a network among executing entities, is essentialfor monitoring and supervision ofproject activities. The technical capacity of all participants must be strengthened as necessary through well-designed technical assistance. * Innovative and effective educational concepts and programs may be difficult to implement broadly unless adequate attention is paid to the political dimensions of change that impinge on educational policy making and may cross conventional dichotomies such as urban-rural school differences and poor versus middle-class students. * Centrally managed programs that target local governments will require flexible project designs to accommodate the enormous diversity in capabilities, needs, and educational and socioeconomic conditions.  COLOMBIA SECOND SUBSECTOR PROJECT FOR PRIMARY EDUCATION (Ln. 3010-CO) IMPLEMENTATION COMPLETION REPORT (ICR) PART I: PROJECT IMPLEMENTATION EVALUATION A. Project Objectives Background 1. Both public and private education began to expand rapidly in Colombia beginning in the mid- 1960s, but maintaining satisfactory levels of quality proved more difficult. By the end of the 1980s, nearly 80% of children aged 7 to 11 were enrolled in primary schools, while 60-70% attended school in rural areas (SAR, paragraph 1.2). These increases reflected increased public expenditures for education especially for primary schools. Nevertheless, at 3% of GDP, public expenditure in education remained lower than in other countries at similar development levels represented through the 1980s. 2. In the 1980s, most public resources for education went to pay teacher salaries (99%). This was reflected in greater per-student costs and reduced funding for inputs to enhance the quality of basic education (e.g., through better infrastructure and teaching/learning materials). Also, public financing of education was excessively centralized, with a relatively high share allocated to higher education, which significantly reduced funding for primary and secondary education. By the mid- 1980s, approximately 97% of public financing for primary education came from the Central Government. 3. By the end of the 1980s, primary and secondary school student achievement was low, and repetition and dropout rates were high, which pointed to the erosion of educational quality. One of the main factors contributing to low quality was the scarcity of teaching materials and textbooks for both teachers and students. During the 1980s, the Subsector Project for Basic Rural Education (see Table 2) partially financed with a Bank loan, supported the introduction of a revised curriculum for rural primary schools (Escuela Nueva), the supply of educational materials for teachers and students, support to school libraries, and in-service training for teachers. All of this led to measurable improvements in quality. 4. One of the objectives of the Government's National Rehabilitation and Poverty Alleviation Program-1986-90 was poverty reduction. The strategy was to increase the productivity of the poor through strengthened delivery of basic services, including education. Increasing primary education quality and improving access for all school-age children were the twin policy objectives for education. 5. These objectives subsequently were incorporated into a six-year investment program focused on expanding primary school coverage in marginal rural and urban areas, improving efficiency in the allocation of primary education resources, and stimulating resource mobilization at the sub-national level. Ln. 3010-CO was designed to support this ambitious investment program, with its numerous objectives.  - 2- Description of Project Objectives 6. Ln. 3010-CO was intended to support the Universal Coverage Program for Primary Education Plan, coordinated by the National Ministry of Education (MEN). The overall objective corresponded to that of the National Rehabilitation and Poverty Alleviation Program, 1986-90, namely to strengthen resource management and mobilization in the educational sector and raise the quality of primary education, while simultaneously expanding access to all school-age children. These objectives would be supported by: (a) limiting new teacher appointments and financing incentives to relocate surplus teachers, mainly to rural areas, and by increasing the share of non- personnel inputs in recurrent expenditures; (b) introducing a new student promotion policy to reduce repetition and dropout rates, and lower average cost per graduate; (c) increase investment in primary education and direct the largest share to schools in disadvantaged (mainly rural) areas; and (d) encourage local resource mobilization to help finance civil works. 7. The strategies to achieve these multiple objectives involved active participation by national and regional governments. Activities included execution of civil works, purchase of furniture and materials, pedagogical training of teachers, and technical assistance for strengthening implementation capacity and purchase and use of materials (see Table 6). 8. The Project comprised the following three components: A policy component which sought to: (a) increase the participation of non-personnel inputs in recurrent expenditures by limiting the creation of new teacher positions and financing incentives to relocate surplus teachers (mainly to rural areas); (b) introduce a student promotion policy aimed at reducing repetition and dropout rates and lowering average cost per graduate to promote efficiency; (c) encourage subnational contributions to finance the subsector; and (d) conduct studies to assess alternatives for future financing of higher education. ? A primary education development component which financed: (a) student and teacher materials and school libraries for 4,000 schools, and in-service training for 20,000 teachers based on the successful rural curriculum development effort known as Escuela Nueva; (b) essential furniture for 13,000 rural schools, and classroom rehabilitation for about 4,300 rural schools to improve access; (c) textbooks and in- service teacher training to support a revised urban curriculum, and furniture for some 2,100 urban schools in marginally served areas; and (d) a system to assess student achievement in primary education, extension of basic rural education beyond the primary cycle on a pilot basis, and a policy study related to the future development of secondary education. * A management component which provided technical assistance, staff training, and studies to: (a) complete the introduction of an improved information system and strengthen medium-term education planning and policy monitoring capacity; (b) develop budgeting and financial management skills within MEN; (c) improve technical support capabilities in the central and local bureaus of MEN, as well as their program planning and implementation capabilities; and (d) develop the capacity to prepare future projects.  -3 - 9. The policy component and the physical goals of the project were articulated by the GOC in a letter of intent that included an action plan with benchmarks for the evaluation of project outcomes (see Tables 5 and 6). 10. Project base cost was estimated of US$144.1 million, with total project cost including contingencies estimated at US$169.2 million. Despite cancellation of an undisbursed loan balance of US$4.2 million at loan closing, actual investment in the Project totaled US$220.9 million (30% higher than estimated total project cost) as a result of substantial increases in Government counterpart contribution to the Project. The initial financing plan called for 59.1% of project costs to be financed by the Loan, while the Central Government would contribute 33.4% and participating departments and municipalities would provide the remaining 7.5% (Table 8B). As a result of the Government's decision to increase its contribution to the Project, the final financing plan shows 43.1% from the Loan, 53.1% from the Central Government and 3.5% from decentralized governments. While the contributions from decentralized entities was US$4.8 million lower than planned, the figure does not include in-kind contributions in the form of storage services, materials, and furniture, which were not estimated in Project accounts. 11. The original plan called for allocation of 74.7% of base costs to the primary education development component, 23.7% to the policy component, and 1.6% to the management development component (Table 8A). Owing to the increase in total investment under the Project, all components received larger allocations of funding than originally planned. It is notable, however, that the largest increase in funding was channeled to learning materials and teacher training, while slightly less was spent on infrastructure compared with original plans. This, combined with substantial increases in support for technical assistance, staff training and teacher incentives, reflects the increasing importance placed on human resources development during project implementation as a critical factor in achieving goals of both education quality improvement and expansion of coverage. This emphasis was important in achieving project objectives. B. Achievement of Project Objectives Overview 12. The Project's key development objective was to contribute to the achievement of universal access to primary education, to raise the quality of primary education, especially from rural and marginal urban populations, and to strengthen mobilization and management of resources in the sector, including the local level. The project exceeded its targets related to expansion of access, bringing nearer attainment of universal coverage at primary level. By 1997, total enrollment (including private) in primary schools reached 5.5 million-a 71 percent increase over 1989 levels, compared with the project target of 20%. The rate of increase was slightly higher in rural areas. 13. Given the 1989 net enrollment rate of less than 66% for children aged 7-11, the goal of universal coverage was ambitious. The project's significant achievements in raising access notwithstanding, even more progress might have been made had the priorities of the UPEP been consistently maintained throughout the period. The original program strategy was to extend primary education coverage through automatic promotion and by encouraging enrollment of children not in school. In 1994, given the poor results registered on the standardized test (SABER) administered by Instituto SER in 1992, emphasis shifted to improving the quality of primary education. Activities targeted 325 rural and underprivileged urban municipalities with low educational quality and high poverty levels. As these municipalities also had weaker-than-average institutional capacity, project execution was rated "low" and remedial actions taken. By mid-1996,  -4 - this slow progress, together with concerns that the reformulated targets could result in additional inequities, led to a change back to the original country-wide scope. This series of shifts in emphasis and focus resulted in implementation delays. The closing date of the loan subsequently was extended several times, and the loan finally closed March 31, 1998. 14. A key quality improvement goal focused on improving teaching conditions in the classroom. Nearly 80% of investments went to improve the teaching and learning environment and pedagogical conditions, factors directly related to quality gains. The Integrated Achievement and Value Assessment Survey (IAVAS) conducted in 1997 as part of the program impact evaluation (Part I, paragraphs 31-32) indicates that the quality of primary education did improve. Significantly higher test scores were obtained in the mathematics, science and values dimensions among students whose schools received furniture through the plan. School environment (e.g., condition of physical infrastructure, availability of furniture, lighting, desks, materials) increased educational achievement by 29%. Students in rural schools modeled on the Escuela Nueva (two-thirds of the total schools) and those in urban schools using the Escuela Nueva curriculum obtained significantly higher average performance scores than those in traditional schools. 15. The project partially achieved its objective of strengthening the provision and management of educational services. A new computer-based management information system was developed for use at decentralized levels, and the system is still in use. Unfortunately, the utility of the new system has been limited by problems of incompatibility with other elements of a proposed management information system at central level, which impedes integration of the systems. This problem is still pending resolution. Training and technical assistance were also provided to strengthen the existing management system. The objective of increasing mobilization of funds for education was achieved at both national and local levels. This is reflected in the larger-than- anticipated provision of counterpart funds for project investments by the Central Government. Increases in local funding were mainly devoted to cover teachers' salaries. Resource Use and Mobilization Policy Component (US$38.S million) 16 Expenditure Composition and Teacher Employment Policy. The National Planning Department (DNP) estimated that primary education absorbs about 56% of investment in basic education (primary school plus 4 years of secondary school). In the 1994-98 period, teacher payroll was nearly 85% of total public education expenditures. Thus, expenditures for educational inputs other than personnel more than doubled during implementation to 9 percent, surpassing the goal of 3 percent. Local administrative contributions to basic education finance are growing, but are dedicated entirely to payroll expenditures, limiting outlays on non-wage inputs that could contribute to improving educational quality. 17. The policy to limit the creation of new positions for teachers to lower the overall wage bill and introduce incentives to relocate surplus teachers to rural and marginal urban areas was implemented in two phases. The first phase (1989-95) financed teacher relocations from urban to rural areas. The second phase (1996/97) was more systematically carried out nationwide. The local educational communities actively participated, encouraging greater responsibility by schools and an improved organizational climate and stimulating greater emphasis on training and improvement of school facilities. 18. Expenditure per Primary School Graduate and Promotion Policy. Though it may still be early to assess the full impact of the project on educational quality and efficiency, repetition and dropout rates remain unsatisfactory despite implementation of the automatic promotion policy. The Expenditure Rationalization and Public Finance Committee (1997) reported that only 20 out of 100  - 5 - children who enrolled in first grade completed basic education (20%), and only 7 do so without repeating a grade. There is evidence that this partly reflects the failure of traditional teaching methods and that the problem is most severe in the early grades. Indirect data for the 1988-91 period suggest that school retention increased substantially in rural areas when the Escuela Nueva methodology was used. On the other hand, no significant changes occurred in urban areas where only the traditional methodology was applied. 19. Modest efficiency gains were achieved in reduction of repetition rates and improvement of resource use. Continued application of the automatic promotion policy has achieved only slight efficiency gains. The cohort that began primary school in 1988 (the first year of the Project) was automatically promoted and subsequently graduated and required one month less to complete five years of primary school than the cohort that started school in 1978. The pattern of resource use suggests some efficiency gains. The share of resources inefficiently spent on repeating and overage students was 37.5 percent in 1993-an 8.7 percent improvement since 1985. In 1994, continued improvement was evident, with students repeating grades or who later dropped out representing 35% of total primary school enrollment (Expenditure Rationalization and Public Finance Committee, 1997). 20. Along with a slight improvement in efficiency, total expenditures per student rose. According to DNP, between 1987 and 1994, total Central Government expenditures per primary student increased 16.2%; total salary expenditures increased by 10.6% over this period. There was little change in productivity and internal efficiency. The student-teacher ratio for Colombian public education was 22 in 1997, compared to an "optimal" ratio assumed to be 30 in rural and 40 in urban schools. A student-teacher ratio of 22 would imply a cost overrun of approximately 30%. 21. Investment Expenditure, Resource Mobilization, and Cost Sharing. Education expenditures increased from 2.5% to 4% of GDP during the project, which is more in line with international benchmarks for countries at similar levels of development. Resources dedicated to the basic education sector (primary school plus four years of high school) grew in real terms 2.3 times between 1990 and 1997. Based on this, public financing for basic education rose from about 1.7% to 2.8% of GDP between 1990 and 1997, excluding resources allocated by departments and municipalities. Although contributing only marginally to this increase, the real impact of the project may be in prompting the Central Government (as well as local governments) to increase public investment in basic education. Consistent with the decentralization objectives expressed in the 1991 Constitution, the municipal share of total sector financing shows a 29-fold increase between 1990 and 1997, rising from 1.5% to 19.5% of total basic education spending for the period. By 1997, the departmental share represented 5% of total basic education spending. The project targeting criteria (focusing benefits on rural and marginal urban areas) were largely met, with textbooks (61%) and furniture (56%) distributed to rural areas. As decentralized executing entities did not keep disaggregated project records, however, it is not possible to estimate distribution of benefits for all activities. 22. Financing Policy for Higher Public Education. Consultant studies helped the Ministry of Education design a Secondary Education Coverage Enhancement Plan (between 1992-1994) and to implement it, with Bank participation, beginning in 1995. Primary Education Subsector Development Component (US$127.9 million) 23. Educational Quality Improvement: Textbooks and Libraries. Provision of materials for teachers and students and in-service training for rural teachers were the main inputs for improving the quality of classroom conditions for teaching and learning. The purchase and distribution of  -6 - textbooks and libraries followed a similar pattern: sets of textbooks and basic libraries were assembled based on recommendations of teacher-training workshops. Both textbook and library coverage extended to nearly all schools in Colombia, surpassing prescribed goals. Approximately 25 million textbooks were delivered, of which 63% went to students in rural schools, 18% to urban students, and 9% for teacher's guides. In rural areas and urban schools that adopted the Escuela Nueva approach, textbooks reflected Escuela Nueva content. Each set of books was to be shared by three students in rural areas and two in urban zones. On average, each public primary school student received six books. 24. Book and library distribution took place through departmental secretariats, which received acquisitions directly, and then distributed them among the municipalities and schools. Final distribution to municipalities and their schools required an agreement between national and local jurisdictions. Provisioning of rural and urban libraries was tailored to local circumstances. School enrollment and the number of municipalities in a department governed the number of books distributed. These criteria accounted for 83 percent of textbook allocations. In rural areas, between 60 and 70 titles went to each library for use by all school grades, totaling nearly 47,000 volumes. In urban schools, libraries were grade-specific, with 15 titles per grade, for a total of nearly 137,000 volumes. Initially, beneficiaries were unaware of the arrival of textbooks and their envisaged educational benefits, due to lack of adequate promotional activities. This changed in 1996 when a more aggressive national communications strategy took shape. 25. Teacher Training. Despite the acknowledged importance of in-service teacher training, project design did not include a plan for monitoring the program. Similarly, the effectiveness of the teacher training in improving classroom teaching practices and qualitative gains in learning were not assessed. Estimates based on employment mobility data (30 percent annually), suggest that training of some type reached at least 50% of the teachers. In-service training occurred throughout implementation, in two phases. Between 1988 and 1992, agreements were signed with departments and departmental secretariats selected teachers and made logistical arrangements. From 1993-1997, training was organized and administered through agreements with the United Nations Development Program (UNDP), Secretaria del Convenio Andres Bello, SECAB (Andres Bello Agreement Secretariat), and the Organization of Ibero-American States (OEI). 26. Since the first phase coincided with the restructuring and decentralization of MEN, the preparation of a training strategy for rural teachers was delayed. The principal problems detected at this stage were poor coordination and lack of instructional materials; the use of lectures instead of experiential, interactive modes of teaching; and inconsistent criteria for selection of trainers. During the second phase, reading, writing and arithmetic instruction were stressed, with training reflecting regional initiatives. In response to MEN's request, the departments turned to universities and NGOs for help in identifying teacher training priorities. This led to a better understanding of the educational environment in municipal schools by all participants. However, universities and NGOs were unfamiliar with student textbooks or teacher manuals and had no access to them during teacher training. This created a disconnection between the content of teacher training and the suitability of the methodologies used and proposed. 27. In 1996, MEN signed agreements with Corporaci6n Mixta and SECAB to develop a new teacher-training strategy and to produce other proposals based in part on teachers' perceptions of needed changes in curriculum content among the six departments surveyed. The size of the sample is not known, however, and there was no separation of responses by rural and urban teachers. Teaching materials and textbooks were produced and training provided for Departmental Training Committees, Education Secretariats, and NGOs. The short duration of implementation (6 to 8 months) made it difficult to assess the effect of the training on teaching practices.  -7- 28. Teacher training ultimately was not consolidated into a cohesive program and no new teacher training policy emerged. Training generally took place haphazardly and was administered mainly as a publicly offered, central MEN function. Moreover, it had the effect of blurring the distinction between rural and multi-grade schools and, for a time, led to more emphasis on traditional and urban schools. Planning and organization of training was helped by the creation of demonstration classrooms (micro-centers) where training teams were organized. Furthermore, teacher training helped disseminate information about Escuela Nueva and its congruence with the new Decentralization Law at national level (teachers, teaching staff involved in management, SED, and the universities) as well as delivery of the Escuela Nueva pedagogical guidelines. Despite earlier positive experience with teacher training in Escuela Nueva demonstrating that such training can be instrumental in changing teaching practices and attitudes, the Escuela Nueva approach to training was not adopted owing to lack of consensus between central education policy-makers and Escuela Nueva advocates over the model's suitability for system-wide implementation. 29. Test Development. In 1989, MEN proposed developing a National Quality Assessment System (SABER) to assess achievement and quality in primary schools. A first phase occurred in the second semester of 1990. Three departments agreed to pilot mathematics and language achievement tests based on teacher inputs. These results were used to develop the current SABER test, first given in 1991 to a representative sample of students in grades three and five in urban and rural public and private schools. The test differentiated between academic and non-academic dimensions of school achievement. 30. Test results showed students failed to achieve at grade level in math and language arts, and cognitive skills appeared weak. The highest achievement scores were recorded in private schools. Rural public schools, followed by urban public schools, posted the lowest scores. School characteristics significantly influenced achievement. Low achievement correlated with inadequate physical infrastructure, limited availability of textbooks and educational materials, and limited access to basic secondary education. Other significant variables were family socioeconomic characteristics and time spent commuting to and from school. Students in Escuela Nueva schools achieved higher scores in Spanish and mathematics, civic behavior, and creativity and showed higher self-esteem than students in schools where conventional teaching practices were used. 31. MEN later implemented an Integrated Achievement and Value Assessment System (IAVAS). IAVAS was intended to provide information about qualitative improvements in public primary schools, as well as to study variables associated with investment in the UPEP. These variables included teacher training; provision of desks, textbooks and libraries; establishment of Municipal Education Resource Centers (CREM); and classroom improvement and extended coverage. The design of IAVAS took into account educational achievement (knowledge, aptitudes and skills) in the areas of language, natural science, and mathematics, and attitudes and values. The assessment instruments and questionnaires were administered in October 1997 to a sample of 400 schools in 100 municipalities that participated in the UPEP. Results were analyzed using descriptive and inferential single- and multivariate statistical techniques. 32. Test results revealed improvements in academic achievement in mathematics, language arts, and science. Urban primary schools scored highest in mathematics and science, while "social values" scores were higher in Escuela Nueva rural schools. Among school factors associated with achievement, the most important were teacher training, availability of textbooks and libraries supplied through the UPEP, receiving furniture, and being in remodeled and rehabilitated buildings. Urban scores were lower on the IAVAS than on the SABER test. Data from these tests have not been used to create a baseline by which to gauge academic quality gains, and MEN has not yet selected which instrument to use in future testing efforts.  -8 - 33. Improved Access and Equipment. Public school enrollment grew from 2.7 million in 1989 to 3.5 million by 1994-an increase of nearly 30 percent, compared with the target of 20% growth for the entire project period. Enrollment growth was slightly higher in rural areas (31 percent). By 1997, total enrollment in primary schools (including private schools) reached 5.5 million-a 71 percent increase over 1989 levels. 34. Bookshelves, filing cabinets, desks and chairs were provided for students and teachers. Beneficiaries included approximately 700,000 rural students and 586,000 urban students. There was a 44% overall improvement in equipment provision for rural students, with bookshelves and filing cabinets distributed exclusively to rural schools. Furniture provision surpassed project goals of 10,700 rural schools. In all, 19,000 filing cabinets and 25,000 bookshelves were distributed, covering 59% and 75%, respectively, of the total of number of schools in urban and rural areas. 35. Financial information at departmental level is the only source of data about investments in school and classroom rehabilitation and construction. Records of physical works were the responsibility of municipalities, which did not systematically report on the type and number of physical works undertaken; thus, meaningful comparison with original targets is not possible. Investments in infrastructure exceeded planned levels, primarily because of significant increases in counterpart funds (from local as well as national sources). The higher level of local contributions to civil works permitted a reallocation of loan funds from the civil works category to teacher training, technical assistance, and educational materials. Despite the increases in local investment in education, total levels of investment per student are still less than optimal, raising concerns about the sustainability of benefits under the project. 36. Extension of Basic Education in Rural Areas. In 1997 an experimental component attempted to extend the primary school cycle to include grades 7 and 8 in selected schools. Known as the Rural Basic Education Quality Strengthening Program (post-primary), it used the Escuela Nueva model in the Department of Caldas, among other territorial jurisdictions, in partnership with the private sector and the Federaci6n de Cafeteros (Federation of Coffee Growers). The effort was incorporated into MEN's post-primary school program. Program characteristics included training and supervision, use of life-experience strategies, community participation, and decentralized implementation. Outcomes have not been formally evaluated. Management Development Component (US$2.8 million) 37. Integrated Information System for Education (SIED) Subcomponent. The SIED project planning and management subsystems were intended to strengthen medium-term planning and monitoring. A study was undertaken by Instituto SER (see Table 7) to analyze functions, processes, organization and administration of the UTPEP as the basis for developing a management information system for use by decentralized entities. At central level, the Ministry could not integrate the entire system developed, owing to incompatibility with other modules of a management information system already developed and in use. 38. Budget and Financial Management. Initially, the Ministry did not possess the requisite accounting and financial management capabilities needed for satisfactory project execution and accounting. Overall project management was very weak in the early years. Internal control inquiries by the General Accounting Office of Colombia in 1991 revealed deficiencies in loan administration, particularly in planning, contracting, and communications with project execution entities. Auditors and the Bank raised questions regarding project accounts and supporting documentation from 1989 to 1993. A satisfactory action program was agreed to correct the difficulties, including development of an adequate accounting and financial management  -9- information system and training of staff in its application. A satisfactory system was developed and was used during the final three years of the project, which was also adopted for use in the subsequent Secondary Education Project. Full documentation for all expenditures was provided by end 1997. 39. Project and Program Implementation. Improvement of technical support at central and departmental levels of MEN as well as departmental and municipal planning, programming, and implementation capabilities evolved slowly. The Ministry designed a technical assistance and training strategy for municipalities and departments consisting of education and financial management tools. A few municipalities, with assistance from universities and NGOs, used the strategy in the final two years of the project (1996/97). The subsequent Secondary Education Project also used the methodology based on this experience. C. Main Factors Affecting Project Implementation 40. Factors Not Subject to MEN Control. Both constitutional and legislative reforms occurred during the project period, with a significant impact on implementation. The new Constitution of 1991 introduced a major decentralization initiative, with emphasis on community participation. The General Education Law of 1994 further decentralized management of funds within the education sector, necessitating a change in coordination arrangements for the Project. The 1991 constitutional changes and the 1994 General Education Law were in consonance with the basic goals of the project. In fact, some of the educational and regulatory reforms adopted by the new law, such as the community participation processes and the active school concept, were directly derived from experiences under the Project. At the same time, they complicated implementation by increasing the number of entities with executing responsibility. The many jurisdictions executing investments varied greatly in technical and managerial capacity, and generally lacked the communications links (especially between municipalities and departments) necessary to meet the new requirements. The Staff Appraisal Report discussed the possibility of substantial legal and institutional reforms and had identified the risk that they could delay implementation. 41. Factors Subject to MEN Control. The high turnover of MEN management and technical staff hindered project implementation. During the life of the project there were six ministers of education, seven planning coordinators, and four teacher training directors. A new project management orientation process was required with each change. This also interrupted the planned sequence of activities and weakened institutions charged with implementation. An integrated information system did not emerge to monitor progress at national, departmental or municipal levels. Information systems that existed were not coordinated and attempts to develop a well- organized, integrated information strategy proved futile. 42. The UPEP was conceived within the framework of a centralist institutional structure in Colombia. Implementation of the overall UPEP program was hampered, however, by lack of clear operational guidelines, despite its having been structured in stages to facilitate its management. The 1991 constitutional reforms set in motion decentralization and community participation initiatives that raised further uncertainties with respect to the roles of national and subnational entities in UPEP program implementation. Moreover, little effort was made to develop monitoring and evaluation mechanisms for execution of the UPEP at the national and subnational levels or among the NGOs. This complicated administration and the legal aspects of implementation. Following the enactment of the Education Law of 1994, The Education Secretariats at departmental level (SEDs) assumed responsibility for managing the UPEP along with financial control of services rendered, including responsibility for storage and distribution of textbooks, libraries, and school furniture-  - 10- responsibilities that previously fell to the MEN's Regional Education Funds. Adaptation to this change took time and slowed implementation. 43. Project management was weak in the early years of implementation. Over half the spending under the project (51.4%) was undertaken between 1989 and 1993, the period during which the national government allocated most of its counterpart resources. While this was an acceptable pace of project execution, financial management and accounting controls were lax during this time (para. 38). During 1989-1992, resources were executed by the MEN Fund. In 1993, the MEN abolished the MEN Fund and directly executed the resources through trust funds. Starting in 1994, the Social Investment Fund (FIS) became the clearinghouse for project disbursements. At first, the FIS faced many organizational problems related to its structure and operational setup. As a result of these combined problems, execution slowed during 1994-1995, with corresponding drops in disbursements. The subsequent requests for loan extension resulted from the need to obtain a better balance between loan and counterpart funds, to accommodate institutional adjustments (after the creation of the FIS), and to clarify the accounting and financial arrangements. The pace of implementation accelerated in 1996 and both quality and pace were excellent in 1997. 44. In the last two years of implementation, the steps taken by MEN to correct earlier management problems were evident in both the PCU and the MEN directorates, resulting in more timely identification, programming, and preparation of subprojects. At the same time, the MEN gave greater attention to its coordinating role, and coordination among all government entities significantly improved. Flexibility and responsiveness on the Bank's part combined with these elements to permit timely resolution of procedural issues, reduced paperwork, and facilitated disbursements from the Special Account. 45. Recommendations made by Bank missions were not always adopted, and subsequent technical assistance, evaluations, and follow-up to correct operational deficiencies in operations were often ineffective. On the other hand, improvement occurred in integrating various MEN functional units into the components, thus obtaining better leadership from MEN line entities, especially during the last two years of implementation. 46. Cumbersome, time-consuming national procurement procedures contributed to delays in contracting under some components. Conversely, the provision of furniture to municipalities through international competitive bidding proved both more rapid and more economical compared to contracting in the first years of the project, when individual contracts were awarded to numerous suppliers. 47. Non-MEN Implementing Agencies. In 1994, the Social Investment Fund (FIS) became the agency responsible for disbursements for project counterpart and credit expenditures. Inadequate administrative capacity seriously hampered its operations. FIS initially lacked adequately trained personnel in sufficient numbers to manage the volume and number of expenditures, along with the numerous tasks related to monitoring, contract agreement settlements, and allocation and transfer of resources. Eventually, FIS developed adequate institutional capacity but was merged with FINDETER in 1997. (This did not have a material effect on the Project). 48. The departments executing the UPEP lacked adequate technical capacity, as well as the communication links with municipalities necessary for smooth project implementation. As a result, the MEN and other central entities (FIS) often worked directly with municipalities, bypassing the departmental level. Nevertheless, technical support provided by the MEN under the project  -11 - achieved some progress in strengthening capacity at departmental level, which proved critical in implementing many activities. 49. The municipalities were able to allocate budgetary appropriations as part of the decentralization process and local financial capacity-building. This generated upward pressure from local to central level (MEN) and fostered the achievement of many objectives related to decentralization of fiscal and program responsibilities in education. D. Project Sustainability 50. The mobilization of financing at municipal level was important not only to the success of the project but also to help ensure the financial sustainability of the UPEP. As noted, however, subsector investtnents are still less than optimal. Continued financial support from central level will be necessary both to sustain operating costs of the education system and to continue and deepen efforts to improve quality. 51. The UPEP's objectives included educational quality improvement as well as expansion of coverage, especially in rural areas. Its principal strategy for quality improvement was to promote use of pedagogical innovations used in Escuela Nueva, using inputs such as textbooks, furniture, and school rehabilitation or construction. Funding instruments were developed for the civil works and training components, and a technical assistance strategy was developed to strengthen educational decentralization, especially at the municipal level. The experience gained was useful in the development of a secondary education project. 52. Additional factors contributing to sustainability of project outcomes and impact include training provided to municipalities by regional universities and NGOs and partnerships developed at central and local levels with the private sector, universities, and NGOs around various issues. E. Bank Performance 53. The performance of the Bank was generally satisfactory. The Bank worked closely with the Government and implementing agencies and supported technical assistance to facilitate implementation of decentralization initiatives, including design of a training strategy for this purpose. The Bank also supported the development of more flexible procurement procedures for purchase of educational materials and school furniture-a bottleneck at the beginning of the project. In addition, Bank promotion of technical capacity building in procurement within MEN and FIS was key to resolving critical implementation problems, and guidance toward effective use of international bidding was key to achieving more competitive, transparent, and less costly procurement of goods. The Bank carried out two supervision missions each year (except in 1991), and local Bank staff held frequent meetings and worked closely with MEN's PCU. This facilitated monitoring of progress in scheduled activities and timely response to proposals to adjust project strategy. Nevertheless, project management problems, especially the absence of an adequate accounting and financial management system, could have been detected earlier. F. Borrower Performance 54. To promote project coordination, MEN created a unit to coordinate the project, initially called the National Executive Secretariat for the UPEP, and thereafter referred to as the Coordinating Unit for Programs Financed by Foreign Credit. Its effectiveness was hampered, however by lack of clarity regarding its role in the Ministry, as well as the respective roles of the  -12 - different jurisdictional levels involved in the execution of the project and UPEP in general. This lack of definition of roles was never fully corrected. Various technical units of MEN were responsible for monitoring achievement of UPEP objectives. Effective monitoring of the program was impeded by high staff turnover in these units. 55. The MEN's project coordination generally focused on budget, administrative, and accounting processes. PCU management through 1994 was generally weak and employed inconsistent administrative and financial criteria (para. 38). Once the problems were identified, however, the Borrower readily agreed with the Bank on an action plan to correct them. All requirements were met satisfactorily by end 1997. The Government was slow to develop procurement capacity, especially for international bidding. Once this capacity was developed, however, the benefits extended to procurement in other MEN projects as well. G. Assessment of Project Outcomes 56. The project outcome was generally satisfactory. Growth in primary school enrolment exceeded targets, although the goal of universal coverage at primary level has not yet been reached. New teaching strategies and innovative methodologies were encouraged, and the quality of primary education was enhanced through in-service training of teachers, distribution of teaching materials, and provision of textbooks and furniture. A national assessment system for student achievement was produced and implemented, and an assessment methodology relating achievement to school characteristics was developed. Both of these instruments have been valuable in planning and assessing alternative interventions in the sector. A dissemination strategy using mass media was developed, including production of television programs, adaptation of international programs, and use of educational software. Additional resources were mobilized for the sector, with municipalities assuming more responsibility for primary education. Participation in the project also led municipalities to develop partnerships with the private sector to expand and improve the quality of primary education, achieving greater participation of civil society in education. Nevertheless, expected improvements to internal efficiency of primary education were limited. H. Future Operations 57. Following the strategy agreed by the Bank and the Government, the Bank initiated two new projects in the education sector, both of which aimed at improving quality of basic education in a decentralized system. The Department of Antioquia and the Municipality of Pasto implement the respective projects. Preparation is underway for an operation to improve the quality of rural education based on partnerships with the private sector and civil society. Lessons learned during the execution of Ln. 3010-CO are incorporated into preparation of this project. I. Key Lessons Learned * Major institutional and political reforms, such as decentralization efforts, that take place during project implementation often delay project implementation-even if, in principle, they support project objectives. If this occurs, the Bank and the Borrower will need time to analyze and reformulate project activities and revise implementation timetables. Project design should be sufficiently flexible to permit timely realignment of activities in line with reform priorities.  - 13 - * Projects that undertake comprehensive national educational reform programs-such as achieving universal basic education, improving quality, and establishing sector finances at the local level-should identify concrete and measurable objectives that can be attained within the project period * Monitoring performance objectives in coordination with representatives of participating national and sub-national entities is crucialfor effective project implementation. In a decentralized context, a basic monitoring and auditing system is essential to provision of timely assistance to local authorities. * An integrated management information system, as well as a network among executing entities, is essentialfor monitoring and supervision ofproject activities. The technical capacity of all participants must be strengthened as necessary through well-designed technical assistance. * Innovative and effective educational concepts andprograms may be difficult to implement broadly unless adequate attention is paid to the political dimensions of change that impinge on educational policy making and may cross conventional dichotomies such as urban-rural school differences and poor versus middle-class students. * Centrally managedprograms that target local governments will require flexible project designs to accommodate the enormous diversity in capabilities, needs, and educational and socioeconomic conditions.   - 15- COLOMBIA SECOND SITBSECTOR PROJECT FOR PRIMARY EDUCATION (Ln. 3010-CO) IMPLEMENTATION COMPLETION REPORT (ICR) PART II: STATISTICAL TABLES PAGE TABLE 1: SUMMARY OF ASSESSMENTS ............................................................ 16 TABLE 2: RELATED BANK CREDITS ............................................................ 17 TABLE 3: PROJECT TIMETABLE ............................................................ 1 8 TABLE 4: PROJECT DISBURSEMENT: CUMULATIVE, ESTIMATED AND ACTUAL (US$MILLION) .................. 18 TABLE 5: KEY INDICATORS FOR PROJECT IMPLEMENTATION ............................................................ 19 TABLE 6: KEY INDICATORS FOR PROJECT OPERATIONS ....................... ..................................... 20 TABLE 7: STUDIES INCLUDED IN THE PROJECT .21 TABLE 8 A: PROJECT COSTS BY COMPONENT .22 TABLE 8 B: PROJECT FUNDING .22 TABLE 9: ECONOMIC COSTS AND BENEFITS .23 TABLE 10: STATUS OF LEGAL COVENANTS .23 TABLE 11: COMPLIANCE WITH OPERATIONAL MANUAL STATEMENTS: .24 TABLE 12: BANK RESOURCES: STAFF INPUTS .24 TABLE 13: BANK RESOURCES: MISSIONS .................... 24  - 16 - Table 1: Summary of Assessments A. Achievement of Objectives Substantial Partial Negligible Not applicable Macro Policies U U 0 E Sector Policies E Il U Il Financial Objectives [l El U a Institutional Development U El U F Physical Objectives El Il U II Poverty Reduction U El E l Gender Issues a El