Mozambique - Education Sector Strategic Program (essp)

 ICRR 12572 Report Number : ICRR12572 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/14/2007 PROJ ID : P001786 Appraisal Actual Project Name : Education Sector US$M ): Project Costs (US$M): 717.2 n.a. Strategic Program (essp) Country : Mozambique Loan/ US$M): Loan /Credit (US$M): 71.8 73.86 Sector Board : ED Cofinancing (US$M): US$M ): Sector (s): Primary education (70%) Tertiary education (18%) General public administration sector (10%) Vocational training (1%) Adult literacy/non-formal education (1%) Theme (s): Education for all (23% - P) Participation and civic engagement (22% - P) Gender (22% - P) Rural services and infrastructure (22% - P) Social analysis and monitoring (11% - S) L/C Number : C3172 Board Approval Date : 02/18/1999 Partners involved : Closing Date : 06/30/2004 06/30/2006 Evaluator : Panel Reviewer : Group Manager : Group : Helen Abadzi Kris Hallberg Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The development objective of the Education Sector Strategy Program was to provide increased and equitable access to higher quality education through improvement in the management of education in order to promote economic and social development in the country . The key performance indicators were : (a) Increased proportion of students passing key primary and lower secondary examinations : (i) Grade 5 from 54% to 74%t; (ii) Grade 7 from 37 to 60%; and (iii) Grade 10 from 33% to 55%. (b) A reduction in the average repetition by half for both primary and lower secondary levels . (c) An increase in gross enrollment rates : (i) Grade 1-5 from 67% to 86%; (ii) Grades 6-7 from 5% to 30%. (d) An increase in school enrollments, including an increase of at least 75% of the new capacity added in districts where classrooms are built. (e) Implementation of at least 80% of the work program for each year, measured by physical targets, budget spent and routine activities. (f) Achievement of the agreed rate of decentralization of management to the provinces . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (a) Quality of education to improve high repetition and low completion rates (US$25. 8m appraisal, US$16.9m actual) through (i) provision of teacher training; (ii) ensuring pedagogical support for teachers; (iii) curriculum transformation; (iv) preparation of learning materials; (v) reform in assessment and examinations; (vi) Introducing direct support for schools; and (v) training of school directors. (b) Access to education to increase access and ensure equity (US$36.4m appraisal, US$52.2m actual) through: (i) school and classroom construction and rehabilitation; (ii) initiatives for girls' education; (iii) expanding non-formal education; and (iv) improving special education for children with disabilities . (c) Institutional capacity to strengthen management capacity (US$8.3m at appraisal, US$2.7m actual) through (i) reform in the organizational structure and strengthening decentralization; (ii) enhancing skills for policy analysis and planning; (iii) improving financial management; and (iv) ensuring systematic monitoring and evaluation; (d) Developing a strategy for technical and vocational education . (IDA: US$0.5m at appraisal, US$0.4m actual) . (e) Strategic planning for expanding and improving the quality of higher education (No cost provided). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: This was a sector-wide operation with total budget costs of US$ 717.2 million, of which various cofinanciers would provide US$118.8 million; IDA contributed $71.0 million for specific items. The closing date of the project was extended by 24 months due to implementation delays . At closing SDRs 3.0m were canceled.. 3. Relevance of Objectives & Design: The objectives were in line with the Bank's Country Assistance Strategy (CAS) for Mozambique, which emphasized the expansion of human capital and building -up of partnerships with donors for development . The CAS objectives corresponded with the 1990 Paris Declaration, which mandated a shift in donor support to coordinate and streamline aid to developing nations. Building partnerships was especially pertinent to the Mozambican context with multiple and disparate donors and Non-governmental Organizations (NGOs) working in the education sector . 4. Achievement of Objectives (Efficacy): overall modest ). Many activities took place, that led Increased and equitable access to higher quality education (overall to the following changes in key indicators : (a) Increased proportion of students passing key primary and lower secondary examinations (modest): (i) Grade 5 from 54% to 74%; (ii) Grade 7 from 37% to 60%; and (iii) Grade 10 from 33% to 55%. According to the ICR, project documents showed only small improvements in pass percentages since 2000,with grade 5 pass rates around 67%, grade 7 around 53% and grade 10 around 45%. In 2005 pass rates increased and exceeded targets : they were 75% for grades 5 and 7, and 60% for grade 10. The increase was hard to explain because learning improvements had not been implemented, and field visits showed that more than half the students lacked textbooks . Furthermore, completion rates remained low. Only 48% of the students enrolled in grade 5 completed this level of education, 29% in grade 7, 29%,and 5% in grade 10. Provinces in the north and center had lower completion rates than the others . Although there were expectations of improved learning outcomes in secondary education, no attention was given to secondary education curricula and issues . This is one possible reason why some important targets were missed . (b) A reduction in the average repetition by half for both primary and lower secondary levels (modest). The project largely succeeded in reducing the repetition rates by half, as expected . In grade 5 repetition decreased from 24% in 1999 to 11% in 2005; for grade 7 from 25% to 6%; for grade 10 from 25% to 19%; and for grade 12 from 12% to 8%. However, the reduction partly resulted from applying social promotion policies in primary school; the extent to which the reduction was due to learning improvements is unknown . (c) An increase in gross enrollment rates : (i) Grade 1-5 from 67% to 86%; (ii) Grades 6-7 from 5% to 30% (substantial). Enrollments steadily increased at all levels of education and surpassed targets . In grade 5 gross enrollment was 131% (net enrollment increase from about 50 to 83%), In actual numbers enrollment was 2 million in 1999 and about 3.4 million in 2005. The gross enrollment rate for grade 7 was 47% and the net enrollment increased from 2.5 to 6.7%. The percentage of girls enrolled relative to boys remained uneven (42.1% to 46.4%) while in the remaining levels (upper primary, lower and senior secondary ) the increase was around one percentage point ). (d) An increase in school enrollments , including an increase of at least 75% of the new capacity added in districts where classrooms are built (substantial). The target was achieved. About 6,000 primary school classrooms were built ( 650 by the IDA Credit) and 30 secondary schools were built or rehabilitated .  80 % of the work program for each year, measured by physical targets, budget spent (e) Implementation of at least 80% and routine activities (substantial) In every year but the first this objective was achieved . (f) Achievement of the agreed rate of decentralization of management to the provinces (modest). It is unclear whether this objective was met, because the institutional development component was implemented only to a limited extent. 5. Efficiency (not applicable to DPLs): The ICR states that additional education may have led to increased individual wages . The project estimated that an extra year of education would increase wages by about 3% in the agriculture sector and between 5% and 15% in non-farm work. There is also an increase in the rate of return between 1996 and 2002 for graduates at the secondary and higher secondary levels . However it is uncertain whether this increase was obtained, particularly for persons who did not learn much in school . The PAD included a detailed economic and financial analysis (Annex 4), but the ICR did not calculate an ex -post rate of return, so little is known about project efficiency in terms of primary -school student earnings. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project achievements have been substantial with regard to increasing access and enrollments, especially in the lower levels of education. However, improvements in completion, repetition and dropout are not commensurate with this increase in enrollment and fall short of targets . (Indicators shown above). Although the objectives were not changed, there were some reallocations of funds away from "quality" and towards "access". Perhaps these reallocations had an impact on the achievement of quality -oriented outcomes. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: It is unknown how sustainable enrollments are if students learn little and drop out without obtaining the expected knowledge from schools. Similarly, quality objectives may be at risk if large numbers of students are enrolled in classrooms with little ability to teach . Also, further development of the sector depends on the continued collaboration of multiple donors, which has been quite complex . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at entry was unsatisfactory . This project was designed as an investment operation within an overall sector-wide approach framework that partly depended on donor support and funding . The design proved overly complex and ambitious, and there was no legal framework for ensuring funding and coordination of 14 donors. However, the Bank had limited options given the post -conflict environment in Mozambique. Supervision was diligent, and overall performance was satisfactory . at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The borrower showed commitment to educational reform during appraisal and supervision and implemented the project with timeliness and vigor as well as detailed accounting processes . Country-wide plans were prepared for the sector. Despite financial constraints and uncertainty, the government showed skill and a willingness to comply with donor requirements as well as work with the limitations of its own system . Staff also worked on building consensus and putting together strategic plans for different areas, including teacher training and curriculum development. a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design, utilization, and implementation are all rated modest . The PAD outlined an extensive plan for monitoring and evaluating the project, that was implemented to a considerable extent. Ministry of Education units were to develop monitoring instruments relevant to their respective components, and financial monitoring reports were to consolidate statistical, financial and physical data on the rate of implementation. The donor group was to monitor and evaluate their respective components and provide their own reports. However, the plan proved too complex . The Ministry has provided has provided timely and comprehensive financial monitoring reports but no reports on outcomes and component implementation . Also, donor reports were not generated. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): In the first few years of the project it was hard to find skilled staff; however, overall there have been no fiduciary significant issues, and financial management has been satisfactory . However, implementation paid limited attention to the means necessary to obtain the targeted increases in learning outcomes, such as secondary education curricula. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: - To improve quality of education, specific inputs must be provided, such as curricula and textbooks, and teacher training. In particular, it is not possible to improve the conditions of secondary education without inputs for this level. - Complex arrangements involving large numbers of donors require much foresight and planning in terms of harmonization and agreement on sectoral strategy . A shared financing arrangement should incorporate a vision for the development of the sector . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR described frankly and in detail the implementation and issues of the project . However, in annex 3, the total final amount of the proceeds used is 0. Final disbursements are shown as US$ 72.2m, but annex 6 shows at least US$73.86m. The final project cost is unknown, because it is a part of government budget, and much of the promised donor assistance did not materialize . Mentioned in the ICR cover is a loan component of US$ 3.11m. This appears to be an error that will be corrected in a revised ICR copy . a.Quality of ICR Rating : Satisfactory