Malawi - Second Education Sector Credit

 ICRR 10285 Report Number : ICRR10285 ICR Review Operations Evaluation Department 1. Project Data : OEDID : OEDID: C2083 Project ID : P001649 Project Name : Malawi Second Education Sector Credit Country : Malawi Sector : Other Education L/C Number : C2083 Partners involved : Not Applicable Prepared by : Angela Ransom Reviewed by : L.Dove Group Manager : G. Ingram Date Posted : 04/06/1999 2. Project Objectives, Financing, Costs and Components : Objectives Improve the quality of education at all levels Expand access to primary and secondary education Strengthen sector management, budgeting, and planning Improve resource mobilization and allocation policies Components (a) Quality improvements Curriculum revision at primary (PS) and secondary (SS) levels Textbook and materials preparation and distribution Improve distance learning at the PS and SS levels Introduce low cost approaches to improving science instruction and library systems at the SS level Improve teacher training and upgrade teacher skills at the PS level Strengthen measurement of achievement and school inspection at the PS level (b) Access improvements Improve school facilities at the primary level Improve utilization of school facilities (double shifts at the SS level and day students at the university level ) Support staff development at the Faculty of Commerce through TA and provision of equipment (c) Institutional Development Support capacity building in education management, planning, and research Support devolution of key education functions and services to the regions and districts (d) Resource Mobilization Introduce fee structure reform at all education levels Increase the share of the national budget allocated to education Reform the reporting requirements for recurrent and investment expenditure plans Finance Appraisal Estimate: IDA: US$36.90M; Govt.: 4.10M Actual/Latest: IDA: US$41.30M; Govt.: 3.30M Costs Appraisal Estimate: US$41.00M Actual/Latest: US$44.6 3. Achievement of Relevant Objectives : (a) Some institutional strengthening objectives of the project were largely achieved : The technical capacity and expertise of the Malawi Institute of Education (MIE) was strengthened in the area of management of textbook and instructional materials preparation, printing, and distribution . The result was the creation of in-country publishing capacity at MIE . The project strengthened the Malawi National Examinations Board's (MANEB) technical capacity to diversify PS examinations through the incorporation of standardized multiple choice items . The project also increased MANEB's computing capacity. DfID provided TA in the fields of testing and measurement . (b) Other institutional development objectives were partially achieved : The project provided initial support for the MOE plan to decentralize educational management to the districts . The project financed local training and seminars for supervisors and headteachers as well as operating costs and staff houses for district inspectors . Government began to devolve personnel functions and to a lesser extent accounting functions, but additional financial and technical efforts are required . Support to the Faculty of Commerce at the Polytechnic through staff exchange /twinning arrangements, post-graduate studies, fellowship training schemes together with development of local teaching materials and teaching equipment (mostly computers) improved the quality of the Faculty of Commerce and increased modestly the number of accountants produced (from 50 to 65 per year). However high staff turnover and inadequate funding for research have prevented the development of a mature academic community of students and scholars. (c) The project's physical objectives were largely achieved : Textbooks and teachers' guides in four subjects for Standards 1 to 4 were provided as planned at appraisal . Coverage was extended to include Standards 5 to 8. Roofing material for 228 schools were financed as intended . In addition the scope of the component was extended to include the rehabilitation of over 2, 000 unusable classrooms to respond to the sudden and massive expansion of PS enrollment in 1994. (The funds for rehabilitating classrooms were taken from funds originally allocated for furniture and equipment resulting in unsatisfactory provision of the latter ). Quantitative targets for upgrading PS teachers in a short period of time were achieved for the most part . Consistent with policy changes for secondary education (SE), 3,200 additional places in day distance education centers were created through double shifting 4. Significant Achievements : There were some indications of positive project outcomes /impacts: The project served as a vehicle for IDA and the Government to pursue policy dialogue for mobilizing additional resources to the sector. Education's share of total Government expenditures exceeded appraisal targets by almost two-fold, from 11% at appraisal to 23.8% in 1994-95. School fees were increased at the secondary and university levels. Boarding school fees were increased and the Government introduced non -boarding SE as official policy. An innovative and more cost-effective model for upgrading teaching skills of PS teachers, developed in the First Education Sector Credit, was refined and has been adopted by the MOE as the main vehicle for upgrading PS teachers in Malawi. The model, which combines residential training with supervised teaching and field seminars, received positive ratings by a DfID mid -term evaluation team in 1994 as well as by the Center for Education and Research (CERT) at the conclusion of the program . The model has been replicated by the Bank and other donors . The project successfully responded to the urgent need to assist with the rehabilitation of school facilities to accommodate the massive expansion of primary school enrollment (over one million additional children in the first year) following the Governments abolition of school fees in 1994. The critical shortage of textbooks was reduced and an in -country publishing capacity was created at MIE . 5. Significant Shortcomings : (a) Important quality improvement objectives were not achieved : According to the ICR, provision of equipment and furniture for PSs and science kits and learning materials for SSs was unsatisfactory. Only 4% of the furniture (3,000 out of 75,000) planned at appraisal was provided . Funds were reallocated for use in PS classroom rehabilitation . Strengthening of school supervision was unsuccessful . Inspectors were unable to undertake more frequent inspection as planned due to the inadequate recurrent budget for fuel and maintenance of vehicles provided under the project. SS curriculum revision was not undertaken, (b) Project objectives were too broad and ambitious and project design overly complex in relation to sector implementation capacity: The PIU was still completing the previous IDA project when this project was launched . PIU staff were over stretched and this delayed project start up . The Bank introduced new bidding and tendering procedures which project staff had to study and learn how to apply appropriately. This took time and contributed to implementation delays . Economic and financial problems, successive years of drought, and disruption in external assistance constrained Government's ability to release counterpart funds in a timely manner . (c) The indicators of school quality listed in Annex Table 7 of the SAR were not used to measure achievement of project outcomes or as indicators of project impacts . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Partial Modest Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory 7. Lessons of Broad Applicability : ICR Inadequate assessment of implementation capacity during project preparation and appraisal can lead to unsuccessful implementation of key project components . Implementation staff (technical, managerial, and financial ) should be in place before a project is initiated . OED At appraisal in addition to standard reporting requirements, explicit provision for M&E and collection of baseline data against which to measure project progress should be built into project design . This should include explicit short-term output indicators and medium-term results/outcomes expected, as well as long term impact. 8. Audit Recommended? Yes No Why? The project's institutional support to strengthen testing and assessment capacity makes it a likely candidate for input into the proposed learning outcomes study . 9. Comments on Quality of ICR : The ICR provided no explanation for why the relevant set of school quality indicators designed at appraisal were not used in assessing project outcomes . Also the ICR did not assess the scope and quality of data and analysis provided by the M&E committee established in the previous project to set up a viable M&E system to support education policymaking and planning . The ICR's plan for future project operation did not include performance indicators nor a plan for M&E .