Algeria - Basic and Secondary Education Support Project

Document of The World Bank FOR OFFICIAL USE ONLY Report No: 22455-ALG IMPLEMENTATION COMPLETION REPORT (CPL-35730; SCL-3573A; SCPD-3573S) ON A LOAN IN THE AMOUNT OF 40US$ MILLION TO THE DEMOCRATIC AND POPULAR REPUBLIC OF ALGERIA FOR A BASIC & SECONDARY EDUCATION SUPPORT PROJECT 09/10/2001 Human Development Sector Middle East and North Africa Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  CURRENCY EQUIVALENTS (Exchange Rate Effective ) Currency Unit = Algerian Dinar 1.00 DZD = US$ 0.013 US$ 1.00 = 77.06 DZD FISCAL YEAR January I st December 31 ABBREVIATIONS AND ACRONYMS CAMEMD Centre d'Approvisionnement et de Maintenance des Equipements et des Moyens Dicdactiques (Center for Procurement and Maintenance of Equipment and Pedagogic Materials) CNF Centre National de Fonnation des Cadres (National Staff Training Center) COSP Centre d'Orientation Scolaire et de Selection Professionnelle (School Orientation and Selection Center) DOE Directions Chargees de l'Orientation et de l'Evaluation (Department for Orientation and Evaluation) IPN Institut Pedagogique National (National Pedagogic Institute) INRE Institut National de la Recherche en Education (National Institute of Education Research) MNE Ministry of National Education PMU Project Management Unit Vice President: Jean-Louis Sarbib Country Manager/Director: Christian Delvoie Sector Manager/Director: Jacques Baudouy Task Team Leader/Task Manager: David Steel  FOR OFFICIAL USE ONLY ALGERIA BASIC & SECONDARY EDUCATION SUPPORT PROJECT CONTENTS Page No. 1. Project Data I 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 1 4. Achievement of Objective and Outputs 4 5. Major Factors Affecting Implementation and Outcome 7 6. Sustainability 8 7. Bank and Borrower Performance 9 8. Lessons Leamed 11 9. Partner Comments 12 10. Additional Information 13 Annex I. Key Performance Indicators/Log Frame Matrix 15 Annex 2. Project Costs and Financing 18 Annex 3. Economic Costs and Benefits 20 Annex 4. Bank Inputs 21 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 22 Annex 6. Ratings of Bank and Borrower Performance 23 Annex 7. List of Supporting Documents 24 This document has a restricted distribution and may be used by recipients only in the perforrnance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.   Project ID: P004954 Project Name: BASIC & SECONDARY EDUCATION SUPPORT PROJECT Team Leader: David J. Steel TL Unit: MNSHD ICR Type: Core ICR Report Date: September 17, 2001 1. Project Data Name: BASIC & SECONDARY EDUCATION SUPPORT L/C/TFNumber: CPL-35730; PROJECT SCL-3573A; SCPD-3573S Country/Department: ALGERIA Region: Middle East and North Africa Region Sector/subsector: EP - Primary Education KEY DATES Original Revised/Actual PCD: 06/20/1991 Ef4&ctive: 03/23/1994 Appraisal: 11/19/1992 MTR: 04/02/1999 Approval: 03/16/1993 Closing: 03/31/2001 Borrower/Implementing Agency: GOVT. OF ALGERIA/MIN. OF EDUCATION Other Partners: STAFF Current At Appraisal Vice President: Jean-Louis Sarbib Caio Koch-Weser Countly Manager: Christian Delvoie Pieter P. Bottelier Sector Manager: Jacques Baudouy Roslyn Hees Team Leader at ICR: David Steel ICR Primary Author: Dung-Kim Pham 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainability: L Instituetional Development Impact: M Bank Performance: S Borrower PerJbrmance: S QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: As indicated in the appraisal report, the project will be the first step in a long-termn process of  improving the quality of learning acquired by students and increasing the efficiency of resource use in basic and secondary education. This objective is clearly stated and correctly reflects the government's strategy with respect to educational quality and efficiency as well as the government's decision to limit the project scope to the immediate needs of those sub-sectors. In the absence of a Country Assistance Strategy, this objective emerged from a sector study: "Democratic and Popular Republic of Algeria - Improving Quality and Efficiency in Basic and Secondary Education," World Bank Report No. 8912-AL, August 1990, prepared jointly with the government. It identified and analyzed a number of organizational/structural and pedagogical problems and offered potential solutions. The project's objective was thus both realistic and relevant. Since the quantitative goals of increasing school enrollment were largely achieved by the Government, the project focused its priority to improving quality and efficiency of the sub-sectors in the areas of assessment of learning achievements, pedagogic research, and pedagogic materials. Furthermore, the project simply aimed to initiate the process of quality and efficiency reform, which will be a long and demanding undertaking. 3.2 Revised Objective: The project objective was not revised. 3.3 Original Components: The Project comprises three components. A description of the Government entities involved in the implementation of the project appears in Section 10, "Additional Information." Component I: Design and introduction of a pilot system for assessing learning achievements, quality, and efficiency. Cost: US$ 3.7 million. This component, managed by the Centres d'Orientation Scolaire and de Selection Professionnelle - COSP (School Orientation and Selection Centers) in six wilayas aimed to (i) increase the capabilities of the Ministry of National Education (MNE) in the area of knowledge and quality assessment; and (ii) disseminate infornation on new assessment techniques, methods, and practices within the education community and among decision-makers. Component II: Pedagogic research. Cost: $US 2.3 million. This component supported training and implementation activities linked to the pedagogic research programs of Institut Pedagogique National - IPN (National Pedagogic Institute). In 1996, the name of IPN was changed to Institut National de Recherche en Education - INRE (National Institute of Education Research). In the context of the project, this change did not affect Component II because INRE's mission was still to perform education-related research, evaluate the education system, and develop and test teaching methods. Component III: Improved provision and more efficient use of pedagogic materials. Cost: US$ 58 million. The purpose of this component was to improve the supply and use of pedagogic materials in schools through: (i) provision of pedagogic materials for about 10,500 existing and 2,500 grade 1-6 schools under construction, and 750 grade 7-9 and 250 secondary schools under construction; (ii) improved provision and use of school science laboratory products in about 3,500 grade 7-12 schools, rehabilitation of the Centre d'Approvisionnement et de Maintenance des Equipements et des Moyens Didactiques - CAMEMD (Center for Procurement and Maintenance of Equipment and Pedagogic Materials) storage and redistribution facilities for such materials and the provision of related equipment and staff training; and (iii) - 2 -  provision of spare parts, equipment, technical assistance, staff training and improved CAMEMD maintenance facilities for improved maintenance services to schools. The design of the three components was well in line with the project objectives. Although aware of the decline in educational quality and output, the MNE lacked the proper tools to measure the system's performance and to assess the impact of the reform measures. The activities of Components I and II filled this gap by providing the MNE with an essential lever. Component I also included a powerful tool for disseminating the results of the assessments to decision-makers and obtaining feedback from them. Other elements essential to improving the quality and efficiency of the education system are an adequate supply of pedagogic materials and their efficient utilization. Most schools were without any pedagogic materials, and the supplies and equipment necessary to implementation of the reforms at the basic education level had not been provided. Component III helped make up this deficit by supplying the schools with the appropriate pedagogic materials. It also included an important element of ensuring a more efficient utilization of those materials by training high-level technicians and a large number of science teachers in their use and application. The design of the components took into account experience from prior loans by choosing a simple implementation structure. Thus, implementation of each component was entrusted to one of the entities operating under the authority of the MNE, the project activities being arranged to fit into each one's work program. However, this design did not take into account the restrictive administrative arrangement for Component I, in which technical implementation and budgetary execution were entrusted to two different bodies (one institution under the ministry's supervisory authority, INRE; and a central MNE directorate, the Direction de l'Evaluation, de l'Orientation et de la Communication - DEOC (Directorate of Assessment, Orientation, and Communications). This arrangement was an obstacle to the efficient implementation of Component I (see details below). 3.4 Revised Components: Whereas the objectives were not revised, the "quantitative" side of Component III was slightly modified. In physical terms, the laboratory products packaging and storage unit was dropped and the overall spare parts requirement scaled down. The unutilized funds were reallocated to supply a number of additional schools whose facilities had been destroyed by acts of terrorism, a total of 193 primary schools and 76 general secondary schools. In financial terms, of an unutilized balance of US$11.8 million, US$5 million was canceled in July, 2000 at the request of the Government, while over US$6 million financed the additional schools. 3.5 Quality at Entry: The quality of project preparation is considered satisfactory based on several factors, of which the following are the most notable: (i) the project design was the result of an in-depth sector study, directly meeting the priorities of the Algerian govenmment, which, at the time, was looking at the qualitative aspects of the system after achieving substantial quantitative improvements in school enrollment; (ii) the modest scope of the objective which addressed the early stages of a longer process of improving the quality of learning acquired by the students and increasing the sector's efficiency; (iii) given the risks associated with Algeria's political and economic situation and with the limited project implementation capacity of government agencies, a simple project design was essential; (iv) preparatory steps taken in advance of the project, such as the issue of invitations to bid on educational materials and of requests for proposals from - 3 -  international technical consultants prior to loan effectiveness. However, the quality of project preparation could have been enhanced by the presence of performance indicators, with clear distinction between output and outcome indicators, with base values and target values. These indicators would have contributed to a better assessment of achievement of the development objectives and project sustainability. Also, viewed retrospectively, while most on-site missions were not feasible because of security concerns, including in particular the mid-term review mission, the project could have provided for an impact evaluation study towards the end to evaluate outcomes as well as identify areas in need of additional action. Furthermore, the study evaluating the system of textbook production and distribution, and the study on the maintenance and security of pedagogic materials in schools should also have benefitted from advance preparation, as indicated in (iv) above. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Despite the difficult economic and political climate in Algeria during the project implementation period, the Government has supported the project objective and continued to assign priority to improving the quality and efficiency of the system. Overall, the project has achieved its objective. It has provided the MNE with a basic capacity for assessment of learning achievements and education research which will serve as a foundation on which the MNE can continue to build in order to better gauge what students are learning, what is being taught, and how it is being taught (Section 6 "Sustainability"). Pilot system for assessing learning achievements, quality, and efficiency: The objective has been partially achieved. The pilot system has not yet been set up. However, substantial experience in pedagogic techniques of assessment and in survey preparation and monitoring has been acquired as a result of the work carried out during the project in connection with pedagogic performance surveys of the following disciplines and levels: Arabic language (3rd, 6th and 9th grade of basic education), mathematics (3rd, 6th and 9th grade of basic education and I st grade of secondary education), French language (9th grade of basic education). DEOC also succeeded in establishing operational indicators, based on those of the OECD, to facilitate assessment of educational quality and efficiency. This task will be continued systematically to serve as an assessment tool. Pedagogic research: The objective was achieved, and the outcomes are encouraging, as shown by the mastering of the application of pedagogic research methodologies and techniques by INRE. As a corollary to the analysis of school programs undertaken by INRE, curricula and textbook content were revised, using in-house capacities. The outcomes were widely distributed (through the publication of study and training workshop reports), including to the decision-makers in the education system, particularly the National Commission on Education Reform set up by the President of the Republic. Furthemmore, the education community now has more information available in the area of pedagogic research and new teaching methods thanks to the INRE library, which has modemized and strengthened its documentation resources. Improvedprovision and more efficient use ofpedagogic materials: This objective was achieved but with one significant qualification. While the "physical" inputs of pedagogic materials were fully achieved and beneficial, CAMEMD's services remained centralized and their quality did not improve much. An assessment based on a sampling of teachers and students in establishments served by the Center, in Guelma and Tlemcen, revealed that the availability of equipment had led to better application of curricula and progressions and greater student interest in classroom and practical work. However, it - 4 -  appeared that the services delivered by CAMEMD in the areas of procurement, repair, and maintenance were inefficient. The quality of the materials received was variable, their arrival was often not synchronized with the start of the school year, packages of supplies were sometimes incomplete, and schools were often turning to the private sector for repairs. The maintenance study, which was not completed, would have enabled CAMEMD to redefine its responsibilities within the context of decentralization and of a closer association with the private sector. 4.2 Outputs by components: The project components had varying outputs. Component I - Design and introduction of a pilot system for assessing learning achievements, quality and efficiency (implementing agencies: DEOC and INRE) The MNE did not have sufficient capacity to assess the students' learning achievements. Although the educators were aware of the decline in the quality and efficiency of education, the reasons were not fully understood, let alone acknowledged. This component aimed to improve the MNE's capacities for assessing learning achievements and quality and to disseminate the new assessment techniques and practices to the education community and the decision-makers. This component was achieved only partially. The activities linked to the design of a pilot system for assessing learning achievements were not fully accomplished. The discipline surveys targeting the different education levels did take place, but this activity was suspended at the critical phase of analyzing the results and preparing the reports. Consequently, neither the activities linked to tests of the system in the six COSPs nor those scheduled for the second phase in seven additional wilaya took place. The multiplier effect of the training received abroad in assessment methodology was very modest, and the training of on-site inspectors did not come about. Within the framework of this component, it was also planned to develop quality and efficiency indicators for the education system, based on the OECD model. This task was performed satisfactorily. Indicators for 1999 and 2000 were established. The 2000 report, containing 29 indicators, has been sent to the printers and will be distributed to the education community. Although the extent to which this component was accomplished fell short of expectations, it did contribute to the strengthening of the DEOC's institutional capacities in the areas of survey preparation and execution and data quality verification. It also led to the development of capacities for devising performance indicators for the education system. Component II - Pedagogic research (Implementing agency: INRE) Given the inadequacy of INRE's pedagogic research capacities it was unable to make a systematic evaluation of the impact of the reform measures, and, consequently, of the quality of the instruction provided, or to develop curricula based on pedagogic programs. The purpose of this component was to strengthen INRE's capacities for analyzing pedagogic programs so as to improve the consistency between disciplines and basic and secondary education cycles. Within the project framework, only two disciplines (Arabic and science) were covered. Implementation of this component was satisfactory. Both the quantitative and the qualitative goals - 5 -  were achieved, and sometimes even exceeded. As planned by the project, 36 experts in pedagogy were trained in curriculum and textbook design. This training was then passed on to wilaya personnel. Pedagogic research programs, supported by international technical assistance, were also conducted, and the findings distributed to large numbers of inspectors and teachers by means of seminars and publications. An assessment of the programs covered by the component (Arabic and science) was carried out and even expanded to other programs and other textbooks in subjects such as philosophy, history, geography, mathematics, and French. The results of the programs assessment were published in a memorandum, of which 6,000 copies were printed for distribution within the education community. A large number of seminars were organized and brochures on their conclusions also produced. Two outcomes are worth mentioning: - A number of textbooks in Arabic for the different education cycles, for the disciplines of science, letters, and mathematics were revised to take account of the results of the programs assessment carried out by INRE. In future, INRE will include among its functions the work of preparing textbook specifications. - INRE was able to make the results of its pedagogic research available to the National Commission on Education Reform and thereby contribute to the Commission's work. Also within the framework of the component, INRE was able to add around 5,000 technical documentation to its library. Although the library still lacks infrastructure and furniture, its filing system is up and running and, with a satisfactory volume of documentation, it has been able to open its doors to educators, researchers and students in the area of education science. The attendance rate at the library, which has become a benchmark, is a good indicator of the availability and quality of its documents and of the services offered to researchers and teachers. Component II - Improved supply and more effective use of pedagogic materials (Implementing agency: CAMEMD) In the absence of materials (particularly for grades I - 9, since implementation of the new basic education system required new teaching inputs), and given that pedagogic materials represent one of the essential inputs contributing to an improved educational environment, this component was designed to provide appropriate pedagogic materials to around 14,000 primary schools (representing 85 percent of existing schools) and 250 secondary schools (35 percent of existing schools) and to establish a central laboratory products storage and packaging unit. Its purpose was also to make the use of pedagogic materials more efficient by strengthening CAMEMD's maintenance facilities. The quantitative goals of this component, i.e. number of schools receiving pedagogic materials, were achieved: the 100 percent coverage rate set for the year 2000 was met, and stocks are available for the new primary schools. A further equipment tranche was obtained to supply 269 additional establishments. This financing was provided by funds coming from the dropped laboratory products storage and packaging unit, and from the reduced needs in equipment spare parts. Activities connected with the unit were abandoned for reasons of security, but arrangements were made to provide the laboratory products to the schools. A total of 27 professionals received training abroad (nine in IT and management, ten in machinery/tools, four in reeling/coiling, and four in electronics). However, the maintenance and security study included in the project, which was to have been completed by December 31, 1994, was launched in only early 1999. Because of the country's security situation at the time, the consultants were unable to conduct the study and the contract was canceled by - 6 -  mutual agreement between CAMEMD and the Bank. Since then, with the evolving economic environment and the expansion in the scope of CAMEMD's responsibilities, it was decided to redefine the purpose of the study, dividing it into two different studies, one concerning a strategy for the procurement and maintenance of school equipment, and the other concerning the development of a maintenance master plan. At the end of the project, no action had been taken on these two studies. CAMEMD did state, however, that it planned to undertake the studies on its own after the project closing date as it recognized their importance. 4.3 Net Present Value/Economic rate of return: Not applicable. 4.4 Financial rate /f return: Not applicable. 4.5 Institutional development impact: The project's contribution to the building of institutional capacities varies among the different implementing agencies within the MNE. Those capacities were acquired thanks to the commitment of the MNE's professionals, their determination to improve the system, and the productive use of the international technical assistance furnished within the project framework. Where INRE is concerned, the impact has been substantial. Its staff has acquired an understanding of the techniques of analysis and design of school programs and assessment of subjects taught, such that INRE has been able to expand these activities to cover subjects not included in the project. INRE professionals are now performing tasks that will enable them to upgrade teaching methods and develop pedagogic objectives for textbook content. In the case of DEOC and CAMEMD, the impact is less marked because of the non-completion of "qualitative" activities. Thanks to the surveys conducted, DEOC and the education community (inspectors and teachers) in the wilaya have acquired a familiarity with techniques for assessing learning achievements, supervision and survey methods, and validation of results. Using its own capacities, DEOC set up test surveys during a period when no technical assistance was available. In CAMEMD's case, this division has strengthened its conpetence in terms of project management, enabling it to understand the need to plan, monitor, and synchronize the different stages of procurement and delivery of the materials to schools. 5. Major Factors Affecting Implementation and Outcome 5.] Factors outside the control of government or implementing agency: The economic crisis of 1998 and 1999 resulting from the drop in the price of oil in 1998 was a hindrance to project implementation because it led to a substantial reduction in the project budget. This was a particular obstacle to Component I activities, since it affected funding for the surveys. In addition, a succession of changes in the Bank's project teams affected "qualitative" activities to a certain degree. The section entitled "Bank Performance" (see below) discusses this factor in detail. In general, inadequate attention was paid to the monitoring and analysis of the assessment and pedagogic research activities. 5.2 Factors generally subject to government control: Thanks to the Ministry of National Education 's commitment to its policy of improving the quality and efficiency of basic and secondary education, the project was able to continue working toward its - 7 -  objective. This was in spite of the country's security situation which had an adverse influence on project implementation. Site visits by international technical assistance consultants and the Bank were not able to take place. In addition, loan disbursements were held up because the suppliers were having difficulties establishing letters of credit. Budgetary constraints during two budget years (1998 and 1999) particularly affected the implementation of Component I, which required funding to pay the DEOC staff participating in the surveys. The Government's employment policy encouraging early retirement lessened the multiplier effects of training, since certain staff took the technical training offered and then left government service. In addition, the complex administrative procedures involved in payment and customs clearance in general delayed loan disbursements, but also delayed, and sometimes even prevented, the procurement of certain technical periodicals and journals for INRE's library. 5.3 Factors generally subject to implementing agency control: Because of the lack of coordination, combined with a string of changes in DEOC and INRE professional staffs, Component I was only partially satisfactory. As noted above (see end of paragraph 3.3), technical implementation and budgetary control under Component 1, entrusted to different entities, disrupted the progress of that component's activities. The slow-down in the management and overall project coordination on the part of the Project Management Unit also affected project implementation, as the difficulties were not acknowledged in time for corrective decisions to be made. 5.4 Costs andfinancing: The total project cost was estimated at US$64 million, of which US$40 million was financed by the Bank loan and the remainder by the Algerian Government. During implementation, the unused funds resulting from the decision to drop the laboratory products storage and packaging unit and reduce spare parts requirements (resulting from an initial overestimation) were reallocated to an additional tranche of pedagogic materials for 193 primary schools and 76 secondary schools whose facilities had been damaged by acts of terrorism. At the Goverrnent's request, US$5 million was canceled from the loan in August 2000 following a review of disbursement forecasts. At the closing of the project books (31 July 2001), after the 4 month-grace period, a total of US$30.7 million had been disbursed. The undisbursed balance will be canceled. The loan withdrawal rate was primarily affected by the following factors: (i) complex payment procedures (see 5.2 above) and (ii) the cancellation of two contracts for pedagogic materials under the additional round of procurement, since the suppliers were unable to respect the delivery date imposed by the loan closing date. 6. Sustainability 6.1 Rationale for sustainability rating: The sustainability of this project is probable because the quality and efficiency of education are central elements of the reform process introduced by the National Commission on Education Reform and under way since May 2000. Moreover, the Government is aware that the actions supported by the project are essential to continued improvement of the system. They contribute to an understanding of the causes of declining quality and efficiency so that the appropriate steps may be taken to bring about improvements. - 8 -  The Govermment has also asked the Bank to undertake a review of public expenditure in the social sectors, to help it identify institutional reforms likely to improve efficiency and enable it to rationalize resource utilization. This report was completed in August 2001. More specifically, the project's sustainability is already assured by INRE, which has expanded its work program and textbook analysis and revision to other subjects. DEOC will continue to work on developing a system for assessing learning achievements, and will continue to produce quality and efficiency indicators, since these tasks fall within the scope of its mission. The project's sustainability would be enhanced by the development of a strategy in pedagogic equipment procurement and maintenance which will redefine CAMEMD's role within the context of decentralization and of the expected role of the private sector. In addition, although the project did not contemplate an impact evaluation study towards the end of the project (see paragraph 3.5), the MNE has included this study in its own program. Likewise, it has plans to train additional specialists in the development of educational indicators. 6.2 Transition arrangement to regular operations: The project activities are already incorporated in the MNE's normal program, so that no special transition arrangements are required. They will continue as at present, and will be financed out of the MNE budget. The education sector, being part of human resource development, falls within the Government's budgetary priorities for the 2001-2004 economic recovery program. The Bank's mission was given the following commitments by the implementing agencies. Continuation of the learning assessment work With respect to the project's two principal activities, the following measures will be undertaken: (i) first, the unfinished task of analyzing the survey results and preparing the reports will be completed. The task of establishing a system of leaming assessment will definitely be the next step, since this is part of DEOC's mission; (ii) the report on education system indicators will be updated every two years and distributed to the education community and the decision-makers. Continuation of pedagogic research. As discussed above, the fact that assessment and revision of the programs have been extended to other disciplines is evidence that the project activities have already been incorporated into INRE's normal program. Improvement of the system of pedagogic equipment procurement and maintenance. Implementation of this component has already helped CAMEMD begin to develop capacities in planning and management of procurement work. In addition, a new management structure is in place, demonstrating a commitment to redefining CAMEMD's role, with the purpose of making the procurement and maintenance system more efficient. The two studies establishing the strategy for the system will be undertaken during 2001. One of the studies will outline a strategy for school equipment procurement and maintenance, and the other will develop a maintenance master plan. 7. Bank and Borrower Performance Bank 7.1 Lending: During project preparation, the Bank's performance was satisfactory. The project team had a range of pertinent qualifications, with expertise in the areas of learning assessment, pedagogic materials, and project management. Project design was based on an in-depth study of the basic and secondary education sector, and reflected government priorities. Therefore, it focused on key elements for -9-  improvement of the quality and efficiency of the education system. The project team clearly identified the principal risks in terms of the country's political and economic situation and the implementation capacities of the government agencies, and thus balanced the design of the project accordingly. In addition, since the textbook publication and distribution system represents a determining factor in the quality and efficiency of the sub-sectors, and wishing to maintain the dialogue with the Government on this matter, the project team included in a supplemental letter to the loan agreement the legal requirement that a study be conducted conceming the assessment of that system. In retrospect, the terms of reference of that study should have been a condition for loan effectiveness. Also, it would have been useful to include an impact evaluation study towards the end of the project in order to assess the project's achievements and identify the challenges still remaining (particularly since the country's difficult security circumstances prevented supervision missions and the mid-term review mission from taking place within Algeria). The Bank prepared an appraisal report clearly describing the problems of the sub-sectors and the activities supported by the project. The report also contains a detailed implementation timetable for project activities and performance indicator matrices. These indicators would have been more useful had they been more clearly defined, with a distinction between indicators measuring outputs and those measuring the project's outcomes and impact. 7.2 Supervision: Bank performance was only moderately satisfactory. This outcome was partly affected by Algeria's difficult security circumstances, which hindered project supervision and led to high staff turnover. Faced with the circumstances, the Bank adopted innovative solutions, organizing missions outside Algeria and calling on the services of local consultants. Despite those efforts, the continuity of the work was not secured and decisions could not be made rapidly when difficulties arose, particularly during implementation of Component 1. Turn over in task management of the project happened several times following loan effectiveness, and information sharing among the different teams was poor. Inadequate attention was given to the monitoring and analysis of activities related to the project's "qualitative" aspects. In addition, the Bank failed to enforce the deadlines for two studies stipulated in the loan agreement, namnely a study on the maintenance and safety of pedagogic materials in the schools (December 31, 1994) and a study conceming assessment of the textbook publication and distribution system (December 31, 1995). 7.3 Overall Bank performance: Overall, the Bank's performance was satisfactory, despite its only moderately satisfactory performance for project supervision. The Bank continued to support the Government during the difficult circumstances, adopting innovative solutions to get around the difficulties. With government support, it organized missions outside Algeria, called upon the services of local consultants, and also had staff go to Algeria when security conditions there were not favorable. Borrower 7.4 Preparation: The Government's performance was satisfactory. Its participation in the Bank's sector work resulted in production of the study entitled "Improving Quality and Efficiency in Basic and Secondary Education in Algeria - Report No. 8912-AL," which identified the priorities endorsed by the project. The - 10-  Government also participated actively at the project design stage, which resulted in a tightening of the project's scope to limit it to the immediate needs of the sub-sectors. 7.5 Government implemnentation performnance: The Government's performance was only moderately satisfactory. Although the Government remained committed to the project's overall objectives, and collaborated fully with the Bank during difficult periods, it did not meet its contractual obligations. The deadlines for two required studies were not met (a study on the maintenance and safety of pedagogic materials, and a study concerning assessment of the textbook publication and distribution system). In addition, no decision was taken to solve the conflict between DEOC and INRE, which persisted throughout the project and hampered the achievement of results under Component I. Also, the lack of government counterpart funds restricted project activities during 1999. 7.6 Implementinig Agency: The implementing agencies' performance was highly satisfactory. Despite such unfavorable factors as a succession of changes of management, a lack of budget resources, and a scaling down of international technical assistance, the implementing agencies remained committed to their task and succeeded in accomplishing most of the project activities, putting in practice the experience acquired through the limited international technical assistance that it was possible to provide. In the same spirit, DEOC and INRE should have been able to override the administrative constraints and to assure a better outcome for Component I. The overall coordination of project activities by the Project Management Unit (PMU) was less satisfactory. Meetings were rarely held with the MNE, and progress reports on the project were prepared less frequently during the final years of the project. 7.7 Overall Borrower performance: Overall performance was satisfactory. Despite difficult security and economic situations, the Government remained committed to its policy of improving the education system, and provided a good framework for project implementation under difficult circumstances. Also, thanks to its help, the organization of project supervision missions was able to take place outside Algeria, and those Bank missions that were carried out within the country benefited from heightened attention on the part of the Government. 8. Lessons Learned A. Positive lessons leamed from the project (i) A project with realistic objectives. designed to fit in with the Government's priorities. has a greater chance of success. Thanks to those characteristics, this project was able to keep going during difficult periods, requiring little decision-making on the part of the Government or the Bank. (ii) A commitment by the government officials directly involved in project activities is essential. This type of commitment contributes to the continued pursuit of project objectives and lessens the risks of changes of priorities, even at times of changes in the political leadership. - 1 1 -  (iii) The necessary preparatory work prior to project start-up enables a project to maintain its momentun and to move steadily forward as soon as the loan is effective. Within the framework of this particular project, invitations to bid were sent to the suppliers of pedagogic materials and requests for proposals for technical assistance and training were issued prior to loan effectiveness. This preparatory work has significantly contributed to the degree of achievement of the project. B. Negative lessons learned from the project (i) It is important to establish distinct output and outcome indicators as well as outcome/impact indicators with a base value and a target value that will help assess the degree of achievement of the development objectives and the project's sustainability. Those indicators are particularly necessary for projects with objectives of an intangible nature, such as those relating to the quality and efficiency of the education system. (ii) It is worthwhile to undertake an impact evaluation of the project to provide an objective analysis of project achievements and identify the challenges still to be met. These results can serve as a basis for new operations. Such a study would have benefited this project, particularly since supervision missions were so restricted. (iii) Greater attention should have been paid to the enforcement of contractual obligations, with an action plan for handling cases of noncompliance with the provisions of the loan agreement. Activities designated contractual obligations are regarded as essential to a project's development. In the event of tardy compliance, or outright noncompliance, action should be taken not only to satisfy the "legal" aspects of the case but also to ensure that the project's objectives were fully met. (iv) It is necessarv to strengthen the system for the passing on of information when there are changes in Bank proiect supervision teams. This ensures continuation of the dialogue with the borrowers, and reduces the risk of unintentional changes in the project's priorities. The following approach would be helpful: (i) presence of the person in charge of project preparation during the first year of project implementation; (ii) updating of project progress reports and more complete project status reports, with greater emphasis put on the achievement of the project development objectives. C. General lessons. To have a significant impact, projects in the education sector need to place an emphasis on reforming the system. Without far-reaching reforms addressing such issues as quality, efficiency, and equity, the education system cannot continue to advance. Investing in such reforms will be a longer process and will demand greater effort. The following factors should be considered: (i) maintenance of an ongoing dialogue with borrowers; (ii) involvement of all parties making decisions concerning the system or directly benefiting from it (teachers, community, parents, etc.); (iii) in-depth assessment of the organization and capacity of the institutions; (iv) presence of an efficiently performing system of assessment of learning outcomes, making it possible to evaluate curricula, textbooks, teaching methods and programs, examination systems, etc. 9. Partner Comments (a) Borrower/implenmenting agency: During its review of the report prepared by the Bank, the Government commented that overall the report gives an accurate picture of the project outcomes, the stages through which the project passed and - 12-  clearly identifies the positive and negative factors that affected its implementation and somewhat impacted its outcomes. The MNE also made a few small comments regarding form, which were included in the report. Two comments are reproduced in detail here, since they express MNE's position and are therefore worthy of note. (i) Paragraph 4-2. Outcomes by component. Component I: Contrary to what is stated in the report, the Ministry of Education has always, since the seventies, had sufficient human capacities to carry out leaming outcome assessments (as far back as 1976, trials were carried out by school and job orientation counselors on tests of learning in several school sub-sectors). (ii) Paragraph 7-5: Government implementation performance. The Government did not fail to honor its contractual obligations, evidence of this being the study on maintenance entrusted in 1995 to IBF, a Belgian consulting firm. All the equipment surveys were carried out using CAMEMD's own resources in 1997 and 1998, as was the data input. It was not until 1999, during the supervision mission, that the decision to cancel the contract was taken, since IBF had failed to meet its contractual obligations. Furthermore, the Bank indicated that terms of reference would be sent to CAMEMD the next month, i.e. in June 1999, so that the study could be completed by a domestic consulting firm (see aide-memoire of May 1999). As a result of rapid turn over in task management, the terms of reference were never sent. The copy of the Government's cornments remains in the project files. A version of the Government's report is annexed hereto. (b) Cofinanciers: Not applicable. (c) Other partners (NGO.s/private sector): Not applicable. 10. Additional Information A brief description of the principal education sector bodies involved in implementation of the project is given below. A. The Directorate of Assessment, Orientation and Communication (DEOC) is the body within the Ministry of National Education that monitors student learning achievements. It has authority over the School Orientation and Selection Centers (COSPs), and provides them with technical supervision (definition of the broad lines of the activities program - general guidelines for orientation programs and methods). The COSPs are external centers responsible for activities linked to the orientation of students enrolled in the different education cycles, particularly at the time of transition from one cycle to another, the provision of information to students and parents, and the assessment of learning achievements and of teaching and training programs and methods, in liaison with the other bodies and agencies concerned. At the local level, the COSPs report to the Education Director of the respective wilaya. The technical personnel of the COSPs comprises school and job orientation counselors who have received specialized university training in school orientation and selection and applied psychology. - 13-  The COSPs have an activities program that covers the school year. A COSP is headed by a Director, who has direct oversight over the technical staff (school and job orientation counselors) and administrative and service staff. It has an operating budget allocated by the Government. COSPs are managed under the direct administrative supervision of the Education Director of the respective wilaya. B. The National Institute of Education Research (INRE) was established in 1996 to take the place of National Pedagogic Institute (IPN). INRE is responsible for pedagogic research, the continuous assessment of the education system, and the preparation of instructional materials. INRE also assess the impact of education policy. INRE is administered by an orientation council and managed by a director general, and has a scientific council and a committee responsible for approving approve pedagogic materials and supporting resources. C. The National Center for Procurement and Maintenance of Equipment and Pedagogic Materials (CAMEMD) was set up in 1986. Its supervisory authority is the MNE and it is financially autonomous. It is responsible for the procurement and maintenance of pedagogic equipment. It works in close collaboration with the Planning Directorate, which it responsible for registration and monitoring of equipment procurement operations and standardization. CAMEMD also trains technicians (pre- and in-service training courses) and monitors pedagogic equipment use. CAMEMD maintains links with the wilaya through its eight regional antennas. - 14 -  Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: Indicator/Matrix Projected in last PSR Actual/Latest Estimate Component 1: Data non available Component 1: 1.1 Improved measurement techniques of 1.1, 1.2, 1.3 Results were not totally pupils' leaming achievement in the DOE at achieved, as the expected activities remained central level and in six participating COSP. uncompleted. 1.2 Introduction of improved local training 1.4 Education indicators were produced but techniques for inspectors and teachers in have not been used as assessment tools. testing, assessment and orientation and selection of pupils in the six pilot wilaya. 1.3 Expansion of the assessment system to seven additonal wilaya which are to be added and financed by the MEN in a second phase (expected 1997-98). 1.4 Improved quality and relevance of curricula in the four covered subjects as compared to international standards (OECD education indicators model). Component 2: Component 2: 2.1 Improved measurement techniques of 2.1 Assessment techniques of INRE were pupils' learning achievement with appropriate improved through intemational technical pedagogic research at classroom and school cooperation and training seminars, thereby level and at central level and in the six enabling INRE to expand its pedagogic participating COSP. research work to other subjects (see below) and to contribute to the work of the National 2.2 Improved feed-back to and from Commission for the Education System inspectors and teachers on pedagogic Reform. research results in the topics covered. 2.2 This was achieved through numerous 2.3 Improved linkages between curricula and information seminars and the publishing of revised texts for the prescribed schoolbooks study reports broadly disseminated to the at 1, 2, and 3 cycles of basic and secondary educabon community. (documents available levels in Arabic and Physical Sciences. in the Bank's archives). 2.4 Development of improved teacher 2.3 This was achieved and went beyond documentation for Arabic and Physical project expectations as the INRE work Sciences at all levels of basic and secondary reached more topics than prescribed by the education. project. 2.5 Improved availability of pedagogic 2.4 The following results assessment background informaton and improved reports were produced : (i) Arabic (grade 1, functioning of the INRE library. 3, 6, and 9 BE and 3 SE); (ii) Mathematics (grade 9 BE, 3 SE); (iii) Physics and Chemistry (grade 1 and 3 SE); (iv) French (1 st foreign language at all cycles, 2nd foreign language at grade 8 BE and 1 SE); (v) English (grade 4 and 9 BE and 3 SE). 2.5 The INRE library modernization and strengthening in terms of documentation contributed to increased availability of pedagogic information. The Bank's mission noticed the presence of education science students and researchers in the library. Component 3: Component 3: 3.1 Improved provision of pedagogic 3.1 All the materials expected under the materials for 14,000 basic and secondary project were purchased and distributed to schools, as follows: school facilities. Field visits could not be Dec. 1994 - 10% covered carried out during the ICR mission but the Dec. 1995 - 25% covered mission was informed that the 100% Dec. 1996 - 40% covered coverage rate has been reached. - 15-  Dec. 1997 - 60% covered Dec. 1998 - 80% covered 3.2, 3.5 Field visits could not be carried out Dec. 1999 - 90% covered during the ICR mission. The assessment Dec. 2000 - 100% covered tools have not yet been used to assess the quality of the system. However, during project 3.2 Better use of pedagogic materials by implementation, an assessment survey teachers in classrooms among teachers and students showed that the availability of materals enabled an 3.3 Better training courses in maintenance improved application of the program and and securty for technicians progress, and a greater interest of students 3.4 Improved provision, repair and in courses and application sessions. maintenance services by CAMEMD and its 3.3 Overseas training activities took place. regional antennas for all basic and secondary The trained technicians organized seminars schools. and workshops for the CAMEMD staff as 3.5 Better teachers education and improved well as for the teachers in topics using those equipments. pupils' leaming achievements. 3.4 CAMEMD services lack effectiveness. School facilities sometimes rely on the prvate sector for repair services. Output Indicators: Indicator/Matrix Projected in last PSR Actual/Latest Estimate Information source: Section "Current Status of the Project" in last PSR (5 March 2001). Component 1: Component 1: 1.1 Six surveys have been canied out: Component 1: 1.1 Nine surveys activities on a - Arabic Language grade 3 BE 1.1 Eight surveys have been carried out: representative sample: - Arabic Language grade 9 BE - Arabic Language grade 3 BE -4 basic subjects (Arabic, Sciences, French) - French Language grade 9 BE - Arabic Language grade 6 BE -4 school levels (grade 1, 2, and 3 in BE - Natural Sciences grade 1 SE - Arabic Language grade 9 BE and SE). - Mathematics grade 9 BE - Mathematics grade 3 BE - Mathematics grade 1 SE - Mathematics grade 6 BE 1.2 (i) Communicaton of the results to the - Mathematics grade 9 BE teachers and the inspectorate. - Mathematics grade 1 SE (ii) Diffusion of the final report to the - French Language grade 9 BE education community. (iii) Short publications on assessment tools to The survey on Sciences in grade 1 SE did teachers. not take place. 1.3 (i) Creation of data banks and A national seminar was organized in May standardized test items. 2000 for the pre experimentation phase. (ii) Pedagogic support to teachers on pedagogic assessment procedures. 1.2, 1.3 The system results and tests 1.4 Seven individuals have benefited from a diffusion did not take place. 1.4 (i) Overseas training of staff (14 training in Canada for 4 to 6 weeks. individuals) in designing, implementing, and 1.4 Fourteen individuals benefited from a 2 evaluating internal efficiency studies. phases training, in 1995 and 1996. (ii) Local training of staff (about 1560 inspectors). 1.5 Inidicators for 1999 and 2000 have been established. 1.5 (i) Setting up of a quality indicators 1.5 Education system quality and efficiency system aimed at providing useful, reliable, indicators, based on the OECD model were and viable information, on the schools set up for 1999 and 2000. system. (ii) Local training of staff (inspectors, teachers, COSP counselors, university psycho-pedagogy specialists). Component 2: Component 2: Component 2: 2.1 To establish a protocol for pedagogic research. - 16 -  2.2 Improve coherence between programs 2.2 In addition to Arabic Language and objectives and subjects content in Arabic and Physical Sciences, the programs concerning Physics for grade 1, 2, and 3 SE. the following subjects were evaluated: mathematics, English, French, biology, 2.3 Overseas training in programs and history, geography, and philosophy (in schoolbooks evaluation. 2.4 The results of the programs and process). 2.4 Dissemination of brochures on the schoolbooks evaluation have been produced 2.3 Two teams of eighteen individuals were conclusions of pedagogic research seminars in a memorandum. 6,000 copies were trained in Canada in 1995. aimed at inspectors and school managers in disseminated to the educaton community. the wiaae 2.4 A memorandum on programs evaluation the wilayate. (6,000 copies) was diffused. An important 2.5 Acquisition of basic documentation number of seminars was organized and (about 5.000 copies). brochures on the conclusions of these seminars were also published. 2.6 Subscription to foreign specialized (documentation available within the anchive magazines. system of the Bank.) 2.5, 2.6 The INRE library benefited from an asset of 5,000 books. Subscriptions to specialized magazines were partial due to cumbersome administrative procedures. Component 3: Component 3: Component 3: 3.1 Purchase of 13,000 compendiums for 3.1 The materials were purchased and 3.1 The expected materials were totally 10,500 built primary schools, plus 2,500 to distributed. An additional tranche of materials purchased and distributed. An additional be built; provision of scientific equipments for is in the process of being acquired. tranche of matenals for facilities having 750 basic schools under construction; suffered from acts of terrorism is in process. provision of scientific equipments for 250 This tranche will be partially executed due to high schools under construction. the closing of the loan. 3.2 (Indicators are not mentioned here as the sub-component related to the establishment of a laboratory products conditioning unit has 3.3 Following the cancellation of the contract bee cneld).. olwgtecnelao ftecnrc been cancelled). with the consulting firm for the initial study, 3.3 No progress on the study. 3.3 To conduct the study on maintenance the study was split into two studies on: (i) a needs. school materals purchase and maintenance strategy, and 3.4 Activity achieved, releasing a balance 3.4 Purchase spare parts. (ii) the development of a master plan for which was used to purchase the additional maintenance. No action was undertaken tranche of materials mentioned above. 3.5 Overseas training of 16 high technicians concerning the execution of (i) and (ii). and 10 CAMEMD executives. 3.5, 3.7 Overseas training took place for 27 executives (9 in information technologies and 3.6 Increase in personnel (about 30 management; 10 in machines/tools; 4 in individuals) in regional annexes. reeling/coiling; and 4 in electronics). As a result, seminars and workshops were 3.7 Skill upgrading of about 100 high organized for other CAMEMD staff and technicians and CAMEMD engineers as well teachers in subjects benefiting from the as about 1,000 laboratory managers and equipments. 29,000 teachers. 3.6 Infommation non available. 3.8 Acquisition of technical documentation. End of project 6/13/2000 - 17-  Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Project Cost By Component US$ million US$ million Pilot national assessment system 3.30 3.60 Pedagogic research 2.00 2.90 Improvement of supply and use of pedagogic material 51.00 50.60 Total Baseline Cost 56.30 57.10 Physical Contingencies 2.00 89.2 Price Contingencies 5.70 89.2 Total Project Costs 64.00 57.10 Total Financing Required 64.00 57.10 Including contingencies Project Costs by Procureme nt Arrangements (Appraisal Estimate) (US$ million equivalent) Procurement Method Expenditure Category ICB NCB Other2 N.B.F. Total Cost 1. Works 0.00 0.00 0.00 1.60 1.60 (0.00) (0.00) (0.00) (0.00) (0.00) 2. Goods 32.70 0.00 4.00 0.00 36.70 (32.50) (0.00) (4.00) (0.00) (x36.50) 3. Services 0.00 0.00 6.60 0.00 6.60 Technical coop. & Trng. (0.00) (0.00) (3.40) (0.00) (3.40) 4. Miscellaneous 0.00 0.00 19.10 0.00 19.10 Operating Costs (0.00) (0.00) (0.10) (0.00) (0.10) 5. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 6. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) Total 32.70 0.00 29.70 1.60 64.00 (32.50) (0.00) (7.50) (0.00) (40.00) Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) Procurement Method Expenditure Category ICB NCB Other2 N.B.F. Total Cost 1. Works 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 2. Goods 33.40 0.00 2.00 0.00 35.40 (31.30) (0.00) (1.20) (0.00) (32.50) -18 -  3. Services 0.00 0.00 2.60 0.00 2.60 Technical coop. & Trng. (0.00) (0.00) (2.60) (0.00) (2.60) 4. Miscellaneous 0.00 0.00 19.10 0.00 19.10 Operating Costs (0.00) (0.00) (0.00) (0.00) (0.00) 5. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 6. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) Total 33.40 0.00 23.70 0.00 57.10 1 (31.30) (0.00) (3.80) (0.00) (35.10) "Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 21Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) Percentage of Appraisal Component Appraisal Estimate Actual/Latest Estimate Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF. Pilot national assessment 1.40 2.30 1.70 1.90 121.4 82.6 system Pedagogic research 2.00 0.30 2.30 0.60 115.0 200.0 Improvement of supply and 36.60 21.40 31.10 19.50 85.0 91.1 use of pedagogic materials - 19-  Annex 3. Economic Costs and Benefits Not Applicable - 20 -  Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Perfornance Rating (e.g. 2 Economists, I FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 12/1991 4 1 ASS, I ED, I 00, I HR Appraisal/Negotiation 11/1992 3 1 ASS, 1 ED, I 00, Supervision 9/1994 2 1 ASS, I ED S S 4/1995 2 2 ED S S 10/1996 2 1 AC, S S 12/1998 1 1 HR S S 5/1999 1 I ED S S 11/1999 3 1 HR, 1 IMP, I MON U S 6/2000 1 1 MON S S 5/1993 2 1 ASS, I ED ICR 4/2001 2 1 00, 1 OA AC: Architect; ASS: Assessment Specialist, ED: Education Specialist; HR: Human Resources Spec.; MON: Monitoring Spec.; 00: Operations Officer Note: 5/1993 Rating for: Impl. Prog and DO = 1 (b) Staff: Stage of Project Cycle | Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation 20 158.5 Appraisal/Negotiation 10 60.0 Supervision 18 163.7 ICR 5 30.0 Total 53 412.2 - 21 -  Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating L Macro policies O H OSUOM O N * NA O Sector Policies OH OSUOM ON * NA E Physical O H *SUOM O N O NA E Financial O H OSUOM O N * NA El Institutional Development O H O SUO M O N 0 NA E Environmental O H OSUOM O N * NA Social ] Poverty Reduction O H O SU O M O N * NA O Gender O H OSUOM O N * NA El Other (Please specify) O H OSUOM O N * NA O Private sector development 0 H O SU O M 0 N * NA El Public sector management 0 H O SU O M 0 N * NA O Other (Please specify) 0 H O SU O M 0 N * NA - 22 -  Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bankperformance Rating E Lending OHS Os Ou OHU E Supervision OHS OS O U O HU O Overall OHS Os O U O HU 6.2 Borrower performance Rating OJ Preparation O HS * S O U O HU O Government implementation performance 0 HS 0 S 0 U 0 HU C0 Implementation agency performance O HS O S O u O HU O Overall OHS OS Ou O HU - 23 -  Annex 7. List of Supporting Documents Staff Appraisal Report (SAR), Basic and Secondary Education Improving Quality and Efficiency in Basic and Secondary Education (Report No. 8912-AL) Project Status Reports Synthese des Rapports d'Evaluation des Programmes Scolaires des Enseignements Fondamental et Secondaire (French and Arabic - Fevrier 1999) Rapport Final dEvaluation - Evaluation de Programme (Decembre 1998) Indicateurs du Systeme Educatif (Mars 2001) Evaluation de Rendement Scolaire (Mai 2000) Evaluation de Rendement Scolaire (Juin 1998) Evaluation de Programmes de l'Enseignement Fondamental et Secondaire (French and Arabic - Mars 1999) Manuel Sciences Physiques (1999) Rapport Final d'Evaluation - Evaluation de Programme - Physique Enseignement Secondaire (Decembre 1998) Recueil des Questionnaires d&Evaluation des Programmes et des Manuels Scolaires (1998- 1999) Evaluation des Manuels Scolaires du Secondaire *Physique lere AS* - Rapport Final (Janvier 2000) Rapport d'Evaluation du Manuel de Francais 8eme AF - 2eme langue etrangere (Decembre 1999) Evaluation des Manuels de Chimie de l'Enseignement Secondaire (Fevrier 2000) Evaluation des Manuels de Chimie de l'Enseignement Secondaire (Fevrier 1999) R6sultats de l'Evaluation Pedagogique des Examens de Fin de Cycle (2000) - 24 -  BORROWER'S REPORT  FROM Unite de Glestin PHONE NO. : +02 232898 Jul. 2B 200 10:42FM1 P2 Republicue Al