Mozambique - Agricultural Sector Public Expenditure Program

 ICRR 12695 Report Number : ICRR12695 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/17/2007 PROJ ID : P001799 Appraisal Actual Project Name : Agricultural Sector US$M ): Project Costs (US$M): 216.5 160.9 Public Expenditure Program Country : Mozambique Loan/ US$M): Loan /Credit (US$M): 30 30.09 Sector Board : RDV Cofinancing (US$M ): US$M): 164.5 NA Sector (s): Agricultural extension and research (45%) Central government administration (45%) Irrigation and drainage (5%) Forestry (5%) Theme (s): Public expenditure financial management and procurement (23% - P) Other environment and natural resources management (22% - P) Other rural development (22% - P) Rural markets (22% - P) Administrative and civil service reform (11% - S) L/C Number : C3171 Board Approval Date : 02/18/1999 Partners involved : UNDP, AfDB, AusAID, Closing Date : 06/30/2004 12/31/2006 DFID, EU, FAO, USAID, Canada, Finland, Italy, Sweden, Ireland, IFAD, Government of Netherlands, Danida, Austria Evaluator : Panel Reviewer : Group Manager : Group : Nalini B. Kumar Ridley Nelson Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The objectives of this first five year phase of the Agriculture Sector Public Expenditure Program (PROAGRI) were to (i) improve institutional arrangements for financing and delivery of core public agricultural services; and (ii) strengthen the Borrower's capacity within MAP [Ministry of Agriculture and Fisheries ] at provincial and local levels, to provide core public sector functions for the agricultural sector, in a cost -effective manner. These objectives are used as the benchmark for evaluation in this ICR Review . The overall purpose of PROAGRI which adopted a sector -wide approach (SWAP) to agriculture development was to improve the impact of public expenditure in securing an enabling environment for sustainable and equitable growth in the rural sector such that poverty is reduced and food security improved while the physical and social environment is protected . The program design involved providing non -earmarked funding, through the government budget system, to finance the entire expenditure program of the MAP over a five year period . IDA financing was to be provided through an APL predicated upon Government compliance with agreed Basic Principles and milestones . The Basic Principles provided the framework to review implementation progress by government and the donors and empowered MAP to coordinate the donors. The Eight Basic Principles were: poverty reduction; decentralization and empowerment; good governance; gender issues; smallholders access to land, inputs and markets; social and environmental sustainability; market orientation; and Ministry activities limited to core functions and strengthened capacity to carry them out . The project objectives were not revised . However, since PROAGRI funded the totality of MAP's activities the scope of the program was altered by two changes in the structure and scope of the Ministry during the project's lifetime . In January 2000, MAP was expanded into the Ministry of Agriculture and Rural Development (MADER). In 2005, it was reorganized around its core function as the Ministry of Agriculture (MINAG). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): PROAGRI comprised eight thematic components congruent with MAP's functional divisions . (i) Institutional Development (appraisal estimate US$59 million, actual US$68.10 million). To improve public administration of the agricultural sector, transforming MAP into a modern institution focused upon policy formulation and implementation, regulation, and the creation of a sectoral environment that nurtures private sector development . (ii) Agricultural Research (appraisal estimate US$25.4 million, actual US$17.30 million). To support upgrading of the quality and effectiveness of agricultural research by establishing mechanisms for competitive funding of priority agricultural research projects and institutional reform of research institutes, development of mechanisms to focus research on the generation of practical crops and livestock technologies for smallholder farming systems, steps to privatize and contract out some types of research and to make research more demand -driven. (iii) Agricultural Extension (appraisal estimate US$16.70 million, actual US$7.80 million) To support the National Directorate of Extension in the establishment of mechanisms for empowering farmers' associations and local authorities to work together with local offices of the MAP in contracting out the provision of extension services, upgrading of agricultural extension management, and establishment of pilot extension programs to test alternative institutional arrangements and extension methodologies . (iv) Support for Crop Production (appraisal estimate US$17.70 million, actual US$46.70 million) To support the National Directorate of Agriculture in the establishment and enforcement of legal and regulatory framework for plant protection, seed and input supply, migratory /pandemic pest control, and crop forecasting . (v) Livestock Support Services (appraisal estimate US$19.10 million, actual US$5.10 million) To support strengthening of the National Directorate of Livestock to deliver services in animal production and health, improving regulatory capability and field operations and transportation . (vi) Forestry and Wildlife Management (appraisal estimate US$20.60 million, actual US$6.00 million) To support strengthening of the Directorate of Forestry and Wildlife at central and provincial levels in planning and monitoring and development of a regulatory framework for the management of forest and wildlife resources, development of extension support to the private sector and communities involved in forestry and wildlife management,and surveys and reviews of the status of existing parks and forest conservation areas and the launching of a program of rehabilitation and management of conservation areas, classification of forest areas, gazettement and field demarcation, identification and award of forest and hunting areas concessions . (vii) Irrigation (appraisal estimateUS$14.60 million, actual US$3.00 million) To support the National Directorate of Irrigation in the preparation of a policy and strategy for irrigation development focussed on the development of small-scale irrigation schemes to be managed by water users' associations and the development of a restructuring proposal for the management system of existing large schemes . (viii) Land Management (appraisal estimate US$26.80 million, actual US$6.90 million) To support development of land management activities being carried out by the Directorate of Land and Cadastre including social surveys, cadastre activities, land demarcation and adjudication concentrating in areas affected by the granting of concessions and strengthening MAP's capacity in the areas of land policy formulation and reforms . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Disbursement of IDA funds lagged during the first half of the project implementation period because of preferential use by the Government of grant funds from other donors over the IDA credit and slow disbursement of IDA funds until the Bank joined the Common Flow of Funds Mechanism (see section 4 below). There were three extensions of the IDA credit with the project finally closing two years later than scheduled on December 31, 2006. The initial extensions were made to allow implementation of some key activities which would help project objectives. The last extension was made to facilitate, among others, implementation of activities in support of the Avian Flu Surveillance and Preparedness Plan . At project closure, after three extensions about US$ 4 million of IDA credit remained undisbursed . During the remaining four months the Government was able to disburse only part of these funds due to delay in submission of audit reports leaving an outstanding balance of US$ 2.7 million IDA credit by April 30, 2007 which had to be canceled. There was a significant reallocation of resources within components over the lifetime of the project . The main increase occurred in the crop production component and there was some additional spending on the Institutional Development component. Amounts spent on research, extension, livestock, land management, forestry and wildlife and irrigation were significantly lower than anticipated at appraisal . It is not possible to tell from the ICR how much the Bank contributed in dollar amounts to the implementation of each of the eight components . The appraisal report (page 30), however, shows that as planned, 80 percent of IDA resources were to be distributed among three components: institutional development, research and extension . 3. Relevance of Objectives & Design: With respect to the relevance of objectives , PROAGRI I was the first phase of a long term sector program in Mozambique and was relevant to conditions in the country where development of the agriculture sector was critical to poverty alleviation. The project was aimed at supporting the Government's agricultural development goals by rationalizing sectoral public expenditure and the role of the Ministry and improving its effectiveness in policy formulation, regulation and service provision . However, the objectives were ambitious given the short five year time frame and the limited institutional capacity in the country . With respect to the relevance of design, PROAGRI was selected as a pilot for enhanced harmonization of donor procedures within the framework of the Special Program for Africa . Past fragmented project based donor assistance with limited in-country institutional capacity had been a major handicap to agriculture sector development in the country. However, its design was complex as it involved actions across a wide spectrum of activities as seen in section 2 which required significant capacity in the implementing agencies . A design weakness was also lack of an adequate system to assess progress . As noted in the ICR (page 16) it was not easy to measure progress on achievement of the objectives and a clearer articulation of expected outcomes would have improved results orientation in implementation and supervision and the overall accountability of the program . In addition, an institutionalized way of keeping costs down and ensuring quality with respect to Ministry of Agriculture activities was not built in. According to the ICR (page 27) this was important in a process where ownership /responsibility was with the Ministry for deciding annual plan activities and there was no other institution or system to which the Ministry was accountable for both quantity and quality of its output . 4. Achievement of Objectives (Efficacy): The weakness in project monitoring and evaluation makes assessment of the two objectives very difficult . Objective (i): Improve institutional arrangements for financing and delivery of core public agricultural services : As regards the first objective, institutional arrangements for financing of core public agricultural services seem to have substantially improved from a low base . The additional information provided by the Region shows that the total budget to MINAG has been increasing . There are concerns, however, about how far the ministry will be able to ensure delivery of core public agricultural services as the evidence is very mixed . The ICR notes that the project was able to transform the Ministry of Agriculture from a weak, marginalized ministry into one of the foremost ministries regarding restructuring and reform issues . The ICR notes several changes : that the share of the provinces in the Ministry of Agriculture budget increased from 40 to 60 percent, the planning and budgeting process improved, a modern financial management system was instituted, physical infrastructure, equipment and working conditions improved, and agriculture research has been restructured . However, the ICR also notes (page 23) that "delay, uncertainty and partial measures have undermined efficacy of reforms even in areas showing success ." The additional information provided by the Region also provides a mixed picture as seen below .  An evaluation shared by the Region notes on the one hand that " ..... PROAGRI I successfully replaced fragmented donor-driven projects with a comprehensive agricultural development programme that allowed donors to focus their assistance behind a government vision . This helped MINAG to realign its functions, organization and approaches with the requirements of a market economy and has put MINAG at the forefront of Mozambican ministries regarding restructuring issues and provided a promising perspective for progress ." (page vii)(Final Evaluation of the First Phase March 2007) But on the other hand it notes that "However, the combination of changes in organizational responsibility (MADER to MINAG) and sense of ownership of the PROAGRI process at the senior level of the Ministry has made implementation of envisioned structural reforms very difficult. Thus the Ministry remains in a limbo of partial transformation . The implications for PROAGRI II is that it may display the same pattern of slow "start and halting" progress as PROAGRI I" (page vii). The same report also notes that " As a consequence of functional analysis in 2001, it was proposed that MINAG should be restructured ...........However, this has not happened during the course of PROAGRI I due to lack of consensus within MINAG/government to make effective changes . Proposals from senior managers for restructuring, in general, have been superficial and are unlikely to bring about any significant changes . Therefore, the implementation of this decision can be judged as not effective during the course of PROAGRI I " (page 64 of the above evaluation). On the one hand the evaluation notes that "The annual activity planning and budgeting tool that has been introduced by PROAGRI, PAAO, was considered to be reasonably successful by central planners and the financial managers because of its ability to provide information in the form specified by the PROAGRI supervisory requirements." On the other hand the report notes that "field investigation has revealed the following serious practical conflicts between central management needs and activity implementation needs : complaints have been made that the PAAO is not flexible enough to allow insertion of local programming concerns, thus counteracting the attempts to include local communities; and field managers and technical staff felt unable to assess their activities because there remains no way to link funds to activities and results of those activities " (page xii) On the one hand the evaluation notes that "PROAGRI I increased the capacity of DPAs [Provincial Directorate of Agriculture] and DDAs [District Director of Agriculture] in terms of local planning and execution to meet its mandate of institutional development ." On the other it notes that "However, this decentralization was perceived to be only partial given that all procedures were determined at central level including the financial ceilings . The definition of budgets at central level failed to consider sectoral and local priorities . The distribution of budgets at provincial level was also too concentrated in hands of the Provincial Director " (page 64 above evaluation). The appraisal report had visualized that the reform process would result in a down -sized Ministry focused on a reduced set of activities. The overall picture here also is mixed . While the Region provides additional information that the overall ministry staff are reduced the ICR notes there is no information on skill mix changes which was a process related indicator visualized at appraisal . ii): Strengthen the Borrower's capacity within MAP [Ministry of Agriculture and Fisheries ] at provincial Objective (ii): and local levels, to provide core public sector functions for the agricultural sector, in a cost -effective manner . As regards the second objective it is difficult to say how far the project was able to strengthen the Borrower's capacity within MAP at the provincial and local levels to provide core public sector functions for the agricultural sector in a cost effective manner. While extension staff numbers increased, the evaluation report cited above found that a number of senior government officials were of the view that PROAGRI "did not make any considerable impact in terms of assisting farmers to increase production and productivity (page 74)." Neither is there information on the impact indicators noted at appraisal namely the extent to which competition and volume are increased in agricultural input and output markets, increases in the contribution of marketed agricultural produce to household income for small-holder subsistence level families, the number of wildlife present in national parks and reserves, the area of forest under sustainable commercial and community management, among others . It is also not clear whether these indicators were the right ones to assess achievements in an intervention which was to last only five years and focussed primarily on institutional reforms . The ICR, however, states that PROAGRI interventions are likely to have had a poverty alleviating impact through support for cashew production and improvements in basic food crop production and agricultural growth though it is difficult to say how much and what can be attributed to either the overall program or the Bank. The evaluation shared by the Region reports a very mixed picture on results as seen below : On the one hand it notes that "Research has done better on service provision, i .e.basic research," on the other hand it notes that "but less well on establishment of a research system with different centres of activity . .....The major concern is the lack of improved linkages between research and extension " (page xiii). On livestock development the report notes that "The provincial surveys indicated that after the management implementation of the decision to decentralise dip tanks to farmer associations and the private sector, the provision of animal health services has deteriorated . Although DPA staff were able to visit more locations using PROAGRI I money, this technical assistance was not translated into better performance as farmers had no resources to implement the recommendations provided . ......The team assessment is that, government failed to develop an appropriate exit strategy to move from providing direct assistance to farmers as a result : Farmers Associations were not prepared to take over these tasks, nor was the private sector in a position to do so " (page  66) On the one hand it notes that "The area and production of maize has increased during the period under analysis but the yield per unit area has remained steady as indicated " on the other it notes that "It is, however, difficult to establish a direct link of these results to the investments made on institutional development (page 64). ...." On the one hand it notes that "Land tenure security is improved for all those who have registered titles ." On the other it notes that "However, achievements of land management seems modest " (page 70) On the one hand it notes that "Partnership for forestry and wildlife management between local communities and the private sector has been one of the major achievements ." (page 80) On the other it notes that "Despite the achievements in this component, it faces an array of poor practices which will prove unsustainable and hurt the sector" (page 82) 5. Efficiency (not applicable to DPLs): Efficiency is very difficult to assess in this intervention due to limited attributable qualitative or quantitative evidence. No analysis is available for cost -effectiveness of resources spent on the Ministry of Agriculture either at appraisal or completion. At the appraisal stage the returns to investments were estimated at the component and sub-component levels and also at the overall agriculture sector level for the project as a whole . At the program level it was estimated that if PROAGRI induced an incremental annual growth rate of not less than 0.76 percent in agricultural GDP an ERR of 16 percent could be realized for the PROAGRI investments .At the completion stage it was estimated that an incremental annual growth rate of not less than 0.34 percent per annum was required for a 12 percent ERR and 0.38 percent for a 16 percent ERR. On the basis of an analysis of growth rates of crops the ICR concludes that "it is not implausible to suppose that PROAGRI I investments could have generated sufficient returns over capital." However, the ICR also shows that actual yields were significantly lower than those forecast at appraisal. The rate of return for overall crop production was assumed to have improved because of cashew activity which was not considered at appraisal . It is not clear how much reliance can be based on the analysis in the ICR which also notes that it is problematic to use the incremental growth rate to assess project efficiency as it is not clear how the incremental impact of institutional improvements in the Ministry on overall performance of the agricultural economy can be calculated and also how the incremental impact of services like extension can be estimated . The ICR also acknowledges that "project cost data was not available at a disagrregated level by project components and activities making it difficult to link the observed impacts to the project activities ." For these reasons the ERR percentages are not reported below . It is worth acknowledging, however, that in interventions like PROAGRI I given the short implementation period it is difficult to demonstrate results at the field level . However, qualitative evidence could have been provided to demonstrate benefits from the project commensurate with resources utilized . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Overall, the weak M&E offers very little evidence on project outcome and impact . With respect to the first objective, while harmonization of donors and the financial and budgetary process reforms were significant achievements, there were significant shortcomings, in the extent to which the more fundamental ministry structural transformation was really achieved, although some allowance may be given for the very low post -conflict starting point. With respect to the second objective, while there was some positive institutional reform on research, small gains in land processes, an increase in numbers of extension staff, and a few other potential gains, very limited evidence of attributable impact on farmers was found. This is notwithstanding the extended, nearly eight year, project implementation period and the $160.9 million total project cost. It is concluded therefore that overall there were considerable shortcomings on efficacy. Together with questions on relevance (section 3 of the ICR Review), and the lack of adequate information to make an assessment of efficiency (section 5 ICR Review), this assessment rates outcome as moderately unsatisfactory. a. Outcome Rating : Moderately Unsatisfactory  7. Rationale for Risk to Development Outcome Rating: Although the ICR notes that the World Bank's decision to withdraw from PROAGRI after the first phase created uncertainty about long term commitment to the sector program by partners, the Region has provided additional information to show that sufficient support is now available from donors to support PROAGRI II . The Region also notes that there is an increasing emphasis of the PRSC process on the provision of agricultural services at the provincial and district level. In addition, the Region also notes that the MINAG budget has been increasing over the years and that more resources are channelled to the local level although the evaluation quoted above notes that extension budget has fallen since 1999. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: While an important process related achievement was on donor harmonization there are several concerns about Bank performance both at Quality at Entry and Supervision . Quality at Entry : The project objectives were ambitious given the time period for project implementation and the weak institutional capacity. The project design was complex as it involved actions across a wide spectrum of activities as seen in section 2 which required significant capacity in the implementing agencies .In addition, an institutionalized way of keeping costs down and ensuring quality with respect to Ministry of Agriculture activities was not built in. Another design weakness was lack of an adequate system to assess progress despite the unusually long (45 months) and costly project preparation phase . As noted in the ICR (page 16) it was not easy to measure progress on achievement of the objectives and a clearer articulation of expected outcomes would have improved results orientation in implementation and supervision and the overall accountability of the program . Quality of Supervision :Throughout the implementation phase there was no effective M&E system which made it difficult to gauge implementation success . Supervision was preoccupied with ensuring fiduciary compliance and relatively less attention was given to achievement of development outcomes of various interventions . Supervision also suffered from inadequate/inappropriate skill mix given the range of technical and institutional issues faced . Moreover the supervision ratings also did not give Bank management a realistic assessment of problems in the project. The ICR notes (page 27), "...in Bank reports, while shortcomings were properly identified, the ratings were at times more positive than supported by the technical analysis ." The Bank withdrawal of direct funding for PROAGRI II was resented at the time by some donor partners who saw it as breaking away from a commitment. However, the Bank move to provide future support through direct budget support at the economy-wide level and the trend of donors in the same direction may have allayed these concerns. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: The ICR notes that government commitment to PROAGRI was strong and that despite changes in leadership at the top the Government maintained overall commitment to the reform process . The Government also increased budget support for the Ministry of Agriculture . Despite capacity limitations, the Government was also able to undertake reform efforts in several areas . During implementation the Government maintained a good dialogue and coordination with donors and key stakeholders . However, this review has concerns about Government commitment since the scope of the program was altered by the two changes in the structure and scope of the Ministry (see section 2 above). Other areas where Government performance was weak include lack of timely support to the Ministry of Agriculture's human resource reform process, especially in areas related to general civil service issues of pay and work conditions, insufficient oversight in key areas relating especially to cost management and control and lack of timely availability of funds to the implementing agency, especially in the initial phase. Weaknesses in implementation relate to delay and weakness in compliance in several areas like implementation of functional analysis and human resource management reform, persistent M&E problems, weakness in linking investments in Ministry of Agriculture capacity and skills to ensure more effective delivery of support services . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: M&E Design According to the appraisal document the impact of the first phase of PROAGRI was to be assessed against process-related indicators related to progress toward bringing about institutional change, such as change in MAP staffing (number of employee mix, skill mix, capacity, level of deconcentration ), and the success with which some services formerly carried out by MAP have been privatized or contracted out . PROAGRI's achievements were to be also assessed against impact indicators such as the extent to which competition and volume increased in agricultural input and output markets as evidenced by the number of traders participating in a given market, the narrowing of the gap between prices received by farmers and those prevailing in local markets and at the border, increases in agricultural production and productivity, among others . However, according to the ICR (page 16) the indicators and milestones listed in the appraisal document varied in their suitability being at times too "low" (process/output level) or too "high" (impact level) on the causal chain often leaving out the outcomes level, with the chain itself not always being clear . Further the expected level of indicators were not necessarily specified in measurable terms and baseline or benchmark information was generally missing . M&E Implementation Throughout the implementation phase there was no effective M&E system . It is not clear why beneficiary surveys were not utilized . An attempt was made towards the end of the project (in 2003-04) to establish a suitable M&E system. The M&E unit was finally established by end -2005 after 5 years. However, it continued to suffer from problems such as lack of good staff . M&E Utilization In the absence of an appropriate M&E framework no special arrangement appears to have been made to collect relevant "impact", process or even cost data at regular intervals . As a result relevant information was not available to inform decision making and evaluate relevant options . As acknowledged by the ICR (page 17), "M&E was a disappointing aspect of this program's design and implementation ." a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Fiduciary compliance was given a large amount of attention in project supervision (see section 8 above). Substantial weaknesses were noted in quality and compliance with financial management guidelines in the 2001 and 2002 Audit reports. Following supervision inputs from the Bank and donors, there was improvement in quality and compliance with guidelines. The development of common procurement procedures was also accorded high priority and completed in May 2001. The 2003 Procurement Review showed that the Ministry's procurement system was working satisfactorily; however, the 2005 Review showed some decline in the quality of procurement procedures, especially in the provinces . The high staff turnover and institutional changes arising from the decentralization process contributed to the weakness in financial management and to the decline in quality of procurement procedures . The project is reported to be in compliance with all applicable safeguard policies by the ICR (page 18). 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Overall, the weak M&E offers very little Unsatisfactory evidence on outcome and impact . With respect to the first objective, while harmonization of donors and the financial and budgetary process reforms were significant achievements, there were significant shortcomings, in the extent to which the more fundamental ministry structural transformation was really achieved, although some allowance may be given for the very low post-conflict starting point. With respect to the second objective, while there was some  positive institutional reform on research, small gains in land processes, an increase in numbers of extension staff, and a few other potential gains, very limited evidence of attributable impact on farmers was found. This is notwithstanding the extended, nearly eight year, project implementation period and the $160.9 million total project cost. It is concluded, therefore, that overall there were considerable shortcomings in efficacy. Together with questions on relevance (section 3 of the ICR Review), and the lack of adequate information to make an assessment of efficiency (section 5 ICR Review), this assessment rates outcome as moderately unsatisfactory. Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately There are several concerns about Bank Satisfactory Unsatisfactory performance both at Quality at Entry and Supervision. Quality at Entry: The project objectives were ambitious given the time period for project implementation and the weak institutional capacity. The project design was complex as it involved actions across a wide spectrum of activities as seen in section 2 which required significant capacity in the implementing agencies.In addition, an institutionalized way of keeping costs down and ensuring quality with respect to Ministry of Agriculture activities was not built in. Another design weakness was lack of an adequate system to assess progress despite the unusually long (45 months) and costly project preparation phase. As noted in the ICR (page 16) it was not easy to measure progress on achievement of the objectives and a clearer articulation of expected outcomes would have improved results orientation in implementation and supervision and the overall accountability of the program. Quality of Supervision :Throughout the implementation phase there was no effective M&E system which made it difficult to gauge implementation success. Supervision was preoccupied with ensuring fiduciary compliance and relatively less attention was given to achievement of development outcomes of various interventions. Supervision also suffered from inadequate/inappropriate skill mix given the range of technical and institutional issues faced. Moreover the supervision ratings also did not give  Bank management a realistic assessment of problems in the project . The ICR notes (page 27), "...in Bank reports, while shortcomings were properly identified, the ratings were at times more positive than supported by the technical analysis." Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Based on the ICR the following lessons are emphasized : Project experience highlights the importance of having an effective and well functioning M&E system right from the beginning, results from which provide continuous feedback for implementation and assessment of project achievements. Sustained donor coordination around commonly agreed programs has a significant reinforcing effect on government ownership of and commitment to the reform . Capacity building efforts also need to give adequate attention to staff retention strategies (including performance and work place conditions ). However, civil service issues of pay and work conditions are beyond the scope of one sectoral ministry and need to be tackled as a part of the wider policy dialogue . Project experience highlights the importance of significant Bank Management oversight to ensure that the Bank either not support a complicated process of reform that requires long term commitment in a country or that the main elements of the reform process are picked up adequately in another program . 14. Assessment Recommended? Yes No Why? There will be lessons for other sectors and countries where the Bank has supported sector -wide approaches. 15. Comments on Quality of ICR: On balance, the ICR is rated satisfactory but only marginally so . While the assessment of the design and project experience is candid, there is little evidence on the achievement of project objectives (partly a reflection of the project's poor M&E). It is not clear why the ICR does not quote the March 2007 evaluation of the first phase which was subsequently shared by the Region with IEG . a.Quality of ICR Rating : Satisfactory