Colombia - Community Child and Nutrition Project

 ICRR 10078 Report Number : ICRR10078 ICR Review Operations Evaluation Department 1. Project Data : OEDID : OEDID: L3201 Project ID : P006869 Project Name : Community Child and Nutrition Project Country : Colombia Sector : Other Population Health & Nutrition L/C Number : L3201 Partners involved : None Prepared by : Susan A. Stout, OEDST Reviewed by : Christopher J. N. Gibbs Group Manager : Roger H. Slade Date Posted : 06/08/1998 2. Project Objectives, Financing, Costs and Components : The project was designed to support the national plan for poverty reduction through supporting the expansion and improved quality of a national effort to improve the quality of child care for children under age seven, managed by the Instituto Colombiano de Beinestar Familiar (ICBF). The project's goal was to support expansion of a Child Care Homes Program (Hogares de Bienstar Infantil, HBI) and reduce expenditures on a less cost effective institution-based program of child care (CAIP). It had three objectives: i) introduce and develop policies aimed at improving the cost effectiveness of ICBF operations; ii ) support the institutional development of ICBF, particularly its technical support, planning and monitoring capabilities; and iii ) evaluate, improve and maintain at an appropriate level of quality in HBI services during the expansion . The project included three components : i) policy development, which supported studies and technical assistance to enhance the effectiveness of ICBF resource use; ii ) institutional development, to strengthen ICBF management; and iii ) service support, to include a major investment in training and technical supervision and a credit program for improving homes included in the HBI program . Total project costs were estimated to be US$40.2 million at appraisal. Actual project costs were $34.8 million due to a slower than expected rate of expansion of the HBI program . The project funded most of the policy and institutional development activities undertaken during the expansion of HBI . Support for services (training and the credit program for home improvement) made up about 75% of the project, but only 5% of the support costs of the overall program . 3. Achievement of Relevant Objectives : The project achieved its objective of influencing expansion of coverage of the HBI program, with high incidence among the target population. Of an enrollment goal of one million children by the end of 1994, enrollment reached 799,000 in 1994 and about 827,000 in 1996. 91% of the beneficiary families belong to lower -income strata or live in rural areas, 87% of the families' earned one minimum salary or less . Results from the impact evaluation survey show disappointing results for program impact on children's welfare in terms of nutrition and health status and psycho-social development. Multivariate analysis indicated that the major factors influencing nutritional status, psycho-social development and morbidity of participants were : a) physical condition of the community home, b ) community mother's (MC) educational level, c) quality of growth monitoring, d) food preparation, and e) monitoring activities. Training of MCs had no effect on children's health and nutritional status . Project results suggest that the relative impact and cost-effectiveness of the HBIs and CAIPs need to be re -evaluated, since CAIPs were found to be of better quality and the cost advantage of HBIs decreased over time, as indirect costs and the need for additional inputs were taken into account . At inception the unit costs of CAIP were 2.6 times those of HBI -- this ratio was estimated at 1.2. in 1995. 4. Significant Achievements : The project achieved a large share of its policy and institutional development objectives . The ICBF reduced its contributions to the CAIP program by 30% by transferring costs to parents, municipalities and private companies . The management development component supported two restructuring studies which helped tailor ICBF to new constitutional requirements and encouraged better links between this government agency, civil society and municipalities (though improvements are still needed ) and initiated decentralization to regional centers . Despite significant delays, an improved information system for monitoring ICBF programs was developed, and after a four-year delay, a First Impact Evaluation Survey was designed and conducted . The latter suffered from several design flaws, but provides a sampling base for future continuous monitoring of the nationwide impact of the HBI program. The micronutrient content of Bienesfarina (the food supplement used in the program ) was improved, production plants modernized and marketing and distribution made more efficient . The service support component succeeded in the provision of training to participating MCs, though the results of the evaluation survey suggest that training quality was inadequate, and supervision and monitoring systems for MCs remains weak . Targets for the provision of credits for MC home improvements were exceeded, though about 28% were used for activities outside project guidelines, suggesting a need for stricter control of future operations . 5. Significant Shortcomings : Key shortcomings include significant delays in the establishment of improved monitoring and impact evaluation survey systems, identified as key constraints to effectiveness during project design . Moreover, despite recognition of the important role of monitoring and impact evaluation in determining program effectiveness, the design lacked specific indicators for monitoring project performance, which made supervision of performance difficult . Training quality also fell short of expectations, and the continuing weakness in systems for supervising and monitoring improved program quality undermine HBI program effectiveness . The ICBF, a semi-autonomous but centralized government agency had difficulty establishing effective relations with other agencies whose support was important for overall effectiveness, including the Ministries of Health (for childcare) and Education (for pre-school education), housing and cooperative agencies, the Ministry of Finance, municipalities and civil society . Weak inter-agency relations led to delays and less extensive expansion than originally planned . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Partial Modest Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory 7. Lessons of Broad Applicability : Projects which rely on a several agencies to support objectives should establish agreements with concerned authorities and institutions during preparation, and provision should be made for periodic review and revision throughout implementation. Centralized procedures are poorly suited to highly decentralized and participatory service delivery models such as employed in the HBI program . Involvement of NGOs might have led to greater impact. Project design should reflect and incorporate baseline data and plans for evaluation follow up . New information systems may be difficult to design and implement and assumptions about the length of time and technical difficulties associated with impact evaluations efforts need to be realistic . 8. Audit Recommended? Yes No Why? Despite some shortfalls, the project succeeded in establishing an 'evaluation culture' in a nutrition and early childhood intervention program , an area of increasing relevance to Bank human development and poverty reduction strategies. An audit would enable more detailed examination of the processes that led to this accomplishment, as well as an opportunity to elaborate the question of the 'scalability' of participatory service delivery models aimed at poverty groups . 9. Comments on Quality of ICR : The ICR is thorough and clearly written, and draws lessons of considerable relevance to current issues in human development lending strategy .