Evaluation

REGIONAL COMMITTEE Provisional Agenda item 7.4 Seventy-first Session SEA/RC71/7 New Delhi, India 3-7 September 2018 8 August 2018 Evaluation The purpose of this Agenda item is to update Member States on the various initiatives undertaken by the Organization and the South-East Asia Region to strengthen the culture of evaluation. The Region recognizes the importance of, – and is committed to advancing, the culture of evaluation as outlined in the WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development and the South-East Asia Region Evaluation Workplan for 2018–2019. In addition to providing an update on progress in evaluation at the global level, this document provides an overview of the evaluations conducted in the Region and ad-interim progress report in the implementation of the South-East Asia Region Evaluation Workplan for 2018–2019. Consistent with the Regional Framework on evaluation a full report should be presented to Regional Committee in 2019. The attached working paper was presented to the Eleventh Meeting of the Subcommittee on Policy and Programme Development and Management (SPPDM) for its review and recommendations. The recommendations made by the Subcommittee for consideration by the Seventy-first Session of the Regional Committee are: Action by Member States  Continue to participate in and support the evaluation exercises. Actions by WHO  Continue to strengthen the evaluation culture and use the results and recommendations for institutional learning and development.  Continue to monitor implementation of the SEA Regional Evaluation Workplan. This working paper and the SPPDM recommendations are submitted to the Seventy-first Session of the WHO Regional Committee for South-East Asia for its consideration and noting. SEA/RC71/7 Background 1. The Executive Board approved the WHO Evaluation Policy at its 131st Session in May 2012 [decision EB131 (1)] and asked the Secretariat to report annually to the Board on the progress of evaluation activities. After an independent review of the Evaluation Policy and the Framework for Strengthening Evaluation and Organizational Learning in 2017, it was further revised with inputs from the Member States during the 142nd Session of the Executive Board in January 2018 and the Independent Expert Oversight Advisory Committee in March 2018. 2. The WHO South-East Asia has made great progress since the adoption of the WHO Evaluation Policy. A Regional Framework for Evaluation was developed and, in 2016, two evaluations were carried out; namely, an evaluation of WHO’s contribution to maternal and child health in five countries (Bangladesh, Indonesia, Myanmar, Nepal and Sri Lanka), and an evaluation of the Regional Office contribution to the implementation of the national immunization programme in Bangladesh. Further, in 2017, as part of the global evaluation workplan, an evaluation of the Thailand Country Office was carried out. Furthermore, an evaluation of tobacco control through MPOWER measures in the Member States of the South- East Asia Region and an evaluation of collaboration between WHO and WHO collaborating centres in South-East Asia have been completed and report preparations are in the final stages. 3. The Annual Report presented to the 143rd Session of the Executive Board in May 2018: (i) provided information on the progress made in implementing the WHO Evaluation Policy, including the Organization-wide evaluation workplans for 2016–2017 and 2018–2019; and (ii) presented summaries of the following five recent evaluations for which management responses were available in order to document organizational learning linked to the findings and recommendations: (1) Country Office evaluation: Thailand (2) European Union/Luxembourg-WHO Universal Health Coverage Partnership (2011– 2016) (3) Pandemic Influenza Preparedness Framework (4) Pandemic Influenza Preparedness Partnership Contribution: high-level implementation plan 2013–2016 (5) International Coordinating Group on Vaccine Provision. 4. The Annual Report further noted the efforts made by the South-East Asia Region to mainstream and strengthen the culture of evaluation in the Region by introducing the Regional Framework for Strengthening Evaluation for Learning and Development along with the Regional Evaluation Workplan for 2018–2019, submitted to the Seventieth session of the Regional Committee for South-East Asia in September 2017. 5. The Independent Evaluation Office is actively engaged in both corporate evaluations and providing support to decentralized evaluations. Regarding evaluation capacity and resources, the engagement of regional and cluster focal points of the Global Network on Evaluation in corporate and decentralized evaluations has enabled greater coordination of evaluation activities at the three levels of the Organization. SEA/RC71/7 Page 2 Regional perspective 6. Consistent with WHO’s Global Policy and the Region’s commitment and high importance accorded to evaluation, the WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development (SEA/RC70/6-Inf. Doc. 1-Rev.1) and SEAR Evaluation Workplan for 2018–2019 (SEA/RC70/6-Inf. Doc. 2) were developed in 2017. The Framework and Workplan were presented to the Seventieth session of the Regional Committee in September 2017. These key documents are aimed at further expanding the culture of evaluation in the Regional Office and country offices. 7. Prior to the development of the SEA Region Evaluation Workplan 2018–2019, the following evaluation work was done or begun in the Region in 2017: a. Evaluation undertaken by the WHO Evaluation Office in Thailand in 2017 provided valuable insight for developing the Global Evaluation Workplan for 2018–2019. b. Two evaluations are completed and reporting writing are in the final stages: (i) Evaluation of tobacco control through MPOWER measures in SEA Region Member States, and (ii) Evaluation of collaboration between WHO and WHO collaborating centres of the South-East Asia Region. Final reports are expected by August– September 2018. 8. The SEA Region Evaluation Workplan 2018–2019 includes the ambitious scope of undertaking 16 evaluations including on various programmatic and country-specific topics across all technical categories of WHO’s General Programme of Work. As in mid July 2018, the status is as follows: (1) Three evaluations are underway and are expected to be completed by the end of 2018: (i) 10 years of SEARHEF implementation in the SEA Region; (ii) Relevance and impact of National Immunization Technical Advisory Groups in countries; and (iii) Evaluation of community clinics in Bangladesh; (2) Six are planned for 2018; these will be completed by the end of 2018 or early 2019; (3) Three are planned for 2019 and will be completed before the end of the biennium; and (4) Four are planned to start in the latter part of 2019 or early 2020. 9. Details on the status by evaluation topic is provided in Annexure A. Time constraints and securing funds for some programmatic and specific country evaluations is a common challenge. 10. As the implementation of the SEA Region Evaluation Workplan progresses mechanisms will be put in place to systematically collect and disseminate the results and recommendations to facilitate learning and development. Consistent with the stipulations of the Framework, the approach to Evaluation in the Region should be reviewed in 2019. The review will serve to identify lessons learned and recommendations to ensure that the evaluation work serves the objective set out in the Framework along with other Organizational initiatives: to nurture the culture of evaluation for improving performance, increasing accountability for results, and promoting Organizational learning and development. SEA/RC71/7 Page 3 Annex A: SEA Region Evaluation Workplan 2018–2019, status and progress (as on 16 July 2018) Budget Centre/ Lead Office Evaluation type Area of Evaluation Justification Category in Twelfth General Programme of Work Status Comments Provide relevant information according to status SEA regional/centralized evaluations Cross- departmental/Country Support & Reforms Programmatic Evaluation of 10 years SEARO’s support to AMR activities in SEA Region Significance/ utility Cross-category Planned for 2019– 2020 Plan to start in the latter part of 2019 Cross- departmental/Director, Programme Management Office Thematic Analysis of the technical quality of the SEARO publications for the past 10 years Significance/ utility Cross-category Planned for 2019– 2020 Plan to start in the latter part of 2019 Health Systems Development Thematic Extent to which WHO activities in the SEA Region have adapted to reflect the more inclusive and integrated approach to development Significance Cross-category Planned for 2019– 2020 This evaluation is planned to start in the latter part of 2019 and should be completed in 2020. This will allow for consideration of ongoing work on SDGs and developments related to implementation of WHO GPW13. WHO Health Emergency Programme Thematic Evaluation of 10 years of SEARHEF implementation in SEA Region Significance/ utility Emergency (Category 5/12) In progress The evaluation has been commissioned. Interim update with preliminary findings and recommendations is expected by September 2018 and final report by October 2018. SEA/RC71/7 Page 4 Budget Centre/ Lead Office Evaluation type Area of Evaluation Justification Category in Twelfth General Programme of Work Status Comments Provide relevant information according to status Nepal Country- specific Evaluation of WHO contribution to preparedness, response readiness and recovery of the Health system from multi- hazard emergencies Significance/ utility Emergency (Category 5/12) and Category 4 Planned for 2019 Draft terms of reference have been prepared. Work is expected to begin in January and end by June 2019. Sri Lanka Country- specific Use and relevance of WHO information products and processes including knowledge management for NCDs and Health systems Significance/ utility Category 2 and Category 4 Planned for 2019 Draft conceptual framework developed and work is expected to start in 2019 once funds are secured. Given the work underway with key information products and processes (i.e. SDGs and UHC), the evaluation is being considered for the latter part of the biennium to incorporate such products. Timor-Leste Country- specific Evaluation of the contribution of the WHO Country Office in Timor-Leste for the implementation of the national immunization programme in Timor-Leste Significance/ utility Category 1 Planned for 2018 Work is scheduled to be conducted in November/December 2018. SEA/RC71/7 Page 5 Budget Centre/ Lead Office Evaluation type Area of Evaluation Justification Category in Twelfth General Programme of Work Status Comments Provide relevant information according to status Family Health, Gender and Life Course* Programmatic The use and perceived impact of WHO guidelines on RMNCAH at country level Significance/ utility Category 3 Planned for 2019 Concept note is ready and contractual partners are being identified. Funds are not allocated due to resource constraints. If funds are made available it can be undertaken in 2019. Thematic The relevance and impact of National Immunization Technical Advisory Groups in countries Significance Category 1 In progress Potential contractual partner identified. Report will be ready in 2019. Thematic An in-depth situation analysis of unmet needs and gaps for adolescent health in countries Significance Category 3 Postponed beyond 2019 Not to be done in this biennium. Decentralized evaluations Bangladesh Country- specific Evaluation of community clinics in Bangladesh Significance/ utility Category 4 In progress Work is ongoing and the final report should be ready by December 2018. Data has been collected and key informant interviews and focus group discussions have been conducted. Interim report has been shared with advisory committee. SEA/RC71/7 Page 6 Budget Centre/ Lead Office Evaluation type Area of Evaluation Justification Category in Twelfth General Programme of Work Status Comments Provide relevant information according to status India Country- specific Programmatic Evaluation of WCO’s contribution to advancing the agenda of viral hepatitis B and C prevention and treatment in India Significance/ utility Category 1 Planned for 2018 Concept note developed and request for proposal is being prepared. Funds are secured. Country- specific Programmatic Evaluation of WCO’s contribution to kala-azar elimination programme in India Significance/ utility Category 1 Planned for 2018 Concept note developed. Funds are being explored. Countryspecific thematic WHO’s role in the development of National Multisectoral Action Plan for prevention and control of NCDs in India. Significance/ utility Category 2 Planned for 2018 Draft concept note developed. Funds to be explored. Nepal Country- specific Programmatic Evaluation of WHO contribution to enhance/strengthen the Health Information System in Nepal Utility Category 4 Planned for 2018 Terms of reference have been developed and arrangements made to hire experts. Work is expected to be completed by November 2018. Sri Lanka Country- specific Programmatic Role of WHO Sri Lanka during the last 10 years in addressing triple burden of malnutrition (under nutrition, overweight and obesity and micronutrient deficiencies) Significance/ utility Category 2 Planned for 2018 Scope of work has been prepared. Preparations for consultations with national stakeholders and identification of appropriate experts are underway. Work is scheduled to begin in the latter part of 2018. Funding is being explored.