Management and Governance matters - Evaluation: Annual report

REGIONAL COMMITTEE Provisional Agenda item 11.2 Seventy-fourth Session SEA/RC74/16 (Virtual) Nepal 6–10 September 2021 29 July 2021 Management and Governance matters Evaluation: Annual report The WHO South-East Asia Region recognizes the importance of, and is committed to, advancing the culture of “Evaluation” as outlined in the WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development and the South-East Asia Region Evaluation Workplan 2020−2021. This document provides an update on the progress of implementation of the South-East Asia Regional Evaluation Workplan for 2018−2019 and 2020−2021 and key considerations for the way forward. The Region collaborated with the WHO Global Evaluation Office in headquarters to improve the management of evaluations in line with the Global Evaluation Policy and the principles of and guidance provided by the WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development. The Working Paper was presented to the High-Level Preparatory Meeting in July 2021 for its review and recommendations. The HLP Meeting reviewed the paper and made the following recommendations for consideration by the Seventy-fourth Session of the WHO Regional Committee: Actions by WHO (1) Revise the South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development including criteria for selection of topics for evaluation. (2) Utilize the recommendations of the evaluations to improve programme planning and implementation. This Working Paper and the HLP Meeting recommendations are submitted to the Seventy-fourth Session of the WHO Regional Committee for South-East Asia for its consideration and decision. SEA/RC74/16 Introduction 1. At its 143rd session in 2018, the Executive Board approved the amended WHO Evaluation Policy, which requires the Secretariat to report annually to the Executive Board on the progress in implementation of evaluation activities. The present annual report: (i) provides information on the progress made in implementing the WHO Evaluation Policy, including the Organization-wide Evaluation Workplan for 2020–2021; and (ii) documents how evaluations inform policy- and decision-making. 2. The approved evaluation workplans for 2018–2019 and 2020–2021 provide the basis for current activities and the report provides a comprehensive update on the different corporate and decentralized evaluations carried out under Evaluation Office and the progress in strengthening the capacity to implement the corporate evaluation functions. All completed corporate evaluation reports and management responses are available on the webpage of the Evaluation Office.1 3. The Report also includes the progress of the South-East Asia Region’s implementation of its Regional Evaluation Workplan for 2020–2021 and progress of evaluations from the Evaluation Workplan for 2018–2019, which were in progress during reporting in 2019. 4. The report of the Myanmar country programme evaluation was finalized in February 2021. It has identified key achievements, challenges and areas for improvement, and the documentation of best practices and innovations of WHO’s work in the country. 5. Furthermore, it is reported that due to the travel restrictions as a result of the COVID-19 pandemic, evaluation work has been affected or delayed where country visits are an essential methodological component in selected and country programme evaluations planned. 6. The 147th Session (resumed) of the Executive Board in November 2020, based on guidance from its Programme, Budget and Administration Committee, proposed the inclusion of an evaluation of the WHO response to the COVID-19 pandemic in the Evaluation Workplan for 2021, while ensuring there is no duplication with other ongoing evaluation efforts in this regard. The Evaluation Office at WHO headquarters may proceed with an evaluation in 2021. In the meantime, the Evaluation Office has been actively engaged with UN and other partners in the evaluation of the broader COVID-19 response. 7. An independent evaluation of WHO Transformation was conducted to assess its progress to date and the status of implementation of the WHO Transformation Plan and Architecture. The executive summary of the evaluation of the WHO Transformation was also presented to the 149th session of the Executive Board with the evaluation annual report, and the full report is made available now. Management response for the recommendations is being drafted by the WHO Secretariat. 1 World Health Organization. Evaluation office. Geneva. https://www.who.int/about/what-we-do/evaluation - accessed 21 June 2021. SEA/RC74/16 Page 2 8. The WHO Evaluation Office also partners with evaluation counterparts in other entities to participate in joint evaluations in areas of shared substantive and strategic interest, and is also an active member of the United Nations Evaluation Group. It is actively engaged in both corporate evaluations and providing support to decentralized evaluations. The engagement of regional and cluster focal points of the Global Network on Evaluation in corporate and decentralized evaluations has enabled greater coordination of evaluation activities at the three levels of the Organization. Regional perspective 9. The SEA Region continues to accord high importance to evaluation. The ”WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development”, which was presented to the Seventieth session of the Regional Committee for South-East Asia in September 2017, is followed in the Region along with the WHO Global Policy 2018 on evaluation to further strengthen the culture of evaluation in the Regional Office and country offices. 10. This report provides the progress of the SEA regional evaluation workplans for 2018−2019 and 2020−2021. All completed evaluations of the evaluation workplan 2018−2019 were published on the Regional Office website, during the Seventy-third session of the Regional Committee for South-East Asia held virtually from Bangkok, Thailand, in September 2020, based on the WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development. 11. There are two evaluations included in the global evaluation workplan 2020–2021 from the SEA Region and both the evaluations have been completed and published on the Region’s website: a. Evaluation of the use and perceived impact of WHO guidelines on reproductive, maternal, neonatal, child and adolescent health at the country level in the South-East Asia Region. b. Evaluation of the implementation of the Regional Flagship Priority Areas in the South- East Asia Region 2014−2018. 12. India Country Office evaluation (2019) and Myanmar Country Programme evaluation (2021) were completed by the independent Evaluation Office of WHO. The report of the Myanmar Country Programme evaluation was finalized in February 2021. 13. Mid-term evaluation of the Thailand Country Cooperation Strategy 2017–2021, the final evaluation of the Timor-Leste Country Cooperation Strategy 2015–2019 in 2020 and Mid-Term Review of the Nepal Country Cooperation Strategy 2018–2022 were completed in collaboration with WHO headquarters and the Regional Office. 14. The SEA Region Evaluation Workplan 2020–2021 included six evaluations (see Annex 1) covering various programmatic and country-specific topics across technical categories. As of June 2021, three evaluations have been completed (one at the country level and two at the regional level), and the remaining three are in progress and expected to be completed by the end of 2021. SEA/RC74/16 Page 3 15. The following three evaluations have been completed: o Evaluation of the Implementation of Regional Flagship Priority Areas in the WHO South-East Asia Region 2014–2018. The report has been published on the Regional Office website. o Evaluation of the national immunization technical advisory groups in the WHO South- East Asia Region. o Evaluation of UN Partnership on the Rights of Persons with Disabilities (UNPRPD) Project – Advancing the rights of persons with disabilities in Bhutan, 2021. 16. The SEA Region Evaluation Workplan 2018–2019 included 16 evaluations (Annex 1) covering various programmatic and country-specific topics across most technical categories. By the end of the period of the 2018–2019 workplan, nine evaluations were completed (seven at the country level and two at the regional level) and one rolled over; six have been cancelled (details are included in Annex 2). 17. The delays, cancellations and postponements of some evaluations in the SEA Region Evaluation Plan 2018–2019 were mainly due to the constraints in securing funds, non-availability of qualified evaluators, time taken for commissioning and conducting evaluations and restrictions faced at the country level due to the ongoing COVID-19 pandemic. 18. The Region has improved its evaluation management while ensuring consistency with the Global Evaluation Policy and Regional Framework principles and guidance on independence and objectivity, quality assurance by developing standard concept notes, the following of consistent processes for review and approvals, and monitoring of recommendations and management response. 19. Approach to Evaluation in the Region is being reviewed along with the WHO South-East Asia Regional Framework for Strengthening Evaluation for Learning and Development and the Global Evaluation Policy 2018 in order to provide robust guidance to strengthen regional- and country- level evaluations. 20. The SEA Region has addressed the challenge of resource limitation for implementing evaluation workplans by encouraging and allocating flexible funds to budget centres to conduct planned evaluations. However, challenges related to the availability of qualified evaluators for some programmatic and specific country evaluations and the limited time to fully implement the Regional Evaluation Workplan continue, especially due to the COVID-19 pandemic. SEA/RC74/16 Page 4 The way forward 21. As the implementation of the SEA Regional Evaluation Workplan progresses, mechanisms will be put in place to systematically collect and disseminate the results and recommendations to facilitate learning and development from such evaluations. Further, the recommendations from evaluations will be considered during operational planning for the Programme Budget 2022–2023 by the respective technical units and country offices. 22. Based on the experiences to date, it is important to define criteria to prioritize evaluations at country and regional levels, including the relevance and feasibility of evaluations planned for a biennium, as well as the complementarity of audits, reviews and monitoring exercises that also contribute to organizational learning and development. 23. The approach to Evaluation in the Region will be reviewed and criteria outlined for the Regional Evaluation Workplan 2022–2023 incorporating lessons learnt from the COVID-19 pandemic response and the implementation of the regional evaluation plans. SEA/RC74/16 Page 5 Annex 1 SEA Region Evaluation Workplan 2020–2021, status and progress Budget Centre/ Lead office Evaluation type Title of evaluation Outcome in Thirteenth General Programme of Work Status/suggestions Progress as on June 2021 SEA regional/centralized evaluations Director, Programme Management Office Thematic Evaluation of implementation of Regional Flagship Priority Areas in WHO South-East Asia Region 2014-2018 Across all outcomes Started in 2019 and roll over to 2020–2021 Completed in 2020 and report published Communicabl e diseases/ & vaccine development Thematic Evaluation of national immunization technical advisory groups (NITAGs) in the WHO South-East Asia Region Outcome 1.1 Rolled over from 2018–2019 to 2020–2021 Completed. Report to be published Decentralized evaluations Nepal Thematic Evaluation of WHO contribution to strengthen maternal perinatal death surveillance and response (MPDSR) in Nepal. Outcome 1.1 & 1.3 New and ongoing Planned for 1st quarter of 2021 but due to COVID situation in Nepal, evaluation could not be started. If situation improves evaluation will begin in 3rd quarter and be completed by December 2021 Bhutan Thematic Evaluation of UN Partnership on the Rights of Persons with Disabilities (UNPRPD) Project - Advancing the rights of persons with disabilities in Bhutan, 2021 Outcome 3.1 New Completed. Report published SEA/RC74/16 Page 6 Budget Centre/ Lead office Evaluation type Title of evaluation Outcome in Thirteenth General Programme of Work Status/suggestions Progress as on June 2021 India Thematic WHO contribution through its teams to different initiatives (TB, India Hypertension Control Initiative, NTD and health systems) at state level in India Outcome 1.1 New and ongoing Concept note under finalization Thailand Thematic Evaluation of National Road Safety Master Plan implementation 2018–2021. Outcome 3.1 New and ongoing Concept note and tools developed SEA/RC74/16 Page 7 Annex 2 SEA Region Evaluation Workplan 2018–2019, status and progress (as on 31 December 2019) Budget Centre/ Lead Office Evaluation type Title of Evaluation Category in Twelfth General Programme of Work Status as of 31 Dec 2019 SEA regional/centralized evaluations Cross- departmental/ country support and reforms Programmatic Evaluation of WHO’s contribution to combat antimicrobial resistance in the South-East Asia Region Cross- category Cancelled The evaluation was cancelled due to completed evaluation on the Regional Flagship Priority Areas which also covered antimicrobial resistance (Flagship 5) Cross- departmental/ Director, Programme Management Office Thematic Analysis of the technical quality of WHO Regional Office publications for the past 10 years Cross- category Cancelled. This evaluation is cancelled as already a corporate evaluation on the subject is available Health Systems Development Thematic Extent to which WHO activities in the SEA Region have adapted to reflect a more inclusive and integrated approach to development Cross- category Cancelled. This evaluation is cancelled due to continuing and rapid developments in WHO’s role on advancing the SDGs WHO Health Emergency Programme Thematic Evaluation of SEARHEF – 10-year milestones Emergency (Category 5/12) Completed The evaluation is published on the website Nepal Country- specific Evaluation of WHO’s contribution to preparedness, response readiness and recovery of the health system from multi-hazard emergencies Emergency (Category 5/12) and Category 4 Cancelled Due to the COVID-19 pandemic the evaluation was not feasible for roll over to 2020–2021 SEA/RC74/16 Page 8 Budget Centre/ Lead Office Evaluation type Title of Evaluation Category in Twelfth General Programme of Work Status as of 31 Dec 2019 Sri Lanka Country- specific Evaluation of the impact of WHO information products related to NCDs and health systems in Sri Lanka, 2012–2019 Category 2 and 4 Completed The evaluation is published on the website Timor-Leste Country- specific Evaluation of the contribution of the WHO Country Office in Timor- Leste for implementation of the National Immunization Programme in Timor-Leste Category 1 Cancelled Due to the COVID-19 pandemic the evaluation was not feasible for roll over to 2020–2021 Family health, gender and life course Programmatic Evaluation of the adaptation and use of WHO guidelines on reproductive, maternal and newborn health (RMNH) in the WHO South-East Asia Region Category 3 Completed The evaluation is published on the website Thematic Evaluation of national immunization technical advisory groups (NITAGs) in the WHO South-East Asia Region Category 1 Rolled over to 2020–2021. The evaluation is commissioned in two phases. First phase is completed and phase 2 will be conducted in first quarter of 2020 Thematic An in-depth situation analysis of unmet needs and gaps for adolescent health in countries Category 3 Cancelled. As a rapid assessment of adolescent health programme implementation in 10 countries carried out Decentralized evaluations Bangladesh Country- specific Evaluation of community clinics in Bangladesh Category 4 Completed. Evaluation is published on the website SEA/RC74/16 Page 9 Budget Centre/ Lead Office Evaluation type Title of Evaluation Category in Twelfth General Programme of Work Status as of 31 Dec 2019 India Country- specific programmatic Evaluation of WCO’s contribution to advancing the agenda of viral hepatitis B and C prevention and treatment in India Category 1 Completed. The evaluation is published on the website Country- specific programmatic Evaluation of WCO’s contribution to kala-azar elimination programme in India Category 1 Completed. The evaluation is published on the website Country- specific thematic WHO’s role in the development of National Multisectoral Action Plan for prevention and control of NCDs in India Category 2 Completed. The evaluation is published on the website Nepal Country- specific programmatic Evaluation of WHO contribution to enhance/strengthen the health information system in Nepal Category 4 Completed The evaluation is published on the website Sri Lanka Country- specific programmatic Evaluation of WHO contribution towards nutrition outcomes in Sri Lanka (2014–2018) Category 2 Completed. The evaluation is published on the website