Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries

Health system resilience indicators an integrated package for measuring and monitoring health system resilience in countries

Health system resilience indicators an integrated package for measuring and monitoring health system resilience in countries Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries ISBN 978-92-4-008898-6 (electronic version) ISBN 978-92-4-008899-3 (print version) © World Health Organization 2024 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial- ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules/). Suggested citation. Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries. Geneva: World Health Organization; 2024. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see https://www.who.int/publications/book- orders. To submit requests for commercial use and queries on rights and licensing, see https://www.who.int/copyright. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. Design and layout by 400 Communications. Contents Acknowledgements v Abbreviations vi 1 Introduction 1 1.1 Scope and objectives 3 1.2 Target audience 3 1.3 Approach and guiding principles 4 2 Using the package of HSR indicators 5 2.1 Step-by-step process 7 2.2 Using the technical specifications or metadata 13 3 Health system resilience indicators 14 3.1 Overview 15 3.2 Summary list 16 4 Health system resilience indicators with metadata 21 4.1 Service delivery 23 4.2 Health workforce 39 4.3 Health information 45 4.4 Access to medicines and other health products and technologies 58 4.5 Health financing 70 4.6 Governance and leadership 80 4.7 General/composite indicators 119 iii 5 Supplementary indicators of relevance to heal th system resilience 121 6 Resources of indicator sets and monitoring and evaluation frameworks as sources 140 References 146 Annex. Methodological approach to HSR indicator development 148 Defining health system resilience capacities 148 Data collection 149 Data analysis and indicator selection 150 References: Annex 151 ivHealth system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Acknowledgements This development of this work was led by the Special Programme on Primary Health Care, Health System Resilience and Essential Public Health Functions Team: Sohel Saikat (Team Lead), Saqif Mustafa, Redda Seifeldin, Yu Zhang, and Geraldine McDarby, in collaboration with the Integrated Health Services Department, Health Service Assessment Unit: Kathryn O’Neill (Unit Head), Dirk Horemans, Kavitha Viswanathan, and Briana Rivas-Morello. Appreciation goes to Gerard Schmets (Deputy Director, Special Programme on Primary Health Care), Suraya Dalil (Director, Special Programme on Primary Health Care), Rudi Eggers (Director, Integrated Health Services Department) and other expert technical colleagues in WHO headquarters and regional offices for their support and technical contributions to the development of this package. Gratitude also goes to the Korea International Cooperation Agency (KOICA) Global Disease Eradication Fund (GDEF) (within the scope of the project on “Making Health Services Resilient with Quality and Emergency Preparedness for Response in Ethiopia and Liberia”) and the Universal Health Coverage Partnership for the funding support for the development of this package. v Abbreviations DHS Program Demographic and Health Surveys Program EPHF essential public health functions HHFA Harmonized Health Facility Assessment HSR indicator health system resilience indicator IHR International Health Regulations JEE joint external evaluations M&E monitoring and evaluation PAHO Pan American Health Organization PHC primary health care PHCMFI Primary Health Care Measurement Framework and Indicators SARA Service Availability and Readiness Assessment SDG Sustainable Development Goal SPA Service Provision Assessment UHC universal health coverage WHO World Health Organization vi 1 Introduction Health system resilience means that health actors (including institutions and populations) are able to effectively anticipate, prevent, prepare for, absorb and adapt in response to, and recover from a wide variety of shocks and stressors while delivering high- quality individual and population health services as needed, utilizing lessons from experiences within and outside their settings to continuously improve on their baseline capacities and performance in all contexts (1–3). Global experiences with public health emergencies, such as Ebola virus disease outbreaks and the COVID-19 pandemic, have reinforced the need for strengthening health system resilience, as reflected in the proceedings of the World Health Assembly (4), the Intergovernmental Negotiating Body (5), the World Health Organization (WHO) and other global partners. While health system resilience is typically a response to disruptive shocks such as infectious disease outbreaks, acute climate change-related events, and war and conflict, health system resilience is equally important in the face of everyday stressors such as evolving population health needs, resource and infrastructure challenges, and staff shortages. In fact, the development of resilience capacities can be seen as a continuum, with the same capabilities required to ensure the delivery of quality essential health services in the face of chronic stressors also providing the base required to support the maintenance of those services in the midst of responding to a large-scale shock event. While the concept of health system resilience is supported by a rapidly growing knowledge base, there remains an unmet need and demand for its measurement and monitoring as part of sustainably strengthening health systems and building resilience at country level (6, 7). This package of health system resilience indicators (HSR indicators) addresses this gap, complementing the body of experiences and resources identified in the Health Systems Resilience Toolkit (2) and supporting implementation of the recommendations in WHO’s position paper on building health system resilience for universal health coverage and health security (8). The added value of this package includes bringing together what health systems require to be functional during periods of relative normalcy, in the face of day-to- day challenges and stressors, sometimes referred to as “everyday resilience” (Box 1), as well as what is needed when there are small or large shocks. Box 1. Resilience against different types of stressors Health systems must be resilient against a range of shocks and stressors, from acute external events such as natural disasters or infectious disease outbreaks to more chronic internal challenges such as insufficient funding or a chronic shortage of human resources. In academic literature, this differentiation is sometimes reflected in the definition of different types of resilience, such as event-based resilience related to acute events, and everyday resilience for more chronic challenges. Regardless of the cause, acute and chronic shocks and stressors share the underlying principle of disruption, which varies in size and onset. It is important that efforts to build resilience do not focus on a specific type of shock or stressor but rather develop the baseline capacities required to ensure the delivery of quality health services in all contexts. This is in recognition of the fact that it is the same capabilities required to ensure the delivery of quality essential services in the face of chronic challenges that provide the foundation for the provision of services during more acute events. 2Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 1.1 Scope and objectives The package of HSR indicators serves as a dedicated resource for those who wish to measure and monitor health system resilience on a routine basis and in the context of disruptive shocks as well as everyday stressors, and to expand their capacity to do so as their population health needs and health system and monitoring capabilities evolve. It promotes and supports building health system resilience through an integrated approach to health system strengthening1 that embeds essential public health functions, including health security capacities, within health systems to comprehensively address population health needs. The HSR indicators can be adapted for use based on population health needs, country or regional risk profiles, specific health system stressors, disease burden, epidemiological and demographic profile, and income level, and can be applied in a range of contexts, including fragile, conflict-affected and vulnerable settings, humanitarian disasters, and specific country contexts, such as Small Island Developing States (SIDS). The package is not intended to be prescriptive or to serve as a separate assessment tool or framework, but rather to act as a feasible, contextualised and pragmatic resource tool for measuring and monitoring health system resilience. The indicators should therefore be integrated with and complement existing health information systems, for example, by being included in national health sector monitoring and evaluation frameworks – though it is recognized that this kind of integration may take time. The overall aim of this package is to support countries to progressively expand their capabilities to measure, monitor and build health system resilience from national level to health facilities and other service delivery platforms in order to enable the delivery of individual- and population-focused health services in all contexts. The specific objectives include: 1 Health system strengthening comprises the means (for example, policy instruments) to achieve health goals. It can be understood as (a) the process of identifying and implementing the changes in policy and practice in a country’s health system, so that the country can respond better to its health and health system challenges; and (b) any array of initiatives and strategies that improves one or more of the functions of the health system and that leads to better health through improvements in access, coverage, quality or efficiency. • to provide a consolidated reference of indicators that can be adapted and utilized to support measuring and building health system resilience, complementary to and integrated with routine national health information systems and existing WHO and other authoritative guidance and tools; • to promote an integrated approach to strengthening existing measuring and monitoring of health and allied systems in countries by including considerations for making health systems more resilient, thus advancing progress towards universal health coverage and health security; • to support countries in identifying targeted interventions for building health system resilience across policy, planning and operational levels within an integrated, whole-of-system approach. 1.2 Target audience The primary target audience for this package is national and subnational health authorities (including planners and managers) and service providers, as well as local, regional, and global technical organizations and partners working on health system strengthening, including WHO, United Nations country teams, donors, nongovernment organizations, development and humanitarian agencies, and other health-related technical agencies. 31. Introduction 1.3 Approach and guiding principles A multipronged strategy was employed to inform the development of this package. Literature reviews underpinned by systematic methods and technical consultations with selected experts at the national (ministries of health, national public health institutes, WHO country offices and academia), regional (WHO regional offices) and global (WHO headquarters and technical partners) levels, were conducted to identify, adapt, and develop indicators and technical specifications (metadata) and accompanying guidance (see Annex for detailed methodological approach). The set of HSR indicators was also informed by and builds on existing monitoring and evaluation frameworks, guidance, and indicator sets, including Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies (9); Primary health care measurement framework and indicators (10); Continuity of essential health services: facility assessment tools (11); International Health Regulations (2005) monitoring and evaluation framework (12); Universal health and preparedness review (UHPR) (13); health system performance assessments (14); and other health system, health security, and disease-, program- and life course-specific indicator sets and measurement tools (see section 6). It has been aligned with and is complementary to the Primary Health Care Measurement Framework and Indicators (PHCMFI). While not comprehensive, the PHCMFI includes a list of resilience-relevant indicators highlighted within the associated technical annex (15). Both documents will remain as “live” documents to allow the incorporation of evolving evidence and knowledge. Throughout the development of this package, contextualization and utility for end users, and integration with existing health system measuring and monitoring mechanisms, have been considered as guiding principles. It was recognized that feasibility in terms of technical and financial resources required to measure and monitor each indicator would vary by context. Therefore, suggested criteria for assessing feasibility were defined and those indicators categorized as having “high” or “medium” feasibility by consultees and end users were incorporated into this package (Box 2). The same principles guiding the development of this package equally apply in its application as end users select, adapt, and adopt the HSR indicators based on their contexts and needs. Box 2. Feasibility criteria It was recognized that feasibility in terms of technical and financial resources required to monitor each indicator would vary by health system. Therefore, the following criteria for assessing feasibility were defined to guide the selection of HSR indicators. • “High” feasibility was defined as: the necessary resources to measure the indicator are already available or the data are already being collected. • “Medium” feasibility was defined as: one or two of the necessary resources to measure the indicator are currently missing; or, with ongoing efforts, the necessary resources could be made available in the next 6 to 12 months to measure the indicator. • “Low” feasibility was defined as: none or very few of the necessary resources are available to measure the indicator and the necessary level of technical resources required to measure the indicator is unlikely to be available in the near future. 4Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 2 Using the package of HSR indicators In developing this package, it was recognized that countries have existing mechanisms for monitoring health system performance and improvements. Many of the indicators within these frameworks represent aspects of the health system that contribute or are even essential to the development of health system resilience. Others can be adapted to measure and monitor resilience with minor alterations or the addition of specific attributes, while even more can be progressively adapted as needed and feasible. The HSR indicators are not meant to replace existing measures but are intended to be used alongside existing approaches to health system monitoring, broadening the scope where necessary, such that resilience is also explicitly captured within these mechanisms. The indicators within the package of HSR indicators are designed to be applicable across different contexts and settings regardless of the level of maturity of the health system. For this reason, some indicators are high-level composite indicators and others represent specific and foundational aspects of the health system that may require specific focus in the early stages of building resilience or within specific settings, for example in relation to resources, development stages, health system arrangements or priority health needs. Some of the indicators represent similar and even overlapping aspects of resilience, allowing countries to select the indicators most clearly aligned with their measurement approaches and goals and most relevant to their setting. In addition, the indicators within the package can be selected and adapted to align with national priorities, population health needs and health system context. The indicators, their technical specifications (metadata), and associated guidance can support end users in the data collection required for measuring and monitoring health system resilience, and can inform policy-making, planning and system improvements. It is important to note that in the context of this package, “health facilities” refers to all service delivery platforms, including facilities and other units of service delivery, where individual or population health services are provided (for example, facilities, departments, units and programs responsible for health, social care or public health services). In addition, some of the indicators contained within the package are also applicable to the private sector (including for-profit, not-for-profit, informal and other types of service providers). The package, intended to be a live resource and repository of health system resilience indicators, contains: General guidance on how to use the health system resilience indicators Section 2 Section 3&4 Section 5 A set of 64 recommended health system resilience indicators with technical specifications A supplementary 101 indicators of relevance to health system resilience 6Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 2.1 Step-by-step process There are six steps that end users may consider while using this package to support measurement, monitoring and building of health system resilience (Figure 1). Figure 1. Suggested steps in using the package of HSR indicators to enhance measurement, monitoring and building of health system resilience 1. Map HSR indicators 2. Select HSR indicators 3. Define targets 4. Build measurement capacity 5. Measure and monitor 6. Utilize information for improvements a. Map HSR indicators against existing national monitoring and evaluation framework and indicators and health information system Within countries, there are various assessments tools and monitoring and evaluation mechanisms for the health system and allied sectors, including data collection for routine health, health sector performance, health emergencies, and disease-, life course-, humanitarian- or disaster-specific programs. It is unlikely that it will be feasible to monitor the entire set of HSR indicators at one time in a given country context. Instead, the indicator package should be reviewed, and a selection or suite of indicators chosen to measure and monitor health system resilience. It is not necessary for the process of building resilience to be delayed in order to synchronize with ongoing health sector planning or monitoring. Building resilience can begin alongside existing planning and monitoring, for example, within the context of recovery from a public health emergency, with the aim of being integrated and aligned over time. The HSR indicators should ultimately be incorporated into routine health information system monitoring. 72. Using the package of HSR indicators The first step towards this is to map the HSR indicators against existing indicators and approaches to monitoring and evaluation, including those within routine health information systems. This can inform selection of an appropriate suite of indicators by establishing which, if any, aspects of health system resilience are being captured within existing measurement mechanisms, as well as any critical gaps and areas for improvement across health system building blocks and essential public health functions (Table 1). b. Select and prioritize HSR indicators After identifying existing indicators, if any, that are currently being collected in relation to resilience, additional indicators from the package of HSR indicators can be identified to complement these in order to more comprehensively monitor and build resilience. Indicators may be selected for inclusion as they are, or currently collected indicators may be adapted to provide a greater focus on resilience. Adaptation may include tailoring existing data collection tools, such as national-, subnational- or facility-level surveys, or embedding new indicators within routine health information systems. The final suite of indicators chosen to represent health system resilience will be highly context specific and should be informed by the following. • National population and health system context. The choice of indicators should be informed by an understanding of current national health priorities, supported by available population and health system data, including population health needs assessment and health system risk profiling. • Health system foundations. The indicators chosen should reflect the critical gaps in foundational elements required to build resilience in a specific context. Activities or resources dedicated to strengthening resilience benefit from dedicated indicators to measure performance and impact. • People and communities. The involvement of people and communities is integral to the development of health system resilience, and should therefore inform the selection of indicators, such that people and communities are at the centre of informing health system priorities and the development and evaluation of services. This may require attention to building the capacity of people and communities to engage with health system development. A number of indicators with a focus on community engagement are presented in Box 3. • Specificity. The indicators chosen should be granular enough to allow identification of specific issues that need to be addressed, as well as measurement of those issues in the short to medium term. For this reason, some indicators in the package measure a single attribute or component of indicators presented in other frameworks. This supports a focus on measuring and addressing specific deficits. Indicators should be adapted and adjusted with progress, with the ultimate goal of integrating them within existing health system frameworks. • Relevance and feasibility. The indicators chosen should be useful to inform planning and delivery of services in support of building health system resilience, and the technical and financial resources to collect the necessary data should be available. • Integration and duplication. The development of capacities to measure and monitor health system resilience should not duplicate or compete with existing assessments, tools and systems; rather, they should build on existing data collection, as available. There should be a plan to integrate, over time, the HSR indicators into existing monitoring and evaluation mechanisms, such as national- and facility-level assessments and tools, and routine health information systems. • Balance. The final suite of indicators chosen to represent health system resilience in a given context should provide a balanced picture of the system, representing all health system building blocks and public health functions as well as presenting a balance between indicators with specific relevance to disruptive shock events and those measuring aspects of everyday resilience. 8Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Box 3. Examples of indicators on community engagement and participation to be selected across health system building blocks • % facilities using community voice to inform service planning (service delivery) • % facilities providing outreach according to community needs (service delivery) • Mechanism available to assess community trust (health information) • % subnational health workers trained in community engagement (health workforce) • Mechanism for multistakeholder participation and community engagement (governance and leadership) • Mechanism to ensure community engagement in service planning and organization (governance and leadership) c. Define targets for selected HSR indicators As resilience is an ongoing and context-specific process, there are no absolute targets or levels universally applicable to the indicators. In the context of this package, targets are specific, planned levels of results that are to be achieved, usually within a specified time frame, depending on the indicator type and other factors, including administrative level of data collection (national, subnational, community or facility). Understanding the baseline and then defining targets can enable end users such as national and subnational authorities to determine whether progress is being made in line with population health needs, health system context and expectations originally envisioned. This stage allows stakeholders to consider and clarify the mechanisms (including the who, where and how) that are required for collecting the data on set targets. Due to the cross-cutting, multisectoral and interdisciplinary nature of health system resilience, targets should be defined and monitored through a consultative and participatory process involving a relevant and diverse range of actors. Box 4 describes principles for defining targets, while Box 5 provides examples of illustrative targets that can enable an understanding of baselines and progress made towards attainment of targets. 92. Using the package of HSR indicators Box 4. Principles for consideration in defining targets for selected HSR indicators Rational Targets should be set based on best available evidence. International or national guidance and recommendations (for example, International Health Regulations (2005) Monitoring and Evaluation Framework, health system performance assessments) can also inform national and subnational targets in relation to national identified priorities and lessons. Measurable and specific Targets should be described in a way that avoids ambiguity and subjective interpretation. Achievable Targets should be determined based on the current baseline of health system resilience, and gaps between the baseline and goals. Realistic Setting realistic targets means being fair to the organizations, agencies or people who are accountable for reaching them. It is important to ask for improvements in health systems and population health outcomes in the scope that the government, organizations and health workers can actually influence, with appropriate resources. Time-bound Targets should be achieved within a defined time frame as part of the commitment and accountability mechanism. Progress towards health goals can be facilitated if organizations and people have a clear sense of the timeline against which the progress is monitored and evaluated. Depending on the type of indicator and contextual considerations, the targets can be defined as one off achievements or in a progressive manner in relation to timelines (for example, a gradual increase in proportions). Examples are given in Box 5 below. 10Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Box 5. Examples of targets of HSR indicators Indicator: Percentage of facilities offering services according to a nationally defined essential health service package Targets can be defined in terms of percentage of facilities offering a core set of services. For example, if the baseline is “50% of facilities are offering the core package of services”, then the target can be set as “75% of facilities are offering the core package of services by the end of the five-year strategy period”. This can be measured at the national or subnational level. Alternatively, targets can be defined for specific services or disease areas: for example, the target could be to increase the percentage of facilities offering services for communicable disease prevention (for example, immunization) from 85% to 100% by the end of the five-year strategy period. Indicator: Designated team or focal persons for emergency management and service continuity in health facility Targets can be based on consideration of the following levels: 1. No team or focal persons for coordinating emergency management and service continuity available. 2. Team or focal persons for coordinating emergency management and service continuity is available on an ad hoc basis. 3. Designated team or focal persons for coordinating emergency management and service continuity is available with terms of reference specifying roles and responsibilities. 4. Designated team or focal persons for coordinating emergency management and service continuity is available with terms of reference specifying roles and responsibilities, and have been tested through simulation exercises. 5. Designated team or focal persons for coordinating emergency management and service continuity is available with terms of reference specifying roles and responsibilities reviewed and updated on a regular basis. Indicator: Availability of national and subnational multisectoral structure for emergency management with participation of all health service levels and health system resilience specified as a core function Targets can be based on consideration of the following levels: 1. Multisectoral structure for coordinated emergency management is not in place, under development or occurs on an ad hoc basis. 2. Multisectoral structure for emergency management with terms of reference specifying participants, roles and responsibilities is available at the national level. 3. Multisectoral structure for emergency management is available and terms of reference specify health system resilience as a core function. 4. Multisectoral structure for emergency management with health system resilience as a core function is available and specifies participation of all relevant stakeholders at all health service levels. 5. Multisectoral structure for emergency management is evaluated and updated on a regular basis. 112. Using the package of HSR indicators d. Define and build capacity for measuring HSR indicators After selecting HSR indicators and setting targets, it is necessary to determine whether the required data can be collected from existing sources, with or without adaptation, or whether additional data collection will be required to address any critical data gaps. The adaptation of indicators to measure and build resilience can support countries in sustainably strengthening existing data monitoring systems, while identifying opportunities to invest in innovative methods to collect data for new indicators. Capacity building can be achieved by strengthening national and subnational qualitative key informant surveys, facility surveys, routine health information systems and health management information systems, which are the main recommended data sources for HSR indicators and other monitoring and evaluation frameworks for health systems. This should also involve investment in patient- and community-based data systems (for example, regular patient surveys or interviews, community assessments) to support the comprehensive assessment of health system resilience from the community and population side to ensure people-centeredness. Information on gaps and what is required should be fed into existing processes for strengthening capacities in data collection and analysis. This will require consideration of data management in terms of how data are created, stored, processed and destroyed, as well as their confidentiality (where applicable), availability and accessibility (that is, the availability of data to those who require and are authorized to access them). Ensuring the required capacities in data quality, analysis and communication goes hand in hand with strengthening data collection mechanisms. Good-quality data enable better interpretation of the results of analyses and have greater utility for performance improvement, thereby strengthening the data capacities of health ministry officials, public health institutes, national statistics offices, district and facility managers, health professionals, health analysts, and individual providers (public and private). e. Collect and analyse information to measure and monitor HSR indicators Once the suite of indicators has been identified and the capacity to measure indicators is ensured, data and information can be collected from a broad range of sources, such as routine or ad hoc assessments, including facility assessments; clinical reporting systems; population-based surveys; national health accounts; health databases and records; human resource, infrastructure or medical records; policy and planning data; civil registration data; and others, depending on national and subnational contexts. Results of data analysis should be reported and communicated to stakeholders at all levels. The reporting process should include clear and accessible visualization of findings, which can be utilized for decision-making and action. Analysing data supports the discovery of insights and transformation into knowledge of health system performance and utilization. Communication of data and results supports decision-making and action for strengthening health system resilience to meet population health needs. Countries could benefit from a resilience-dedicated dashboard page. In section 4, data sources and existing data collection tools are suggested for each of the core HSR indicators. Data from different sources of data collection can be pooled to formulate targeted understanding or give a more comprehensive picture of the resilience of the health system. f. Utilize information for improvements In order for the chosen HSR indicators to build health system resilience they must be systematically applied to drive performance improvements. To support this, the data and information collected should be linked with and integrated within wider national health sector and allied or related planning – for example, national action plans for health security or antimicrobial resistance plans – to ensure their utility and to guide more targeted interventions for building health system resilience and achieving universal health coverage, health security and healthier populations. Progress towards targets should be reviewed regularly, making adjustments to the indicator set as appropriate. These adjustments may include adding additional resilience indicators to support a more targeted focus on a specific area of weakness or deleting indicators that are no longer useful to inform planning or improvements. This supports the continual process of building health system resilience and is reflective of the learning aspect of resilient health systems. 12Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 2.2 Using the technical specifications or metadata To support the step-by-step approach to using the package of HSR indicators, in section 4, technical specifications are provided for each of the 64 proposed core HSR indicators. Table 1 provides a summary overview of the structure of the technical specifications (or metadata). These include information such as the short and long names of the indicator for ease of identification; definitions (including details on the criteria or attributes needed to measure the indicator); the rationale; administrative or service level and potential disaggregation of the data; numerator and denominator (if applicable); recommended data sources; and available data collection tools if and where relevant. Explanations for the technical specifications are provided in Table 1. Table 1. Structure of technical specifications (metadata) for indicators, with explanations of information provided for each indicator in this package Metadata Explanation of information provided Indicator short name Contains the indicator’s short name. Indicator name Contains the indicator’s full name. Domain Categorizes the indicator according to the WHO health system framework; where indicators are considered cross-cutting, the primary building block or function is in bold. Definitiona Defines and provides further information and criteria to measure the indicator. May include relation with the PHCMFI indicators. Rationale Provides the reason for and importance of this indicator for measurement of health system resilience. Level Designates for which level (facility, subnational or national) the data for measuring the indicator are predominantly collected. Disaggregation Describes the possible separation or disaggregation of compiled information into smaller units. Numerator If the indicator requires a calculation (common in percentage or proportion indicators), the recommended numerator is stated. Denominator If the indicator requires a calculation (common in percentage or proportion indicators), the recommended denominator is stated. Recommended data source Provides a recommendation for potential data sources that can be used to measure the indicator. Type (M&E domain) Categorizes the indicator on the input/structures–process–output scale. For example, indicators related to building blocks such as financing or workforce are typically a part of structures or inputs, and service delivery indicators are usually processes and outputs. Additional reading and references Provides recommended additional reading and key references used to select or develop the indicator. Existing data collection tools Describes existing data collection tools that can be used to measure this indicator. a. For HSR indicators that have close alignment with the PHCMFI, the table includes information on the corresponding PHCMFI indicator number for ease of referral. 132. Using the package of HSR indicators 3 Health system resilience indicators 3.1 Overview This section contains the list of HSR indicators, which build on globally and regionally established frameworks for health system strengthening, primary health care, universal health coverage and health security (see section 6). The indicators represent the critical elements health systems require to be functional during regular times, as well as when the health system is undergoing more acute or large-scale shocks and stressors. In this package, the indicators are broadly organized according to the health system building blocks domain with reference to the essential public health functions, as well as the administrative level (Table 2). Many indicators are associated with more than one building block or function due to the cross-cutting nature of health system resilience. 153. Health system resilience indicators 3.2 Summary list Table 2. Summary list of core HSR indicators with primary domain measured No. Main domain Indicator short name Levela National Subnational Facility 1. Service delivery % facilities offering services according to nationally defined service package ✓ 2. Service delivery % facilities conducted simulation exercise ✓ 3. Service delivery % facilities conducted after- or intra-action reviews ✓ 4. Service delivery % facilities that closed/discontinued services ✓ 5. Service delivery % facilities with specified Infection Prevention and Control Assessment Framework level/score ✓ 6. Service delivery Collaboration between facility-based and community-based delivery ✓ ✓ 7. Service delivery % facilities sharing practices and lessons ✓ 8. Service delivery % facilities using community voice to inform service planning ✓ 9. Service delivery % facilities providing outreach according to community needs ✓ 10. Workforce % subnational health workers trained in community engagement ✓ ✓ 11. Workforce % facilities with focal point for emergency management and service continuity ✓ 12. Workforce Roster of rapid response teams available ✓ ✓ 13. Workforce % facilities covered by occupational health services ✓ 14. Workforce % facilities with staff having received health system resilience training ✓ 15. Health information Current state of essential public health functions delivery ascertained ✓ ✓ 16. Health information Mechanism in place to assess community trust ✓ ✓ 17. Health information Comprehensive surveillance and response system ✓ ✓ 16 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 2 (continued). Summary list of core HSR indicators with primary domain measured No. Main domain Indicator short name Levela National Subnational Facility 18. Health information Completeness of reporting by facilities ✓ 19. Health information Health system resilience measured and monitored in routine health information system ✓ ✓ 20. Health information % facilities with risk profiles ✓ 21. Health information Early warning system established ✓ ✓ 22. Health information Mechanism for multisectoral information sharing ✓ ✓ 23. Health information Vulnerability and risk mapping conducted ✓ 24. Access to medicines and other health products and technologies National list of essential medicines ✓ 25. Access to medicines and other health products and technologies Regulatory mechanisms for essential health products ✓ ✓ 26. Access to medicines and other health products and technologies % facilities with prepositioned essential supplies ✓ 27. Access to medicines and other health products and technologies Availability of essential medicines ✓ 17 3. Health system resilience indicators Table 2 (continued). Summary list of core HSR indicators with primary domain measured No. Main domain Indicator short name Levela National Subnational Facility 28. Access to medicines and other health products and technologies % facilities experiencing water supply interruption ✓ 29. Access to medicines and other health products and technologies % facilities experiencing power outages ✓ 30. Access to medicines and other health products and technologies % facilities with basic WASH amenities ✓ 31. Health financing % facilities with user fee waiver mechanisms ✓ 32. Health financing Mechanism to address financial barriers ✓ ✓ 33. Health financing Dedicated budget line for service continuity ✓ ✓ ✓ 34. Health financing Contingency funds available ✓ ✓ 35. Health financing Contingency funds accessible to facilities ✓ ✓ ✓ 36. Health financing Resource mapping conducted ✓ ✓ 37. Health financing Public health services funded ✓ ✓ 18 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 2 (continued). Summary list of core HSR indicators with primary domain measured No. Main domain Indicator short name Levela National Subnational Facility 38. Governance Service package meeting criteria ✓ ✓ 39. Governance Availability of protocol for prioritization of services ✓ ✓ ✓ 40. Governance Availability of priority disease and event case management protocols ✓ ✓ ✓ 41. Governance System for conducting simulation exercises ✓ ✓ 42. Governance % facilities part of collaborative networks ✓ 43. Governance All-hazards emergency preparedness and response plan defines role of health services ✓ ✓ 44. Governance % facilities with emergency management plans incorporating service continuity ✓ 45. Governance % facilities with plans or service delivery models for hard-to-reach populations ✓ 46. Governance Health facility infrastructure standards for health facility resilience ✓ ✓ 47. Governance % facilities that meet standards for infrastructure ✓ 48. Governance Mechanism for multistakeholder participation and community engagement ✓ ✓ 49. Governance Mechanism to ensure community engagement in service planning and organization ✓ ✓ 50. Governance % facilities with standard operating procedures for ensuring essential supplies ✓ 51. Governance % facilities with standard operating procedures for repurposing resources ✓ 52. Governance Institutionalizing learning from public health events ✓ ✓ 53. Governance Health system resilience as a function in emergency management structure ✓ ✓ 54. Governance Emergency policy defines role of health services ✓ 55. Governance Health sector policy defines roles of health services for emergencies ✓ 19 3. Health system resilience indicators Table 2 (continued). Summary list of core HSR indicators with primary domain measured No. Main domain Indicator short name Levela National Subnational Facility 56. Governance Health sector plan includes preparedness activities ✓ ✓ 57. Governance Designated entity or structure for health system resilience ✓ ✓ 58. Governance Guideline on equity and ethics for service delivery ✓ 59. Governance Institutional capacity for essential public health functions coordination ✓ ✓ 60. Governance Focal point designated for IHR health services provision assessment ✓ 61. Governance Health in All Policies approach being implemented ✓ 62. Governance Recovery planning guidance ✓ ✓ 63. Governance Designated authority with responsibility for recovery ✓ ✓ 64. Composite IHR SPAR health services provision capacity score ✓ a. “Level” refers to the level at which data are generated and collected. Definitions for indicators can be adapted to be applicable to different levels. 20 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries 4 Health system resilience indicators with metadata This section provides the metadata and technical specifications for the 64 recommended health system resilience indicators. 22Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 4.1 Service delivery Indicator 1. Percentage of facilities offering services according to nationally defined essential health service package for their level of care Indicator short name % facilities offering services according to nationally defined service package Indicator name Percentage of facilities offering services according to nationally defined essential health service package for their level of care Domain Service delivery; Governance and leadership Definition Service package of essential health services (including primary care services) and public health functions is developed and meets following criteria: • Addresses comprehensive essential individual and population health services, including: – Health protection – Prevention – Promotion – Management (diagnosis, treatment, rehabilitation, resuscitation) – Palliation • Includes key life course needs and disease programs: – Foundations of care management of emergency syndromes and common pres- entations in primary care – Reproductive and sexual health, including pregnancy, childbirth and family plan- ning – Growth, development, disability and ageing – Communicable diseases – Noncommunicable diseases – Mental health, neurological and substance use disorders – Violence and injury • The package addresses disease burden and other national priorities, including risk factor profiles and projections • The process for development of the service package involves a wide range of stakeholders • The package is based on an evaluation of existing resources • The package is routinely revised as part of national planning processes • The package includes and designates key services related to emergency events for which the country is at risk This indicator is in the Primary Health Care Measurement Framework and Indicators (2022) [indicator 66]. Rationale Availability of health services should be aligned with a country’s defined package of essential health services. This measures the availability of individual and public health services in the relevant health care settings (for example, primary care, hospital, and long-term care). It indicates the functionality and everyday resilience of the health system for delivering services required to meet population health needs, also considering the public health landscape, including risk profile. Where the minimum health services are routinely unavailable, they are even more unlikely to be provided as needed in times of crisis. Level Facility 234. Health system resilience indicators with metadata Indicator 1 (continued). Percentage of facilities offering services according to nationally defined essential health service package for their level of care Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practitioner practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, or other National Urban/rural Numerator Number of facilities offering the total package of core services; number of facilities offering each service Denominator Total number of facilities examined Recommended data source Facility survey or facility census or routine health information system Type (M&E domain) Output Additional reading World Health Organization. 2021. 21st century health challenges: can the essential public health functions make a difference? Discussion paper. Geneva: WHO (https://apps.who.int/iris/handle/10665/351510) World Health Organization. 2021. UHC compendium: health interventions for universal health coverage. Geneva: WHO (https://www.who.int/universal-health-coverage/ compendium) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools From existing health facility survey tools such as WHO’s Service Availability and Readiness Assessment (SARA) and Harmonized Health Facility Assessment (HHFA), and the Demographic and Health Surveys (DHS) Program’s Service Provision Assessment (SPA): World Health Organization. Service Availability and Readiness Assessment (https://www.who.int/data/data-collection-tools/service-availability-and-readiness- assessment-(sara)) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) DHS Program. Service Provision Assessment, May 2022 (https://dhsprogram.com/ publications/publication-spaq8-spa-questionnaires-and-manuals.cfm) 24Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 2. Percentage of health facilities that participated in a simulation exercise to test health system resilience within the last year Indicator short name % facilities conducted simulation exercise Indicator name Percentage of health facilities that participated in a simulation exercise to test health system resilience within the last year Domain Service delivery Definition The percentage of health facilities (for example, in a geographical area or within a network of health facilities) that have participated in a simulation exercise that specifically tests health system resilience within the last year This indicator is included as one of the attributes of indicator 61 in the Primary Health Care Measurement Framework and Indicators (2022 version) Rationale In addition to simulation exercises conducted at the national or subnational/ regional level, it is important to conduct simulated interactive exercises to test the capability of health facilities or groups of health facilities with inter- and multidisciplinary participation as well as with a variety of stakeholders, such as the community the health facility serves. Much like simulation exercises conducted specifically to test emergency preparedness and response capacities, regular and routine participation in simulation exercises with a focus on health system resilience can facilitate learning and improvement of services and systems, which can contribute to enhanced maintenance of essential health services in all contexts as well as emergency preparedness and response. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practitioner practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, or other National Urban/rural Numerator Number of health facilities that have participated in a simulation exercise to test health system resilience in the last year Denominator Total number of health facilities assessed in the last year Recommended data source Facility survey Type (M&E domain) Process 254. Health system resilience indicators with metadata Indicator 2 (continued). Percentage of health facilities that participated in a simulation exercise to test health system resilience within the last year Additional reading World Health Organization. 2021. Health systems resilience simulation exercises. Geneva: WHO (https://www.who.int/teams/primary-health-care/health-systems- resilience/integrated-health-system-strengthening/health-systems-resilience- simulation-exercises) World Health Organization, Pan American Health Organization. 2011. Guidelines for developing emergency simulations and drills. Area on Emergency Preparedness and Disaster Relief. Washington, D.C.: WHO, PAHO World Health Organization Regional Office for South-East Asia. 2006. A guide for conducting table-top exercises for national influenza pandemic preparedness. New Delhi: WHO Regional Office for South-East Asia (https://iris.who.int/ handle/10665/204728) World Health Organization Regional Office for the Western Pacific. 2006. Creating and tracking pandemic preparedness plans: a guide. Manila: WHO Regional Office for the Western Pacific World Health Organization Regional Office for the Western Pacific. 2006. Exercise development guide for validating influenza pandemic preparedness plans. Manila: WHO Regional Office for the Western Pacific World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of these data 26Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 3. Percentage of health facilities that have conducted or participated in after-action reviews (AAR) and/or intra-action reviews (IAR) that evaluated health system performance including maintenance of essential health services in the last year Indicator short name % facilities conducted after- or intra-action reviews Indicator name Percentage of health facilities that have conducted or participated in after-action reviews (AAR) and/or intra-action reviews (IAR) that evaluated health system performance including maintenance of essential health services in the last year Domain Service delivery Definition The percentage of health facilities (for example, in a geographical area or within a network of health facilities) that have either conducted, or participated in, after- action reviews (AAR) and/or intra-action reviews (IAR) that evaluate health system performance including maintenance of essential health services in the last year Rationale AARs and IARs are qualitative reviews of actions usually taken in response to an event of public health concern as a means of identifying best practices, gaps, and challenges. AARs are usually conducted after an event whereas IARs can be conducted during response to an event. They can enable identification of actions that need to be implemented immediately to ensure better preparation for the next or future events as well as more medium- and long-term actions needed to strengthen the health system. Regular participation of facilities in AARs and IARs which specifically evaluate health systems performance including maintenance of essential health services can enhance emergency preparedness, response, and recovery as well as build health systems resilience in the short, medium, and longer term. Learning and improving health systems performance through experience is a key capacity of resilient health systems. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational Urban/rural Numerator Number of health facilities that have conducted or participated in AARs and/or IARs that evaluated health system performance including maintenance of essential health services in the last year Denominator Total number of health facilities assessed in the last year Recommended data source Facility survey Type (M&E domain) Process Additional reading and references World Health Organization. 2019. Guidance for after action review (AAR). Geneva: WHO (https://www.who.int/publications/i/item/WHO-WHE-CPI-2019.4) World Health Organization. 2020. Guidance for conducting a country COVID-19 intra- action review (IAR). Geneva: WHO (https://www.who.int/publications/i/item/WHO- 2019-nCoV-Country_IAR-2020.1) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 274. Health system resilience indicators with metadata Indicator 4. Percentage of health facilities that were closed or discontinued routine health services (completely or partially for one day or more) in the last year Indicator short name % facilities closed or discontinued services Indicator name Percentage of health facilities that were closed or discontinued routine health services (completely or partially for one day or more) in the last year Domain Service delivery Definition The percentage of health facilities (for example, in a geographical area or within a network of health facilities) that were closed or discontinued delivery of routine and/ or essential health services (completely or partially for any duration) in the last year Rationale The maintenance of routine and essential health services can suffer from health facility closures due to unexpected public health events (such as infectious disease outbreaks, chemical, radiological, or nuclear events, or natural disasters) or routine health system stressors (such as staff unavailability, economic downturn, changes in policy, planning and organization of service delivery). Understanding the proportion of health facilities that are closed or that discontinued routine and essential health services can enable greater understanding of the extent of disruptions to routine functionality and resilience of the health system. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational Urban/rural Numerator Number of health facilities that were closed or discontinued health services in the last year Denominator Total number of health facilities assessed in the last year Recommended data source Routine health information system (master facility list) Key informant interviews at a subnational administrative office (for example district office) Type (M&E domain) Output 28Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 4 (continued). Percentage of health facilities that were closed or discontinued routine health services (completely or partially for one day or more) in the last year Additional reading and references Barnard M, Mark S, Greer SL, Trump BD, Linkov I, Jarman H. Defining and analyzing health system resilience in rural jurisdictions. Environ Syst Decis. 2022;42(3):362-371. doi: 10.1007/s10669-022-09876-w. Fleming P, O’Donoghue C, Almirall-Sanchez A, Mockler D, Keegan C, Cylus J et al. Metrics and indicators used to assess health system resilience in response to shocks to health systems in high income countries-A systematic review. Health Policy. 2022 Dec;126(12):1195-1205. doi: 10.1016/j.healthpol.2022.10.001. World Health Organization. 2021. Continuity of essential health services: facility assessment tool: a module from the suite of health service capacity assessments in the context of the COVID-19 pandemic: interim guidance, 12 May 2021. Geneva: WHO (https://apps.who.int/iris/handle/10665/341306) World Health Organization. 2021. Health service continuity planning for public health emergencies: a handbook for health facilities. Interim version for field testing. Geneva: WHO (https://www.who.int/publications/i/item/9789240033337) Existing data collection tools Robust and up-to-date master facility lists should have this information available. No specific data collection tool is necessary. Otherwise, a more manual process of going through the facility list with the district office can show what health facilities have closed/open in the last year. 294. Health system resilience indicators with metadata Indicator 5. Percentage of facilities that meet specified Infection Prevention and Control Assessment Framework (IPCAF) level/score Indicator short name % facilities with specified Infection Prevention and Control Assessment Framework level/score Indicator name Percentage of facilities that meet specified Infection Prevention and Control Assessment Framework (IPCAF) level/score Domain Service delivery; Governance and leadership; Health infrastructure Definition The Infection Prevention and Control Assessment Framework (IPCAF) at the facility level is a tool to support implementation of WHO guidelines on core components of Infection Prevention and Control (IPC) programs such as: • IPC programs • IPC guidelines • IPC education and training • Healthcare-associated (HAI) surveillance • Multimodal strategies for implementation of IPC • Monitoring/audit of IPC practices and feedback • Workload, staffing and bed occupancy • Built environment, materials and equipment for IPC Through a structured, close-formatted questionnaire and associated scoring system, the IPCAF can assess the current IPC situation in a facility. The score can be interpreted and used to determine the assigned “IPC level” in a facility as inadequate, basic, intermediate or advanced. This indicator is in Primary Health Care Measurement Framework and Indicators (2022 version) [indicator 69]. Rationale Resilient health facilities and systems implement strong IPC measures including implementation of IPC programs and guidelines, standardized protocols, staff training, HAI surveillance, monitoring and evaluation, bed occupancy, and built environment considerations. Strong IPC measures prevent infection, re-infection and transmission of disease, alleviating pressure on the health system and preventing larger scale outbreaks. The IPC score as defined by the IPCAF can also provide an indication as to the level of progress needed from an improvement perspective which is a key capacity of resilient health systems. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc.) Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of facilities by IPC level (inadequate, basic, intermediate, or advanced) Denominator Total number of facilities assessed Recommended data source Facility survey Self-assessment; joint assessment (for example, with facility staff, ministries of health, WHO or other stakeholders); external assessors 30Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 5 (continued). Percentage of facilities that meet specified Infection Prevention and Control Assessment Framework (IPCAF) level/score Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2016. WHO Guidelines on core components of IPC programmes at the national and acute health care facility level. Geneva: WHO (http://www.who.int/infection-prevention/publications/core-components/en/) World Health Organization. 2018. Infection prevention and control assessment framework at the facility level. Geneva: WHO (https://www.who.int/publications/i/ item/WHO-HIS-SDS-2018.9) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2018. Infection prevention and control assessment framework at the facility level. Geneva: WHO (https://www.who.int/publications/i/ item/WHO-HIS-SDS-2018.9) 314. Health system resilience indicators with metadata Indicator 6. Existence of formal linkages between facility-based and community-based service delivery Indicator short name Collaboration between facility-based and community-based service delivery Indicator name Existence of formal linkages between facility-based and community-based service delivery Domain Service delivery; Governance and leadership; Workforce Definition Facility level: Percentage of primary care and first referral hospitals within a defined catchment area that have established formal linkages with community-based service providers, including community health workers (CHWs). These linkages may include: • Community-based providers are integrated in the facility management structures, facility teams, and data systems; • Supportive supervision and training opportunities are made available by primary- care facility to the community-based service providers; • Other formal mechanisms of clinical decision support provided by the facility to community service providers; • Shared protocols for two-way referral of patients between the facility and community-based providers and receive referrals from community-based providers. Qualitative measurement at aggregate levels (for example, national/subnational): Evidence for formal linkages can be defined as either clear national or regional guidelines that define the roles between the different service delivery platforms or written agreements that formalize this relationship locally. This indicator is in Primary Health Care Measurement Framework and Indicators (2022 version) [indicator 56]. Rationale Creating sustainable, effective linkages between facilities and community settings can improve people’s use of promotion and preventive services, their timely access to facility-based services and their adherence to treatment. These positive outcomes are achieved when community-based service providers are trusted by the community they serve and by facility-based providers and when they are partnering to ensure continuity of care and improved clinical quality (i.e., through training or formative supervision). In addition, community-based providers have a role to alert facility- based providers of public health issues and help carry the voice of the people they serve to improve responsiveness of primary care services. They can act as an effective broker between communities and district or facility managers. These linkages connect clinical providers, community organizations, and public health agencies. Strong linkages between different providers enhance continuity of care and enables and promotes integrated, people-centred health services attuned to the needs of the community. Integrated delivery of essential health services builds resilience in the system through enhancing the patient experience and community trust in the health system and reducing inefficiencies and errors. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator N/A Denominator N/A 32Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 6 (continued). Existence of formal linkages between facility-based and community-based service delivery Recommended data source Facility survey Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references McIntyre H, Reeves V, Loughhead M, Hayes L, Procter N. Communication pathways from the emergency department to community mental health services: A systematic review. Int J Ment Health Nurs. 2022 Dec;31(6):1282-1299. doi: 10.1111/inm.13024 World Health Organization. 2017. WHO community engagement framework for quality, people-centred and resilient health services. Geneva: WHO (https://apps.who.int/iris/ handle/10665/259280) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2012. African partnerships for patient safety: patient safety situational analysis (Long form). Geneva: WHO (https://apps.who.int/iris/ handle/10665/330052) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) WHO is currently revising its facility survey modules to incorporate and address primary health care specific elements. 334. Health system resilience indicators with metadata Indicator 7. Percentage of health facilities that participate in a platform to share good practices and lessons learned from public health challenges (including emergencies) Indicator short name % facilities sharing practices and lessons Indicator name Percentage of health facilities that participate in a platform to share good practices and lessons learned from public health challenges (including emergencies) Domain Service delivery; Governance and leadership Definition The proportion of health facilities surveyed that participates in a platform through which good practices and lessons learned from public health challenges (for example, public health emergencies, public health incidents, everyday operations) from local contexts and beyond are shared Examples of platform to share good practices and lessons include regular regional meetings, network of health facilities, communication or reporting channels with the function to collect and disseminate good practices and lessons among health facilities and other stakeholders, designated online platform to disseminate good practices and lessons, etc. Rationale Learning from past lessons and good practices for health facility and service improvement is an important capacity of resilient health system which enables them to adapt, transform and better respond to health threats while maintaining core functions. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Local catchment areas Urban/rural Numerator Number of health facilities that participate in the platform Denominator Total number of health facilities assessed Recommended data source Facility survey Type (M&E domain) Process Additional reading and references World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) Existing data collection tools Not at present 34Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 8. Percentage of facilities that use community health needs and priorities to inform service prioritization Indicator short name % facilities using community voice to inform service planning Indicator name Percentage of facilities that use community health needs and priorities to inform service prioritization Domain Services delivery Definition There is evidence that health facilities service planning and organization is informed by community health needs and priorities as identified through, but not limited to, the following activities: • community health needs and asset assessments or equivalent • participatory processes for priority setting • patients and relatives’ surveys • training of patient advocates • membership of community representatives in advisory board at the local level or in supervisory boards of facilities Rationale Providing services to communities aligned with local health needs and priorities ensure the most effective support for those greatest in need. Responding to local needs and priorities provides an opportunity for health facilities to improve health outcomes within a population and groups with specific health needs, or within a specific geographical area, such as exposure to a specific environmental hazard, or infectious disease outbreaks in a particular group of schools. It also builds trust between the community and health providers. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational Local catchment areas Urban/rural Numerator Number of facilities that provide services to communities according to local health needs and priorities Denominator Total number of facilities assessed Recommended data source Facility survey Type (M&E domain) Process 354. Health system resilience indicators with metadata Indicator 8 (continued). Percentage of facilities that use community health needs and priorities to inform service prioritization Additional reading and references Centers for Disease Control and Prevention (CDC). 2022. Community Health Assessments & Health Improvement Plans. Atlanta: CDC (https://www.cdc.gov/ publichealthgateway/cha/plan.html) Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization Regional Office for Europe. 2001. Community Health Needs Assessment: An introductory guide for the family health nurse in Europe. Copenhagen: WHO Regional Office for Europe (https://www.euro.who.int/__data/ assets/pdf_file/0018/102249/E73494.pdf) Existing data collection tools From existing health facility survey tools such as WHO’s Service Availability and Readiness Assessment (SARA) and Harmonized Health Facility Assessment (HHFA), and the Demographic and Health Surveys (DHS) Program’s Service Provision Assessment (SPA): DHS Program. 2022. Service Provision Assessment, May 2022 (https://dhsprogram. com/publications/publication-spaq8-spa-questionnaires-and-manuals.cfm) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) World Health Organization. Service Availability and Readiness Assessment (https://www.who.int/data/data-collection-tools/service-availability-and- readiness-assessment-(sara)) 36Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 9. Percentage of facilities providing outreach according to community needs Indicator short name % facilities providing outreach according to community needs Indicator name Percentage of facilities providing outreach according to communities needs Domain Services delivery Definition Percentage of facilities that provide community outreach services based on local needs and priorities, informed by the following (not exhaustive): Community health needs and asset assessment or equivalent Participatory processes for priority setting at local levels Patient and relatives’ surveys Rationale Community outreach services increase the range and effectiveness of health services, including protective and promotive services. Outreach services are often targeted at the most vulnerable and can overcome access issues, increasing engagement among this population group. Ensuring the involvement of communities in planning and organization of health services, including outreach services, orients local providers to the needs of all those in the community, including the most vulnerable, promotes trust and increases accessibility of services. This advances UHC and supports uptake of services including preventive and promotive services while also promoting compliance with public health advice during emergencies. Level Facility Disaggregation Facility type (as relevant to context); including primary care facilities (for example, general practices, health centres, community health posts), first level hospitals, second-level hospitals, specialty hospitals, long term care facilities, continuing care facilities, etc.) Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of facilities where outreach services are informed by local need Denominator Total number of facilities assessed Recommended data source Qualitative assessment based on interview with key informant and/or desk review of country documents Type (M&E domain) Output 374. Health system resilience indicators with metadata Indicator 9 (continued). Percentage of facilities providing outreach according to community needs Additional reading and references Primary Health Care Performance Initiative. 2019. Primary Health Care Progression Model Assessment Tool (measure 26 – community engagement) (https:// improvingphc.org/sites/default/files/PHC-Progression%20Model%202019-04-04_ FINAL.pdf) World Health Organization. 2017. WHO community engagement framework for quality, people-centred and resilient health services. Geneva: WHO (https://apps.who.int/iris/ handle/10665/259280) World Health Organization. 2021. Voice, agency, empowerment: handbook on social participation for universal health coverage. Geneva: WHO (https://www.who.int/ publications/i/item/9789240027794) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization Regional Office for Europe. 2019. Indicator passport - WHO European Primary Health Care, Impact, Performance and Capacity Tool. Copenhagen: WHO Regional Office for Europe (https://iris.who.int/handle/10665/346478) Existing data collection tools World Health Organization. 2020. Community Engagement: A health promotion guide for universal health coverage in the hands of the people. Geneva: WHO (https://www. who.int/publications/i/item/9789240010529) 38Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 4.2 Health workforce Indicator 10. Percentage of health workers at subnational (for example, district) and primary care levels trained in community engagement and risk communication Indicator short name % subnational health workers trained in community engagement Indicator name Percentage of health workers at subnational (for example, district) and primary care levels trained in community engagement and risk communication Domain Health workforce; Community engagement Definition At the facility level: Percentage of facilities with at least one health worker that has completed training in community engagement and risk communication within the last 5 years that includes but is not limited to: • Communication skills • Management of information (for example related to an emergencies) • Authentic engagement • Cultural responsiveness and adaptation • Collaboration • Advancing equity • Using the principles of crisis and risk communication At the health worker level: Percentage of health workers that have undergone training in community engagement and risk communication meeting criteria defined above. Rationale Health workers at the primary care level are often the first point-of-contact between communities and the health system. Engaging communities and being able to effectively communicate risk is an important role of health workers as part of emergency preparedness and response, and maintenance of essential health services. Level Subnational; facility Disaggregation No disaggregation for key informant qualitative assessments Facility surveys: Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first level hospitals, second level hospitals, etc.) Managing authority: public, private Sub-national Urban/rural Health worker surveys: by health worker type (as relevant to context) Numerator No numerator for key informant qualitative assessments Facility level assessments: Number of facilities that have at least one health worker that has been trained in community engagement and risk communication Health worker survey: Number of health workers trained in community engagement and risk communication 394. Health system resilience indicators with metadata Indicator 10 (continued). Percentage of health workers at subnational (for example, district) and primary care levels trained in community engagement and risk communication Denominator No denominator for key informant qualitative assessments Facility level assessments: Total number of facilities Health worker survey: Total number of health workers assessed Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Facility-level reviews or surveys Health worker survey Type (M&E domain) Process Additional reading and references World Health Organization. 2017. Communicating risk in public health emergencies: a WHO guideline for emergency risk communication (ERC) policy and practice. Geneva: WHO (https://www.who.int/publications/i/item/9789241550208) World Health Organization. 2017. WHO community engagement framework for quality, people-centred and resilient health services. Geneva: WHO (https://apps. who.int/iris/handle/10665/259280) World Health Organization. 2021. Voice, agency, empowerment: handbook on social participation for universal health coverage. Geneva: WHO (https://www.who. int/publications/i/item/9789240027794) Existing data collection tools World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/ publications/i/item/9789240051980) 40Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 11. Percentage of facilities with a designated team or focal person(s) for emergency management and service continuity Indicator short name % facilities with focal point for emergency management & service continuity Indicator name Percentage of facilities with a designated team or focal person(s) for emergency management and service continuity Domain Health workforce Definition A designated team or focal person with terms of reference which include responsibility for leading and coordinating emergency management and essential health services continuity in a coordinated manner at the facility. Rationale Experiences from public health emergencies and events highlight that coordination structures and capabilities for emergency management and maintenance of essential health services are less established at the subnational and facility level versus the national level. The designation of a focal team or person with responsibility for such functions can enhance their effectiveness, ensure their sustainability, enhance accountability, and improve health outcomes. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practitioner practice, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc.) Managing authority: public, private Subnational Urban/rural Numerator Number of facilities with a designated team or focal person(s) for emergency management Denominator Total number of facilities assessed Recommended data source Facility survey Type (M&E domain) Input and structure Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, et al. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255-268. doi: 10.1093/ heapol/czab089 World Health Organization. 2020. Maintaining essential health services: operational guidance for the COVID-19 context: interim guidance, 1 June 2020. Geneva: WHO (https://www.who.int/publications/i/item/WHO-2019-nCoV-essential_health_ services-2020.2) World Health Organization. 2021. Continuity of essential health services: facility assessment tool: a module from the suite of health service capacity assessments in the context of the COVID-19 pandemic: interim guidance, 12 May 2021. Geneva: WHO (https://apps.who.int/iris/handle/10665/341306) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2021. Continuity of essential health services: facility assessment tool: a module from the suite of health service capacity assessments in the context of the COVID-19 pandemic: interim guidance, 12 May 2021. Geneva: WHO (https://apps.who.int/iris/handle/10665/341306) 414. Health system resilience indicators with metadata Indicator 12. Functional, up to date (within the last 12 months) roster of a readily available multidisciplinary rapid response team for emergency response and surge capacity in place Indicator short name Roster of rapid response team available Indicator name Functional, up to date (within the last 12 months) roster of a readily available multidisciplinary rapid response team for emergency response and surge capacity in place Domain Health workforce; Governance and leadership Definition Roster of rapid response team is readily available with the following criteria: • Multidisciplinary (for example, surveillance officers, epidemiologists, public health officers, clinicians, laboratory technicians, risk communication officers, point of entry officers, social scientists, and other relevant disciplines to the context) • Have technical knowledge and skills to investigate and rapidly respond to public health emergencies such as infectious disease outbreaks • Terms of references for emergency response and surge capacity • Surge personnel includes other sectors (for example, chemical, radiation, animal health) • Terms of references for emergency response and surge capacity Rationale Rapid response teams for emergency response and surge capacity enable effective management of unexpected threats to health. Rapid response teams respond to emergencies when and where they arise and can stop small scale events from becoming larger scale emergencies or disasters. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as terms of reference/job descriptions of provincial/district rapid response teams Type (M&E domain) Input Additional reading and references S Tweed, DE Stewart, E Hornsey, W Graham, Increasing role of Public Health Rapid Response Teams in infectious disease outbreaks, European Journal of Public Health, Volume 32, Issue Supplement_3, October 2022, ckac130.022, https://doi.org/10.1093/ eurpub/ckac130.022 World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/ item/9789240051980) Existing data collection tools World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/ item/9789240051980) 42Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 13. Percentage of facilities with access to or being covered by dedicated services for occupational safety and health Indicator short name % facilities covered by occupational health services Indicator name Percentage of facilities with access to or being covered by dedicated services for occupational safety and health Domain Health workforce; Governance and leadership Definition Percentage of faculties being covered by services for occupational safety and health hazards, identified within their context (for example, occupational health, health and safety services, including services for those health and social care workers affected by public health emergencies or events, etc.). Arrangements for occupational safety and health services established (for example, services in facility; services provided by local health authorities) can be considered as facilities being covered. Examples of risks includes (non-exhaustive): • Harmful chemical and biological agents/substances • Psychosocial risks and stress at work • Electrical hazards • Fire • Accidents • Noise • Working in overcrowded spaces • Failure of personal protective equipment • Work overload • Attacks on health workers Rationale Ensuring the fundamental right to a safe and healthy working environment is essential to prevent work-related accidents and diseases and protect and promote the health and well-being of workers. Protecting health workers is key to ensuring the effective delivery of quality and safe essential health services in all contexts and the resilience of the health system. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practice, health centres, community health posts), hospitals, temporary health facilities, etc. Managing authority: public, private Subnational Urban/rural Numerator Number of facilities with access to or being covered by occupational safety and health management systems and services Denominator Total number of facilities assessed Recommended data source Facility survey Type (M&E domain) Input and structure Additional reading and references International Labour Organization (ILO). Occupational Safety and Health Management Systems. Geneva: ILO (https://www.ilo.org/safework/areasofwork/occupational- safety-and-health-management-systems/lang--en/index.htm) Existing data collection tools Currently not measured. It is to be included in forthcoming facility survey tools. 434. Health system resilience indicators with metadata Indicator 14. Percentage of facilities with personnel that have received training with a focus on building health system resilience Indicator short name % facilities with staff having received health system resilience training Indicator name Percentage of facilities with personnel that have received training with a focus on building health system resilience Domain Health workforce Definition At the facility level: Percentage of facilities with at least one health worker that has completed training in health systems or service resilience within the last 5 years (for example, WHO’s integrated approach to building resilience, or other global actor’s resilience- specific training). The training should include (but is not limited to) the following contents: Conceptual understanding of health systems resilience and linkages to other key global health concepts (for example, universal health coverage, health security, essential public health functions); identifying key stakeholders and their roles in building health systems resilience; key actions required to build health systems resilience (for example, service continuity planning, simulation exercises, post- event reviews); monitoring and measuring of health systems resilience. At the health worker level: Percentage of health workers that have undergone training in health systems or service resilience meeting criteria defined above. Rationale To build and sustain health systems resilience, health workers require specific training and orientation on the concept and its operationalization. To conduct key health systems resilience building activities such as service continuity planning, simulation exercise and post-event reviews, facilities require at least one member of staff (preferably more) who has received dedicated training on building health systems resilience and can apply it in decision making, and ensuing transfer of the knowledge to others. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), hospitals, temporary health facilities, etc. Managing authority: public, private Subnational Urban/rural Numerator Number of facilities with personnel that have received training with a focus on building health systems resilience Denominator Total number of facilities assessed Recommended data source Facility survey Type (M&E domain) Process Additional reading and references OpenWHO.org. An integrated approach to building health systems resilience. Geneva: WHO (https://openwho.org/courses/health-service-resilience) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 44Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 4.3 Health information Indicator 15. Current state of delivery of the essential public health functions (EPHFs) has been ascertained Indicator short name Current state of essential public health functions delivery ascertained Indicator name Current state of delivery of the essential public health functions (EPHFs) has been ascertained Domain Health Information; Governance and leadership Definition There is evidence that the current state of EPHFs delivery has been comprehensively reviewed/assessed and documented, for example, within the last 5 years. The review of EPHFs should consider the areas: If the list of EPHFs has been prioritized based on national context including population health needs and health system risks • If EPHFs are considered and integrated into broader national health and allied sectors’ planning, policies, strategies or plans (for example, health workforce, water, sanitation and hygiene (WASH), health protection, health promotion, disease prevention, antimicrobial resistance (AMR)) • If the coordination mechanism(s) for the delivery of the essential public health functions has been reviewed • If there are mechanism(s) for monitoring and evaluation of essential public health functions at the national level. Mechanisms can be in the form of periodic qualitative review of EPHFs, a part of wider routine health system monitoring and evaluation framework, government audit of delivery of EPHFs, intersectoral etc. • If monitoring and evaluation for EPHFs is linked to follow-up planning and actions on findings from monitoring and evaluation WHO lists 12 EPHFs (please see below), as a minimum requirement for Member States to assure public health in a holistic, integrated, and sustainable manner. Countries may utilize this list, or the lists suggested by other global and regional entities, or develop their own list based on global consensus on EPHFs and reflective of their population health needs. • Public health surveillance and monitoring: Monitoring and surveillance of population health status, risk, protective and promotive factors, threats to health, and health system performance and service utilization • Public health emergency management: Managing public health emergencies for international and national health security • Public health stewardship: Establishing effective public health institutional structures, leadership, coordination, accountability, regulations and laws • Multisectoral planning, financing and management for public health: Supporting effective and efficient health systems and multisectoral planning, financing and management for public health • Health protection: Protecting populations against health threats, for example, environmental and occupational hazards and communicable and noncommunicable diseases, including mental health conditions, food insecurity, and chemical and radiation hazards • Disease prevention and early detection: Prevention and early detection of communicable and noncommunicable diseases, including mental health conditions and injuries • Health promotion: Promoting health and well-being as well as actions to address the wider determinants of health and inequity 454. Health system resilience indicators with metadata Indicator 15 (continued). Current state of delivery of the essential public health functions (EPHFs) has been ascertained Definition • Community engagement and social participation: Strengthening community engagement, participation and social mobilization for health and well-being • Public health workforce development: Developing and maintaining an adequate and competent public health workforce • Health service quality and equity: Improving appropriateness, quality and equity in provision of and access to health services • Public health research, evaluation and knowledge: Advancing public health research and knowledge development • Access to and utilization of health products, supplies, equipment and technologies: Promoting equitable access to and rational use of safe, effective and quality-assured health products, supplies, equipment and technologies Rationale Providing and maintaining EPHFs is a cornerstone for public health and resilient systems. EPHFs are acknowledged as a cost-effective and efficient means for advancing universal health coverage (UHC), other health-related targets of the United Nations Sustainable Development Goals (SDGs), and health security. The COVID-19 pandemic, climate-related threats, conflicts and other public health challenges have exposed weaknesses in the public health capacities necessary for resilient health systems. Routine, proactive public health activities have been chronically under-prioritized, in terms of investment and stakeholder action, compared with hospital-based health care and disease-specific interventions. In addition, there has been a fragmented approach to public health capacities building and a disproportionate focus on responding to crises – to the detriment of long-term measures including health promotion and disease prevention. These have left health systems and populations vulnerable to public health threats. Applying the EPHFs can enable a comprehensive and integrated operational approach to public health. Current state of EPHFs delivery and consideration of EPHFs in health and allied sectors being ascertained at national and subnational level indicates the government’s commitments and efforts to applying and strengthening EPHFs. Moreover, understanding the current state of EPHFs delivery can help identify strengths, critical gaps and key areas for improvement. Ascertaining the consideration of EPHFs in health and allied sectors at national and subnational level provides evidence that national and subnational governments applies an integrated approach to public health capacities strengthening for resilient health systems. Level National; subnational Disaggregation Subnational Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process 46Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 15 (continued). Current state of delivery of the essential public health functions (EPHFs) has been ascertained Additional reading and references Pan American Health Organization, Centers for Disease Control and Prevention, and Centro Latino Americano de Investigación en Sistemas de Salud. 2001. Public health in the Americas: Instrument for Performance Measurement of Essential Public Health Functions. Washington, D.C.: WHO, PAHO (https://iris.paho.org/handle/10665.2/42814) World Health Organization. 2018. Essential public health functions, health systems and health security: developing conceptual clarity and a WHO roadmap for action. Geneva: WHO (https://apps.who.int/iris/handle/10665/272597) World Health Organization. 2021. 21st century health challenges: can the essential public health functions make a difference?: discussion paper. Geneva: WHO (https://apps.who.int/iris/handle/10665/351510) World Health Organization. 2023. Application of the essential public health functions: an integrated and comprehensive approach to public health. Geneva: WHO. World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization Regional Office for Europe. 2015. Self-assessment tool for the evaluation of essential public health operations in the WHO European Region. Copenhagen: WHO Regional Office for Europe (https://apps.who.int/iris/ handle/10665/344398) World Health Organization Regional Office for the Easter Mediterranean. 2017. Assessment of essential public health functions in countries of the Eastern Mediterranean Region. Assessment tool. Cairo: WHO Regional Office for the Eastern Mediterranean (https://apps.who.int/iris/handle/10665/254383) Existing data collection tools Not at present 474. Health system resilience indicators with metadata Indicator 16. Mechanism in place to assess community trust Indicator short name Mechanism in place to assess community trust Indicator name Mechanism in place to assess community trust Domain Health information; Governance and leadership Definition There is a mechanism in place in the area to assess, track and monitor community trust through patient reported experiences and/or outcomes. Patient reported experiences or outcomes can be used as a proxy to assess community trust in the health system, essential health services and public health interventions. Additionally, public trust and perceptions of health services can be captured using community or population-based surveys or through community-based interventions. Rationale Positive patient experiences and health outcomes can foster trust between the health system and service providers, and the community. Trust is key to the success of public health interventions such as vaccination campaigns, adoption of healthy behaviours, risk communication, etc. Community trust contributes to the resilience of a health system by ensuring continued utilization of essential health services, reducing pressure on acute health services as needed, preventing more serious disease through uptake of health promotion and disease prevention interventions, and ensuring reliable evidence-based health information is effectively delivered to communities. Mechanisms that track and monitor community trust in the health system can be used to inform policy, planning and implementation further ensuring community engagement and participation in the design and delivery of public health functions and health services. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references Larson E, Sharma J, Bohren MA, Tunçalp Ö. When the patient is the expert: measuring patient experience and satisfaction with care. Bull World Health Organ. 2019 Aug 1;97(8):563-569. doi: 10.2471/BLT.18.225201 World Health Organization. 2021. Voice, agency, empowerment: handbook on social participation for universal health coverage. Geneva: WHO (https://www.who. int/publications/i/item/9789240027794) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/ publications/i/item/9789240051980) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/ publications/i/item/9789240051980) 48Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 17. Comprehensive public health surveillance and response system in place at all service delivery levels Indicator short name Comprehensive surveillance and response system Indicator name Comprehensive public health surveillance and response system in place at all service delivery levels Domain Health information; Service delivery Definition There is a comprehensive public health surveillance and response system with at least the following characteristics: • The system conducts systematic identification, collection, collation, analysis and interpretation of disease occurrence and public health events data, for the purpose of taking timely and robust action • Data are collected, analysed, interpreted, and reported in a consistent manner, for example, by the same focal point who normally submit routine report forms on health-related data A comprehensive public health surveillance and response system is based on an integrated approach to disease surveillance that aims to collect health data for multiple conditions using standardized tools. To ensure robust early warning and support prompt response, the system incorporates indicator-based and event- based surveillance as integral parts of an Early Warning Alert and Response (EWAR) system. Rationale Comprehensive public health surveillance and response makes surveillance, response, and laboratory data more usable and aids public health decision makers improve detection and response to the leading causes of illness, disability and death contributing to the resilience of the health system. Level National; subnational Disaggregation Service delivery levels: primary; secondary; tertiary; quaternary Numerator N/A Denominator N/A Recommended data source Qualitative assessment based on interview with key informant and/or desk review of country documents Electronic IHR States Parties Self-Assessment Annual Reporting Tool (eSPAR) Type (M&E domain) Input and structure Additional reading and references OpenWHO.org. Integrated Disease Surveillance and Response Course Series. Geneva: WHO (https://openwho.org/channels/idsr) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization Regional Office for Africa. 2010. Technical Guidelines for Integrated Disease Surveillance and Response in the African Region October 2010. Brazzaville: WHO Regional Office for Africa. (https://www.afro.who.int/publications/technical-guidelines-integrated-disease- surveillance-and-response-african-region-third) Existing data collection tools Not at present 494. Health system resilience indicators with metadata Indicator 18. Percentage of health facilities providing complete reports according to district and/or national requirements Indicator short name Completeness of reporting by facilities Indicator name Percentage of health facilities providing compete reports according to district and/ or national requirements Domain Health information; Health infrastructure Definition Percentage of health facilities that use information systems for capturing and reporting comprehensive patient and facility data and report this according to district and/or national requirements within the required timeframe This indicator is in Primary Health Care Measurement Framework and Indicators (2022 version) [indicator 34]. Rationale Routine health information systems are one of the key building blocks of efficient, nationally led, integrated and resilient health systems. Routine health information systems are systems that provide health data at regular intervals of less than a year to meet predictable health information requirements. They include paper records or electronic/digital health records as well as facility- and district-level health information management systems (for example, DHIS2) and regular surveillance and epidemiological data. A well-functioning health information system (HIS) has the following attributes: • Generation of individual-level, facility-based and population-based data from multiple sources: public health surveillance platforms, medical records, civil registration data, household surveys, censuses, health service coverage and health system input data (for example, human resources, health infrastructure and financing). • Capacity to detect, investigate, communicate, and contain events that threaten public health security at the place they occur, and as soon as they occur. • Ability to synthesize information and apply this knowledge. A good HIS improves both demand for and supply and use of data – in health systems and services, clinical and public health management, financing, planning, and implementation. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: public, private Urban/rural Subnational Service/Program: for example, immunization, maternal child health, noncommunicable diseases, etc. Numerator Number of health facilities that are a part of the routine health information system Denominator Total number of health facilities assessed 50Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 18 (continued). Percentage of health facilities providing complete reports according to district and/or national requirements Recommended data source Routine health information system Facility survey The denominator can be assessed through, for example, the list of registered facilities in authority’s database in a defined area; or through the total number of health facility surveyed in a catchment area. Type (M&E domain) Process Additional reading and references Hotchkiss DR, Diana ML, Foreit KG. How can routine health information systems improve health systems functioning in low- and middle-income countries? Assessing the evidence base. Adv Health Care Manag. 2012;12:25-58. doi: 10.1108/ s1474-8231(2012)0000012006 World Health Organization. WHO Toolkit for Routine Health Information Systems Data (https://www.who.int/data/data-collection-tools/health-service-data/toolkit- for-routine-health-information-system-data/modules) World Health Organization Regional Office for South-East Asia. 2017. UHC Technical Brief: Strengthening health information systems. New Delhi: WHO Regional Office for South-East Asia (https://apps.who.int/iris/handle/10665/259716) Existing data collection tools Routine health information system 514. Health system resilience indicators with metadata Indicator 19. Health system resilience is measured and monitored as part of routine health information system Indicator short name Health system resilience measured and monitored in routine health information system Indicator name Health system resilience is measured and monitored as part of routine health information system Domain Health information Definition Health system resilience is being measured and monitored as part of routine health information system and can be assessed against the following criteria: • A well-balanced set of indicators of health systems resilience should be identified, harmonized, monitored, and utilized for system-wide improvement within and outside emergency contexts in countries, at national, subnational, and service-delivery levels. All indicators should have well-defined baseline and targets; specify disaggregation including by age, sex, gender, and by other equity dimensions; include specifications on data collection methods, digital architecture required for reporting of key indicators • Data for identified health system resilience indicators are collected in routine health information systems, as appropriate • Measuring and monitoring health systems resilience includes data quality assurance mechanisms Rationale Systematic, timely and regular monitoring and evaluation using contextualized and integrated measurement approaches are essential for identifying areas for improvement, targeting interventions, and ensuring accountability to stakeholders when investing and building health systems resilience. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator N/A Denominator N/A Recommended data source Qualitative assessment based on interview with key informant and/or desk review of country documents Type (M&E domain) Process Additional reading and references World Health Organization. 2022. Health systems resilience toolkit: a WHO global public health good to support building and strengthening of sustainable health systems resilience in countries with various contexts. Geneva: WHO (https://www.who.int/publications/i/item/9789240048751) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 52Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 20. Percentage of facilities that have documented up to date risk profiles for potential shocks and stressors Indicator short name % facilities with risk profiles Indicator name Percentage of facilities that have documented up to date risk profiles for potential shocks and stressors Domain Health information Definition Percentage of facilitates that have an up to date (for example, once in the past five years) risk profile for potential shocks and stressors informed by assessment of risks and structural, non-structural, functionality and preparedness of health care facilities The risk profile of a facility informs all considerations in leading and managing its health and safety risks, and can be assessed against the following criteria. The risk profile includes: • the nature and extent of the threats to health services delivery, quality, utilization • the likelihood of an adverse event or effect occurring • the degree of disruption and costs associated with the different types of risk • the effectiveness of mitigation measures in place to manage risks Rationale Risk profiling for potential shocks and stressors leads to priority risks being identified and prioritized for action. It informs mitigation measures and enables health facilities to prepare for, adapt to and respond to risks and threats while maintaining core functionality thereby building the resilience of the overall health system. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of facilities with up-to-date risk profiles Denominator Total number of facilities assessed Recommended data source Facility survey Qualitative assessment based on interview with key informant and/or desk review of country documents Type (M&E domain) Input and structure Additional reading and references United Kingdom Health and Safety Executive (HSE). Managing risks and risk assessment at work. Merseyside: HSE (https://www.hse.gov.uk/simple-health- safety/risk/index.htm) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 534. Health system resilience indicators with metadata Indicator 21. Early-warning, alert and response system (EWARS) is established Indicator short name Early-warning system established Indicator name Early-warning, alert and response system (EWARS) is established. Domain Health information; Service delivery Definition Early warning systems or early warning, alert, and response systems (EWARS) are designed to improve disease outbreak detection in emergency settings such as in countries in conflict or following natural disaster. Whether a country has an effective early-warning, alert and response system can be assessed using the IHR State Party Self-Assessment Annual Report (SPAR) (2nd edition) indicator C5.1 Early warning surveillance function: • Level 1 - National guidelines and/or SOPs for surveillance are not available or under development • Level 2 - National guidelines and/or SOPs for surveillance have been developed but not implemented. The surveillance system is functioning but lacks systematic immediate reporting or weekly reporting of events and/or data • Level 3 - National guidelines and/or SOPs for surveillance have been developed and are being implemented at the national level and provide immediate and weekly reporting of events and/or data • Level 4 - National guidelines and/or SOPs for surveillance have been developed and are being implemented at the national and intermediate levels and provide immediate and weekly reporting of events and/or data • Level 5 - National guidelines and/or SOPs for surveillance have been developed and implemented at national, intermediate and local levels; and the system is exercised (as applicable), reviewed, evaluated and updated on a regular basis, with improvement at all levels in the country This indicator is included as one of the attributes of indicator 41 in Primary Health Care Measurement Framework and Indicators (2022 version). Rationale Early warning systems are often a part of surveillance systems. Early warning systems are simple and cost-effective ways to enable the health system to prepare for and respond to health threats. Indicator-based surveillance is the systematic (regular) collection, monitoring, analysis and interpretation of structured data, i.e., of indicators produced by several well-identified, mostly health-based, formal sources, such as when health care facilities (including primary care settings) regularly report the numbers of cases and deaths caused certain priority diseases that are predefined and mandated. Event-based surveillance is the organized collection, monitoring, assessment and interpretation of mainly unstructured ad hoc information regarding health events or risks which may represent an acute risk to human health. It is a functional component of the early warning and response system (such as media screening that is conducted in a systematized manner to identify events of public health interest). All surveillance data are systematically analysed for informed decision-making and dissemination. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A 54Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 21 (continued). Early-warning, alert and response system (EWARS) is established Denominator N/A Recommended data source Electronic IHR States Parties Self-Assessment Annual Reporting Tool (eSPAR) Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2018. Guidance document for the State Party self- assessment annual reporting tool -International Health Regulations (2005). Geneva: WHO (https://apps.who.int/iris/handle/10665/272438) World Health Organization. Early Warning, Alert and Response System (EWARS) (https://www.who.int/emergencies/surveillance/early-warning-alert-and- response-system-ewars) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools eSPAR (https://extranet.who.int/e-spar) 554. Health system resilience indicators with metadata Indicator 22. Existence of a mechanism for sharing of relevant public health information with other sectors Indicator short name Mechanism for multisectoral information sharing Indicator name Existence of a mechanism for sharing of relevant public health information with other sectors Domain Health information; Governance and leadership Definition There is evidence of a mechanism for sharing of relevant public health information with between sectors (for example, veterinary, environment, agriculture, port health authorities, transportation, education, finance, commence, internal affairs, private sector, etc.). Mechanism for sharing relevant public health information should have the following attributes: • There are information system structures for data sharing • Regulations and standards are in place for data sharing • Data sharing should involve all relevant key sectors This indicator is included as one of the attributes of indicator 1 in Primary Health Care Measurement Framework and Indicators (2022). Rationale Sharing of health information and data with actors within and outside the health sector is critical for mitigation, preparedness, response and recovery from public health events, shocks and stressors. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools Not at present 56Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 23. Vulnerability and risk analysis and mapping has been conducted at the subnational level Indicator short name Vulnerability and risk mapping conducted Indicator name Vulnerability and risk analysis and mapping has been conducted at the subnational level Domain Health information; Governance and leadership Definition There is evidence that a vulnerability and risk analysis and mapping, using the strategic tool for assessing risk (STAR), or equivalent has been conducted at the subnational level with reports disseminated to health facilities. Rationale Such mapping enables national and subnational government to rapidly conduct a strategic and evidence-based assessment of public health risks for planning and prioritization of health emergency preparedness and disaster risk management activities. Level Subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references World Health Organization. 2021. Strategic toolkit for assessing risks: a comprehensive toolkit for all-hazards health emergency risk assessment. Geneva: WHO (https://www.who.int/publications/i/item/9789240036086) Existing data collection tools Not at present 574. Health system resilience indicators with metadata 4.4 Access to medicines and other health products and technologies Indicator 24. National list of essential medicines is developed Indicator short name National list of essential medicines Indicator name National list of essential medicines is developed Domain Access to medicines and other health products and technologies Definition There is a national defined list of essential medicines. The list of essential medicines should consider the national demographic and disease profiles. Rationale Essential medicines are those that satisfy the priority health care needs of a population. They are selected with due regard to national contexts (for example, disease prevalence and public health relevance), evidence of efficacy and safety and comparative cost-effectiveness. They are intended to be available in functioning health systems at all times, in appropriate dosage forms, of assured quality and at prices individuals and health systems can afford Level National Disaggregation N/A Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2023. WHO Model List of Essential Medicines - 23rd list, 2023. Geneva: WHO (https://www.who.int/publications/i/item/WHO-MHP-HPS- EML-2023.02) Existing data collection tools Not at present 58Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 25. Regulatory mechanisms for medicines and other essential health products are established Indicator short name Regulatory mechanisms for essential health products Indicator name Regulatory mechanisms for medicines and other essential health products are established Domain Access to medicines and other health products and technologies Definition There are regulatory mechanisms for medicines and other essential health products (for example, vaccines, medical devices, in vitro diagnostics, protective equipment and vector-control tools, and assistive devices), measured against the following criteria, as applicable: • National regulatory authority • Marketing authorization • Licensing of manufacturers • Licensing of importers, exporters, wholesalers and distributors • Licensing pharmacies and retail outlets • Registration of pharmacy personnel • Post-marketing surveillance and controls • Control of drug promotion and advertising • Pharmacovigilance • Regulation of clinical trials • Regulatory inspections • Laboratory quality control • Control of narcotics, psychotropic substances and precursors This indicator is linked to indicator 30 in the Primary Health Care Measurement Framework and Indicators (2022). Rationale Health system resilience relies on access to health products including medicines, vaccines, medical devices, in vitro diagnostics, protective equipment and vector- control tools, and assistive devices. These must be of assured safety, efficacy/ performance and quality. In addition, they must be appropriate, available and affordable. Poor or inadequate regulation can lead to the prevalence of poor standard, counterfeit, harmful and ineffective drugs on national markets and in the international commerce. This can result in serious harm to the health of individual consumers and even to the health of a wider population. Therefore, countries must continuously strengthen key drug regulatory responsibilities to ensure the safety, quality and efficacy of drugs and the accuracy of product information. Level National; subnational Disaggregation N/A Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure 594. Health system resilience indicators with metadata Indicator 25 (continued). Regulatory mechanisms for medicines and other essential health products are established Additional reading and references World Health Organization. 2014. Good governance for medicines: model framework, updated version 2014. Geneva: WHO (https://apps.who.int/iris/ handle/10665/129495) World Health Organization. 2020. WHO Expert Committee on Specifications for Pharmaceutical Preparations: fifty-fourth report (WHO technical report series; no. 1025). Geneva: WHO (https://www.who.int/publications/i/item/978-92-4-000182-4) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2007. WHO Data Collection Tool for the Review of Drug Regulatory Systems. Geneva: WHO. 60Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 26. Percentage of facilities with prepositioned public health emergency-relevant health products Indicator short name % facilities with prepositioned essential supplies Indicator name Percentage of facilities with prepositioned public health emergency-relevant health products Domain Access to medicines and other health products and technologies Definition Percentage of facilities that have prepositioned public health emergency-relevant health products (for example, examination equipment, oxygen, consumable supplied, diagnostic imaging technology, medical equipment for treatment, medicines) that meet national or international requirements in accordance with facility types and national and local risk profiles. Please note: The list of essential supplies to be prepositioned should align with local needs and services being provided in the facility. Some reference list of essential emergency supplies that can be adapted to local contexts are: WHO Standards Health Emergency kits; WHO General essential emergency equipment list; and UNICEF Emergency supplies lists. Rationale During public health emergencies, there can be a surge in demand for emergency- relevant health products such as disease-specific medicines, prophylactic agents, oxygen supply, or personal and protective equipment. Prepositioning stock ensures that delivery of emergency case management and routine essential health services are not disrupted. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Local catchment areas Urban/rural Numerator Number of facilities that have prepositioned public health emergency-relevant health products that meet national or international requirements in accordance with facility types. Denominator Total number of facilities surveyed or in the catchment area. Recommended data source Facility-level reviews or surveys (for example, Site visit; storage logs) Type (M&E domain) Input and structure 614. Health system resilience indicators with metadata Indicator 26 (continued). Percentage of facilities with prepositioned public health emergency- relevant health products Additional reading and references World Health Organization. 2012. WHO generic essential emergency equipment list. Geneva: WHO (https://www.who.int/docs/default-source/integrated-health- services-(ihs)/csy/surgical-care/imeesc-toolkit/equipment-lists-and-needs- assessment/essential-emergency-equipment-list.pdf?sfvrsn=cb54324f_5) World Health Organization. Access to medicines and health products (https://www.who.int/our-work/access-to-medicines-and-health-products) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) WHO Regional Office for South-East Asia. Essential Medicines (https://www.who. int/southeastasia/health-topics/essential-medicines) Existing data collection tools Facility survey 62Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 27. Percentage of health facilities that have a core set of relevant essential medicines available and affordable on a sustainable basis Indicator short name Availability of essential medicines Indicator name Percentage of health facilities that have a core set of relevant essential medicines available and affordable on a sustainable basis Domain Access to medicines and other health products and technologies Definition Percentage of health facilities that have a core set of relevant essential medicines available and affordable on a sustainable basis The indicator is a multidimensional index reported as a proportion (%) of health facilities that have a defined core set of quality-assured medicines that are available and affordable relative to the total number of surveyed health facilities at national level. A medicine is available in a facility when it is found in this facility by the interviewer on the day of data collection, based on the national reference list or other reference list that is fit-for-purpose to the facility. Below is a reference list from the Primary Health Care Measurement Framework and Indicators (2022): Category Medicines Noncommunicable diseases Salbutamol; Beclomethasone (NCD) respiratory Gliclazide, Metformin, insulin regular [soluble] NCD Diabetes Any two of the following hypertensives: Amlodipine, Enalapril, Hydrochlorothiazide or Chlorthalidone, Bisoprolol NCD Cardiovascular Simvastatin, Acetylsalicylic acid (aspirin), Furosemide Pain and palliative care Morphine, paracetamol, ibuprofen for adults Central nervous system Fluoxetine; Phenytoin or Carbamazepine Anti-infective Gentamicin, Amoxicillin for adults, Ceftriaxone, Procaine benzylpenicillin or Benzathine benzylpenicillin Contraception - maternal child health (MCH) One of the following contraceptives: Ethinylestradiol + Levonorgestrel, Levonorgestrel (30 mcg cap/tab), Medroxyprogesterone acetate injection, progesterone- releasing implant (Etonogestrel or Levonorgestrel), Levonorgestrel (750 mcg or 1.5 mg tablet) MCH Oral rehydration salts, zinc sulphate, Oxytocin, magnesium sulphate, folic acid Anti-malarial One of the artemisinin-based combination therapies (ACT): Artemether + Lumefantrine, Artesunate + Amodiaquine, Artesunate + Mefloquine, Dihydroartemisinin + Piperaquine, Artesunate + Sulfadoxine + Pyrimethamine; Anti-malarial Artesunate Antiretroviral (ARV) One of combination ARV first-line treatment for HIV: Efavirenz + Emtricitabine + Tenofovir disoproxil fumarate, Efavirenz + Lamivudine + Tenofovir disoproxil fumarate Neonatal care Chlorohexidine Nutrition Ready-to-use therapeutic food (RUTF) Antituberculosis Isoniazid + pyrazinamide + rifampicin 634. Health system resilience indicators with metadata Indicator 27 (continued). Percentage of health facilities that have a core set of relevant essential medicines available and affordable on a sustainable basis Chronic kidney disease Erythropoietin Antiallergics and medicine used in anaphylaxis (optional) One of the following: Epinephrine injection, Dexamethasone injection Anti-fungal medicines (optional) Fluconazole, Nystatin Thyroid hormones (optional) Levothyroxine A medicine is affordable when no extra daily wages are needed for the lowest-paid unskilled government sector worker to purchase a monthly dose treatment of this medicine after fulfilling basic needs represented by the national poverty line. Affordability is measured as a ratio of 1) the sum of the national poverty line and the price per daily dose of treatment of the medicine, over 2) the lowest-paid government worker salary. This measures the number of extra daily wages needed to cover the cost of the medicines in the core set and that can vary between 0 and infinity. This indicator is in Primary Health Care Measurement Framework and Indicators (2022) [indicator 31]. Rationale Access to medicines is a composite multidimensional concept that is composed of the availability of medicines and the affordability of their prices. Information on these two dimensions has been collected and analysed since the 54th World Health Assembly in 2001, when Member States adopted the WHO Medicines Strategy (resolution WHA54.11). This resolution led to the launch of the joint project on Medicine Prices and Availability by WHO and the international non-governmental organization Health Action International (HAI/WHO), as well as a proposed HAI/WHO methodology for collecting data and measuring components of access to medicines. To this day, this methodology has been widely implemented to produce useful analyses of availability and affordability of medicines, however the two dimensions have been evaluated separately. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational Local catchment areas Urban/rural Numerator Number of facilities that have a core set of relevant essential medicines available and affordable Denominator Total number of surveyed facilities per country Recommended data source Facility survey Type (M&E domain) Input and structure 64Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 27 (continued). Percentage of health facilities that have a core set of relevant essential medicines available and affordable on a sustainable basis Additional reading and references World Health Organization. 2018. 2018 Global reference list of 100 core health indicators (plus health-related SDGs). Geneva: WHO (https://www.who.int/ publications/i/item/2018-global-reference-list-of-100-core-health-indicators-(-plus- health-related-sdgs)) World Health Organization. 2019. Model List of Essential Medicines, 21st List, 2019. Geneva: WHO (https://www.who.int/publications/i/item/WHOMVPEMPIAU2019.06) World Health Organization and United Nations Children’s Fund (UNICEF). 2020. Operational framework for primary health care: transforming vision into action. Geneva: WHO and UNICEF (https://www.who.int/publications/i/item/9789240017832) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) United Nations Department of Economic and Social Affairs Statistics Division. United Nations Sustainable Development Goals Indicators Metadata repository (https://unstats.un.org/sdgs/metadata/) Existing data collection tools While existing health facility survey tools such as the World Health Organization’s facility survey assessments, World Bank’s Service Delivery Indicators), and Demographic and Health Surveys (DHS) program’s Service Provision Assessment (SPA) measure availability of essential medicines, they are not all fully aligned to the SDG definition, and they also do not collect information on affordability. DHS Program. 2022. Service Provision Assessment, May 2022 (https://dhsprogram. com/publications/publication-spaq8-spa-questionnaires-and-manuals.cfm) World Bank. Service Delivery Indicators (https://www.sdindicators.org/) World Health Organization. 2015. Service Availability and Readiness Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/service-availability- and-readiness-assessment-(sara)) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) To note: WHO is currently revising its facility survey modules to incorporate/address specific elements on primary health care and health system resilience. 654. Health system resilience indicators with metadata Indicator 28. Percentage of facilities that have experienced interruption in water supply in the previous week Indicator short name % facilities experiencing water supply interruption Indicator name Percentage of facilities that have experienced interruption in water supply in the previous week Domain Access to medicines and other health products and technologies; Service Delivery Definition The percentage of health facilities that have experienced an interruption in running water supply or another source of safe water in the previous week Rationale There is a need to ascertain how frequently there are disruptions to water supply over a relatively short duration of time, i.e., in the last week (depending on how) as well as the proportion of facilities affected. Countries can decide how frequently the data is collected and adapt this indicator, for example, interruption in water supply in the previous month. This can enable identification and targeted interventions for service improvement. The availability of water to a facility is considered adequate when the facility has running water or another source of safe water 24 hours a day, 365 days a year. Water supply is a foundation for public health and the maintenance of safe, quality essential health services. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), hospitals, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of health facilities that have experienced interruption in water supply in the previous week Denominator Total number of health facilities assessed Recommended data source Facility survey Routine health information system Type (M&E domain) Input and structure Additional reading and references World Bank. Service Delivery Indicators (https://www.sdindicators.org/) World Health Organization. 2009. Vision 2030: the resilience of water supply and sanitation in the face of climate change: technical report. Geneva: WHO (https://www.who.int/publications/i/item/WHO-HSE-WSH-10.01) World Health Organization Regional Office for the Western Pacific. 2010. Safe hospitals in emergencies and disasters: structural, non-structural and functional indicators. Manila: WHO Regional Office for the Western Pacific (https://apps.who.int/iris/ handle/10665/207689) Existing data collection tools World Health Organization. 2012. African partnerships for patient safety: patient safety situational analysis (Long form). Geneva: WHO (https://apps.who.int/iris/ handle/10665/330052) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) World Health Organization Regional Office for Africa. 2003. Tools for Assessing the Operationality of District Health Systems. Brazzaville: WHO Regional Office for Africa. 66Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 29. Percentage of facilities that have experienced power outages in the previous week Indicator short name % facilities experiencing power outages Indicator name Percentage of facilities that have experienced power outages in the previous week Domain Access to medicines and other health products and technologies; Service delivery Definition The percentage of health facilities (for example, in a given geographical area or group of facilities) that have experienced electrical power outages in the previous week Rationale There is a need to ascertain how frequently there are disruptions to electrical power over a relatively short duration of time i.e., in the last week, as well as the proportion of facilities experiencing power outages by disaggregation. This can enable identification and targeted interventions for service improvement. The availability of electrical power to a facility is essential for lighting, use of information technology, power supply to medical equipment and devices, refrigeration of medicines and health products, temperature regulation, and maintenance of safe, quality essential health services. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), hospitals, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of health facilities that have experienced power outages in the previous week Denominator Total number of health facilities assessed Recommended data source Facility survey Routine health information system Type (M&E domain) Inputs and structure Additional reading and references World Bank. Service Delivery Indicators (https://www.sdindicators.org/) World Health Organization Regional Office for the Western Pacific. 2010. Safe hospitals in emergencies and disasters: structural, non-structural and functional indicators. Manila: WHO Regional Office for the Western Pacific (https://apps.who. int/iris/handle/10665/207689 Existing data collection tools World Health Organization. 2012. African partnerships for patient safety: patient safety situational analysis (Long form). Geneva: WHO (https://apps.who.int/iris/ handle/10665/330052) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) World Health Organization Regional Office for Africa. 2003. Tools for Assessing the Operationality of District Health Systems. Brazzaville: WHO Regional Office for Africa. 674. Health system resilience indicators with metadata Indicator 30. Percentage of facilities with availability of water, sanitation, and hygiene (WASH) amenities Indicator short name % facilities with basic WASH amenities Indicator name Percentage of facilities with availability of water, sanitation, and hygiene (WASH) amenities Domain Access to medicines and other health products and technologies; Service delivery; Governance and leadership Definition Percentage of facilities that have basic WASH amenities as defined by: • Water: available from an improved source, on premises • Sanitation: Improved facilities are usable, with at least one toilet for staff, one sex-separated with menstrual hygiene facilities and at least one accessible for those with limited mobility • Hand hygiene: functional hand hygiene facility (water with soap and/or ABHR) at points of care and within 5 meters of toilets • Health care waste: waste is safely segregated into three bins and sharps and infectious waste and treated and disposed of safely • Cleaning: basic protocols for cleaning are available and staff with cleaning responsibilities have received training This indicator is in Primary Health Care Measurement Framework and Indicators (2022) [indicator 23]. Rationale The availability of basic WASH amenities in health facilities is fundamental to delivering quality and resilient health services and adhering to infection prevention and control standards. Without basic WASH amenities, health services and systems are more prone to healthcare-acquired infections, staff infections, poorer health outcomes, larger scale outbreaks and disruptions to health services and systems. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: government, non-government; public, private Urban/rural Numerator Number of health facilities that meet basic WASH standards Denominator Total number of facilities examined Recommended data source Facility survey Type (M&E domain) Input and structure 68Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 30 (continued). Percentage of facilities with availability of water, sanitation, and hygiene (WASH) amenities Additional reading and references World Health Organization. 2020. Global progress report on WASH in health care facilities: Fundamentals first. Geneva: WHO (https://www.who.int/publications/i/ item/9789240017542) World Health Organization and United Nations Children’s Fund (UNICEF). 2018. Core questions and indicators for monitoring WASH in health care facilities in the Sustainable Development Goals. Geneva: WHO and UNICEF (https://www.who.int/ water_sanitation_ health/publications/monitoring-wash-in-health-care-facilities- aug-2018.pdf) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization and United Nations Children’s Fund (UNICEF). WHO/UNICEF Joint Monitoring Programme. Latest database: (http://washdata.org/data/healthcare). Existing data collection tools From existing health facility survey tools such as WHO’s Service Availability and Readiness Assessment (SARA) and Harmonized Health Facility Assessment (HHFA), and the Demographic and Health Surveys (DHS) Program’s Service Provision Assessment (SPA): DHS Program. Service Provision Assessment, May 2022 (https://dhsprogram.com/ publications/publication-spaq8-spa-questionnaires-and-manuals.cfm) World Bank. Service Delivery Indicators (https://www.sdindicators.org/) World Health Organization. 2015. Service Availability and Readiness Assessment (https://www.who.int/data/data-collection-tools/service-availability-and- readiness-assessment-(sara)) World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) 694. Health system resilience indicators with metadata 4.5 Health financing Indicator 31. Percentage of health facilities with user fees waiver mechanisms for public health emergency-related health services Indicator short name % facilities with user fees waiver mechanisms Indicator name Percentage of health facilities with user fees waiver mechanisms for public health emergency-related health services Domain Health financing; Governance and leadership Definition There is evidence of user fee waiver mechanisms for public health emergency- related health services such as consultations, treatment, investigations, and provision of medicines. Evidence can be policies or guidelines on user fee waiver in public health emergencies; patient experiences, financial reports, etc. Rationale User fees can be a barrier to the uptake of essential health services (both emergency case management and routine essential health services). Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Local catchment areas Urban/rural Numerator Number of facilities with user fee waiver mechanisms Denominator Total number of health facilities assessed Recommended data source Facility survey Type (M&E domain) Process Additional reading and references Inter-Agency Standing Committee Global Health Cluster. 2010. Removing user fees for PHC services during humanitarian crises. Geneva: WHO (https://www.who.int/ docs/default-source/documents/publications/removing-user-fees-for-primary- health-care-services-during-humanitarian-crises.pdf?sfvrsn=19631353_1) Existing data collection tools Not at present 70Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 32. Mechanism in place to ensure financial barriers do not impede diagnosis and treatment to a range of health threats Indicator short name Mechanism to address financial barriers Indicator name Mechanism in place to ensure financial barriers do not impede diagnosis and treatment to a range of health threats Domain Health financing; Governance and leadership Definition Availability of functioning system to ensure financial barriers do not impede the process of diagnosing and treating cases in different contexts (including infectious disease outbreaks, natural disasters, etc.). A functioning system should have the following attributes: consideration of the health needs of vulnerable populations public financing for provision of health services in need financial support to populations in accessing health services Examples of such mechanisms can include: essential services included in health benefits package; contingency funds available for emergencies should have dedicated budget for providing essential diagnosis and treatment services; etc. Rationale Financial barriers can impede patients from seeking essential health services thereby impeding timely diagnosis and treatment, and associated public health actions like notification of relevant authorities and others, risk communication, contact tracing, etc. This can lead to worsened health outcomes and greater pressure on secondary and tertiary levels of the health system. Level National, subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Local catchment areas Urban/rural Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references World Health Organization Regional Office for Europe. Financing (https://www. who.int/europe/emergencies/our-work-in-emergencies/health-systems-for- emergencies/financing) World Health Organization Regional Office for the Western Pacific. 2017. Removing financial barriers to accessing quality health services. Manila: WHO Regional Office for the Western Pacific (https://www.who.int/china/activities/removing-financial- barriers-to-accessing-quality-health-services) Existing data collection tools Not at present 714. Health system resilience indicators with metadata Indicator 33. Availability of dedicated budget line to support health services continuity in all contexts Indicator short name Dedicated budget line for service continuity Indicator name Availability of dedicated budget line to support health services continuity in all contexts Domain Health financing Definition As a key informant qualitative assessment at national and subnational administrative levels: There is evidence in the financial system of a dedicated budget line or space for maintenance and continuity of essential health services in all contexts (including in routine settings and during public health emergencies). The budget line for health services continuity can be within health service continuity plan, multi-hazards management plans, health sector development plan, health disaster and emergency management plans, etc. At the facility level: Percentage of facilities that have a dedicated budget line on provision and continuity of essential health services in all contexts (including in routine settings and during public health emergencies). The budget line can be a part of local health departments’ budgets. The facility level measurement of this indicator is an attribute in indicator 61 in the Primary Health Care Measurement Framework and Indicators (2022 version). Rationale During health systems shocks, stressors, and public health emergencies, the focus can be on the acute need such as emergency preparedness and response activities, however, there is a need to augment budgets for essential health services continuity and ensure resilience of the health system. Provision of essential health services in routine time is also an attribute of the everyday resilience of health systems. The budget line creates a space to pool financial resources to the maintenance and continuity of essential health services. Level National; subnational; facility Disaggregation No disaggregation for key informant qualitative assessments Facility surveys: Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first level hospitals, second level hospitals, temporary health facilities, etc.) Managing authority: public, private Sub-national Urban/rural Numerator No numerator for key informant qualitative assessments Facility level assessments: Number of facilities that have a dedicated budget line for supporting health services continuity. Denominator No denominator for key informant qualitative assessments Facility level assessments: Total number of facilities. Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Facility-level reviews or surveys 72Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 33 (continued). Availability of dedicated budget line to support health services continuity in all contexts Type (M&E domain) Input and structure Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, etc. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255-268. doi: 10.1093/heapol/czab089 Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) World Health Organization. 2020. Maintaining essential health services: operational guidance for the COVID-19 context: interim guidance, 1 June 2020. Geneva: WHO (https://apps.who.int/iris/handle/10665/332240) Existing data collection tools Not at present 734. Health system resilience indicators with metadata Indicator 34. Contingency funds available in the country for emergencies Indicator short name Contingency funds available Indicator name Contingency funds available in the country for emergencies Domain Health financing Definition Contingency funds available for health emergencies, measured against the following criteria: • A contingency fund exists at the subnational, national, regional, or international level, with which a national or subnational authority can coordinate the reception and distribution of funds to the health facility for responding to emergencies is in place at the national, intermediate, and local levels. (IHR SPAR C1.3). • A contingency fund having explicit coverage on maintenance of essential health services, including primary care services. • Financing can be executed and monitored in a timely and coordinated manner at all levels and for all relevant sectors, with an emergency contingency fund in place, for response to an acute public health emergency. (IHR JEE P1.3) This indicator is in Primary Health Care Measurement Framework and Indicators (2022) [indicator 18]. Rationale Contingency funds for emergencies that allow access funds to respond to emergencies, often in 24 hours or less, are a critical part of emergency response preparedness. Ability to quickly respond to emergencies can stave off unnecessary suffering and save lives. This emergency fund also serves to support continuity of services during an emergency when there are gaps. This indicator measure if the budget space for contingency and health services continuity is filled with financial resources. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2019. Health emergency and disaster risk management framework. Geneva: WHO (https://apps.who.int/iris/handle/10665/326106) (Caveat: it does not cover funding “maintenance of essential health services” aspects. World Health Organization. 2022. Joint External Evaluation Tool: International Health Regulations (2005) 3rd ed. Geneva: WHO (https://www.who.int/publications/i/ item/9789240051980) World Health Organization. State Party Annual Report for IHR (e-SPAR) (https://extranet.who.int/e-spar). World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2018. IHR (2005) State Party Self-Assessment Annual Reporting Tool (SPAR). Geneva: WHO (https://www.who.int/publications/i/item/WHO- WHE-CPI-2018-16) 74Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 35. Contingency or service continuity funds are accessible to the facility Indicator short name Contingency funds accessible to facilities Indicator name Contingency or service continuity funds are accessible to the facility Domain Health financing Definition At system level: Contingency funds or service continuity funds are accessible to facilities for response to an acute public health emergency, measured against the following criteria: • Existence of mechanism to execute emergency funds (for example, allocate or release contingency funds) to health facilities OR evidence showing that contingency funds were allocated to facilities to ensure continuity of health services in previous public health emergencies • Allocation of contingency funds is monitored in a timely and coordinated manner at all levels and for all relevant sectors, with an emergency contingency fund in place At facility level: Percentage of facilities that have access to contingency funding in the context of emergencies, supported by clear mechanisms (for example, defined triggers of contingency fund, stewardship of contingency fund) OR that were allocated with contingency funds to ensure continuity of health services in previous public health emergencies Rationale Contingency funds for emergencies that allow access funds to respond to emergencies, often in 24 hours or less, are a critical part of emergency response preparedness. Ability to quickly respond to emergencies can stave off unnecessary suffering and save lives. This emergency fund also serves to support continuity of services during an emergency when there are gaps. This indicator measures if the available contingency funding can be used, i.e. allocated and accessed by health facilities to implement activities in relation to continuity of essential health services. Level National; subnational; facility Disaggregation No disaggregation for key informant qualitative assessments Facility surveys: Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first level hospitals, second level hospitals, temporary health facilities, etc.) Managing authority: public, private Sub-national Urban/rural Numerator No numerator for key informant qualitative assessments Facility level assessments: Number of facilities that have access to contingency funding in context of emergencies Denominator No denominator for key informant qualitative assessments Facility level assessments: Total number of facilities Recommended data source At system level: qualitative assessment based on interview with key informant and/or desk review of facility documents At facility level: Facility survey Type (M&E domain) Process 754. Health system resilience indicators with metadata Indicator 35 (continued). Contingency or service continuity funds are accessible to the facility Additional reading and references World Health Organization. 2019. Health emergency and disaster risk management framework. Geneva: WHO (https://apps.who.int/iris/handle/10665/326106). (Caveat: it does not cover funding “maintenance of essential health services” aspects) World Health Organization. 2022. Joint External Evaluation Tool: International Health Regulations (2005) 3rd ed. Geneva: WHO (https://www.who.int/publications/i/ item/9789240051980) World Health Organization. State Party Annual Report for IHR (e-SPAR) (https:// extranet.who.int/e-spar). Existing data collection tools Not at present 76Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 36. Mapping of all health sector assets (resources) has been conducted in the last two years Indicator short name Resources mapping conducted Indicator name Mapping of all health sector assets (resources) has been conducted in the last two years Domain Health financing; Health Information Definition Health sector assets and resources mapping has been conducted against the following criteria: • existing assets (for example, infrastructure, supplies) and resources (for example, financial, human) are comprehensively identified and documented in the mapping process • mapping should be conducted relatively regularly (for example, at least within the last two years) Rationale Mapping of all health sector assets and resources creates an informational resource that can be used to enhance planning and delivery of health services and ensure health service continuity in all contexts. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references Institute for Healthcare Improvement (IHI). How Can Asset Mapping Improve Community Health? Boston: IHI (https://www.ihi.org/education/IHIOpenSchool/ resources/Pages/Activities/Bintz-AssetMapping.aspx) Existing data collection tools Not at present 774. Health system resilience indicators with metadata Indicator 37. Health financing arrangement includes public funding of public health services Indicator short name Public health services funded Indicator name Health financing arrangement includes public funding of public health services Domain Health financing Definition • There is evidence of health financing arrangements for a defined set of basic public health services, against the following criteria: • There is a set of explicitly defined package of essential health services for the entire population • The defined package of essential health services includes public health services within the scope of essential public health functions, often including health promotion, disease prevention, health protection, public health surveillance and monitoring • There are clear budget lines for basic public health services in health financing arrangement Public health services refer to services with the primary purpose of protecting and promoting the health and well-being of a defined population as a whole. The scope often includes health promotion, disease prevention and health protection services, as well as the legislative, regulatory, administrative, technical and behaviour-modifying interventions that impact on determinants of health. Rationale On many occasions, public health services are cost-effective and efficient to protect and improve population health. But routine, proactive public health services have been chronically under-prioritized, in terms of investment and stakeholder action compared with hospital-based health care and disease-specific interventions. A clear budget line and implementation of a budget indicates political commitment to public health services and supports delivery of public health services. There are different approaches to what constitutes expenditure on public health services. According to the 2011 edition of the System of Health Accounts, “prevention and public health services” are defined as “services designed to enhance the health status of the population as distinct from curative services, which repair health dysfunction. An upcoming WHO publication on operationalizing EPHFs will describe a compendium of public health services for reference. Level National; subnational Disaggregation Type of services: for example, prevention, promotion, treatment/rehabilitation, palliation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source National health account (NHA) Qualitative assessment based on interview with key informant or review of national budgets, financial report, policies, strategies, or plans National government audit Type (M&E domain) Input and structure 78Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 37 (continued). Health financing arrangement includes public funding of public health services Additional reading and references European Observatory on Health Systems and Policies, Rechel, Bernd, Jakubowski, Elke, McKee, Martin. et al. 2018. Organization and financing of public health services in Europe. Copenhagen: World Health Organization Regional Office for Europe (https://apps.who.int/iris/handle/10665/326254) Organisation for Economic Co-operation and Development. 2017. Expenditure on Prevention Activities under SHA 2011: Supplementary Guidance. Paris: OECD (https://www.oecd.org/els/health-systems/Expenditure-on-prevention-activities- under-SHA-2011_Supplementary-guidance.pdf) World Health Organization. 2016. Strengthening essential public health functions in support of the achievement of universal health coverage. Geneva: WHO (https:// apps.who.int/iris/handle/10665/252781) World Health Organization. 2018. Essential public health functions, health systems and health security: developing conceptual clarity and a WHO roadmap for action. Geneva: WHO (https://apps.who.int/iris/handle/10665/272597) World Health Organization. 2021. 21st century health challenges: can the essential public health functions make a difference?: discussion paper. Geneva: WHO (https://apps.who.int/iris/handle/10665/351510) World Health Organization. 2021. Measuring primary health care expenditure under SHA 2011: technical note, December 2021. Geneva: WHO (https://apps.who. int/iris/handle/10665/352307) World Health Organization. 2023. Operationalizing the essential public health functions: an integrated and comprehensive approach to public health. Geneva: WHO World Health Organization, Organisation for Economic Co-operation and Development & Statistical Office of the European Communities. 2017. A system of health accounts 2011: revised edition. Paris: OECD (https://doi. org/10.1787/9789264270985-en) Existing data collection tools Not at present 794. Health system resilience indicators with metadata 4.6 Governance and leadership Indicator 38. Service package for essential health services and public health functions is developed and meets criteria Indicator short name Service package meeting criteria Indicator name Service package for essential health services and public health functions is developed and meets criteria Domain Governance and leadership Definition Service package of essential health services (including primary care services) and public health functions is developed and meets following criteria: • Addresses comprehensive essential individual and population health services including: 1. Health protection 2. Prevention 3. Promotion 4. Management (diagnosis, treatment, rehabilitation, resuscitation) 5. Palliation 6. Includes key life course needs and disease programs 7. Foundations of care management of emergency syndromes and common presentations in primary care 8. Reproductive and sexual health, including pregnancy, childbirth, and family planning 9. Growth, development, disability and ageing 10. Communicable diseases 11. Noncommunicable diseases 12. Mental health, neurological and substance use disorders 13. Violence and injury • The package addresses disease burden and other national priorities including risk factor profiles and projections • The process for development of the service package involves a wide range of stakeholders • The package is based on an evaluation of existing resources • Is routinely revised as part of national planning processes • The package includes and designates key services related to emergency events for which the country is at risk This indicator is in Primary Health Care Measurement Framework and Indicators (2022) [indicator 45]. 80Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 38 (continued). Service package for essential health services and public health functions is developed and meets criteria Rationale For health systems to be resilient, they must be able to maintain essential health services in all contexts. A prerequisite for this capacity, is to have a nationally defined and prioritized set of essential health services, often referred to as an ‘essential package of health services’. For health systems to comprehensively meet population health needs, in routine contexts and during shocks, the defined package of essential health services should prioritize public as well as individual health services encompassing promotive, preventive, curative, rehabilitative and palliative services at all levels of service delivery. Applying the PHC approach supports this holistic and comprehensive approach to service delivery with equity and whole-of-society participation to meet population health needs throughout the life course. The exercise of specifying a core package is a value-laden process, looking to decision-makers and system stewards to establish a strategic policy position and equitable framework for protected access to quality individual and population health services when faced with competing priorities (such as short-term shocks or chronic stressors to the system). Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure 814. Health system resilience indicators with metadata Indicator 38 (continued). Service package for essential health services and public health functions is developed and meets criteria Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, Kelley E, Saikat S. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255- 268. doi: 10.1093/heapol/czab089 World Health Organization. 2014. Making fair choices on the path to universal health coverage. Final report of the WHO Consultative Group on Equity and Universal Health Coverage. Geneva: WHO (https://www.who.int/publications/i/ item/9789241507158) World Health Organization. 2018. Integrating health services: brief. Geneva: WHO (https://apps.who.int/iris/handle/10665/326459) World Health Organization. 2018. Primary health care: closing the gap between public health and primary care through integration. Geneva: WHO (https://apps. who.int/iris/handle/10665/326458) World Health Organization. UHC Compendium (https://www.who.int/universal- health-coverage/compendium) World Health Organization and the United Nations Children’s Fund (UNICEF). 2020. Operational framework for primary health care: transforming vision into action. Geneva: WHO and UNICEF (https://www.who.int/publications/i/ item/9789240017832) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization Regional Office for the Eastern Mediterranean. Universal health coverage (UHC) – priority benefits package. Existing data collection tools National authority websites, for example, website of ministries of health, can be checked for evidence of national service package. A qualitative service delivery assessment to measure this indicator is under development by WHO. 82Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 39. Availability of a protocol or guidance for prioritization of health services to be maintained following health systems shocks or stressors Indicator short name Availability of protocol for prioritization of services Indicator name Availability of a protocol or guidance for prioritization of health services to be maintained during health systems shocks or stressors Domain Governance and leadership Definition As a key informant qualitative assessment at national and subnational administrative levels: There is a protocol or guidance in place at facility level that supports the prioritization of those services to be maintained, including key elements, in all contexts, when it is not possible to maintain all routine health services, such as: During shocks or stresses to the health system including: • infectious disease outbreaks • natural disasters • chemical or radiological threats • sudden conflict Under routine pressures or challenging conditions as a demonstration of ‘everyday resilience’: • changing patient and community expectations • evolving disease profile and burden • altered governance structures and changes in policy directives • payment delays • health workforce issues • protracted conflict Assessment at the facility level: Percentage of facilities that have received the protocol or guidance to prioritize services in different contexts, including shocks, stresses to the health system, routine pressures and challenging conditions. An attribute in indicator 61 in the Primary Health Care Measurement Framework and Indicators (2022 version, at facility level measurement only). Rationale Health systems often have limited resources and face additional, unexpected, and/ or routine fluctuations in demand and pressures. The availability of a protocol or guidance which identifies those routinely provided essential health services (such as acute and/or critical interventions) that must be maintained when it is not possible to deliver all routine health services, can minimize the impact of disruptions and reduce avoidable morbidity and mortality. Level National; subnational; facility Disaggregation No disaggregation for key informant qualitative assessments Facility surveys: Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first level hospitals, second level hospitals, etc.) Managing authority: public, private Sub-national Urban/rural 834. Health system resilience indicators with metadata Indicator 39 (continued). Availability of a protocol or guidance for prioritization of health services to be maintained following health systems shocks or stressors Numerator No numerator for key informant qualitative assessments Facility level assessments: Number of facilities that have a protocol or guidance to prioritize services. Denominator No denominator for key informant qualitative assessments Facility level assessments: Total number of facilities Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Facility-level reviews or surveys Type (M&E domain) Process Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, Kelley E, Saikat S. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255- 268. doi: 10.1093/heapol/czab089 World Health Organization. 2020. Maintaining essential health services: operational guidance for the COVID-19 context: interim guidance, 1 June 2020. Geneva: WHO (https://apps.who.int/iris/handle/10665/332240) World Health Organization. 2021. Analysing and using routine data to monitor the effects of COVID-19 on essential health services: practical guide for national and subnational decision-makers: interim guidance, 14 January 2021. Geneva: WHO (https://apps.who.int/iris/handle/10665/338689) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 84Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 40. Availability of clinical protocols for case management of common and high priority infectious diseases and hazards Indicator short name Availability of priority disease and event case management protocols Indicator name Availability of clinical protocols for case management of common and high priority diseases and events at facility level Domain Governance and leadership Definition Clinical protocols for case management are structured plans that include clinical guidance and map the routes of care through the health system for individuals with specific clinical problems. Core conditions can be defined as common medical conditions for which preventive, diagnostic and treatment approaches are well established and for which a lack of treatment can cause significant harm to a patient. As a key informant qualitative assessment at national and subnational administrative levels: There is a clinical case management protocol or equivalent for that includes the following conditions (or include other priority diseases and events identified): A. FOUNDATIONS OF CARE 1. Diarrhoea 2. Difficulty in breathing 3. Fever 4. Sepsis B. GROWTH, DEVELOPMENT AND AGEING 5. Undernutrition C. REPRODUCTIVE AND SEXUAL HEALTH 6. Complications of pregnancy (maternal) D. COMMUNICABLE DISEASES 7. Lower respiratory infection 8. Malaria E. NONCOMMUNICABLE DISEASES 9. Asthma 10. Breast cancer 11. Chronic heart disease 12. Chronic kidney disease 13. Chronic obstructive pulmonary disease 14. Depression 15. Diabetes [mellitus] 16. Hearing impairment F. VIOLENCE AND INJURY 17. Serious injury AND specifies the following attributes: a. Key care elements are based on evidence and best practice b. Details on communication among the team members and with patients and families are included c. Roles and responsibilities, including sequencing of activities across the multidisciplinary care team, patients and their relatives are defined d. Guidance on monitoring and evaluation of variances and outcomes is included e. Health practitioner training in the use of care pathways 854. Health system resilience indicators with metadata Indicator 40 (continued). Availability of clinical protocols for case management of common and high priority infectious diseases and hazards Assessment at the facility level: Percentage of facilities that have received the case management protocol or equivalent for priority diseases and events This system-level measurement of the indicator is similar to indicator 50 in Primary Health Care Measurement Framework and Indicators (2022 version). The facility-level measurement of the indicator is included as one of the attributes of indicator 61 in Primary Health Care Measurement Framework and Indicators (2022 version). Rationale The resilience of health systems is tested everyday by common or high priority diseases and events, some of which can become public health emergencies. Protocols, guidelines and/or equivalent which consider system-wide issues (for example, workforce training and availability, essential health products and medicines, care pathways, multisectoral considerations such as discharge into community or social care) can enable provision of optimal care throughout the entirety of such diseases, illnesses, conditions, or events. They can also prevent small events including outbreaks of infectious diseases from becoming larger, more consequential events which disrupt essential health services and capacities and functional systems for health. The availability of protocols or equivalent can standardize clinical practice, reduce error, enhance quality of service delivery, reduce the risk of complications, and increase the chance of positive health outcomes thereby reducing pressure on the health system and, ultimately, the need for more costly services and interventions further down the line. Level National; subnational; facility Disaggregation No disaggregation for key informant qualitative assessments Facility surveys: Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first level hospitals, second level hospitals, etc.) Managing authority: public, private National/sub-National Urban/rural Numerator No numerator for key informant qualitative assessments Facility level assessments: Number of facilities that have a protocol or guidance to prioritize services Denominator No denominator for key informant qualitative assessments Facility level assessments: Total number of facilities Recommended data source Review of national/subnational policies, plans or guidance Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Facility-level reviews or surveys Type (M&E domain) Process 86Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 40 (continued). Availability of clinical protocols for case management of common and high priority infectious diseases and hazards Additional reading and references World Health Organization. 2022. Clinical management and infection prevention and control for monkeypox: interim rapid response guidance, 10 June 2022. Geneva: WHO (https://apps.who.int/iris/handle/10665/355798) World Health Organization. 2022. Clinical management of COVID-19: living guideline, 13 January 2023. Geneva: WHO (https://www.who.int/teams/health- care-readiness/covid-19) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) Of note, WHO is currently revising its facility survey modules. 874. Health system resilience indicators with metadata Indicator 41. National or subnational system in place for conducting simulation exercises that meets criteria Indicator short name System for conducting simulation exercises Indicator name National or subnational system in place for conducting simulation exercises that meets criteria Domain Governance and leadership Definition Nationally or sub-nationally, there is a system for conducting simulation exercises that includes the following elements: • Monitored annual requirement to conduct • Key stakeholders involved in/responsible for routine and emergency health services are included • Relevant stakeholders from outside the health sector are included • The simulation exercise focuses on testing the resilience of the health system including plans for essential health services continuity Rationale Simulation exercises are fully simulated, interactive exercises that test the capability of organizations or groups of organizations/health facilities with intra- and inter-sectoral participation to respond to simulated emergency, disaster, crisis, or routine situations. Regular participation in simulation exercises enable learning and improvement of services delivery as well as decision-making, planning, and other required system inputs which contributes to building resilience. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Type of simulation exercises Numerator N/A Denominator N/A Recommended data source Review of national and subnational policies, plans or guidance Qualitative or key informant survey or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure 88Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 41 (continued). National or subnational system in place for conducting simulation exercises that meets criteria Additional reading and references World Health Organization. 2021. Health Systems Resilience Simulation Exercises. Geneva: WHO (https://www.who.int/teams/integrated-health-services/health- service-resilience/integrated-health-system-strengthening/health-systems- resilience-simulation-exercises) World Health Organization Regional Office for the Western Pacific. 2006. Creating and tracking pandemic preparedness plans: a guide. Manila: WHO Regional Office for the Western Pacific World Health Organization Regional Office for the Western Pacific. 2006. Exercise development guide for validating influenza pandemic preparedness plans. Manila: WHO Regional Office for the Western Pacific World Health Organization Regional Office for South-East Asia. 2006. A guide for conducting table-top exercises for national influenza pandemic preparedness. New Delhi: WHO Regional Office for South-East Asia (https://iris.who.int/ handle/10665/204728) World Health Organization, Pan American Health Organization. 2011. Guidelines for developing emergency simulations and drills. Area on Emergency Preparedness and Disaster Relief. Washington, D.C: WHO, PAHO (https://www.nab.vu/sites/ default/files/documents/SimulationsGuide.pdf) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 894. Health system resilience indicators with metadata Indicator 42. Percentage of health facilities that are a part of a collaborative network or mutual aid arrangements for managing public health challenges and maintaining routine functions Indicator short name % facilities part of collaborative networks Indicator name Percentage of health facilities that are a part of a collaborative network or mutual aid arrangements for managing public health challenges and maintaining routine functions Domain Governance and leadership Definition The percentage of health facilities and other platforms/units of service delivery (for example mobile/outreach clinics, community health organizations, school health services, public health departments), in a given geographical area or within a network/group, that are a part of a collaborative network or mutual aid arrangement for managing public health challenges including emergencies, and maintaining routine functions. A collaborative network or mutual aid (voluntary exchange of services and resources for mutual benefit) arrangement can include an agreement, sometimes a legal document, that provides a formal framework for assistance between parties. Rationale Being a part of a collaborative network of health facilities and/or mutual aid arrangement can facilitate efficient transfer of resources (for example financial, human) and technical support, and even lending of support such as emergency responders across jurisdictional, geographical or sectoral boundaries. This is of mutual benefit to parties as such an agreement can facilitate the rendering of aid for another during disruptive public health events or even routine/everyday shocks and stressors to the health system. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of health facilities that are a part of a collaborative network or mutual aid arrangement Denominator Total number of health facilities assessed Recommended data source Facility survey Type (M&E domain) Input and structure Additional reading and references Stier DD, Goodman RA. Mutual aid agreements: essential legal tools for public health preparedness and response. American Journal of Public Health, 2007, 97 (Supplement 1): 62–68 World Health Organization. 2014. Hospital preparedness for epidemics. Geneva: WHO (https://www.who.int/publications/i/item/hospital-preparedness-for- epidemics) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 90Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 43. Existence of an all-hazard emergency preparedness and response plan (or equivalent) which defines the role of health services (including primary care) in emergency management and the maintenance of essential health services Indicator short name All-hazard emergency preparedness and response plan defines role of health services Indicator name Existence of an all-hazard emergency preparedness and response plan (or equivalent) which defines the role of health services (including primary care) in emergency management and the maintenance of essential health services Domain Governance and leadership Definition There is an all-hazard emergency preparedness and response plan (or equivalent) that specifies the routine role of health services (including primary care services) in emergency management (i.e., from prevention to preparedness, response and recovery) and the maintenance of essential health services Rationale Planning for and reducing the health risks and consequences of public health emergencies, including the maintenance of essential health services, requires the specification of roles and responsibilities of health service actors at all service delivery levels. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references World Health Organization. 2018. Primary health care and health emergencies. Geneva: WHO (https://apps.who.int/iris/ handle/10665/328105) World Health Organization. 2019. Health emergency and disaster risk management framework. Geneva: WHO (https:// apps.who.int/iris/handle/10665/326106) World Health Organization. State Party Annual Report for IHR (e-SPAR) (https:// extranet.who.int/e-spar) Existing data collection tools Not at present. Qualitative assessment tool with recommended scoring methodology under development by WHO. 914. Health system resilience indicators with metadata Indicator 44. Percentage of health facilities that have emergency management plans incorporating consideration of health services continuity Indicator short name % facilities with emergency management plans incorporating service continuity Indicator name Percentage of health facilities that have emergency management plans that incorporate considerations of health services continuity Domain Governance and leadership Definition Percentage of health facilities (for example, in a geographical area or within a network of health facilities) that have an emergency management and routine/essential health services continuity plan Rationale Service continuity planning is a process that identifies and prioritizes the critical functions of a health facility, evaluates the potential impact of various hazards, and identifies actions to ensure the continuity of critical functions (that is, essential health services) in all contexts (for example, in response to public health events/emergencies, shocks, or routine/everyday stressors. The inclusion of service continuity considerations within emergency management plans in health facilities contributes to minimizing disruptions and therefore, health system resilience. Such plans (or equivalent) should also consider context specific considerations, if applicable, such as special considerations for health in fragile, conflict and violence (FCV) settings. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational Urban/rural Numerator Number of health facilities that have emergency management and routine health services continuity plans Denominator Total number of health facilities assessed Recommended data source Facility survey Type (M&E domain) Process 92Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 44 (continued). Percentage of health facilities that have emergency management plans incorporating consideration of health services continuity Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, Kelley E, Saikat S. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255-268. doi: 10.1093/heapol/czab089 World Health Organization. 2021. Continuity of essential health services: facility assessment tool: a module from the suite of health service capacity assessments in the context of the COVID-19 pandemic: interim guidance, 12 May 2021. Geneva: WHO (https://apps.who.int/iris/handle/10665/341306) World Health Organization. 2021. Health service continuity planning for public health emergencies: a handbook for health facilities. Interim version for field testing. Geneva: WHO (https://www.who.int/publications/i/item/9789240033337) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data. 934. Health system resilience indicators with metadata Indicator 45. Percentage of facilities that have plans or service delivery models that take essential services to hard-to-reach populations Indicator short name % facilities with plans or service delivery models for hard-to-reach populations Indicator name Percentage of facilities that have plans for service delivery models to take essential services to hard-to-reach populations Domain Governance and leadership Definition The percentage of health facilities (for example, in a geographical area or within a network of health facilities) that have plans or service delivery models to take essential health services to hard-to-reach populations in their areas of responsibility Service delivery models are approaches to delivering health services (for example, centralized national health services, managed care, concierge services, self-directed services, telemedicine, community of care model). Populations that are hard-to- reach such as the disadvantaged and marginalized, migrants, refugees, displaced, geographically distant, homeless, criminal offenders and chronically mental ill have distinct health needs that may not be readily accessible at health facilities. Therefore, tailored plans and service delivery models may be needed to reach them and provide the necessary health services to meet their needs. Rationale Without specific plans to take essential health services to those that are hard-to- reach, significant proportions of the population are left without contact with the health system which can delay notification of infectious diseases and other health hazards. This can mean that preventable health issues are not picked up early and can cause greater strain on the health system in future, as well as reducing general population resilience through ill-health. Moreover, smaller outbreaks of infectious diseases can circulate un-reported or undetected in hard-to-reach populations increasing the chance of becoming larger scale emergencies. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, etc. Managing authority: public, private Subnational Urban/rural Numerator Number of facilities that have plans for service delivery models to take essential services to hard-to-reach populations Denominator Total number of facilities assessed Recommended data source Facility survey Type (M&E domain) Process Additional reading and references Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014 Mar 25;14:42. doi: 10.1186/1471-2288-14-42 Expanding universal health coverage among refugees and migrants: challenges and opportunities. East Mediterr Health J. 2021;27(4):427–428 https://doi. org/10.26719/2021.27.4.427 Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data. 94Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 46. Existence of a standard, guideline, specification or equivalent that defines infrastructural standards for health facilities Indicator short name Health facility infrastructure standards for health facility resilience Indicator name Existence of national or subnational standard, guideline, specification or equivalent that defines infrastructural standards for health facilities Domain Governance and leadership; Health infrastructure Definition Existence of standards or equivalent that mandate the development of health facility infrastructure such that the physical structures are appropriate to provide essential medical services as well as withstand threats such as natural disasters, security or other threats, in line with local or national risk assessments. The standard, guideline or specification is recommended to cover the following areas: • structural safety in the context of priority risks • adequate, safe and accessible infrastructure (beds, stations, rooms, etc.) including in the context of a surge • sustainable and safe management of water, sanitation, hygiene (WASH), and health care waste services • sustainable energy services Rationale The standard, guideline, specification or equivalent of health facilities can provide a useful and standardized mechanism for ensuring compliance with requirements for infrastructures for their resilience to routine/every day and unexpected shocks and stressors. Understanding of the safety and functionality of facilities in each geographical area or other disaggregation can be useful for policy, planning and designing service and system improvements to build resilience. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2020. WHO guidance for climate-resilient and environmentally sustainable health care facilities. Geneva: WHO (https://www.who. int/publications/i/item/climate-resilient-and-environmentally-sustainable-health- care-facilities) World Health Organization. 2022. Health systems resilience toolkit: a WHO global public health good to support building and strengthening of sustainable health systems resilience in countries with various contexts. Geneva: WHO (https://www. who.int/publications/i/item/9789240048751) World Health Organization, Pan American Health Organization. 2019. Hospital Safety Index. Guide for Evaluators. Second Edition. Washington, D.C.: WHO, PAHO (https:// iris.paho.org/handle/10665.2/51448) Existing data collection tools World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) 954. Health system resilience indicators with metadata Indicator 47. Percentage of health facilities that meet standards for infrastructure Indicator short name % facilities that meet standards for infrastructure Indicator name Percentage of health facilities that meet standards for infrastructure Domain Governance and leadership; Access to medicines and other health products and technologies Definition The percentage of health facilities that meet national or subnational standard, guideline, specification or equivalent to guide the development of infrastructure such that the physical structures are appropriate to provide essential medical services as well as withstand threats such as natural disasters, security or other threats, in line with local or national risk assessments. The facilities should meet standards for infrastructure in the following areas: • structural safety in the context of priority risks • adequate, safe and accessible infrastructure (beds, stations, rooms, etc.) including in the context of a surge • sustainable and safe management of water, sanitation, hygiene (WASH), and health care waste services • sustainable energy services • appropriate information technologies in place Rationale The standard, guideline, specification or equivalent of health facilities can provide a useful and standardized mechanism for ensuring compliance with requirements for their resilience to routine/every day and unexpected shocks and stressors. These include their suitability for essential health service continuity, safety, WASH, utilities, conducting simulation exercises and learning activities, and space for adequate workforce, considering their risk profile. Understanding of the safety and functionality of facilities in each geographical area or other disaggregation can be useful for policy, planning and designing service and system improvements to build resilience. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: public, private; government, non-government Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of facilities that meet standards for health facility infrastructure Denominator Total number of facilities examined / sampled Recommended data source Facility survey Type (M&E domain) Input and structure 96Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 47 (continued). Percentage of health facilities that meet standards for infrastructure Additional reading and references World Health Organization. 2020. WHO guidance for climate-resilient and environmentally sustainable health care facilities. Geneva: WHO (https://www.who. int/publications/i/item/climate-resilient-and-environmentally-sustainable-health- care-facilities) World Health Organization. 2022. Health systems resilience toolkit: a WHO global public health good to support building and strengthening of sustainable health systems resilience in countries with various contexts. Geneva: WHO (https://www.who. int/publications/i/item/9789240048751) Existing data collection tools World Health Organization. 2023. Harmonized Health Facility Assessment. Geneva: WHO (https://www.who.int/data/data-collection-tools/harmonized-health-facility- assessment/introduction) It can be used to capture some of the data. 974. Health system resilience indicators with metadata Indicator 48. A national coordination mechanism for multistakeholder participation and community engagement with a focus on public health exists Indicator short name Mechanism for multistakeholder participation and community engagement Indicator name A national coordination mechanism for multistakeholder participation and community engagement with a focus on public health exists Domain Governance and leadership; Community engagement Definition A national coordination mechanism exists meeting the following criteria: • Responsible for coordinating, monitoring and implementing health (for example, EPHF, PHC and/or UHC-related strategies and policies within the national health sector policy, strategies and plans) • Engagement and participation include a broad range of stakeholders, including: a. Community groups, including vulnerable, marginalized and excluded popula- tions (for example, ethnic minorities, women, the elderly, etc.) b. Members of parliamentary health committee c. Health worker associations, patient groups d. Civil society organizations and advocacy groups e. Health insurance bodies f. Provider organizations/associations g. Private sector h. Academia and research institutes i. UN agencies and other international organizations operating within the national context • The coordination mechanism has accountability for the range of health activities defined by national health policies and plans • The coordination mechanism/authority has adequate budget and sufficient staff • The mandate includes the public sector as well as oversight and regulation of the private sector where feasible This indicator is in Primary Health Care Measurement Framework and Indicators (2022) [indicator 7]. Rationale A key role of the ministry of health is to plan, initiate, coordinate, and oversee strategies, policies and plans, where relevant, through health sector coordination mechanisms. Policymakers must thus lead the process, ensure broad and meaningful stakeholder participation and engagement including with the communities they serve, ensure that the priorities that are set reflect stakeholder input in a balanced way, and be held accountable for the results. The process must be transparent, with clear roles and responsibilities, especially when it comes to evaluating and discussing evidence from different viewpoints. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process 98Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 48 (continued). A national coordination mechanism for multistakeholder participation and community engagement with a focus on public health exists Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) Primary Health Care Performance Initiative (PHCPI). Primary health care progression model (https://improvingphc.org/primary-health-care-progression-model) World Health Organization. 2021.Voice, agency, empowerment: handbook on social participation for universal health coverage. Geneva: WHO (https://www.who.int/ publications/i/item/9789240027794) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools World Health Organization. 2018. IHR (2005): State Party self-assessment annual reporting tool, 1st ed. Geneva: WHO (https://www.who.int/publications/i/item/WHO- WHE-CPI-2018-16) 994. Health system resilience indicators with metadata Indicator 49. Mechanism to ensure community voices informing planning and organization of services at the local level Indicator short name Mechanism to ensure community engagement in service planning and organization Indicator name Mechanism to ensure community voices informing planning and organization of services at the local level Domain Governance and leadership; Community engagement Definition There is a mechanism to ensure the planning and organization of services is informed by the voices of the population, communities, and civil society, at the local level (district or local health systems and facilities). Such mechanism(s) should involve: • recognizing that communities can take on roles and responsibilities in planning and organization • utilizing the role of the community effectively by communicating needs upwards to policymakers as well as downwards by gathering and coordinating community voices • enhancing technical knowledge and skills of decision-makers, managers, and health workers to fully engage in discussions and engagement with the community • engaging communities in processes such as needs assessment, community development, planning, design, development, delivery, and evaluation. Such mechanism(s) include but not limited to the following activities and demonstrates involvement of communities including vulnerable groups in the planning process: • Community health needs and asset assessment • Participatory processes for priority setting • Patient and relatives’ surveys • Training of patient advocates • Membership of community representatives in advisory boards at the local level (for example, council boards) or in supervisory boards of facilities This indicator in linked to indicator 57 in Primary Health Care Measurement Framework and Indicators (2022). Rationale Community engagement is the inclusion of local health system users and community members in all aspects of health planning, provision, and governance. It is a central component of ensuring that the services delivered are tailored to population needs, priorities and values, which can be achieved through the involvement of communities in the design, financing, governance, and implementation of PHC. To ensure that the needs of all community members are met, it is important that community engagement efforts include representation from diverse members of the community. This may require multiple mediums for engagement, to best capture the needs and opinions of traditionally underrepresented community members. Ensuring that community voices are systematically used to inform the planning and organization of health services leads to increased trust in services and providers, more aligned decision making and addressing of the real needs of the community, enhanced community satisfaction, trust, and patient reported outcomes in relation to health services, greater chance of community acceptance and compliance to public health advice and interventions, and more politically robust planning, policymaking and implementation. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. Numerator N/A 100Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 49 (continued). Mechanism to ensure community voices informing planning and organization of services at the local level Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/handle/10665/250221) Primary Health Care Performance Initiative (PHCPI). 2019. Primary Health Care Progression Model Assessment Tool (https://improvingphc.org/sites/default/files/ PHC-Progression%20Model%202019-04-04_FINAL.pdf ) World Health Organization. 2017. WHO community engagement framework for quality, people-centred and resilient health services. Geneva: WHO (https://apps.who.int/iris/ handle/10665/259280) World Health Organization. 2020. Community Engagement: A health promotion guide for universal health coverage in the hands of the people. Geneva: WHO (https://www. who.int/publications/i/item/9789240010529) World Health Organization. 2021. Voice, agency, empowerment: handbook on social participation for universal health coverage. Geneva: WHO (https://www.who.int/ publications/i/item/9789240027794) World Health Organization Regional Office for Europe. 2019. Indicator passports: WHO European Primary Health Care, Impact, Performance and Capacity Tool (PHC- IMPACT): version 1. Copenhagen: WHO Regional Office for Europe (https://iris.who.int/ handle/10665/346478) Existing data collection tools World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/ item/9789240051980) A qualitative assessment tool with recommended scoring methodology is under development by WHO and will be forthcoming. 1014. Health system resilience indicators with metadata Indicator 50. Percentage of facilities that have standard operating procedures (SOPs) for ensuring essential supplies Indicator short name % facilities with SOPs for ensuring essential supplies Indicator name Percentage of facilities that have standard operating procedures (SOPs) for ensuring essential supplies Domain Governance and leadership; Access to essential health products Definition Facility has standard operating procedures for ensuring essential supplies such as medicines, personal protective equipment (PPE), oxygen, bed nets are available during a shock event or surge. Rationale The availability of essential supplies is critical for ensuring maintenance of essential health services including emergency case management. Standard operating procedures (SOPs) reduce errors and increase efficiency for intended outcomes such as ensuring essential supplies (for example, medicines, personal protective equipment, oxygen, bed nets) are available during a shock event or surge in need. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), first-level hospitals, second-level hospitals, specialty hospitals, long-term care facilities, continuing care facilities, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Local catchment areas Urban/rural Numerator Number of health facilities that have SOPs for ensuring essential supplies Denominator Total number of health facilities assessed Recommended data source Facility survey Type (M&E domain) Input and structure Additional reading and references World Health Organization. Access to medicines and health products (https://www.who.int/our-work/access-to-medicines-and-health-products) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 102Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 51. Percentage of facilities that have standard operating procedures (SOPs) to enable health facility staff to repurpose resources Indicator short name % facilities with SOPs for repurposing resources Indicator name Percentage of facilities that have standard operating procedures (SOPs) to enable health facility staff to repurpose resources Domain Governance and leadership; Access to medicines and supplies Definition Percentage of facilities that have SOPs to enable health facility staff to repurpose resources without disrupting essential health services, in response to evolving population health needs (for example, infectious disease outbreaks, natural disasters, or everyday stressors such as staff absences, temporary budgetary issues, etc), and include SOPs for: • repurposing of infrastructure (for example, hospital beds) • staff (for example, redeployment to areas of greater need) • medical supplies (for example, redirecting oxygen supplies for acute, high- dependency and intensive care) SOPs represent the translation of policies, guidelines, standards, etc into practice at the service delivery level. SOPs reduce errors and increase efficiency for intended outcomes. Rationale Ensuring SOPs to enable health facility staff to repurpose resources can enhance the ability of the system to better prepare for, adapt to, respond to, and recover from evolving health needs, shocks, and stressors, while maintaining core functionality. Level Facility Disaggregation Facility type (as relevant to context): including primary care facilities (for example, general practices, health centres, community health posts), hospitals, temporary health facilities, etc. Managing authority: public, private Subnational (as relevant to context): region, state, province, canton, municipality, etc. Urban/rural Numerator Number of facilities that have standard operating procedures to enable staff to repurpose resources Denominator Total number of facilities assessed Recommended data source Facility survey Type (M&E domain) Input and structures Additional reading and references World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://www.who.int/publications/i/item/WHO-UHL- PHC-SP-2021.01) Existing data collection tools Not at present but existing routine health information systems or health facility assessments and tools could be adapted to incorporate collection of this data 1034. Health system resilience indicators with metadata Indicator 52. Mechanisms in place to support the systematic capture and translation of lessons identified from public health shocks, incidents and events Indicator short name Institutionalizing learning from public health events Indicator name Mechanisms in place to support the systematic capture and translation of lessons identified from public health shocks, incidents and events Domain Governance and leadership; Health information Definition There is evidence of mechanisms or processes to support the systematic capture and translation of lessons identified from public health shocks, incidents and events, to inform decision making, planning and policymaking, resource allocation and health systems performance improvement. Examples include: • Systematic inclusion of intra- and after-action review (IAR/AAR) findings and recommendations within health sector and system planning process • Integration and alignment between National Action Plans for Health Security and health sector development planning process Rationale A key capacity of resilient health systems is the ability to apply lessons learnt from past and ongoing experiences to adapt, transform, and improve. Lessons are not automatically translated into improvements even when systematic mechanisms to capture those lessons (for example, intra- and after-action reviews) are in place. In order to institutionalize learning, lessons must be systematically integrated into existing processes. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references McDarby G, Seifeldin R, Zhang Y, Mustafa S, Petrova M, Schmets G, Porignon D, Dalil S and Saikat S (2023) A synthesis of concepts of resilience to inform operationalization of health systems resilience in recovery from disruptive public health events including COVID-19. Front. Public Health. 11:1105537. doi: 10.3389/ fpubh.2023.1105537 Existing data collection tools Not at present 104Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 53. Existence of health system resilience function within emergency management structures at all levels Indicator short name Health system resilience as a function in emergency management structures Indicator name Existence of health system resilience function within emergency management structures at all levels Domain Governance and leadership Definition There is evidence that a health system resilience function, for example, continuity of essential health services during emergencies, and support to health system recovery and strengthening based on lessons learnt form emergencies, exists within emergency management structures at all administrative levels. This includes: • Health system resilience is identified within the terms of reference of emergency structures at all levels (i.e., national, subnational, local) • Focal point for health system resilience is identified in the organograms of emergency management structure • Activities include a focus on health system resilience involving relevant stakeholders at system and service delivery levels Rationale Effective emergency management with maintenance of essential health services requires intra- and inter-sectoral coordination and participation with a focus on resilience of the health system. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, guidance or terms of reference of emergency structures, etc. Type (M&E domain) Inputs and structure Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, Kelley E, Saikat S. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255- 268. doi: 10.1093/heapol/czab089 World Health Organization. 2017. Emergency response framework (ERF), 2nd edition. Geneva: WHO (https://www.who.int/publications/i/item/9789241512299) World Health Organization. 2020. Maintaining essential health services: operational guidance for the COVID-19 context: interim guidance, 1 June 2020. Geneva: WHO (https://apps.who.int/iris/handle/10665/332240) World Health Organization. 2020. Multisectoral preparedness coordination framework: Best practices, case studies and key elements of advancing multisectoral coordination for health emergency preparedness and health security. Geneva: WHO (https://www.who.int/publications/i/item/9789240006232) Existing data collection tools Not at present 1054. Health system resilience indicators with metadata Indicator 54. National emergency policies and strategies define the role of health services in emergency preparedness and response, and recovery Indicator short name Emergency policy defines role of health services Indicator name National emergency policy and strategy (or equivalent) defines the role of health services in emergency preparedness and response, and recovery Domain Governance and leadership Definition There is evidence that national emergency policies and strategies define the roles of health services in emergency preparedness and response and recovery, against the following criteria: • Describing the roles and responsibilities of health services at all levels • Describing the roles and responsibilities of health services comprehensively in terms of health emergency management cycle, i.e., addresses prevention, preparedness, response and recovery measures • Describing roles in relation to (but not limited to): a. designating focal points in health services providers health emergency and disaster risk management, or incorporating health services in emergency management structures b. simulation exercises c. continuity and the maintenance of quality essential health services during response d. safe restoration of services and addressing the backlog of health care needs in the recovery phase e. after action review or intra-action review f. ensuring equity and addressing the needs of vulnerable populations and communities National emergency policies and strategies can include health emergency and disaster risk management strategies, national pandemic preparedness plans, etc. This indicator is included as one of the attributes of indicator 5 in the Primary Health Care Measurement Framework and Indicators (2022). Rationale Health services functions play a key role in emergency preparedness and response and require formally defining roles and responsibilities to ensure effective emergency management in tandem with maintaining core functions of the broader health system. National emergency policy defining the roles of health services in emergency management supports the integration between efforts for health security and health systems strengthening for resilience. Level National Disaggregation N/A Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process 106Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 54 (continued). National emergency policies and strategies define the role of health services in emergency preparedness and response, and recovery Additional reading and references Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako-Egbe L, McDarby G, Kelley E, Saikat S. COVID-19 Preparedness and Response Plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022 Feb 8;37(2):255- 268. doi: 10.1093/heapol/czab089 World Health Organization. 2020. Maintaining essential health services: operational guidance for the COVID-19 context: interim guidance, 1 June 2020. Geneva: WHO (https://apps.who.int/iris/handle/10665/332240) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools Not at present 1074. Health system resilience indicators with metadata Indicator 55. National health sector policies and strategies define the roles of health services at all levels for public health emergencies Indicator short name Health sector policies define roles of health services for emergencies Indicator name National health sector/health system policies and strategies (or equivalent) define the roles of health services at all levels for public health emergencies Domain Governance and leadership Definition There is evidence of national health sector policies and strategies (or equivalent) which define the role of health services (primary, secondary, tertiary care) during public health emergencies, for example, roles in relation to providing essential health services including public health services on disease prevention, health promotion, health protection, public health surveillance and monitoring, etc. Rationale Specifying the roles of health services for public health emergency management in national health sector plans can support the integration of health security and health systems strengthening for resilience. Primary care is often the first point of contact between communities and the health system and as such play a key role in emergency management activities such as surveillance and disease notification, testing, contact tracing, vaccine delivery, etc. Level National Disaggregation N/A Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) World Health Organization. 2018. Primary health care and health emergencies. Geneva: WHO (https://iris.who.int/handle/10665/328105) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) Existing data collection tools Not at present 108Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 56. Annual operational health sector plan includes emergency preparedness activities Indicator short name Health sector plan includes preparedness activities Indicator name Annual operational health sector plan includes emergency preparedness activities Domain Governance and leadership Definition There is evidence of emergency preparedness activities in the annual operational health sector plan. Emergency preparedness activities should include activities in relation to: • Simulation exercises • After-action reviews or intra-action reviews • Training on health systems resilience or equivalent training covering key conceptual and operational aspects on health systems resilience The evidence should be in the form of activities specified in written plans. Rationale National annual operational health sector plans set the operational scope of health sector activities and to ensure health systems are resilient there is a need to incorporate emergency preparedness and other health systems activities in tandem. This would allow mainstreaming of health system resilience considerations in routine health system functions including at service delivery levels. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) Existing data collection tools Not at present 1094. Health system resilience indicators with metadata Indicator 57. Availability of a designated entity or structure for health system resilience function Indicator short name Designated entity or structure for health system resilience Indicator name Availability of a designated entity or structure for health system resilience function Domain Governance and leadership Definition There is evidence of the availability of a designated authority or entity mandated with responsibility for coordination and implementation of health system resilience functions in the government. Evidence is usually demonstrated through clear terms of reference or mandates that describe roles of coordination and oversight of health system resilience in the country. The designated authority can be in the form of a unit in ministry, a technical group, etc. as long as coordination and oversight of health system resilience function is presented in the terms of reference or list of mandates. Rationale A designated responsible authority or entity for health systems functions ensures intended activities and outcomes are pursued as well as accountability for implementation. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/handle/10665/346515) Existing data collection tools Not at present 110Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 58. Availability of national guideline on equity and ethics for delivery of routine and emergency-related health services in the context of public health emergencies Indicator short name Guideline on equity and ethics for service delivery Indicator name Availability of national guideline on equity and ethics for delivery of routine and emergency related health services in the context of public health emergencies Domain Governance and leadership Definition There is national guideline(s) on equity and ethics for delivery of routine and emergency-related health services in the context of public health emergencies. The national guideline should have the following characteristics: • It should provide guidance on equity and ethics for providing both routine essential health services and emergency-related health services • It should have specific consideration for meeting the needs of vulnerable populations • It should be available and disseminated to health facilities It could be in the format of a part of other documents. It does not have to be standalone guideline. Rationale During emergencies, vulnerable populations such as women, children, elderly, migrants, etc. may be disproportionately affected by the impacts and as such there is a need for guidelines on the equitable and ethical delivery of essential health services (both emergency case management and routine). Level National Disaggregation N/A Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references ICRC, IFRC, IOM, NRC, UNICEF, UN-HABITAT, UNHCR, and WHO in consultation with IASC members. 2020. Inter-Agency Standing Committee (IASC). Public health and social measures for COVID-19 preparedness and response in low capacity and humanitarian settings (https://www.who.int/publications/m/item/public-health- and-social-measures-for-covid-19-preparedness-and-response-in-low-capacity- and-humanitarian-settings) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) Existing data collection tools Not at present 1114. Health system resilience indicators with metadata Indicator 59. Existence of a national entity or structure that is responsible for the coordination of the essential public health functions (EPHFs) in an integrated manner Indicator short name Institutional capacity for EPHF coordination Indicator name Existence of a national entity or structure that is responsible for the coordination of the essential public health functions (EPHFs) in an integrated manner Domain Governance and leadership; service delivery Definition There is a dedicated national entity or structure (for example, standalone national public health institute, semi-autonomous institution under a national health authority, department within the MoH, network of agencies with the responsibility to carry out public health functions collectively, etc.) with a clear mandate for coordinating the planning and delivery of essential public health functions in the country. This entity or structure has the characteristics: • It is a public institution operating as part of the government or with the concurrence of the government. • Coordination of all or most of the essential public health functions at the national level is clearly defined in its mandate or terms of reference • It is empowered by the minister or the parliament to coordinate among different agencies in the planning and delivery of the essential public health functions. Rationale The COVID-19 pandemic, climate-related threats, conflicts and other public health challenges have exposed weaknesses in the public health capacities necessary for resilient health systems. Applying the EPHFs is a holistic, integrated operational approach to public health. It is important to have institutions such as NPHIs that are responsible for leading, and able to coordinate the planning, delivery and monitoring and evaluation of EPHFs reflective of the national context. Without dedicated responsible entity(ies) these public health functions may not be carried out adequately or in an integrated, coordinated and holistic manner. Furthermore, NPHIs provide independent scientific evidence to inform policymaking in a national context as well as provide visibility and prominence to public health. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure 112Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 59 (continued). Existence of a national entity or structure that is responsible for the coordination of the essential public health functions (EPHFs) in an integrated manner Additional reading and references International Association of National Public Health Institutes (IANPHI). 2007. Framework for the Creation and Development of National Public Health Institutes. Paris: IANPHI (https://ianphi.org/_includes/documents/sections/tools-resources/ all-frameworks/frameworkfornphi.pdf) World Health Organization. 2016. Strengthening essential public health functions in support of the achievement of universal health coverage. Geneva: WHO (https:// apps.who.int/iris/handle/10665/252781) World Health Organization. 2018. Essential public health functions, health systems and health security: developing conceptual clarity and a WHO roadmap for action. Geneva: WHO (https://apps.who.int/iris/handle/10665/272597) World Health Organization. 2018. Primary health care: closing the gap between public health and primary care through integration. Geneva: WHO (https://apps. who.int/iris/handle/10665/326458) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) World Health Organization. 2023. Operationalizing essential public health functions – an integrated and comprehensive approach to public health. Geneva: WHO. World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) World Health Organization Regional Office for Europe. 2021. A guide to establishing national public health institutes through mergers. Copenhagen: WHO Regional Office for Europe (https://apps.who.int/iris/handle/10665/340282) Existing data collection tools Not at present 1134. Health system resilience indicators with metadata Indicator 60. Availability of a designated health system focal point responsible for health services provision assessment process in IHR monitoring and evaluation Indicator short name Focal point designated for IHR health services provision assessment Indicator name Availability of a designated health system focal point responsible for health services provision assessment process in IHR monitoring and evaluation Domain Governance and leadership Definition There is evidence of availability of a designated health system focal point (for example, a focal person or team) responsible for providing, drawing and coordinating inputs from health system and multisectoral stakeholders to the health services provision assessment process in IHR monitoring and evaluation (for example, State Party Self-Assessment Annual Report second edition, C8 health services provision; Joint External Evaluation third edition, R3 health services provision). Evidence for availability of a designated focal point can be in the form of clear terms of reference describing the roles and responsibilities. Rationale State Party Self-Assessment Annual Report (SPAR) second edition C8 health services provision requires specific health systems technical expertise for accurate and effective evaluation. Level National Disaggregation N/A Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2021. International Health Regulations (2005): State Party Self-assessment annual reporting tool, second edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005), third edition. Geneva: WHO (https://www.who.int/ publications/i/item/9789240051980) Existing data collection tools Not at present The IHR State Party Self-assessment annual reporting tool and Joint external evaluation tool collects data on the existence and functions of IHR focal point. 114Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 61. Implementation of a Health in All Policies (HiAP) approach Indicator short name Health in All Policies approach being implemented Indicator name Implementation of a Health-in-all-Policies (HiAP) approach Domain Governance and leadership Definition The country has implemented an HiAP approach that includes the following elements: • Existence of a national HiAP strategy and plan of action involving multiple sectors • Existence of recognized functional mechanisms to manage and monitor HiAP development and implementation • Mechanism for monitoring and oversight to examine the impact on health and equity of outcomes of HiAP • Evidence of collaborations across sectors to address health issues or determinants of health including: – Existence of operational policy/strategy/action plan to reduce physical inactivity – Age limits alcohol service/sales – Alcohol taxation – Drunk driving laws – Alcohol advertising restrictions – Alcohol licensing requirements – Existence of a national seat-belt law – Existence of national speed limit – MPOWER measures fully implemented (tobacco) – Existence of any policies to reduce population salt consumption – Existence of policies on marketing of foods to children – Existence of tax on sugar-sweetened beverages • Training opportunities and knowledge change for health workforce and institutions • Opportunities for community engagement through consultations and level of community participation. This indicator is in the Primary Health Care Measurement Framework and Indicators (2022) [Indicator 1]. Rationale Multisectoral policies and action are a core component of primary health care. To bring about policy changes in other sectors, the health community needs to advocate for change and to generate evidence on the health impacts of multisectoral determinants. This is particularly important because several the policy changes that are most important for improving health and well-being involve vested commercial interests, which often have significant influence over policymakers. HiAP is a whole-of- government approach to multisectoral policy and action at the national, subnational, and regional levels: “an approach to public policies across sectors that systematically takes into account the health implications of decisions, seeks synergies, and avoids harmful health impacts in order to improve population health and health equity” (WHA67.12). HiAP underscores the alignment of interests across policies to serve all people’s basic right to a healthy, productive life. It provides a framework for addressing determinants by developing the needed leadership and governance and providing an umbrella for multiple sets of actions across sectors. In an HiAP approach, the health sector is seen as the champion for health, keeping health on the agenda but aware of the need for policy action with mutual benefit with other sectors, seeking overall societal gains. National health assemblies can bring together key stakeholders, including those from other sectors, to shape policymaking. Level National Disaggregation N/A 1154. Health system resilience indicators with metadata Indicator 61 (continued). Implementation of a Health in All Policies (HiAP) approach Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Process Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/handle/10665/250221) World Health Organization. 2014. Health in all Policies (HiAP). Framework for Country Action. Geneva: WHO (https:// www.who.int/healthpromotion/hiapframework.pdf) World Health Organization. 2018. Health in All Policies as part of the primary health care agenda on multisectoral action. Geneva: WHO (https://apps.who.int/iris/ handle/10665/326463) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Existing data collection tools A qualitative assessment tool with recommended scoring methodology is currently under development by WHO and forthcoming. 116Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Indicator 62. Availability of national and subnational guidance (or equivalent) on health system recovery planning and actions Indicator short name Recovery planning guidance Indicator name Availability of national and subnational guidance (or equivalent) on health system recovery planning and actions Domain Governance and leadership Definition There is evidence that guidance document exists for health systems recovery planning and actions, with the following attributes: describing comprehensive and specific approach to recovery, for example, guiding principles and steps informed by situational reviews, evidence, and analyses The guidance could be a distinct document or a part of other national or subnational guidance, for example, for emergency management or health sector planning including a focus on recovery aspects. Rationale Health systems recovery planning and actions can be overlooked during and beyond health emergencies/events but offer significant opportunities to adapt, transform and improve the health system. Facilities can benefit from evidence- based guidance based on global, national and subnational lessons learned and best practices. Level National; Subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references Schmets G, Rajan D, Kadandale S, editors. 2016. Strategizing national health in the 21st century: a handbook. Geneva: WHO (https://apps.who.int/iris/ handle/10665/250221) World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) WHO has an upcoming publication on Health System Recovery Planning Guide. Existing data collection tools Not at present 1174. Health system resilience indicators with metadata Indicator 63. Existence of a designated entity or structure with the responsibility for recovery process following a public health event Indicator short name Designated entity with responsibility for recovery Indicator name Existence of a designated entity or structure with the responsibility for recovery process following a public health event Domain Governance and leadership Definition There is evidence of the availability of a designated entity or structure (for example, a multiagency working group; an existing governmental agency or department; etc.) mandated with responsibility for multisectoral coordination and implementation of recovery processes following public health emergencies/ events, humanitarian crises, natural disasters; etc. Examples of responsivities for recovery process include transitioning government arrangements from response to recovery, planning for health system recovery, etc. Evidence for availability of a designated entity or structure can be in the form of clear terms of reference covering responsibility for recovery process. Rationale A designated responsible authority or dedicated structure for health systems recovery functions ensures intended activities and outcomes are pursued as well as accountability. Level National; subnational Disaggregation Subnational (as relevant to context): region, state, province, canton, municipality, etc. National Numerator N/A Denominator N/A Recommended data source Qualitative key informant survey and/or desk review with verification from key country documents such as national, subnational polices, plans, and guidance Type (M&E domain) Input and structure Additional reading and references World Health Organization. 2021. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: WHO (https://apps.who.int/iris/ handle/10665/346515) Existing data collection tools Not at present 118Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 4.7 General/composite indicators Indicator 64. IHR State Party Self-Assessment Annual Report (SPAR) health service provision capacity score Indicator short name IHR SPAR health services provision capacity score Indicator name IHR State Party Self-Assessment Annual Report (SPAR) health services provision capacity score Domain Composite; service delivery Definition The SPAR (second edition, 2021) C8 health services provision capacity score is calculated through self-evaluation using different levels of indicators in the following areas: C8.1. Case management Level 1 National clinical case management guidelines for priority health events are not available or under development Level 2 National clinical case management guidelines for priority health events are developed but not being implemented Level 3 National clinical case management guidelines for priority health events are developed and being implemented at national level Level 4 National clinical case management guidelines for priority health events are developed and being implemented at national and subnational levels Level 5 National clinical case management guidelines for priority health events are implemented at all levels and are exercised (as applicable), reviewed, evaluated and updated on regular basis C8.2. Utilization of health services Level 1 Very low levels of service utilization (number of outpatient department visits per person per year < 1.00 visit/person/ year in both urban and rural areas) Level 2 Low levels of service utilization (number of outpatient department visits per person per year 1.0 ≤ X < 2.0 visit/ person/year, in both urban and rural areas) Level 3 Satisfactory levels of service utilization in tertiary health care facilities at national level (number of outpatient department visits per person per year ≥ 2.0 visit/person/year, in both urban and rural areas) Level 4 Strong levels of service utilization at all tertiary and secondary health care facilities at intermediate and national levels and geographical contexts (number of outpatient department visits per person per year ≥ 3.0 visit/person/ year, in both urban and rural areas) Level 5 Strong levels of service utilization at all tertiary, secondary and primary health care facilities at national, intermediate and local levels and geographical contexts (number of outpatient department visits per person per year ≥ 3.0 visit/ person/year, in both urban and rural areas) and information on service utilization is reviewed, evaluated and updated on a regular basis to inform policy and planning 1194. Health system resilience indicators with metadata Indicator 64 (continued). IHR State Party Self-Assessment Annual Report (SPAR) health service provision capacity score Definition C8.3. Continuity of essential health services (EHS) Level 1 A package of EHS is not defined and there are no plans or guidelines for continuity EHS during emergency Level 2 A package of EHS is defined but plans/guidelines on continuity of EHS in emergencies is not developed Level 3 A package of EHS and plans/guidelines on continuity of EHS in emergencies are developed and mechanism for monitoring service continuity during emergency is in place at national level Level 4 A package of EHS and plans/guidelines on continuity of EHS in emergencies are developed and mechanism for monitoring service continuity during emergency is in place at national and intermediate levels Level 5 A package of EHS, plans/guidelines on continuity of EHS in emergencies, and mechanisms for monitoring service continuity based on existing guidelines are defined and functional at national, intermediate and local levels and exercised, reviewed, evaluated and updated, with improvements based on simulation exercise (SimEx) and lessons learned from real- world events, for example, IARs or AARs Rationale Resilient health systems are essential if countries are to prevent, detect, respond to and recover from public health events while also ensuring the continuity of health services at all levels. Health services provision for both event-related case management and routine health services are as equally important. Ensuring minimal disruption in health services utilization before, during, and beyond an emergency and across the varied contexts within a country is a critical aspect of health systems resilience. This existing indicator in IHR monitoring and evaluation framework can be used in conjunction with other health system resilience indicators to understand health services provision capacities in both routine and emergency settings. Level National Disaggregation N/A Numerator N/A Denominator N/A Recommended data source IHR State Party Self-Assessment Annual Report questionnaire Electronic IHR States Parties Self-Assessment Annual Reporting Tool Type (M&E domain) Composite / Output Additional reading and references World Health Organization. 2021. International Health Regulations (2005): State Party Self-assessment annual reporting tool, second edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) Existing data collection tools World Health Organization. 2021. International Health Regulations (2005): State Party Self-assessment annual reporting tool, second edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. Electronic IHR States Parties Self-Assessment Annual Reporting Tool (https://extranet.who.int/e-spar) 120Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries 5 Supplementary indicators of relevance to health system resilience This section provides a set of supplementary indicators that have been identified as of relevance to health system resilience and can contribute to the comprehensive measurement, monitoring and analysis of the resilience of health systems and services. 122Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries Table 5. Supplementary indicators and sources Indicators Main sources and references Service delivery Number of inpatient beds per 10 000 population World Health Organization. Global Health Observatory (https://www.who.int/data/gho) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Average bed occupancy rate World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Average number of health workers per inpatient bed Related indicators in: World Health Organization. Global Health Observatory (https://www.who.int/data/gho) Outpatient department, primary health care and emergency department service utilization rate (before, during and after emergencies) Organisation for Economic Co-operation and Development. Health at a glance 2021: OECD indicators. Paris: OECD (https://doi.org/10.1787/19991312) Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Number of patients who are admitted to or leave a hospital after staying at least one night per 1000 population (includes death following inpatient care but excludes same-day discharges) Related indicators in: Organisation for Economic Co-operation and Development. Health at a glance 2021: OECD indicators. Paris: OECD (https://doi. org/10.1787/19991312) World Health Organization. Global Health Observatory (https://www.who.int/data/gho) Average length of stay (ALOS) at hospital or health facility (average number of days a patient has stayed at the facility from admission to discharge) Organisation for Economic Co-operation and Development. Health at a glance 2021: OECD indicators. Paris: OECD (https://doi. org/10.1787/19991312) World Health Organization. Global Health Observatory (https://www.who.int/data/gho) World Health Organization Regional Office for Europe. European health information gateway. Indicators explorer. Copenhagen: WHO Regional Office for Europe (https://gateway.euro.who.int/en/hfa-explorer/) Average number of consultations per capita per year Organisation for Economic Co-operation and Development. Health at a glance 2021: OECD indicators. Paris: OECD (https://doi. org/10.1787/19991312) Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) 123 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Implementation of a triage process in the outpatient department (OPD) Similar indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). Geneva: WHO (https://indicator-inventory.hhfa.online/) General readmission rate at the facility (hospital readmission within specified days of discharge) Related indicators in: World Health Organization. 2015. Global reference list of 100 core health indicators. Geneva: WHO (https://apps.who.int/iris/ handle/10665/173589) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Readmission rates for a tracer condition (relevant to the country context, for example, cases of malnutrition, caesarean section or surgical procedure) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Rate of a specific healthcare associated infection (HAI) (which is relevant to the country context) in patients Similar indicator in: World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Rates of return to intensive care unit (ICU) (return to ICU within specified hours of hospital discharge) Related indicators in: World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Availability of an antimicrobial resistance (AMR) stewardship programme? (yes or no) Food and Agriculture Organization of the United Nations, United Nations Environment Programme, World Health Organization, and World Organization for Animal Health. Global Database for Tracking Antimicrobial Resistance (AMR). Country Self- Assessment Survey (TrACSS). (http://www.amrcountryprogress.org/) World Health Organization. 2016. Global action plan on antimicrobial resistance. Geneva: WHO (https://www.who.int/ publications/i/item/9789241509763) World Health Organization. 2021. WHO policy guidance on integrated antimicrobial stewardship activities. Geneva: WHO (https://apps.who.int/iris/handle/10665/341432) 124 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Percentage of bloodstream infections due to selected antimicrobial-resistant organisms United Nations Statistics Division. Sustainable development goals (SDGs) Indicators (https://unstats.un.org/sdgs/metadata/) Percentage of health facilities that have a core set of relevant essential medicines available and affordable on a sustainable basis (disaggregated by medicine types, for example, antibiotics). World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Immunization coverage rate for measles/ yellow fever/polio or other country specific tracer vaccine World Health Organization. Global Health Observatory. Immunization coverage. (https://www.who.int/data/gho/data/ themes/topics/immunization-coverage) World Health Organization. WHO immunization data portal. (https://immunizationdata.who.int/listing.html?topic=coverage) United Nations Statistics Division. Sustainable development goals (SDGs) Indicators (https://unstats.un.org/sdgs/metadata/) Percentage of children aged six months to 15 years who have received measles vaccination, on completion of a measles vaccination campaign World Health Organization: Global Health Observatory. Immunization coverage. (https://www.who.int/data/gho/data/ themes/topics/immunization-coverage) Percentage of children aged six months to 59 months who have received an appropriate dose of vitamin A, on completion of a measles vaccination campaign World Health Organization. 2011. Guideline: Vitamin A supplementation in infants and children 6–59 months of age. Geneva: WHO ( https://www.who.int/publications/i/item/9789241501767) World Health Organization. Global Health Observatory (https://www.who.int/data/gho) Percentage of children aged 12 months who have had three doses of the combined diphtheria, tetanus toxoid and pertussis vaccine (DPT) World Health Organization. Global Health Observatory (https://www.who.int/data/gho) World Health Organization Regional Office for the Eastern Mediterranean. Eastern Mediterranean Health Observatory (https://rho.emro.who.int/metadata-Registry) 125 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Percentage of primary care facilities that offer basic Expanded Program on Immunization (EPI) services at least 20 days/month Related indicators and concepts in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) World Health Organization. Essential Programme on Immunization. (https://www.who.int/teams/immunization-vaccines- and-biologicals/essential-programme-on-immunization) Percentage of noncompliance to tuberculosis (TB) treatment disaggregated by type of TB case Related concepts in: World Health Organization. 2022. WHO consolidated guidelines on tuberculosis: module 4: treatment: drug-susceptible tuberculosis treatment. Geneva: WHO (https://apps.who.int/iris/handle/10665/353829) World Health Organization. 2022. WHO consolidated guidelines on tuberculosis: module 4: treatment: tuberculosis care and support. Geneva: WHO (https://apps.who.int/iris/handle/10665/353399) Annual in-patient cause-specific mortality rates for a tracer NCD condition (for example, cardiovascular diseases, diabetes or other specific condition, to be defined with reference to disease burden) at the last measurement point Related indicators in: World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Under-five crude mortality rate - deaths/10,000 children under 5 years/day United Nations Children’s Fund (UNICEF). Under-five mortality (https://data.unicef.org/topic/child-survival/under-five- mortality/#data) United Nations Statistics Division. Sustainable development goals (SDGs) Indicators (https://unstats.un.org/sdgs/metadata/) Is effective anti-malarial treatment provided in a timely manner to all children under age five years presenting with malaria? (yes or no) Related indicators in: World Health Organization. The Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/imr- details/14) Is oral rehydration salts (ORS) and zinc supplementation provided in a timely manner to all children under age five years presenting with diarrhoea? (yes or no) Related indicators in: World Health Organization. The Global Health Observatory. Health inequality monitor: Newborn and child health interventions: Care-seeking for sick children. https://www.who.int/data/gho/data/themes/topics/indicator-groups/indicator- group-details/GHO/health-equity-monitor-care-seeking-for-sick-children 126 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Is appropriate care provided in a timely manner to all children under age five years presenting with pneumonia? (yes or no) Related indicators in: World Health Organization. The Global Health Observatory. Health inequality monitor: Newborn and child health interventions: Care-seeking for sick children. https://www.who.int/data/gho/data/themes/topics/indicator-groups/indicator- group-details/GHO/health-equity-monitor-care-seeking-for-sick-children Presence of a specimen referral system (or protocol) between facilities and labs Related indicators and technical questions in: World Health Organization. 2021. International health regulations (2005): state party self-assessment annual reporting tool, 2nd ed. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) Proportion of samples within recommended turnaround time (for priority diseases) to/at the reference laboratory Related guidance: World Health Organization. Monitor adherence to the turnaround times as determined for each examination (https://extranet. who.int/lqsi/content/monitor-adherence-turnaround-times-determined-each-examination) Percentage of facilities located in a hazardous/at risk zone (hazardous zone: for example, a zone prone to earthquakes, natural disasters, periodic/ seasonal health risks and hazards/ outbreaks) Related concepts in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization, Pan American Health Organization. Smart Hospitals initiative. (https://www.paho.org/en/health- emergencies/smart-hospitals-initiative) Incidence of major communicable diseases is stable or not increasing against pre-crisis level? (yes or no) Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of suspected cases confirmed by a diagnostic method as determined by an agreed protocol Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of health facilities with protocols for the acutely injured including formal triage instruments Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) 127 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Percentage of health facilities with staff that have received basic training in the approach to the acutely injured Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of health facilities implementing quality improvement measures to reduce baseline morbidity and mortality according to available data Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of secondary healthcare facilities with trained and supervised staff and systems for managing mental health conditions Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of primary healthcare facilities providing care for priority NCDs Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of population that can access primary healthcare (PHC) within one hour’s walk from dwellings (or live within 5km of a PHC facility) Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Number of beds designated for isolation of infectious diseases at the healthcare facility Related indicators in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. Global Health Observatory (https://www.who.int/data/gho/data/themes/topics/topic-details/ GHO/health-service-delivery) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) 128 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Number of ICU beds per 10,000 population Similar indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). Geneva: WHO (https://indicator-inventory.hhfa.online/) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Percentage of healthcare facilities delivering the full essential package of health services (EPHS) routinely Similar indicators in: Health systems standard 1.1: Health service delivery in, Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https:// spherestandards.org/handbook-2018/) Percentage of the population that is requiring a referral, to the next level or specialty of healthcare, is seen at the next level or specialty of healthcare Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of patients referred in adequate time using standardized treatment protocols Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of healthcare facilities supporting a crisis-affected population using standardized treatment protocols for a specified illness due to PHE Communicable diseases standard 2.1.3: Diagnosis and case management in, Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) 129 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Case fatality rate is reduced to an acceptable level: for example, Cholera <1 per cent Meningitis <15 per cent Hepatitis E <4 per cent in general population, 10–50 per cent in pregnant women in third trimester Diphtheria (respiratory) <5–10 per cent Pertussis <4 per cent in children aged one year, <1 per cent in those aged one to four years Dengue <1 per cent Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of health facilities that have an emergency/disaster plan including management of mass casualties, reviewed and rehearsed on a regular basis Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) Percentage of the community population with access to essential health services during a public health emergency (PHE) Related concepts in: Maintaining essential health services: operational guidance for the COVID-19 context: interim guidance, 1 June 2020. https://www.who.int/publications/i/item/WHO-2019-nCoV-essential_health_services-2020.2 World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) Percentage of facilities meeting minimum standards to deliver tracer services World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Community satisfaction rates on specific health services (determined by country) in emergency context Similar indicators measuring community satisfaction on certain specific services can be found in: World Health Organization. Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) 130 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Health Workforce Number of health workers per 10,000 population by occupation World Health Organization. Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Number of community health workers (CHW) per 1,000 population World Health Organization. Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Average number of staff, by occupation in primary care facilities and hospitals Similar indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA) (https://indicator-inventory.hhfa.online/) Health worker absenteeism rate Related concept in: World Health Organization. 2010. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva: WHO (https://apps.who.int/iris/handle/10665/258734) Health workforce attrition rate Related indicators and concepts in: Health Finance and Governance Project. Human Resources for Health Indicators. Washington, D.C.: United States Agency for International Development (https://www.hfgproject.org/human-resources-health-indicators/) World Health Organization. 2017. National health workforce accounts: a handbook. Geneva: WHO (https://www.who.int/ publications/i/item/9789241513111) World Health Organization. Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/) 131 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Rural-to-urban distribution ratio of health workers Related indicators on health worker density in: World Health Organization. Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) United Nations Statistics Division. Sustainable development goals (SDGs) Indicators (https://unstats.un.org/sdgs/metadata/) Related concepts in: World Health Organization. 2010. Increasing access to health workers in remote and rural areas through improved retention: global policy recommendations. Geneva: WHO (https://www.who.int/publications/i/item/increasing-access-to-health- workers-in-remote-and-rural-areas-through-improved-retention) Percentage of births attended by skilled personnel (doctors, nurses, midwives) United Nations Statistics Division. Sustainable development goals (SDGs) Indicators (https://unstats.un.org/sdgs/metadata/) World Health Organization. Global Health Observatory (https://www.who.int/data/gho/indicator-metadata-registry/) Skilled care is available for emergency obstetrics and new-born care at all times (yes or no) Similar indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA) (https://indicator-inventory.hhfa.online/) Percentage of health workers that are vaccinated against Hepatitis B (and other vaccinations specific to the country context) World Health Organization. 2016. Monitoring and evaluation for viral hepatitis B and C: recommended indicators and framework. Geneva: WHO (https://apps.who.int/iris/handle/10665/204790) Rate of hospital acquired infections (HAIs) (specific to country context) among health workers World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) 132 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Percentage of health workers that received training and qualifications from an accredited educational or training institution or body Related indicators in: World Health Organization. 2017. National health workforce accounts: a handbook. Geneva: WHO (https://www.who.int/ publications/i/item/9789241513111) World Health Organization. 2021. International health regulations (2005): state party self-assessment annual reporting tool, 2nd ed. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) All health staff performing clinical work in emergencies have received training in clinical protocols and case management (yes or no) Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). Geneva: WHO (https://indicator-inventory.hhfa.online/) Percentage of health staff in high-risk areas trained on outbreak response plan and protocols Related indicators in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. Harmonized Health Facility Assessment (HHFA) (https://indicator-inventory.hhfa.online/) All health facilities have trained staff, sufficient supplies and equipment for clinical management of rape survivor services based on national or international protocols (yes or no) Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA) (https://indicator-inventory.hhfa.online/) National total expenditure on health workforce World Health Organization. 2017. National health workforce accounts: a handbook. Geneva: WHO (https://www.who.int/ publications/i/item/9789241513111) Percentage of facilities with a reporting mechanism for occupational hazard incidents Related concepts and rationale in: International Labour Organization. 1996. Recording and notification of occupational accidents and diseases. Geneva: ILO (https://www.ilo.org/safework/info/standards-and-instruments/codes/WCMS_107800/lang--en/index.htm) 133 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Health Information Percentage of notifiable public health events reported to the appropriate authorities within 24 hours Related indicators or technical questions in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. 2021. International health regulations (2005): state party self-assessment annual reporting tool, 2nd ed. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) Percentage of complete Early Warning, Alert and Response (EWAR)/surveillance reports submitted on time Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of reported alerts being verified within 24 hours Related indicators or technical questions in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. 2021. International health regulations (2005): state party self-assessment annual reporting tool, 2nd ed. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) Frequency of health information reports (for example, epidemiological bulletin) produced by the lead health actor Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Use of electronic information and surveillance systems? (yes or no) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) World Health Organization. Benchmarks for IHR Capacities. (https://ihrbenchmark.who.int/document/9-surveillance) Health workers training on surveillance and information systems? (yes or no) Related technical questions and standards in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. Benchmarks for IHR Capacities. (https://ihrbenchmark.who.int/document/9-surveillance) 134 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Defined procedures and deadlines for the transmission of health information between facilities and across different levels? (yes or no) Related technical questions and standards in: Health systems standard 1.5: Health information in, Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/ handbook-2018/) Percentage of facilities with access to communications system World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Percentage of facilities that experienced any interruption to access to telephone service (landline or mobile) in previous week Related technical questions in: World Health Organization. 2021. Continuity of essential health services: facility assessment tool: a module from the suite of health service capacity assessments in the context of the COVID-19 pandemic: interim guidance, 12 May 2021. Geneva: WHO (https://apps.who.int/iris/handle/10665/341306) Proportion of households in at-risk areas that report receiving appropriate information/education from assigned community health workers, on specified priority public health threats Similar indicators in: World Health Organization. 2021. International health regulations (2005): state party self-assessment annual reporting tool, 2nd ed. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) Access to Essential Health Products Is the availability of essential medicines and supplies being monitored? (yes or no) Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) Number of days essential medicines are not available during a month Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) System in place for the emergency procurement (ordering) of medicines and medical supplies in the event of a public health emergency (yes or no) Related definitions and standards in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) 135 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Presence of a national and/or regional list of emergency medicines and supplies identified based on priority diseases Related definitions and indicator in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of health facilities with essential medicines World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) Are all medicines dispensed to patients within the expiry date? (yes or no) Related indicator in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) Availability of Infection Prevention and Control supplies for example, exam gloves; long plastic gloves; protective masks; plastic apron; PPE; triple packaging kit; stock of soap; stock of disinfectants Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) Availability of standard medical equipment (essential medical equipment can be defined by WHO list of priority medical equipment and devices, or local/ priority need) Similar indicator in: World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Facility has access to a blood bank with blood and blood products, and standard operating procedures (SOPs) and guidelines for their correct storage and handling Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) All transfused blood is screened and is free of transfusion-transmissible infections, including HIV Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) 136 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Facilities with access to functional transport services for logistics needs for essential supplies and services Related definitions and standards in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) Percentage of facilities that have access to a functional ambulance service (as recommended in national guidelines) Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). Geneva: WHO (https://indicator-inventory.hhfa.online/) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Are supply chain management protocols in place for activating and coordinating stockpiles during a public health emergency (PHE)? (yes or no) Related indicators in: World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) Percentage of health facilities that have an appropriate set of essential in vitro diagnostics (IVDs) and associated laboratory equipment and consumables for their health care facility level available on a sustainable basis Similar indicator in: World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Percentage of facilities that have a functional waste management system? Related indicators in: World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) Are all inpatients in healthcare settings use long-lasting insecticide nets (LLINs) in malarial zones? (yes or no) Related indicators in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization. Harmonized Health Facility Assessment (HHFA). (https://indicator-inventory.hhfa.online/) 137 5. Supplem entary indicators of relevance to health system resilience Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Resilience of health facility infrastructure to disasters is assessed as part of accreditation process Related concepts in: World Health Organization. 2022. Health care accreditation and quality of care: exploring the role of accreditation and external evaluation of health care facilities and organizations. Geneva: WHO (https://www.who.int/publications/i/ item/9789240055230) Facility safety has been assessed within the past 12 months Related indicators and concepts in: Sphere Association. 2018. The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition. Geneva: Sphere Association (https://spherestandards.org/handbook-2018/) World Health Organization, Pan American Health Organization. 2019. Hospital Safety Index. Guide for Evaluators. Second Edition. Washington, D.C.: WHO, PAHO (https://iris.paho.org/handle/10665.2/51448) Percentage of facilities with access to emergency transport for interfacility transport World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Percentage of facilities that use an electrical power source (excluding standalone medical devices) at least some of the time World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Governance There is a national policy, strategy or plan guiding the engagement of the private sector in health service delivery that includes WHO-recommended behaviours World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Private sector representatives are involved in development of national crisis management plans World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) 138 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries Table 5 (continued). Supplementary indicators and sources Indicators Main sources and references Availability of multisectoral emergency management forum with community representation Similar indicators in: World Health Organization. 2021. International health regulations (2005): state party self-assessment annual reporting tool, 2nd ed. Geneva: WHO (https://www.who.int/publications/i/item/9789240040120) World Health Organization. 2022. Joint external evaluation tool: International Health Regulations (2005) - third edition. Geneva: WHO (https://www.who.int/publications/i/item/9789240051980) World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Financing Current percentage of health expenditure that is externally sourced World Health Organization. Global Health Expenditure Database (GHED). (https://apps.who.int/nha/database) World Health Organization. Global Health Observatory. (https://www.who.int/data/gho/indicator-metadata-registry/) Domestic general government expenditure on PHC as a share of domestic general government health expenditure nationally World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Appropriate provider payment methods are in place in national as measured against criteria World Health Organization and United Nations Children’s Fund (UNICEF). 2022. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: technical specifications. Geneva: WHO and UNICEF (https://apps.who.int/iris/handle/10665/352201) Out-of-pocket (OOP) payment for health (% of current expenditure on health) World Health Organization. Global Health Observatory. (https://www.who.int/data/gho/indicator-metadata-registry/) World Health Organization. Global Health Expenditure Database (GHED). (https://apps.who.int/nha/database) 139 5. Supplem entary indicators of relevance to health system resilience 6 Resources of indicator sets and monitoring and evaluation frameworks as sources Main resources of indicator sets and monitoring and evaluation frameworks which informed the development of this health system resilience package and metadata are listed below. Links to these resources are provided for convenient reference. No. Resource Year Source/ Author(s) Link to the resource 1 Assessment of essential public health functions in countries of the Eastern Mediterranean Region 2017 WHO Regional Office for the Eastern Mediterranean https://www.emro.who.int/about-who/public-health-functions/ assessment-public-health-functions.html 2 Building resilient health systems: a proposal for a resilience index 2017 Kruk ME, Ling E J, Bitton A, Cammett M, Cavanaugh K, Chopra M et al. https://doi.org/10.1136/bmj.j2323 3 Checklist and Indicators for Monitoring Progress in the Development of IHR Core Capacities in States Parties 2013 WHO https://iris.who.int/handle/10665/84933 4 Continuity of essential health services: facility assessment tool 2021 WHO https://iris.who.int/handle/10665/341306 5 Harmonized Health Facility Assessment (HHFA) 2022 WHO https://www.who.int/data/data-collection-tools/harmonized-health- facility-assessment/introduction 6 Health at a Glance 2021: OECD Indicators 2021 OECD https://www.oecd-ilibrary.org/social-issues-migration-health/health- at-a-glance-2021_ae3016b9-en 7 Health Emergency and Disaster Risk Management Framework 2019 WHO https://www.who.int/publications/i/item/9789241516181 8 Health emergency preparedness self- assessment tool 2018 European Centre for Disease Prevention and Control https://www.ecdc.europa.eu/en/publications-data/hepsa-health- emergency-preparedness-self-assessment-tool-user-guide 9 Health financing country diagnostic: a foundation for national strategy development 2016 WHO, McIntyre D, Kutzin J https://iris.who.int/handle/10665/204283 10 Health Resources and Services Availability Monitoring System (HeRAMS) - WHO https://www.who.int/initiatives/herams 141 6. Resources of indicator sets and m onitoring and evaluation fram ew orks as sources No. Resource Year Source/ Author(s) Link to the resource 11 Health Sector Self-Assessment Tool for Disaster Risk Reduction 2010 WHO, Pan American Health Organization https://iris.paho.org/handle/10665.2/34974 12 Health system performance assessment 2022 WHO https://iris.who.int/handle/10665/352686 13 Health system preparedness for emerging infectious disease: A synthesis of the literature 2019 Palagyi A, Marais BJ, Abimbola S, Topp SM, McBryde ES, Negin J https://doi.org/10.1080/17441692.2019.1614645 14 Health System Strengthening - A Compendium of Indicators 2017 Diana M, Yeager V, Hotchkiss D https://www.measureevaluation.org/resources/publications/tr-17- 167b.html 15 Hospital emergency response checklist: an all-hazards tool for hospital administrators and emergency managers 2011 WHO https://iris.who.int/handle/10665/349374 16 Hospital safety index: guide for evaluators, 2nd ed. 2015 WHO https://iris.who.int/handle/10665/258966 17 IHR (2005): guidance document for the State Party self-assessment annual reporting tool 2018 WHO https://www.who.int/publications/i/item/WHO-WHE-CPI-2018.17 18 IHR Core Capacity Monitoring Framework: Questionnaire for Monitoring Progress in the Implementation of IHR Core Capacities in States Parties 2017 WHO https://iris.who.int/handle/10665/255756 19 IHR-PVS National Bridging Workshops - FAO, OIE, WHO https://extranet.who.int/sph/ihr-pvs-bridging-workshop#parallax- our-work 20 Infection Prevention and Control Assessment Framework at the Facility Level 2018 WHO https://www.who.int/publications/i/item/WHO-HIS-SDS-2018.9 142 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries No. Resource Year Source/ Author(s) Link to the resource 21 Joint Assessment of a National Health Strategy (JANS) 2014 International Health Partnership https://www.uhc2030.org/what-we-do/improving-collaboration/ health-systems-strengthening/jans-tool-and-guidelines/ 22 Joint External Evaluation (JEE) - WHO https://extranet.who.int/sph/jee 23 Joint External Evaluation (JEE) Tool, 3rd Edition 2022 WHO https://www.who.int/publications/i/item/9789240051980 24 Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies 2010 WHO https://iris.who.int/handle/10665/258734 25 National health workforce accounts: a handbook 2017 WHO https://iris.who.int/handle/10665/259360 26 PATH – Performance Assessment Tool for Quality Improvement in Hospitals 2007 WHO https://iris.who.int/handle/10665/107808 27 Patient Safety Assessment 2012 WHO Regional Office for the Eastern Mediterranean https://www.who.int/publications/i/item/9789290221203 28 Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens 2022 WHO and UNICEF https://www.who.int/publications/i/item/9789240044210 29 Primary Health Care Vital Signs 2018 Primary Health Care Performance Initiative https://www.improvingphc.org/sites/default/files/2018PHCPI_ VitalSignsReport_3.pdf 30 Public health emergency preparedness: a framework to promote resilience 2018 Khan Y, O’Sullivan T, Brown A, Tracey S, Gibson J, Généreux M, Henry B, Schwartz B https://bmcpublichealth.biomedcentral.com/articles/10.1186/ s12889-018-6250-7 31 Recovery toolkit: supporting countries to achieve health service resilience: a library of tools and resources available during the recovery period of a public health emergency 2016 WHO https://iris.who.int/handle/10665/205944 143 6. Resources of indicator sets and m onitoring and evaluation fram ew orks as sources No. Resource Year Source/ Author(s) Link to the resource 32 Resilience Quality Indicators Report - WHO Available on request 33 Resilient Health System as Conceptual Framework for Strengthening Public Health Disaster Risk Management: An African Viewpoint 2017 Olu O https://www.frontiersin.org/articles/10.3389/fpubh.2017.00263/full 34 Safe hospitals in emergencies and disasters: structural, non-structural and functional indicators 2010 WHO https://iris.who.int/handle/10665/207689 35 Sendai Framework for Disaster Risk Reduction 2015–2030 2015 United Nations https://www.undrr.org/publication/sendai-framework-disaster-risk- reduction-2015-2030 36 Service availability and readiness assessment (SARA) 2015 WHO https://www.who.int/data/data-collection-tools/service-availability- and-readiness-assessment-(sara) 37 Service Delivery Indicators 2010 World Bank https://www.worldbank.org/en/programs/service-delivery-indicators 38 Service Provision Assessment (SPA) 2012 DHS Program, USAID https://dhsprogram.com/methodology/Survey-Types/SPA.cfm 39 State Party Self-Assessment Annual Reporting Tool (SPAR), 2nd edition 2021 WHO https://www.who.int/publications/i/item/9789240040120 40 Strategic Partnership for Health Security and Emergency Preparedness (SPH) Portal - WHO https://extranet.who.int/sph/ 41 Strengthening health-system emergency preparedness: toolkit for assessing health-system capacity for crisis management 2012 WHO https://iris.who.int/handle/10665/352566 https://iris.who.int/handle/10665/352568 42 Support tool to assess health information systems and develop and strengthen health information strategies 2015 WHO Regional Office for Europe https://iris.who.int/handle/10665/172761 144 Health system resilience indicators: an integrated package for m easuring and m onitoring health system resilience in countries No. Resource Year Source/ Author(s) Link to the resource 43 The sphere handbook: humanitarian charter and minimum standards in humanitarian response, fourth edition 2018 Sphere Association https://www.spherestandards.org/handbook/ 44 Tools for Assessing the Operationality of District Health Systems 2003 WHO Regional Office for Africa, Sambo LG, Chatora RR and Goosen ESM Available on request 45 Tools for District Health Systems functionality assessment for IHR - WHO Available on request 46 UN Common Guidance on Helping Build Resilient Societies 2021 United Nations https://unsdg.un.org/resources/un-common-guidance-helping-build- resilient-societies 47 Universal Health and Preparedness Review (UHPR) - WHO https://www.who.int/emergencies/operations/universal-health--- preparedness-review 48 WHO country assessment tool on the uses and sources for human resources for health (HRH) data 2012 WHO https://iris.who.int/handle/10665/77947 49 WHO Simulation Exercise (SimEx) Manual 2017 WHO https://www.who.int/publications/i/item/WHO-WHE-CPI-2017.10 50 Worldwide Governance Indicators - World Bank https://datacatalog.worldbank.org/search/dataset/0038026 DHS: Demographic and Health Surveys; FAO: Food and Agriculture Organization; OECD: Organisation for Economic Co-operation and Development; OIE: World Organization for Animal Health; UNICEF: United Nations Children’s Fund; USAID: United States Agency for International Development 145 6. Resources of indicator sets and m onitoring and evaluation fram ew orks as sources References 1. McDarby G, Seifelden R, Zhang Y, Mustafa S, Petrova M, Schmets G et al. A synthesis of concepts of resilience to inform operationalization of health systems resilience in recovery from disruptive public health events including COVID-19. Front Public Health. 2023:11. 2. Health Systems Resilience Toolkit: a WHO global public health good to support building and strengthening of sustainable health systems resilience in countries with various contexts. Geneva: World Health Organization; 2022 (https://www.who.int/publications/i/ item/9789240048751, accessed 28 October 2023). 3. Mustafa S, Zhang Y, Zibwowa Z, Seifeldin R, Ako- Egbe L, McDarby G et al. COVID-19 preparedness and response plans from 106 countries: a review from a health systems resilience perspective. Health Policy Plan. 2022;37(2):255–68. doi:10.1093/heapol/czab089. 4. WHA64.10: Strengthening national health emergency and disaster management capacities and resilience of health systems. In: Sixty-fourth World Health Assembly, May 2011. Geneva: World Health Organization; 2011 (https://apps.who.int/ iris/handle/10665/3566, accessed 28 October 2023). 5. Zero draft of the WHO CA+ for the consideration of the Intergovernmental Negotiating Body at its fourth meeting: WHO convention, agreement or other international instrument on pandemic prevention, preparedness and response (“WHO CA+”). A/INB/4/3. In: Fourth meeting of the Intergovernmental Negotiating Body to draft and negotiate a WHO convention, agreement or other international instrument on pandemic prevention, preparedness and response, February 2023. Geneva: World Health Organization; 2023 (https://apps.who.int/gb/ inb/e/e_inb-4.html, accessed 28 October 2023). 6. Kruk ME, Ling EJ, Bitton A, Cammett M, Cavanaugh K, Chopra M et al. Building resilient health systems: a proposal for a resilience index. BMJ. 2017;357:j2323. 7. Fleming P, O’Donoghue C, Almirall-Sanchez A, Mockler D, Keegan C, Cylus J et al. Metrics and indicators used to assess health system resilience in response to shocks to health systems in high income countries: a systematic review. Health Policy. 2022;126(12):1195–205. doi:10.1016/j.healthpol.2022.10.001. 8. Building health systems resilience for universal health coverage and health security during the COVID-19 pandemic and beyond: WHO position paper. Geneva: World Health Organization; 2021 (https://iris.who.int/handle/10665/346515, 3 November 2023). 9. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva: World Health Organization; 2010 (https://iris.who.int/handle/10665/258734, accessed 3 November 2023). 10. World Health Organization & United Nations Children’s Fund (UNICEF). Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Geneva: World Health Organization; 2022 (https://iris.who.int/ handle/10665/352205, accessed 3 November 2023). 11. Continuity of essential health services: facility assessment tool: a module from the suite of health service capacity assessments in the context of the COVID-19 pandemic: interim guidance, 12 May 2021. Geneva: World Health Organization; 2021 (https:// iris.who.int/handle/10665/341306, accessed 3 November 2023). 146 12. International Health Regulations (2005): IHR monitoring and evaluation framework. Geneva: World Health Organization; 2018 (https:// iris.who.int/handle/10665/276651, accessed 3 November 2023). 13. Universal Health and Preparedness Review. Geneva: World Health Organization; 2023 (https://www.who.int/emergencies/operations/ universal-health---preparedness-review, accessed 3 November 2023). 14. Papanicolas I, Rajan D, Karanikolos M, Soucat A, Figueras J, editors. Health system performance assessment: a framework for policy analysis. Geneva: World Health Organization; 2022 (https://iris.who.int/handle/10665/352686, accessed 3 November 2023). 15. Primary health care measurement framework and indicators: monitoring health systems through a primary health care lens. Web annex: Technical specifications. Geneva: World Health Organization: 2022 (https:// iris.who.int/handle/10665/352201, accessed 3 November 2023). 147References Annex. Methodological approach to HSR indicator development Defining health system resilience capacities Definitions of resilience vary among different actors and communities of practice (for example, peace and security, development, human rights, disaster risk reduction and climate change). However, within the United Nations system, there is a unified definition, as follows (1): [Resilience is] the ability of individuals, households, communities, cities, institutions, systems and societies to prevent, resist, absorb, adapt, respond and recover positively, efficiently and effectively when faced with a wide range of risks, while maintaining an acceptable level of functioning and without compromising long-term prospects for sustainable development, peace and security, human rights and well-being for all. In the context of health systems, health system resilience can be defined as the capacity of health actors, institutions, and populations to prepare for and effectively respond to public health events and regular stressors; maintain essential, routine and core functions in all contexts; and, informed by lessons learned, adapt and improve (2). Health systems are resilient if they protect human life and produce good health outcomes for all during a crisis and in its aftermath (2). Health system resilience means that the system is able to adapt its functioning to absorb shocks and daily stressors and transform, if necessary, to recover and maintain functionality (3). The ability of a health system to respond and adapt to external shocks and stressors – including infectious disease outbreaks and natural disasters – is seen as one of the key elements of health system resilience (3, 4). Through an emergency and disaster risk management lens, resilience can be further delineated as “the ability of a system, community or society exposed to hazards to resist, absorb, accommodate, adapt to, transform and recover from the effects of a hazard in a timely and efficient manner, including through the preservation and restoration of its essential basic structures and functions through risk management” (5). For the purpose of developing this package of indicators, it was necessary to consider how definitions of health system resilience can be expressed in functional terms of relevance to health service delivery at health facilities. To do so, a desk review of the literature and existing relevant indicator sets was conducted, as well as technical consultations with WHO experts. From these exercises, four categories of health system resilience capacities were identified: • the capacity to forecast, prevent, and prepare for public health needs (such as emergencies, shocks and routine stressors); • the capacity to maintain the delivery of routine and essential health services in all contexts (such as emergencies, shocks and routine stressors); • the capacity to absorb, adapt and respond to changes in demand and the need for health services; • the capacity to learn and improve, as required, based on experience, maintaining the course towards long-term objectives. 148 Data collection Literature review Iterative literature reviews underpinned by a systematic search protocol were conducted throughout October 2019 to December 2022. The research questions addressed through the review were as follows: • How is health system resilience defined? • What core capacities must be in place to achieve resilience? • How is resilience measured, and how might it be measured more comprehensively and effectively? The findings from the review were used to: • develop a conceptual framework for the monitoring and evaluation of health system resilience; • identify key resilience capacities and attributes; • identify and define key phases in the emergency cycle and consider what structures, resources, policies, mechanisms and outputs are required for resilience outcomes and impacts; • identify relevant indicators and indicator sets; • consider current constraints at the operational level (feasibility and affordability) and future goals for health systems and health security; • enable the specification of key questions for the next stage of data collection – the technical consultation, as described below. Technical consultation Consultations with key experts were conducted at WHO headquarters in Geneva from 2019 to 2022. Participants were recruited according to their expertise in key technical areas, including: • emergency preparedness and response (including in fragile, conflict-affected and vulnerable settings); • humanitarian intervention; • service delivery before, during and in recovery from health emergencies; • animal health and One Health approaches; • health system governance, policy and effectiveness; • primary health care; • risk management for health emergencies and disasters. The data generated from these consultations were used to: • ensure that the conceptual framework for resilience was comprehensive; • generate information on relevant indicators and sets of indicators on topics related to the four capacities; • assist in determining an inclusion criterion for the selection of indicators for the package. Consultations with key experts in Monrovia, Liberia, were conducted within the scope of the KOICA- funded health system resilience project. Participants included representatives of the ministries of health, national public health institutes, health facilities, universities, research institutes, and WHO country offices in Liberia and Ethiopia. Consultations were conducted by way of key informant interviews, discussion groups and plenary sessions. The data generated were consolidated and used to refine the indicators, with the added consideration of feasibility and applicability to the local context. Further rounds of technical consultations were conducted throughout 2021 and 2022 at WHO headquarters and with regional offices to further review the indicators and inform the next steps. Indicator set review A review of indicator sets related to health system strengthening, primary health care, emergency management and health system resilience was conducted. An initial list of indicator sets was identified from the literature review and expert consultations. Academic databases and online searches were used to identify additional indicator sets. Drawing on the conceptual framework, and insights from the literature review and expert consultations, the indicator sets were assessed for relevance to the four capacity categories defined above. The indicators assessed as most relevant were used to develop an initial compendium (or “longlist”) of potential indicators for inclusion. 149Annex. Methodological approach to HSR indicator development Data analysis and indicator selection The indicators were grouped according to the WHO health system building blocks framework. From the “long list” of indicators (n=830), each indicator was assessed for relevance to the four capacities with a focus on an integrated approach to health system resilience and its determinants; those deemed most relevant were selected. The remaining indicator set (n=432) was then assessed, and indicators that were unlikely to be measurable, or were primarily relevant to high-income country settings, were excluded. Duplicates were excluded, while others that were substantively related were combined. Figure A.1 illustrates how the final set of indicators was generated. Figure A.1 Generation of final set of indicators Identification Screening Grouping Included Indicators deems as feasible (n=173) Final HSR indicator set (relevant and feasible) (n=64) + Supplementary indicators of HSR relevance (n=101) Indicators aer similar/duplicate indicators were removed (n=432) Low feasibility indicators excluded (n=259) Compendium of indicators relating to one or more resilience capacities (n=830) Similar/duplicate indicators excluded (n=398) Relevant indicator sets identified via literature review, expert consultation and online search Inclusion criteria: must relate to one or more of the four resilience capacities (n=43 indicator sets; n=748 individual indicators) Individual indicators extract from search of academic databases and grey literature Inclusion criteria: must relate to one or more of the four resilience capacities (n=82) 150Health system resilience indicators: an integrated package for measuring and monitoring health system resilience in countries References: Annex 1. United Nations Development Group/Inter-Agency Standing Committee (2015). Risk and Resilience: A Proposed Approach for better UN Cross-Pillar Integration. 2. Kruk ME, Myers M, Varpilah ST, Dahn BT (2015). What is a resilient health system? Lessons from Ebola. The Lancet 385(9980):1910-12. 3. Blanchet K (2015). Thinking shift on health systems: from blueprint health programmes towards resilience of health systems. Int J Health Policy Manag. 4(5):307-9. 4. Health Systems Global (2016). Resilient and responsive health systems for a changing world (http://healthsystemsresearch.org/hsr2016/ about/theme/, accessed 3 November 2023). 5. United Nations, General Assembly (2016). Report of the open-ended intergovernmental expert working group on indicators and terminology relating to disaster risk reduction. Note by the Secretary-General. 151

World Health Organization 20 Avenue Appia 1211 Geneva 27 Switzerland www.who.int