TOOL TO ASSESS THE IMPACT OF HUMAN RESOURCES FOR HEALTH INVESTMENTS ON HIV, TB AND MALARIA SERVICES AND HEALTH OUTCOMES
TOOL TO ASSESS THE IMPACT OF HUMAN RESOURCES FOR HEALTH INVESTMENTS ON HIV, TB AND MALARIA SERVICES AND HEALTH OUTCOMES Tool to assess impact of human resources for health investments on HIV, TB and malaria services and health outcomes (Human Resources for Health Observer Series No. 27) ISBN 978-92-4-004206-3 (electronic version) ISBN 978-92-4-004207-0 (print version) © World Health Organization 2022 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules/). Suggested citation. Tool to assess impact of human resources for health investments on HIV, TB and malaria services and health outcomes. Geneva: World Health Organization; 2022 (Human Resources for Health Observer Series No. 27). Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see https://www.who.int/copyright. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. Design and layout by gardeners Printed in Switzerland Contents Acknowledgements iv Abbreviations and acronyms v Executive summary vi 1. Introduction 1 2. Methods 2 2.1. Conceptual framework 2 2.2. Structure of the model and key assumptions 3 3. Results 13 3.1. Illustrative examples of impact estimates 13 3.2. Country case studies 17 4. Discussion 20 4.1. Limitations 20 4.2. Data requirements 21 4.3. Policy implications 21 Annex 1. Assumptions of the calculator 22 Overview 22 Assumptions 22 Annex 2. Targeted literature review to identify empirical estimates for specific assumptions in the model 24 a) CHW productivity 24 b) Studies on pre-service education 25 c) Studies on salary pay raises and retention 26 d) Studies on in-service training and retention 28 Annex 3. Approaches to estimating the relationship between HRH density and service coverage 29 a) DEA for the benchmarking alternative 29 b) Aggregation of four treatment service indicators into a single metric 32 References 36 Web Annexes A. Lives saved calculator. WHO/UHL/HWF/HWP/2022.1. https://apps.who.int/iris/bitstream/ handle/10665/351524/WHO-UHL-HWF-HWP- 2022.1-eng.xlsm B. Coverage target calculator. WHO/UHL/HWF/HWP/2022.2. https://apps.who.int/iris/bitstream/ handle/10665/351525/WHO-UHL-HWF-HWP- 2022.2-eng.xlsm C. User guide for web annexes A and B. https://youtu.be/pWm6HjoxahM iii Acknowledgements The development of this document was overseen by an inter-organizational working group led by the World Health Organization (WHO). Members of the working group included representatives from the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund), the Joint United Nations Programme on HIV and AIDS (UNAIDS), the United States Agency for International Development (USAID) and the President’s Emergency Plan for AIDS Relief (PEPFAR). The group was coordinated by Giorgio Cometto (Health Workforce Department). Other WHO colleagues who contributed in several ways included the following: Khassoum Diallo and Onyema Ajuebor (Health Workforce Department); Nathan Paul Ford (Global HIV, Hepatitis and STIs Programmes); Richard Cibulskis, John Jairo Aponte Varon and Noor Abdisalan (Global Malaria Programme); Lana Syed (Global Tuberculosis Programme); Gulin Gedik (Regional Adviser, WHO Regional Office for the Eastern Mediterranean), Jamaal Nasher and Muhammad Safdar Kamal Pasha (WHO Country Office, Pakistan). Working group members from other organizations included: Olga Bornemisza, Alexander Rowe, Shufang Zhang and Mehran Hosseini (the Global Fund); Erik Philippe Alain Lamontagne (UNAIDS); Vasireddy Vamsi (PEPFAR) and Gordon Akudibillah (USAID). The efforts of the following Global Fund portfolio managers to collect and appraise data are greatly appreciated: Werner Buehler, Marion Hachmann-Gleixner, Paul McCarrick, Musoke J. Sempala and Qi Cui. Amos Royle Bemeyani of the Malawi Ministry of Health is also appreciated in this regard. The writing of the methodology document and the development of the calculator tools were led by Tim Bruckner and Jenny Liu of the University of California, Berkeley, and Tracy Lin at the University of California, San Francisco. WHO is grateful to the Global Fund for funding the development of the methodology and tools, and for providing key data that informed their development. iv Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Abbreviations and acronyms ART antiretroviral therapy CEA cost–effectiveness analysis CHW community health worker CO clinical officer DALY disability-adjusted life-year DCE discrete choice experiment DEA data envelopment analysis DNMs doctors, nurses and midwives FTE full-time equivalent GHE Global Health Estimates Global Fund Global Fund to Fight AIDS, Tuberculosis and Malaria HRH human resources for health LICs low-income countries LMICs low- and middle-income countries MES median effect size OLS ordinary least squares PEPFAR President’s Emergency Plan for AIDS Relief PMTCT prevention of mother-to-child transmission TB tuberculosis UHC universal health coverage UNAIDS Joint United Nations Programme on HIV/AIDS USAID United States Agency for International Development WHO World Health Organization WTP willingness to pay v Executive summary The WHO Global strategy on human resources for health: workforce 2030 (1) encourages development partners and global health initiatives to leverage their support to health systems in countries to sustainably strengthen the health workforce. The Global Fund makes substantial investments in human resources for health (HRH) to “incentivize use of country allocations for strategic priorities…contributing to resilient and sustainable systems for health”. To assess the impact of these investments, a methodology was developed and pilot tested by WHO. The impact assessment tool (consisting of an MS Excel calculator with two subsets) presented in this paper can do the following respectively: • assess and quantify the health impact of HRH investments made in the context of HIV, tuberculosis (TB) and malaria programmes through their modelled effect on health service coverage of these three diseases; and • provide aggregate indicative estimates of the range of health workers required to attain high coverage of selected health services. The approach adopted in this analysis entailed the development of a deterministic user-friendly tool to calculate associations between HRH investment inputs and effects on reduced morbidity and mortality. Three distinct conceptual steps enable assessment of the causal pathway and modelling of the correlations between: • investment in HRH (through education, deployment, remuneration of health workers) and improved availability, distribution and performance of HRH; • improved availability and performance of HRH and its effect on coverage of selected services related to HIV, TB and malaria; and • improved coverage of services related to the three diseases and reduced morbidity and mortality. Empirical estimates were identified from the literature to populate the model with relevant assumptions. The calculator was tested using several sample calculations based on a fictitious country, and subsequently pilot tested using real data from Global Fund grants in four low- and middle- income countries (LMICs). In every country where the tool was tested, it was possible to model an estimate of the number of lives saved through the HRH investments made. The development of this impact assessment tool, which stems directly from an integrated conceptual model, demonstrates the feasibility of estimating improvements in service treatment and lives saved for HIV, TB and malaria deriving from investments in HRH. The integration of the literature, as well as novel empirical results, indicate positive associations among HRH investments, four key treatment service indicators and health impact on the burden of HIV, TB and malaria. In addition, pilot tests from LMICs support the feasibility of using the tool to assist with HRH impact assessment in the context of grants supported by global health initiatives. Limitations are discussed. The feasibility and applicability of the workforce impact assessment tool depend on the availability of HRH output data (e.g. the number of new health workers entering the workforce and the existing number of health workers trained, remunerated, incentivized or otherwise supported). Programmes that wish to assess the impact of their HRH investments should consider strengthening their monitoring and reporting systems to capture such data more systematically. The workforce impact assessment tool can be part of a suite of tools and approaches to assess the broader health and socioeconomic benefits of health workforce investments. vi Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes 1. Introduction The WHO Global strategy on human resources for health: workforce 2030 (1) encourages development partners and global health initiatives to leverage their support to health systems in countries to sustainably strengthen the health workforce. The Global Fund makes substantial investments in HRH. In order to assess the results of the Global Fund’s catalytic investments to “incentivize use of country allocations for strategic priorities… contributing to resilient and sustainable systems for health”, a methodology was developed and pilot tested by WHO. The tool consists of a calculator with two subsets of MS Excel files (web annexes A and B): A. Lives saved calculator B. Coverage target calculator See contents table for links to the web annexes. The objectives of the tool presented in this paper are to: • estimate the health impact of HRH investments made in the context of HIV, TB and malaria programmes through their modelled effect on health service coverage of these three diseases; and • provide aggregate indicative estimates of the range of density of health workers required to attain high coverage of selected health services. The research questions guiding the analysis, accordingly, were the following: • To what extent and through which pathways does investment in HRH improve health outcomes through improved coverage of HIV, TB and malaria services? • What are the health workforce requirements to allow scaled-up coverage of HIV, TB and malaria services? 1 Introduction 2. Methods 2.1. CONCEPTUAL FRAMEWORK A deterministic, user-friendly MS Excel calculator was developed to calculate associations between HRH investment inputs and effects on reduced morbidity and mortality. Conceptually, three distinct steps (Fig. 1) are envisaged for assessing the causal pathway and modelling the correlations between: • investment in HRH (through education, deployment, remuneration of health workers) and improved availability, distribution and productivity of HRH; • improved availability and performance of HRH and its effect on coverage of selected services related to HIV, TB and malaria; and • improved service coverage and reduced morbidity and mortality. Fig. 1. Conceptual model for assessing the health impact of HRH investment Inputs • Pre-service education • In-service training • Full salary remuneration • Salary pay raises • Incentives (e.g. pay for performance) Outputs • Improved HRH availability • Improved HRH performance Outcomes Increased service coverage for • HIV • TB • Malaria Impact • Reduced mortality • Reduced incidence/ prevalence HRH: human resources for health; TB: tuberculosis. The conceptual model was created to estimate the effects of five main modalities of HRH investment (Fig. 2) with the potential to improve the availability of health workers – in terms of full-time equivalent (FTE) health personnel – to deliver services and/or their productivity. These modalities are as follows: • HRH availability: Investments can help to increase the number of health workers or hours worked to deliver services. 1. Pre-service education: Augmenting inflows into the health workforce from health education programmes, factoring in the need for remuneration to support employment of new graduates. 2. Employment: Increasing the number of qualified health workers to be recruited (or finding opportunities for self-employment) in the health sector to add directly to the active health workforce stock. 3. Increased remuneration: Higher salaries to reduce attrition (e.g. leaving the country or going to work in another sector). 2 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes • HRH productivity: Salary increases in the form of incentive pay and in-service training can enhance the productivity of the available pool of health workers. 4. Incentive schemes: Offering financial rewards for workers who meet a specific target or goal, such as through pay-for-performance schemes. Under the right circumstances, such rewards have the potential to incentivize more effective and efficient care. 5. In-service training: Providing training courses and educational resources to workers. As an element of multipronged strategies, training can add to their skills set and job-related knowledge, thus contributing to the quality and relevance of the care they offer. This document describes how these pathways are operationalized in the calculator, the empirical basis for the assumptions made on the magnitude of associations used, and how associations between HRH density and service coverage were estimated. The document also provides an illustrative example for how the calculator works for each of the different HRH investment options. Fig. 2. HRH investment pathways to availability and performance Full salary for graduates Hiring Salary pay raise Financial incentives In-service training Pre-service education Current stock Added stock Density Service coverage Health impact ProductivityEntry of newly trained HCWs Employment of trained HCWs Retention among trained HCWs Investment 1 Investment 2 Investment 3 Investment 4 Investment 5 Availability Productivity HCWs: healthcare workers; HRH: human resources for health. 2.2. STRUCTURE OF THE MODEL AND KEY ASSUMPTIONS The model is built on a number of methodological and data assumptions, which are explained below for each of the steps of the conceptual framework depicted in Fig. 1. This section illustrates the main features, evidence base and assumptions of the model linking HRH investments to improved availability, distribution and performance of HRH. A full list of these assumptions is provided in Annex 1. STEP 1: INVESTMENT IN HRH THROUGH EDUCATION, DEPLOYMENT, REMUNERATION OF HEALTH WORKERS AND IMPROVED AVAILABILITY, DISTRIBUTION AND PERFORMANCE OF HRH. a) Investment 1: Pre‑service education It is assumed that any HRH investments occur above and beyond existing levels of investment; associations between investment options and HRH density or performance were based on the peer-reviewed literature. Few studies examined the association between the number of entrants in health 3 Methods education institutions and the number of graduates. No quantitative study was found that provided a systematic estimate of the graduation rate among health education institutions in LMICs or the rate of employment of new graduates. A default graduation rate of 80% was set for students enrolled in pre-service education programmes. Among these graduates, the model postulates that additional full-time employment salary support (as compared to pre-existing employment levels) is needed to increase the number of graduates who go on to be fully employed as health workers (above and beyond steady-state inflows); the tool allows modification of these parameters if context-specific empirical data are available. b) Investment 2: Hiring Simply increasing the employment of already-trained health workers in the health sector will add to the HRH workforce. It is assumed that each new job employs one (assuming an 8-hour workday). c) Investment 3: Salary raises Increasing the salary of currently employed health workers is one possible approach, among others, to retain health workers. The base wage is excluded because the incremental increase is conceptualized to target currently employed workers. Hiring additional workers via full salary support is reflected in Investment 2. Literature on the impact of wages on labour (2, 3) includes quantitative analyses such as discrete choice experiments (DCEs). Remuneration was included as an intervention in most of the DCE studies (n = 24). Most DCE studies in LMICs find that remuneration is a statistically significant factor for encouraging healthcare workers to relocate to, or remain at, specific locations that may otherwise have fewer desirable characteristics (see Table B2 in Annex 2). We examined published studies that provided an estimate translatable to several outcomes: (i) retention in the country, (ii) willingness to work in poorly equipped facilities,1 (iii) remaining at a specific location (e.g. rural) or position and (iv) relocation to a rural/remote location. Retention of the workforce in the healthcare sector or the country was the most relevant outcome for estimating the effect of salary pay raises on HRH availability. Many of the DCE studies calculated the willingness-to-pay (WTP) value that health workers were willing to trade for other job characteristics. In the context of HRH, the WTP value provides a monetary value of how much it is worth for healthcare workers to stay in or leave their current position, thus affecting the labour supply. Using the reported WTP value, the WTP can be calculated as a percentage of workers’ salary, which can be interpreted as the increase in resources needed to retain a healthcare worker among those in the study that took up the offer. Three studies (from LMICs that included doctor and/or nurse/midwife occupational groups) met the inclusion criteria (see Annex 2 for further details) (4, 5, 6). However, only one of the three studies – i.e. Rockers et al. (5) – reported the fraction of study participants willing to take the pay raise, which enabled calculation of an estimate (Example 1). Example 1. Rockers et al. (5) reported that given a 71% increase in pay, 46% of medical students reported being willing to work in a lower- quality facility. Dividing the increase in pay that would incentivize working in a poorer-quality location by the percentage found to accept the incentive converts this association to FTEs: 71% / 46% = 1.54. This quotient is interpreted as meaning that a 54% increase in salary is necessary to retain 1 FTE. Given the limited number of studies, the estimates across income categories and across occupational groups were consolidated to generate a single overall estimate that can be applied to all groups. 1 In qualitative interviews conducted to identify attributes for DCEs, this outcome has been consistently found as a salient characteristic influencing individual decisions to leave or remain in a country. 4 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes d) Investments 4 and 5: Incentives and in‑service training A recent meta-review on the effects of health worker interventions on HRH productivity was conducted by Rowe et al. (2018) (7). This study is the most comprehensive review conducted to date that is directly relevant to the objective of obtaining elasticities related to performance outcomes. Rowe et al. presented study results for improving healthcare provider practice expressed either as percentages (e.g. percentage of patients treated correctly) or as continuous measures (e.g. number of medicines prescribed per patient). The summary measure to aggregate measured effect sizes across studies was the median effect size (MES). The MES values from the included studies were derived from 670 reports from 337 peer-reviewed publications, including 118 types of strategies, many of which included multiple intervention components. In studies that focused on either financial incentives or training as the sole intervention, the MES calculated provides information on the intervention’s impact on specific outcomes. Studies reviewed that describe the effect of financial incentives or in-service training on HRH performance (Investments 4 and 5, respectively) tended to evaluate incentive packages or policies which involved a combination of different intervention approaches. For example, Perez-Cuevas et al. (8) examined the effect of the interventions of group problem solving, supervision, other management techniques and training conjointly. The MESs reflect the change in performance outputs in response to the presence of any training or financial incentives. Table 1 summarizes these estimates and the number of studies reviewed on which they were based. The information is from Rowe et al. (2018) (7) and the associated supplementary appendix, and Rowe et al. 2019 (9) and the Health Care Provider Performance Review online database (10). Table 1. Studies examining the association between remuneration or training only, and performance Intervention Number of studies Percentage point difference Median of MES values (expressed in percentage) Interquartile range Minimum Maximum Training 82 11.5 8 3.1–18.4 –21.3 68.1 Financial incentives 2 1.2 25.9 N/A 11.1 40.7 MES: median effect size; N/A: not applicable. Composite estimates in the column for percentage point difference were used for Investments 4 and 5. Proceeding with the aggregate MES, the following additional assumptions were made: • Service coverage was defined broadly as improvement in service delivered; the variable is not disease specific. • Performance was defined as the increase in correct behaviour and treatments administered (or alternatively as the reduction in incorrect or inefficient behaviour and treatments administered). • Any resulting performance improvement was linearly additive onto the HRH service coverage that results from HRH density. Example 2. Given that the MES for training on performance corresponds to 11.5 percentage points, this increase can be multiplied by the fraction of the workforce targeted and added to the existing service coverage level: 50.0% service coverage + (11.5 percentage points * 10% of workforce targeted) = 51.15% new total service coverage. Because service coverage is not specific to disease according to the empirical basis used to estimate these coefficients, the assumptions on increase in coverage adopted in the model are the same for HIV, TB and malaria services. 5 Methods e) Multiplier for community healthcare workers Estimates of the empirical association between health worker density and service coverage for HIV/AIDS, TB and malaria are based on actual cross-country data on density of skilled health professionals, and specifically of the workforce of doctors, nurses and midwives (DNMs, see Chapter 4). Cross-country data for community healthcare workers (CHWs), however, are less complete and thus insufficient to use to separately estimate an empirical relationship with service coverage. This was a notable limitation of the available data: future work could benefit from refinement of methods as more data on additional occupational groups become more widely available. To estimate how HRH investments on CHWs can affect treatment service coverage and lives saved (and vice versa), a productivity multiplier was used, reflecting the ratio of the productivity of a doctor or nurse/midwife to CHW. A literature search identified studies that provided suitable estimates of this relative productivity (see Annex 2). As an example, one study found that when tasks were shared between DNMs and CHWs, compared to solely being performed by DNMs, the follow-up use of antihypertensive medication increased from 46.3% to 49.7% (11). Another study found a similar change in the TB treatment completion rate from 64.3% to 83.0% when CHWs were added to the skills mix (12). Two systematic literature reviews on cost–effectiveness analyses (CEAs) and optimal skills mix (13, 14) were helpful in developing specific estimates for comparing changes in effectiveness of a team of CHWs and DNMs, as compared to DNMs providing care alone. The changes in effectiveness between the two groups were then translated into changes in overall productivity for each intervention, allowing calculation of a productivity multiplier for converting DNM to CHW equivalents. Four CEA estimates were extracted from the literature which enabled conversion of DNM productivity into CHW productivity (see Excel sheet “X.CHW_calc” in the Lives Saved and Coverage Target calculators and Annex 2). The multiplier was calculated by taking the change in service coverage percentage points given the involvement of CHWs, and assuming that the change is due to the increase or decrease in productivity of CHWs compared to DNMs (15).2 The descriptive statistics for the multipliers are included in Table 2 (individual study estimates are provided in Table B1 in Annex 2). Given the range of estimates, the median of the productivity ratios was used as the multiplier. Table 2. Descriptive statistics for the ratio of productivity of DNMs to CHWs Observations Median Min Mean SD Max 4 1.1065 1.020 1.105 0.090 1.187 CHWs: community health workers; DNMs: doctors, nurses and midwives; Max: maximum; Min: minimum; SD: standard deviation. Example 3. If the HRH investments focus on CHWs, the model converts this input into the DNM equivalent. The multiplier of 1.179 is interpreted as an 18% gain in team productivity for one added CHW (relative to DNM) when that CHW is added to a team of DNMs. STEP 2: RELATIONSHIP BETWEEN HRH DENSITY AND INCREASED COVERAGE OF SELECTED SERVICES RELATED TO HIV, TB AND MALARIA. The relationship between DNM density and HIV, TB and malaria service coverage was estimated using the data on HRH density and service coverage available to WHO as of January 2019. These data include DNM workforce stock, with country-level United Nations population estimates serving as the denominator. For each country, DNM density was calculated by summing the separate densities of these three occupational groups for the most recently available year. Over 75% of countries recorded DNM data from 2014 to 2016. DNM data for countries with missing information were not imputed for the purposes of the estimation strategy. 2 For example, for estimates from Okello et al. (15) the productivity multiplier was calculated by dividing the TB treatment success for DNMs (74%) by that when CHWs were added to the skills mix (56%), which yields the multiplier of 1.32. 6 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Given the Global Fund’s focus on addressing the burden of HIV, TB and malaria in LMICs, the search of treatment service indicators focused on those with non-missing data since 2010 (for at least one data year) for at least 50 LMICs. The 29 candidate indicators (18 for HIV, 5 for TB and 6 for malaria; see the “Candidates” tab in the Excel file for the Lives Saved calculator) represent the full set of candidate indicators for which data are publicly available. These data sets were acquired from various sources, including the World Bank Health, Nutrition and Population Statistics data. This repository compiles several sources of data from WHO, UNAIDS and other international organizations. Many of these candidate indicators gauge incidence or prevalence of conditions, and thus were discarded as unrelated to the conceptual framework and purpose of this model, which focuses on treatment service indicators. a) Primary analysis of service coverage indicators Eleven of the 29 candidate indicators were examined using ordinary least squares (OLS) regression (see the “Z.MD_only” and “Z.NM_only” tabs in Lives saved calculator: web annex A). This framework assumes a curvilinear, or “diminishing returns”, relationship between DNMs and HIV, TB and malaria treatment coverage. In this approach, the greatest increases in treatment service coverage are associated with initial investments in DNMs. At a certain level of DNMs, however, there is an inflection point at which the rate of increase in treatment service coverage is smaller with each additional DNM investment. First, the relationship between each of the 11 candidate indicators (dependent variables) and the natural logarithm of the number of DNMs per 1000 population (independent variable) was estimated. In the base specification, the number of observations was the total number of LMICs analysed. This analysis yielded four treatment service indicators with a positive and statistically detectable (i.e. P < 0.05) association with DNM concentration (see Table 3): • antiretroviral therapy (ART) coverage (percentage of people living with HIV); • percentage of pregnant women with HIV who receive antiretroviral medicine for prevention of mother-to-child transmission (PMTCT); • TB treatment coverage: the number of new and relapse TB cases per number of incident cases; and • the percentage of children < 5 years with fever who sought treatment at any facility (a proxy indicator of treatment for malaria). Each treatment service indicator serves as a separate dependent variable in a separate regression. When each of the four regressions was restricted to LMICs only, all but the first HIV indicator (ART coverage for people living with HIV) showed a positive and statistically detectable association. However, when using the full country data set without any income-based country restrictions, all four indicators – two for HIV, one for TB and one for malaria – showed a positive and statistically detectable association. The full-country results of each of the four OLS regression analyses appear in Table 3. Note that the number of countries analysed differed for each indicator, owing to varying data availability for treatment service indicators across countries. Table 3. Four separate regression results for four treatment service indicators Indicator* No. of countries DNM coefficient SE P value Intercept ART coverage for HIV 129 5.30 2.20 < .001 41.70 ART coverage for PMTCT 104 7.42 2.27 .002 60.27 TB treatment coverage 201 9.22 1.30 < .001 63.59 Children < 5 sought treatment for fever 70 6.05 1.8 .002 61.86 ART: antiretroviral therapy; DNM: doctors, nurses and midwives; PMTCT: prevention of mother-to-child transmission; SE: standard error; TB: tuberculosis. *Each treatment service indicator serves as the dependent variable in that respective regression; the natural logarithm of DNMs per 1000 population is the independent variable in each regression. 7 Methods These parameters allowed estimation of treatment service coverage for a particular DNM density. For example, for TB treatment coverage a few levels of DNMs per 1000 population, corresponding to a specific level of treatment service coverage, are provided for purposes of illustration (Table 4). Table 4. Sample levels of TB treatment service coverage by value of DNM concentration DNM concentration per 1000 population Estimated TB treatment coverage 1.0 63.59 1.5 67.33 2.0 69.98 2.5 72.03 4.0 76.37 DNM: doctors, nurses and midwives; TB: tuberculosis. The regression equation results illustrated in Tables 3 and 4 were then used to estimate the gains of treatment service coverage associated with investments in DNMs. Users may also identify lower and upper confidence bound estimates of “lives saved” for additional coverage of each treatment service indicator. These confidence bounds use the standard errors of coefficient estimates (Table 3) to derive 95% confidence intervals of lives saved. b) Benchmarking alternative A complementary function of the model and calculator developed entails a possibility for the user to set a predetermined level of treatment service coverage for HIV, TB and malaria and then find the level of aggregate workforce requirements to attain that service coverage. The benchmarking alternative entails use of empirically derived maximum “caps” as well as minimum “floors” (and the range in between the cap and floor) of general DNM concentrations that correspond with existing treatment service coverage levels. The maximum cap was derived by identifying the median level of DNMs observed for countries that meet the service coverage target for that particular indicator (e.g. 10.85 DNMs for the 12 countries which exceeded 90% treatment coverage for TB). The minimum floor of DNMs, by contrast, was created using results from a data envelopment analysis (DEA), which ranks countries according to their efficiency in delivering treatment service coverage per level of DNMs. DEA is a statistical analysis tool often applied to microeconomics and operations management. In this context, the tool has been applied to identify countries that maximize the utility of existing resources to achieve a desired end. Specific details for the DEA approach are covered in Annex 3. Once the DNM maximum cap and floor levels for each treatment service indicator had been determined, a log-linear function was fitted through these points (given the economic principle of diminishing returns to additional DNM, especially at higher levels of DNMs). c) Other approaches attempted but not further utilized Several other approaches to estimating the relationship between investments in DNMs and treatment service coverage for HIV, TB and malaria were attempted. These approaches included developing a composite treatment service metric, a disability-adjusted life-year (DALY)-weighted composite estimate, and expanding treatment service coverage indicators beyond the four items listed in Table 3. None of these approaches, however, were deemed to be more robust statistically or more user-friendly than the approaches described earlier in the primary analysis (section a) and the benchmarking alternative (section b). Methodological details that were attempted, including regression results, appear in Annex 3. 8 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes d) Accounting for comorbidity In countries with a relatively high incidence of HIV and TB, there is a dual burden from HIV/TB coinfection (16). Despite the comorbidity associated with HIV/TB coinfection, many coinfected patients remain undetected and undertreated (17). In light of this challenge, WHO outlined guidelines and policy recommendations for enhancing collaborative HIV/TB activities (18). Based on empirical estimates available in the studies quoted in this subsection, estimates of HIV/TB coinfection and HIV/TB detection rates were integrated when estimating how HRH investments could augment treatment service coverage across these two conditions. In a study from a setting with a high HIV burden, among HIV-positive persons who had screened positive for TB at the time of the study, only 6.3% were receiving TB therapy (19). This study underscores the sometimes low level of integration of HIV and TB control activities, but indicates that persons who test positive for HIV may also undergo additional TB screening. It is therefore assumed that, in a high HIV/TB coinfection context, additional treatment of, say, 100 HIV-positive persons would increase TB screening for ~6.3 TB-positive persons (i.e. 6.3%). Next, the increase in TB service treatment coverage attributable to an increase in HIV treatment coverage was calculated for countries with lower HIV/TB incidence. Estimates of HIV/TB incidence were taken from the Global Burden of Disease project (20). It was assumed that countries with a lower HIV/TB coinfection rate would necessarily have a slightly lower “cross-benefit” for TB detection among persons undergoing HIV treatment. This cross-benefit in TB detection ranged from 3.3% to 6.3% and was automatically integrated into the final TB service treatment coverage percentage. e) Interpretation Table 3 provides estimates of the relation between DNM density and treatment service coverage for four treatment service coverage indicators referring to specific HIV, TB and malaria services. Given the instability of the HIV ART treatment indicator results in the alternative approaches, which exclude high-income countries and have a relatively low sample size (n = 62; see Annex 3), further stratification of our regression approaches by country characteristics (e.g. low-income countries – LICs – only) attenuates the DNM/treatment coverage associations or renders them nondetectable. The tool can be modified depending on the specific aspects of a country’s situation. For example, countries without endemic malaria may wish to incorporate an assumption that HRH investments are not related to malaria burden in that country. This context-specific aspect, as well as the “high or low HIV burden” button, can thus be factored into modelling estimates. STEP 3: TRANSLATING SERVICE COVERAGE INTO HEALTH IMPACT. The OLS regression analysis described earlier (Step 2) yielded four treatment service coverage indicators for HIV, TB and malaria that show a positive association with DNM density: • ART coverage (percentage of people living with HIV); • percentage of pregnant women with HIV who receive antiretroviral medicine for prevention of PMTCT; • TB treatment coverage: the number of new and relapse TB cases per number of incident cases; and • the percentage of children < 5 years with fever who sought treatment at any facility (an indicator of treatment for malaria). These four indicators were used to model how improvements in service coverage, via HRH investments, may translate into lives saved. Relevant empirical data from the literature were reviewed to derive estimates of lives saved. 9 Methods a) Lives saved due to increased ART coverage A study reported trends in HIV incidence, ART coverage and mortality statistics from the 30 countries with the greatest AIDS mortality burden (21). Based on results using data from the Spectrum/EPP software AIDS Impact Module, estimates of lives saved per unit increase in ART coverage were derived. The study reports results for two countries that account for 27% of the global HIV burden. The first country reported a 20% ART coverage rate for persons with HIV. If this country were to maintain its current level of ART coverage, the researchers estimated 485 564 fewer AIDS-related deaths over a 7-year period from 2014 to 2020. This estimate equates to an average of 69 366 lives saved per year associated with 20% ART coverage. Given this estimate, a 1 percentage point increase in ART coverage could avert 3468 deaths per year (i.e. 69 366 / 20). If these lives saved were scaled to the population size of that country, a 1% increase in ART coverage would correspond to 1.89 lives saved per 100 000 population. If, instead, we used figures for the second country that were generated by the AIDS Impact Module, the reported ART coverage of 42% would correspond to a projected 1 363 201 fewer AIDS-related deaths from 2014 to 2020 (21). This projection equates to an average of 194 743 lives saved per year. A 1% increase in ART coverage, therefore, could avert 4637 deaths (i.e. 194 743 / 42). If these lives saved estimates were scaled to the population size of the second country, a 1% increase in ART coverage would thus correspond to 8.33 lives saved per 100 000 population. If we take the average number of AIDS-related lives saved per 100 000 population (in the first and second country) for a 1 percentage point increase in ART coverage, which assumes additivity of health benefits per unit increase in treatment coverage given low HIV incidence: (1.89 + 8.33) / 2 = 5.11 lives saved per 100 000 population These two countries, however, have different incidence and prevalence rates of HIV. These epidemiologic factors, as well as other drivers of regional variation in HIV burden and treatment, likely explain the different estimates in lives saved per 100 000 population per unit increase in ART coverage. However, the estimate of 5.11 lives saved per 100 000 population can represent a starting point in estimating the health impact of increasing ART coverage in the LMIC context. In a context of low HIV burden, the literature reports a lower number of lives saved per 100 000 population relative to the high HIV burden context. Specifically, using Latin American countries classified as middle-income, one report finds that for every 1 percentage point increase in ART coverage, there are 0.3 fewer AIDS-related deaths per 100 000 population (22). This estimate is also used in the calculator for a “low HIV burden” context option. b) Lives saved due to ART to prevent mother‑to‑child transmission WHO estimates that in the absence of ART, ~30% of mothers transmit HIV to their infants (with a range of 15–45%) (23). With ART for PMTCT, this transmission rate can be reduced to below 5% (23). In 2014, 21 priority sub-Saharan African countries provided ART to 77% of pregnant women living with HIV (24). Among these women, mother-to-child HIV transmission fell to 9% (24). Using the estimates above, in the absence of ART for PMTCT, for every 100 HIV-positive pregnant women, ~30 cases of HIV-positive infants are estimated. An estimated 50% of children living with HIV die before their second birthday (24). In the absence of ART for PMTCT, for every 100 HIV- positive pregnant women, there would be 15 child deaths (i.e. 50% * 30 cases). With ART for PMTCT, for every 100 HIV-positive pregnant women, ~5 children would be HIV-positive, of which 50% (i.e. 2.5) are estimated to die prematurely. Based on these calculations, the number of HIV-related child deaths averted for every 100 additional HIV-positive women on ART for PMTCT would be 12.5 (i.e. 15 child deaths without ART for PMTCT minus 2.5 child deaths in the presence of ART for PMTCT). The model can input country- specific ART for PMTCT coverage levels to estimate child lives saved per percentage point increase in ART for PMTCT coverage. 10 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Example 4. We use one sub-Saharan country as an example for performing the calculation. This country has about 1.5 million births per year (25). Researchers estimate that the country has an HIV prevalence of 6.2% for women aged 15–49 years (26). If we assume that the likelihood of having a child is unrelated to a woman’s HIV status, then 93 000 HIV-positive women give birth per year. The country reports 80% ART coverage for PMTCT, which suggests that 74 400 HIV-positive women received treatment. If the country were to augment PMTCT treatment by 1 percentage point (i.e. to 81%), an additional 930 mothers would be covered. According to the assumptions made, this additional coverage would translate to 116 (i.e. 930 * .125) additional infant lives saved in that country in that year. c) Lives saved due to TB treatment coverage A report by the Global Fund (27) includes the following assumption for lives saved due to TB treatment coverage: Currently it is assumed that one life is saved for every three cases of TB that are treated, based on the difference in the case fatality ratio of treated and untreated TB. The 3 : 1 ratio could be applied to country-level estimates of total TB cases that are treated. For example, Sri Lanka shows 64% TB treatment coverage and ~10 000 detected cases per year (28). Using these inputs, the total estimate of all TB cases is 15 625 (i.e. 10 000 / 0.64). Sri Lanka’s stated TB treatment success rate is 85%. Given these inputs, the current TB treatment coverage (i.e. 10 000 identified cases * 0.85) leads to 8500 successfully treated cases. If we increase TB treatment coverage by 1 percentage point, this increase would show 8633 successfully treated cases – or an additional 133 (i.e. 8633 – 8500) cases treated. Application of the 3 : 1 ratio to these 133 newly treated cases yields 44.3 lives saved per year in Sri Lanka associated with a 1 percentage point increase in TB treatment coverage. d) Lives saved due to seeking cure for fever among children under 5 WHO’s World malaria report (29) found that in sub-Saharan Africa, 36% of febrile children are not brought to treatment facilities for care. Based on this statistic, 64% of children sought treatment for a fever at any facility. Moreover, the proportion of febrile children who received a malaria diagnostic test in the public sector rose from 29% in 2010 to 51% in 2015, yielding a percentage increase of (51 – 29) / 29 = 76% (29). If a child seeks treatment for fever at a facility in LMICs, it is assumed that the child would receive a diagnostic test. Over the same time period, under-5 mortality due to malaria declined by 35% (29). A decline in malaria is assumed to be related to treatment seeking for fever. Given these parameters, every 1 percentage point increase in children under 5 with fever seeking treatment at any facility was assumed to correspond with a 0.46% decline (i.e. 35 / 76) in malaria mortality for under-5 children. The model can factor in estimates of malaria mortality for under-5 children in 1 year and multiply them by 0.0046 (i.e. 0.46%) to estimate children’s lives saved for each 1 percentage point increase in children seeking care for fever. e) General considerations on inputs needed and assumptions of the model Empirical literature was used to derive estimates of lives saved due to increased HIV, TB and malaria treatment service coverage. The gains in service coverage for four specific treatments are positively associated with DNM concentration, which supports the notion that investments in DNMs result in additional lives saved. The lives saved calculations, albeit omitting more granular information on the health services landscape and the dynamics of HIV, TB and malaria, rely on a set of relatively straightforward epidemiological assumptions. For each indicator, Table 5 illustrates the additional lives saved by a 1 percentage point increase in coverage, the input(s) needed by the model to estimate lives saved and the key assumptions for that estimate. As the table shows, the number and level of complexity of the inputs required by the model are modest. We remind the reader of the important caveat that correlation estimates do not represent causal relationships. Rather, the model illustrates the potential health impact that may be realized 11 Methods from additional HRH investments. Including the lower and upper bound estimates of lives saved further underscores the uncertainty inherent in these estimates. Table 5. Summary of lives saved estimation approach for HIV, TB and malaria Indicator Lives saved Country-level inputs needed for the model Key assumptions • ART coverage for HIV • 5.11 lives saved per 100 000 population for each 1 percentage point increase in ART coverage • Population size (optional: HIV prevalence) • Effectiveness of ART coverage is similar to average effectiveness in countries that the underlying studies refer to. • ART coverage for PMTCT • 12.5 child deaths averted for every 100 additional HIV-positive women on ART for PMTCT • Percent of HIV+ pregnant women who receive ART for PMTCT • Effectiveness of ART coverage for PMTCT is similar to that of 21 sub-Saharan countries. • TB treatment coverage • 1 life saved for every 3 TB cases treated • Estimated number of all TB cases (detected) • TB treatment success rate • TB treatment success rate remains fixed, regardless of level of TB treatment coverage. • Children < 5 years sought treatment for fever • 0.46% decline in count of < 5 years malaria deaths for each 1 percentage point increase in treatment seeking • Number of malaria-related deaths for children < 5 years • If a child seeks treatment for fever at a facility, the child receives a diagnostic test for malaria. ART: antiretroviral therapy; PMTCT: prevention of mother-to-child transmission; TB: tuberculosis. 12 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes 3. Results 3.1. ILLUSTRATIVE EXAMPLES OF IMPACT ESTIMATES Several illustrative examples based on the methods and assumptions described in this document help to clarify how the deterministic model and calculator work. • Examples 5–7 begin with investment inputs as a starting point. • Example 8 illustrates the functionality for calculating the HRH availability required to achieve a target service coverage level. These examples are based on a fictitious lower-middle-income country with a population of 40 million, a current DNM density of 1.85 per 1000 population and ART coverage of 64%. In addition, this example is performed only for ART coverage rather than the larger set of service coverage indicators. Nevertheless, the same incremental increase in resulting service coverage would apply to other indicators. The following context-specific assumptions of elasticities are adopted (and would ideally be replaced by local empirical data in application of the model to a real- world context): • Investment 1: Assuming a cohort of 400 students, 90% of whom graduate per year, increasing support for new graduate jobs by 80% leads to an additional density of 0.007. • Investment 2: There is a 1 : 1 correspondence between the number of health workers hired and those added to the workforce. • Investment 3: A 54% increase in remuneration leads to retaining 1 FTE. • Investment 4: Increasing remuneration leads to a performance increase of 1.2 percentage points. • Investment 5: Providing in-service training leads to a performance increase of 11.5 percentage points. Example 5. Increase entry of pre-service education graduates into the workforce Investment: Increase the number of graduates entering the labour market by supporting 80% more jobs for new graduates. 1. Starting with the number of students going into pre-service education (400) and calculating the numbers who graduate based on a 90% graduation rate: 400 students * 90% graduation rate = 360 DNMs 2. Assuming that the HRH investment provides full salary coverage for an additional 80% of new graduates’ employment in the health sector (funded either by domestic or external sources), the additional DNM density will be: 360 * 80% given full salary coverage / 40 000 000 population * 1000 = 0.007 additional DNM density 3. The investment in pre-service education and 80% full salary coverage for the augmented number of new labour market entrants leads to an additional 0.007 DNM / 1000 population. 1.85 current DNM / 1000 + 0.007 additional DNM / 1000 = 1.857 DNM / 1000 4. The increase in service coverage is then calculated according to the predicted values resulting from the estimated marginal effect from the individual regression of DNM concentration with HIV ART service coverage (Table 3). In other words, the estimated slope and intercept from the regression equation are used to predict the service coverage level given a certain concentration of DNMs. This calculation is done for the 13 Results beginning level of DNM concentration (1.85 DNM / 1000) and the ending level of DNM concentration (1.857 DNM / 1000). The difference in resulting ART coverage is taken to determine the incremental improvement in service coverage resulting from the investments. 44.96 + 5.30 * ln(1.85) = 48.22 44.96 + 5.30 * ln(1.857) = 48.24 48.24 – 48.22 = 0.02 percentage point increase in ART coverage 5. The additional service coverage increase from performance is then added to the service coverage level resulting from increases in HRH density, to obtain the final total increase in service coverage. 64.00% Current ART coverage level 0.02% Gain from pre-service education 64.02% Total ART coverage level achieved 6. To convert the additional service coverage into lives saved, the percentage of additional ART coverage is multiplied by the estimation of lives saved and the country’s population: 0.02 percentage point coverage gain * (5.11 additional lives saved / 100 000 population) * 40 000 000 population = 40.9 additional lives saved in that country 7. The confidence interval estimates for total lives saved are calculated using a lower bound intercept and a higher bound intercept to calculate coverage changes and then adding the value to the new coverage. Total lives saved: 64.02 * (5.11 additional lives saved / 100 000 population) * 40 000 000 population = 130 856.9 total lives saved in that country Lower bound: 42.31 + 5.30 * ln(1.857) = 45.59 lower bound ART coverage 45.59 – 48.24 = –2.65 (64.00 + –2.65) * (5.11 additional lives saved / 100 000 population) * 40 000 000 population = 125 399.4 total lives saved in that country Upper bound: 47.61 + 5.30 * ln(1.857) = 50.94 upper bound ART coverage 50.94 – 48.24 = 2.70 (64.00 + 2.70) * (5.11 additional lives saved / 100 000 population) * 40 000 000 population = 136 334.8 total lives saved in that country 14 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Example 6. Increase health worker salaries Investment: Increase salary pay by 10% for 10 000 DNMs. 1. The identified estimate for Investment 3 is 1.54 (i.e. a 54% increase in salaries leads to retaining 1 FTE). As such, the additional FTEs gained from the cohort of 10 000 DNMs are calculated as: 10% / 1.54 for each 1 FTE * 10 000 DNM = 649 additional FTEs 2. The total FTE change is then divided by the total population of that country to arrive at the incremental increase in DNM density. 649 FTEs / 40 000 000 population * 1000 = 0.016 DNM / 1000 increase 3. The increase in service coverage is then calculated per the predicted values resulting from the estimated marginal effect from the individual regression of DNM concentration with HIV ART service coverage (Table 3). The estimated slope and intercept from the regression equation are used to predict the service coverage level given a certain concentration of DNMs. This step is performed for the beginning level of DNM concentration (1.85 DNM / 1000) and the ending level of DNM concentration (1.863 DNM / 1000). The difference in resulting ART coverage is calculated to determine the incremental improvement in service coverage resulting from the investments. 41.70 + 5.30 * ln(1.850) = 44.97 41.70 + 5.30 * ln(1.866) = 45.00 45.00 – 44.97 = 0.03 percentage point increase in ART coverage 4. For Investment 4 (financial incentive), the MES for pay for performance reported by Rowe et al. (2018) (7) is an increase of 1.2 percentage points in performance. We perform this calculation assuming that 5000 workers (i.e. 10% of the current workforce) are targeted for in- service training. 1.2 percentage points * 10% targeted * 1.86 DNM / 1000 = 0.002 percentage point coverage increase 64.00% Current ART coverage level 0.03% Gain from salary pay raise investment 0.002% Gain from incentive investment 64.032% Total ART coverage level achieved 5. To convert the additional service coverage into lives saved, the percentage of additional ART coverage is multiplied by the estimation of lives saved and the population of Kenya: 0.032 percentage point coverage gain * (5.11 additional lives saved / 100 000 population) * 40 000 000 population = 65.4 lives saved 15 Results Example 7. Provide in-service training Investment: Provide in-service training to 10 000 CHWs. 1. For Investment 5 (in-service training), the MES for in-service training reported by Rowe et al. (2018) (7) is an increase of 11.5 percentage points in performance. The estimates from the literature indicate that one CHW has 1.179 times the productivity of one DNM. This calculation is conducted assuming that 20% of the HRH are targeted for in-service training. 11.5 percentage points * 1.179 productivity * 20% targeted * 1.86 DNM / 1000 = 0.05 percentage point coverage increase 2. The additional service coverage increase from performance is then added to the service coverage level resulting from increases in HRH density, to obtain the final total increase in service coverage. 64.00% Current ART coverage level 0.05% Gain from investment in in-service training 64.05% Total ART coverage level achieved 3. To convert the additional service coverage into lives saved, the percentage of additional ART coverage is multiplied by the estimation of lives saved and the population. 0.05 percentage point coverage gain * (5.11 additional lives saved / 100 000 population) * 40 000 000 population = 102.2 lives saved Example 8. HRH numbers needed to achieve 80% ART coverage Question: What is the number of additional HRH needed to achieve 80% ART coverage? 1. Based on the results from the DEA benchmarking alternative calculation (see “Estimators” tab in the Coverage target calculator: web annex B), the DNM density needed to achieve 80% ART coverage is 4.56 DNM / 1000. If the country’s current ART coverage is 64% and they have 1.85 DNM / 1000, the additional DNM density corresponding to 80% ART coverage can be calculated as: 4.56 DNM / 1000 – 1.85 DNM / 1000 = 2.71 additional DNM / 1000 needed 2. This estimate of additional workers needed is based on the DEA results and the log-linear relation between DNM and HRH investments (see Annex 3). The equation then multiplies the concentration by the estimated population (40 000 000). The product is the total stock of DNMs needed. (DNM / 1000) * 40 000 000 = 108 400 DNMs needed 16 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes 3.2. COUNTRY CASE STUDIES To understand the impact of the Global Fund’s HRH investments on treatment outcomes for HIV, TB and malaria, and to test the methodology and applicability of the model and calculator described in this document, WHO tested the calculator tool through case studies based on data from real grants awarded by the Global Fund to countries. Global Fund Secretariat staff and, where possible, officials from principal recipients of Global Fund grants and governments from five countries were requested to provide grant-specific information and health workforce and epidemiological data to test the functionality of the calculator. These case studies were used to pilot the methodology and achieve the following goals: • provide country-specific estimates of the value of specific HRH investments to lives saved across HIV, TB and malaria; • provide country-specific estimates of the DNM concentration needed to attain desired target treatment coverage levels for HIV, TB and malaria services; • assess the feasibility and user-friendliness of the calculator in terms of ease of data input and interpretation of output; and • identify and correct gaps in the functionality of the tool. The case studies were desk-based analyses using secondary data available in administrative records. The choice of countries for testing the model was made jointly by the Global Fund and the WHO Secretariat. The choice was informed by the volume and nature of HRH investments made in Global Fund grants and by the availability of relevant programmatic and administrative data. The Global Fund Secretariat, in collaboration with relevant principal recipient officials managing the grants, provided the necessary HRH and health data to analyse the health impact of HRH investments. Where possible, relevant government officials were requested to validate the data and estimates provided. Results from the five case studies for Malawi, Pakistan, Lesotho, the Philippines and Zambia follow. The feasibility analysis and subsequent modifications to the calculator have been incorporated into the final tool. Case Study: Malawi Malawi is a low-income country in south-eastern Africa with a population of 18.1 million. The current DNM density is 0.25 per 1000 population. Malawi faces a considerable HIV, TB and malaria burden. The number of pregnant women who received ART for PMTCT is 47 148 according to the latest available data. Yearly TB cases are estimated at 29 000, and the number of malaria-related deaths estimated at 8090. The workforce investments supported by the Global Fund in Malawi include pre-service training for doctors (n = 30) and nurses/midwives (n = 48), full remuneration of new hires (n = 1509) and in-service training (n = 1432). These investments span across the DNM workforce and CHWs. The modelled estimates suggest that investments supported by the Global Fund in HRH result in 162 additional lives saved across the four treatment coverage areas. But the highest proportion of lives saved appear to occur through increased coverage of ART for HIV treatment. If Malawi’s goal was to increase treatment service coverage by, for example, 5%, the additional DNM density needed would range from 0.51 to 2.87 DNMs per 1000 population depending on the specific treatment service provided. Increasing the total number of health workers in Malawi represents a crucial long-term investment to improve service treatment coverage rates and reduce the burden of HIV, TB and malaria. 17 Results Case Study: Pakistan Pakistan is a lower-middle-income country in the WHO Eastern Mediterranean Region. It has a population of 189.4 million people and a current DNM density of 0.32 per 1000 population. Pakistan faces the challenge of a substantial HIV, TB and malaria burden. The yearly number of pregnant women who receive ART for PMTCT is 532, and (notified) TB cases are estimated at 334 742. Malaria is endemic, and the number of malaria-related deaths is estimated at 138 per annum. The workforce investments currently supported by the Global Fund in Pakistan include pre-service training for CHWs (n = 51), doctors (n = 25), nurses and midwives (n = 20), full remuneration of new hires (n = 254) and salary pay raises (n = 1844). These investments span across medical personnel, the nursing and midwifery workforce, support personnel and CHWs. The modelled estimates suggest that HRH investments supported by the Global Fund result in approximately 367 additional lives saved across the four treatment coverage areas. But the highest proportion of lives saved appear to occur through increased coverage of ART for HIV treatment. Pakistan has a goal of increasing treatment service coverage by 9%, 28%, 23% and 13% for ART for HIV, and ART for PMTCT, TB and children under 5 seeking treatment for fever, respectively. The additional DNM density needed will range from 0.09 to 7.03 DNMs per 1000 population depending on the specific treatment provided for the aforementioned service areas. Increasing the total number of HRH workers in Pakistan represents a crucial long-term investment to improve service treatment coverage rates and reduce the burden of HIV, TB and malaria. Case Study: Lesotho Lesotho is a lower-middle-income country in southern Africa with a population of about 2 million people, and a current DNM density of 0.72 per 1000 population. The level of HIV burden is high in Lesotho, and the number of pregnant women who received ART for PMTCT is 8065. TB cases are estimated at 13 000. Malaria is not endemic, and the number of malaria-related deaths is estimated at 0. The workforce investments supported by the Global Fund in Lesotho include pre-service training for doctors (n = 21), nurses/midwives (n = 1) and CHWs (n = 1); full remuneration of new hires (n = 101); salary pay raises (n = 88); in-service training (n = 351) and incentives (n = 3284). These investments focus primarily on the nursing and midwifery workforce and CHWs. The modelled estimates suggest that HRH investments supported by the Global Fund result in 31 additional lives saved across treatment coverage areas in HIV and TB (with no lives saved in malaria treatment, as Lesotho has eliminated malaria). If Lesotho’s goal was to increase treatment service coverage by 5%, the additional DNM density needed would range from 0.0 to 0.77 DNMs per 1000 population depending on the specific treatment service provided. Increasing the total number of HRH workers in Lesotho represents a crucial long-term effort to reduce the burden of HIV and TB. 18 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Case Study: Philippines The Philippines is a lower-middle-income country in South-East Asia with a population of 93.7 million and a current DNM density of 4.61 per 1000 population. The Philippines faces a considerable TB burden; yearly TB cases are estimated at 581 000. The level of HIV burden is low, and the number of pregnant women who received ART for PMTCT is 67. Malaria is endemic in rural archipelagos but not in major cities, and the number of malaria-related deaths is estimated at 34. The workforce investments currently supported by the Global Fund in the Philippines include full remuneration of new hires (n = 572), in-service training (n = 4314) and incentives (n = 138). These investments span across the DNM workforce and CHWs. The modelled estimates suggest that HRH investments supported by the Global Fund result in 23 additional lives saved across the HIV and TB treatment coverage areas (with no additional lives saved in malaria treatment).The highest proportion of lives saved appear to occur through increased coverage of ART for HIV treatment. If the goal of the Philippines was to increase treatment service coverage by 5%, this could be achieved within the existing availability of health workers. The tool indicates that attainment of high targets of specific treatment coverage indicators may be obtained by more efficient use or distribution of existing workers, without necessarily increasing the total number of health workers. Case Study: Zambia Zambia is a lower-middle-income country in south-central Africa with a population of 16.6 million and a current DNM density of 0.54 per 1000 population. Zambia is challenged by a substantial burden of HIV, TB and malaria. The yearly number of pregnant women who received ART for PMTCT is 56 543, and TB cases are estimated at 62 000 annually. Malaria is endemic, and the number of malaria-related deaths is estimated at 7618 per annum. The workforce investments currently supported by the Global Fund in Zambia include pre-service training only for CHWs (n = 216), full remuneration of new hires (n = 557) and salary pay raises (n = 557). These investments span across the nursing and midwifery workforce, clinical officers and CHWs. The data indicate that no investment has been made in doctors. The modelled estimates suggest that HRH investments supported by the Global Fund result in 176 additional lives saved across the four treatment coverage areas. But the highest proportion of lives saved appear to occur through increased coverage of ART for HIV treatment. If Zambia’s goal was to increase treatment service coverage by 5%, the additional DNM density needed would range from 0.45 to 2.42 DNMs per 1000 population depending on the specific treatment service provided. Increasing the total number of HRH workers in Zambia represents a crucial long-term investment to improve service treatment coverage rates and reduce the burden of HIV, TB and malaria. 19 Results 4. Discussion 4.1. LIMITATIONS The correct interpretation of the tool and its use requires an understanding of its main limitations. The associations which underpin the calculator tool should not be interpreted as causal relations. Other methodological approaches (e.g. simultaneous equation modelling – see, for example, Scheffler et al.) (30) may be better equipped to estimate causal effects. Such approaches would, however, require HRH and health data with finer geographic and temporal resolution – but with much less country coverage – than the publicly available data sets we analysed. A number of simplifying assumptions (see Annex 1) had to be made to build the model. For instance, we assumed that new HRH investments do not affect existing HRH dynamics (e.g. inflows from immigration, outflows from departures or retirements). Another important limitation lies in the design of the tool itself. The tool attempts to model effects only through coverage indicators referring to HIV, TB and malaria services for which statistically significant associations can be found. However, the tool does not estimate the positive effects of HRH on other service delivery areas or on HIV, TB and malaria services for which no empirical correlation can be found between HRH density and service coverage. Therefore, the estimates of lives saved through HRH investments are in all likelihood an underestimate, as only the effects through some service coverage indicators were modelled. The development of estimates of the existence and strength of associations between HRH density and service coverage was constrained by HRH data availability. Only data on the density of skilled health professionals (the DNM workforce were available, whereas CHW data were unavailable for a sufficient number of countries to derive similar estimates of an empirical relationship. While this was addressed by identifying a “CHW multiplier”, additional data on CHWs or other additional occupational groups could lead to more reliable estimates of the associations. The estimation of coefficients for the tool relies on global population, service coverage, health workforce density and programme-specific data estimates, which are regularly updated. This tool was developed using data that were available at the time the analysis was conducted; some of the underlying data (e.g. the World malaria report and the Global Health Estimates), therefore, were no longer up to date by the time of publication. The effects of changes in individual data used in the modelling on the calculation of the coefficients and on the overall results reported are nevertheless negligible; the estimates in any case are not meant to be interpreted as entirely accurate but mostly to serve an illustrative purpose. Attempts were made to develop uncertainty bounds for the estimates. However, the combination of methods adopted, and the underlying evidence and data available, meant that this was not ultimately possible. In particular, application of the delta method is constrained by the fact that standard errors and MESs were only available for a minority of the parameters of interest built into the modelling approach. Attempting to run the delta method with the parameters and data available yielded errors or results that were difficult to interpret. An alternative approach entailed an attempt to estimate confidence intervals using the regression parameters. It is possible to develop uncertainty bounds for the overall numbers of lives saved, but not for the estimate of lives saved specifically through HRH investments/interventions, which is the focus of this model. The calculation of additional lives saved relies on the estimated slope and intercept from the regression equation used to predict the service coverage level, given a certain DNM concentration. Additional lives saved are calculated by taking the difference in service coverage between the previous concentration and increased concentration. If the lower bound intercept and the upper bound intercept are used to conduct the calculation, the resulting difference in service coverage will not vary (because the slope is unchanged). The same service coverage numbers will then result in the number of additional lives saved for the lower and upper bound, since the correlation coefficient r is the same as the point estimate. 20 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes For these reasons, the model had to assume that the relations between the many health worker inputs and treatment service outputs are measured with no uncertainty. This simplifying assumption permits calculation of point estimates in terms of additional lives saved, for example, but does not provide confidence bounds for each point estimate. The inability of the tool to calculate uncertainty bounds represents an area of enquiry for future research and development of methodological adaptations relevant to this purpose. Given that each of our estimates is measured with uncertainty, we caution the reader against overinterpreting results from the calculator. We also note that a better understanding of these sources of uncertainty, and identification of strategies to increase precision in parameter estimates, represents an important avenue for improving workforce calculators. Such estimates of uncertainty, for instance, could include the delta method. The delta method incorporates the statistical variability of each of the various steps of the calculator and integrates this uncertainty across all steps to yield a summary measure. Future work that refines this and other calculators may benefit from exploring the use of the delta method to create credible confidence bounds for HRH investments. Finally, while this tool does factor in different types of HRH investments and different occupational groups, it is not an allocative efficiency tool. For this reason, the tool cannot be used as a guide for which type of HRH investment or which occupational group (based on a marginal investment logic) should be prioritized. 4.2. DATA REQUIREMENTS The feasibility and applicability of the workforce impact assessment tool depend on availability of HRH output data. This information includes the number of new health workers entering the workforce and the existing number of health workers trained, remunerated, incentivized or otherwise supported by the programme for which an impact assessment is sought. Programmes and development partners, such as the Global Fund, that make a substantial investment in HRH might consider establishing a requirement that reporting of HRH-specific information be included in their routine monitoring and reporting systems if they intend to quantify the health impact of HRH investments based on this methodology. Adopting such a reporting requirement would allow tracking of the results and modelling of the impact of HRH investments, and would enable more precise estimates of the association between HRH inputs and specific treatment service outputs. The programmatic and administrative reporting requirements on HRH could also contribute to strengthening national health workforce information systems by aligning to the methodology and data standards of WHO’s National health workforce accounts (31). 4.3. POLICY IMPLICATIONS The development of the workforce impact assessment tool, which stems directly from an integrated conceptual model, demonstrates the feasibility of estimating improvements in service treatment and lives saved for HIV, TB and malaria deriving from investments in HRH. The integration of the literature, as well as novel empirical results, indicates positive associations among HRH investments, four key treatment service indicators and health impact on the burden of HIV, TB and malaria. In addition, pilot tests from LMICs support the feasibility of using the tool to assist with HRH impact assessment in the context of grants supported by global health initiatives. By strengthening a core element of health systems in an integrated manner, investments in HRH have the potential to achieve positive results for a range of health services and health outcomes, beyond the three diseases we examined. Future research may explore the application of our methodology to other health service areas. This impact assessment tool can be relevant in a suite of broader tools and approaches to assess the benefits of health workforce investments. It is also acknowledged that HRH investments have broader positive development outcomes, including through the creation of qualified employment opportunities, particularly for women, spurring sustainable economic growth and contributing to gender empowerment. These other dimensions are of critical importance and might be the subject of future research endeavours and alternative or complementary methodologies. 21 Discussion Annex 1. Assumptions of the calculator OVERVIEW The calculator includes options for specifying HRH investments for four main investment types: • pre-service education aimed at increasing the number of health workers entering the healthcare labour market from medical training programmes; • salary pay raises, broadly defined as any change in base compensation (wages, benefits) given to existing health workers; • incentives (e.g. pay for performance) provided for meeting target performance outputs to increase the productivity of individuals privy to these schemes; and • in-service training for existing health workers to increase the skill and quality of services a given health worker can produce. Conceptually, pre-service education and salary pay raises are thought to affect the HRH labour market on the extensive margin, or the availability of HRH workers to be employed in a health service delivery capacity. This includes increasing the number of health workers who may enter the labour market either by retaining (through salary pay raises) health workers in service delivery positions who would otherwise have exited, or by encouraging employed workers to put in more hours for those who may be suboptimally employed (and thus could be measured in terms of fractions of FTEs). Incentives and in-service training investments can affect the HRH labour market on the intensive margin, or the productivity (services delivered per worker) of labour. While theoretically in-service training (viewed as an occupational benefit) may also influence health workers’ decision to join the health workforce labour market or increase their hours worked on the extensive margin, our search of the literature found no empirical data that could be used to estimate this relationship (see Annex 2, section d). ASSUMPTIONS 1. Investments are made on top of existing HRH workforce dynamics. It was assumed that the labour market operates at a steady state in which inflows (newly graduated workers or immigrants) and exits (due to death, departure or retirement) occur as they historically have occurred. 2. The labour market in LMICs is currently operating at full capacity. Many countries in LMICs do not have enough demand to support a larger HRH workforce (32). As such, any additional workers to be employed will need to be fully supported for compensation, whether for new graduates from medical education (above and beyond steady-state inflows) or for retaining workers in their current position. 3. The user can input the number of workers targeted for the investment. To calculate the association between HRH investments and treatment service coverage, the starting point for calculating the impact of investments will be the number of health workers targeted by occupational group and, for the case of pre-service education and salary pay raises, the relative size of investment to be made (i.e. 10% increase in salary, increasing the number of graduates to be supported in the labour market by 20%). 4. The calculator tool will not estimate the cost implications of the investments. The costs of HRH investments will vary considerably across country and health system contexts. In addition, it is assumed that the salary support given to health workers is discounted over the full lifetime of each worker. 5. Investment effects will be static. While all investments will take time to implement and to take effect in the health system, the estimates are simplified to reflect the result after all effects have worked through the system. 6. The main effects of different HRH investment options will be additive. If multiple investment types are chosen, resulting estimates of FTEs will be assumed to be linearly additive in a steady-state labour market. 22 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes 7. Estimates will be stratified by occupational group (DNMs and CHWs) and country income level (low, lower-middle and upper-middle) to the extent supported by the empirical literature. Because associations of HRH investments with density and performance may vary substantially by occupational group and the depth of the labour market for health workers across income levels, calculations will be sensitive to occupational group targeted by country income level to the extent possible. Note that given the data requirements of using all countries with available data (including those beyond LMICs) to estimate the relation between HRH density and service coverage levels, identified associations reflect an average across all country income levels. 8. Estimates for CHWs will be based on a productivity multiplier. Data on the availability of CHWs across countries is not as readily obtainable as it is for DNMs. The peer-reviewed literature will therefore be used to estimate CHW performance using a productivity multiplier (see Annex 2, section a). For countries without CHW stock data, the median ratio of DNM density to CHW density will be used, calculated from the set of countries for which data for all three health worker categories are available. 9. Estimates for clinical officers (COs), a default option for an additional category of healthcare workers, will be based on a productivity multiplier. This productivity ratio is estimated from a study which calculates sub-Saharan Africa CO salaries relative to nurse salaries (i.e. US$ 1915 / US$ 1865 = 1.026) (33). By default, the productivity ratio of COs relative to nurses and midwives is 1.03. The number of COs will be multiplied by the productivity ratio and then folded into the total number for nurses and midwives. The lives saved from additional COs will therefore be calculated in terms of nurses and midwives. 10. Associations between investment options and HRH availability or productivity will be based on aggregated estimates from the empirical literature. Estimates from different empirical sources are standardized for use in the calculator. 11. HRH availability and productivity are the intermediate outputs that feed into service coverage outcomes and, ultimately, into estimates of impact in terms of lives saved. Based on a review of global HRH data, past experience and a literature search on empirical estimates of association elasticities to be estimated by the calculator, estimates for HRH availability and productivity measures can plausibly be calculated across country contexts. Although other measures of HRH distribution have been developed, they have not been consistently calculated and used across settings and require significantly more data (e.g. comparable subnational units) in order to compute. Thus, HRH distribution measures are excluded from the set of HRH investment outcomes. 12. Service coverage increases resulting from HRH productivity investments will not be occupation specific. The optimal skills mix for producing health services, whether specific to HIV, TB, malaria or more generally, is unknown and likely to be highly specific to health systems structures in different countries. Additional assumptions would need to be made to identify the relative attribution of service coverage increases due to HRH productivity across health workforce categories. Furthermore, no assumptions or limits are made about scale economies, the marginal rates of substitution between capital and labour inputs or the pathway for labour market expansion in the production function for health. 13. There are diminishing returns to HRH availability investments on service coverage outcomes. This relationship is reflected in the log functional form transformation used in the regression analysis for Step 2 (see Chapter 3). The assumption of diminishing returns is not applied to service coverage increases resulting from HRH productivity investments, as there is no empirical basis for this from the existing literature. 14. The empirical relationship between HRH density and service coverage is based on regression results that aggregate DNMs. While regression estimates were analysed for doctors and nurses/midwives separately, the resulting statistical significance of these relationships did not hold as strongly across service coverage indicators as analyses that combined these groups. 15. The relationship between HRH inputs and HRH availability and productivity are measured without uncertainty estimates. Given that a key goal of the calculator involves arriving at a point estimate of lives saved due to HRH investments, we used the peer-reviewed literature as well as regression-based parameter estimates to derive the quantitative inputs. Whereas we acknowledge that each of these inputs is measured with error, for ease of interpretation of the calculator tool we did not apply formal uncertainty analyses (e.g. the delta method) to the standard errors of each input. Rather, we used only the standard error estimates from the regressions in Table 3 (of health workers and treatment service coverage for HIV, TB and malaria) to arrive at lower and upper bounds of total lives saved in the calculator tool. 23 Annex 1. Assumptions of the calculator Annex 2. Targeted literature review to identify empirical estimates for specific assumptions in the model a) CHW productivity While estimates of the empirical relationship between health worker density and service coverage for HIV/AIDS, TB and malaria were based on actual cross-country data on density for DNMs, cross-country data for CHWs were less complete and insufficient for separately estimating an empirical relationship with service coverage. This is a notable limitation of the available data to date, and an area where this work can benefit from refinement in the future as more data on additional occupational groups become systematically available. To be able to estimate how HRH investments in CHWs affect health outcomes, a productivity multiplier was used that reflects the ratio of the productivity of a doctor or nurse/midwife to the combined productivity of CHWs and a doctor or nurse/midwife providing collaborative care. A literature search was conducted to identify studies that provided suitable estimates of this relative productivity. The literature review searches were performed using the WHO and International Labour Organization’s definition of a CHW: Community health workers provide health education and referrals for a wide range of services, and provide support and assistance to communities, families and individuals with preventive health measures and gaining access to appropriate curative health and social services. They create a bridge between providers of health, social and community services and communities that may have difficulty in accessing these services (34). Given the definition, the terms “community healthcare worker”, “community health worker”, “lay health worker”, “community-based health worker” and “task shifting” were searched. Search results indicated that there is an expansive literature examining the efficacy of CHWs in providing healthcare. To identify estimates that would allow us to convert the productivity between DNMs and the collaborative care provided by nurses, midwives and CHWs, the literature review focused on quantitative studies that examine task sharing. Many of the studies in the task-sharing literature involved simulation and estimation of DNM and CHW stock (35), which are not applicable for this project. However, two relatively recent systematic literature reviews were identified on studies that conducted CEA on CHW involvement and task sharing (13, 14). The studies included in these two systematic literature reviews were evaluated for reliable estimates of the effectiveness of CHW, because CEA studies require specific estimates on the effectiveness of the intervention in question and the comparator. Estimators identified specifically measured sharing of tasks between doctors, nurses and midwives, or DNMs and CHWs (as opposed to DNMs shifting the tasks in their entirety to CHWs). The focus on these estimates enabled comparison of the changes in effectiveness of DNMs and CHWs providing collaborative care, compared to only DNMs providing care. The changes in effectiveness between the two groups were then translated into changes in overall productivity in each task-sharing intervention, thus enabling direct comparison of relative output among the groups. 24 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Of the CEA studies reviewed, those with the following criteria were included: • The intervention focused on task sharing between CHWs and professionally trained healthcare workers (i.e. DNMs). • CHW involvement was the only part of the intervention. In other words, no other interventions were examined simultaneously. • Outcome variables could be translated to productivity, focusing on service coverage (e.g. TB cure rate, capacity of TB treatment programme). • All the estimates provided and used in the CEA resulted from primary data collected by the study, rather than from secondary data analysis (e.g. in simulations) on previously published estimates. In one of the studies included in the systematic literature review (38), the primary data were reported in a separate study. Therefore, that study was reviewed to extract the estimates, which were then translated to productivity for this analysis. • The estimates represent statistically significant differences in service coverage between the task-sharing group and the doctors, nurses and midwives or DNM-only groups. Consequently, the exclusion criteria included the following: • studies that only examined changes in prevalence and did not report service coverage outcomes; • studies that had costs but no estimates of effectiveness; and • studies that used estimates generated from other studies. If the target of HRH investments is CHWs and users enter the number of CHWs to be targeted, the number of CHWs is then converted to their DNM equivalent (unobserved to the user). The subsequent calculations then proceed according to the estimates used for DNMs. Table B1 presents the study estimates, converted productivity multipliers and the final median productivity multiplier included in the Lives saved calculator. Table B1. Studies that quantitatively estimate the relative productivity of DNMs to CHWs Citation Outcome measure Performance outputs Multiplier (DNMs/CHWs) DNMs CHWs Buttorff et al. (36) Reduction in psychiatric symptoms 39.9% 67.5% 1.02 Dick (12) Treatment completion 64.3% 83.0% 1.187 Jafar et al. (11) Medication use 46.3% 49.7% 1.034 Sabin et al. (37) Death averted – 17.9 / 1000 1.179 Median 1.1065 CHWs: community health workers; DNMs: doctors, nurses and midwives. b) Studies on pre‑service education There is an absence of studies which examine the relationship between the number of medical professional school enrollees and the number of graduates in LMICs. Among quantitative studies assessing interventions that increased the number of enrollees and graduates in medical education, none were found that generated a usable number for this project. The literature on pre-service education contains mostly qualitative and prescriptive studies outlining the need to train and retain HRH in low-income settings. One study examined Ethiopia’s “flood and retain” policy for increasing the supply of HRH by building professional schools (38). The qualitative interviews conducted in the study show that the government had increased the number of students, but not the number of teachers, equipment and other resources. In general, these studies highlight the need to ensure that there are enough resources to recruit and retain HRH; simply increasing pre-service education is insufficient (39). These findings underscore 25 Annex 2. Targeted literature review to identify empirical estimates for specific assumptions in the model the importance of including provisions for eventual full health worker salary support for any HRH investment in pre-service education that aims to augment the inflows of workers into the HRH labour supply. In addition, these studies provide no relational estimates between enrolment, graduation and DNM density, most likely due to the lack of transparency and systematic data collection (40). A study surveyed all identified medical schools in sub-Saharan Africa and found that 81% of responding schools indicated that they had no tracking system for their graduates and could not determine whether they were practicing medicine or where (41). Consequently, there is a lack of data to determine the relationship between healthcare-related professional school enrolment and its resulting effect on labour force inflows. One study conducted by WHO collected data on the number of new graduates in 12 countries in Africa (42). This study provides the most comprehensive information on the number of qualified HRH personnel in sub-Saharan Africa; but the information is somewhat outdated and still not comprehensive enough for use in this exercise. c) Studies on salary pay raises and retention There is an extensive literature on the labour market and the impact of wages on labour supply. Nevertheless, most studies that apply labour market theories on HRH in LMICs are theoretical (2, 3) and thus do not provide usable empirical estimates for this project. Furthermore, studies which calculated specific elasticity estimates for healthcare workers in LMICs are scarce. In terms of quantitative analysis, only a few studies (43) have generated estimates of the relationship between remuneration and training and retention from regression models. As is evident in findings from numerous systematic literature reviews (44, 45, 46), studies examining the relationship between remuneration and training and retention and performance suffer from several limitations: • Many studies are purely descriptive, that is, survey- and questionnaire-based studies which highlight the importance of remuneration and training but provide no estimates on the relationship between variables (47). • The primary outcome of focus is often the satisfaction of healthcare workers with their career and position (48). These outcomes are not connected to retention or performance and cannot be translated into elasticity estimates. A broad literature search and review showed the type of quantitative analysis most useful in the context of this project to be findings from DCEs in LMICs. For this reason, our subsequent literature search focused on DCE studies. The following terms were searched individually and in combination: “remuneration”, “wages”, “healthcare work force”, “human resources for health”, “doctors”, “nurses”, “supply-oriented interventions”, “DCE”, “training”, “development”, “retention” and “low- and middle-income”. Posters and PowerPoint presentations were excluded, as studies presented in these formats do not provide adequate information for us to determine the usability and reliability of the estimates generated. Published studies of the effect of HRH investments on HRH density were examined to identify estimates translatable to several outcomes: (i) retention in the country, (ii) willingness to work in poorly equipped positions, (iii) remaining at a specific location (e.g. rural) or position and (iv) relocation to a rural/remote location. Retention in the healthcare sector or the country is the most relevant outcome for extracting an estimate of the effect of salary pay raises on HRH availability via retention among currently trained HCWs. In qualitative interviews conducted to outline attributes for DCEs, the outcomes for willingness to work in poorly equipped positions consistently emerged as a salient characteristic influencing individual decisions to leave or remain in a country. Theoretically, these various outcomes can be translated into an equivalent number of FTEs, which can then be used to calculate HRH density (further explanation follows). To ensure the relevance of the findings and their applicability to the current HRH context, estimates in our final model were restricted to studies published between 2008 and 2019. Within the included studies, estimates of the relationship between remuneration and training and retention generally surveyed current healthcare workers or students in medical or nursing schools in LMICs. Many of the DCE studies also calculated the WTP value that healthcare workers were willing to trade for other job characteristics. The WTP measurement is useful, as it provides a monetary value for how much a person is willing to pay for a given good or experience or to avoid an undesired outcome. In the context of HRH, the WTP value provides a monetary value of how much 26 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes it is worth for a healthcare worker to stay in or leave his/her current position, thus affecting the DNM labour supply. The WTP value is calculated as the percentage of the average salary of each occupational group, which indicates the additional resources needed to retain a healthcare worker. This body of literature is the most applicable for extracting estimates on how increases in remuneration and training may lead to increased HRH density. The included estimates are presented in Table B2. Table B2. Studies examining the association between salary pay raises and willingness to work in poorly equipped facilities Doctors Nurses and midwives Low Three studies: • Ethiopia: 27% increase in base salary leads to selection of remote post. • Ethiopia: 26% increase in base salary leads to staying in condition with poorer equipment. • Malawi: Two times current monthly salary leads to selection of work in rural areas. • Uganda: Additional USh 1 million (US$ 426) (1.43 times the base salary) per month leads to a 46% increase in preference for poorer-quality facilities. Six studies: • Ethiopia: 72.2% increase in base salary leads to selection of remote post. • Ethiopia: 50% increase in base salary leads to staying in conditions with poorer equipment. • Malawi: MWK 10 000 (US$ 10) increase in salary leads to increase in choosing rural post by 50% point change. • Tanzania: 80–100% increase in salary leads to selection of rural post. • Tanzania: TZS 57 151 (US$ 25) leads to selection of remote post. • Uganda: Additional USh 1 million (US$ 426) per month leads to a 17% increase in preference for poorer-quality facilities. Lower middle Three studies: • India: 33% increase in salary increases the likelihood of doctors accepting a rural post by 13% point change. • Kenya: 20% increase in salary increases the likelihood of doctors accepting a rural post by 22.8% point change. • Viet Nam: Doctors are willing to pay 7.04 million VND (US$ 303) to be located in an urban post. Three studies: • India: INR 10 000 (US$ 140) increase in salary increases the likelihood of nurses accepting a rural post by 31% point change. • India: 2.5 times current salary leads to 61% point change in acceptance of rural positions. • Indonesia: IDR 7 million (US$ 503) increase in salary leads to selection of remote post. Upper middle Five studies: • China: 10.8% of current income leads to increased likelihood of doctors choosing to work in less well-equipped health centres. • China: Doctors are willing to pay CNY 4020 (US$ 496) to be located in an urban post. • Iran: 36.5% salary increase (or additional US$ 730) leads doctors to be willing to work in poorer facilities. • Peru: PEN 1000 (US$ 300) increase/month increases the odds ratio of staying in a rural post by 2.82. • Thailand: 45% increase in salary leads to selection of rural post. Four studies: • China: 8.1% of current income leads to increase in the likelihood of nurses choosing to work in less well-equipped health centres. • China: CNY 3000 (US$ 444) increase in income leads to 75.2% of nurses choosing a rural job. • Peru: PEN 1000 (US$ 300) increase/month increases the odds ratio of staying in a rural post by 2.95. • South Africa: ZAR 250 000 (US$ 18 475) annual salary leads to choosing to remain at current position. There were three studies (from LMICs that included doctor and/or nurse/midwife occupational groups) that met the inclusion criteria (see Table B3 for further details) (4, 5, 6). However, only one of the three studies actually reported the fraction of study participants willing to accept and stay for the pay raise offered – data required to calculate an estimate and make the study usable. 27 Annex 2. Targeted literature review to identify empirical estimates for specific assumptions in the model Table B3. Included studies on salary pay raise and retention Study % salary increase % of respondents willing to accept Estimator Hanson and William (4) 26% for D, 50% for NMW – Rockers et al. (5) 71% for NMW 46% 1.54 Song et al. (6) 11% for D, 8% for NMW – D: doctor; NMW: nurse or midwife. Rockers et al. (5) reported that given a 71% increase in pay, 46% of medical students would be willing to work in a lower-quality facility. Dividing the increase in pay that would incentivize working in a poorer-quality location by the percentage found to accept the incentive converts this association to FTEs: 71% / 46% = 1.54. This quotient is interpreted as a 54% increase in salary being necessary to retain 1 FTE. Given the limited number of studies, the estimates across income categories and across occupational groups relied on one overall estimate (1.54), which can be applied to all groups. d) Studies on in‑service training and retention A literature search was conducted to identify DCE studies that include in-service training and outcomes related to retention. Given the few studies that focus on in-service training, the content, duration, cost or frequency of in-service training is not considered further.3 A review of the available empirical estimates showed that there are few to no studies that report any results relating in-service training to worker retention. Thus, the empirical basis for this investment pathway is not supported by the current state of the literature. 3 Training can vary substantially across settings, health conditions and occupational groups. However, most studies do not provide enough information to characterize the content and intensity of training in order to make more specific estimates of these attributes. 28 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Annex 3. Approaches to estimating the relationship between HRH density and service coverage a) DEA for the benchmarking alternative Data envelopment analysis is a nonparametric estimation strategy for identifying a country’s efficiency in covering treatment services at particular levels (49). DEA borrows from tools in economics and operations management to identify groups that maximize the utility of their existing resources to achieve a desired end. In our scenario, countries serve as the “group” and DNMs per 1000 are the “inputs” that produce treatment service coverage for HIV, TB and malaria. Countries are then ranked by DEA according to an efficiency score, which is calculated by the successful attainment of a certain threshold of treatment coverage per DNM worker in that country. DEA has been used by WHO in past reports to provide specific DNM benchmarks for attaining desired treatment coverage targets (50). Performed using the “dea” command in STATA, DEA ranks all countries according to efficiency (51). DEA was applied separately for each of the four treatment service coverage indicators. The 60% coverage level was chosen as a level of coverage at the medium range. The DEA then identified the top 20 countries as exemplary in their efficiency for attaining > 60% service coverage. The 20 countries included represented the Americas, Africa, and South-East Asia and the Western Pacific regions. If the mean DNM value is taken from this list of 20 efficient countries (separately for each treatment coverage, given that countries may be efficient in, say, TB coverage but not in HIV coverage), the following benchmark levels of DNMs are produced at a treatment coverage level of 60%: • ART coverage for HIV: 1.54 DNMs • ART coverage for PMTCT: 0.64 DNM • TB treatment coverage: 0.77 DNM • Children < 5 sought treatment for fever: 0.71 DNM. Although these numbers are lower than previously published reports of desired DNMs, results cohere with the notion that DEA seeks efficiency and ranks countries favourably if they attain treatment service coverage with relatively fewer health workers. With these DEA-derived DNM benchmarks at 60% treatment coverage, a log-linear function was fit through this point and the “cap” DNM level for 99% treatment coverage which was empirically identified previously (by taking the median DNM level of existing countries that attained > 90% treatment service coverage). The log-linear function implies a curvilinear relation between DNM and treatment service coverage, such that after a certain level of DNM investments, there are diminishing returns to increases in treatment service coverage. This cap was set to make it logically impossible for a country to achieve DNM levels higher than the level that the highest DNM countries in the present day have attained. In addition, it is assumed that no country could have a negative DNM (at low levels of treatment service coverage). Results of this approach appear in the figures below (the x-axis is DNMs per 1000 population, and the y-axis is treatment coverage percentage). Note that the curves show diminishing returns at the higher end of treatment coverage, provide specific DNM values across the entire distribution of treatment coverage and do not show any gross discontinuities (or jumps) in DNMs as coverage percentage is increased. 29 Annex 3. Approaches to estimating the relationship between HRH density and service coverage If this logic is applied to the “Coverage Target” tab, for a country with fewer DNMs than indicated by these benchmark DNM levels for a particular treatment coverage percentage, the country’s desired value would move up to the DNM benchmark. Alternatively, if that country already has more DNMs than indicated by the benchmark, the “Coverage Target” tab does not suggest a higher level of DNMs than the country has already attained. HIV ART 0 2 4 6 8 10 12 14 DNMs/1000 0 10 20 30 40 50 60 70 80 90 100 % C ov er ag e ART: antiretroviral therapy; DNMs: doctors, nurses and midwives. HIV PMTCT 0 1 2 3 4 5 6 DNMs/1000 0 10 20 30 40 50 60 70 80 90 100 % C ov er ag e DNMs: doctors, nurses and midwives; PMTCT: prevention of mother-to-child transmission. 30 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes TB treatment 0 2 4 6 8 10 12 14 DNMs/1000 0 10 20 30 40 50 60 70 80 90 100 % C ov er ag e DNMs: doctors, nurses and midwives; TB: tuberculosis. Malaria treatment 0 0.5 1 1.5 2 2.5 3 DNMs/1000 0 10 20 30 40 50 60 70 80 90 100 % C ov er ag e DNMs: doctors, nurses and midwives. 31 Annex 3. Approaches to estimating the relationship between HRH density and service coverage b) Aggregation of four treatment service indicators into a single metric The “composite regression method” was explored as a method of integrating the four regression results from Table 3 (i.e. one for each treatment service indicator shown to have an empirical association with DNM concentration) into a single metric. The benefit of the single metric is that one can produce a summary indicator of overall increase in treatment service coverage for HIV, TB and malaria associated with investment in DNMs. This approach has intuitive appeal, as DNMs do not typically treat single conditions to the exclusion of other conditions. Thus, a single composite index may better capture the healthcare reality in which treatment service coverage gains occur concurrently across HIV, TB and malaria as a result of augmenting DNM investments. Four ways were explored for estimating the association between a composite treatment service coverage metric and DNM concentration. 1. Continuous index, unweighted To create the composite index, the percentage attainment of the four treatment coverage indicators (Table 3) for each country was summed. For example, one country shows the following treatment coverage: • ART coverage (percentage of people living with HIV): 64%; • percentage of pregnant women with HIV who receive antiretroviral medicine for PMTCT: 80%; • TB treatment coverage: the number of new and relapse TB cases per number of incident cases: 45%; and • percentage of children < 5 years with fever who sought treatment at any facility (an indicator of treatment for malaria): 27.1%. The sum of these four percentages (i.e. 64 + 80 + 45 + 27.1) for this country produces a composite index score of 216.1. This summation step was used for all countries. Next, the composite score was regressed as a function of log(DNM). In the basic analysis, the composite index is retained as a continuous measure. Results, shown in the “continuous, unweighted” row (Table C1), indicate a positive and statistically detectable (P < 0.05) association between DNM and the composite index. Table C1. Composite regression method results, four treatment indicators combined Summary index n DNM coef SE P value Intercept Composite index (continuous, unweighted) 62 21.60 7.57 0.006 226.02 Composite index (DALY weight) 62 20.52 7.86 0.01 216.97 Composite index (median split, unweighted) 62 0.50 0.17 0.005 1.79 DALY: disability-adjusted life-years; DNM: doctors, nurses and midwives; SE: standard error. 2. Continuous index, DALY weighted In LMICs, the burden of disease due to HIV exceeds that of either TB or malaria. Additional treatment service coverage for ART, therefore, could be considered as reducing more DALYs than would similar gains in treatment coverage for malaria due to children seeking care for fever. To adjust the continuous composite index results by the DALYs of the condition that each treatment service seeks to address, an additional DALY-weighting strategy is provided. 32 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes For LMICs only, specific rows were identified in the WHO Global Health Estimate (GHE) summary tables (52) for HIV, TB and malaria. Table C2 lists the four treatment service coverage indicators, the specific row used in the Global Burden of Disease table and the total DALYs lost in LMICs due to that condition. Table C2. LMICs only: DALYs due to HIV, TB and malaria Treatment coverage GHE cause Estimated DALYs (’000s) • ART overall • ART for PMTCT HIV/AIDS 59 139.70 • TB treatment coverage TB 51 362.70 • Children seeking treatment for fever Malaria 37 368.30 ART: antiretroviral therapy; DALYs: disability-adjusted life-years; GHE: Global Health Estimates; PMTCT: prevention of mother-to-child transmission; TB: tuberculosis. An “analytic weight” was calculated by dividing DALYs for a particular disease by the sum of DALYs from all three diseases (Table C3). Table C3. Analytic DALY weight, by disease Condition Analytic DALY weight HIV 0.40 TB 0.35 Malaria 0.25 DALY: disability-adjusted life-years; TB: tuberculosis. The DALY-weighted composite regression slightly adjusts results from the unweighted regression by “upweighting” the importance of attaining HIV treatment service coverage indicators, and by “downweighting” the malaria treatment service coverage indicator. Here, the TB DALY weight is the “base” weight. HIV treatment indicators are upweighted by a factor of 1.14 (i.e. 0.40 / 0.35), and malaria is downweighted by a factor of 0.71 (i.e. 0.25 / 0.35). Given the use of two HIV treatment indicators, each of these HIV indicators was weighted by 1.07. The TB service treatment indicator remained unchanged (i.e. 0.35 / 0.35, or a DALY weight of 1.0). After applying these analytic weights to each country’s value for the treatment service coverage indicators, the regression equation was re-estimated. Results from the DALY-weighted analysis appear very similar to the unweighted analysis (see Table 3), although the intercept and DNM coefficients are slightly smaller. To assist the reader in contextualizing the composite index regression results from Table C1, Table C4 provides a few fitted values of treatment service coverage for specific levels of DNMs. For example, 2.5 DNMs per 1000 population corresponds with an unweighted composite index of 246 (i.e. an average of 61.5% coverage per each of the four treatment service indicators, since 4 * 61.5 = 246). In addition, raising the DNM level from 2.5 to 4.0 per 1000 population is associated with a 10 unit increase in the unweighted composite index. 33 Annex 3. Approaches to estimating the relationship between HRH density and service coverage Table C4. Sample values of composite index coverage by level of DNMs DNMs per 1000 population Composite index* (unweighted) Composite index* (DALY weighted) 1.0 226 217 1.5 234 225 2.0 241 231 2.5 246 236 4.0 256 245 DALY: disability-adjusted life-year; DNMs: doctors, nurses and midwives. * Values rounded to the nearest integer. 3. Median split In the median split approach, each of the four continuous measures of treatment service indicators was converted into a binary variable: scores above the grand median for LMICs received a 1 and scores below the median received a zero. Next, the four binary indicator scores were summed to produce a composite index. The range of each country’s median split composite index is from zero to 4, with 4 indicating attainment of all four treatment indicators above the country medians. The dependent variable in the OLS regression was specified as the median split composite index score, and the DNM coefficient was the independent variable. Results (last row of Table C1) show a positive association between DNM and the composite treatment service index. However, interpretation of the coefficient is challenging in that movement of the median split index from, say, 3 to 4 does not seem intuitive. Therefore, this analytical model was not retained in the development of the calculator. 4. Expanded number of service coverage indicators The last approach to producing a composite index assumes that increasing HRH investments may show “on the ground” improvements in treatment service coverage rates for HIV, TB and malaria in areas that do not show statistically detectable correlations in isolated OLS regressions using aggregate level data. In this scenario, three candidate treatment service indicators were added to the composite index. These candidate indicators focus on treatment modules for HIV, TB and malaria in LMICs which DNMs may directly provide to patients: • percentage of HIV/TB coinfected population who receive ART; • TB treatment success rate (percentage of new cases); and • malaria: percentage of women aged 15–49 with a live birth who received 2+ doses of sulfadoxine-pyrimethamine (SP/Fansidar). In total, for each country this process sums seven treatment service indicators: three new HIV, TB and malaria indicators, and the four “base” indicators that show empirically robust association with DNM concentration. Regression results from this augmented model are shown in Table C5. The P value indicates that the DNM coefficient for these seven treatment indicators does not reject the null; there is no association between DNM and the expanded composite index of treatment service indicators. In addition, only 38 countries have non-missing values on all treatment service indicators. Imputation of median treatment service values for LMICs with missing treatment indicators produces regression results very similar to those shown in Table C5 with the larger country set. 34 Tool to assess the impact of human resources for health investments on HIV, TB and malaria services and health outcomes Table C5. Augmented composite regression results, seven treatment indicators Summary index n DNM coef SE P value Intercept Composite index (continuous, unweighted) 38 21.27 14.79 0.16 421.76 DNM: doctors, nurses and midwives; SE: standard error. In addition, different transformations of the dependent and independent variables were assessed in the OLS regression. The inverse sine transformation for treatment coverage, previously implemented by WHO (53), was assessed, as well as a square root transformation for treatment coverage and DNM (one at a time). None of these transformations produces a DNM coefficient which rejects the null. Based on this result, there is no empirically supported approach to expanding the list of treatment service indicators beyond the four indicators described in Table 3. Any inclusion of additional treatment service indicators for HIV, TB and malaria would require assumptions that WHO and the Global Fund, given their expertise in this area, are able to make about the relation between DNMs and service coverage. 35 Annex 3. Approaches to estimating the relationship between HRH density and service coverage References 1. Global strategy on human resources for health: workforce 2030. Geneva: World Health Organization; 2016 (https://apps.who.int/iris/bitstream/ handle/10665/250368/9789241511131-eng.pdf, accessed 29 July 2021). 2. Andalón M, Fields G. A labor market approach to the crisis of health care professionals in Africa. IZA discussion paper no. 5483. 2011 (https:// papers.ssrn.com/sol3/papers.cfm?abstract_id=1765648, accessed 19 July 2021). 3. Scheffler R, Bruckner T, Spetz J. The labour market for human resources for health in low-and middle-income countries. Human Resources for Health Observer no. 11. Geneva: World Health Organization; 2012. 4. Hanson K, William J. Incentives could induce Ethiopian doctors and nurses to work in rural settings. Health Affairs. 2010;29:1452–60. 5. Rockers PC, Jaskiewicz W, Wurts L, Kruk ME, Mgomella GS, Ntalazi F et al. Preferences for working in rural clinics among trainee health professionals in Uganda: a discrete choice experiment. BMC Health Serv Res. 2012;12:212. 6. Song K, Scott A, Sivey P, Meng Q. Improving Chinese primary care providers’ recruitment and retention: a discrete choice experiment. Health Policy Plan. 2013;30:68–77. 7. Rowe AK, Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. Lancet Glob Health. 2018;6(11):e1163–75. 8. Perez-Cuevas R, Guiscafre H, Munoz O, Reyes H, Tome P, Libreros V et al. Improving physician prescribing patterns to treat rhinopharyngitis: intervention strategies in two health systems of Mexico. Soc Sci Med. 1996;42(8):1185–94. 9. Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D and Rowe AK. A systematic review of the effectiveness of strategies to improve health care provider performance in low- and middle-income countries: methods and descriptive results. PLoS ONE. 2019;14:e0217617. 10. Health Care Provider Performance Review [website]; 2021 (https://www.hcpperformancereview.org/, accessed 1 March 2021). 11. Jafar TH, Islam M, Bux R, Poulter N, Hatcher J, Chaturvedi N. Cost-effectiveness of community-based strategies for blood pressure control in a low-income developing country: findings from a cluster-randomized, factorial-controlled trial. Circulation. 2011;124:1615–25. 12. Dick J, Clarke M, van Zyl H, Daniels K. Primary health care nurses implement and evaluate a community outreach approach to health care in the South African agricultural sector. Int Nurs Rev. 2007;54(4):383–90. doi:10.1111/j.1466-7657.2007.00566.x. 13. Seidman G, Atun R. Does task shifting yield cost savings and improve efficiency for health systems? A systematic review of evidence from low- income and middle-income countries. Hum Res Health. 2017;15(1):29. 14. Vaughan K, Kok MC, Witter S, Dieleman M. Costs and cost-effectiveness of community health workers: evidence from a literature review. Hum Res Health. 2015;13(1):71. 15. Okello D, Floyd K, Adatu F, Odeke R, Gargioni G. Cost and cost–effectiveness of community-based care for tuberculosis patients in rural Uganda. Int J Tuberc Lung Dis. 2003;7:S72–9. 16. Marais BJ, Lönnroth K, Lawn SD, Migliori GB, Mwaba P, Glaziou P. Tuberculosis comorbidity with communicable and non-communicable diseases: integrating health services and control efforts. Lancet Infect Dis. 2013;13(5):436–48. 17. Duarte R, Lönnroth K, Carvalho C, Lima F, Carvalho ACC, Munoz-Torrico M. Tuberculosis, social determinants and co-morbidities (including HIV). Pulmonology. 2018;24(2):115–9. 18. WHO policy on collaborative TB/HIV activities: guidelines for national programmes and other stakeholders. Geneva: World Health Organization; 2012. 19. Wood R, Middelkoop K, Myer L, Grant AD, Whitelaw A, Lawn SD. Undiagnosed tuberculosis in a community with high HIV prevalence: implications for tuberculosis control. Am J Respir Crit Care Med. 2007;175(1):87–93. 20. Wang H, Wolock TM, Carter A, Nguyen G, Kyu HH, Gakidou E. Estimates of global, regional, and national incidence, prevalence, and mortality of HIV, 1980–2015: the Global Burden of Disease Study 2015. Lancet HIV. 2016;3(8):e361–87. 21. Granich R, Gupta S, Hersh B, Williams B, Montaner J, Young B et al. Trends in AIDS deaths, new infections and ART coverage in the top 30 countries with the highest AIDS mortality burden; 1990–2013. PLoS ONE. 2015:10(7);e0131353. 22. Gonzalez MA, Martin L, Munoz S, Jacobson JO. Patterns, trends and sex differences in HIV/AIDS reported mortality in Latin American countries: 1996–2007. BMC Pub Health, 2011;11(1):605. 36 23. Mother-to-child transmission of HIV. In: WHO/Teams/Global HIV Programme [website]. Geneva: World Health Organization; 2021 (https://www. who.int/teams/global-hiv-hepatitis-and-stis-programmes/hiv/prevention/mother-to-child-transmission-of-hiv, accessed 13 November 2021). 24. 2015 progress report on the Global Plan towards the elimination of new HIV infections among children and keeping their mothers alive. Geneva: UNAIDS; 2015 (http://www.unaids.org/sites/default/files/media_asset/JC2774_2015ProgressReport_GlobalPlan_en.pdf, accessed 19 July 2021). 25. United Nations Population Division. World population prospects: 2019 revision. New York: United Nations Population Division; 2019 (https:// population.un.org/wpp/, accessed 19 July 2021). 26. Country: Kenya [website]. Geneva: UNAIDS; 2021 (http://www.unaids.org/en/regionscountries/countries/kenya, accessed 19 July 2021). 27. Report of the first meeting of the expert panel on health impact of Global Fund investments. Geneva: The Global Fund; 2014 (https://www. theglobalfund.org/media/8049/corporate_expertpanelhealthimpactinvestmentsmeeting_report_en.pdf, accessed 19 July 2021). 28. Dissanayake C. Sri Lanka fast “developing” into TB hub of Asia. Sunday Times, 30 March 2014 (http://www.sundaytimes.lk/140330/news/sri- lanka-fast-developing-into-tb-hub-of-asia-91034.html, accessed 19 July 2021). 29. World malaria report 2016. Geneva: World Health Organization; 2016 (https://apps.who.int/iris/bitstream/hand le/10665/252038/9789241511711-eng.pdf, accessed 1 August 2021). 30. Scheffler R, Cometto G, Tulenko K, Bruckner T, Liu J, Keuffel EL. Health workforce requirements for universal health coverage and the Sustainable Development Goals. Human resources for health observer series no. 17. Geneva: World Health Organization; 2016 (https://apps. who.int/iris/bitstream/handle/10665/250330/9789241511407-eng.pdf?sequence=1&isAllowed=y, accessed 19 July 2021). 31. National health workforce accounts – a handbook. Geneva: World Health Organization; 2017 (https://apps.who.int/iris/bitstream/hand le/10665/259360/9789241513111-eng.pdf, accessed 19 July 2021). 32. Liu J, Goryakin Y, Maeda A, Bruckner T, Scheffler R. Global health workforce labor market projects for 2030. Hum Resour Health. 2017;15(1):11. https://doi.org/10.1186/s12960-017-0187-2. 33. McCoy D, Bennett S, Witter S, Pond B, Baker B, Gow J. Salaries and incomes of health workers in sub-Saharan Africa. Lancet. 2008;371(9613):675–68. 34. WHO guideline on health policy and system support to optimize community health worker programs. Geneva: World Health Organization; 2018 (https://apps.who.int/iris/bitstream/handle/10665/275474/9789241550369-eng.pdf?ua=1, accessed 22 February 2019). 35. Fulton BD, Scheffler RM, Sparkes SP, Auh EY, Vujicic M, Soucat A. Health workforce skill mix and task shifting in low income countries: a review of recent evidence. Hum Resour Health. 2011;9(1):1. 36. Buttorff C, Hock RS, Weiss HA, Naik S, Araya R, Kirkwood BR. Economic evaluation of a task-shifting intervention for common mental disorders in India. Bull World Health Organ. 2012;90:813–21. 37. Sabin LL, Knapp AB, MacLeod WB, Phiri-Mazala G, Kasimba J, Hamer DH. Costs and cost–effectiveness of training traditional birth attendants to reduce neonatal mortality in the Lufwanyama Neonatal Survival study (LUNESP). PLoS ONE. 2012;7(4):e35560. 38. Mengistu BS, Vins H, Kelly CM, McGee DR, Spicer JO, Derbew M. Student and faculty perceptions on the rapid scale-up of medical students in Ethiopia. BMC Med Educ. 2017;17(1):11. 39. Willcox ML, Peersman W, Daou P, Diakité C, Bajunirwe F, Mubangizi V. Human resources for primary health care in sub-Saharan Africa: progress or stagnation? Hum Resour Health. 2015;13(1):76. 40. Pemba S, Macfarlane SB, Mpembeni R, Goodell AJ, Kaaya EE. Tracking university graduates in the workforce: information to improve education and health systems in Tanzania. J Pub Health Policy. 2012;33(1):S202–25. 41. Mullan F, Frehywot S, Omaswa F, Buch E, Chen C, Greysen SR. Medical schools in sub-Saharan Africa. Lancet. 2011;377(9771):1113–21. 42. Kinfu Y, Dal Poz MR, Mercer H, Evans DB. The health worker shortage in Africa: are enough physicians and nurses being trained? Bull World Health Organ. 2009;87(3):225–30. 43. Kolstad JR. How to make rural jobs more attractive to health workers. Findings from a discrete choice experiment in Tanzania. Health Econ. 2011;20(2):196–211. 44. Dambisya YM. A review of non-financial incentives for health worker retention in east and southern Africa. EQUINET discussion paper no. 44. 2007:49–50. 45. Willis-Shattuck M, Bidwell P, Thomas S, Wyness L, Blaauw D, Ditlopo P. Motivation and retention of health workers in developing countries: a systematic review. BMC Health Serv Res. 2008;8(1):247. 37 46. Henderson LN, Tulloch J. Incentives for retaining and motivating health workers in Pacific and Asian countries. Hum Resour Health. 2008;6(1):18. 47. Kotzee TJ, Couper ID. What interventions do South African qualified doctors think will retain them in rural hospitals of the Limpopo province of South Africa? Rural Remote Health. 2006;6(3):581. 48. Agyepong IA, Anafi P, Asiamah E, Ansah EK, Ashon DA, Narh‐Dometey C. Health worker (internal customer) satisfaction and motivation in the public sector in Ghana. Int J Health Plann Manage. 2004;19(4):319–36. 49. Brockett PL, Golany B. Using rank statistics for determining programmatic efficiency differences in data envelopment analysis. Manage Sci. 1966;42(3):466–72. 50. Health workforce requirements for universal health coverage and the Sustainable Development Goals. Human Resources for Health Observer no. 17. Geneva: World Health Organization; 2016. 51. Ji YB, Lee C. Data envelopment analysis. Stata J. 2010;10(2):267–80. 52. Estimated DALYs (’000) by cause, sex and WHO Member State (1), 2012. In: WHO Global Health Estimates 2016 summary tables [Excel workbook]. Geneva: World Health Organization; 2016 (https://www.who.int/healthinfo/global_burden_disease/GHE2016_Deaths_ WBInc_2000_2016.xls, accessed 13 November 2021). 53. Scheffler RM, Liu JX, Kinfu Y, Dal Poz MR. Forecasting the global shortage of physicians: an economic- and needs-based approach. Bull World Health Organ. 2008;86:516–23B. 38
Health Workforce Department World Health Organization 20 Avenue Appia CH - 1211 Geneva 27 Switzerland www.who.int/health-topics/health-workforce#tab=tab_1