Global research agenda for improving the health safety and dignity of sanitation workers
Global research agenda for improving the health safety and dignity of sanitation workers Global research agenda for improving the health safety and dignity of sanitation workers This publication is the update of the document published in 2015 entitled “Sanitation safety planning: manual for safe use and disposal of wastewater, greywater and excreta”. ISBN 978-92-4-006290-0 (electronic version) ISBN 978-92-4-006291-7 (print version) © World Health Organization 2022 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https:// creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http:// www.wipo.int/amc/en/mediation/rules/). Suggested citation. Global research agenda for improving the health safety and dignity of sanitation workers. Geneva: World Health Organization; 2022. Licence: CC BY-NC- SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see https://www.who.int/copyright. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. Design by L’IV Com Sàrl iii Contents Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Research priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Characteristics of respondents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Top 15 research questions ranked by sanitation experts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Top 15 research questions ranked by sanitation workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Comparing rankings by experts and sanitation workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Annex 1: Results and themes from the initial literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Annex 2: Ranking results of the research question by sanitation experts and sanitation workers . . . . . . . 14 Annex 3: Ranking of all research questions by sanitation workers and experts . . . . . . . . . . . . . . . . . . . 16 Annex 4: Correlation between ranked scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 iv Acknowledgements This research agenda was developed by the World Health Organization (WHO) as part of the Sanitation Workers Initiative (SWI) co-led by WaterAid, the World Bank Group, International Labour Organization (ILO), SNV Netherlands Development Organisation, and WHO. The research agenda development process was coordinated by Abisola Osinuga (University of North Carolina, Center for Health Equity Research) with support from Jennifer Barr (freelance consultant) and direction from Kate Medlicott and Sophie Boisson of WHO. This research agenda could not have been produced without the valuable input of sanitation workers and practitioners, and advisers who identified and prioritized research questions, including: • Center for Affordable Water and Sanitation Technology, Canada: Kelly James • Center for Law and Justice, Pakistan: Mary James Gill • Dalberg Associates, India: Nirat Bhatnagar • I-San, United Kingdom: Pippa Scott • Independent WASH consultant, Nigeria: Sulaiman Ahmad • International Labour Organisation, Switzerland: Carlos Carion-Crespo and David Kapya • Lancaster Environment Centre, Lancaster University, United Kingdom: Sally Cawood • Practical Action, United Kingdom: Lucy Stevens • The Pan-African Association of Sanitation Actors (PASA), Kenya: Eva Muhia, Ibra Sow • SNV Netherlands Development Organisation, Netherlands: Antoinette Kome, Emily Banda, Leyla Khalifa and Rajeev Munankami • Urban Management Center, India: Prerana Somani • WaterAid, United Kingdom: Andrés Hueso González and Kanika Singh • Water services regulator board (WASREB), Kenya: Peter Njaggah • Women in Informal Employment: Globalizing and Organizing (WIEGO), United Kingdom: Sonia Dias • World Bank Group, United States of America: Ndeye Awa Diagne, Miguel Vargas-Ramirez and Ruth Kennedy- Walker WHO gratefully acknowledges the financial support provided by the Bill & Melinda Gates Foundation (BMGF), the Swiss Agency for International Development (SDC) and Agence Française de Développement (AFD). vAEA Average expert agreement score CHNRI Child health nutrition research initiative FSM Faecal sludge management ILO International Labour Organisation ISW Initiative for sanitation workers JMP Joint monitoring programme LMIC Low- and middle-income countries PPE Personal protective equipment RPS Research priority score SDG Sustainable development goals SMS Safely managed sanitation UNICEF United Nations Children’s Fund WASH Water, sanitation, and hygiene WHO World Health Organisation Acronyms vi Definitions Research theme Broad areas of research gaps identified in the initial literature review (Annex 1). Research domain Refers to the CHNRI 4D Framework— Description, Delivery, Development and Discovery Research areas Research areas comprise of multiple inter-related research questions. A research area would likely not be answered by an individual study. Research questions Questions that are specific enough to be answered by an individual study or programme. Sanitation experts scoring criteria Answerable How feasible or practical will it be to implement this research idea into a programme or intervention? Translatable What is the potential for scaling up this idea or research across multiple contexts? Can this idea be translated into initiatives that will be relevant to diverse types of sanitation workers? Effective Can this research idea inform the development of solutions, policies or infrastructure that will improve work conditions and minimize the health risks of sanitation workers? Equitable Can this research idea be used to inform solutions, policies or infrastructure that are accessible to all, particularly the most marginalized sanitation workers? Maximum potential for impact If successfully implemented, can this research idea markedly reduce the health risks of sanitation work and improve the working and living conditions of sanitation workers? Sustainable Considering the potential cost, scope, and relevance of the research idea to multiple kinds of sanitation work, can this research idea generate long-term solutions that will improve the health, safety and dignity of sanitation work? Sanitation workers scoring criteria Important If successfully implemented, can this research idea markedly reduce the health risks of sanitation work and improve the working and living conditions of sanitation workers? Feasible How feasible or practical will it be to implement this research idea into a programme or intervention? Applicable/relevant to multiple groups What is the potential for scaling up this idea or research across multiple contexts? Can this idea be translated into initiatives that will be relevant to diverse types of sanitation workers? 1Introduction In the Sustainable Development Goals (SDGs), leaders set out an ambitious target for sanitation with the goal of achieving safely managed sanitation for all by 2030. This means that everyone, everywhere would have access to and use a safe toilet where the waste is contained and safely disposed of onsite or safely transported and adequately treated before being disposed of or used (1). However, as of 2020, approximately 46% of the world’s population lacked access to safely managed sanitation services (2). To achieve the SDG target, the world needs to quadruple the rate of progress made between 2000 and 2017 in increasing sanitation coverage (3). Safe and resilient sanitation systems are essential to prevent pollution and the spread of disease, and improve health, social and economic wellbeing for everyone. Sanitation systems require significant amounts of physical labour at every step of the sanitation chain: for waste collection, waste transportation, equipment maintenance, cleaning, and other tasks. The transition to safely managed sanitation systems for all will require many more sanitation workers in safe and dignified employment. The term sanitation workers refers to people responsible for cleaning, maintaining, operating, or emptying a sanitation technology at any step of the sanitation chain (Figure 1). The term includes toilet cleaners and caretakers in domestic, public, and institutional settings; people who empty latrine pits and septic tanks; faecal sludge handlers; people who clean sewers and manholes; and people who work at sewage and faecal waste treatment and disposal sites (1, 4). While the sanitation workforce is vital to a community’s wellbeing, workers are often ignored, neglected, or actively stigmatized, leading to increased risk of physical, physiological, mental, or social harms including illness, injury, and death. Many of these jobs exist in the informal economy, and workers are often not recognized for the work that they do, nor are they protected by basic labour rights. Recurrent news items about sewer deaths, pit collapses, and illnesses are often dismissed as isolated incidents rather than part of a systemic issue. Growing and protecting the sanitation workforce is vital to improving sanitation globally – not only in ensuring the human rights and development of the workers themselves, but as a foundation of achieving and sustaining SDG commitments and ensuring that everyone benefits from them. A joint report Health, Safety and Dignity of Sanitation Workers: An Initial Assessment jointly prepared by the World Bank, World Health Organization (WHO), the International Labour Organisation (ILO), and WaterAid identified key challenges, good practices, and areas for action to improve conditions of sanitation workers (5) (see Figures 1 and 2). One of the critical action areas identified was building the evidence base. Fig. 1. Types of sanitation work (adapted from World Bank, 2019) Toilet/containment Emptying Conveyance Treatment End use/disposal Sweeping Latrine cleaning Domestic work Community/public toilet keeping School toilet cleaning Municipalities, government and private office cleaning Manual emptying Faecal sludge handling Mechanical emptying (including septic tank desludging) Manual transport Mechanical transport Sewer cleaning Sewer and pumping station maintenance Manhole cleaning Treatment plant work Sewage treatment plant cleaning wastewater and sludge handing and wastewater treatment plants Manual disposal Mechanical disposal 2Fig. 2. Key challenges, good practices, and action areas identified in previous report on sanitation workers (adapted from World Bank, 2019) Key challenges Identified good practices Areas for action Multiple occupational and environmental hazards Weak legal protection of an invisible workforce Financial insecurity Social stigma and discrimination Acknowledgement and formalization Mitigating occupational health risks Delivering health services Standard operational procedures and guidelines Workers’ empowerment though unions and associations Policy, legislative and regulatory reform Development ad adoption of operational guidelines Advocacy and empowerment Building the evidence base A systematic review of health evidence identified a lack of research on sanitation workers particularly in low- and middle-income countries (LMIC) (6). This document lays a foundation for building the evidence base by setting out a research agenda focused on LMICs. This agenda was developed using a consultative process involving a wide range of stakeholders including researchers, practitioners, as well as sanitation workers and their representatives. This research agenda is intended for researchers, practitioners and funders who seek to focus research activities on topics of greatest relevance to understanding and addressing challenges faced by sanitation workers globally. 3Methods The sanitation workers research prioritization exercise was undertaken following the WHO systematic guide to setting research priorities and was coordinated by a team of WHO staff and independent consultants (7). Research priority- setting is a transparent and systematic process to assist decision- and policy-makers with selecting core areas to focus on amidst numerous competing research to ensure proper allocation and effective utilization of resources such as capacity, time, and funds (7, 8). The steps below were followed: Step 1: Rapid review of literature The team conducted a rapid literature review of sanitation worker-related research to identify and thematically categorize research recommendations and knowledge gaps. Step 2: Advisory group creation and engagement The team assembled an advisory group consisting of 15 sanitation experts primarily from non-governmental organizations (NGOs) and academic institutions, primarily linked to sanitation implementation activities in LMICs. The advisory group and the team collaborated throughout this exercise to define the scope, design methods, contextualise tools, and refine the research themes. The advisory group also assisted in identifying and reaching out to stakeholders working on this issue. Step 3: Selection of methods for prioritizing research Several methods exist to prioritise health research and they typically fall into two main categories: consensus-based or metrics-based approaches (7). The team selected the approach from the child health and nutrition research initiative (CHNRI). It is one of the most popular approaches for health research priority- setting. It consists of a collaborative process to identify, compile, and list competing research questions and to evaluate the questions against a set of pre-defined criteria (8–10). This method was chosen because it is systematic, transparent, replicable, and can include a wide range of stakeholders (10). In addition, we chose a method that was could be easily adapted to a topic with a scope wider than health only. Step 4: Context definition and criteria specification Research questions were organised into four broad domains based on the adapted CHNRI “4D framework” (11): Description: Questions to assess the risks and challenges faced by sanitation workers and to understand their determinants. This may include describing workers typology and demographics, identifying and quantifying health risks and disease burden, understanding the social, legal, and institutional challenges faced by sanitation workers, and gender aspects of the work. Delivery: Questions to examine existing policies and interventions designed to protect sanitation workers. This may include understanding implementation processes for given technological or behavioural interventions; reviewing and analysing current policies and guidelines that are designed to professionalize work, improve legal and social protection, improve partnerships, or facilitate mobilization and empowerment of workers. Development: Questions to improve the implementation of existing interventions in terms of quality, affordability, acceptability, and deployment at scale. Discovery: Questions that may lead to innovations or generation of new knowledge to develop completely new interventions. Six of the CHNRI criteria were selected by the team and advisory group to rank research ideas: the extent to which the research idea is answerable, translatable, effective, equitable, sustainable, and has potential for maximum impact (see definitions section). A simplified version of the ranking survey was produced for sanitation workers with only three criteria: the extent to which the research idea is important, feasible and applicable (i.e., applicable to many types of workers). 4Step 5: Research question elicitation An online survey was developed using QualtricsTM to solicit research ideas from sanitation experts and workers. The survey was shared via multiple channels including emails, newsletters, online sanitation discussion platforms and conferences. Respondents were asked to provide at least one research idea under themes identified via the initial rapid literature review. Between 15 July and 3 December 2021, a total of 79 ideas were collated. This included 52 ideas from 56 respondent via online surveys and 27 ideas from 20 respondents during sessions at the 2021 University of North Carolina Water and Health conference and the 2021 Sanitation Workers Forum. Step 6: Consolidation and refinement The 79 research ideas were consolidated into 55 ideas to remove duplicates and improve clarity. They were grouped into 12 major research themes across the four domains: description (four themes), development (three themes), delivery (three themes), and discovery (two themes) (Table 4). The ranking survey was pilot tested with 20 conferences attendees during a session at the 2021 Sanitation Workers Forum. A simplified survey containing 42 reworded ideas was developed for sanitation workers following input from the advisory group and six representatives from sanitation organizations in LMICs. Step 7: Ranking research ideas Two separate ranking surveys were sent: one to sanitation experts and one to sanitation workers and union leaders in LMICs. The electronic survey sent to sanitation experts involved them rating each of the research ideas based on six CHNRI criteria while the survey administered to sanitation workers involved rating against three criteria. Sanitation workers ranking surveys were translated as needed and administered electronically (29), via focus group discussions (19), and interviews (3). Step 8: Analysis Sanitation experts ranked each idea against the 6 criteria by selecting ‘Yes’ (1 point), ‘Undecided’ (0.5 point), ‘No’ (0 point) or’ Insufficiently informed’ (no input). Sanitation workers ranked each idea against three criteria by choosing ‘Very well’ (1 point), “Moderately’ (0.75 point), “Slightly (0.5 point), Not at all (0 point) or insufficiently informed (no input). A research priority score (RPS) of 0-100% was calculated for each idea by estimating the mean RPS score across the criteria. The level of agreement between respondents’ answer was assessed through the average expert agreement (AEA), a proportion of scorers who gave the most frequent score for an idea divided by the number of scorers who scored that idea: where q is an idea that experts are being asked to evaluate each competing research investment option again, ranging from 1 to n (n= 6 for experts and n=3 for workers). An AEA score of 0 would mean no agreement at all; 100 would mean perfect agreement between all respondents. Annex 1 Results and themes from the initial literature review summarizes the results of the initial literature search. Of the 34 studies on sanitation workers in LMICs that this review examined, a majority of them focused on technological innovation or occupational health and safety. Far fewer papers looked at the key stakeholders involved, institutional arrangements, professionalisation of work, or gender aspects. 5Research priorities Characteristics of respondents Table 1 describes participant characteristics and survey completion rates. In total, 76 individuals provided research ideas and 96 individuals ranked them. In total, 38 sanitation experts or practioners ranked 55 ideas based on six criteria, while 58 workers from 9 LMICs ranked 42 ideas based on three criteria. Respondents involved were from over 20 countries spread across different geographical areas while sanitation workers were primarily from African and South-Asian countries. Experts providing research ideas were mostly from NGOs or academic institutions, while experts ranking the ideas were more evenly spread across different constituencies and mostly from Africa and South Asia. Sanitation workers who ranked research ideas worked mostly as manual emptiers or sewer cleaners. Table 1. Summary of characteristics of respondents Respondents characteristics Eliciting Ideas (Survey 1) Ranking Ideas Survey 2 (Experts) Survey (Workersa) Total number of respondents 76 38 58 Complete responses 52 16 49 Number of countries represented 21 20 9b Region Europe 15 (37.5%) 2 (6.5%) – Africa 11 (27.5%) 18 (58.1%) 29 (55.1%) South Asia 7 (17.5%) 7 (22.6%) 21 (40.4%) The Americas 5 (12.5%) 3 (9.6%) 1 (1.9%) Western Pacific 2 (5.0%) 1 (3.2%) 1 (1.9%) Organization NGO 13 (31.0%) 6 (15.8%) Academic institution 12 (28.6%) 4 (10.5%) Sanitation service providers 7 (16.7%) 3 (7.9%) Private consultants 6 (14.3%) 6 (15.8%) Sanitation workers association 1 (2.4%) 4 (10.3%) National or local government 1 (2.4%) 9 (23.7%) Others 2 (4.8%) 6 (15.8%) Sex Male 15 (39.5%) 20 (57.1%) 49 (92.4%) Female 23 (60.5%) 15 (42.9%) 4 (8.2%) Ideas provided or ranked 79 55 42 a Sanitation workers were engaged in the following type of work: manual emptying (n=19), sewer cleaning (n=13), sanitary work (n=5), mechanical emptying (n=3), WASH consultant (n=3), sanitary supervisor (n=2), sanitation union leaders (n=2), sweeper (n=1). b Countries included Pakistan, India, Nigeria, Zambia, Tanzania Kenya, Madagascar, Uganda, Niger 6Top 15 research questions ranked by sanitation experts Table 2 summarizes the 15 highest-scoring research questions by experts. About two thirds of the top 15 questions belonged to the delivery domain on understanding policies, systems and implementation processes. The remaining fell largely within the development domain, i.e., research on improving the quality, affordability, sustainability, and acceptability of existing interventions. Questions focused on improving the legal and social protection, and professionalization of work (n=5) and on improving the translation and scale-up of existing health and safety measures, behavioural interventions, and policies for sanitation workers (n=4). Table 2. Top 15 research questions ranked by sanitation experts Rank Research question De sc rip tio n De liv er y De ve lo pm en t Di sc ov er y 1 How can we better understand relevant country laws or schemes to protect sanitation workers and the challenges of implementing them in different contexts? 2 What are the implications of formalizing and increasing the visibility of workforce, particularly for the most marginalized workers? 3 How can we design PPE that is responsive to the needs of workers in different climates, contexts, and for men and women to improve PPE uptake and use? 4 How can collaborations among local and national government, private sector, unions and NGOs be strengthened to increase the dignity and respect of sanitation work? 5 What are the most effective strategies that can be used to reduce the negative impacts of social stigma? 6 How can technologies that reduce physical hazards for workers be scaled up? 7 How can sanitation tools and technologies be better designed and adapted to the needs of sanitation workers to improve their uptake and use? 8 What new technologies and systems can be used to increase emptying speeds and reduce fuel costs? 9 What kinds of behaviour change interventions can be implemented for sanitation users to improve the management and safety of sanitation systems? (e.g., to reduce blocks and risks to workers) 10 How can sanitation workers unions and associations most effectively build capacity and strengthen social capital for sanitation workers? 11 How can sanitation workers, especially marginalized workers, be better represented and involved in sanitation policies decision making? 12 How can a centralized system be developed across institutions researching sanitation work globally to regularly collect data on sanitation workers lived experiences (globally and over time) 13 How can sanitation workers be engaged as educators on safe practices for safely managed sanitation services? 14 How does social stigma affect sanitation workers’ behaviour and adoption of unsafe coping mechanisms? 15 How can global policy guidelines be developed and disseminated to encourage best practices? 7Top 15 research questions ranked by sanitation workers Table 3 summarizes the 15 highest-scoring research questions by sanitation workers. The questions tended to be more evenly spread across the four research domains compared to the questions from experts. Almost half feel within the description domain, i.e., research examining the risks and challenges of sanitation work. More specifically, questions related to assessing the efficacy, coverage, and effectiveness of behavioural and technical interventions (n=8), reducing the risks of sanitation work (n=4), and improving the legal, social protection and the professionalization of work (n=3). Table 3. Top 15 research questions ranked by sanitation workers Rank Research question De sc rip tio n De liv er y De ve lo pm en t Di sc ov er y 1 How and by how much do different sanitation technologies reduce health risks and hazards of sanitation work? 2 How can the capacity of local supply chains and markets be strengthened to maintain sanitation technologies and infrastructure? 3 How can sanitation tools and technologies be better designed and adapted to the needs of sanitation workers to improve their uptake and use? 4 How can sanitation workers, especially marginalized workers, be better represented and involved in sanitation policy level decision making? 5 What are the best ways to support organizations who are already working to address the stigma around sanitation workers and their families? 6 How can sanitation workers’ perspectives and views be better included in evaluation of programs designed to improve their health dignity and safety? 7 How can PPE be better designed to respond to the needs of workers in different climates, contexts, and for men and women improve PPE uptake and use? 8 How do sanitation workers perceive the safety of different technologies? 9 How can sanitation workers unions and associations most effectively build capacity and provide social support structures for sanitation workers? 10 What are the best ways to disseminate success stories showcasing sanitation workers unions and other associations work in strengthening social and legal protections of sanitation work? 11 What kinds of behaviour change interventions can be implemented for sanitation users to improve the management and safety of sanitation systems? (e.g., to reduce blocks and risks to workers) 12 How can global policy guidelines for the sanitation workforce be developed and disseminated to encourage best practices? 13 What are the challenges for sanitation workers using PPE? Would workers’ pay for a better version of PPE? 14 How can sanitation worker injuries and deaths be transparently and systematically tracked at local national and global scale? 15 How can a centralized system be developed across institutions researching sanitation work globally to regularly collect data on sanitation workers lived experiences (globally and over time) 8Comparing rankings by experts and sanitation workers Table 4 and Annexes 2–4 show a detailed comparison in top 15 ranked questions between experts and workers. Overall, experts tended to prioritize questions that are more closely related to improving the legal, social protection, marginalization, and professionalization of sanitation work while workers prioritized topics that are most closely related to documenting and improving the efficacy, and effectiveness of behavioural or technical interventions. Research domain Research theme Experts’ ranking Workers’ ranking Description Theme A: Quantifying and profiling the global and/or national sanitation work force Theme B: Measuring and reporting sanitation work-related accidents, diseases, and deaths 14 Theme C: Understanding the social and behavioural challenges affecting the economic livelihood, social protection, and the marginalization of workers 14 10 Theme D: Evaluating the efficacy, coverage, and effectiveness of behavioural or technical interventions to reduce the risks of sanitation work 1, 6, 8, 13 Delivery Theme A: Development of policies/guidelines or systems Research 1, 15 12 Theme B: Improving the legal, social protection and the professionalization of work 2, 5, 10, 11, 13 4, 5, 9 Theme C: Improving partnerships among stakeholders in sanitation service provision or regulation 4 Development Theme A: Improving affordability and acceptability of existing health, safety, behavioural or technical interventions 8 Theme B: Improving the translation and scale-up of existing policies, and existing health, safety, behavioural or technological interventions among sanitation workers 3, 6, 7, 9 3, 11 Discovery Theme A: Knowledge generation to develop practical, or scalable interventions, guidelines, and policies for sanitation workers 12 7, 15 Theme B: Knowledge that leads to developing sustainable interventions that will improve sanitation workers’ health, safety, and dignity 2 Table 4. Top 15 research questions grouped by the CHNRI domains and research themes 9Discussion This exercise used a well-established framework to identify and prioritize research aimed at improving the working conditions of sanitation workers. The research ideas generated varied widely in scope and covered most of the knowledge gaps identified previously in the literature. While sanitation experts prioritised research on policies, systems and implementation processes, sanitation workers prioritised research on health and safety, social and behavioural challenges, and the effectiveness of interventions to reduce the risk of sanitation work. Interestingly, some of the research areas identified in previous reviews such as understanding and quantifying sanitation work-related risks at local, national and global levels was reflected in the top 15 ranking by sanitation workers but not experts (5, 6). Quantifying these issues is essential to assist decision-makers in understanding the scale and severity of the problem and to inform prioritization and implementation of adequate mitigation measures. Below is a summary of the main priority themes that emerged from this exercise: 1. Examine the legal and social protection of sanitation workers particularly in LMIC. This includes research on developing or strengthening and effective implementation of sanitation regulations and standards to improve the legal and social protection of workers especially in countries where safely managed sanitation is still beyond reach and sanitation workers are exposed to deplorable and hazardous work conditions (12). 2. Ensure sanitation workers have a seat at the table. Results show differences between the type of research that workers perceive as beneficial and what experts see as important. This finding reinforces that worker involvement in research design and implementation is key. It is important to prioritize engagement of sanitation workers in many aspects including design of personal protective equipment (PPE) adapted for use in tropical hot climate, data collection and sharing, participation in sanitation workers associations and unions, and user education on proper usage of sanitation technology among others (13). 3. Identify technologies that are more widely applicable and scalable across different settings. Despite the wide range of sanitation technologies currently available, further research is needed to optimize and scale-up new and existing technological interventions that can be applied in multiple contexts (5). 4. Understand the root-causes of marginalization of sanitation workers to better address them. Some studies, especially in South Asia have examined the influences of social discrimination, stigmatization in predicting an individual’s involvement in sanitation work, including how those influences perpetuates a cycle of poverty (12,14–17). Further research is needed to understand intergenerational factors hindering the socio-economic mobility of marginalized workers. 5. Examine gender aspects of sanitation work. Sanitation work is highly gendered with toilet cleaning more commonly done by women and other aspects such as emptying transport and treatment roles done by men. Issues related to the gendered aspect of work (considering gender when designing PPEs) were highlighted in research priorities however, wider aspects of gender roles and biases in areas such as leadership were not identified. While we found the CHNRI methodology suitable for the scope of this exercise, this exercise had several limitations. First, while the CHNRI approach allowed for including a wide range of stakeholders, ensuring adequate representation of all groups was challenging. It is possible that some research fields were under-represented. In addition, some 10 stakeholders such as funders were not represented in the ranking and more efforts could have been made to engage policy- and decision-makers in countries so that research takes into account government priorities and findings from research are translated into action (7). Second, adapting the CHNRI methodology allowed for a systematic process but the length and complexity of the surveys and ranking may have negatively impacted the response and completion rates (7). Third, we aimed to involve sanitation workers - the beneficiaries of this research - in the ranking of ideas. In order to improve response rate among this group, research questions and criteria used for the ranking were slightly different to those used by experts. While the team and advisory group did a thorough review to ensure the original meaning of the questions was maintained and the three criteria reflected the six originally selected, subtle differences may have affected comparability of the results between the two groups. Notwithstanding these limitations, the present exercise provided a systematic, transparent and inclusive way of identifying and prioritizing future research efforts to better characterise and improve the working conditions of the sanitation workforce in LMICs. 11 Conclusion Historically, research and implementation on sanitation has paid relatively little attention to improving outcomes for health safety and dignity of sanitation workers, particularly in low- and middle-income countries (6). This exercise highlights the need for evidence-based approaches to improve the conditions of work to be incorporated in all sanitation policy and plans by global, national and local stakeholders. The research agenda (Table 2 and Table 3) helps support and coordinate the research and practice to address the most pressing research questions as perceived by experts and workers. It provides guidance to funders and researchers to help direct activities to where evidence is most needed. Additional work is needed by the research community to refine and contextualize research questions and carry out high quality research. It is also important that the research questions are applied to a diverse range of communities, people, and geographies spanning high- middle and low-income countries. Research should also cover the many types of sanitation work along the sanitation chain (Figure 1) encompassing both formal and informal workers, their families and communities, and the modifiers of gender, age, ethnicity and religion. The results highlight the need for researchers to routinely involve sanitations workers in all aspects of policy, research design and implementation. Research should be responsive rather than extractive and should avoid reinforcing local, national and international conditions that have contributed to stigma and marginalization faced by workers. Research should enhance wellbeing of the communities being studied. Compared to other public health and development challenges, the community of practice on sanitation workers is relatively small and diverse. Actors and participants in this exercise come from a variety of backgrounds including WASH, occupational health and labour among others. Successful research will bring together diverse skill sets and ultimately grow the coherence and quality of research as well as the size of the community of practice for this neglected topic. Despite limitations described, this exercise is an important step in understanding and creating a meaningful, driven, and unified research community that thoughtfully tackles the many pressing issues around sanitation workers and contribute to the achievement of the SDGs. 12 References 1. Guidelines on sanitation and health. World Health Organization. 2018. Available from: https://www.who.int/ publications-detail-redirect/9789241514705 2. Progress on household drinking water, sanitation and hygiene 2000-2020: Five years into the SDGs. Geneva: World Health Organization and the United Nations Children’s Fund; 2021. (https://www.who.int/publications/i/ item/9789240030848) 3. State of the world’s sanitation: An urgent call to transform sanitation for better health, environments, economies and societies. Geneva: World Health Organization and the United Nations Children’s Fund; 2020 (https://www. who.int/publications-detail-redirect/9789240014473) 4. Sanitation Worker Safety and Livelihoods in India: A Blueprint for Action—Phase 1: Understanding the Problem. Dalberg Advisors, New Delhi, India.; 2017. (https://dalberg.com/our-ideas/understanding-indian-sanitation- workers-and-finding-solutions-their-challenges/, Accessed on 11 October 2022) 5. Health, Safety and Dignity of Sanitation Workers: An Initial Assessment. Washington, DC: World Bank, International Labour Organization, WaterAid, World Health Organization. 2019 (https://openknowledge. worldbank.org/handle/10986/32640) 6. Oza HH, Lee MG, Boisson S, Pega F, Medlicott K, Clasen T. Occupational health outcomes among sanitation workers: A systematic review and meta-analysis. Int J Hyg Environ Health. 2022 Mar;240:113907. doi: 10.1016/j. ijheh.2021.113907. 7. A systematic approach for undertaking a research priority-setting exercise: guidance for WHO staff. Geneva: World Health Organization; 2020 (https://apps.who.int/iris/handle/10665/334408) 8. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health. 2016 Jun;6(1):010507. doi: 10.7189/jogh.06.010507. 9. Rudan I. Setting health research priorities using the CHNRI method: IV. Key conceptual advances. J Glob Health. 2016 Jun;6(1):010501. doi: 10.7189/jogh.06.010501. 10. Rudan I, Yoshida S, Chan KY, Sridhar D, Wazny K, Nair H et al. Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method. J Glob Health. 2017 Jun;7(1):011004. doi: 10.7189/jogh.07.011004. 11. Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, Black M et al. Child Health and Nutrition Research Initiative. Setting priorities in global child health research investments: guidelines for implementation of CHNRI method. Croat Med J. 2008 Dec;49(6):720-33. doi: 10.3325/cmj.2008.49.720. 12. Assessment of the Health, Safety and Dignity of Sanitation Workers in Dar es Salaam, Dodoma and Arusha, Tanzania. WaterAid; 2022 (https://washmatters.wateraid.org/publications/assessment-health-safety-dignity- sanitation-workers-tanzania, Accessed 11 October 2022) 13. How to support sanitation workers to claim their rights: lessons from other marginalised workers’ movements. WaterAid; 2020 (https://washmatters.wateraid.org/publications/how-to-support-sanitation-workers-to-claim- their-rights, Accessed 11 October 2022) 14. Acharya SS. Health, Safety and Well-Being of Sanitation Workers–Realities of Historical Exclusion and Livelihoods. In: Panneer S, Acharya SS, Sivakami N, editors. Health, Safety and Well-Being of Workers in the Informal Sector in India. Singapore: Springer Singapore; 2019 p. 199–214 (http://link.springer.com/10.1007/978-981-13-8421-9_16) 15. Bharti O, Sood V, Phull A, Kumar V. High Disease Burden among Sanitation Workers of Shimla Municipality in Himachal Pradesh, India - A Leading Cause of Adult Mortality. Int J Trop Dis Health. 2016 Jan 10;14(3):1–7. doi: 10.9734/IJTDH/2016/23747. 16. Raghavendra R H & Kumar RA. Sanitation Workers: A Neglected Community of Indian Civilized Society. Contemporary Voice of Dalit. 2022. https://doi.org/10.1177/2455328X211069683 17. The hidden world of sanitation workers. WaterAid India; 2019. (https://washmatters.wateraid.org/publications/ the-hidden-world-of-sanitation-workers, Accessed 11 October 2022) 13 Annex 1: Results and themes from the initial literature review Fig. 3. Summary of knowledge gaps identified from an initial rapid literature review, 2019 Institutional arrangements Allies and stakeholders Professionalisation and labour Gender aspects Root causes of marginalisation Behavioural change Policy, regulatory and legal Qualification of workforce Sustainable livelihoods Occupational health and safety Technological innovation Kn ow le dg e g ap s 0 Percent (%) across 34 studies 2 4 6 8 10 12 14 16 18 20 14 Annex 2: Ranking results of the research question by sanitation experts and sanitation workers Research question/idea RPS E6 RPS E3 AEA (E6) AEA (E3) Rank (E6) Rank (E3) Rank (W) Systematic approach to understanding labour laws, equality/anti-discrimination acts and social protection schemes that sanitation workers are protected by, and eligible for in different country contexts, and the challenges to implementation of such laws and schemes [NE = 20; NW= 43] 93.2 95.5 90.5 92.6 1 1 17 What is the implication of formalizing, increasing the visibility and upgrading the work of manual emptiers and other marginalized sanitation work groups? [NE = 17; NW= 43] 93.2 93.9 88.2 93.3 2 2 25 Better understand the needs, priorities and working realities of sanitation workers, and build this into design of any OHS/equipment. Take gender, weather and other factors into consideration when designing PPEs. [NE = 18; NW= 44] 92.4 92.4 86.7 85.9 3 4 7 Improve collaboration and partnerships among local, national governmental, private organizations, unions and NGOs to increase the respect, perceived value, visibility and understanding of sanitation work [NE = 14; NW= 44] 90.9 89.4 85.1 92.5 4 5 16 What are the strategies that can be used to address or reduce the negative impacts of cultural taboos, social stigma, and discrimination at local, national and global levels? [NE =15 NW= 44] 90.0 93.3 83.4 92.5 5 3 23 Scale up and increase the visibility of technologies that reduce the physical hazards (heavy lifting, heat, stress, confined spaces etc.) of sanitation work [NE = 13; NW = 44] 87.6 89.4 83.1 85.9 6 6 34 Include Sanitation Workers themselves in the design of technological tools (people centered design) so that it is adaptable to their needs and to improve uptake of the intervention. [NE = 17; NW= 44] 86.4 86.4 81.9 82.1 7 9 3 Using technology that increases emptying speed and reduces fuel costs including dewatering and decentralized treatment [NE = 14; NW=43] 85.6 86.4 81.8 79.1 8 8 24 Behavioural interventions for Sanitation Users on the implications of improper disposal of solid wastes and household objects into latrines i.e., bottles and clothes drop into toilets which makes emptying difficult [NE = 16; NW= 44] 85.6 89.4 80.6 82.2 9 7 11 Increasing the visibility and importance of sanitation workers union by using unions to create avenues for capacity building and social support for sanitation workers [NE = 17; NW= 43] 85.3 85.5 80.0 80.6 10 16 9 How can we better involve sanitation workers in decision making to improve workers representation and inclusivity when designing policies especially among marginalized work groups? [NE = 15; NW= 43] 84.7 83.3 78.9 81.1 11 18 4 Institutionalize a data registry (partners with local universities) to collect regular survey/ interview data on sanitation workers lived experiences longitudinally and globally [NE = 14; NW=44] 84.1 86.4 78.9 81.5 12 10 15 Using sanitation workers has safely managed sanitation educators in their communities on safe sanitation practices etc. be a part of the social knowledge transfer [NE = 15; NW= 45] 83.3 83.3 80.0 80.8 13 19 18 More research on how stigmatization influences sanitation workers behaviour and adoption of unsafe practices (such as emptying at night) as coping mechanism to deal with stigmatization. [NE = 15; NW= 43] 82.6 81.9 72.7 74.4 14 20 28 Improve the development and dissemination of global policy guidelines specific to sanitation labour force to ensure adoption of policy recommendations at the national and local level [NE = 18; NW= 45] 81.9 83.3 73.6 74.4 15 17 12 Description Delivery Development Discovery Table 5. Top 15 research questions ranked by sanitation experts RPS_E6 = Research priority score for sanitation experts from 6 CHNRI criteria; RPS_E6= score from sanitation experts from 3 criteria; RPS_W = Score from sanitation workers; AEA= Average Expert Agreement Score; NE = Number of experts; NW = Number of workers. 15 Research question/idea RPS (W ) RPS (E3) AEA (W) AEA (E3) Rank (W) Rank (E3) Rank (E6) Identify and group technologies based on their capacity to reduce the health risks and hazards of sanitation work [NE = 18 NW=45] 94.22 81.25 78.4 74.4 1 15 21 Research and funding to strengthen local supply chain markets in order to maintain and operate already existing technologies and sanitation infrastructure [NE=19; NW= 44] 91.86 80.3 75.0 71.04 2 22 17 Create avenues to include sanitation workers in the design of technological tools (people centered design) so that it is adaptable to their needs and to improve uptake of the intervention [NE = 17; NW=44] 90.90 86.36 72.72 82.15 3 9 7 Identify how to better involve sanitation workers in decision making to improve workers representation and voices when designing policies [NE = 15; NW= 43] 90.34 83.33 71.21 77.24 4 18 11 Provide support and funding for organization working to address the impact of stigmatization on sanitation work/workers [NE =16; NW= 45] 90.15 74.24 73.29 68.68 5 29 31 Understand what sanitation workers think represent the success of intervention implemented for them to improve their health, dignity and safety [NE = 15; NW= 45] 89.96 78.20 71.94 74.44 6 26 28 Better understand the needs, priorities and working realities of sanitation workers and build this into design of any OHS/equipment. Take gender, weather and other factors into consideration when designing PPEs [NE = 18; NW= 43] 89.77 92.42 68.93 85.85 7 4 3 Know the extent to which PPEs are perceived as safe by sanitation workers [NE = 16; NW= 44] 89.39 81.00 71.21 73.92 8 15 21 Increase the relevance and value of sanitation workers union by using unions to create avenues for capacity building and social support for sanitation workers [NE = 18; NW= 44] 88.82 84.52 63.63 80.59 9 16 10 Improve the knowledge about the significance of worker unions, cooperatives, and other sanitation workers’ associations in strengthening the legal and social protection of sanitation work [NE = 17; NW= 43] 88.44 76.19 64.39 71.04 10 31 26 Implement behavioural interventions for sanitation user to reduce improper disposal of solid wastes and household objects (bottles, clothes etc.) into latrines which makes emptying difficult [NE = 16; NW= 44] 88.44 89.39 68.93 82.15 11 7 9 Improve the development and dissemination of global policy guidelines specific to the sanitation labour force to ensure the adoption of policy recommendation at the national and local level [NE = 18; NW= 45] 87.87 83.33 68.99 74.44 12 17 15 Understand the challenges that prevent sanitation workers from using PPEs and if they are willing to pay for a better version of PPEs [NE = 15; NW= 43] 87.59 84.61 68.99 74.44 13 14 29 Develop systematic and transparent ways to record and report sanitation work-related injuries, and deaths from the local, to the national and global regional levels [NE = 17; NW= 43] 87.12 81.25 63.63 74.44 14 30 35 Institutionalize a data registry to collect regular survey/interview data on sanitation workers lived experiences longitudinally and globally. [NE = 14; NW=44] 87.12 86.36 62.12 81.48 15 10 12 Description Delivery Development Discovery Table 6. Top 15 research questions ranked by sanitation workers RPS_E6 = Research priority score for sanitation experts from 6 CHNRI criteria; RPS_E6= score frm sanitation experts from 3 criteria; RPS_W = Score from sanitation workers; AEA= Average Expert Agreement Score; NE = Number of experts; NW = Number of workers. 16 Annex 3: Ranking of all research questions by sanitation workers and experts Ideas/Topics Experts (3-criteria ranking) Experts (6-criteria ranking) Workers ranking Q1. Estimate the current side of the global sanitation workforce 34 32 36 Q2. Establish a standard system to classify the types of sanitation workers 32 27 32 Q3. Assess how sanitation work affects sanitation workers and their family’s economic likelihood. 36 33 26 Q4. Understand the global or regional differences in income insecurity and profits derived from sanitation work 36 33 39 Q5: Estimate (estimate) the burden of diseases, accidents, and deaths that happened because of the risks from sanitation work across the globe 21 22 19 Q6. Develop systematic and transparent ways to record and report sanitation work-related injuries, accidents, and deaths from the local, to the national and global regional levels. 35 35 14 Q7. Develop initiatives to address the mental health and other factors (substance abuse, psychosocial problems) that influence the mental well-being of sanitation work(ers) 40 40 38 Q8. Determine the impact of sanitation workers association and how inclusive they are to marginalized sanitation workgroups 28 30 29 Q9. Improve the knowledge about the significance of worker unions, cooperatives, and other sanitation workers’ associations in strengthening the legal and social protection of sanitation work 31 26 10 Q10. Assess the intergenerational effects of sanitation work or quantify the lifetime effects of sanitation work 38 38 42 Q11. Improve the understanding of how multiple issues (such as gender, caste, disability, and rurality) interact together to impact the health, economic livelihood, and social protection of sanitation workers 37 37 20 Q12. Fund more research to understand the how the stigma of sanitation work affects workers’ behaviour and adoption of unsafe practices to cope with stigmatization 35 36 30 Q13. Impact of stigmatization on social capital, including familial and friend relationships. 20 14 28 Q14. Know the extent to which PPEs are perceived as safe by sanitation workers 15 21 8 Q15. Identify and group technologies based on their capacity to reduce the health risks and hazards of sanitation work. 15 21 1 Q16. Understand the challenges that prevent sanitation workers face from using PPEs and if they will be willing to pay for a better version of PPEs 14 29 13 Q17. Understand what sanitation workers think represents the success of intervention implemented for them to improve their health, dignity, and safety 26 28 6 Q18. Increasing cultural sensitivity by understanding how sanitation workers perceive risk, and how they want those risks addressed. 12 18 33 Q19. Develop and disseminate global and national policy guidelines specific to the sanitation labour force to ensure the adoption of policy recommendations at the national and local level 17 15 22 Q20. Improve the development and dissemination of global policy guidelines specific to the sanitation labour force to ensure the adoption of policy recommendations at the national and local level 17 15 12 Q21. Systematic approach to understanding labour laws, equality/anti-discrimination acts and social protection schemes that sanitation workers are protected by in different country contexts, and the challenges to implementing them 1 1 17 Q22. Increase the relevance and value of sanitation workers union by using unions to create avenues for capacity building and social support for sanitation workers 16 10 9 Q23. Identify how to better involve sanitation workers in decision making to improve workers representation and voices when designing policies 18 11 4 Q24. Use sanitation workers as safely managed sanitation educators in their communities on safe sanitation practices etc. be a part of the social knowledge transfer designing policies 19 13 18 Table 7. Ranking of all research questions by sanitation workers and experts 17 Ideas/Topics Experts (3-criteria ranking) Experts (6-criteria ranking) Workers ranking Q25. Identify the legal protections and enforcement currently in place for different kinds of workers. Do they encourage formalization of services or inadvertently encourage informality? 13 19 40 Q26. Improve upon strategies that can be used to address or reduce the negative impacts of cultural taboos, social stigma, and discrimination at local, national and global levels 3 5 23 Q27. Provide support and funding for organizations working to address the impact of stigmatization on sanitation work 29 31 5 Q28. Research and discover the benefits of implementing initiatives to formalize, increasing the visibility and upgrade the work of informal sanitation workers especially manual emptiers and other marginalized sanitation work groups 2 2 25 Q29. Increase the visibility, scale, and funding for organizations working to understand the gender aspects of sanitation work 33 34 41 Q30. Improve collaboration and partnerships among local, national, private organizations, unions, and NGOs to increase the respect, perceived value, visibility and understanding of sanitation work? 5 4 16 Q31. Encourage and support local (country-based) design and production of PPE and equipment for emptying 25 20 21 Q32. Improve technology that increases emptying speed and reduces fuel costs including dewatering and decentralized treatment 8 8 24 Q33. Scale up and increase the visibility of technologies that reduce the physical hazards (heavy lifting, heat, stress, confined spaces etc.) of sanitation work. 6 6 34 Q34. Better understand the needs, priorities and working realities of sanitation workers, and build this into design of any OHS/equipment. Take gender, weather and other factors into consideration when designing PPEs. 4 3 7 Q35. Create avenues to include sanitation Workers in the design of technological tools (people centered design) so that it is adaptable to their needs and to improve uptake of the intervention. 9 7 3 Q36. Implement behavioural interventions for sanitation user to reduces improper disposal of solid wastes and household objects into latrines such as bottles, clothes etc. drop into toilets which makes emptying difficult. 7 9 11 Q37. Institutionalize a data registry (partners with local universities) to collect regular survey/interview data on sanitation workers lived experiences longitudinally and globally 10 12 15 Q38. Prioritize and improve on using sanitation initiatives, forums, conferences as a steppingstone to connect labour, human and legal rights organizations and initiatives to those on sanitation workers in the WASH sector 24 23 27 Q39. How can waste to wealth management organizations and projects improve the visibility and dignity of sanitation work? 27 25 31 Q40. Improving understanding of the barriers to scale up the promising technologies developed for hard to access sanitation facilities. 23 16 35 Q41. Research and funding to strengthen local supply chain markets in order to maintain and operate already existing technologies and sanitation infrastructure. 22 17 2 Q42. Use the COVID-19 Pandemic to generate new knowledge on occupational health and safety of sanitation work and as a driver to professionalize sanitation work. 13 24 37 18 Annex 4: Correlation between ranked scores 1 Important, feasible, applicable/relevant to multiple groups. 2 Answerable, translatable, effective, equitable, maximum potential for impact, sustainable. Fig. 4. Correlation of research priority score between workers and experts using three1 scoring criteria Annex 4: Correlation between ranked scores Figure 4: Correlation of research priority score between workers and experts using three1 scoring criteria Figure 5: Correlation of research priority score between workers and experts using six 2 scoring criteria· Figur 6: Corr lation of research priority core between experts using three and six scoring criteria 1 Important, feasible, applicable/relevant to multiple groups 2 Answerable, translatable, effective, equitable, maximum potential for impact, sustainable R² = 0.1113 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 Ra nk by W or ke rs Rank by Experts_3 R² = 0.1289 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 Ra nk by W or ke rs Rank by Experts R² = 0.843 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 Ex pe rt_ 3 R PS Expert_6 RPS Fig. 5. Correlation of research priority score between workers and experts using six2 scoring criteria Annex 4: Correlation between ranked scores Figure 4: Correlation of research priority score between workers and experts using three1 scoring criteria Figure 5: Correlation of research priority score between workers and experts using six 2 scoring criteria· Figure 6: Correlation of research priority score between experts using three and six scoring criteria 1 Important, feasible, applicable/relevant to multiple groups 2 Answerable, translatable, effective, quitable, maximum potential for impact, sustainable R² = 0.1113 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 Ra nk by W or ke rs Rank by Experts_3 R² = 0.1289 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 Ra nk by W or ke rs Rank by Experts R² = 0.843 0 5 10 15 20 25 30 35 40 45 0 10 20 30 40 50 Ex pe rt_ 3 R PS Expert_6 RPS Fig. 6. Correlation of research priority score between experts using three and six scoring criteria Annex 4: Correlation between ranked scores Figure 4: Correlation of research priority score between workers and experts using three1 scoring criteria Figure 5: Correlation of research priority score between workers and experts using six 2 scoring criteria· Figure 6: Correlation of research priority score between experts using three and six scoring criteria 1 Important, feasible, applicable/relevant to multiple groups 2 Answerable, translatable, effective, equitable, maximum potential for impact, sustainable R² = 0.1113 0 5 0 15 0 25 0 35 0 45 0 10 20 30 40 50 Ra nk by W or ke rs Rank by Experts_3 R² = 0.1289 0 5 0 15 0 25 0 35 0 45 0 10 20 30 40 50 Ra nk by W or ke rs Rank by Experts R² = 0.843 0 5 0 15 0 25 0 35 0 45 0 10 20 30 40 50 Ex pe rt_ 3 R PS Expert_6 RPS
Department of Environment, Climate change and Health Water Sanitation Hygiene and Health World Health Organization 20, avenue Appia 1211 Geneva 27 Switzerland www.who.int/water_sanitation_health/en/