Annex A Annex B Annex C Annex D Annex E Annex F Guidance 1 WHO tool for benchmarking ethics oversight of health-related research involving human participants User guide
WHO tool for benchmarking ethics oversight of health-related research involving human participants User guide WHO tool for benchmarking ethics oversight of health-related research involving human participants: user guide ISBN 978-92-4-008071-3 (electronic version) ISBN 978-92-4-008072-0 (print version) © World Health Organization 2023 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial- ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules/). Suggested citation. WHO tool for benchmarking ethics oversight of health-related research involving human participants: user guide. Geneva: World Health Organization; 2023. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see https://www.who.int/publications/book-orders. To submit requests for commercial use and queries on rights and licensing, see https://www.who.int/copyright. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party- owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. iii Introduction 1 Overview of the tool 1 Conducting assessments 1 Category 1 assessments: the national context 1 Category 2-6 assessments: RECs 3 Category 7 assessments: research institutions 4 After the assessment 5 Contents
1Introduction This user guide is intended to assist WHO Member States in implementing the WHO Tool for Benchmarking Ethics Oversight of Health- Related Research involving Human Participants. As explained more fully below, the tool is designed to facilitate multiple types of benchmarking activities, including self-assessments, collaborate assessments, nationally coordinated assessments, and external assessments. These assessments may be conducted by, or in collaboration with, ministries of health or other regulatory agencies, research ethics committees (RECs) or research institutions, research funders, or international organizations. The primary purpose of the tool is to enable countries to evaluate their existing capacities in ethics review and monitoring, identify strengths and limitations, and develop plans for improvement. The tool is not intended to be used as a means of “grading” or “ranking” countries or the entities being assessed. Rather, the goal is to assist countries in ensuring that their systems comply with international norms. Overview of the tool The tool is comprised of the following elements: Categories – The seven categories of the tool (see Figure 2) represent the essential components of an effective system of research ethics review and monitoring. The first category includes legal and regulatory aspects of the system. This category is to be assessed at the national level. The next five categories (categories 2 through 6) focus on critical aspects of REC functioning. These categories can be used to assess individual RECs; in addition, multiple assessments of representative RECs in a country can be combined to generate information about the functioning of the system overall. The final category applies to institutions whose employees or agents conduct health-related research involving humans, such as academic medical centers. This category can be used to assess individual research institutions; in addition, multiple assessments of representative research institutions in a country can be combined to generate information about the functioning of the system overall. Indicators – Each category is divided into multiple indicators. An indicator is a concise statement of an attribute that is expected to be present in a well- functioning entity. For example, Indicator 03.03 states, “The REC has adequate facilities and equipment.” The ability to satisfy this indicator is essential to the ability of an REC to effectively carry out its responsibilities. Guidance for assessors – The information below each indicator provides additional information to assist in the application of the tool. Description – The description provides more detail on the content of the indicator, including in some cases an explanation of the meaning of key terms. Evidence to review – The list of evidence to review provides examples of the kind of information that can help an assessor determine whether the indicator is satisfied. This list is intended to be illustrative only. Assessors may determine that some items on the list are not necessary to review because the information is obtainable through other means. In addition, they may choose to review evidence that does not appear on the list if they think it will help them make a more thorough assessment. Rating scale – The rating scale provides a metric for evaluating the extent to which the indicator has been adequately implemented. Conducting assessments Category 1 assessments: the national context The indicators in Category 1 relate to the legal and regulatory context of research ethics review. Most of the indicators in this category require assessors to review the country’s laws, regulations, and other legally binding instruments to determine whether 2Figure 1. The structure of the tool Figure 2. The seven categories 4. REC procedures 5. Mechanisms to promote REC transparency 6. Mechanisms for RECs to monitor their performance 7. Responsible research institutions 1. Legal provisions and regulatory framework 2. REC structure and composition 3. REC resources 7 Categories 48 Indicators Guidance for assessors Category 1 The national context Categories 2–6 Research ethics committees Category 7 Research institutions 14 in category 1 5 in category 2 5 in category 3 7 in category 4 6 in category 5 3 in category 6 8 in category 7 • Description • Evidence to be reviewed • Rating scale 3specified provisions exist. One indicator in this category (01.11) requires assessors to assess the legal powers and activities of relevant governmental entities. Two indicators in this category (01.13 and 01.14) require assessors to determine whether specified information is publicly available and up to date. Unlike the indicators in the other categories of the tool, which must be evaluated separately for each REC and research institution, the Category 1 indicators can be evaluated in a single assessment applicable to the country as a whole. Evaluating the Category 1 indicators is a relatively straightforward process, as it does not require interviews or site visits. It can be done as an entirely desk-based process by anyone familiar with relevant legal instruments and other publicly available resources. Some countries may choose to entrust the responsi- bility for conducting the Category 1 assessment to a ministry of health or other regulatory agency. Other countries may prefer to commission a report by an academic expert. In all cases, individuals entrusted with evaluating the Category 1 indicators should have appropriate training in law and familiarity with the country’s research ethics system. The results of their Category 1 assessments should be accompanied by specific references to relevant legal provisions and other sources that substantiate the assessor’s conclusions. Category 2-6 assessments: RECs Categories 2 through 6 address the functioning of RECs. Unlike Category 1 assessments, the indicators in Categories 2 through 6 must be evaluated separately for each individual REC. These evaluations can be conducted in a variety of ways: Self-assessments – Individual RECs can use these categories to conduct self-assessments of their operations, with the goal of identifying strengths and weaknesses and prioritizing quality improvement activities. Self-assessments may be conducted at the REC’s own initiative or at the request of ministries of health or other regulatory bodies. Collaborative assessments – Groups of RECs, such as RECs operating in a particular part of a country, RECs connected to similar types of research institutions, or RECs that focus on specific areas of health-related research, may choose to collaborate in a collegial process of mutual assessments. For example, members and/or staff from different RECs may serve as external assessors for each other’s committees. Collaborative assessments can be a useful way for different RECs to learn from each other and to promote the development of best practices in research ethics review. External assessments – Instead of relying on self-assessments, national authorities—as well as other entities, such as research funders or international organizations—may wish to undertake external assessments of selected RECs. In an external assessment, persons not affiliated with the REC complete the assessment checklists and rating sheets based on their review of REC records and interviews with members, staff, and other stakeholders. Nationally coordinated assessments – Ministries of health or other regulatory bodies interested in evaluating the overall quality of RECs in their country can select a representative sample of RECs to evaluate, either by conducting external assessments or by asking the RECs to conduct self- assessments and submit their results. The results of these assessments can then be aggregated to obtain a snapshot of the status of research ethics review in the country. In determining which RECs to include in this process, national authorities should seek to mirror the overall makeup of RECs in the country, considering factors such as the volume and nature of the research the REC reviews and the REC’s geographic location and institutional affiliation (if any). The process of conducting self-assessments and externally conducted assessments is largely the same, except for the fact that externally conducted assessments require collaboration between the external assessment team and the REC members and staff. All assessments should include each of the following activities: Formation of the assessment team – Assessments should be conducted by a team of persons knowledgeable about the field of research ethics and experienced in working with RECs. For self-assessments, the team will normally consist of REC members and staff, although some RECs may choose to engage outside expert consultants. For external assessments, the entity organizing the assessment can draw on the entity’s own staff 4members, staff from other relevant government agencies or international organizations, academics, and possibly members and staff of other RECs. The size of the team will depend on the volume of research the REC reviews, but in all cases should consist of at least two persons. Some members of external assessment teams should be able to communicate in the REC’s local language. For external assessments, the composition of the assessment team should be communicated to the REC before the process begins. RECs should be permitted to request the replacement of any team member, subject to adequate justification for the request. Training on the benchmarking tool – All assessors should undergo training on the benchmarking tool before the assessment begins. Training can be conducted by the WHO secretariat or consultants experienced in the use of the tool. Scheduling the assessment – The dates for the assessment should be set in advance. This is particularly important for external assessments, which require coordinating the schedules of the assessors and REC members and staff. However, even for self-assessments, it is useful for RECs to carve out specific dates for conducting their evaluations, rather than attempting to perform the assessment on an ad hoc basis on top of other work. In most cases, two days will be sufficient to complete an assessment. Document review – A large part of the assessment involves the review of various documents, including REC guidelines, standard operating procedures, terms of reference, review templates, meeting minutes, and correspondence. If some or all of these documents are available electronically, it may be possible to conduct a portion of the assessment remotely. Assessors should maintain lists of all documents they review, identifying documents as specifically as practicable (e.g., “minutes of REC meeting on XXX date,” or “letter from REC chair to XXX on XXX date”). Facilities and equipment review – All assessments should involve an in-person review of the facilities and equipment available to the REC, including office space, computers and other equipment, and the document storage system. Interviews – To gain an accurate understanding of how the REC operates, assessors will need to spend time talking to REC members, staff, and other stakeholders (e.g., principal investigators and other research staff, research participants, and, if possible, community representatives). This is particularly true for external assessors, but it is also important in self-assessments, as those conducing the assessment cannot assume that they have up-to-date knowledge of all stakeholders’ perspectives. Applying the rating scale – The rating scale requires assessors to determine whether the specified criteria have been satisfied. Assessors should consider a criterion satisfied only if there is evidence that demonstrates full, consistent satisfaction of that criterion. Assessors should indicate their basis for determining whether a particular criterion has been satisfied, such as references to specific documents or interviews. For ratings of “partially implemented” or “not imple- mented,” assessors should identify specific gaps and offer suggestions for addressing them. Assessments will often depend on subjective determi- nations. For this reason, it is important that all members of assessment teams participate in completing the rating scale. Discussions among the team may clarify uncertainties and increase the reliability of determinations. When external assessments are performed, the team’s initial impressions should be discussed with the REC’s members and staff before any final determinations are made. Category 7 assessments: research institutions The process of assessing the Category 7 indicators is identical to the one used for the Category 2 through 6 indicators, except that the entity being evaluated is a research institution rather than an REC. Research institutions consist of any entity whose employees or agents conduct health-related research involving human participants. Depending on the nature of the institution and how it is organized, responsibility for conducting assessments (or for participating in externally conducted assessments) may be entrusted to deans’ offices, academic departments, grants offices, or other units involved in the oversight of research. For institutions that have their own RECs, the REC chair should be able to identify relevant institutional officials. 5After the assessment Once the assessment is completed, the assessment team should prepare a report that includes the rating scales and references to the evidence supporting them, as well as any comments and suggestions for improvement made by the assessors. The entity being evaluated should use this report to develop a written plan for follow-up actions, including a timeline for implementation. If implementation of the plan depends on obtaining additional funding, the approximate amount of funding required should be identified, along with potential funding sources. Before the plan is finalized, it should be shared with the assessment team for feedback.
8 WHO tool for benchmarking ethics oversight of health-related research involving human participants: user guide World Health Organization 20 Avenue Appia 1211 Geneva 27 Switzerland www.who.int