Concept note: a framework for evaluating and publicly designating regulatory authorities as WHO-Listed authorities

WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 139 CONCEPT NOTE: A FRAMEWORK FOR EVALUATING AND PUBLICLY DESIGNATING REGULATORY AUTHORITIES AS WHO-LISTED AUTHORITIES (May 2019) DRAFT FOR COMMENTS Please send any comments you may have to nra_admin@who.int, with a copy to Ms Claire Vogel (vogelc@who.int) by 17 July 2019. Medicines Quality Assurance working documents will be sent out electronically only. They will also be placed on the Medicines website for comment under “Current projects”. If you have not already received our draft working documents, please send your email address (jonessi@who.int) and we will add you to our electronic mailing list. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 140 CONCEPT NOTE: A FRAMEWORK FOR EVALUATING AND PUBLICLY DESIGNATING REGULATORY AUTHORITIES AS WHO-LISTED AUTHORITIES 1. SUMMARY This concept note outlines a proposed framework for evaluating and publicly designating regulatory authorities as ‘WHO-listed authorities’, following upon recommendations from the Fifty-first meeting of the World Health Organization (WHO) Expert Committee on Specifications for Pharmaceutical Products (ECSPP) in October 2017 on the replacement of the term stringent regulatory authority with WHO-Listed Authority (WLA). This concept note presents a proposed definition for WLA; procedures for designating a WLA; and the process for finalizing the definition and the procedures for putting the framework into place. This concept note is meant to provide sufficient information to solicit comments on proposals presented herein, not to provide details of how the framework might be implemented. WHO intends to publish a draft WLA policy document similar in scope to this note by the end of July 2019 and draft operational guidance documents by the end of August 2019 to enable implementation of the WLA framework. Both the WLA policy document and operational guidance documents will be published for public comment. Given the wide interest in and implications associated with the definition and framework, WHO will adopt a multi-prong consultation process as outlined in this concept note. The definition for WLA will also need to be reviewed by WHO Expert Committees1 in the context of its usage in place of stringent regulatory authority in existing WHO guidelines. It is expected that the introduction of the WLA framework will begin with a pilot phase in the first quarter of 2020. 1 Expert Committee on Specification for Pharmaceutical Preparations (ECSPP) for the approval of the definition, consulting the Expert Committee on Biological Standardization (ECBS) to cover all product streams including vaccines and biotherapeutic products. WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 141 2. BACKGROUND The concept of a stringent regulatory authority or SRA was developed by the WHO Secretariat and the Global Fund to Fight AIDS, Tuberculosis and Malaria to guide medicine procurement decisions and is now widely recognized by the international regulatory and procurement community. Since its introduction, the term and corresponding definition have been incorporated in the quality assurance policies of most international organizations involved in the purchase and supply of medicines, the assumption being that products assessed and approved to enter the market by an SRA would consistently meet international requirements for safety, efficacy and quality. The concept has additionally served to promote reliance on the product evaluations and decisions of SRAs by other authorities when making their own regulatory decisions. WHO prequalification procedures and several other WHO guidance documents relating to the quality of medicines provide mechanisms to rely on SRAs, defining an SRA as a regulatory authority which is a member or an observer of the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH), or is associated with an ICH member through a legally binding mutual recognition agreement. In recent years, however, there have been increasing calls for a change in the term and definition given the inference regarding other authorities2; the expansion of ICH membership, as well as the fact that ICH is a harmonization initiative and does not have the mandate to assess regulatory capacity; as well as the interest of other regulators to be considered an ‘SRA’. A subsequent, interim definition was proposed by WHO and adopted in October 2017 by the ECSPP while WHO formulated a proposal for a more suitable term and definition. The interim definition essentially maintained the original definition based on the pre-reform membership of ICH. A proposal on the elements of a replacement definition for SRA was posted by WHO for public comment in July 2017 that was intended to provide a more transparent, robust and equitable measure of regulatory capacity and performance. The proposal set out a number of principles by which “stringent” regulatory authorities would qualify to be “on a list” established by WHO based on a formal assessment by WHO using the Global Benchmarking Tool (GBT) against requirements established for maturity level 4 (ML 4), a level which corresponds to a regulatory system operating at advanced level of performance and continuous improvement. 2 This concern led to a recommendation at the Seventeenth International Conference of Drug Regulatory Authorities (ICDRA) on ‘moving away from using the term stringent national regulatory authority.’ Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 The proposal also provided for a modular approach that would allow regulatory authorities to be designated as ML 4 for a specific regulatory function (such as inspection) or product group (such as the vaccine program). Following review of public comments, the ECSPP recommended that: • the term SRA be replaced by the term WHO-Listed Authority (WLA); • regulatory authorities currently identified as SRAs be regarded as WHO-listed authorities; • the designation of additional national regulatory authorities (NRAs) as WLAs be based on an assessment using the GBT, and a “performance verification process” to confirm consistency of performance against international standards and best practices; and • the procedure for listing WLAs be developed through WHO’s public consultation process. A similar concept of authorities exhibiting ‘a high level of performance’ was endorsed by the WHO Expert Committee on Biological Standardization (ECBS) at its Sixty-first meeting (in October 2010) within the context of streamlining the prequalification process for vaccines3. The procedure for selecting eligible regulatory authorities was based on experience gained by WHO in the evaluation of influenza H1N1 (2009) pandemic vaccines. The procedure was meant to serve as an interim measure pending the development of a revised NRA assessment tool with additional performance indicators that would be able to distinguish levels of regulatory functionality (maturity levels) and performance. This now translates into the GBT and the proposed performance evaluation framework. The harmonized medicines and vaccines GBT and the concept of categorizing regulatory authorities based on maturity level were introduced in 2016 (see Annex 1).4 The GBT builds upon previous regulatory system benchmarking tools used within the WHO family beginning in 1997 and also takes into account regulatory system evaluation tools used by organizations external to the WHO5. The benchmarking of regulatory authorities serves as a basis for formulating institutional development plans and implementing recommended improvements. 3http://www.who.int/immunization_standards/vaccine_quality/TRS_978_61st_report_Annex_6_PQ_vaccine_ procedure.pdf 4 The WHO had previously introduced the concept of classifying regulatory systems according to “maturity level” following a series of international consultations and adaptations of International Standard ISO 9004 and the Benchmarking of European Medicines Agencies (BEMA) initiative. 5 For further information on the official GBT (Revision VI), please refer to the following link: https://www.who.int/medicines/regulation/benchmarking_tool/en/ which presents the complete listing of GBT indicators and fact sheets according to regulatory functions and overarching system. A revised draft user manual will also be posted in the near future. 142 WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 143 The goal of regulatory system strengthening is ultimately to promote access to quality assured medical products, consistent with the United Nations (UN) Sustainable Development Goal (SDG) 36, World Health Assembly Resolution (WHA) 67.20 on Regulatory system strengthening for medical products78 and the Access Roadmap9. This concept note presents a proposed framework for using the evaluation tool (GBT) to generate and analyse evidence of regulatory system performance, as mandated by WHA Resolution 67.20, to allow for the public listing of regulatory authorities as WLAs. Given the implications associated with the proposed change in definition and approach, WHO is seeking feedback from Member States and all interested parties through a public consultation process. Details on the implementation of this framework will be guided by input received. 3. PURPOSE The purpose of this concept note is to: • present a proposed definition for WLA; • define the proposed process and timelines for designating a regulatory authority as a WLA; • describe the proposed process and timelines for finalizing the definition and the procedures for evaluating and designating and re-designating a WLA. 6 SDG 3: ‘healthy lives and promoting the well-being at all ages…including [through]… access to safe, effective, quality and affordable essential medicines and vaccines for all.’ 7 WHA Resolution 67.20 provides WHO the mandate to apply evaluation tools to generate and analyze evidence of regulatory system performance and facilitate the formulation and implementation of institutional development plans. 8 Several WHO regional committee resolutions on regulatory system strengthening have also been adopted, including, for example, Regional Committee Resolution (CD50.R9), 2010, in the WHO Regional Office for the Americas (AMRO/PAHO), Regional Strategy for Improving Access to Essential Medicines in the Western Pacific Region (2005-2010), and document AF/RC63/7 of the WHO Regional Office for Africa (AFRO). 9 The road map for access to medicines, vaccines and other health products (WHA72/17) highlights regulatory system strengthening as an integral part of a health systems approach to improving access to safe and effective medical products of assured quality. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 The designation of a regulatory authority as a WLA is ultimately meant to promote access, supply and the intended use of safe, effective and quality medicines and vaccines by: • providing a robust and transparent framework to promote trust, confidence and reliance between regulatory authorities and thereby enable the efficient use of regulatory resources; • providing a pathway for regulatory authorities to be globally recognized as meeting WHO and other international recognized standards and practices and thereby help guide international and national procurement decisions on medical products, including for products not eligible for prequalification; • increasing the pool of regulatory authorities contributing to the efficiency of the WHO Prequalification (PQ) program through the increased use of abridged procedure or alternative pathways to PQ listing;10 • promoting investments in, and the continuous improvement of, regulatory systems; • creating an enabling regulatory environment for innovation and local production; and • helping contribute to regulatory collaboration and capacity building efforts and more effective global regulatory oversight of medical products. 4. DEFINITIONS WHO proposes to publicly list regulatory authorities that satisfy the requirements for designation as WLA, as defined below: Maturity level 3 WHO listed authority (ML 3 WLA) A regulatory authority11 which has been documented to comply with all of the indicators and requirements specified by WHO for maturity level 3 based on an established benchmarking process. Represents a stable, well-functioning and integrated regulatory system. 10 Regulatory networks are increasingly important in building capacity and promoting convergence, harmonization, work-sharing and reliance. This in turn should result in greater regulatory efficiencies, effectiveness and transparency of operations. WHO also intends to take greater advantage of joint assessments and inspections performed by regulatory networks in the PQ process. Details on how this alternative PQ listing process will operate will be described in a separate document. 11 A regulatory authority is meant to cover all the institutions, working together in an integrated and effective manner, that are responsible for the regulatory oversight of medical products in a given country or region. 144 WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 145 Maturity level 4 WHO listed authority (ML 4 WLA) A regulatory authority which has been documented to comply with all of the indicators and requirements specified by WHO for maturity level 4 and to consistently adhere to WHO and other internationally recognized standards based on an established benchmarking and enhanced performance evaluation process. Represents a regulatory system operating at an advanced level of performance and continuous improvement, currently known as a stringent regulatory authority (medicines) and an authority exhibiting ‘a high level of performance’ (vaccines). Regional regulatory system A system composed of individual regulatory authorities, or a regional body composed of individual regulatory authorities, operating under a common regulatory framework. The common framework must ensure equivalence between the members in terms of regulatory requirements, practices and quality assurance policies. The regional body, where it exists, may have enforcement powers to ensure compliance with the common regulatory framework. A regional regulatory system so described may be considered a single entity and therefore eligible for listing as a WLA. Note: A regional regulatory system may be designated a WLA for those regulatory functions subject to a unified set of requirements, processes and set of controls. At present, the GBT is designed to evaluate national regulatory systems. The WHO is in the process of developing a system for evaluating the performance of regional regulatory networks or systems. Maturity level (ML) An estimation of the effectiveness and performance of a regulatory system or regulatory function as graded on a scale, based on an evaluation of the legal framework, regulations and guidelines; regulatory practices and procedures; organizational structure; management and administration; and human resource capacity and development 12. See Annex 1 for further details on maturity levels as they relate to the classification of authorities responsible for the regulation of medical products. 12 Adapted from ISO and BEMA definitions. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 In the initial implementation phase of the WLA framework, a regulatory authority may be designated a WLA for the regulation of generic medicines, for new medicines, and/or for vaccines, reflecting the current scope of the WHO benchmarking program13. The scope of program and listing options will gradually expand, beginning with medical devices (including in-vitro diagnostics) and blood and blood products. A regulatory authority may also be designated a WLA for one or more regulatory functions such as inspection. In all cases, the listing would specify the scope of designation (see Annex 2). In situations where the regulatory system is divided across different levels within the country, for example central/national and provincial/state/municipal entities, the initial designation as a WLA may be restricted to central and certain provinces and states, taking into consideration the size and administrative complexity of the country. The principal difference between the designation of ML 3 and ML 4 WLAs relates to the fact that: i. an ML 4 WLA must comply with ML 4 indicators relevant for the designation being sought14; and ii. the performance, that is, the ability of the regulatory authority to consistently adhere to international standards and best practices, has been more fully evaluated and documented. The designation of WLAs is meant to substantiate the maturity and performance of regulatory authorities using an international benchmark, as defined in the GBT. It is not meant to make any inference regarding the maturity or performance of regulatory authorities that have not been evaluated by WHO under the proposed framework. 14 It is important to note that an ML 3 WLA must comply with all the GBT indicators specified for a ML 3 regulatory system for medicines and/or vaccines, as documented by a formal benchmarking by WHO. This means that the authority must meet indicators for ML 1 to ML 3, inclusively. An ML 4 WLA must additionally comply with relevant ML 4 indicators specified for the scope of listing being sought, for example, generic medicines program or regulatory inspection. Further details will be available with the publication of the draft GBT manual, including with respect to the scoring of indicators, later in 2019. 146 WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 147 Similarly, while the performance of ML 3 WLAs will not be evaluated to the same level as for agencies seeking ML 4 WLA designation, the same considerations stated above apply. It is also important to recognize the significance of attaining ML 3, the objective of WHO regulatory system strengthening efforts for most countries, and of WHA Resolution 67.20, representing an international baseline for effective regulation. As per current practice, a regulatory authority that complies with ML 3 requirements for vaccines satisfies a prerequisite for application by vaccine manufacturers in that country to the WHO PQ program. 5. PROCESS FOR APPLICATION, EVALUATION AND LISTING AS A WLA15 A written application must be submitted to WHO from the regulatory authority requesting WLA listing that would include the authority’s agreement with the general evaluation methodology, the obligations of the regulatory authority throughout and following the process, and the publication of the outcome (the listing) and a summary basis of the decision on the WHO website. The authority is encouraged to meet with the WHO Secretariat to discuss the WLA evaluation and listing process prior to the formal application process. Regulatory authorities applying for listing should have an established history of interaction with WHO (see below) and be in a position to comply with the requirements for listing within a reasonable period of time. This will be considered by WHO in the application evaluation step. As each regulator and situation is unique, WHO will work with the authority to establish a roadmap to reach a listing decision. Should it become evident during the course of the combined benchmarking-performance evaluation exercise that the regulatory authority is unlikely to meet requirements for listing within a 6 – 12-month timeframe from the start of the benchmarking-evaluation process, depending on the scope of designation, the authority would be invited to reapply at a later time once identified areas for improvement have been addressed. Upon successful review of the application, the WHO Secretariat will meet with the regulatory authority to discuss and agree upon a roadmap for the WLA evaluation process that considers the scope of designation requested, the state of readiness of the authority and WHO, and all available information supporting the maturity and performance level of the authority. 15 Does not apply to transitional arrangements referred to on page 11. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 148 Depending on the number of requests, WHO may prioritize eligible requests based on the estimated impact on the regional or global supply of quality assured medicines or vaccines. The listing on the WHO website would specify the product categories (generic and/or new medicines and/or vaccines) or regulatory functions (such as regulatory inspection) for which the designation applies (see Listing Process below). The scope of listing for WLA product centric regulatory programs - generic medicines, new medicines and vaccines - includes the overarching regulatory system and all functions related to the regulatory oversight of the products that are within the scope of the program. With a view to encouraging incremental investment in regulatory systems and the attainment of an international level of capacity and performance, regulatory authorities would also be eligible to apply for listing in a regulatory function defined by the GBT, such as regulatory inspection, marketing authorization, laboratory testing or vigilance (see Annex 2). 6. CONSIDERATIONS IN THE DESIGNATION AND LISTING PROCESS • It is essential that the outcome (listing decision) and summary basis for the listing decision be made public in order to have the intended impact in promoting trust, reliance, sound procurement decisions and investments in regulatory systems. • Related to this point, regulatory authorities, procurement agencies and other stakeholders must have confidence in the process and outcomes. This means that WLA authorities must be recognized as complying with robust standards for evaluating regulatory systems, as developed with the input of experts from regulatory authorities and as published on the WHO website. • All available evidence and outcomes, including from previous benchmarking/audit exercises, will be taken into consideration when determining compliance with the requirements for designation as a WLA. • WHO supports the practice of reliance and recognition by even the most advanced and resourced regulatory authorities as a means of addressing the complexities of the global regulatory environment and promoting effective use of resources. A WLA authority, however, is expected to have the capacity to perform all regulatory functions related to medicine and/or vaccine regulation. WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 149 7. PERFORMANCE EVALUATION PROCESS WHO will take account of all existing evidence supportive of adherence to international standards and best practices to expedite the performance evaluation process, including WHO’s experience interacting with the regulatory authority. The performance evaluation activity is expected to provide a more detailed picture of how a regulatory system operates. It should serve to document consistency in adherence to procedures and in producing outputs which are consistent with the application of WHO and other international regulatory requirements and best practices. The framework will consider the nature and extent of evaluation required to provide a high degree of confidence in the authority’s performance. The GBT is designed to provide a robust and structured approach to analysing the required inputs (legal framework, organizational structure and resources), regulatory processes and intended outputs that together determine how well a regulatory authority is configured to ensure the safety, efficacy and quality of medical products in an effective, transparent manner. The benchmarking process requires independent experts to gather and review evidence described in fact sheets for all GBT indicators in order to document the level of implementation of a particular regulatory function, for example, regulations and guidelines for market authorization that define the types and scope of product variations and the required documentation for each type of variation. In finalizing the current version of the GBT, performance indicators and the accompanying instructions on evidence to collect and review were elaborated for many of the regulatory functions. For example, in relation to Registration and Market Authorization (MA): MA01.09: Specific guidelines on the quality, nonclinical and clinical aspects are established and implemented: Evidence to review: • Examples of MA application submissions that are in compliance with the published guidelines. • Evidence that documents submitted were appropriate for the type of product and type of application. • Copies of all quality, safety, efficacy reports for recently approved MA applications to determine whether reviews were done thoroughly and based on guideline requirements. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 150 While the GBT is designed to measure the existence and level of implementation of inputs and processes against specified indicators, as well as the performance of the regulatory system - that is, how the component inputs and processes result in desired regulatory outputs - the challenge has been the time required to fully evaluate the consistent performance during a formal benchmarking mission. WHO intends to address this challenge through an expansion of performance measurement within the overall benchmarking process. Performance measurement would be more comprehensive in relation to a ML 4 thereby providing a more detailed evaluation of regulatory outputs over a defined period of time or number of regulatory activity units, such as product assessment, good manufacturing practice (GMP) inspection or laboratory analysis. Figure 1 - Standard benchmarking process WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 151 The development of the performance evaluation framework will be guided by comments received and draw upon similar frameworks developed to establish performance and equivalence between regulatory systems. The framework will also take into consideration experience gained from evaluations of performance currently conducted by WHO, for example, in relation to GMP inspection and the management of adverse events following immunization (AEFI) for vaccine programs. It will also provide an opportunity to pilot the WHO global competency framework for evaluating human resource capacity and development.16 WHO will assemble a group of regulatory experts to assist in developing the performance evaluation framework. As the performance evaluation exercise will require investment in resources on the part of the regulatory authority and WHO, authorities applying for listing should be in a position to complete the process within 6-12 months from the start of the integrated benchmarking- performance evaluation process, depending on the scope of the evaluation. 8. LISTING PROCESS Following the successful completion of the WLA evaluation process, a regulatory authority will be listed on the WHO website. To bring further impartiality to the process, it is proposed that a recommendation to list a regulatory authority be made following a review by an independent committee of experts designated by WHO based on the report and recommendation from the evaluation team17. The listing would indicate the scope of the designation, for example, generic medicines, vaccines or pharmacovigilance as well as the maturity level of the authority. The period of validity of the listing would also be indicated (see below). 16 Human resource constraints have been identified as a challenge for regulatory systems globally, especially in low- and middle-income countries (LMICs), impacting on access to quality, safe and efficacious medical products in those settings. The Institute of Medicine (IOM) report Ensuring Safe Foods and Medical Products Through Stronger Regulatory Systems Abroad identified the current ad hoc and inconsistent trainings offered to LMICs as part of the problem. While WHO has established a well-recognized process for benchmarking and strengthening regulatory systems, there is a growing recognition that the current approach in regulatory capacity development must include a common global competency framework to achieve the desired public health outcomes. 17 The terms of reference and composition of the review committee are under development. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 The list would be accompanied by summary information, including: • an outline of the organizational structure and responsibilities of the regulatory system evaluated; • evidence reviewed, and the process undertaken to support the listing; • scoring (overall percentage of indicators met) and maturity level per regulatory function (see Annex 3); • a summary of recommendations on further improvement; and • a link to relevant website/webpages of the regulatory authority. It may also include additional information, for example, regarding the pharmaceutical profile of the country. Full details of the evaluation are available to the regulatory authority in question. Regulatory authorities are free to share any information related to the evaluation with other parties. 9. RENEWALS AND VALIDITY PERIOD Renewal of listing would be based on a risk-based re-evaluation process that takes into consideration the fact that WLA authorities should be in a state of continual improvement. Typically, re-evaluation would involve a desk-based review by WHO appointed regulatory experts supplemented, if required, by on-site evaluation. The re-evaluation would be based on a self-assessment by the authority. Additionally, WLAs would be required to confirm on a periodic basis18, or as appropriate, that no changes have taken place that could negatively impact the WLA listing, in addition to any significant developments that should be reflected in updated summary information published on the WHO website. A validity period of seven (7) years is proposed for ML 3 WLAs, provided no changes have taken place that could negatively impact the WLA listing. Following the initial evaluation by WHO, it is proposed that ML 4 WLAs would not be subject to subsequent re-evaluation in relation to the program or regulatory function listed as ML 4, provided that no event has taken place which could cause concern and trigger a re-evaluation of the authority. WHO nonetheless reserves the right to conduct a re-evaluation of an authority based on concerns related to the continued validity of evidence supporting WLA designation (see below). 18 To be defined in operational guidance. 152 WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 153 This approach takes into account the fact that these regulatory systems may be considered the most advanced and transparent, with continual improvement processes in place. In all cases, reporting requirements described above apply. Consistent with the above, WHO will continue to adopt a risk-based approach that devotes greater resources and effort to strengthening less mature systems, taking into account the potential impact on the production and supply of quality assured medical products. 10. DELISTING Similar to the practice established by the WHO PQ program in relation to the delisting of prequalified products, WHO reserves the right to delist a regulatory system should, upon evaluation and subsequent discussion with the regulatory authority, WHO concludes that the basis for supporting the listing is no longer valid. This could be due to evidence of serious and persistent lapses in regulatory oversight, ineffective enforcement activities, a major downsizing or re-organization of the authority, or a lack of response to requests for re-evaluation. It could also be a result of the voluntary decision of the regulatory authority. The decision to delist would be made based on a recommendation from the WHO independent review committee of experts19 following a meeting with the regulatory authority, during which the authority would have an opportunity to present its case. Should a delisting occur, the authority would be provided the basis for the decision, which would then be made public. The authority would be permitted to reapply for listing based on an agreed corrective action plan and a possible re-benchmarking by WHO. 11. TRANSITIONAL ARRANGEMENTS Transitional arrangements are proposed to ensure that the introduction of the listing process does not disrupt the supply of prequalified and other quality assured medical products for purchase by UN agencies and countries. Arrangements also take in consideration ECSPP recommendations and evidence already available to the WHO. Regulatory authorities identified as SRAs20 for medicines regulation by the current interim definition will be entered onto the list of ML 4 WLAs for a period of five years as of the date the WLA framework comes into effect. The same would apply to regulatory authorities that comply with the requirements of Technical Report Series (TRS) 978 (Annex 6) in relation to vaccines regulation. 19 The same committee involved in the recommendation to list the WLA. 20 For the purpose of transitional arrangements, SRA within the context of the European Union (EU), applies to market authorisations granted through centralised, decentralised or mutual recognition procedures. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 National Regulatory Authorities of Regional Reference (NRArr)21 evaluated by Pan American Health Organization (PAHO/WHO) within the last three (3) years from the effective date of the WLA framework will be entered onto the list as ML 3 WLAs for a period of five (5) years. During this period, WHO will work with the above authorities at their request to develop a plan for documenting their maturity and performance as a pre-requisite to re-listing. All available supporting evidence would be considered in developing the plan and could include a desk-based assessment performed by a team of peers under the direction of WHO against the GBT performance evaluation framework using information in the public domain whenever possible22. The plan for relisting of an NRArr could target ML 4 at the request of the regulatory authority based on the previous benchmarking by PAHO/WHO and a gap analysis against the GBT (Revision VI). Similarly, for authorities that have been formally benchmarked since 2016 using the previous version of the GBT (Revision V) and found compliant with indicators for ML 3, a gap analysis against GBT (Revision VI) would form the basis for developing a plan for listing. 12. PROCESS AND TIMELINES FOR FINALIZING THE DEFINITION AND THE PROCEDURES FOR EVALUATING AND DESIGNATING A WHO-LISTED AUTHORITY WHO intends to complete the drafting of the WLA policy document by the end of July 2019 and implement the framework in phases beginning with a six (6) month pilot in the first quarter of 2020. The definition for WLA will also need to be reviewed and endorsed by WHO Expert Committees in the context of its usage in existing WHO guidelines in place of the term SRA. The WLA framework would also have implications regarding the abridged/streamlined procedures for prequalification of medicines/vaccines. WHO invites interested and eligible regulatory authorities to volunteer for the pilot in order to gain experience with and help shape refinements to the proposed framework. 21 An NRArr is a regulatory authority that has reached Level 4 as established using the benchmarking tool and process established by PAHO/WHO in conjunction with the regulatory authorities of the Americas Region. 22 Based on experience, transparency is a proxy for a well-performing regulatory authority. 154 WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 155 Given the wide interest in and implications associated with the definition and policy, WHO will adopt a multi-prong consultation process that includes: • circulation of the draft policy including definitions through the routine WHO Expert Committee consultation processes, which includes posting for comment on the WHO website, circulation through the Expert Committee network and publication in WHO Drug Information; • circulation through established Member State contact lists; • presentation at relevant conferences and international/regional fora; and • organization of dedicated consultative meetings (dates and location to be confirmed). ANNEX 1 Maturity level The WHO Global Benchmarking Tool (GBT) incorporates the concept of maturity levels, adapted from the International Standard ISO 9004:2009. This concept is not new within the context of regulatory systems benchmarking, having been implemented through the Benchmarking of the European Medicines Agencies (BEMA) since 2004. The concept has also been extensively discussed within WHO as well as during two WHO international consultations conducted in Geneva, Switzerland, in January and December 2015. By applying the concept of maturity levels according to a well-defined algorithm, regulatory authorities are able to ascertain their level of development or ‘regulatory maturity’. The concept of maturity level also allows for the definition of more advanced systems that in turn should facilitate reliance and greater regulatory cooperation. In applying the maturity level concept to the tool, the approach of defining critical indicators used in the earlier WHO vaccine tool has been eliminated. Maturity of regulatory systems is divided into four levels: (1) some elements of regulatory systems exist; (2) evolving national regulatory system that partially performs essential regulatory functions; (3) stable well-functioning and integrated regulatory system; and (4) regulatory system operating at advanced level of performance and continuous improvement. The attributes under each of the four levels are well defined, with full consideration of the WHO good regulatory practice principles. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 156 Figure 2: Maturity levels WHO Drug Information, Vol 33, No. 2, 2019 Consultation Documents 157 ANNEX 2 Examples of WHO-Listed Authority designations As noted, a regulatory authority may be designated a maturity level (ML) 3 or ML 4 WHO- Listed Authority (WLA) for the regulation of generic medicines, for new medicines and/or for vaccines, reflecting the current scope of the WHO benchmarking program. A regulatory authority may also be designated a ML 4 WLA for one or more regulatory functions, such as inspection. In all cases, the listing would specify the scope of designation. To qualify for ML 4 listing for a specific regulatory function, the regulatory authority must at a minimum meet ML 3 WLA requirements for either medicine or vaccine regulation. Examples of both types of listings are provided below, together with the scope of regulatory functions that would be assessed. Example 1: Regulatory authority seeking ML 3 WLA recognition for generic medicines program The National Regulatory System (RS) and following regulatory functions would be evaluated: Registration and Marketing Authorization (MA), Vigilance (VL), Market Surveillance and Control (MC), Licensing Establishments (LI), Regulatory Inspection (RI), Laboratory Access and Testing (LA). As the targeted product type/family is generic medicines, Clinical Trials Oversight (CT) and NRA Lot Release (LR) will be excluded from the evaluation. Note: Good Clinical Practice (GCP) inspection is covered under Regulatory Inspection (RI) in the GBT. Example 2: Regulatory authority seeking ML 4 WLA recognition for good manufacturing practices (GMP) inspection function Having attained ML 3 WLA status for generic medicines program, the same regulatory authority may subsequently decide to target ML 4 for a specific regulatory function, for example, GMP inspection. The GMP inspection function would then need to comply with relevant ML 4 indicators and be documented to consistently adhere to international standards before being designated ML 4. Consultation Documents WHO Drug Information, Vol 33, No. 2, 2019 158 ANNEX 3 Example of scoring and maturity level Example of scoring results and maturity level by regulatory function for the evaluation of the regulation of medicines: *** NRA Function Assessed Percentage Implemented Maturity Level 01-NATIONAL REGULATORY SYSTEM (RS 81.0% 3 02-REGISTRATION AND MARKETING AUTHORIZATION (MA 90.0% 3 03-VIGILANCE (VL 85.0% 3 04-MARKET SURVEILLANCE AND CONTROL (MC 85.0% 3 05-LICENSING ESTABLISHMENT (LI 86.0% 3 06-REGULATORY INSPECTION (RI 100.0% 4 07-LABORATORY TESTING (LT 88.0% 3 08-CLINICAL TRIAL’S OVERSIGHT (CT 90.0% 3 09-NRA LOT RELEASE (LR Not Applicable