Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera

Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment TSS-5 Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment: Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera Published 15 March 2018 ISBN 978-92-4-151371-5 © World Health Organization 2018 Some rights reserved. This work is available under the Creative Commons Attribution- NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization. Suggested citation. Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment: Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera. Geneva: World Health Organization; 2018. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Contents Contents .................................................................................................................................................. 1 Acknowledgements ................................................................................................................................. 2 List of contributors .................................................................................................................................. 2 Abbreviations .......................................................................................................................................... 4 A. Introduction .................................................................................................................................... 5 B. Other guidance documents ............................................................................................................ 6 C. Performance principles for WHO Prequalification ......................................................................... 6 C.1 Intended use ................................................................................................................................. 6 C.2 Diversity of specimen types, users and testing environments and impact on required studies .. 6 C.3 Applicability of supporting evidence to IVD under review ........................................................... 7 D. Explanation of terms applied to Vibrio cholerae for the purposes of describing strains detected by an IVD ................................................................................................................................................. 8 E. Table of requirements..................................................................................................................... 9 Part 1 Establishing analytical performance characteristics ......................................................... 10 Part 2 Establishing clinical performance characteristics ............................................................. 18 F. References .................................................................................................................................... 21 Page 1 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Acknowledgements The document Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment: rapid diagnostic tests used for surveillance and detection of an outbreak of cholera was developed with support from the Bill & Melinda Gates Foundation and Unitaid. The document was prepared in collaboration with C. Yansouni, Associate Director, J.D. MacLean Centre for Tropical Diseases, Assistant Professor, Division of Infectious Diseases, Department of Microbiology, McGill University Health Centre, Montreal, Quebec, Canada., R. J. S. Duncan, London, United Kingdom of Great Britain and Northern Ireland; D. Healy, R. Meurant, M. Lanigan Prequalification Team – Diagnostic Assessment, WHO, Geneva, Switzerland with input and expertise from E. Houpt, Jack Gwaltney Professor of Medicine in the Division of Infectious Diseases and International Health, University of Virginia, Charlottesville, Virginia, United States of America (USA); J. Liu, Assistant Professor, University of Virginia, Charlottesville, Virginia, USA who both provided unpublished data on cholera bacterial loads, and technical and programmatic input from K Alberti, D. Legros and D. Olson, High Threat Pathogens, Department of Infectious Hazard Management and P. Olliaro, Intervention and Implementation Research, Special Programme for Research and Training in Tropical Diseases, WHO, Geneva. This document was produced under the coordination and supervision of R. Meurant and I. Prat, Prequalification Team – Diagnostic Assessment, WHO, Geneva, Switzerland. List of contributors A technical consultation on WHO Prequalification requirements for rapid diagnostic tests used for surveillance and detection of an outbreak of cholera was held in Geneva, Switzerland from 16 to 18 October 2017. Meeting participants: N. Ahmed, Senior Director, International Centre for Diarrhoeal Diseases Research Bangladesh (ICDDR,B), Dhaka, Bangladesh; J. Carter, Technical Director, Clinical and Diagnostics Programme, Amref Health Africa, Nairobi, Kenya; R. J. S. Duncan, London, United Kingdom of Great Britain and Northern Ireland; D. Goldfarb, Medical Microbiologist, Department of Pathology & Lab Medicine, BC Children's & Women's Hospitals; Clinical Associate Professor, University of British Columbia, Vancouver, Canada1; M. Guillerm, Diagnostics Consultant, France; C Ilonze, Head, Biologics, Vaccines and Medical Devices Registration and Regulatory Affairs, National Agency for Food & Drug Administration & Control (NAFDAC), Lagos, Nigeria; L. Li, Scientific Reviewer, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration,Silver Spring, Maryland, USA2; J. Liu, Assistant Professor, University of Virginia, Charlottesville, Virginia, USA; A. L. Page, Epidemiologist, Epicentre, Paris, France; E. Piriou, Médecins Sans Frontières, Amsterdam, The Netherlands; M. L. Quilici, Enteric Bacterial Pathogens Unit, French National Reference Centre for Vibrios and Cholera, Institut Pasteur, Paris, France3; U. 1 Participated via web conferencing 2 Participated via web conferencing 3 Participant with observer status Page 2 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Scherf, Director, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA4; W. Sebonego, Technical Specialist, UNICEF Supply Division Health Technology Centre, Copenhagen, Denmark; E. Tran, International quality assurance coordinator for medical devices, Médecins Sans Frontières International, Switzerland; and C. Yansouni, Associate Director, J.D. MacLean Centre for Tropical Diseases, Assistant Professor, Division of Infectious Diseases, Department of Microbiology, McGill University Health Centre, Montreal, Quebec, Canada. WHO Secretariat: D. Healy; M. Lanigan; R. Meurant; W.Urassa, Prequalification Team – Diagnostic Assessment Group, Department of Essential Medicines and Health Products; D. Olson; and K. Alberti, High Threat Pathogens, Department of Infectious Hazard Management, Geneva. The draft technical specifications document was posted on the WHO website for public consultation on 09 November 2017. Various stakeholders, including manufacturers submitting to WHO Prequalification of IVDs, IVD manufacturing industry associations, various national and international regulatory bodies, and IVD standards organizations were informed of the consultation in order to solicit feedback. A one- month response period was provided. Public comments were received for consideration from Asian Harmonization Working Party (AHWP) Working group 2, China, Hong Kong SAR; ARKRAY Healthcare Pvt. Ltd, Dist. Surat, India; College of American Pathologists, Northfield, Illinois, USA; Global Health Investment Fund, New York, USA; P. W. Kamau, Pharmacy and Poisons Board, Nairobi Kenya; In Vitro Diagnostic Devices Evaluation Division, Medical Devices Bureau, Therapeutic Products Directorate, Health Canada, Government of Canada, Canada; L. Li, Scientific Reviewer, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA; J. Liu, Assistant Professor, University of Virginia, Charlottesville, Virginia, USA; A. L. Page, Epidemiologist, Epicentre, Paris, France; E Piriou, Médecins Sans Frontières, Amsterdam, The Netherlands; M. L. Quilici, Enteric Bacterial Pathogens Unit, French National Reference Centre for Vibrios and Cholera, Institut Pasteur, Paris, France; M. B. Rumaney, Goodwood, Western Cape, South Africa; U. Scherf, Director, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA; E. Tran, International quality assurance coordinator for medical devices, Médecins Sans Frontières International, Switzerland. 4 Participated via web conferencing Page 3 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Abbreviations ATCC American Type Culture Collection CFU colony forming units E. coli Escherichia coli EIA enzyme immunoassay IFU instructions for use IVD in vitro diagnostic medical device LPS lipopolysaccharide O1 serogroup 1 O139 serogroup 139 RDT rapid diagnostic test spp. species TSS Technical Specifications Series U.S. FDA U.S. Food and Drug Administration V. cholerae Vibrio cholerae WHO World Health Organization Page 4 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 A. Introduction The purpose of this document is to provide technical guidance to in vitro diagnostic medical device (IVD) manufacturers that intend to seek WHO Prequalification of rapid diagnostic tests (RDTs) for the detection of toxigenic Vibrio cholerae. This document is relevant to RDTs that can be used near to the patient, in settings outside of a laboratory environment, and that detect toxigenic V. cholerae strains that are associated with epidemic spread. It must be able to be used by health care workers and trained community workers. A V. cholerae IVD submitted for WHO Prequalification is expected to detect V. cholerae O1 or O1/O139 in combination, (being the strains consistently considered toxigenic), or are capable of detecting O1/O139 toxin prior to strain confirmation. IVDs will only be prequalified based on the evidence that supports their use for the detection of outbreaks or surveillance for this disease. The requirements outlined in this document do not include those that demonstrate that the IVD can be used for other purposes, such as the diagnosis of an individual, or for patient management. Nor do they include requirements for use of the product for environmental testing. If claimed, these additional intended uses must be clearly stated and supported by relevant evidence. Likewise, field-deployable rapid tests for cholera that are quantitative or are based on other technologies, such as nucleic acid amplification tests, will have different and/or additional requirements for WHO Prequalification and these are not included in this TSS. For the purpose of this document, the verbal forms used follow the usage described below: • “shall” indicates that the manufacturer is required to comply with the technical specifications; • “should” indicates that the manufacturer is recommended to comply with the technical specifications but it is not a requirement; • “may” indicates that the technical specifications are a suggestion to undertake the testing but it is not a requirement. Minimum requirements for WHO Prequalification to demonstrate performance are summarized in this document, and where possible, WHO performance requirements are aligned with published guidance, standards and/or regulatory documents. Although references to source documents are provided, in some cases WHO Prequalification has additional requirements. A documented justification and rationale shall be provided by the manufacturer when the WHO Prequalification submission does not comply with the required technical specifications outlined in this document. For WHO Prequalification purposes, manufacturers shall provide evidence in support of the clinical performance of an IVD that can demonstrate that reasonable steps have been taken to ensure that a properly manufactured IVD, being correctly operated in the hands of the intended user, will reproducibly detect the target analyte and fulfil its intended use. If the manufacturer supplies a reader for use with the IVD, safety and performance data shall be provided in the dossier with and without the use of the reader. WHO Prequalification requirements summarized in this document do not extend to the demonstration of clinical utility, i.e. the effectiveness and/or benefits of an IVD, relative to and/or in combination with other measures, as a tool to inform clinical intervention in each population or health care setting. To demonstrate clinical utility, a separate set of studies is required. Clinical utility Page 5 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 studies usually inform programmatic strategy and are thus the responsibility of programme managers, ministries of health and other related bodies in individual WHO Member States. Such studies do not fall under the scope of WHO Prequalification. B. Other guidance documents This document should be read in conjunction with other WHO guidance documentation, including: • Technical Guidance Series for WHO Prequalification – Diagnostic Assessment • Sample Product Dossiers for WHO Prequalification – Diagnostic Assessment • Instructions for Compilation of a Product Dossier, WHO document PQDx_018. These documents are available at: http://www.who.int/diagnostics_laboratory/evaluations/en/ C. Performance principles for WHO Prequalification C.1 Intended use An IVD intended for WHO Prequalification shall be accompanied by a sufficiently detailed intended use statement. This should allow an understanding of at least the following: • the function of the IVD (e.g. to specifically detect O1 and O139 strains of V. cholerae, etc. as appropriate) and that the result is qualitative; • the testing population for which functions are intended, e.g. patients presenting with symptoms of acute watery diarrhoea in suspected outbreak settings; • the intended operational setting (e.g. primary health care level with no access to standard laboratory facility or settings and no access to electricity, or, community settings outside health facilities where cholera outbreak is suspected); • clinical indication, e.g. for detection and monitoring of epidemic cholera; and • the intended user. C.2 Diversity of specimen types, users and testing environments and impact on required studies Depending on the intended use of the IVD, performance studies shall be designed to consider the diversity of knowledge and skills of potential IVD users, and the likely operational settings in which testing is likely to occur. It is a manufacturer’s responsibility to ensure that the risk assessment for an IVD reflects the intended operational settings, including service delivery complexity, and the likely user population conducting the test. Prequalified IVDs in low- and middle-income countries are likely to be used by laboratory professionals5, healthcare workers6 as well as trained lay providers7. 5 Medical technologists, medical laboratory technicians or similar, who have received a formal professional or paraprofessional certification or tertiary education degree. 6 Any person who performs functions related to healthcare delivery and has received a formal professional or paraprofessional certification or tertiary education degree 7 Any person who performs functions related to healthcare delivery and has been trained to deliver specific services but has received no formal professional or paraprofessional certification or tertiary education degree. Page 6 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 In addition, health care workers (or other users) may be required to perform testing on specimens that are collected either by a different health care worker or by the patient themselves. This shall be considered during the risk assessment. The complexity of a test, from specimen collection to result interpretation, shall be clearly elucidated in the IVD instructions for use (IFU) and reflected in risk analysis. C.3 Applicability of supporting evidence to IVD under review Performance studies shall be undertaken using the version of the IVD intended to be submitted for WHO Prequalification. This version of the IVD shall be made to the finalized, approved documentation, including the quality assurance and quality control specifications and the performance studies shall be performed according to the finalized method documented in the IFU. Where this is not possible a justification shall be provided; additional supporting evidence may also be required. This may occur in the case of minor variations to the design where no negative impact on performance has been demonstrated. Where alternative methods are proposed in the IFU, such as the possible addition of a sample enrichment step, performance studies shall be performed for each method documented in the IFU. If the methods produce difference performances, these shall be reported clearly in the IFU. Specific information is provided in Parts 1 and 2 of this document for the numbers of lots required for particular studies. Each lot should comprise different production (or manufacturing, purification, etc.) runs of critical reagents, representative of routine manufacture. It is a manufacturer’s responsibility to ensure that, via risk analysis of its IVD, minimum numbers of lots chosen for estimating performance characteristics are commensurate with the variability in performance likely to arise from the diversity of key components and their formulation. The manufacturer shall also provide appropriate justifications for the number of lots chosen for estimating performance characteristics. The true V. cholerae status and, if discrimination is claimed, the serogroup or serotype of a specimen shall be determined using a suitable reference standard. For WHO purposes this should be a standard that currently is at a developed stage of technical capability based on the relevant consolidated findings of science, technology and experience (commonly referred to as state of the art). Justification for the choice of standard shall be provided. Estimation (and reporting) of IVD performance shall include the rate of invalid test results and the 95% confidence interval around the estimated values for key performance metrics, as appropriate. Analytical studies should include testing for all specific characteristic (e.g. serogroups, serotypes, or toxin production) for which detection is claimed. For IVDs that include a claim for differentiation (e.g. differentiation of O1 or O139, etc.), evidence of performance shall be provided for each claimed analyte. Where no claim of differentiation is made, it may be acceptable for some analytical studies to generate most of evidence of performance in a single toxigenic strain (e.g. O1 Ogawa) as a surrogate for performance in other related serotypes. A justification for this approach, including a relevant demonstration of equivalence between serotypes/serogroups, shall be provided. Example: for an IVD in which detection of several serogroups or serotypes is claimed, analytical sensitivity shall be determined for each specified serogroup or serotype. The use of characterized repository specimens may be acceptable provided they are relevant to the IVD under assessment taking into consideration: • collection media, collection device (swabs, etc.) • in addition to any requirement for testing in fresh (unpreserved and unfrozen) specimens only. Page 7 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 For some analytical studies, it may be acceptable to use contrived specimens (e.g. bacterial cultures spiked into normal human specimens or into media used in the IVD or, a negative clinical matrix, as appropriate). All bacterial strains used for contrived specimens must be characterized – either by the catalogue number of the commercial source or by providing details of the laboratory from where they are sourced as well as a description of what methods were used for identification of the species and strain. For WHO Prequalification submission, clinical performance studies shall be conducted using each specimen type and collection material (e.g. swabs, collection media) that are claimed in the IFU. Clinical studies shall be based on testing in natural specimens only (a stool specimen originating from a person suspected of having cholera) for V. cholerae O1. However, for V. cholerae O139 where clinical specimens are difficult to isolate, contrived specimens may be acceptable. D. Explanation of terms applied to Vibrio cholerae for the purposes of describing strains detected by an IVD Only specific strains of V. cholerae have the potential for epidemic spread, characterized by specific “O” serogroups and the production of cholera toxin. Individual strains within a serogroup are described according to both serotype and biotype. Serogroup: More than 200 serogroups of the “O” lipopolysaccharide (LPS) are currently recognized within the species V. cholerae. These include the toxigenic O1 and O139 serogroups which are associated with epidemic cholera. Importantly, some strains of V. cholerae O1 and O139 do not produce cholera toxin, and these do not cause cholera disease or epidemics. Some serogroups other than O1 or O139 can cause sporadic disease, but these are not associated with epidemic spread. Serotype: These are subdivisions within serogroups based on specific antigens of the O-group LPS. There main serotypes within the O1 serogroup are Inaba, Ogawa and Hikojima. A given O1 strain can occasionally shift between serotypes. Biotype: Biotypes represent a set of specific phenotypic properties that characterize particular strains of V. cholerae within a serogroup. For example, the O1 serogroup has two biotypes, “classical” and “El Tor”. Biotypes are not specified by serotypes, and are defined by the acquisition of phenotypic characteristics unrelated to O-group LPS antigens. Cholera toxin: The enterotoxin produced by toxigenic V. cholerae strains, responsible for the massive watery diarrhoea seen with cholera infection. Similar in mechanism and sequence to the heat-labile enterotoxin secreted by some strains of Escherichia coli. Page 8 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 E. Table of requirements PART 1 Establishing analytical performance characteristics 1.1 Specimen types 1.1.1 Specimen types 1.2 Specimen collection, storage and transport 1.2.1 Specimen stability 1.3 Precision of measurement 1.3.1 Repeatability, reproducibility 1.4 Performance panels 1.4.1 Performance panels 1.5 Validation of reading times 1.5.1 Validation of reading times 1.6 Analytical sensitivity 1.6.1 Analytical Sensitivity 1.7 Establishment of reader cut-off 1.7.1 Establishment of reader cut-off 1.8 High dose hook effect 1.8.1 High dose hook effect 1.9 Analytical specificity 1.9.1 Potentially interfering substances 1.9.1.1 Endogenous 1.9.1.2 Exogenous 1.9.2 Cross-reactivity, including selectivity with non-virulent O1 strains of V. cholerae 1.10 Stability 1.10.1 Shelf-life (including transport stability). 1.10.2 In-use stability (open pack or open vial stability). 1.11 Flex studies 1.11.1 Flex studies/robustness PART 2 Establishing clinical performance characteristics 2.1 Diagnostic sensitivity and specificity 2.1.1 Diagnostic sensitivity 2.1.2 Diagnostic specificity 2.2 Qualification of usability 2.2.1 Labelling comprehension study (including IFU) 2.2.2 Results interpretation study Page 9 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Part 1 Establishing analytical performance characteristics Aspect Testing requirements Notes on testing requirements Source documents 1.1 Specimen types 1.1.1 Specimen types For each claimed specimen type, testing of at least: • 25 V. cholerae matched negative specimens; • 25 V. cholerae matched positive specimens. Equivalence shall be determined for each claimed V. cholerae serogroup/serotype, as appropriate The relationship between IVD performance in claimed specimen 1. types and materials used for analytical studies shall be clearly established. The design of subsequent studies shall then take that relationship into account. If there is no equivalence between claimed specimen types 2. and/or the materials used for analytical studies, then the impact that this will have on each subsequent performance claim shall be fully understood and described. Example: an IVD intended for testing rectal swabs as well as fresh stools for which the performance is evaluated using a dilution of a cell culture into assay diluents or into stool specimens. • The relationship between the characteristic in diluents to that in a natural specimen (fresh stools and that obtained by using rectal swabs) shall be understood. • The relationship between analytical sensitivity in stool specimens or diluents to that of the same characteristic in rectal swabs shall be understood. If rectal swabs are recommended for use with the IVD, the details 3. (brand, product code etc.) of swab shall be identified and its use validated. The performance for each specimen type shall be established. A prospective study comparing paired rectal swabs and stool specimens may be performed. Positive specimens shall be chosen so that a majority are at a 4. concentration near the limit of detection. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment – TGS-3 (1) 1.2 Specimen collection, storage and transport 1.2.1 Specimen stability Real time studies considering: • storage conditions (duration at different temperatures and variation in humidity, Attention shall be paid to the length of time likely to elapse 1. between specimen collection and its addition to the IVD in the settings where this IVD may be used. Page 10 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents temperature limits, freeze/thaw cycles); • transport conditions, where applicable; • intended use (see note 1); • specimen collection and/or transfer devices, whether these contain transport media, and whether they can be sealed. Unless all specimens are expected to be processed as fresh 2. samples within a specified time frame, the IVD performance shall be established under different storage conditions. The IVD shall generate similar results for the stored specimens at 3. several time points throughout the duration of the recommended storage period and at the upper and lower limit of the recommended storage temperature range. The specimen storage studies shall include specimens with bacterial load majority are at a concentration near the limit of detection. In addition to the analysis of qualitative results, there shall be an 4. analysis of the semi-quantitative results. In all cases, swab types and transport medium shall be specified 5. (brand, product code etc.) and their use validated when claimed as part of the intended use. 1.3 Precision of measurement 1.3.1 Repeatability, reproducibility Both repeatability (within-condition – see note 1) and reproducibility (between-condition – see note 1) shall be estimated by replicate testing at limit of detection (end-point dilutions) of several analyte-positive specimens. Specimens chosen for the testing panel shall include panel members that reflect the main specimen types intended for use with the IVD (e.g. swab or stool specimen). Each panel member shall be tested: • using 3 different lots (see note 4); • over 5 days (not necessarily consecutive) with one run per day (alternating morning/afternoon); • at each of 3 different testing sites. E.g. within- or between-run, -lot, -day, -operator, -site, etc. 1. The testing panel shall be the same for all operators, lots and 2. sites. Low-reactivity specimens shall be in sufficient replication for each 3. run to allow imprecision to be quantitated. Each lot shall comprise different production (or manufacturing, 4. purification, etc.) runs of critical reagents. Results shall be statistically analysed to identify and isolate the 5. sources and extent of any variance. The percentage of correctly identified, incorrectly-identified and 6. invalid results shall be tabulated for each specimen and be separately stratified according to each site, lot, etc. This type of analysis is especially important for RDTs that may not have any numerical values. To understand irregularities in results obtained, at least 2 lots 7. should be tested at each of the 3 testing sites. The effect of operator-to-operator variation on IVD performance 8. CLSI EP12-A2 (2) CLSI EP17-A2 (3) EN 13612:2002 (4) Page 11 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents The effect of operator-to-operator variation on IVD performance is to be included as part of the precision studies (see also note 8 and 9). Testing should be performed: • by personnel representative of intended users; • unassisted; • using only those materials provided with the IVD (e.g. IFU, labels and other instructional materials). may be used in conjunction with other studies to qualify the usability of the IVD. Users shall be selected based on a pre-determined and 9. contextually appropriate level of education, with literacy and auxiliary skills that will challenge the usability of the IVD and reflect the diversity of intended users and operational settings. These characteristics shall be detailed in the submission. 1.4 Performance panels 1.4.1 Performance panels Testing of the IVD shall be undertaken using suitable performance panels (e.g. comprising relevant organism variants, subtypes, etc.) as available. Testing should be performed using more than one lot of the final 1. design (locked-down). It is acceptable to use contrived specimens that comprise stool 2. specimens spiked with microorganisms from culture collections. 1.5 Validation of reading times 1.5.1 Validation of reading times For IVDs where a reading interval is specified (i.e. time when result can first be read; time beyond which result should not be read), validation of critical time points shall be provided. Performance studies shall be conducted at the extremes of the intended operational temperature range; the effect of humidity on reading times shall also be investigated. The ranges of temperature and humidity tested for should be 1. risk-based, taking into consideration knowledge of conditions likely to be experienced in intended operational settings. The intended operating temperature range, upon which reading 2. time has been validated, shall be clearly stated in the IFU. This work shall be performed with IVDs near the beginning and 3. near the end of their assigned shelf-lives. Some of these aspects may be evaluated within the flex studies 4. (1.11.1). Technical Guidance Series for WHO Prequalification – Diagnostic Assessment TGS-3 (1) 1.6 Analytical sensitivity 1.6.1 Analytical Sensitivity Analytical sensitivity shall be determined for each claimed target strain as appropriate (see Introduction section C3) as follows: • using 3 lots (see note 1) Each lot shall comprise different production (or manufacturing, 1. purification, etc.) runs of critical reagents. Ideally the lots should be chosen from across the assigned shelf-life of the IVD. This study may alternatively be performed as part of the shelf-life 2. CLSI EP17-A2 (3) WHO Prequalification – Diagnostic Assessment Page 12 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents • a minimum of 5 replicate tests of each member of an 8-member dilution series of a suitable biological material, the concentrations of which are closely spaced and which cross the limit of detection. Analytical sensitivity shall be estimated by determining the lowest concentration for which the rate of detection is 95%. verification of the IVD. Analytical sensitivity shall be demonstrated in a clinical sample 3. matrix and shall use the entire assay system from sample preparation to interpretation. Justification shall be provided for the choice of reference cultures 4. and dilution media used in the dilution series. The estimate of analytical sensitivity should be confirmed by 5. separately testing an additional 20 replicates. Where a claim is made for detection of specific V. cholerae 6. serogroups/serotypes, performance characteristics shall be determined in each strain type claimed. If analytical sensitivity for one or more given strain types of V. cholerae is not appropriate for intended users, then this limitation of the IVD shall be clearly reported in the IFU. PQDx_018 (5) 1.7 Establishment of reader cut-off 1.7.1 Establishment of reader cut- off In V. cholerae RDTs provided with a reader, the way in which the reader has been designed to differentiate positive specimens from negative specimens shall be demonstrated. 1.8 High dose hook effect 1.8.1 High dose hook effect The potential for a high dose hook effect shall be determined using each of the claimed serotype or serogroup with at least three finalized lots of the IVD. This study may be undertaken using cultures expressing high 1. concentration of the target analytes claimed to be detected, or with natural specimens found to have a very high titre in enzyme immunoassays (EIA) or by molecular methods. Strains shall be representative of the diversity among each 2. serotype or serogroup claimed. This would require showing the specimens yield positive results 3. over the range of 109 – 1012 CFU/mL for the claimed serogroup/serotype. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment TGS-6 (7) Page 13 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents 1.9 Analytical specificity 1.9.1 Potentially interfering substances The potential for false results (false-negatives and false-positives) arising from interference from, at least, the substances/conditions listed below shall be determined (see note 1). • a minimum of 100 specimens; • each substance represented by at least 3-5 specimens from different individuals. Testing shall be undertaken in both V. cholerae negative and -positive specimens spiked with each potentially interfering substance at physiologically relevant dosages. The risk assessment conducted for an IVD should identify 1. substances where the potential for interference can reasonably be expected for the analyte being detected (e.g. laxatives). Any observed interference shall be investigated and performance 2. limitations of the IVD reported in the IFU. Results shall be reported with respect to each condition and not be reported as an aggregate of the total number of specimens tested in the study. Where a claim is made for identification of different V. cholerae 3. serogroups/serotypes, performance characteristics shall be determined for each. At a minimum this shall include detection in specimens positive for Ogawa, Inaba and O139. (Note that evaluation with uncharacterized specimens is insufficient). Where testing in these serogroups/serotypes has not been undertaken, this limitation of IVD performance shall be reported in the IFU. Evaluation of endogenous interfering substances may be 4. addressed as part of the clinical studies. In this case, a description of the faecal form shall be provided. European Commission decision on CTS (6) CLSI EP07-A2 (8) 1.9.1.1. Endogenous Clinical specimens selected for the presentation of: • blood; • excessive lipids; • excessive mucus (see note 4). 1.9.1.2. Exogenous Relevant medicines, including: • antimalarial, antiretroviral and anti- tuberculosis medications; • laxatives; • common anti-diarrhoeal medicines; • common over-the-counter anti- inflammatory/analgesic medications (paracetamol, ibuprofen); • ethanol; • caffeine; • azithromycin; • ciprofloxacin; • co-trimoxazole; • doxycycline; • ivermectin; • albendazole; Page 14 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents • conjugates of loperamide. 1.9.2. Cross- reactivity including selectivity with non-virulent O1 strains of V. cholerae. Strains used may be either well characterized clinical isolates from symptomatic individuals, or a pathogenic type-strain from a recognized culture collection bank such as the American Type Culture Collection (ATCC). • each potentially cross-reacting pathogens listed should be represented by at least 3-5 specimens from different individuals. The following list represents the main attributable diarrhoeal pathogens in low-resource settings where cholera epidemics may occur that require testing for cross-reactivity: Bacteria: • Non-O1/O139 V. cholerae (for IVDs that detect the “O” LPS); • Non-toxigenic V. cholerae including non- virulent O1 strains of V. cholerae (for IVDs that detect cholera toxin itself) • STEC (Shiga toxin-producing E. coli); • ETEC (enterotoxigenic E. coli). Viruses: • Rotavirus; • Adenovirus 40/41 Identification of microorganisms as potentially cross-reacting 1. should be risk-based and may include: Bacteria: • Shigella spp; • Campylobacter spp; • Non-pathogenic E coli; • EAEC (enteroaggregative E. coli); • EPEC (enteropathogenic E. coli); • Aeromonas spp; • Salmonella spp. (see note 2); • Plesiomonas spp; • Vibrio parahaemolyticus; • Vibrio fluvialis • Clostridium difficile The specific species of a genus, or serovar of Salmonella enterica, 2. or pathotype of E coli etc. that are tested shall be described. Page 15 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents 1.10 Stability Replicate testing shall be undertaken using a panel consisting, for each claimed analyte, of at least: • 1 negative specimen; • 2 low positive specimens (with a concentration of analyte just above the limit of detection such that results of repeated tests of this sample are positive approximately 95% of the time); • 1 moderately positive specimen (with a concentration at which one can anticipate positive results approximately 100% of the time e.g., approximately five to ten times the concentration of the limit of detection). 1. Stability studies shall be conducted using the conditions expected in the environment of intended use. 2. Each lot should comprise different production (or manufacturing, purification, etc.) runs of critical reagents. 3. The numbers of invalid tests with each kit lot shall be reported. 4. Transport simulation shall be performed using simulated extreme stress conditions, ensuring that application of those conditions is consistent and controlled. 5. Claims for stability shall be based on the second-last successful data point from the least stable lot, with, if lots are different, a statistical analysis showing that the bulk of lots will be expected to meet the claimed life. For example: for testing conducted at 3, 6, 9, 12 and 15 months, if stability was observed at 15 months, then the maximum stability claim can be 12 months. 6. Accelerated studies do not replace the need for real time studies. 7. In-use stability of labile components shall be conducted using components in their final configuration. 8. In-use stability should be conducted with lots at the beginning and end of their shelf-lives. ISO 23640:2011 (9) CLSI EP25-A (10) Technical Guidance Series for WHO Prequalification – Diagnostic Assessment TGS-2 (11) ASTM D4169-14 (12) 1.10.1 Shelf-life (including transport stability) • real time studies using a minimum of 3 lots of final design product in final packaging. • lots are “transport stressed” (simulated) before real time studies are undertaken on these lots. 1.10.2 In-use stability • minimum of one lot, using panel(s) compiled as above; • testing of all labile components (e.g. buffers vials etc. see note 7). 1.11 Flex studies 1.11.1 Flex studies/ robustness The influence of the following factors on expected results (both reactive and non-reactive) shall be considered: • specimen and/or reagent volume; • reading time; • buffer pH (measure of robustness – e.g. due to evaporation of the buffer); • IVD sturdiness including robustness of 1. Refer to WHO document PQDx_018 “Instructions for compilation of a product dossier” for other flex studies that may be relevant, taking into consideration the broad range of operational and environmental conditions consistent with intended use. 2. The factors listed opposite should be investigated in ways that not only reflect, but also exceed, likely operating conditions in lower- and middle-income countries so that the limitations of the device can be understood. For example, in addition to investigating WHO Prequalification – Diagnostic Assessment PQDx_018 (5) Page 16 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents packaging and labelling. IVD in final packaging shall be subjected to drop-shock testing; • lighting and humidity (See note 3); • operating temperature. Instrumentation (if applicable) including: • ruggedness (see note 4); • impact of dust and mould on componentry (e.g. optics). deviations of temperature within those claimed in the IFU (in the middle and at both lower and upper extremes of a claimed temperature range), temperature ranges should be investigated that exceed those of claimed operating conditions and which cause test failure (incorrect/invalid results). 3. The impact of lighting can be twofold – i.e. the impact of lighting on packaging (e.g. fading), and the sufficiency of lighting to read the test lines. 4. For the purposes of this document, ruggedness means the ability to resist environmental shocks. Page 17 Part 2 Establishing clinical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Part 2 Establishing clinical performance characteristics Aspect Testing requirements Notes on testing requirements Source documents 2.1 Diagnostic sensitivity and specificity Diagnostic sensitivity and specificity shall be determined for each of the specimen types claimed: • by a variety of intended users (e.g. 9 - 12 users); • using more than one lot. 1. For WHO Prequalification purposes, testing should be undertaken in fresh specimens. If more than one specimen collection method is intended for use with the IVD, each of those methods must be evaluated during the clinical studies. 2. In line with the intended use of this product, clinical performance characteristics should be established on symptomatic patients (i.e. with watery diarrhoea). 3. Prequalified RDTs to detect V. cholerae will generally be used by trained lay providers and trained health care workers. For WHO Prequalification purposes, these should also be considered as the intended user in addition to a laboratory professional. 4. A separate aliquot of the specimen or of the extract from the swab shall be collected in parallel to establish the reference result. The testing algorithm used to determine the reference results should include culture and nucleic acid testing since there might be non-viable microorganisms in the material. Justification for the use of the testing algorithm shall be provided. 5. Contrived specimens may be used for determination of diagnostic sensitivity in IVDs to detect V. cholerae O139. 6. Each lot (design locked-down) should comprise different production (or manufacturing, purification, etc.) runs of critical reagents. 7. All discrepant results (between assay under evaluation and the reference results) shall be repeated on the same lot, and then on all available lots and the variability noted. Performance characteristics shall be reported using initial results only. The results of further testing of specimens with U.S. FDA (13) European Parliament IVD regulations (14) 2.1.1 Diagnostic Sensitivity For determination of diagnostic sensitivity, testing shall be conducted: • 150 positive specimens, as confirmed by the reference methods (see note 4) for each of the serogroup or serotype claimed (see note 5). 2.1.2 Diagnostic Specificity For determination of diagnostic specificity, testing should be conducted at different geographical and epidemiological settings representative of intended users (minimum of 2 regions). Testing shall be conducted in: • at least 200 V. cholerae negative specimens from a symptomatic population (i.e. those with watery diarrhoea or non-formed stools). Page 18 Part 2 Establishing clinical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents discrepant results shall be reported separately as additional information about IVD performance. 8. All invalid results shall be recorded and evaluated in comparison to the reference result. Invalid results shall be analysed separately in the final performance calculations. 9. Estimates of diagnostic/clinical sensitivity and specificity shall be reported with 95% confidence intervals. 10. Results shall be reported with respect to each study site and not be reported as an aggregate of the total number of specimens tested to establish these characteristics. 11. Where possible, sensitivity should be established different geographical and epidemiological settings. 2.2 Qualification of usability 2.2.1 Label comprehension study Testing of subjects shall be undertaken to assess ability of intended users to correctly comprehend key messages from packaging and labelling: • understanding key warnings, limitations and/or restrictions; • proper test procedure; • test result interpretation. Studies shall include at least 15 intended users including those whose native language may not be the language of the IFU if necessary, in their usual working environment, not employees of the manufacturer, from two geographically diverse populations to to demonstrate comprehension of key messages in each user group. Intended users include those who are collecting the stool 1. specimen as well as those who are performing the testing of the stool specimen. Trained lay providers and health care workers should both be considered as intended users. If the labelling is available in different languages, the labelling 2. comprehension study should be performed for each language. Requirements listed may be investigated as separate studies 3. or included as part of clinical studies. Testing may be conducted using questionnaire based surveys. 4. European Parliament IVD regulations (14) Backinger CL and Kingsley PA (15) U.S. FDA (16) 2.2.2 Results interpretation study Intended users shall interpret the results of contrived IVDs (e.g. static/pre-made tests) to assess their ability to correctly interpret pre-determined test results. Contrived tests shall be made to demonstrate the following potential test results: Page 19 Part 2 Establishing clinical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents • non-reactive; • range of invalid results; • reactive; • weak reactive. Testing subjects shall consist of: • at least 15 intended users, including those whose native language may not be the IFU language if necessary, • in their usual working environment, not employees of the manufacturer, • from two geographically diverse populations to demonstrate correct interpretation of simulated test results. Page 20 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 F. References 1. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment. Principles for Performance studies, TGS– 3. Geneva: World Health Organization; 2016 http://apps.who.int/iris/bitstream/10665/258985/1/WHO-EMP-RHT- PQT-TGS3-2017.03-eng.pdf?ua=1 accessed 09 October 2017. 2. Evaluation of Precision of Quantitative Measurement Procedures, 3rd Edition - CLSI EP05-A3. Wayne, PA: Clinical and Laboratory Standards Institute; 2014. 3. Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline- Second Edition. CLSI EP17-A2; Wayne, PA: Clinical and Laboratory Standards Institute; 2012. 4. EN 13612:2002. Performance evaluation of in vitro diagnostic medical devices. Brussels: European Committee for Standardization; 2002. 5. WHO Prequalification – Diagnostic Assessment: Instructions for Compilation of a Product Dossier. WHO PQDx_018 v3, 27 August 2014. Geneva: World Health Organization; 2014 (http://www.who.int/entity/diagnostics_laboratory/evaluations/141015_pqdx_018_dossier_instructions_v4.pdf?ua= 1 accessed 09 September 2017. 6. Technical Guidance Series for WHO Prequalification – Panels for quality assurance and quality control of in vitro diagnostic medical devices TGS–6. Geneva: World Health Organization; 2017 http://apps.who.int/iris/bitstream/10665/259408/1/WHO-EMP-RHT-PQT-2017.10-eng.pdf?ua=1 accessed 15 December 2017. 7. Commission Decision of 3 February 2009 amending Decision 2002/364/EC on common technical specifications for in vitro-diagnostic medical devices (2009/108/EC). O. J. E. U. 2009, L39/34-L39/49; (http://eur-lex.europa.eu/legal- content/EN/ALL/?uri=uriserv:OJ.L_.2009.039.01.0034.01.ENG accessed 09 October 2017). 8. Interference Testing in Clinical Chemistry. Approved Guideline - Second Edition. CLSI EP07-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2005. 9. ISO 23640:2011. In vitro diagnostic medical devices - Evaluation of stability of in vitro diagnostic reagents. Geneva: International Organization for Standardization; 2011. 10. Evaluation of Stability of In Vitro Diagnostic Reagents; Approved Guidelines. CLSI EP25-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2009. 11. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment. Establishing stability of in vitro diagnostic medical devices, TGS–2. Geneva: World Health Organization; 2017 http://apps.who.int/iris/bitstream/10665/259742/1/WHO-BS-2017.2304-eng.pdf?ua=1/ accessed 08 October 2017. 12. ASTM International. ASTM D4169-14. Standard Practice for Performance Testing of Shipping Containers and Systems. West Conshohocken, PA: ASTM International; 2014. 13. Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests. Silver Spring, Maryland: Food and Drug Administration; 2007 http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm071287.pd f, accessed 09 October 2017). 14. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in-vitro diagnostic medical devices. O. J. E. U. 2017 L 117/176 http://eur-lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX:32017R0746&from=DE accessed 08 September 2017. 15. Backinger CL and Kingsley PA. Write It Right: Recommendations for Developing User Instructions for Medical Devices Used in Home Health Care. HHS Publication FDA 93-4258; 1993. Silver Spring, Maryland: Food and Drug Administration; 1993 (http://www.fda.gov/downloads/MedicalDevices/.../ucm070771.pdf, Accessed 08 September 2017. 16. Applying Human Factors and Usability Engineering to Medical Devices. Silver Spring, Maryland: Food and Drug Administration; 2016. https://www.fda.gov/downloads/MedicalDevices/.../UCM259760.pdf accessed 09 October 2017. Page 21 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 List of related WHO Publications of related interest WHO Prequalification Team- Diagnostic Assessment. Technical Guidance Series for WHO Prequalification – Diagnostic assessment (available online) WHO Prequalification Team – Diagnostic Assessment. Instructions for Compilation of a Product Dossier. WHO document WHO/PQDx_18 (available online) The Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment set out appropriate performance evaluation criteria to meet prequalification requirements. Each Technical Specification provides information on the minimum performance requirements for WHO Prequalification that should be met by a manufacturer to ensure that their in vitro diagnostic medical device is safe and performs optimally. Further information on these and other WHO publications can be obtained from WHO Press, World Health Organization, 1211Geneva 27, Switzerland http:/www.who.int/bookorders tel.: +41 22 791 3264; fax: +41 22791 4857; email bookorders@who.int ISBN-978-92-4-1513715

Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment TSS-5 Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment: Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera Published 15 March 2018 ISBN 978-92-4-151371-5 © World Health Organization 2018 Some rights reserved. This work is available under the Creative Commons Attribution- NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization. Suggested citation. Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment: Rapid diagnostic tests (RDTs) used for surveillance and detection of an outbreak of cholera. Geneva: World Health Organization; 2018. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Contents Contents .................................................................................................................................................. 1 Acknowledgements ................................................................................................................................. 2 List of contributors .................................................................................................................................. 2 Abbreviations .......................................................................................................................................... 4 A. Introduction .................................................................................................................................... 5 B. Other guidance documents ............................................................................................................ 6 C. Performance principles for WHO Prequalification ......................................................................... 6 C.1 Intended use ................................................................................................................................. 6 C.2 Diversity of specimen types, users and testing environments and impact on required studies .. 6 C.3 Applicability of supporting evidence to IVD under review ........................................................... 7 D. Explanation of terms applied to Vibrio cholerae for the purposes of describing strains detected by an IVD ................................................................................................................................................. 8 E. Table of requirements..................................................................................................................... 9 Part 1 Establishing analytical performance characteristics ......................................................... 10 Part 2 Establishing clinical performance characteristics ............................................................. 18 F. References .................................................................................................................................... 21 Page 1 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Acknowledgements The document Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment: rapid diagnostic tests used for surveillance and detection of an outbreak of cholera was developed with support from the Bill & Melinda Gates Foundation and Unitaid. The document was prepared in collaboration with C. Yansouni, Associate Director, J.D. MacLean Centre for Tropical Diseases, Assistant Professor, Division of Infectious Diseases, Department of Microbiology, McGill University Health Centre, Montreal, Quebec, Canada., R. J. S. Duncan, London, United Kingdom of Great Britain and Northern Ireland; D. Healy, R. Meurant, M. Lanigan Prequalification Team – Diagnostic Assessment, WHO, Geneva, Switzerland with input and expertise from E. Houpt, Jack Gwaltney Professor of Medicine in the Division of Infectious Diseases and International Health, University of Virginia, Charlottesville, Virginia, United States of America (USA); J. Liu, Assistant Professor, University of Virginia, Charlottesville, Virginia, USA who both provided unpublished data on cholera bacterial loads, and technical and programmatic input from K Alberti, D. Legros and D. Olson, High Threat Pathogens, Department of Infectious Hazard Management and P. Olliaro, Intervention and Implementation Research, Special Programme for Research and Training in Tropical Diseases, WHO, Geneva. This document was produced under the coordination and supervision of R. Meurant and I. Prat, Prequalification Team – Diagnostic Assessment, WHO, Geneva, Switzerland. List of contributors A technical consultation on WHO Prequalification requirements for rapid diagnostic tests used for surveillance and detection of an outbreak of cholera was held in Geneva, Switzerland from 16 to 18 October 2017. Meeting participants: N. Ahmed, Senior Director, International Centre for Diarrhoeal Diseases Research Bangladesh (ICDDR,B), Dhaka, Bangladesh; J. Carter, Technical Director, Clinical and Diagnostics Programme, Amref Health Africa, Nairobi, Kenya; R. J. S. Duncan, London, United Kingdom of Great Britain and Northern Ireland; D. Goldfarb, Medical Microbiologist, Department of Pathology & Lab Medicine, BC Children's & Women's Hospitals; Clinical Associate Professor, University of British Columbia, Vancouver, Canada1; M. Guillerm, Diagnostics Consultant, France; C Ilonze, Head, Biologics, Vaccines and Medical Devices Registration and Regulatory Affairs, National Agency for Food & Drug Administration & Control (NAFDAC), Lagos, Nigeria; L. Li, Scientific Reviewer, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration,Silver Spring, Maryland, USA2; J. Liu, Assistant Professor, University of Virginia, Charlottesville, Virginia, USA; A. L. Page, Epidemiologist, Epicentre, Paris, France; E. Piriou, Médecins Sans Frontières, Amsterdam, The Netherlands; M. L. Quilici, Enteric Bacterial Pathogens Unit, French National Reference Centre for Vibrios and Cholera, Institut Pasteur, Paris, France3; U. 1 Participated via web conferencing 2 Participated via web conferencing 3 Participant with observer status Page 2 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Scherf, Director, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA4; W. Sebonego, Technical Specialist, UNICEF Supply Division Health Technology Centre, Copenhagen, Denmark; E. Tran, International quality assurance coordinator for medical devices, Médecins Sans Frontières International, Switzerland; and C. Yansouni, Associate Director, J.D. MacLean Centre for Tropical Diseases, Assistant Professor, Division of Infectious Diseases, Department of Microbiology, McGill University Health Centre, Montreal, Quebec, Canada. WHO Secretariat: D. Healy; M. Lanigan; R. Meurant; W.Urassa, Prequalification Team – Diagnostic Assessment Group, Department of Essential Medicines and Health Products; D. Olson; and K. Alberti, High Threat Pathogens, Department of Infectious Hazard Management, Geneva. The draft technical specifications document was posted on the WHO website for public consultation on 09 November 2017. Various stakeholders, including manufacturers submitting to WHO Prequalification of IVDs, IVD manufacturing industry associations, various national and international regulatory bodies, and IVD standards organizations were informed of the consultation in order to solicit feedback. A one- month response period was provided. Public comments were received for consideration from Asian Harmonization Working Party (AHWP) Working group 2, China, Hong Kong SAR; ARKRAY Healthcare Pvt. Ltd, Dist. Surat, India; College of American Pathologists, Northfield, Illinois, USA; Global Health Investment Fund, New York, USA; P. W. Kamau, Pharmacy and Poisons Board, Nairobi Kenya; In Vitro Diagnostic Devices Evaluation Division, Medical Devices Bureau, Therapeutic Products Directorate, Health Canada, Government of Canada, Canada; L. Li, Scientific Reviewer, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA; J. Liu, Assistant Professor, University of Virginia, Charlottesville, Virginia, USA; A. L. Page, Epidemiologist, Epicentre, Paris, France; E Piriou, Médecins Sans Frontières, Amsterdam, The Netherlands; M. L. Quilici, Enteric Bacterial Pathogens Unit, French National Reference Centre for Vibrios and Cholera, Institut Pasteur, Paris, France; M. B. Rumaney, Goodwood, Western Cape, South Africa; U. Scherf, Director, Division of Microbiology Devices, Office of In Vitro Diagnostics and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA; E. Tran, International quality assurance coordinator for medical devices, Médecins Sans Frontières International, Switzerland. 4 Participated via web conferencing Page 3 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Abbreviations ATCC American Type Culture Collection CFU colony forming units E. coli Escherichia coli EIA enzyme immunoassay IFU instructions for use IVD in vitro diagnostic medical device LPS lipopolysaccharide O1 serogroup 1 O139 serogroup 139 RDT rapid diagnostic test spp. species TSS Technical Specifications Series U.S. FDA U.S. Food and Drug Administration V. cholerae Vibrio cholerae WHO World Health Organization Page 4 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 A. Introduction The purpose of this document is to provide technical guidance to in vitro diagnostic medical device (IVD) manufacturers that intend to seek WHO Prequalification of rapid diagnostic tests (RDTs) for the detection of toxigenic Vibrio cholerae. This document is relevant to RDTs that can be used near to the patient, in settings outside of a laboratory environment, and that detect toxigenic V. cholerae strains that are associated with epidemic spread. It must be able to be used by health care workers and trained community workers. A V. cholerae IVD submitted for WHO Prequalification is expected to detect V. cholerae O1 or O1/O139 in combination, (being the strains consistently considered toxigenic), or are capable of detecting O1/O139 toxin prior to strain confirmation. IVDs will only be prequalified based on the evidence that supports their use for the detection of outbreaks or surveillance for this disease. The requirements outlined in this document do not include those that demonstrate that the IVD can be used for other purposes, such as the diagnosis of an individual, or for patient management. Nor do they include requirements for use of the product for environmental testing. If claimed, these additional intended uses must be clearly stated and supported by relevant evidence. Likewise, field-deployable rapid tests for cholera that are quantitative or are based on other technologies, such as nucleic acid amplification tests, will have different and/or additional requirements for WHO Prequalification and these are not included in this TSS. For the purpose of this document, the verbal forms used follow the usage described below: • “shall” indicates that the manufacturer is required to comply with the technical specifications; • “should” indicates that the manufacturer is recommended to comply with the technical specifications but it is not a requirement; • “may” indicates that the technical specifications are a suggestion to undertake the testing but it is not a requirement. Minimum requirements for WHO Prequalification to demonstrate performance are summarized in this document, and where possible, WHO performance requirements are aligned with published guidance, standards and/or regulatory documents. Although references to source documents are provided, in some cases WHO Prequalification has additional requirements. A documented justification and rationale shall be provided by the manufacturer when the WHO Prequalification submission does not comply with the required technical specifications outlined in this document. For WHO Prequalification purposes, manufacturers shall provide evidence in support of the clinical performance of an IVD that can demonstrate that reasonable steps have been taken to ensure that a properly manufactured IVD, being correctly operated in the hands of the intended user, will reproducibly detect the target analyte and fulfil its intended use. If the manufacturer supplies a reader for use with the IVD, safety and performance data shall be provided in the dossier with and without the use of the reader. WHO Prequalification requirements summarized in this document do not extend to the demonstration of clinical utility, i.e. the effectiveness and/or benefits of an IVD, relative to and/or in combination with other measures, as a tool to inform clinical intervention in each population or health care setting. To demonstrate clinical utility, a separate set of studies is required. Clinical utility Page 5 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 studies usually inform programmatic strategy and are thus the responsibility of programme managers, ministries of health and other related bodies in individual WHO Member States. Such studies do not fall under the scope of WHO Prequalification. B. Other guidance documents This document should be read in conjunction with other WHO guidance documentation, including: • Technical Guidance Series for WHO Prequalification – Diagnostic Assessment • Sample Product Dossiers for WHO Prequalification – Diagnostic Assessment • Instructions for Compilation of a Product Dossier, WHO document PQDx_018. These documents are available at: http://www.who.int/diagnostics_laboratory/evaluations/en/ C. Performance principles for WHO Prequalification C.1 Intended use An IVD intended for WHO Prequalification shall be accompanied by a sufficiently detailed intended use statement. This should allow an understanding of at least the following: • the function of the IVD (e.g. to specifically detect O1 and O139 strains of V. cholerae, etc. as appropriate) and that the result is qualitative; • the testing population for which functions are intended, e.g. patients presenting with symptoms of acute watery diarrhoea in suspected outbreak settings; • the intended operational setting (e.g. primary health care level with no access to standard laboratory facility or settings and no access to electricity, or, community settings outside health facilities where cholera outbreak is suspected); • clinical indication, e.g. for detection and monitoring of epidemic cholera; and • the intended user. C.2 Diversity of specimen types, users and testing environments and impact on required studies Depending on the intended use of the IVD, performance studies shall be designed to consider the diversity of knowledge and skills of potential IVD users, and the likely operational settings in which testing is likely to occur. It is a manufacturer’s responsibility to ensure that the risk assessment for an IVD reflects the intended operational settings, including service delivery complexity, and the likely user population conducting the test. Prequalified IVDs in low- and middle-income countries are likely to be used by laboratory professionals5, healthcare workers6 as well as trained lay providers7. 5 Medical technologists, medical laboratory technicians or similar, who have received a formal professional or paraprofessional certification or tertiary education degree. 6 Any person who performs functions related to healthcare delivery and has received a formal professional or paraprofessional certification or tertiary education degree 7 Any person who performs functions related to healthcare delivery and has been trained to deliver specific services but has received no formal professional or paraprofessional certification or tertiary education degree. Page 6 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 In addition, health care workers (or other users) may be required to perform testing on specimens that are collected either by a different health care worker or by the patient themselves. This shall be considered during the risk assessment. The complexity of a test, from specimen collection to result interpretation, shall be clearly elucidated in the IVD instructions for use (IFU) and reflected in risk analysis. C.3 Applicability of supporting evidence to IVD under review Performance studies shall be undertaken using the version of the IVD intended to be submitted for WHO Prequalification. This version of the IVD shall be made to the finalized, approved documentation, including the quality assurance and quality control specifications and the performance studies shall be performed according to the finalized method documented in the IFU. Where this is not possible a justification shall be provided; additional supporting evidence may also be required. This may occur in the case of minor variations to the design where no negative impact on performance has been demonstrated. Where alternative methods are proposed in the IFU, such as the possible addition of a sample enrichment step, performance studies shall be performed for each method documented in the IFU. If the methods produce difference performances, these shall be reported clearly in the IFU. Specific information is provided in Parts 1 and 2 of this document for the numbers of lots required for particular studies. Each lot should comprise different production (or manufacturing, purification, etc.) runs of critical reagents, representative of routine manufacture. It is a manufacturer’s responsibility to ensure that, via risk analysis of its IVD, minimum numbers of lots chosen for estimating performance characteristics are commensurate with the variability in performance likely to arise from the diversity of key components and their formulation. The manufacturer shall also provide appropriate justifications for the number of lots chosen for estimating performance characteristics. The true V. cholerae status and, if discrimination is claimed, the serogroup or serotype of a specimen shall be determined using a suitable reference standard. For WHO purposes this should be a standard that currently is at a developed stage of technical capability based on the relevant consolidated findings of science, technology and experience (commonly referred to as state of the art). Justification for the choice of standard shall be provided. Estimation (and reporting) of IVD performance shall include the rate of invalid test results and the 95% confidence interval around the estimated values for key performance metrics, as appropriate. Analytical studies should include testing for all specific characteristic (e.g. serogroups, serotypes, or toxin production) for which detection is claimed. For IVDs that include a claim for differentiation (e.g. differentiation of O1 or O139, etc.), evidence of performance shall be provided for each claimed analyte. Where no claim of differentiation is made, it may be acceptable for some analytical studies to generate most of evidence of performance in a single toxigenic strain (e.g. O1 Ogawa) as a surrogate for performance in other related serotypes. A justification for this approach, including a relevant demonstration of equivalence between serotypes/serogroups, shall be provided. Example: for an IVD in which detection of several serogroups or serotypes is claimed, analytical sensitivity shall be determined for each specified serogroup or serotype. The use of characterized repository specimens may be acceptable provided they are relevant to the IVD under assessment taking into consideration: • collection media, collection device (swabs, etc.) • in addition to any requirement for testing in fresh (unpreserved and unfrozen) specimens only. Page 7 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 For some analytical studies, it may be acceptable to use contrived specimens (e.g. bacterial cultures spiked into normal human specimens or into media used in the IVD or, a negative clinical matrix, as appropriate). All bacterial strains used for contrived specimens must be characterized – either by the catalogue number of the commercial source or by providing details of the laboratory from where they are sourced as well as a description of what methods were used for identification of the species and strain. For WHO Prequalification submission, clinical performance studies shall be conducted using each specimen type and collection material (e.g. swabs, collection media) that are claimed in the IFU. Clinical studies shall be based on testing in natural specimens only (a stool specimen originating from a person suspected of having cholera) for V. cholerae O1. However, for V. cholerae O139 where clinical specimens are difficult to isolate, contrived specimens may be acceptable. D. Explanation of terms applied to Vibrio cholerae for the purposes of describing strains detected by an IVD Only specific strains of V. cholerae have the potential for epidemic spread, characterized by specific “O” serogroups and the production of cholera toxin. Individual strains within a serogroup are described according to both serotype and biotype. Serogroup: More than 200 serogroups of the “O” lipopolysaccharide (LPS) are currently recognized within the species V. cholerae. These include the toxigenic O1 and O139 serogroups which are associated with epidemic cholera. Importantly, some strains of V. cholerae O1 and O139 do not produce cholera toxin, and these do not cause cholera disease or epidemics. Some serogroups other than O1 or O139 can cause sporadic disease, but these are not associated with epidemic spread. Serotype: These are subdivisions within serogroups based on specific antigens of the O-group LPS. There main serotypes within the O1 serogroup are Inaba, Ogawa and Hikojima. A given O1 strain can occasionally shift between serotypes. Biotype: Biotypes represent a set of specific phenotypic properties that characterize particular strains of V. cholerae within a serogroup. For example, the O1 serogroup has two biotypes, “classical” and “El Tor”. Biotypes are not specified by serotypes, and are defined by the acquisition of phenotypic characteristics unrelated to O-group LPS antigens. Cholera toxin: The enterotoxin produced by toxigenic V. cholerae strains, responsible for the massive watery diarrhoea seen with cholera infection. Similar in mechanism and sequence to the heat-labile enterotoxin secreted by some strains of Escherichia coli. Page 8 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 E. Table of requirements PART 1 Establishing analytical performance characteristics 1.1 Specimen types 1.1.1 Specimen types 1.2 Specimen collection, storage and transport 1.2.1 Specimen stability 1.3 Precision of measurement 1.3.1 Repeatability, reproducibility 1.4 Performance panels 1.4.1 Performance panels 1.5 Validation of reading times 1.5.1 Validation of reading times 1.6 Analytical sensitivity 1.6.1 Analytical Sensitivity 1.7 Establishment of reader cut-off 1.7.1 Establishment of reader cut-off 1.8 High dose hook effect 1.8.1 High dose hook effect 1.9 Analytical specificity 1.9.1 Potentially interfering substances 1.9.1.1 Endogenous 1.9.1.2 Exogenous 1.9.2 Cross-reactivity, including selectivity with non-virulent O1 strains of V. cholerae 1.10 Stability 1.10.1 Shelf-life (including transport stability). 1.10.2 In-use stability (open pack or open vial stability). 1.11 Flex studies 1.11.1 Flex studies/robustness PART 2 Establishing clinical performance characteristics 2.1 Diagnostic sensitivity and specificity 2.1.1 Diagnostic sensitivity 2.1.2 Diagnostic specificity 2.2 Qualification of usability 2.2.1 Labelling comprehension study (including IFU) 2.2.2 Results interpretation study Page 9 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Part 1 Establishing analytical performance characteristics Aspect Testing requirements Notes on testing requirements Source documents 1.1 Specimen types 1.1.1 Specimen types For each claimed specimen type, testing of at least: • 25 V. cholerae matched negative specimens; • 25 V. cholerae matched positive specimens. Equivalence shall be determined for each claimed V. cholerae serogroup/serotype, as appropriate The relationship between IVD performance in claimed specimen 1. types and materials used for analytical studies shall be clearly established. The design of subsequent studies shall then take that relationship into account. If there is no equivalence between claimed specimen types 2. and/or the materials used for analytical studies, then the impact that this will have on each subsequent performance claim shall be fully understood and described. Example: an IVD intended for testing rectal swabs as well as fresh stools for which the performance is evaluated using a dilution of a cell culture into assay diluents or into stool specimens. • The relationship between the characteristic in diluents to that in a natural specimen (fresh stools and that obtained by using rectal swabs) shall be understood. • The relationship between analytical sensitivity in stool specimens or diluents to that of the same characteristic in rectal swabs shall be understood. If rectal swabs are recommended for use with the IVD, the details 3. (brand, product code etc.) of swab shall be identified and its use validated. The performance for each specimen type shall be established. A prospective study comparing paired rectal swabs and stool specimens may be performed. Positive specimens shall be chosen so that a majority are at a 4. concentration near the limit of detection. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment – TGS-3 (1) 1.2 Specimen collection, storage and transport 1.2.1 Specimen stability Real time studies considering: • storage conditions (duration at different temperatures and variation in humidity, Attention shall be paid to the length of time likely to elapse 1. between specimen collection and its addition to the IVD in the settings where this IVD may be used. Page 10 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents temperature limits, freeze/thaw cycles); • transport conditions, where applicable; • intended use (see note 1); • specimen collection and/or transfer devices, whether these contain transport media, and whether they can be sealed. Unless all specimens are expected to be processed as fresh 2. samples within a specified time frame, the IVD performance shall be established under different storage conditions. The IVD shall generate similar results for the stored specimens at 3. several time points throughout the duration of the recommended storage period and at the upper and lower limit of the recommended storage temperature range. The specimen storage studies shall include specimens with bacterial load majority are at a concentration near the limit of detection. In addition to the analysis of qualitative results, there shall be an 4. analysis of the semi-quantitative results. In all cases, swab types and transport medium shall be specified 5. (brand, product code etc.) and their use validated when claimed as part of the intended use. 1.3 Precision of measurement 1.3.1 Repeatability, reproducibility Both repeatability (within-condition – see note 1) and reproducibility (between-condition – see note 1) shall be estimated by replicate testing at limit of detection (end-point dilutions) of several analyte-positive specimens. Specimens chosen for the testing panel shall include panel members that reflect the main specimen types intended for use with the IVD (e.g. swab or stool specimen). Each panel member shall be tested: • using 3 different lots (see note 4); • over 5 days (not necessarily consecutive) with one run per day (alternating morning/afternoon); • at each of 3 different testing sites. E.g. within- or between-run, -lot, -day, -operator, -site, etc. 1. The testing panel shall be the same for all operators, lots and 2. sites. Low-reactivity specimens shall be in sufficient replication for each 3. run to allow imprecision to be quantitated. Each lot shall comprise different production (or manufacturing, 4. purification, etc.) runs of critical reagents. Results shall be statistically analysed to identify and isolate the 5. sources and extent of any variance. The percentage of correctly identified, incorrectly-identified and 6. invalid results shall be tabulated for each specimen and be separately stratified according to each site, lot, etc. This type of analysis is especially important for RDTs that may not have any numerical values. To understand irregularities in results obtained, at least 2 lots 7. should be tested at each of the 3 testing sites. The effect of operator-to-operator variation on IVD performance 8. CLSI EP12-A2 (2) CLSI EP17-A2 (3) EN 13612:2002 (4) Page 11 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents The effect of operator-to-operator variation on IVD performance is to be included as part of the precision studies (see also note 8 and 9). Testing should be performed: • by personnel representative of intended users; • unassisted; • using only those materials provided with the IVD (e.g. IFU, labels and other instructional materials). may be used in conjunction with other studies to qualify the usability of the IVD. Users shall be selected based on a pre-determined and 9. contextually appropriate level of education, with literacy and auxiliary skills that will challenge the usability of the IVD and reflect the diversity of intended users and operational settings. These characteristics shall be detailed in the submission. 1.4 Performance panels 1.4.1 Performance panels Testing of the IVD shall be undertaken using suitable performance panels (e.g. comprising relevant organism variants, subtypes, etc.) as available. Testing should be performed using more than one lot of the final 1. design (locked-down). It is acceptable to use contrived specimens that comprise stool 2. specimens spiked with microorganisms from culture collections. 1.5 Validation of reading times 1.5.1 Validation of reading times For IVDs where a reading interval is specified (i.e. time when result can first be read; time beyond which result should not be read), validation of critical time points shall be provided. Performance studies shall be conducted at the extremes of the intended operational temperature range; the effect of humidity on reading times shall also be investigated. The ranges of temperature and humidity tested for should be 1. risk-based, taking into consideration knowledge of conditions likely to be experienced in intended operational settings. The intended operating temperature range, upon which reading 2. time has been validated, shall be clearly stated in the IFU. This work shall be performed with IVDs near the beginning and 3. near the end of their assigned shelf-lives. Some of these aspects may be evaluated within the flex studies 4. (1.11.1). Technical Guidance Series for WHO Prequalification – Diagnostic Assessment TGS-3 (1) 1.6 Analytical sensitivity 1.6.1 Analytical Sensitivity Analytical sensitivity shall be determined for each claimed target strain as appropriate (see Introduction section C3) as follows: • using 3 lots (see note 1) Each lot shall comprise different production (or manufacturing, 1. purification, etc.) runs of critical reagents. Ideally the lots should be chosen from across the assigned shelf-life of the IVD. This study may alternatively be performed as part of the shelf-life 2. CLSI EP17-A2 (3) WHO Prequalification – Diagnostic Assessment Page 12 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents • a minimum of 5 replicate tests of each member of an 8-member dilution series of a suitable biological material, the concentrations of which are closely spaced and which cross the limit of detection. Analytical sensitivity shall be estimated by determining the lowest concentration for which the rate of detection is 95%. verification of the IVD. Analytical sensitivity shall be demonstrated in a clinical sample 3. matrix and shall use the entire assay system from sample preparation to interpretation. Justification shall be provided for the choice of reference cultures 4. and dilution media used in the dilution series. The estimate of analytical sensitivity should be confirmed by 5. separately testing an additional 20 replicates. Where a claim is made for detection of specific V. cholerae 6. serogroups/serotypes, performance characteristics shall be determined in each strain type claimed. If analytical sensitivity for one or more given strain types of V. cholerae is not appropriate for intended users, then this limitation of the IVD shall be clearly reported in the IFU. PQDx_018 (5) 1.7 Establishment of reader cut-off 1.7.1 Establishment of reader cut- off In V. cholerae RDTs provided with a reader, the way in which the reader has been designed to differentiate positive specimens from negative specimens shall be demonstrated. 1.8 High dose hook effect 1.8.1 High dose hook effect The potential for a high dose hook effect shall be determined using each of the claimed serotype or serogroup with at least three finalized lots of the IVD. This study may be undertaken using cultures expressing high 1. concentration of the target analytes claimed to be detected, or with natural specimens found to have a very high titre in enzyme immunoassays (EIA) or by molecular methods. Strains shall be representative of the diversity among each 2. serotype or serogroup claimed. This would require showing the specimens yield positive results 3. over the range of 109 – 1012 CFU/mL for the claimed serogroup/serotype. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment TGS-6 (7) Page 13 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents 1.9 Analytical specificity 1.9.1 Potentially interfering substances The potential for false results (false-negatives and false-positives) arising from interference from, at least, the substances/conditions listed below shall be determined (see note 1). • a minimum of 100 specimens; • each substance represented by at least 3-5 specimens from different individuals. Testing shall be undertaken in both V. cholerae negative and -positive specimens spiked with each potentially interfering substance at physiologically relevant dosages. The risk assessment conducted for an IVD should identify 1. substances where the potential for interference can reasonably be expected for the analyte being detected (e.g. laxatives). Any observed interference shall be investigated and performance 2. limitations of the IVD reported in the IFU. Results shall be reported with respect to each condition and not be reported as an aggregate of the total number of specimens tested in the study. Where a claim is made for identification of different V. cholerae 3. serogroups/serotypes, performance characteristics shall be determined for each. At a minimum this shall include detection in specimens positive for Ogawa, Inaba and O139. (Note that evaluation with uncharacterized specimens is insufficient). Where testing in these serogroups/serotypes has not been undertaken, this limitation of IVD performance shall be reported in the IFU. Evaluation of endogenous interfering substances may be 4. addressed as part of the clinical studies. In this case, a description of the faecal form shall be provided. European Commission decision on CTS (6) CLSI EP07-A2 (8) 1.9.1.1. Endogenous Clinical specimens selected for the presentation of: • blood; • excessive lipids; • excessive mucus (see note 4). 1.9.1.2. Exogenous Relevant medicines, including: • antimalarial, antiretroviral and anti- tuberculosis medications; • laxatives; • common anti-diarrhoeal medicines; • common over-the-counter anti- inflammatory/analgesic medications (paracetamol, ibuprofen); • ethanol; • caffeine; • azithromycin; • ciprofloxacin; • co-trimoxazole; • doxycycline; • ivermectin; • albendazole; Page 14 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents • conjugates of loperamide. 1.9.2. Cross- reactivity including selectivity with non-virulent O1 strains of V. cholerae. Strains used may be either well characterized clinical isolates from symptomatic individuals, or a pathogenic type-strain from a recognized culture collection bank such as the American Type Culture Collection (ATCC). • each potentially cross-reacting pathogens listed should be represented by at least 3-5 specimens from different individuals. The following list represents the main attributable diarrhoeal pathogens in low-resource settings where cholera epidemics may occur that require testing for cross-reactivity: Bacteria: • Non-O1/O139 V. cholerae (for IVDs that detect the “O” LPS); • Non-toxigenic V. cholerae including non- virulent O1 strains of V. cholerae (for IVDs that detect cholera toxin itself) • STEC (Shiga toxin-producing E. coli); • ETEC (enterotoxigenic E. coli). Viruses: • Rotavirus; • Adenovirus 40/41 Identification of microorganisms as potentially cross-reacting 1. should be risk-based and may include: Bacteria: • Shigella spp; • Campylobacter spp; • Non-pathogenic E coli; • EAEC (enteroaggregative E. coli); • EPEC (enteropathogenic E. coli); • Aeromonas spp; • Salmonella spp. (see note 2); • Plesiomonas spp; • Vibrio parahaemolyticus; • Vibrio fluvialis • Clostridium difficile The specific species of a genus, or serovar of Salmonella enterica, 2. or pathotype of E coli etc. that are tested shall be described. Page 15 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents 1.10 Stability Replicate testing shall be undertaken using a panel consisting, for each claimed analyte, of at least: • 1 negative specimen; • 2 low positive specimens (with a concentration of analyte just above the limit of detection such that results of repeated tests of this sample are positive approximately 95% of the time); • 1 moderately positive specimen (with a concentration at which one can anticipate positive results approximately 100% of the time e.g., approximately five to ten times the concentration of the limit of detection). 1. Stability studies shall be conducted using the conditions expected in the environment of intended use. 2. Each lot should comprise different production (or manufacturing, purification, etc.) runs of critical reagents. 3. The numbers of invalid tests with each kit lot shall be reported. 4. Transport simulation shall be performed using simulated extreme stress conditions, ensuring that application of those conditions is consistent and controlled. 5. Claims for stability shall be based on the second-last successful data point from the least stable lot, with, if lots are different, a statistical analysis showing that the bulk of lots will be expected to meet the claimed life. For example: for testing conducted at 3, 6, 9, 12 and 15 months, if stability was observed at 15 months, then the maximum stability claim can be 12 months. 6. Accelerated studies do not replace the need for real time studies. 7. In-use stability of labile components shall be conducted using components in their final configuration. 8. In-use stability should be conducted with lots at the beginning and end of their shelf-lives. ISO 23640:2011 (9) CLSI EP25-A (10) Technical Guidance Series for WHO Prequalification – Diagnostic Assessment TGS-2 (11) ASTM D4169-14 (12) 1.10.1 Shelf-life (including transport stability) • real time studies using a minimum of 3 lots of final design product in final packaging. • lots are “transport stressed” (simulated) before real time studies are undertaken on these lots. 1.10.2 In-use stability • minimum of one lot, using panel(s) compiled as above; • testing of all labile components (e.g. buffers vials etc. see note 7). 1.11 Flex studies 1.11.1 Flex studies/ robustness The influence of the following factors on expected results (both reactive and non-reactive) shall be considered: • specimen and/or reagent volume; • reading time; • buffer pH (measure of robustness – e.g. due to evaporation of the buffer); • IVD sturdiness including robustness of 1. Refer to WHO document PQDx_018 “Instructions for compilation of a product dossier” for other flex studies that may be relevant, taking into consideration the broad range of operational and environmental conditions consistent with intended use. 2. The factors listed opposite should be investigated in ways that not only reflect, but also exceed, likely operating conditions in lower- and middle-income countries so that the limitations of the device can be understood. For example, in addition to investigating WHO Prequalification – Diagnostic Assessment PQDx_018 (5) Page 16 Part 1 Establishing analytical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents packaging and labelling. IVD in final packaging shall be subjected to drop-shock testing; • lighting and humidity (See note 3); • operating temperature. Instrumentation (if applicable) including: • ruggedness (see note 4); • impact of dust and mould on componentry (e.g. optics). deviations of temperature within those claimed in the IFU (in the middle and at both lower and upper extremes of a claimed temperature range), temperature ranges should be investigated that exceed those of claimed operating conditions and which cause test failure (incorrect/invalid results). 3. The impact of lighting can be twofold – i.e. the impact of lighting on packaging (e.g. fading), and the sufficiency of lighting to read the test lines. 4. For the purposes of this document, ruggedness means the ability to resist environmental shocks. Page 17 Part 2 Establishing clinical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Part 2 Establishing clinical performance characteristics Aspect Testing requirements Notes on testing requirements Source documents 2.1 Diagnostic sensitivity and specificity Diagnostic sensitivity and specificity shall be determined for each of the specimen types claimed: • by a variety of intended users (e.g. 9 - 12 users); • using more than one lot. 1. For WHO Prequalification purposes, testing should be undertaken in fresh specimens. If more than one specimen collection method is intended for use with the IVD, each of those methods must be evaluated during the clinical studies. 2. In line with the intended use of this product, clinical performance characteristics should be established on symptomatic patients (i.e. with watery diarrhoea). 3. Prequalified RDTs to detect V. cholerae will generally be used by trained lay providers and trained health care workers. For WHO Prequalification purposes, these should also be considered as the intended user in addition to a laboratory professional. 4. A separate aliquot of the specimen or of the extract from the swab shall be collected in parallel to establish the reference result. The testing algorithm used to determine the reference results should include culture and nucleic acid testing since there might be non-viable microorganisms in the material. Justification for the use of the testing algorithm shall be provided. 5. Contrived specimens may be used for determination of diagnostic sensitivity in IVDs to detect V. cholerae O139. 6. Each lot (design locked-down) should comprise different production (or manufacturing, purification, etc.) runs of critical reagents. 7. All discrepant results (between assay under evaluation and the reference results) shall be repeated on the same lot, and then on all available lots and the variability noted. Performance characteristics shall be reported using initial results only. The results of further testing of specimens with U.S. FDA (13) European Parliament IVD regulations (14) 2.1.1 Diagnostic Sensitivity For determination of diagnostic sensitivity, testing shall be conducted: • 150 positive specimens, as confirmed by the reference methods (see note 4) for each of the serogroup or serotype claimed (see note 5). 2.1.2 Diagnostic Specificity For determination of diagnostic specificity, testing should be conducted at different geographical and epidemiological settings representative of intended users (minimum of 2 regions). Testing shall be conducted in: • at least 200 V. cholerae negative specimens from a symptomatic population (i.e. those with watery diarrhoea or non-formed stools). Page 18 Part 2 Establishing clinical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents discrepant results shall be reported separately as additional information about IVD performance. 8. All invalid results shall be recorded and evaluated in comparison to the reference result. Invalid results shall be analysed separately in the final performance calculations. 9. Estimates of diagnostic/clinical sensitivity and specificity shall be reported with 95% confidence intervals. 10. Results shall be reported with respect to each study site and not be reported as an aggregate of the total number of specimens tested to establish these characteristics. 11. Where possible, sensitivity should be established different geographical and epidemiological settings. 2.2 Qualification of usability 2.2.1 Label comprehension study Testing of subjects shall be undertaken to assess ability of intended users to correctly comprehend key messages from packaging and labelling: • understanding key warnings, limitations and/or restrictions; • proper test procedure; • test result interpretation. Studies shall include at least 15 intended users including those whose native language may not be the language of the IFU if necessary, in their usual working environment, not employees of the manufacturer, from two geographically diverse populations to to demonstrate comprehension of key messages in each user group. Intended users include those who are collecting the stool 1. specimen as well as those who are performing the testing of the stool specimen. Trained lay providers and health care workers should both be considered as intended users. If the labelling is available in different languages, the labelling 2. comprehension study should be performed for each language. Requirements listed may be investigated as separate studies 3. or included as part of clinical studies. Testing may be conducted using questionnaire based surveys. 4. European Parliament IVD regulations (14) Backinger CL and Kingsley PA (15) U.S. FDA (16) 2.2.2 Results interpretation study Intended users shall interpret the results of contrived IVDs (e.g. static/pre-made tests) to assess their ability to correctly interpret pre-determined test results. Contrived tests shall be made to demonstrate the following potential test results: Page 19 Part 2 Establishing clinical performance characteristics RDTs used for surveillance and detection of an outbreak of cholera TSS-5 Aspect Testing requirements Notes on testing requirements Source documents • non-reactive; • range of invalid results; • reactive; • weak reactive. Testing subjects shall consist of: • at least 15 intended users, including those whose native language may not be the IFU language if necessary, • in their usual working environment, not employees of the manufacturer, • from two geographically diverse populations to demonstrate correct interpretation of simulated test results. Page 20 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 F. References 1. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment. Principles for Performance studies, TGS– 3. Geneva: World Health Organization; 2016 http://apps.who.int/iris/bitstream/10665/258985/1/WHO-EMP-RHT- PQT-TGS3-2017.03-eng.pdf?ua=1 accessed 09 October 2017. 2. Evaluation of Precision of Quantitative Measurement Procedures, 3rd Edition - CLSI EP05-A3. Wayne, PA: Clinical and Laboratory Standards Institute; 2014. 3. Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline- Second Edition. CLSI EP17-A2; Wayne, PA: Clinical and Laboratory Standards Institute; 2012. 4. EN 13612:2002. Performance evaluation of in vitro diagnostic medical devices. Brussels: European Committee for Standardization; 2002. 5. WHO Prequalification – Diagnostic Assessment: Instructions for Compilation of a Product Dossier. WHO PQDx_018 v3, 27 August 2014. Geneva: World Health Organization; 2014 (http://www.who.int/entity/diagnostics_laboratory/evaluations/141015_pqdx_018_dossier_instructions_v4.pdf?ua= 1 accessed 09 September 2017. 6. Technical Guidance Series for WHO Prequalification – Panels for quality assurance and quality control of in vitro diagnostic medical devices TGS–6. Geneva: World Health Organization; 2017 http://apps.who.int/iris/bitstream/10665/259408/1/WHO-EMP-RHT-PQT-2017.10-eng.pdf?ua=1 accessed 15 December 2017. 7. Commission Decision of 3 February 2009 amending Decision 2002/364/EC on common technical specifications for in vitro-diagnostic medical devices (2009/108/EC). O. J. E. U. 2009, L39/34-L39/49; (http://eur-lex.europa.eu/legal- content/EN/ALL/?uri=uriserv:OJ.L_.2009.039.01.0034.01.ENG accessed 09 October 2017). 8. Interference Testing in Clinical Chemistry. Approved Guideline - Second Edition. CLSI EP07-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2005. 9. ISO 23640:2011. In vitro diagnostic medical devices - Evaluation of stability of in vitro diagnostic reagents. Geneva: International Organization for Standardization; 2011. 10. Evaluation of Stability of In Vitro Diagnostic Reagents; Approved Guidelines. CLSI EP25-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2009. 11. Technical Guidance Series for WHO Prequalification – Diagnostic Assessment. Establishing stability of in vitro diagnostic medical devices, TGS–2. Geneva: World Health Organization; 2017 http://apps.who.int/iris/bitstream/10665/259742/1/WHO-BS-2017.2304-eng.pdf?ua=1/ accessed 08 October 2017. 12. ASTM International. ASTM D4169-14. Standard Practice for Performance Testing of Shipping Containers and Systems. West Conshohocken, PA: ASTM International; 2014. 13. Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests. Silver Spring, Maryland: Food and Drug Administration; 2007 http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm071287.pd f, accessed 09 October 2017). 14. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in-vitro diagnostic medical devices. O. J. E. U. 2017 L 117/176 http://eur-lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX:32017R0746&from=DE accessed 08 September 2017. 15. Backinger CL and Kingsley PA. Write It Right: Recommendations for Developing User Instructions for Medical Devices Used in Home Health Care. HHS Publication FDA 93-4258; 1993. Silver Spring, Maryland: Food and Drug Administration; 1993 (http://www.fda.gov/downloads/MedicalDevices/.../ucm070771.pdf, Accessed 08 September 2017. 16. Applying Human Factors and Usability Engineering to Medical Devices. Silver Spring, Maryland: Food and Drug Administration; 2016. https://www.fda.gov/downloads/MedicalDevices/.../UCM259760.pdf accessed 09 October 2017. Page 21 RDTs used for surveillance and detection of an outbreak of cholera TSS-5 List of related WHO Publications of related interest WHO Prequalification Team- Diagnostic Assessment. Technical Guidance Series for WHO Prequalification – Diagnostic assessment (available online) WHO Prequalification Team – Diagnostic Assessment. Instructions for Compilation of a Product Dossier. WHO document WHO/PQDx_18 (available online) The Technical Specifications Series for submission to WHO Prequalification – Diagnostic Assessment set out appropriate performance evaluation criteria to meet prequalification requirements. Each Technical Specification provides information on the minimum performance requirements for WHO Prequalification that should be met by a manufacturer to ensure that their in vitro diagnostic medical device is safe and performs optimally. Further information on these and other WHO publications can be obtained from WHO Press, World Health Organization, 1211Geneva 27, Switzerland http:/www.who.int/bookorders tel.: +41 22 791 3264; fax: +41 22791 4857; email bookorders@who.int ISBN-978-92-4-1513715