Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance: policy brief, November 2023

Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance Policy brief Novem ber 2023 © WHO / Ploy Phutpheng

Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance ISBN 978-92-4-008613-5 (electronic version) ISBN 978-92-4-008614-2 (print version) © World Health Organization 2023 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial- ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules/). Suggested citation. Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance. Geneva: World Health Organization; 2023. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at https://iris.who.int/. Sales, rights and licensing. To purchase WHO publications, see https://www.who.int/publications/book-orders. To submit requests for commercial use and queries on rights and licensing, see https://www.who.int/copyright. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third- party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. Policy brief, November 2023 iii Contents Acknowledgements ............................................................................................................... vi Executive summary ............................................................................................................... 1 1. Background .................................................................................................................... 2 2. Approach to development ................................................................................................ 3 3. Minimum criteria for external quality assurance proficiency tests ......................................... 4 3.1 Panel composition ............................................................................................................................ 4 3.1.1 Type of specimen ................................................................................................................. 4 3.1.2 Drug resistance mutations ................................................................................................... 4 3.1.3 HIV-1 subtypes ..................................................................................................................... 4 3.1.4 Plasma vial load ................................................................................................................... 5 3.1.5 Number of samples per proficiency panel ........................................................................... 5 3.1.6 Virus-assay compatibility ..................................................................................................... 5 3.1.7 Plasma volume ..................................................................................................................... 5 3.2 Logistical aspects ............................................................................................................................. 5 3.2.1 Number of laboratories required to generate consensus sequence .................................... 5 3.2.2 Frequency of testing of proficiency panels by participating laboratories ............................ 5 3.2.3 Turnaround time .................................................................................................................. 5 3.2.4 Time to sending a second proficiency panel for laboratories failing a first panel ............... 6 3.3 Experience, financial sustainability and demonstrated ability to effectively manage a proficiency testing scheme ................................................................................................................................. 6 3.3.1 Experience in proficiency test development ........................................................................ 6 3.3.2 Accreditation of a proficiency panel provider ...................................................................... 6 3.3.3 Sustainable funding requirement ........................................................................................ 6 3.3.4 Cost of the proficiency test .................................................................................................. 6 3.4 Evaluation, scoring and reporting of results .................................................................................... 6 3.4.1 Minimum proficiency test evaluation process ..................................................................... 6 3.4.2 Contents of proficiency test evaluation report .................................................................... 7 3.4.3 Situations where proficiency test providers do not meet WHO criteria for analysis and scoring 7 3.4.4 WHO proficiency testing evaluation procedure ................................................................... 7 Annex 1: WHO HIV drug resistance network laboratories providing review .................................... 9 Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance iv Acknowledgements This policy brief is a product of the WHO Department of Global HIV, Hepatitis and STI Programmes, and it is the result of the technical contribution of WHO’s HIV Drug Resistance Laboratory Network Working Group led by Seth Inzaule (Consultant, WHO Department of Global HIV, STI, Hepatitis Programmes, Switzerland) and Michael Jordan (Consultant, WHO Department of Global HIV, STI, Hepatitis Programmes, Switzerland) with guidance from Neil Parkin (Data First Consulting, USA). The World Health Organization (WHO) acknowledges WHO’s HIV Drug Resistance Laboratory Network Working Group and other experts who participated in the consultative meetings including Daniel Kuritzkes (Division of Infectious Diseases, Brigham and Women’s Hospital, USA), Irene Mukui (Drugs for Neglected Diseases Initiative (DNDi), Kenya), Kim Steegen (National Health Laboratory Services, South Africa), Omar Sued (HIV, Tuberculosis, STIs and Viral Hepatitis Unit, Pan American Health Organization and the WHO region for the Americas), Alia Margaret Paul (Communicable Disease Unit, WHO Regional Office for Africa), Fausta Shakiwa Mosha (Communicable Disease Unit, WHO Regional Office for the Eastern Mediterranean), Amalia Giron (Consultant, WHO Department of Global HIV, STI, Hepatitis Programmes, Switzerland), Barbara Bighignoli (National HIV Reference Laboratory, Malawi), Bhavna Hora (Duke University, USA), Chanson Brumme (British Columbia Centre for Excellence in HIV/AIDS, Canada), Cheri Van Der Walt (National Institute for Communicable Diseases, Centre for HIV and STIs, South Africa), Chika Owuama (Nigerian Institute of Medical Research (NIMR), Nigeria), Clement Zeh (US Centers for Disease Control, USA), Deogratius Ssemwanga (Uganda Virus Research Institute, Uganda), Diane Descamps (Agence Nationale de Recherche sur le SIDA et les Hépatites Virales (ANRS), France), Duping Zheng (US Centers for Disease Control, USA), Ewalde Cutler (National Institute for Communicable Diseases, Centre for HIV and STIs, South Africa), Joseph Fokam (Chantal BIYA International Reference Centre for Research on HIV/AIDS Prevention and Management (CIRCB), Cameroon), Giovanny Vilcarino Zevallos (Instituto Nacional de Salud, Perú), Hezhao Ji (Public Health Agency of Canada, Canada), Hloniphile Zwane (National Institute for Communicable Diseases, Centre for HIV and STIs, South Africa), José Carlos Fernandez (Fundación Oswaldo Cruz-FIOCRUZ, Brazil), Keith Crawford (Therapeutic Research Program, Division of AIDS, NIAID/NIH/DHHS, USA), Nguyen Thi Lan Anh (National Institute of Hygiene and Epidemiology, Viet Nam), Lingjie Liao (Chinese Center for Disease Control and Prevention, China), Lisa Guerrero (US Centers for Disease Control, USA), Liuber Yans Machado-Zaldivar (Laboratorio de Investigaciones del SIDA (LISIDA), Cuba), Monalisa Nomhle Kalimashe (National Institute for Communicable Diseases, Centre for HIV and STIs, South Africa), Navin Horthongkham (Faculty of Medicine Siriraj Hospital, Mahidol University, Thailand) Paul Sandstrom (Public Health Agency of Canada, Canada), Rami Kanter (Brown University, USA), Raphael Sanchez (AIDS Research Program-Immunology Reference Laboratory, Puerto Rico), Rob Schuurman (Department Of Virology, University Medical Center Utrecht, Netherlands (Kingdom of the)), Roger Paredes (Hospital Universitari Germans Trias i Pujol, Spain), Rupert Capina (Public Health Agency of Canada, Canada), Santiago Avila (Centro de Investigación en Enfermedades Infecciosas, Instituto Nacional de Enfermedades Respiratorias (CIENI/INER), Sikhulile Moyo (Botswana-Harvard HIV Reference Laboratory (BHHRL), Botswana), Sumonmal Uttayamakul (Bamrasnaradura Infectious Diseases Institute, Department of Disease Control, Ministry of Public Health, Thailand), Susan Espetia (Instituto Nacional de Salud, Perú), Tamyo Mbisa (Public Health England, United Kingdom), Tran Thi Minh Tam (National Institute of Hygiene and Epidemiology, Viet Nam), Don Changsom (National Institute of Health, Department of Medical Sciences, Thailand), Valarie Opollo (Kenya Medical Research Institute and US Center for Disease Control and Prevention, Kenya), Vanessa Rivera-Amill (AIDS Research Program-Immunology Reference Laboratory, Puerto Rico), Vibha Kana (National Institute for Communicable Diseases, Centre for HIV and STIs, South Africa). We acknowledge the review and participation of the WHO HIV drug resistance network laboratories listed in Annex 1. We are also grateful to the external reviewers; Lara Vojnov and Robert Luo (WHO Department of Global HIV, STI, Hepatitis Programmes, Switzerland), Gillian Hunt (Independent consultant, South Africa) and Martine Peeters (Agence Nationale de Recherche sur le SIDA et les Hépatites Virales (ANRS), France). Policy brief, November 2023 1 Executive summary This policy brief provides guidance on the minimum criteria external quality assurance proficiency testing schemes are required to meet to be used to assess the proficiency of laboratories seeking designation or redesignation as WHO-designated laboratories for the purpose of HIV drug resistance surveillance testing. Routine monitoring and surveillance of HIV drug resistance is one of the key pillars of quality and effective antiretroviral therapy (ART) programming and helps ensure the continuous efficacy of ART and supports countries in achieving the 2030 global goal for HIV epidemic control. Population-level HIV drug resistance surveillance is especially critical in low- and middle-income countries, where routine drug resistance testing for individual patient monitoring is limited or non-existent. To support global HIV drug resistance surveillance, WHO has provided guidance and tools since 2003 to support countries in implementing surveys of HIV drug resistance. WHO has also established a network of HIV drug resistance genotyping laboratories (WHO HIVResNet laboratories) that support the generation of reliable quality-assured HIV drug resistance survey data. Participation in external quality assurance programmes, including annual proficiency testing, is required by WHO for laboratories to obtain and maintain HIVResNet designation status. External quality assurance provides a means to assess laboratories’ technical competence and is intended to ensure the reliability and quality of genotyping results, thus providing confidence to public health officials using survey data for decision-making. Since 2007, the WHO external quality assurance of HIV drug resistance proficiency testing has been supported by the United States National Institutes of Allergy and Infectious Diseases and National Institutes of Health through the Viral Quality Assurance programme. One limitation with the existing programme is a restricted number of laboratories that can be supported due to funding constraints. To facilitate the inclusion of additional laboratories into the WHO HIVResNet Laboratory Network, WHO in collaboration with the WHO HIVResNet Laboratory Working Group developed minimum criteria for external quality assurance proficiency testing schemes that can be used to support laboratories seeking designation or redesignation by WHO for the purposes of HIV drug resistance surveillance. Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance 2 Background To obtain and maintain membership in the WHO HIV Drug Resistance Network (WHO HIVResNet) Laboratory Network, a laboratory must participate in a WHO-recognized HIV drug resistance genotyping external quality assurance programme that includes proficiency panel testing. WHO and its partners or selected specialized WHO HIVResNet laboratories are responsible for developing and distributing proficiency test panels to member laboratories and to those applying for membership. Laboratories seeking to obtain or maintain designation need to successfully pass a minimum of one proficiency panel per year. Since 2007, proficiency panels have been provided by the Virology Quality Assurance Program (VQA) under a contract supported by the United States National Institutes of Allergy and Infectious Diseases and National Institutes of Health. Recognizing the limited number of laboratories that can be supported under the existing VQA scheme and seeking to provide flexibility to laboratories regarding selection of an appropriate alternative to the VQA proficiency test, WHO sought to develop minimum standards to be met by HIV drug resistance proficiency testing panels. The recommended minimum standards for a HIV-1 drug resistance genotyping proficiency testing scheme are described here. © WHO / Sergey Volkov Chapter 1 Policy brief, November 2023 3 Approach to development Between September 2022 and March 2023, WHO conducted a landscape assessment of available HIV drug resistance proficiency testing schemes to understand the characteristics of these schemes. A scoping review was first done to identify external quality assurance programmes providing proficiency testing schemes for HIV drug resistance. This list was subsequently shared with the WHO HIVResNet Laboratory Network for review and for further identification of additional schemes. A detailed questionnaire was sent to the manufacturers of the identified proficiency test schemes. The questionnaire included five domains: i) programme characteristics including years of experience and programme sustainability; ii) characteristics of the specimens in the panel including the nature of specimens including whether they are clinically derived or are artificial constructs; iii) proficiency panel characteristics including the number of specimens in the panel, subtypes, viral load range and gene targets; iv) analysis and scoring criteria for the panel; and v) interpretation of the results and quality improvement support. The landscape review was shared with the WHO HIVResNet Laboratory Network for review and was used as a background document to guide the development of minimum criteria for HIV drug resistance proficiency tests. In April 2023, WHO held a series of virtual consultations with its HIVResNet laboratory working Group. The HIVResNet laboratory working group is a sub-group of WHO HIVResNet and is composed of international experts, researchers, laboratorians, organizations, partners, stakeholders, and civil society members with an advisory and implementation role to prevent, monitor and respond to HIV drug resistance. The goal of the virtual consultations was to define the minimum criteria that a proficiency testing programme should meet to be considered eligible to provide proficiency tests to WHO HIVResNet laboratories. During the consultations, while continuing to rely on VQA to the extent possible, WHO affirmed the use of additional external quality assurance schemes and HIV-1 drug resistance proficiency tests providing they meet the minimum standards defined here. Chapter 2 © WHO / Christopher Black Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance 4 Minimum criteria for external quality assurance proficiency tests Chapter 3 3.1 Panel composition 3.1.1 Type of specimen The proficiency panel should preferably comprise specimens containing HIV-1 derived from clinical samples. If appropriate clinical specimens are unavailable, HIV-1 derived from cell culture is acceptable and may be used. Clinically derived plasma specimens are preferred because they mimic the specimen type used by countries implementing surveys of HIV drug resistance. Recognizing the limitation in obtaining large volumes of clinical specimens to support proficiency testing in many laboratories, viruses cultured in vitro are an acceptable alternative. Use of artificial constructs such as infectious molecular clones or site-directed mutants is not acceptable because they are more homogeneous than virus derived from clinical specimens, thus reducing the ability of the proficiency test to discern a laboratory’s ability to accurately recognize and report mixed bases. 3.1.2 Drug resistance mutations Whenever possible, the proficiency panel should include one or more samples with several drug resistance mutations in each of the three target regions of the HIV-1 genome: reverse transcriptase, protease and integrase. Drug resistance mutations do not need to be present in each of the three target regions in a given sample. Including specimens with drug resistance mutations in each of the three target regions in each proficiency panel is highly desirable; however, it is not mandatory, provided that sequence homology analyses performed when writing the evaluation report include all positions in which relevant drug resistance mutations are known to occur.1 Including samples with drug resistance mutations is preferred; however, obtaining clinical specimens with drug resistance mutations in the integrase region of HIV-1 may be difficult because they are relatively rare. However, since the required minimum evaluation assesses the entire nucleotide sequence and because a laboratory’s proficiency in detecting drug resistance mutations and mixed bases may be assessed using dry panels, it is not strictly necessary for drug resistance mutations to be present in all the samples used in a panel. 3.1.3 HIV-1 subtypes A proficiency panel should include at least three different HIV-1 subtypes. WHO-designated laboratories should anticipate HIV-1 subtype variability when testing specimens from countries implementing surveys of HIV drug resistance. Thus, proficiency panels should include a variety of HIV-1 subtypes. Although not possible all HIV-1 subtypes can be included, at least three subtypes should be included in each panel. Some 1. The sequence must include all positions in which drug resistance mutations are known to occur in accordance with the most-updated Stanford HIVdb algorithm (https://hivdb.stanford.edu/hivdb/by-patterns) excluding positions 318 and 348 of reverse transcriptase, which are positions of mutations conferring resistance to drugs that are not currently considered by WHO to be priority treatment or prevention options for use in low- and middle-income countries. Policy brief, November 2023 5 national and regional laboratories may only routinely encounter one or two HIV-1 subtypes circulating in their respective geographical region. In this case, WHO may waive the requirement for a minimum of three different subtypes per panel when designating a laboratory if the subtypes present in the proficiency panel are representative of those that the laboratory is likely to encounter when performing surveys of HIV drug resistance. 3.1.4 Plasma viral load Samples included in a proficiency panel should have a minimum plasma viral load of 2000 copies/mL, with at least one sample having a viral load between 2000 and 10 000 copies/ml. Although some survey specimens are expected to have viral loads as low as 1000 copies/mL, based on the WHO definition of viral non-suppression, most assays cannot achieve 100% amplification sensitivity at this level. Thus, a minimum of 2000 copies/mL is preferred. At least one sample in the panel should have a viral load between 2000 and 10 000 copies/mL to monitor the performance of laboratories in successfully analysing and interpreting sequences derived from samples with lower viral loads. 3.1.5 Number of samples per proficiency panel Any given proficiency panel should include a minimum of four different samples and preferably contain five samples. Having several samples in a panel is a useful way to evaluate the potential for laboratory errors due to cross-contamination or sample mix-ups. 3.1.6 Virus-assay compatibility The viruses included in a proficiency panel should be compatible with most of the assays used by laboratories participating in the proficiency testing scheme. The burden placed on providers of proficiency panels to ensure compatibility with the wide array of in-house assays used by WHO HIVResNet laboratories is recognized. However, proficiency panel providers should confirm that all samples included in a panel can be successfully tested using their own assay and preferably also conduct reference testing in two to three other laboratories using different primers for amplification and sequencing as a best practice. 3.1.7 Plasma volume A minimum sample volume of 1 mL of plasma shipped on dry ice is preferred; however, the use of freeze- dried lyophilized plasma is acceptable. Frozen plasma is preferred since it best mimics the type of specimen that laboratories are likely to encounter when performing HIV drug resistance testing for countries implementing WHO- recommended surveys. Recognizing that the cost of shipping plasma on dry ice could be a limitation, lyophilized specimens are an acceptable alternative. However, proficiency test producers should recognize that using lyophilized specimens is likely to introduce variability and changes in routine procedures during sample reconstitution in laboratories testing the panel, which may potentially affect the panel results. Lyophilized specimens should therefore be well characterized under different conditions likely to be encountered by the testing laboratories and clear instructions for reconstitution should also be shared with the testing laboratories. 3.2 Logistical aspects 3.2.1 Number of laboratories required to generate consensus sequence To yield a meaningful consensus sequence, a minimum of 10 laboratories should test each panel in any given round of testing (such as annual). The generation of a meaningful consensus sequence may be limited if the number of participating laboratories is fewer than 10. A higher number of participating laboratories is preferred. 3.2.2 Frequency of testing of proficiency panels by participating laboratories Panels should be sent to laboratories participating in the testing scheme at least once every year. Although participating in multiple panels every year is preferable for laboratories, this may pose challenges regarding cost and logistics. 3.2.3 Turnaround time The turnaround time from receipt of proficiency testing results from participating laboratories to the return of an evaluation report should not exceed six weeks. A long turnaround time may limit the maximum utility of proficiency test results for improving quality in situations of unsatisfactory performance. However, WHO acknowledges that Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance 6 the timely relay of results may be challenging due to delays in receiving results from all participating laboratories and the requirement to include all laboratory results in the consensus sequence. Nevertheless, attempts should be made to deliver the evaluation report no more than six weeks after receipt of results. Participating laboratories should be notified if the six-week turnaround time is not feasible. 3.2.4 Time to sending a second proficiency panel for laboratories failing a first panel A second proficiency panel should be made available to laboratories with unsatisfactory performance to support their quality improvement within three months of delivery of the evaluation report documenting unsuccessful performance. A proficiency panel must be successfully retested as part of a corrective action and quality improvement plan. The panel used for retesting should meet the requirements of the initial panel described above. In addition, sharing the second proficiency panel in a timely manner is also critical, within three months of delivery of the evaluation report, to enable maximum benefit of the proficiency testing panel for improving the quality of the participating laboratory. 3.3 Experience, financial sustainability and demonstrated ability to effectively manage a proficiency testing scheme 3.3.1 Experience in proficiency test development Providers of proficiency tests should have at least two years of experience in managing a HIV drug resistance proficiency testing programme. This requirement is necessary to demonstrate competence regarding the logistics of sample preparation, shipment, data analysis and reporting, and supporting participating laboratories with quality improvement guided by proficiency test evaluation reports. 3.3.2 Accreditation of a proficiency panel provider The provider of the proficiency panel should either be accredited according to the International Organization for Standardization (ISO) (ISO 17043) or should be actively pursuing ISO accreditation. ISO accreditation provides assurance of the provider’s competence to plan and implement proficiency testing programmes. Having ISO 17043 accreditation is also useful for ISO 15189 accredited WHO HIVResNet laboratories, which are required to participate in external quality assurance schemes from an accredited provider whenever possible. 3.3.3 Sustainable funding requirement Providers of HIV drug resistance proficiency panels should have sufficient funding to guarantee operation at their current level for a minimum of five years from the date when they start supporting WHO HIVResNet laboratories. The test provider should have sufficient funding to ensure the continuity of its proficiency testing scheme for WHO HIVResNet laboratories to minimize the potential for variability and logistical challenges that come with the need to change proficiency test providers. 3.3.4 Cost of the proficiency test The cost of participation by WHO HIVResNet laboratories in low-and middle-income countries should not be prohibitive and should preferably be free. Ideally, the proficiency test should be free of charge to WHO HIVResNet laboratories to secure their participation. However, some providers depend on a service charge to produce and manage operations. Thus, a low-cost service may be acceptable. Although some WHO HIVResNet laboratories are currently enrolled in schemes requiring them to pay, long-term sustainability may be challenging. 3.4 Evaluation, scoring and reporting of results 3.4.1 Minimum proficiency test evaluation process At a minimum, the evaluation process should include an assessment of sequence homology, drug resistance mutation concordance and evaluation for contamination or sample mix-up. Contamination or sample mix-up have been observed during routine evaluation of samples from WHO- recommended surveys of HIV drug resistance. If contamination, specimen mix-up or mislabelling is detected through analysis of sequence homology between samples and with the consensus sequences, the submission will be considered a failure, and incident investigation and corrective action is required. Policy brief, November 2023 7 3.4.2 Contents of a proficiency test evaluation report The evaluation report returned to participating laboratories should contain a sequence alignment that includes the consensus sequence derived from all laboratories that have performed a given proficiency test and, in a deidentified fashion, the sequence submitted by each individual laboratory. In addition, scoring for drug resistance mutation concordance and any other parameters evaluated (such as assessment for cross-contamination or sample mix-up) should be provided in the evaluation report. Where possible, the report should also include information on HIV- 1 subtype, the viral load of the samples tested and overall quality improvement guidance or recommendations, when relevant. The report generated by the proficiency test provider should include all potential information that can help laboratories participating in the evaluation scheme to understand their performance compared with other laboratories and sufficient information to help with troubleshooting in case of unsatisfactory performance. 3.4.3 Situations in which proficiency test providers do not meet WHO criteria for analysis and scoring In circumstances in which an alternative proficiency testing scheme does not meet all of the specified criteria outlined above for analysis and scoring of the proficiency tests, WHO would need to reanalyse the results generated by the network laboratories in accordance with subsection 4.4, and proficiency panel providers would be required to share the raw data necessary for WHO to perform the analysis. Over time, proficiency panel providers would be strongly encouraged to adapt their evaluation and scoring criteria to align with WHO recommendations. With respect to capacity building, not all proficiency test providers may be able to support corrective actions and quality improvement for laboratories with unsatisfactory performance (such as by performing a root-cause analysis and developing and monitoring the success of a corrective action plan). In these circumstances, this function can be performed by one or more WHO- designated specialized network laboratories. 3.4.4 WHO proficiency testing evaluation procedure Sequence data submitted by laboratories testing the proficiency panel are analysed by comparison to the group consensus sequence according to the proficiency testing scheme requirements, preferably following either the established VQA or WHO analysis procedures. The VQA analysis method is described elsewhere.2 The WHO analysis procedure is described below. A consensus sequence is prepared by aligning the sequences submitted by all participating laboratories. At each position in the alignment, the nucleotide or nucleotide mixture observed in >80% of the submitted sequences is included in the consensus. If no nucleotide or mixture is observed in >80% of the sequences, the position is excluded from the analysis for that specimen. The nucleotide sequence concordance of each laboratory’s submission, compared with the consensus sequence over the region spanning amino acids 10–93 of protease, 41– 238 of reverse transcriptase and 51–263 of integrase, is reported as a percentage of concordant nucleotides over the total number of nucleotides in the consensus (nucleotides reaching the >80% threshold for inclusion, even if mixed). Drug resistance mutation positions are defined as those with non-zero penalty scores in the contemporaneous version of the Stanford University HIV drug resistance algorithm (http://hivdb.stanford.edu). Concordance at both major and minor drug resistance mutation sites is determined when a submitted sequence is evaluated. Drug resistance mutation scores are calculated and expressed as a percentage of concordant drug resistance mutation codons detected by the laboratory over the total number of drug resistance mutation codons in the consensus. The definition of “concordance at drug resistance mutation positions” depends on the context and the presence or absence of mixed bases. • When mixtures are absent from both the consensus and the test sequence, the same nucleotide must be reported to be considered concordant, whether or not the nucleotide change results in an amino acid change or whether or not the change is considered wild type (same as the consensus HIV-1 subtype B reference) or mutant (any other amino acid). • If a nucleotide mixture is present in either the consensus or the test sequence, the mixture is treated according to the impact on the encoded amino acid(s), as outlined in Table 1. In addition, 2. Denny T, Louzao R, Carper M, Camp T, Scianna S. HIV-1 drug resistance sequencing: participation requirements and scoring procedures. Durham (NC): Duke Virology Quality Assurance Program, Duke Human Vaccine Institute; 2021 (https://www.hanc.info/content/dam/hanc/documents/ laboratory/vqa-resources/VQA_HIV-1_GEN_DR_PT_Information%20v2.0_20211101.pdf, accessed 23 August 2023). Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance 8 for comparisons involving mixtures to be counted as concordant, the nucleotide in the mixture must be compatible with the unmixed base (for example R versus A or G, Y versus C or T, but not Y versus G or compatible with the corresponding mixture (for example R versus M but not Y versus M). If mutations causing frameshifts are encountered in any test sequence, they will be handled as outlined below. • Erroneous deletion (missing nucleotide): a dash is inserted into the test sequence, and the remainder of the sequence is aligned against the consensus sequence following the standard procedure. When the alignment score is calculated, each dash is assigned a “gap penalty” of 10 (the alignment score reduced by 10 for each erroneous nucleotide deletion). • Erroneous insertion (extra nucleotide): the inserted nucleotide(s) is deleted and a “gap- opening penalty” of 10 is assigned for each erroneous nucleotide insertion. Since the error will not be visible in the group alignment, a separate note about this error is required. • Neither insertions nor deletions will affect the drug resistance mutation site score unless the insertion or deletion occurs at a codon where mutations are known to occur that are associated with drug resistance. To pass the panel, the laboratory must have an average of score across all samples of ≥99% nucleotide concordance for both the entire sequence and at drug resistance mutation positions. Table 1. Scoring matrix for drug resistance mutation positions involving mixtures (1 = concordant, 0 = discordant) Consensus sequence Wild type unmixed Mutant unmixed Mixed (A*, wt) Mixed (A, mut) Mixed (B, wt+mut) Mixed (C, >1 mut) Te st se qu en ce Wild type unmixed 1 0 1 0 0 0 Mutant unmixed 0 1 0 1 1 0 Mixed (A, wt) 1 0 1 0 0 0 Mixed (A, mut) 0 1 0 1 1 0 Mixed (B, wt+mut) 0 1 0 1 1 0 Mixed (C, >1 mut) 0 0 0 0 0 1 Mut=mutant, WT=wild type. *Mixture types: A (silent): the mixture results in codons that only encode one amino acid; discrepancies at the mixed base position are not counted. Example: consensus = GTR (WT = GTA), test sequence = GTG: since both encode valine, the difference is not considered as an error. However, if consensus = GAR (WT=GAA), test sequence = GAM, the mixture results in codons that encode two different amino acids; wild type which is common to both codons and mutant unique to the test sequence; therefore, this is considered an error. B (wt + mut aa): the mixture results in the presence of two or more amino acids, one of which is the wild type. Example: consensus = AYT (WT = ATT), test sequence = ACT: considered concordant; however, if the test sequence = ATT, considered discordant. C (complex): the mixture results in two or more amino acids, none of which is the wild type; if all mutants represented in the consensus sequence are also detected in the test sequence, they are considered as being concordant. Example: consensus = TWC (WT = ACC), test sequence = WCC or TRC: considered discordant since mutant’s Y/F not both detected in either result. Policy brief, November 2023 9 WHO region Name of WHO HIV drug resistance network laboratory WHO African Region Botswana-Harvard HIV Reference Laboratory (BHHRL), Gaborone, Botswana WHO African Region Retrovirus Côte d’Ivoire (Retro-CI), Abidjan, Cote d'Ivoire WHO African Region HIV and Other Viral Disease Research, EHNRI, Addis Ababa, Ethiopia WHO African Region KEMRI/CDC HIV Research Laboratory, Kisumu, Kenya WHO African Region National HIV Reference Laboratory, Nairobi, Kenya WHO African Region Centre For Human Virology & Genomics (CHVG) Nigerian Institute of Medical Research (NIMR), Lagos, Nigeria WHO African Region National HIV Reference Laboratory, Lilongwe, Malawi WHO African Region Bacteriology-Virology UTH A Le Dantec, Dakar, Senegal WHO African Region MRC/UVRI Basic Sciences Laboratory, Entebbe, Uganda WHO African Region National Health Laboratory Services Genotyping Laboratory, Johannesburg General Hospital, Johannesburg, South Africa WHO African Region AIDS Virus Research Unit, National Institute for Communicable Diseases (NICD), Johannesburg, South Africa WHO Region of the Americas Laboratory of AIDS and Molecular Immunology, Oswaldo Cruz Foundation – FIOCRUZ, Rio de Janeiro, Brazil WHO Region of the Americas Laboratorio de Virologia Molecular, Universidade Federal do Rio de Janeiro (LVM-UFRJ), Rio de Janeiro, Brazil WHO Region of the Americas University of British Columbia/Providence Health Care, Vancouver, Canada WHO Region of the Americas National Laboratory for HIV Genetics, PHAC, Winnipeg, Canada WHO Region of the Americas AIDS National Reference Laboratory (LISIDA), Havana, Cuba Annex 1: WHO HIV drug resistance network laboratories providing review Policy brief Novem ber 2023 Minimum criteria for external quality assurance proficiency tests for use by laboratories designated by WHO for the purposes of HIV drug resistance surveillance 10 WHO region Name of WHO HIV drug resistance network laboratory WHO Region of the Americas Service de Virologie Immunologie Centre Hospitalier et Universitaire de Fort-de-France, Fort de France, Martinique WHO Region of the Americas Centro de Investigación en Enfermedades Infecciosas, Instituto Nacional de Enfermedades Respiratorias (CIENI/INER), Mexico City, Mexico WHO Region of the Americas AIDS Research Program-Immunology Reference Laboratory, Ponce, Puerto Rico WHO Region of the Americas Drug Resistance Unit, International Laboratory Branch, DGHA, CGH, CDC, Atlanta, USA WHO European Region Laboratoire de Virologie, Centre Hospitalier Universitaire, Bordeaux, France WHO European Region UMI 233, TransVIHMI, IRD and UM1, Montpellier, France WHO European Region Public Health England, London, United Kingdom WHO South-East Asia Region Department of Clinical Research Tuberculosis Research Centre (ICMR), Chennai, India WHO South-East Asia Region National AIDS Research Institute (NARI), Indian Council of Medical Research, Pune, India WHO South-East Asia Region Clinical Microbiology Laboratory, Department of Microbiology, Faculty of Medicine, University of Indonesia, Jakarta, Indonesia WHO South-East Asia Region Dept Microbiology, Siriraj Hospital, Bangkok, Thailand WHO South-East Asia Region National Institute of Health, Department of Medical Sciences, Bangkok, Thailand WHO Western Pacific Region NSW State Reference Laboratory for HIV and Molecular Diagnostic Medicine, Sydney, Australia WHO Western Pacific Region Shanghai Municipal Center for Disease Control and Prevention, Shanghai, China WHO Western Pacific Region Key Laboratory of Immunology of AIDS, Ministry of Health, Shenyang, China WHO Western Pacific Region Division of Research on Virology and Immunology (DRVI), NCAIDS, Chinese Center for Disease Control and Prevention, Beijing, China WHO Western Pacific Region Laboratory of Molecular Diagnostics, National Institute of Hygiene and Epidemiology, Hanoi, Viet Nam WHO Western Pacific Region HIV/AIDS laboratory, Pasteur Institute, Ho Chi Minh City, Viet Nam Policy brief, November 2023 11 Department of HIV/AIDS World Health Organization 20 Avenue Appia 1211 Geneva 27 Switzerland hiv-aids@who.int who.int/hiv who.int