WHO Regional Training Workshop on Strengthening Evidence-based Decision-making in Immunization: Budva, Montenegro, 8–11 April 2019

1 WHO Regional Training Workshop on Strengthening Evidence-based Decision-making in Immunization Budva, Montenegro 8–11 April 2019 2 Abstract The WHO Regional Training Workshop on Strengthening Evidence-based Decision-making in Immunization was held in Budva, Montenegro on 8–11 April 2019. The countries represented included Albania, Belarus, Bosnia and Herzegovina, Georgia, Montenegro, North Macedonia, Romania, Serbia and Ukraine. The participants discussed the composition and functioning of national immunization technical advisory groups and collaboration with policy-makers; introduced the concept and discussed procedures for making evidence-based recommendations on immunization policy and practice; reviewed the criteria for making recommendations on introducing new vaccines and revising immunization schedules; described possible sources of evidence and opportunities for international collaboration; presented methods to analyse and synthesize evidence; and conducted case studies to apply evidence-based approaches for making recommendations on introducing a new vaccine through a simulated experience. Keywords EVIDENCE-BASED DECISION MAKING NATIONAL IMMUNIZATION TECHNICAL ADVICORY GROUP IMMUNIZATION POLICY TRAINING © World Health Organization 2019 All rights reserved. The Regional Office for Europe of the World Health Organization welcomes requests for permission to reproduce or translate its publications, in part or in full. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either express or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use. The views expressed by authors, editors, or expert groups do not necessarily represent the decisions or the stated policy of the World Health Organization. 3 CONTENTS ACRONYMS ................................................................................................................................. 4 INTRODUCTION ......................................................................................................................... 5 PURPOSE OF THE WORKSHOP ............................................................................................. 5 OPENING ...................................................................................................................................... 5 OVERVIEW OF THE WORKSHOP ......................................................................................... 5 FINAL RECOMMENDATIONS ................................................................................................ 9 COUNTRY COMMITMENTS................................................................................................. 10 CLOSING ................................................................................................................................... 10 POST-WORKSHOP EVALUATION ...................................................................................... 11 ANNEX A. List of participants ................................................................................................. 12 ANNEX B. Workshop agenda ................................................................................................... 15 ANNEX C. Workshop evaluation – selected results ............................................................... 20 4 ACRONYMS BCG bacille Calmette-Guérin (tuberculosis vaccine) ETAGE European Technical Advisory Group of Experts on Immunization GRADE Grading of Recommendations, Assessment, Development and Evaluation HPV human papillomavirus JCVI United Kingdom’s Joint Committee on Vaccination and Immunisation NITAG National Immunization Technical Advisory Group PICO population, intervention, comparison and outcome (four elements of a clinical question) SAGE Strategic Advisory Group of Experts on Immunization STIKO Standing Committee on Vaccination (Germany) 5 INTRODUCTION The WHO Regional Training Workshop on Strengthening Evidence-based Decision-making in Immunization for chairs and members of national immunization technical advisory groups (NITAGs) and their secretariats was held in Budva, Montenegro on 8–11 April 2019. The countries represented at the Workshop included: Albania, Belarus, Bosnia and Herzegovina, Georgia, Montenegro, North Macedonia, Romania, Serbia and Ukraine. The WHO Regional Office for Europe arranged the Workshop, which was supported by WHO headquarters and partners from the United States Centers for Disease Control and Prevention, Robert Koch Institute Standing Committee on Vaccination, United Kingdom Joint Committee on Vaccination and Immunisation (JCVI) Scientific Secretariat, Sabin Vaccine Institute and the Task Force for Global Health. Representatives of the WHO Regional Office for the Americas and the WHO Country Office in Montenegro also participated. PURPOSE OF THE WORKSHOP The objectives of the Workshop were to discuss the composition and functioning of NITAGs and collaboration with policy-makers; to introduce the concept and discuss procedures for making evidence- based recommendations on immunization policy and practice; to review the criteria to be considered to make recommendations on introducing new vaccines and revising immunization schedules; to describe possible sources of evidence and opportunities for international collaboration; to present methods to analyse and synthesize evidence; and to conduct case studies to apply evidence-based approaches for making recommendations on introducing a new vaccine through a simulated experience. OPENING The Workshop was opened on behalf of WHO by Miro Knezević, Deputy Minister of Health, Montenegro; Mina Brajovic, WHO Representative in Montenegro; and Siddhartha Datta, Programme Manager, Vaccine- preventable Diseases and Immunization, WHO Regional Office for Europe. OVERVIEW OF THE WORKSHOP Session 1: Overview of the NITAGs and countries’ experiences Presentation 1. Overview of NITAGs in the WHO Regional Office for Europe. The purpose and terms of reference for NITAGs were discussed as well as their value in strengthening decision-making on immunization. The recommendations and the minimum requirements for NITAGs of the Global Vaccine Action Plan and European Vaccine Action Plan were reviewed. The current performance of the European Region NITAGs and identified challenges with the function and composition of NITAGs were also outlined. Presentation 2. NITAG Resource Center and Global NITAG Network. The World Health Assembly and the Global Vaccine Action Plan midterm review both recommended strengthening NITAGs. The NITAG Resource Center provides technical documents and training materials for NITAG members. A Global NITAG Network is also available for NITAG members to join to share and access knowledge, liaise with regional NITAG networks, develop standards for processes and facilitate the evaluation and capacity- building of NITAGs. Countries were encouraged to join the Global NITAG Network. 6 Session 2: Composition and functioning of NITAGs Presentation 3: Country presentations on the composition and functioning of NITAGs. All participating countries reported having a NITAG, but Serbia’s NITAG was established in 2018 and has not yet convened. The countries had a wide variety of experiences with their NITAGs. Most had a legislative basis, but not all had written terms of reference and/or a work plan. Most had external experts representing various disciplines as the voting members. The rotation of members varied from 2 to 4 years, and some countries are using work groups. The frequency of meetings varied and is not a set number per year for most NITAGs, and most had an established voting process for making decisions. Many of the NITAGs collected declarations of interest from voting members, and some required members to sign a confidentiality clause. Discussion. Ukraine said that they include a communication specialist on their NITAG to assist with issues on vaccine hesitancy and to make recommendations on communication strategies. Some countries also included a gynaecologist on their NITAG, which is helpful when discussing human papillomavirus (HPV) vaccine, maternal bacille Calmette-Guérin vaccine (BCG) and vaccination of pregnant women. Several countries spoke of the value of using work groups for specific topics, such as developing guidelines on contraindications to immunization. Presentation 4: Composition and modes of functioning of NITAGs. An overview of the recommended composition and function of NITAGs was provided. Information was presented on key terms of reference, recommended stakeholders, NITAG composition and the corresponding roles (core, ex officio and liaison members and the secretariat), the charter and the process before, during and after a NITAG meeting. NITAG work groups, declarations of interest and the process for decision-making were also discussed. Presentation 5. Declaration and management of conflicts of interest. Guidelines on the declaration and management of interests were presented. Conflict of interest was defined, and examples were given. Methods to manage and prevent conflicts of interest were provided, including having a NITAG policy to prevent and manage conflicts of interest. The process for such a policy was outlined, and examples of potential conflicts of interest were reviewed. Presentation 6: Composition and functioning of STIKO. An overview of the composition and functioning of the Standing Committee on Vaccination of the Robert Koch Institute (STIKO, Germany’s NITAG) was presented. The members, Secretariat, work groups, conflict of interest management, process for developing recommendations and communication of decisions were described. Discussion. There was much discussion about conflicts of interest, and some countries reported challenges finding NITAG members who do not have conflicts of interest. A few countries reported that they do not currently have a process in place to evaluate conflicts of interest among NITAG members. It was noted that having a transparent process to evaluate declared interests is important to ensure that there are no perceived conflicts. It was also mentioned that there are different levels of conflict of interest: some may be minor, and others may be major. Countries should have a method in place for evaluating these conflicts of interest. It was mentioned that STIKO allows manufacturers to meet with the NITAG once per year to present data and information, but they are not allowed to be present for discussion. Exercise A: Review of the structure of NITAGs. Each country evaluated the current composition and functioning of their NITAG and developed recommended changes. Some of the proposed changes included making health ministry staff non-voting members, developing processes for declarations of interest and confidentiality agreements, modifying the rotation of members and identifying ways to recruit new NITAG members. 7 Presentation 7: Collaboration between the NITAG and the health ministry. The health ministry’s role in establishing the NITAG, appointing members and supporting the NITAG were discussed. Communication between the NITAG and health ministry, communication of decisions to stakeholders and implementation of NITAG recommendations were also covered. Discussion. It was suggested that NITAGs work with the health ministry to ensure that their recommendations are feasible and to decide when cost–effectiveness studies are needed. Exercise B: Strengthening collaboration between the NITAG and the health ministry. Participants reviewed a sample work plan and two scenarios on introducing new vaccines and provided feedback. Discussion. One country felt that having a template for the NITAG meeting minutes and reporting would be useful. The importance of having a very thorough process for identifying the appropriate stakeholders for each vaccine introduction was discussed, such as including education and reproductive health representatives during HPV vaccine discussions. Presentation 8: Collaboration between global, regional and national technical advisory groups. The functions of the Strategic Advisory Group of Experts on Immunization (SAGE) and European Technical Advisory Group of Experts on Immunization (ETAGE) and the interactions between SAGE, regional immunization technical advisory groups and NITAGs were reviewed. An external evaluation of SAGE and recommendations for improvement were discussed. Discussion. Some countries reported using the SAGE documents and WHO recommendations and position papers to prepare for NITAG recommendations and decisions. One country indicated the need for support in preparing for the introducing HPV vaccine. Session 3: Preparing for NITAG meetings Presentation 9: HPV diseases and vaccines. The participants discussed: the epidemiology of HPV infections and related diseases; HPV vaccine efficacy, immunogenicity and safety; HPV vaccine recommendations and introductions; and the early effects of HPV vaccination in the United States of America and other countries. Discussion. There was discussion about countries that have moved from a three-dose to a two-dose schedule of HPV vaccine and the data behind this decision. Some countries are concerned about communication strategies because of resistance to the HPV vaccine, and some countries are unsure which HPV vaccine to introduce (2, 4 or 9 valent). Presentation 10: From policy question to recommendation. The four steps to move from a policy question to a recommendation were reviewed. The process of developing a broad policy question and then specifying the policy question using the population, intervention, comparison and outcome (PICO) method was introduced. Exercise C: Drafting PICO questions. Participants reviewed broad policy questions and drafted PICO questions for each. Some groups had challenges in completing this exercise. It was recommended that the participants define the outcome broadly, such as using a broad outcome of disease burden rather than a narrower outcome of reduced disease burden. Presentation 11: Criteria for decision-making. Defining the criteria for making a recommendation and ranking of criteria were described. A generic criteria table was introduced, and the speaker gave examples of the data needed for each element and demonstrated how to rank each outcome. 8 Exercise D: Adapting generic criteria and ranking outcomes. The participants worked in groups to make necessary adaptations and rank the outcomes from four criteria tables based on a PICO question on HPV vaccine. The groups varied in how they ranked the outcomes in the criteria tables. The participants were reminded that the criteria tables are meant for the NITAG and not for the health ministry and to consider the PICO question when ranking the criteria. One country said that their NITAG would consider the financial affordability of vaccines when considering an introduction. Presentation 12: Gathering evidence: systematic literature review and evaluating evidence. The process of gathering and analysing evidence and various sources for finding evidence through local data sources and literature reviews were discussed. The steps necessary to conduct a systematic literature review were outlined, and the sources for accessing systematic literature reviews that have already been conducted were provided. Methods to evaluate the quality of the evidence and SAGE evidence profiles were also covered. Exercise E: Reading SAGE tables on GRADE quality assessment and where to find more information. The participants worked in groups to access a SAGE table on GRADE (Grading of Recommendations, Assessment, Development and Evaluation) quality assessment online and then answered questions about the table and ratings. During the second exercise, participants navigated the WHO SAGE website to find documents and answer questions. Discussion. There was discussion about how studies are rated when there is a mix of randomized clinical trials and observational studies. There was some confusion about the HPV safety SAGE table because the PICO question did not include a statement about severe adverse events following immunization. It was noted that SAGE tables may have been developed several years ago and may not include all of the most recent data and, if a country does not want to use a SAGE table, they could look for another data source, such as JCVI or STIKO. Presentation 13: Evidence-to-recommendation framework. The evidence-to-recommendation framework was described, and evidence-to-recommendation tables were reviewed. Presentation 14: Experience from JCVI in making recommendations on HPV vaccine. The decision- making process JCVI used in considering the introduction of HPV vaccine was reviewed from framing the question, gathering evidence, conducting analysis and translating the evidence into a recommendation. Cost–effectiveness is a key consideration of JCVI’s advice. Exercise F: Reviewing the SAGE evidence-to-recommendation framework. Participants worked in groups to review and answer questions about the SAGE evidence-to-recommendation framework on HPV vaccine. The participants liked that the document is compact and provides good information for decision- makers, but some felt that not all the criteria were useful. One participant noted that the document is not dated; how recently the document has been updated is therefore unclear. Most participants felt that these SAGE tools would be useful for their NITAG to review when making recommendations. Session 4: Developing NITAG recommendations Presentation 15: Evolution of the recommendations of the Advisory Committee on Immunization Practices on HPV vaccine. An overview of the evolving HPV vaccine recommendations since 2016 in the United States of America was provided, and there was discussion on how to decide when a recommendation needs to be updated. Presentation 16: Working group and NITAG background documents. The purpose, structure and best practices of a working group were outlined. Background documents to be developed by working groups 9 were described, as were the recommended roles, responsibilities and interactions between NITAG and a working group. Presentation 17: Experience of STIKO with a working group on HPV vaccine. The structure and function of STIKO’s HPV work group were covered. The steps taken in developing an HPV vaccine recommendation were outlined, and communication about the final recommendation was described. Discussion. The participants had questions about the epidemiological and health modelling on HPV vaccine that was done for STIKO and how this work was procured. Exercise G: Establishing a working group. Participants reviewed a scenario about a country planning to introduce HPV vaccine and responded to questions about the responsibilities and composition of a working group. Session 5: Developing NITAG minutes and policy briefs Presentation 18: How NITAGs document their work. The methods NITAGs should use to document their work were discussed, including: meeting minutes, reports of recommendations to the health ministry, publishing recommendations and disseminating information to the general public. Presentation 19: Technical resource package on HPV vaccine. An overview of the resource package on HPV vaccination being developed by the Task Force for Global Health was provided, and the content of the package was summarized. The participants were then asked for their feedback on the package. One suggestion was to include a case study on the introduction of HPV vaccine using a less developed NITAG. FINAL RECOMMENDATIONS WHO recommended that countries arrange a meeting with all members of the NITAG in their country to provide feedback from the training and to discuss the implementation of recommendations from this Workshop. This would include the following. ➢ NITAG structure and collaboration with the health ministry: The NITAG charter and terms of reference (with a focus on the six criteria) should be reviewed and necessary changes should be made. A health ministry focal point should be identified for NITAG communication, and the secretariat should be strengthened to implement an evidence-based approach in developing recommendations. ➢ Working groups: A working group should be formed to prepare a topic requested by the health ministry for NITAG deliberations. The composition of the working group should be defined, terms of reference and a work plan should be developed and the health ministry should give its approval. ➢ NITAG background document: The critical and important criteria for making recommendations should be defined (using the tables introduced during the meeting), and evidence should be collected and evaluated using the methods outlined during the meeting. ➢ NITAG documentation: Meeting minutes and policy briefs should be prepared after each NITAG meeting, and standard operating procedures for developing evidence-based recommendations should be developed. 10 COUNTRY COMMITMENTS The countries were asked what they planned to do as a result of this Workshop and made the following responses. Albania. They will bring this information back to their members and update their terms of reference and will work with the framework and put to use all of the information that was provided. Belarus. They will need to do a lot of work to meet the WHO criteria of NITAG functionality. The training recommendations are very useful however; they need to further discuss how to implement them. The most active members of their NITAG are very busy with their regular jobs. Therefore the Secretariat should use working groups to prepare topics for NITAG discussions. They will consider including PhD students and their professors to collect and evaluate evidence. Bosnia and Herzegovina. They will prepare a report to the health ministry and will use this information in their future work. They will structure their information using the instructions and will try to arrange a meeting upon their return. Georgia. They will use this new information and will organize a meeting with their NITAG to talk about establishing a working group and discuss the Secretariat. They will try to reorganize their roles based on this information. Montenegro. They need to reorganize and make changes to their NITAG and use the information they gathered during this meeting. They will prepare a process to introduce new vaccines. This meeting was very useful for them in further developing their NITAG. North Macedonia. They will prepare a report and send it to the Ministry to inform them about these recommendations. The training gave them many new ideas. They will need to start with the structure and align their NITAG with the WHO requirements. They will share this information with their NITAG. Romania. They plan to meet and discuss what they learned and provide feedback to the person who sent them to this meeting. Hopefully, by the end of 2019, they will have a new NITAG. Serbia. Their NITAG has not yet started, and now they have a lot of information to assist them. They will first report back to their health ministry and will then begin working. Ukraine. They were planning a NITAG meeting in April 2019 to discuss the national immunization schedule. They will prepare a report on the results of this meeting to discuss during the NITAG meeting and will discuss the next steps to modify their Terms of Reference. They will also discuss these issues with the health ministry. CLOSING Participants were asked whether they had ideas about how the various partners (WHO, United States Centers for Disease Control and Prevention, Task Force and Sabin Vaccine Institute) could support them in developing their NITAGs. They were encouraged to send this information later and were assured that there would be follow-up to check on their progress and to offer additional support. 11 POST-WORKSHOP EVALUATION At the close of the Workshop, participants were asked to complete an anonymous evaluation. Twenty- seven participants completed this, and the responses were generally positive. Most felt that the presentations and group exercises were relevant to their NITAG work and that the content was easy to understand. Of the group exercises, drafting the PICO questions was felt to be the most relevant and “where I can find” the least relevant. Overall, the participants reported that their understanding and confidence increased in all areas as a result of the training provided (see Annex C for selected results). Some suggestions for improvement for future training included: more exercises, more experts, more practical examples (such as STIKO’s presentation), opportunities to visit other countries’ NITAGs to exchange information, shorter days and more social events. Fourteen of the respondents felt they would need follow- up technical support to make evidence-based recommendations on HPV vaccine. 12 Annex A. List of participants First name Last name Title Country Najada Como Head, Clinic of Infectious Diseases, University Hospital Center “Mother Teresa” Albania Erida Nelaj Immunization Programme, Institute of Public Health Albania Iria Preza Immunization Programme, Institute of Public Health Albania Irina Hlinskaya Deputy Chief Doctor of Epidemiology, Republican Center for Hygiene, Epidemiology and Public Health and Member, Expert Council on Immunization, Ministry of Health Belarus Elena Samoilovich Head, Laboratory Vaccine-preventable Diseases, Republican Scientific and Practical Center for Etiology and Microbiology and Deputy Chair, Expert Council on Immunization, Ministry of Health Belarus Sanjin Musa Immunization Programme Manager, Public Health Institute, Federation of Bosnia and Herzegovina Bosnia and Herzegovina Damir Sabitovic Head, Immunization Technical Advisory Group, Ministry of Health, Federation of Bosnia and Herzegovina Bosnia and Herzegovina Eric Zelimir Paediatrician, Clinical Centre Banja Luka Bosnia and Herzegovina Lia Jabidze Immunization Program Manager and Member, National Immunization Technical Advisory Group Georgia Paata Imnadze Deputy Director, LEPL National Center for Disease Control and Public Health and Core Member, National Immunization Technical Advisory Group Georgia Teona Kashibadze Senior Specialist, Immunization Division, LEPL National Center for Disease Control and Public Health and Secretary, National Immunization Technical Advisory Group Georgia Edita Basovic Primary health care physician (paediatrician) Montenegro Dragan Laušević Institute of Public Health Montenegro Mileva Mašanović Clinical Centre of Montenegro Montenegro Lidija Poček Institute for Child Diseases, Clinical Centre of Montenegro Montenegro Nelica Ivanović Radović Institute for Child Diseases, Clinical Centre of Montenegro Montenegro Kristina Stavridis Member, National Immunization Group North Macedonia 13 First name Last name Title Country Branka Velichkovska Immunization Program Officer, Division of Preventive Health Care, Department of Preventive and Primary Health Care, Ministry of Health North Macedonia Gratiana Chicin Epidemiology Section Chief, Regional Center of Public Health Timișoara, National Institute of Public Health Romania Mihai Craiu Primary Pediatrician, National Institute for the Health of Mother and Child “Alessandrescu- Rusescu” Romania Jasna Jancic Professor of Child Neurology, Faculty of Medicine, University of Belgrade and member, NITAG Serbia Tanja Jovanovic Institute of Microbiology, Faculty of Medicine Serbia Vesna Knjeginjic Assistant Minister, Sector for Public Health and Health Care Programs, Ministry of Health Serbia Nada Milic Head, Department of Sanitary Inspection, Ministry of Health Serbia Vera Stoiljkovic Acting Managing Director, Institute of Virology, Vaccines and Sera “Torlak” Serbia Olga Bokancha Senior Translator and Interpreter and Head, Translation Unit, Public Health Center, Ministry of Health Ukraine Fedir Lapii Associate Professor, Department of Paediatric Infectious Diseases and Paediatric Immunology, National Medical Academy of Postgraduate Education Ukraine Olena Levchenko Epidemiologist, Public Health Center, Ministry of Health Ukraine Oleksandr Zaika Head, Immunization Strategy Department, Public Health Center, Ministry of Health Ukraine First name Last name Organization Facilitators Erin Kennedy Epidemiologist, Demand, Policy, and Communications Team, Immunization Systems Branch, Global Immunization Division, United States Centers for Disease Control and Prevention United States Centers for Disease Control and Prevention Abigail Shefer Medical Epidemiologist, Science, Policy, and Research Coordinator, Immunization Systems Branch, Global Immunization Division, United States Centers for Disease Control and Prevention United States Centers for Disease Control and Prevention 14 First name Last name Organization Facilitators Elissa Meites Medical Epidemiologist, HPV Team, Division of Viral Diseases, National Center for Immunization and Respiratory Diseases, United States Centers for Disease Control and Prevention United States Centers for Disease Control and Prevention Christopher Lucas JCVI Scientific Secretariat, Immunisation, Hepatitis, Blood Safety and Countermeasures Response, National Infection Service, Public Health England JCVI Thomas Harder Robert Koch Institute, STIKO Robert Koch Institute Stacey Knobler Sabin Vaccine Institute Sabin Vaccine Institute Stephen Hadler Senior Consultant, Center for Vaccine Equity, Task Force for Global Health Task Force for Global Health Nathalie El Omeiri WHO Regional Office for the Americas WHO Kathleen Cavallaro Consultant WHO Brajovic Mina WHO Country Office in Montenegro WHO Louise Henaff WHO headquarters WHO Christoph Steffen WHO headquarters WHO Siddhartha Datta WHO Regional Office for Europe WHO Lisa Jacques- Carroll WHO Regional Office for Europe WHO Liudmila Mosina WHO Regional Office for Europe WHO Natalia Nemirova WHO Regional Office for Europe WHO 15 Annex B. Workshop agenda Monday, 8 April 2019 8:30–9:00 Registration 9:00–9:30 Welcome by the Ministry of Health Welcome by thee WHO Country Office in Montenegro Welcome by the WHO Regional Office for Europe Meeting objectives Session 1: Overview of the NITAGs and countries’ experiences Chair: Siddhartha Datta, WHO Regional Office for Europe 9:30–9:50 Overview of NITAGs in the WHO European Region Liudmila Mosina WHO Regional Office for Europe 9:50–10:20 NITAG Resource Centre and Global NITAG Network Louise Henaff WHO headquarters 10:20–10:30 Discussion 10.30–11.00 Break Session 2: Composition and functioning of NITAGs Chair: Abigail Shefer, United States Centers for Disease Control and Prevention 11:00–12:30 Country presentations: composition and functioning of NITAGs Albania Belarus Bosnia and Herzegovina Georgia Montenegro North Macedonia Romania Serbia Ukraine 12:30–13:30 Lunch break 13:30–14:00 Composition and modes of functioning of NITAGs Kathleen Cavallaro WHO consultant 14:00–14:20 Declaration and management of conflicts of interest Christoph Steffen WHO headquarters 14:20–14:50 Composition and functioning of STIKO Thomas Harder STIKO 16 14:50–15:10 Discussion 15:10–15:30 Group exercise A: Review of the structure of NITAGs 15:30–16:00 Break 16:00–16:25 Group exercise A: Review of the structure of NITAGs (continued) 16:25–17:15 Feedback from countries on the exercise 17:15–17:35 Discussion 17:35–17:50 General announcements and daily adjournment 18:30–19:30 Reception Tuesday, 9 April 2019 Session 2: Composition and functioning of NITAGs (continued) Chair: Abigail Shefer, United States Centers for Disease Control and Prevention 9:00–9:20 Collaboration between the NITAG and the health ministry Liudmila Mosina WHO Regional Office for Europe 9:20–9:40 Discussion 9:40–10:30 Group exercise B: Strengthening collaboration between the NITAG and health ministry 10:30–11:00 Break 11:00–11:30 Feedback from groups on the exercise 11:30–12:00 Discussion 12:00–12:30 Collaboration between global, regional and national technical advisory groups Christoph Steffen WHO headquarters 12:30–13:30 Lunch break 13:30–13:50 Discussion Session 3: Preparing for a NITAG meeting Chair: Christoph Steffen, WHO headquarters 13:50–14:20 HPV diseases and vaccines Elissa Meites United States Centers for Disease Control and Prevention 14:20–14:40 Discussion 17 14:40–14:55 Policy question Abigail Shefer United States Centers for Disease Control and Prevention 14:55–15:30 Group exercise C: Drafting PICO questions 15:30–16:00 Break 16:00–16:25 Group exercise C: Drafting PICO questions (continued) 16:25–17:00 Feedback from groups on the exercise 17:00–17:20 Discussion 17:20–17:30 General announcements and daily adjournment Wednesday, 10 April 2019 Session 3: Preparing for a NITAG meeting (continued) Chairs: Christoph Steffen and Louise Henaff, WHO headquarters 9:00–9:20 Criteria for decision-making Abigail Shefer United States Centers for Disease Control and Prevention 9:20–10:10 Group exercise D: Adapting generic criteria and ranking outcomes 10:10–10:30 Feedback from groups on the exercise 10:30–11:00 Break 11:00–11:40 Gathering evidence: systematic literature review and evaluating evidence Elissa Meites Abigail Shefer United States Centers for Disease Control and Prevention 11:40–12:10 Group exercise: Where to find more information 12:10–12:30 Group exercise E: Reading SAGE tables on GRADE quality assessment 12:30–13:30 Lunch break 13:30–13:50 Group exercise E: Reading SAGE tables on GRADE quality assessment (continued) 13:50–14:20 Feedback from groups on the exercise 14:20–14:40 Discussion 18 14:40–14:55 Evidence-to-recommendation framework Elissa Meites United States Centers for Disease Control and Prevention 15:15–16:15 Break 16:15–16:45 Experience from JCVI in making recommendations on HPV vaccine Christopher Lucas JCVI 16:45–17:15 Group exercise F: Reviewing the SAGE evidence-to-recommendation framework 17:15–17:45 Feedback from groups on the exercise 17:45–18:00 General announcements and daily adjournment Thursday, 11 April 2019 Session 4: Developing NITAG recommendations Chair: Liudmila Mosina, WHO Regional Office for Europe 9:00–9:20 Evolution of the recommendations of the Advisory Committee on Immunization Practices on HPV vaccine Elissa Meites United States Centers for Disease Control and Prevention 9:20–9:40 Working group and NITAG background documents Erin Kennedy United States Centers for Disease Control and Prevention 9:40–10:10 Experience of STIKO with a working group on HPV vaccine Thomas Harder STIKO 10:10–10:30 Discussion 10:30–11:00 Break 11:00–11:45 Group exercise G: Establishing a working group 11:45–12:30 Feedback from groups on the exercise 12:30–13:30 Lunch break Session 5: Developing NITAG minutes and policy briefs Chair: Kathleen Cavallaro, WHO consultant 13:30–13:45 How NITAGs document their work Louise Henaff 19 WHO headquarters 13:45–14:00 Discussion 14:00–14:30 Workshop recommendations and countries feedback 14:30–15:00 Technical resource package on HPV vaccine Stephen Hadler Task Force for Global Health 15:00–15:30 Group exercise: feedback on resource package 15:30–16:00 Break 16:00–16:20 Training ends 20 Annex C. Workshop evaluation – selected results 21 22 23 The WHO Regional Office for Europe The World Health Organization (WHO) is a specialized agency of the United Nations created in 1948 with the primary responsibility for international health matters and public health. The WHO Regional Office for Europe is one of six regional offices throughout the world, each with its own programme geared to the particular health conditions of the countries it serves. Member States Albania Andorra Armenia Austria Azerbaijan Belarus Belgium Bosnia and Herzegovina Bulgaria Croatia Cyprus Czechia Denmark Estonia Finland France Georgia Germany Greece Hungary Iceland Ireland Israel Italy Kazakhstan Kyrgyzstan Latvia Lithuania Luxembourg Malta Monaco Montenegro Netherlands North Macedonia Norway Poland Portugal Republic of Moldova Romania Russian Federation San Marino Serbia Slovakia Slovenia Spain Sweden Switzerland Tajikistan Turkey Turkmenistan Ukraine United Kingdom Uzbekistan World Health Organization Regional Office for Europe UN City, Marmorvej 51, DK-2100 Copenhagen Ø, Denmark Tel.: +45 45 33 70 00 Fax: +45 45 33 70 01 Email: eurocontact@who.int Website: www.euro.who.int