Digital solutions to health risks raised by the COVID-19 infodemic: policy brief

Digital solutions to health risks raised by the COVID-19 infodemic Policy brief Abstract A policy brief is a short publication specifically designed to provide policy-makers with evidence on a policy question or priority. Policy briefs bring together existing evidence and present it in an accessible format and use systematic methods and make these transparent so that users can have confidence in the material. Policy briefs tailor the way evidence is identified and synthesized to reflect the nature of the policy question and the evidence available, and are underpinned by a formal and rigorous open peer review process to ensure the independence of the evidence presented. Policy briefs provide instant access to key information and additional detail for those involved in drafting, informing, or advising on the policy issue. Policy briefs provide evidence for policy-makers, not policy advice. They do not seek to explain or advocate a policy position but to set out clearly what is known about it. They may outline the evidence on different prospective policy options and on implementation issues, but they do not promote a particular option or act as a manual for implementation. Document number: WHO/EURO:2022-5351-45116-64364 © World Health Organization 2022 Some rights reserved. This work is available under the Creative Commons Attribution-Non Commercial-Share Alike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO endorses any specific organization, products or services. The use of the WHO logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition: Digital solutions to health risks raised by the COVID-19 infodemic: policy brief. Copenhagen: WHO Regional Office for Europe; 2022”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules/). Suggested citation. Digital solutions to health risks raised by the COVID-19 infodemic: policy brief. Copenhagen: WHO Regional Office for Europe; 2022. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO to verify the information contained in this publication However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use. Designed by: www.freightdesign.co.uk Cover image: © WHO/Frank Henriksen Digital solutions to health risks raised by the COVID-19 infodemic Policy brief Contents Foreword iv Acknowledgements v Abbreviations v Executive summary vi 1. Introduction 1 2. Methodology 2 3. Health risks and harms due to the COVID-19 infodemic 3 4. Digital solutions to health risks raised by misinformation 5 5. Policy considerations 9 6. What WHO is doing to address the infodemic in the 10 European Region and globally 6.1 Digital communication strategies for infodemic management 10 6.2 Mobile apps disseminating accurate health information 10 6.3 AI-based innovations for rumour tracking, 11 community feedback and social listening 6.4 Training, events and conferences to promote infodemic management 11 6.5 Behavioural insights surveys, better rumour tracking mechanisms, 11 multi-language fact-checking 6.6 Partnering with social media and web companies 11 7. Conclusions 13 References 14 Annex. Bibliographic search strategy 19 Additional reading 21 iii We are delighted to present this policy brief addressing the application of digital solutions to health risks raised by the COVID-19 infodemic. The spread of excessive, false and misleading information on COVID-19 has become a major cause of reduced effectiveness of national COVID-19 response efforts and has the potential to directly threaten lives and livelihoods of populations across the WHO European Region. The infodemic, as it is now commonly known, has the ability to cause confusion through an overabundance of information – including false and misleading information, and to encourage risk-taking behaviours that can seriously harm health and lead to mistrust in health authorities. The European Region contains some of the world’s most connected societies, where social media and internet use are an integral part of daily life and information can spread rapidly. While this level of access to information can be a major benefit in filling knowledge voids and reaching target audiences, it can also be used to amplify harmful messages. Effective management of the infodemic requires systems-level interventions together with measures for individual capacity and skills building at country level. Such actions include equipping Member States with the digital tools and skills to identify and act upon sources and flows of rumours, mis- and disinformation, as well as developing digital, health and media literacy in populations to enable individuals to act on information, question misinformation and evaluate sources. Based on our learnings from the COVID-19 pandemic, the findings and considerations in this policy brief offer some insight into how digital tools can be applied by Member States to prevent and mitigate the effects of misinformation and improve the online environments that are an essential source of information for many. We trust that it will offer health authorities across the European Region the knowledge to help position themselves as trusted sources and make access to online information safer, more reliable and more rights-based for all. Natasha Azzopardi Muscat Director Division of Country Health Policies and Systems WHO Regional Office for Europe Gerald Rockenschaub Director Health Emergencies Programme WHO Regional Office for Europe 1. Foreword Digital solutions to health risks raised by the COVID-19 infodemiciv This policy brief has been authored by members of the Infodemic Management Project Team within the Communication Pillar in the COVID-19 Incident Management Support Team and the Empowerment through Digital Health flagship initiative of the Division of Country Health Policies and Systems at the WHO Regional Office for Europe. The main contributors are, in alphabetical order, Agnese Pastorino, Simon van Woerden and Stefan Voinea. Substantial technical contributions and guidance have been provided by Clayton Hamilton and Cristiana Salvi. The authors thank Natasha Azzopardi Muscat, Dorit Nitzan and Gerald Rockenschaub for their support in creating this policy brief. We also thank Tim Nguyen and Tina Purnat for their leadership and work in the area of infodemic management. Lastly, we thank Nils Fietje, Atsuyoshi Ishizumi, and Brian Yau for their thoughtful comments and suggestions. Abbreviations AI artificial intelligence ECDC European Centre for Disease Prevention and Control RCCE risk communication and community engagement UHC universal health coverage UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children’s Fund Acknowledgements Digital solutions to health risks raised by the COVID-19 infodemic v The COVID-19 pandemic has been accompanied by feelings of uncertainty, worry and fear, which nourished a lack of trust towards health authorities, scientific communities and mass media, enhanced by false information spread on the Internet (1). This phenomenon associated with the pandemic has been called an information epidemic or “infodemic” and defined as “an overabundance of information including false or misleading information in digital and physical environments during a disease outbreak” (2). Due to the spread of false and misleading health- related content, the COVID-19 outbreak response has in some cases become less timely and effective, negatively impacting the livelihoods and emotional well-being of populations (3). The exposure to false information – both online and offline – has been linked to increased health risks (4), having harmful or even deadly effects (5). Examples of negative behaviours include use of wrong or harmful treatments, lower uptake of protective measures including vaccinations (6), impaired mental health and emotional well-being (7), and lower trust in health-care providers (8). In this scenario, the WHO Regional Office for Europe sought to answer the question of how digital solutions can be used to address the infodemic. A rapid review of the evidence revealed that digital solutions that aim to leverage technological innovations are best positioned to respond to the infodemic. It is important to emphasize that online and offline environments are intertwined and the underlying factors of the COVID-19 infodemic are not purely digital, but they are rooted in complex historical and sociocultural contexts. Therefore, approaching the infodemic through digital interventions alone may not be sufficient. The below-illustrated digital solutions need to be combined with already tested offline approaches to outbreak response. Risk communication and community engagement interventions are one of the main examples of best practices to be dovetailed with digital perspectives (9). The identified digital interventions designed to tackle infodemics include: • implementing fact-checking (10) and false information reporting mechanisms (11); • adopting social listening tools augmented by artificial intelligence, which can help analyse the large-scale fast-flowing data, assess risks and identify infodemic signals (12); • creating monitoring programmes (13), multi- stakeholders’ coordination initiatives (14) and national regulatory frameworks which respect the principles of freedom of expression (15); • promoting digital health literacy (16) and inoculation interventions that improve people’s ability to spot misinformation (17). Adopting these digital solutions calls for a whole-of-society approach to infodemic response. National health authorities, journalists, fact- checkers, civil society organizations, empowered citizens and other relevant parties all play an important role in building trust and tackling the spread of harmful health-related information (18). Executive summary Digital solutions to health risks raised by the COVID-19 infodemicvi Six policy considerations The WHO Regional Office for Europe highlights six policy considerations: 1. Reinforcing multistakeholder networks for infodemic management. 2. Strengthening overall risk communication and community engagement. 3. Implementing continuous monitoring of online harmful and false content. 4. Improving digital literacy approaches and organizing infodemic management trainings. 5. Advocating for infodemic management through communication campaigns. 6. Ensuring safe online platforms, which protect people from harmful content. © WHO/Uka Borregaard 1. Introduction As part of a multifaceted approach to tackling the COVID-19 pandemic, the United Nations has called on countries to take urgent action to address what has been described as the “infodemic”(2). Caused by an excess of information, false information and rumours, the infodemic has exacerbated the threat to people’s health and well-being and contributed to the risk of social instability. United Nations Secretary-General António Guterres warned about the combined threat of COVID-19 and false information: “Our common enemy is #COVID19, but our enemy is also an infodemic of misinformation”(19). Likewise, WHO Director- General Dr Tedros Adhanom Ghebreyesus emphasized the need to address both the biological and informational harms of COVID-19: “We are not just fighting an epidemic, we are fighting an infodemic”. At the seventy-third World Health Assembly in 2020, Member States endorsed WHA Resolution 73.1, which calls upon WHO to “address, and where relevant in coordination with Member States, the proliferation of disinformation and misinformation particularly in the digital sphere”. In response to the infodemic, WHO has launched a variety of initiatives at global and regional levels. At regional level these include actions in the areas of: risk communication and community engagement (e.g. by monitoring for and responding to high- risk infodemic signals, including the recent spike in misinformation on protective behaviours, including COVID-19 vaccines) (9,20); digital health (e.g. planned development of e-learning capacity-building targeted towards national health authorities); and behavioural and cultural insights (e.g. the development of a Health Evidence Network study on how inherent uncertainties and gaps in evolving evidence drive infodemic harms during large-scale epidemics and pandemics). This tri-divisional project contributes in various ways to the core priorities and flagship initiatives of the WHO Regional Office for Europe’s European Programme of Work 2020–2025 – “United Action for Better Health in Europe” and its core priorities of protecting people from health emergencies and moving towards universal health coverage (UHC). The infodemic harms, in particular, those population groups who are already vulnerable from a UHC perspective, including those with lower health, digital and information literacy skills (21), and one of the aims of infodemic management is to address these inequities in a sustainable manner. In continuation of these efforts, the WHO Regional Office for Europe has decided to develop this policy brief that identifies digital solutions to the health risks posed by the COVID-19 infodemic. The nature of these solutions is either technological, political, regulatory, educational or communicative. This document collects them and shows relevant examples, retrieved from the WHO COVID-19 Research Database. The policy brief also aims to present policy considerations which can help Member States take concrete actions in addressing the COVID-19 infodemic using digital solutions. 1 Digital solutions to health risks raised by the COVID-19 infodemic The evidence used in this policy brief was obtained through a rapid review of the most recent publications on digital solutions to health risks observed during the COVID-19 infodemic. Keyword searches were performed in the WHO COVID-19 Research Database (22). This database gathers global literature on COVID-19, retrieved and catalogued from some of the main online databases – including Scopus, Medline, Web of Science, Europe PMC, Taylor & Francis and ProQuest Central, among others. In performing the searches, the main words describing the most well-known technological advances and digital solutions were combined with the terms “COVID-19” and “infodemic” appearing in the titles or abstracts. The words “COVID-19” and “infodemic” were, therefore, searched in combination with the terms “fact-checking”, “social listening”, “artificial intelligence”, “deep learning”, “machine learning”, “literacy”, “policy”, “regulation”, “regulatory” or “law”. The study selection was based on the following inclusion criteria: the language of publication (European, precisely English, French, Spanish, Portuguese, Italian, Russian), the year of publication (2020 and 2021), the type of published materials (journal articles), the affiliation of most of the authors and the field study (in Europe). In total, 150 bibliographic titles and abstracts were reviewed, and 38 articles were included in the analysis. Selected evidence was then analysed through a quality assessment, by focusing on research methodology, field study and results. Digital solutions and innovations chosen were considered relevant to the European scenario, even based on the experience of WHO Regional Office for Europe in the fields of Infodemic Management during the COVID-19 pandemic. In the Annex, the bibliographic search results are presented in detail. The sources used to inform this brief are either cited throughout the text and listed in the references or are listed by keyword in the “Additional reading” section. Additional sources were used, that were not identified in the review, in order to provide a fuller picture on some under-investigate topics. Furthermore, selected grey literature was also included, especially governmental documents, newspaper articles, and reports published by relevant stakeholders (i.e. the European Centre for Disease Prevention and Control (ECDC); the Inter-Agency Standing Committee; the European Commission; and the United Nations Educational, Scientific and Cultural Organization (UNESCO)). 2. Methodology Digital solutions to health risks raised by the COVID-19 infodemic 2 3. Health risks and harms due to the COVID-19 infodemic The COVID-19 infodemic has made it more difficult for health institutions to provide the public with scientifically sound information, as an overabundance of information, disinformation, misinformation and rumours are spreading continuously on the Internet (23). Consequently, users exposed to inaccurate information have in certain cases exhibited unhealthy and risky behaviour, both for themselves and for their communities (24). For this reason, the ongoing infodemic constitutes one of the main challenges for governments in delivering a timely and effective COVID-19 response. The main impacts on population behaviour include the uptake of unproven and potentially harmful forms of treatment; neglect of personal protective measures, such as physical distancing and mask-wearing; and increased vaccine hesitancy. In addition to physical harms, the psychological effects of infodemic risks were also observed, and included adverse effects on the emotional well-being of individuals, an increased incidence of mental disorders and increased distrust towards the health system. Among the main cognitive and behavioural contributors to the infodemic were excessive social media usage and low digital health literacy (25). In this section, the policy brief explores some examples of cyber harms through a health-risk lens, thus focusing on physical and psychological harms caused by the COVID-19 infodemic. Adopting wrong or harmful treatments • Media reports on the use of some drugs as possible treatments for COVID-19 resulted in the spread of misinformation with regards to their usage. This led to an increased demand for these products, resulting in shortages, and increased incidence of their misuse due to self-medication, which was associated with harm and in some cases death (26). • More than 100 cases of consuming disinfectant as a possible treatment for COVID-19 were reported by the state of Maryland in the United States of America (27). • The United States National Poison Data System (NPDS) reported a 245% increase in self-medication with Ivermectin to treat COVID-19 between July and August 2021, in the United States, because of false information spread online (28). Reducing uptake of protective behaviours and vaccination • Studies show how COVID-19 misinformation is among the main causes of a delayed outbreak response due to the slowed adoption of preventive measures (29). • Higher exposure to online misinformation has been observed to be associated with vaccine hesitancy, which has contributed to lower vaccination rates for COVID-19 and other childhood vaccine-preventable diseases (30). • Conspiracy theories and rumour acceptance were related to lower willingness to respect protective behaviours, especially physical distancing, mask-wearing and vaccinations (31). • Deliberate exposure to COVID-19 to induce immunity has been shown to lead to severe illness and even death (32). Affecting mental health and emotional well-being • Persistent fears, worries, uncertainties and stress provoked by misinformation may have long-term effects on the population, especially the most vulnerable members of the community (33). 3 Digital solutions to health risks raised by the COVID-19 infodemic • Higher levels of anxiety were seen in subjects who believed in COVID-19 related conspiracy theories – including those concerning biological warfare and 5G networks – which were nourished by unreliable sources. Symptoms of anxiety, depression and post-traumatic stress disorder were associated with exposure to misinforming messages (29). • Some researchers suggest misinformation to be one of the contributing factors to suicide risk, due to its effects on mental health in high-risk and vulnerable groups (34). Decreasing trust in the health system • Conspiracy theories, rumours and misinformation diffused through social media and messaging applications such as WhatsApp, Viber and Telegram have increased mistrust in health workers and institutions, resulting in episodes of violence against medical staff (35). • Distrust towards experts – fuelled by misinformation and distortion of medical statements – led some communities to self-medication, fear and misuse of pharmaceuticals (36). • Health-care workers of the Red Cross in Paris highlighted how the rumours spread on social media platforms undermined the perception of public health authorities and reduced the effectiveness of response strategies. Also, the health workers reported feelings of anxiety and uncertainty due to the huge amount of contradictory information available online and offline (37). © WHO/Blink Media – Amanda Mustard Digital solutions to health risks raised by the COVID-19 infodemic 4 4. Digital solutions to health risks raised by misinformation Since social media and the Internet are an important forum for the dissemination of overabundant, false and misleading information regarding COVID-19, they are also key spaces where solutions must be found and implemented. Several digital innovations have been developed, and previous methods strengthened, to respond to the infodemic. The nature of these solutions is not only technological, but also political, regulatory, educational and communicative. Effective application of these digital solutions requires the involvement of relevant stakeholders in formal or informal COVID-19 infodemic management mechanisms. Digital infodemic interventions include: • the implementation of fact-checking and false information reporting mechanisms; • the adoption of social listening tools augmented by artificial intelligence (AI); • the creation of monitoring programmes, multi-stakeholders’ coordination initiatives and national regulatory frameworks which respect the principles of freedom of expression; • the promotion of digital health literacy and inoculation interventions that improve people’s ability to spot misinformation. Implementing fact-checking and false information reporting mechanisms • One of the most popular interventions to counter misinformation has been fact-checking. In this field, evidence shows how warning labels may reduce the intention to share and the perceived credibility of fake news. Nevertheless, fact-checking effectiveness may be influenced by news formats, topics, sources, and users’ political orientation and prior knowledge (10). • A major challenge is that the pace of the careful work done by fact-checkers cannot keep up with the speed of misinformation spread (38), the amount of which is amplified using social bots (4). • This well-known phenomenon became particularly critical with the enormous amount of COVID-19 misinformation that made it arduous for fact-checkers to respond to false claims (39). • In response, automated fact-checking bots are being developed to aid the work of human fact-checkers, supporting them in accelerating responses, diffusing fact-checks, and identifying fake news or its patterns (40). • Reporting mechanisms for COVID-19 misinformation have been updated by the main social media platforms to respond to the pandemic (11). For example, social media platforms have developed proprietary algorithms to identify false information, while also relying on reports from users and fact- checking organizations in order to remove or hide false information. Examples of fact-checking and reporting mechanisms • Fact-checking organizations are offering their services to verify the truth of the information. Some of the most well-known are AFP Fact Check, FactCheck.org, Full Fact, NewsGuard, Poynter and Snopes. For example, Heath Feedback, a fact-checking organization, has created a worldwide network of scientists and health experts which help sort fact from fiction (41). • Also, web-based platforms for fact-checking are being implemented, such as Logically, BoomLive, Lead Stories, Newschecker and PolitiFact. Several of them include a specific section for COVID-19 related fact-checks. 5 Digital solutions to health risks raised by the COVID-19 infodemic • The main social media and Internet companies proposed tools for fact-checking and reporting, like those of Google, Twitter and Facebook. The Google fact-checking tools consist of an Explorer and a Markup Tool with the purpose of verifying the truth of online information. The Twitter COVID-19 hub was visited by more than 160 million users between June 2020 and June 2021. Since the beginning of the pandemic, 43 010 tweets have been removed (11). In addition, Meta, the company behind Facebook, launched the Third-Party Fact-Checking Program (42). Adopting social listening tools augmented by AI • Social listening tools allow the gathering of information distributed on online platforms to capture and analyse keywords and sentiments with regards to COVID-19, and thus assess the volume, velocity and variety of false information and rumours (12). • Natural language processing, the branch of AI that interprets human language, has been shown to be helpful in the automated detection of misinformation (43). • AI solutions to combat false information require a multidisciplinary approach, using methods from different scientific fields: sociology (complex networks), computer science and linguistics (text mining), mathematics and statistics (machine learning) and physics (agent-based modelling) (38). Examples of social listening tools augmented by AI • Social listening tools are offered by CrowdTangle, Google Alerts, Social Mention, Talkwalker Alerts and TweetDeck, among others – with some features provided cost free and some premium features. Additionally, some tools include the analysis of social media contents, for example Awario, Cision, CrowdTangle, Social Mention, Talkwalker and TweetDeck (12). • Bot Sentinel is a tool that identifies Twitter bots and accounts sharing unreliable contents, by means of AI and machine learning. Another example of a machine learning browser extension is Trusted Times, which classifies fake news and untrustworthy content (44). • The EARS platform (Early AI-assisted Response with Social Listening) was developed by WHO to allow health decision-makers to view a real-time analysis of narratives happening in public online forums in multiple countries and languages (45). Creating monitoring programmes, multi-stakeholders’ coordination initiatives and national regulatory frameworks which respect the principles of freedom of expression • Technological innovations which have an impact on the health of citizens need to be supported by appropriate policy approaches. For example, evidence suggests that national policies may be enforced through good-quality statistics and precise definitions of indicators. Such an effort of national coordination is particularly necessary during an infodemic when the measurement scenario is more uncertain (46). • The various stakeholders involved in infodemic response need to be coordinated through specific mechanisms aimed at allowing intersectoral interaction (14). The rapidity and efficacy of multistakeholder dialogues with online digital platforms in the European Union nevertheless may be strongly impacted by the degree of state involvement (47,48). • National regulatory frameworks which tackle the spread of false information need to be built with utmost respect of freedom of speech and media pluralism. When taking measures to protect their populations from misinformation, states must not unduly infringe on human rights, particularly the freedom of expression (15). • At the same time, self-regulatory monitoring programmes need to be based on principles of transparency in reporting harmful online content. Transparency standards help to ensure that social media platforms provide accurate reports of the size and scope of the infodemic challenge and mitigation measures, instead of underreporting (13). Digital solutions to health risks raised by the COVID-19 infodemic 6 Examples of monitoring programmes, multi- stakeholders’ coordination initiatives and national regulatory frameworks • The COVID-19 disinformation monitoring programme is orchestrated by European Union institutions to encourage social media platforms to share achievements in reporting mechanisms and other initiatives aimed at managing the infodemic (11). • The Online Safety Bill in the United Kingdom represents a positive advance, which includes – for the first time – mis- and disinformation in a national regulation. Through it, the Government of the United Kingdom recommends the following safety-by-design principles for web platforms: 1) platforms should take preventative steps to make sure their service reduces users’ exposure to harm; 2) platforms should consider all types of users when assessing risks; 3) platforms should give users the tools and information they need to make safer choices online; and 4) platforms must be designed to keep children safe (16). • Most adopted legislation considers the perpetrator – either as a person or as a media entity – liable for spreading false information, whereas the German Network Enforcement Act also holds online platforms liable for removing or moderating harmful content. This also implies the implementation of a monitoring programme on reporting of content flagged by users (17). • Other legislative proposals tackle misinformation/ disinformation by also highlighting the responsibilities of web companies. They are, for example, France’s Fight against Online Hate Speech Law, Ireland’s Proposal to Regulate Transparency of Online Political Advertising and Israel’s Proposed Electoral Law Amendments and “Facebook Laws” (15). Promoting digital health literacy and initiatives aimed at inoculating against misinformation • One of the main strategies to counter online misinformation has been the promotion of media and information literacy, with the purpose of bolstering prevention and reducing the appeal of disinformation and conspiracy theories (49). • More specific approaches to tackling COVID-19 related misinformation include introducing educational modules on digital health literacy from a younger age. Digital health literacy has been seen to improve individuals’ comprehension of public health information, increase healthy behaviours, reduce harm, promote protective attitudes towards others and themselves, assist in identifying rumours and misinformation on social media, and decrease vaccine hesitancy and conspiracy theories, while promoting informed health decisions (50). • News literacy programs may be aimed at empowering users’ ability to recognize and share correct information and encouraging them to produce verified news. Consequently, information professionals should be encouraged to build international ethical codes and educate users to recognize verified information (51). • Some perspectives use inoculation theory to develop “psychological vaccines” – interventions such as games and videos aimed at improving people’s ability to spot manipulation techniques commonly used with COVID-19 misinformation (52). • Another approach uses campaigns against the infodemic, such as the United Nation’s Verified, which was launched globally in April 2020 to address the infodemic around COVID-19 (53). Examples of digital health literacy and initiatives aimed at inoculating against misinformation • Media and information literacy packages are made available by UNESCO to counter disinformation, fight discrimination and promote best practices (54). • With regards to news literacy, an example of its application is NewsGuard, a Microsoft-sponsored program which rates the trustworthiness of websites through an extension of the Internet browser Edge (55). • Inoculation games and videos have also been released, like GoViral, a game developed by the University of Cambridge, the media agency DROG and the United Kingdom’s Cabinet Office (56). Digital solutions to health risks raised by the COVID-19 infodemic7 The WHO Regional Office for Europe developed information resources and risk communication interventions intended to provide trustworthy health information, identify infodemic risks and analyse the response landscape. Through risk communication and community engagement (RCCE), the Regional Office has focused on building and maintaining bidirectional trust between at-risk and affected communities and emergency response authorities. During the COVID-19 outbreak, infodemic management has been embedded in RCCE (9). The priority has been to ensure that affected populations have timely access to quality and relevant health information, and that misinformation is properly tackled. To this end, concrete actions have been undertaken by WHO to manage the infodemic in the European Region, as described in section 6 in this policy brief. © WHO/ NOOR / Arko Datto Digital solutions to health risks raised by the COVID-19 infodemic 8 5. Policy considerations This policy brief highlights the need for specific improvements and developments that call upon stakeholders in the European Region to cooperate in infodemic management. To this end, the Regional Office highlights the six policy considerations described here below. 1. Reinforcing multistakeholder networks for infodemic management. Infodemic management networks can contribute to building bi-directional trust between people and institutions, through the amplification of positive health messages by trusted influencers. 2. Strengthening overall risk communication and community engagement. RCCE is a public health intervention essential to emergency response and critical to risk management. Political and financial commitment are vital to develop RCCE capacity at the level needed to support infodemic response. 3. Implementing continuous monitoring of online harmful and false content. Member States can create a process for routinely examining infodemic signals (information voids, overabundant or false information, and rumours) in the main social media channels and other digital and online forums in their country. 4. Improving digital literacy approaches and organizing infodemic management trainings. Infodemics can be prevented or mitigated by redesigning, enhancing and organizing initiatives to improve digital health literacy, critical thinking and infodemic management, with the purpose of increasing resilience to false information. 5. Advocating for infodemic management through communication campaigns. Social mobilization can help empower citizens and engage them to contribute to building a safer online public sphere, for example by flagging and reporting false information. 6. Ensuring safe online platforms, which protect people from harmful content. Infodemic management may involve policy changes that nudge digital platforms to stop the spread of misinformation through regional or national regulatory frameworks which promote transparency and external oversight of data and algorithms, with the aim of ensuring users’ safety, health and well-being, while respecting freedom of expression. 9 Digital solutions to health risks raised by the COVID-19 infodemic 6. What WHO is doing to address the infodemic in the European Region and globally 6.1 Digital communication strategies for infodemic management The WHO Regional Office for Europe is making an unprecedented effort to inform stakeholders about the most recent scientific updates. Via social media, web platforms and traditional media, verified information is disseminated in all countries, and misinformation is directly tackled, by means of weekly news and daily rollouts. Examples of these strategies are the weekly collection of social and news media, web analytics and online search data with regards to the COVID-19 infodemic, as well as the visual network analyses showing misinformation ecosystems. The Regional Office COVID-19 Incident Management Support Team (IMST) Communication Pillar has developed an implementation guidance on advancing infodemic management within RCCE in the European Region, which is in the process of being published. Additionally, WHO has launched digital campaigns to promote accurate health information (57), such as the one on handwashing; other campaigns include #doitall (58), #summersense (59), #safehands (60), #healthyathome (61) and #thankshealthheros (62). Moreover, WHO is publishing myth busters (63), live Q&A interviews with experts, and an Infodemic Management News Flash, which informs readers about the most recent activities and events on a global level, and the main challenges and solutions highlighted locally (64). Additionally, the Regional Office publishes a newsletter that rounds up the latest updates on COVID-19 every two weeks; it includes guidance, interviews, tweets, epidemiological updates and country stories from WHO channels. A weekly update on COVID-19 data and news is provided through Flu News Europe, which was born from the collaboration between the Regional Office and ECDC (65). Press conferences and live media briefings on COVID-19 are run in English and Russian; here, media from across the Region find their answers to an average of 350 media queries per week. This stream of regional risk communication is supported by country-focused communications. 6.2 Mobile apps disseminating accurate health information The Regional Office partnered with the United Nations Children’s Fund (UNICEF) in Europe and central Asia in April 2020 to make a freely available interactive COVID-19 chatbot, called HealthBuddy+, which provides expert advice through AI. The chatbot is available in 20 regional languages, and is accessible on the Internet, mobile applications (iOS/Android), and messaging platforms (Telegram, VKontakte, WhatsApp and Facebook Messenger). The digital solution provides evidence-based content and health advice on COVID-19, and links users to country-specific information. The mobile app also features an interactive poll, where users can voice their thoughts on key elements of the pandemic (18). Embedded in 15 partner platforms, HealthBuddy+ reached about 4 million individuals by the end of 2021. Globally, WHO developed two mobile apps, respectively named WHO Info and WHO Academy, with the purpose of diffusing trustworthy pieces of advice, tips and scientific knowledge. WHO Info publishes health information for the public in six languages, in the form of news, feature stories, fact sheets, disease outbreak updates and public health emergency information. WHO Academy has been specifically created for the COVID-19 outbreak and is aimed at informing health workers by providing them with online guidance, tools, training and workshops (66). Digital solutions to health risks raised by the COVID-19 infodemic 10 Digital solutions to health risks raised by the COVID-19 infodemic 6.3 AI-based innovations for rumour tracking, community feedback and social listening HealthBuddy+ also allows rumour tracking and community feedback through AI. Since its launch, nearly 800 reports in 16 languages have been collected (67). The rumour reporting function helps shed light on emerging and circulating rumours in the Region, which are analysed by language and then across the entire Region. The Regional Office has also used the AI-based news gathering, monitoring and analysis tool uCrawler to monitor fact-checking websites across the Region, and the social listening tool CrowdTangle to help monitor and assess infodemic signals. Globally, WHO established a social listening tool, called EARS (Early AI-supported Response with Social Listening), based on AI, to identify emerging health narratives and “information voids” to help inform health decisions (68). Finally, WHO refined an infodemic observatory to assess the spread of misinformation and disinformation, through AI, data science and network science techniques. 6.4 Training, events and conferences to promote infodemic management The Regional Office has mobilized an Infodemic Management Team to help Membe States detect, assess, analyse and respond to infodemic risks. To this end, national authorities and infodemic responders can benefit from a three- module training on infodemic management. The Leadership module equips national health authorities and partner organizations at the managerial and policy-making levels with necessary infodemic background and key technical skills to engage and manage national and subnational infodemic management stakeholders operationally. The Response module provides infodemic responders with advanced technical skills and knowledge of tools that will enable them to better understand, respond to and recover from infodemics. Finally, the Coordination module facilitates network building and planning between, on the one hand, national health authorities, WHO country offices and partner organizations and, on the other, fact- checkers, misinformation and disinformation researchers, journalists and other stakeholders involved in infodemic management. Globally, WHO boosted international dialogue through events, conferences and trainings. Three WHO infodemic management conferences have been organized, where intersectoral dialogue on misinformation between all stakeholders involved was encouraged (69). WHO, together with the academic sector, has developed a public health research agenda for infodemic management, and the new discipline of infodemiology has been strengthened. The WHO Information Network for Epidemics (EPI-WIN) has organized several training activities on infodemic management. 6.5 Behavioural insights surveys, better rumour tracking mechanisms, multi-language fact- checking The Regional Office collaborated with a community of 2000 young people in the European Region for message outreach, rumour tracking and community-feedback initiatives; and with religious leaders from diverse faiths to debunk vaccines myths and increase vaccination uptake (70). The Regional Office also created a behavioural insights methodological tool for monitoring public knowledge, risk perceptions, behaviours and trust in a rapid, flexible and cost-effective way, and has conducted surveys in many countries in the Region throughout the pandemic. One of the goals of the tool was to highlight potential risks of misinformation and distrust, to better enable early response (71). Furthermore, WHO is creating a repository of about 200 active fact-checking organizations aimed at verifying COVID-19 false facts in more than 40 languages (72). 6.6 Partnering with social media and web companies The regional COVID-19 response landscape is shaped by different stakeholders, in particular government counter-misinformation teams; United Nations agencies and international partners (the International Federation of Red Cross and Red Crescent Societies (IFRC), the United Nations Development Programme (UNDP), UNICEF and WHO); media and journalist groups; community response groups; and influencers and organizations dedicated to fact-checking, disinformation analysis and response. The Regional Office enhanced partnerships with countries, other United Nations agencies, the tech industry, the media system and civil society, and supported national partners in building country tools and developing training for infodemic response. 11 At a global level, several infodemic management initiatives have seen WHO partnering with digital platforms, such as Google, Twitter, Facebook, TikTok, LinkedIn, WhatsApp and YouTube. An example of these initiatives of cooperation to respond to the infodemic is the Google SOS alert, which makes scientific health information more available and visible on Google channels for content search and mapping (73). Another initiative has been the WhatsApp service WHO Health alert, which provides updated health information to users in seven languages (74). Moreover, a devoted section of the WHO website encourages users to use the reporting systems of social media platforms to report false content directly (75). © WHO/Frank Henriksen Digital solutions to health risks raised by the COVID-19 infodemic 12 7. Conclusions The health impact of the COVID-19 pandemic has been exacerbated by the “infodemic”: an overabundance of information, misinformation, disinformation and rumours, spread especially through social media platforms. Harmful effects caused or exacerbated by the infodemic include adoption of wrong or harmful treatments, reduction in uptake of protective behaviours and vaccination, negative impacts on mental health and emotional well-being, and increased distrust towards the health system. To respond to the increasing amount of false or misleading information available online, this policy brief provides an overview of available evidence retrieved through the WHO COVID-19 Research Database and other relevant sources to identify digital solutions to help mitigate the impact of the COVID-19 infodemic. Based on the evidence presented in this brief, viable operational considerations for action by Member States include: implementing fact-checking and false information reporting mechanisms; adopting social listening tools enhanced by artificial intelligence; creating monitoring programmes, multi-stakeholders’ coordination initiatives and national regulatory frameworks which respect the principles of freedom of expression; and promoting digital health literacy and inoculation interventions that improve people’s ability to spot misinformation. The brief also proposes a list of policy considerations for all stakeholders, including: reinforcing multistakeholder networks for infodemic management; strengthening overall risk communication and community engagement; implementing continuous monitoring of online harmful and false content; improving digital literacy approaches and organizing infodemic management trainings; advocating for infodemic management through communication campaigns; and encouraging policy changes to make online platforms safer for users. In section 6, the brief outlines how the WHO Regional Office for Europe is shaping its own role in bringing to life the above operational and policy considerations, in support of and close coordination with Member States and other subnational and international governmental and nongovernmental stakeholders. Concrete examples of these efforts include digital communication strategies; mobile apps disseminating accurate health information; AI- based innovations for rumour tracking, community feedback and social listening; training, events and conferences to promote infodemic management; behavioural insights surveys, better rumour tracking mechanisms and multi-language fact- checking; and partnerships with social media and web companies. With this publication, another valuable tool is added to this list. The implementation of the digital solutions and policy considerations above calls for a whole- of-society approach with the collaboration of all stakeholders involved in infodemic management, ranging from users themselves to social media platforms, the information technology sector, health policy-makers, and fact-checking and civil society organizations, among others – with a shared objective of improving the Region’s public health response to the COVID-19 infodemic and enhancing preparedness for future health emergencies. It also contributes to the European Programme of Work 2020–2025 – “United Action for Better Health in Europe”, and the call to action in World Health Assembly Resolution 73.1 for WHO and its Member States to address the proliferation of disinformation and misinformation. Although this policy brief focuses on digital solutions to infodemics, it also underscores the importance of complementing online approaches with offline measures. This is especially true for further implementation and improvement of already known and tested risk communication and community engagement best practices. 13 Digital solutions to health risks raised by the COVID-19 infodemic References 1. Jakovljevic M, Bjedov S, Mustac F, Jakovljevic I. COVID-19 Infodemic and public trust from the perspective of public and global mental health. Psychiatr Danub. 2020;32(3–4):449–57. doi: 10.24869/psyd.2020.449. 2. Infodemic. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/health-topics/ infodemic#tab=tab_1, accessed 3 February 2022). 3. Cuan-Baltazar JY, Muñoz-Perez MJ, Robledo-Vega C, Pérez-Zepeda MF, Soto-Vega E. Misinformation of COVID-19 on the Internet: Infodemiology study. JMIR Public Health Surveill. 2020;6(2):e18444. doi: 10.2196/18444. 4. Gallotti R, Valle F, Castaldo N, Sacco P, De Domenico M. Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics. Nat Hum Behav. 2020;4:1285–93. doi: 10.1038/s41562-020-00994-6. 5. Love JS, Blumenberg A, Horowitz Z. The parallel pandemic: medical misinformation and COVID-19: Primum non nocere. J Gen Intern Med. 2020;35(8):2435–36. doi: 10.1007/ s11606-020-05897-w. 6. Garett R, Young SD. Online misinformation and vaccine hesitancy. Transl Behav Med. 2021;11(12):2194–99. doi: 10.1093/tbm/ibab128. 7. Gao J, Zheng P, Jia Y, Chen H, Mao Y, Chen S et al. Mental health problems and social media exposure during COVID-19 outbreak. PLoS One. 2020;15(4):e0231924. doi: 10.1371/journal. pone.0231924. 8. Gruzd A, Mai P. Going viral: how a single tweet spawned a COVID-19 conspiracy theory on Twitter. Big Data Soc. 2020;7(2). doi: 10.1177/2053951720938405. 9. Risk communication. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/teams/ risk-communication, accessed 3 February 2022). 10. Edwards L, Stoilova M, Anstead N, Fry A, El- Halaby G, Smith M. Rapid evidence assessment on online misinformation and media literacy: final report for Ofcom. London: London School of Economics and Political Science; 2021 (http://eprints.lse.ac.uk/id/eprint/110866, accessed 3 February 2022). 11. Reports on June Actions – Fighting COVID-19 Disinformation Monitoring Programme. In: European Commission/Shaping Europe’s digital future [website]. Brussels: European Commission; 2021 (https://digital-strategy. ec.europa.eu/en/library/reports-june-actions- fighting-covid-19-disinformation-monitoring- programme, accessed 3 February 2022). 12. Social Listening and Monitoring Tools. COVID-19 Vaccine Confidence: Rapid Community Assessment Tool. Atlanta: Centers for Disease Control and Prevention; 2021 (https://www.cdc.gov/vaccines/covid-19/ vaccinate-with-confidence/rca-guide/ downloads/CDC_RCA_Guide_2021_Tools_ AppendixE_SocialListening-Monitoring- Tools-508.pdf, accessed 3 February 2022). 13. Pollock S. Facebook fined €2m for under- reporting its complaints figures. The Times. 7 July 2019 (https://www.thetimes.co.uk/article/ facebook-fined-2m-for-under-reporting-its- complaints-figures-7fhg26dct, accessed 4 February 2022). Digital solutions to health risks raised by the COVID-19 infodemic 14 14. Kolotaev Y. Европа и Европейский союз в контексте информационных угроз постковидного мира (Europe and the European Union in the Context of the Information Threats of a Post-Covid World). Sovremennaya Evropa. 107(7):143-151, 2021. 15. Milanovic M, Schmitt MN. Cyber attacks and cyber (mis)information operations during a pandemic. Journal of National Security Law & Policy. 2020;11:247. doi: 10.2139/ssrn.3612019. 16. Guidance. Principles of safer online platform design. In: Government of the United Kingdom [website]. London: Government of the United Kingdom; 2021 (https://www.gov.uk/guidance/ principles-of-safer-online-platform-design, accessed 4 February 2022). 17. Balancing act: countering digital disinformation while respecting freedom of expression. Broadband Commission research report on “Freedom of expression and addressing disinformation on the Internet”. Geneva: Broadband Commission for Sustainable Development; 2020 (https://www.broadbandcommission.org/ wp-content/uploads/2021/02/WGFoEDisinfo _Report2020.pdf, accessed 4 February 2022). 18. HealthBuddy+ [website]. HealthBuddy+; 2022 (https://healthbuddy.plus/index, accessed 4 February 2022). 19. UN tackles “infodemic” of misinformation and cybercrime in COVID-19 crisis. In: United Nations [website]. New York: United Nations; 2020 (https://www.un.org/en/un- coronavirus-communications-team/un- tackling-%E2%80%98infodemic%E2%80%99- misinformation-and-cybercrime-covid-19, accessed 3 February 2022). 20. Guljaš S, Bosnić Z, Salha T, Berecki M, Krivdić Dupan Z, Rudan S et al. Lack of informations about COVID-19 vaccine: from implications to intervention for supporting public health communications in COVID-19 pandemic. Int J Environ Res Public Health. 2021;18(11):6141. doi: 10.3390/ijerph18116141. 21. Health Literacy. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/teams/health- promotion/enhanced-wellbeing/ninth-global- conference/health-literacy, accessed 11 February 2022). 22. WHO COVID-19 Research Database [online database]. Geneva: World Health Organization; 2022 (https://search.bvsalud.org/global- literature-on-novel-coronavirus-2019-ncov, accessed 3 February 2022). 23. Mheidly N, Fares J. Leveraging media and health communication strategies to overcome the COVID-19 infodemic. J Public Health Policy. 2020;41(4):410–20. doi: 10.1057/s41271-020- 00247-w. 24. Zhang Y, Tambo E, Djuikoue IC, Tazemda GK, Fotsing MF, Zhou XN. Early-stage risk communication and community engagement (RCCE) strategies and measures against the coronavirus disease 2019 (COVID-19) pandemic crisis. Global Health Journal. 2021;5(1):44–50. doi: 10.1016/j.glohj.2021.02.009. 25. Pian W, Chi J, Ma F. The causes, impacts and countermeasures of COVID-19 “Infodemic”: a systematic review using narrative synthesis. Inf Process Manag. 2021;58(6):102713. doi: 10.1016/j.ipm.2021.102713. 26. Mallhi TH, Khan YH, Alotaibi NH, Alzarea AI, Alanazi AS, Qasim S et al. Drug repurposing for COVID-19: a potential threat of self-medication and controlling measures. Postgrad Med J. 2021;97(1153):742–43. doi: 10.1136/ postgradmedj-2020-138447. 27. Winsor M, Mansell W, Ella T. Maryland agency receives more than 100 disinfectant use calls. ABC News. 25 April 2020 (https:// abcnews.go.com/Health/coronavirus-updates- china-reports-covid-19-deaths-%209th/ story?id=70324535, accessed 3 February 2022). 28. Romo V. Poison control centers are fielding a surge of Ivermectin overdose calls. NPR. 4 September 2021 (https:// www.npr.org/sections/coronavirus-live- updates/2021/09/04/1034217306/ivermectin- overdose-exposure-cases-poison-control- centers, accessed 3 February 2022). 29. Magarini FM, Pinelli M, Sinisi A, Ferrari S, De Fazio GL, Galeazzi GM. Irrational beliefs about COVID-19: a scoping review. Int J Environ Res Public Health. 2021;18(19):9839. doi: 10.3390/ ijerph18199839. Digital solutions to health risks raised by the COVID-19 infodemic15 30. Garett R, Young SD. Online misinformation and vaccine hesitancy. Transl Behav Med. 2021;11(12):2194-99. doi: 10.1093/tbm/ibab128. 31. Pian W, Chi J, Ma F. The causes, impacts and countermeasures of COVID-19 “Infodemic”: a systematic review using narrative synthesis.  Inf Process Manag. 2021;58(6):102713. doi: 10.1016/j.ipm.2021.102713. 32. LaMotte S. 5 reasons you should not deliberately catch Omicron to ‘get it over with’. CNN. 11 January 2022 (https://edition.cnn. com/2022/01/11/health/deliberate-omicron- infection-wellness/index.html, accessed 3 February 2022). 33. Addressing mental health and psychosocial aspects of COVID-19 outbreak. Interim Briefing Note. Geneva: Interagency Standing Committee, Reference Group on Mental Health and Psychosocial Support in Emergency Settings; 2020 (https://interagencystandingcommittee.org/ iasc-reference-group-mental-health-and- psychosocial-support-emergency-settings/ interim-briefing-note-addressing-mental- health-and-psychosocial-aspects-covid-19- outbreak, accessed 3 February 2022). 34. Que J, Yuan K, Gong Y, Meng S, Bao Y, Lu L. Raising awareness of suicide prevention during the COVID-19 pandemic. Neuropsychopharmacol Rep. 2020;40(4):392– 95. doi: 10.1002/npr2.12141. 35. Taylor L. COVID-19 misinformation sparks threats and violence against doctors in Latin America. BMJ. 2020;370:m3088. doi: 10.1136/ bmj.m3088. 36. Hashmi FK, Atif N, Malik UR, Riboua Z, Saleem F, Khan YH et al. Misinformation in wake of the COVID-19 outbreak: fueling shortage and misuse of lifesaving drugs in Pakistan. Disaster Med Public Health Prep. 2021;15(4):e3–e33. doi: 10.1017/dmp.2020.400. 37. Heyerdahl LW, Lana B, Giles-Vernick T. The impact of the online COVID-19 infodemic on French Red Cross actors’ field engagement and protective behaviors: mixed methods study. JMIR Infodemiology. 2021;1(1):e27472. doi: 10.2196/27472. 38. Gradoń KT, Hołyst JA, Moy WR, Sienkiewicz J, Suchecki K. Countering misinformation: a multidisciplinary approach. Big Data Soc. 2021;8(1). doi: 10.1177/20539517211013848. 39. Gruzd A, De Domenico M, Sacco PL, Briand S. Studying the COVID-19 infodemic at scale. Big Data Soc. 2021;8(1):1–6. doi: 10.1177/20539517211021115. 40. Mantas H. Fact-checkers use automation to maximize their impact. In: Poynter [website]. Florida: Poynter; 2021 (https://www.poynter. org/fact-checking/2021/fact-checkers-use- automation-to-maximize-their-impact/, accessed 3 February 2022). 41. Health Feedback [website]. Health Feedback; 2022 (https://healthfeedback.org/, accessed 3 February 2022).  42. Meta’s Third-Party Fact-Checking Program. In: Meta Journalism Project [website]. Meta; 2022 (https://www.facebook.com/ journalismproject/programs/third-party-fact- checking, accessed 3 February 2022). 43. Chen Q, Leaman R, Allot A, Luo L, Wei CH, Yan S et al. Artificial intelligence in action: addressing the COVID-19 pandemic with natural language processing. Annu Rev Biomed Data Sci 4. doi: 10.1146/annurev- biodatasci-021821-061045. 44. Tools That Fight Disinformation Online [online database]. Santa Monica: RAND Corporation; 2021 (https://www.rand.org/research/projects/ truth-decay/fighting-disinformation/search. html, accessed 3 February 2022). 45. Mahajan A, Czerniak C, Lamichhane J, Phuong L, Purnat T, Nguyen T et al. Advances in real- time social listening for an adaptive public health response: WHO’s EARS platform. Eur J Public Health. 2021;31(3). doi: 10.1093/eurpub/ ckab164.501 46. Antolini F, Cesarini S. COVID-19 and possible bias in statistical information: pandemic indicators and the risk of infodemic. Statistica Applicata. 2021;33(1):7–42 (https://www.sa-ijas. org/ojs/index.php/sa-ijas/article/view/33-1/86, accessed 3 February 2022). Digital solutions to health risks raised by the COVID-19 infodemic 16 47. Викторовна, Плотичкина Ната. Public Policy on COVID-19 Infodemic Management. Society: Politics, Economics, Law. (2):43-47, 2021. (Article in Russian) 48. Pivovarova O A, Gorbatov S J, Korotkova E O, Kamynina N N. International aspects of the COVID-19 pandemic. Vestinik Rossiiskoi Akademii Meditsinskikh Nauk. 76(5 S):572- 580, 2021. (Article in Russian) 49. High Level Expert Group (HLEG) on Fake News and Online Disinformation. A multi- dimensional approach to disinformation. Brussels: European Commission; 2018 (https://op.europa.eu/en/publication-detail/ -/publication/6ef4df8b-4cea-11e8-be1d- 01aa75ed71a1/language-en, accessed 4 February 2021). 50. 2.L. Workshop: Health literacy in childhood and adolescence (HLCA): A focus on health literacy in schools. Eur J Public Health. 2021;31(3). doi: 10.1093/eurpub/ckab164.140. 51. Llamas N M. Infodemic and informational dependence: the ethical function of Andalusian public television during the COVID-19 crisis. Comunicacion y Hombre. (17):119-138, 2021. (Article in Spanish) 52. Basol M, Roozenbeek J, Berriche M, Uenal F, McClanahan W, van der Linden S. Towards psychological herd immunity: cross-cultural evidence for two prebunking interventions against COVID-19 misinformation. Big Data Soc. 2021;8(1). doi: 10.1177/20539517211013868. 53. Verified [website]. Verified; 2022 (https:// shareverified.com, accessed 4 February 2022). 54. Visual resources to download and share. In: UNESCO [website]. Paris: United Nations Educational, Scientific and Cultural Organization; 2021 (https://en.unesco.org/ covid19/communicationinformationresponse/ visualresources, accessed 16 December 2021). 55. NewsGuard [website]. NewsGuard; 2021 (https://www.newsguardtech.com/, accessed 4 February 2022). 56. GoViralGame [website]. Go Viral; 2021 (www.goviralgame.com, accessed 4 February 2022). 57. Connecting the world to combat coronavirus. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/campaigns/connecting- the-world-to-combat-coronavirus, accessed 4 February 2022). 58. Join the WHO/Europe #DoItAll campaign. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.euro.who.int/en/health-topics/ health-emergencies/coronavirus-covid-19/ summersense-campaign/join-the-whoeurope- doitall-campaign, accessed 4 February 2022). 59. #SummerSense campaign. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.euro.who. int/en/health-topics/health-emergencies/ coronavirus-covid-19/summersense-campaign, accessed 4 February 2022). 60. #SafeHands challenge. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/ campaigns/connecting-the-world-to-combat- coronavirus/safehands-challenge, accessed 4 February 2022). 61. #HealthyatHome challenge. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https:// www.who.int/campaigns/connecting-the- world-to-combat-coronavirus/safehands- challenge/healthyathome-challenge, accessed 4 February 2022). 62. #ThanksHealthHeroes challenge. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https:// www.who.int/campaigns/connecting-the- world-to-combat-coronavirus/safehands- challenge/thankshealthheroes, accessed 4 February 2022). 63. COVID-19 mythbusters. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who. int/westernpacific/emergencies/covid-19/ information/mythbusters, accessed 4 February 2022). Digital solutions to health risks raised by the COVID-19 infodemic17 64. Infodemic Management News Flash [newsletter]. Issue #28. Copenhagen: WHO Regional Office for Europe; 2021 (https://epi-win.cmail20.com/t/ViewEmail/d/ 9089AC1A19E092202540EF23F30FEDED/ ACCA9FA3A49B9FDB765E7602346EC846? alternativeLink=False, accessed 4 February 2022). 65. Flu News Europe. Joint ECDC-WHO/Europe weekly influenza update [website]. Stockholm and Copenhagen: European Centre for Disease Prevention and Control and WHO Regional Office for Europe; 2022 (https://flunewseurope. org, accessed 4 February 2022). 66. Applications In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/news-room/apps, accessed 4 February 2022). 67. HealthBuddy+. In: WHO Regional Office for Europe [website]. Copenhagen: WHO Regional Office for Europe; 2021 (https://www.euro.who. int/en/health-topics/health-emergencies/ coronavirus-covid-19/healthbuddy, accessed 4 February 2022). 68. EARS Early AI-supported Response with Social Listening [website]. Geneva: World Health Organization; 2022 (https://www.who- ears.com, accessed 4 February 2022). 69. Calleja N, AbdAllah A, Abad N, Ahmed N, Albarracin D, Altieri E et al. A public health research agenda for managing infodemics: methods and results of the first WHO Infodemiology Conference. JMIR Infodemiology. 2021;1(1):e30979. doi: 10.2196/30979. 70. COVID-19: WHO joins forces with young Global Shapers to disseminate health advice. In: WHO Regional Office for Europe [website]. Copenhagen: WHO Regional Office for Europe; 2021 (https://apps.who.int/iris/ handle/10665/333549, accessed 4 February 2022). 71. Survey Tool and Guidance. Rapid, simple, flexible behavioural insights on COVID-19. Copenhagen: WHO Regional Office for Europe; 2020 (https://apps.who.int/iris/ bitstream/handle/10665/333549, accessed 4 February 2022). 72. COVID-19 Fact Checkers Dataset. In: covid19misinfo.org [website]. Toronto: Social Media Lab; 2022 (https://covid19misinfo. org/fact-checking/covid-19-fact-checkers/, accessed 4 February 2022). 73. Google SOS Alert [website]. Google; 2022 (https://support.google.com/sosalerts/?hl=en, accessed 4 February 2022). 74. WHO Health Alert brings COVID-19 facts to billions via WhatsApp. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https://www.who.int/ news-room/feature-stories/detail/who-health- alert-brings-covid-19-facts-to-billions-via- whatsapp, accessed 4 February 2022). 75. How to report misinformation online. In: World Health Organization [website]. Geneva: World Health Organization; 2021 (https:// www.who.int/campaigns/connecting-the- world-to-combat-coronavirus/how-to-report- misinformation-online, accessed 4 February 2022). Digital solutions to health risks raised by the COVID-19 infodemic 18 The bibliographic search strategy consisted firstly in selecting sources per main keywords by identifying all sources where the terms “COVID-19” and “infodemic” appeared in the titles or abstracts within the WHO COVID-19 Research Database (22). The words “COVID-19” and “infodemic” were then searched in combination with the terms “fact- checking”, “social listening”, “artificial intelligence”, “deep learning”, “machine learning”, “literacy”, “policy”, “regulation”, “regulatory” or “law”. The study selection was based on the inclusion criteria indicated in Table A.1, along with the language of publication (European, precisely English, Spanish, Portuguese, Italian, Russian), the year of publication (2020 and 2021), the type of published materials (journal articles), the affiliations of most of the authors and the field study (in Europe). The following provides an example of how the strategy search was carried out for the search term “artificial intelligence”. • References mentioning “COVID-19” and “infodemic” in titles and abstracts and including the term “artificial intelligence” in the body text were included, producing seven in total. • All seven references that included “COVID-19” and “infodemic” in titles and abstract and “artificial intelligence” in the body text were in English. • Among them, only journal articles were retained and only when their research focus was on a European field study or most of the article’s authors were European. Two articles were therefore included. • Following the initial selection, to avoid the mistaken inclusion of the same articles in the total search results, each reference was included only once. This step was needed particularly for terms referring to close research areas, such as “artificial intelligence”, “deep learning” and “machine learning” on the one hand, and “law”, “regulatory” and “regulation” on the other. • Finally, after reading the whole-body text, only references mentioning digital solutions were included. This same procedure for inclusion was applied to all the keywords listed above, as shown in table A.1, leading to 38 articles being included in the analysis. Annex. Bibliographic search strategy 19 Digital solutions to health risks raised by the COVID-19 infodemic Table A.1. Bibliographic search results per keyword and inclusion criteria The sources used to inform this brief are either cited throughout the text as references or are listed by keywords in the “Additional reading” section. Terms Total results Language of publication (European) Type of published materials (journal articles) and either field study (European) and affiliation of most authors (European) Used only once Relevant to digital solutions Artificial intelligence 7 7 2 7 7 Deep learning 7 7 3 Machine learning 15 15 5 Fact-checking 27 24 21 8 7 Law 2 2 2 6 5 Regulatory 7 7 3 Regulation 5 5 2 Literacy 59 55 16 13 12 Policy 28 27 9 4 4 Social listening 11 9 4 4 3 Total 168 158 67 42 38 Digital solutions to health risks raised by the COVID-19 infodemic 20 Additional reading This section lists sources that informed this policy brief in identifying the main digital solutions cited in recently published literature from the WHO European Region and found in the WHO COVID-19 Research Database. As these sources have been selected through the bibliographic search criteria described in the Annex, they are ordered per main keywords and in alphabetical order. The sources are not cited in the body text nor listed in the references. Artificial intelligence, deep learning, machine learning • Bragazzi NL, Dai H, Damiani G, Behzadifar M, Martini M, Wu J. How big data and artificial intelligence can help better manage the COVID-19 pandemic. Int J Environ Res Public Health 2020;17(9):3176. doi:10.3390/ijerph17093176. • Choudrie J, Banejee S, Kotecha K, Walambe R, Karende H, Ameta J. Machine learning techniques and older adults processing of online information and misinformation: a COVID-19 study. Comput Hum Behav. 2021;119:106716. doi:10.1016/j.chb.2021.106716. • Gruber A, Ghiringhelli M, Edri O, Abboud Y, Shiti A, Shaheen N et al. Literature review and knowledge distribution during an outbreak: a methodology for managing infodemics. Acad Med. 2021;96(7):1005–9. doi:10.1097/ ACM.0000000000004073. • Jijon-Vorbeck JA, Segura-Bedmar I. Exploring the impact of COVID-19 on social life by deep learning. Information 2021;12(11):459. doi:10.3390/ info12110459. • Mazzeo V, Rapisarda A, Giuffrida G. Detection of fake news on COVID-19 on web search engines. Frontiers in Physics 2021;9. doi:10.3389/ fphy.2021.685730. • Teodoro D, Ferdowsi S, Borissov N, Kashani E, Vicente Alvarez D, Copara J. Information retrieval in an infodemic: the case of COVID-19 publications. J Med Internet Res. 2021;23(9):e30161. doi:10.2196/30161. Fact-checking • García-Marín D. Global infodemic: information disorders, false narratives, and fact checking during the COVID-19 crisis. El Profesional de la Información 2020;29(4) (in Spanish). • Kishore Shahi G, Dirkson A, Majchrzak TA. An exploratory study of COVID-19 misinformation on Twitter. Online Social Networks and Media 2021;22:100104. doi:10.1016/j.osnem.2020.100104. • Mahajan A, Phuong L, Nguyen T, Czerniak C, Lamichhane J, Purnat T et al. 50 global actions to manage the COVID-19 infodemic: a WHO framework. Eur J Public Health 2021;31(3). doi:10.1093/eurpub/ckab164.277. • 11.N. Skills building seminar: how can local public health services manage infodemics and fight misinformation? Eur J Public Health 2021;31(3). doi:10.1093/eurpub/ckab164.859. • Pulido CM, Villarejo-Carballido B, Redondo-Sama G, Gómez A. COVID-19 infodemic: more retweets for science-based information on coronavirus than for false information. Big Data Soc. 2020;35(4):377–92. doi:10.1177/0268580920914755. • Uakkas S, El Omrani O. Youth-led efforts against infodemic. Eur J Public Health. 2021;30(5). doi:10.1093/eurpub/ckaa165.570. 21 Digital solutions to health risks raised by the COVID-19 infodemic Law, regulatory and regulation • Sander B, Tsagourias N. The COVID-19 infodemic and online platforms as intermediary fiduciaries under international law. J Int Humanit Leg Stud. 2020;11(2):331–47. doi:10.1163/18781527-01102002. • Tuccori M, Convertino I, Ferraro S, Cappello E, Valdiserra G, Focosi D et al. The impact of the COVID-19 “infodemic” on drug- utilization behaviors: implications for pharmacovigilance. Drug Saf. 2020;43:699–709. doi:10.1007/s40264-020-00965-w. • Zielinski C. Infodemics and infodemiology: a short history, a long future. Rev Panam Salud Publica. 2021;45:e40. doi:10.26633/RPSP.2021.40. Literacy • Biasio LR, Bonaccorsi G, Lorini C, Pecorelli S. Assessing COVID-19 vaccine literacy: a preliminary online survey. Hum Vaccin Immunother. 2021;17(5):1304–12. doi:10.1080/ 21645515.2020.1829315. • Dadaczunski K, Okan O, Messer M, Leung AYM, Rosário R, Darlington E et al. Digital health literacy and web-based information- seeking behaviors of university students in Germany during the COVID-19 pandemic: cross-sectional survey study. J Med Internet Res. 2021;23(1):e24097. doi:10.2196/24097. • Guggiari E, De Gani SM, Berger F, Jaks R. Corona-specific health literacy: a longitudinal study in the German-speaking part of Switzerland. Eur J Public Health 2021;31(3). doi:10.1093/eurpub/ckab165.138. • Karabela ŞN, Coşkun F, Hoşgör H. Investigation of the relationships between perceived causes of COVID-19, attitudes towards vaccine and level of trust in information sources from the perspective of infodemic: the case of Turkey. BMC Public Health 2021;21(1):1195. doi:10.1186/ s12889-021-11262-1. • Lo Moro G, Bert F, Minutiello E, Zacchero AL, Sinigaglia T, Colli G et al. COVID-19 fake news, conspiracy beliefs and the role of eHealth literacy: an Italian nationwide survey. Eur J Public Health 2021;31(3). doi:10.1093/eurpub/ckab164.031. • Okan O, Bollweg TM, Berens E-M, Hurrelmann K, Bauer U, Schaeffer D. Coronavirus-related health literacy: a cross-sectional study in adults during the COVID-19 infodemic in Germany. Int J Environ Res Public Health 2020;17(15):5503. doi:10.3390/ijerph17155503. • Pian W, Chi J, Ma F. The causes, impacts and countermeasures of COVID-19 “Infodemic”: a systematic review using narrative synthesis. Information Processing & Management. 2021;58(6):102713. doi:doi.org/10.1016/j.ipm. 2021.102713. • Siebenhaar KU, Köther AK, Alpers GW. Dealing with the COVID-19 infodemic: distress by information, information avoidance, and compliance with preventive measures. Front Psychol. 2020;11:567905. doi:10.3389/ fpsyg.2020.567905. • Vrdelja M, Vrbovšek S, Klopčič V, Dadaczynski K, Okan O. Facing the growing COVID-19 infodemic: digital health literacy and information-seeking behaviour of university students in Slovenia. Int J Environ Res Public Health 2021;18(16):8507. doi:10.3390/ijerph18168507. Policy • Arkhipova A S, Peigin B S. Ministry of truth: how the Russian state fights fake news during the COVID-19 pandemic. Shagi/Steps 2021;7(4):124–50 (in Russian). • Tangcharoensathien V, Calleja N, Nguyen T, Purnat T, D’Agostino M, Garcia-Saiso S et al. Framework for managing the COVID-19 infodemic: methods and results of an online, crowdsourced WHO technical consultation. J Med Internet Res. 2020;22(6):e19659. doi:10.2196/19659. Social listening • Mahajan A, Czerniak C, Lamichhane J, Phuong L, Purnat T, Briand S et al. Listening to community concerns in the COVID-19 infodemic: a WHO digital approach. Eur J Public Health 2021;31(3). doi:10.1093/eurpub/ckab164.032. • Purnat TD, Vacca P, Czerniak C, Ball S, Burzo S, Zecchin T et al. Infodemic signal detection during the COVID-19 pandemic: development of a methodology for identifying potential information voids in online conversations. JMIR Infodemiology 2021;1(1):e30971. doi:10.2196/30971. • Vital Brito D, Miranda D. Local public health professionals contribution to infodemic management. Eur J Public Health 2021;31(3). doi:10.1093/eurpub/ckab164.860. Digital solutions to health risks raised by the COVID-19 infodemic 22 World Health Organization Regional Office for Europe UN City, Marmorvej 51, DK-2100 Copenhagen Ø, Denmark TEL +45 45 33 70 00 FAX +45 45 33 70 01 EMAIL eurocontact@who.int WEB www.euro.who.int The World Health Organization (WHO) is a specialized agency of the United Nations created in 1948 with the primary responsibility for international health matters and public health. The WHO Regional Office for Europe is one of six regional offices throughout the world, each with its own programme geared to the health conditions of the countries it serves. Member States Albania Andorra Armenia Austria Azerbaijan Belarus Belgium Bosnia and Herzegovina Bulgaria Croatia Cyprus Czechia Denmark Estonia Finland France Georgia Germany Greece Hungary Iceland Ireland Israel Italy Kazakhstan Kyrgyzstan Latvia Lithuania Luxembourg Malta Monaco Montenegro Netherlands North Macedonia Norway Poland Portugal Republic of Moldova Romania Russian Federation San Marino Serbia Slovakia Slovenia Spain Sweden Switzerland Tajikistan Turkey Turkmenistan Ukraine United Kingdom Uzbekistan Document number: WHO/EURO:2022-5351-45116-64364