Be he@lthy, be mobile: a handbook on how to implement mSafeListening

A handbook on how to implement mSafeListening A handbook on how to implement mSafeListening A handbook on how to implement mSafeListening Be he@lthy, be mobile: a handbook on how to implement mSafeListening ISBN (WHO) 978-92-4-004478-4 (electronic version) ISBN (WHO) 978-92-4-004479-1 (print version) ISBN (ITU) 978-92-61-36251-5 (electronic version) ISBN (ITU) 978-92-61-36241-6 (print version) © World Health Organization and International Telecommunication Union, 2022 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non- commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO or ITU endorses any specific organization, products or services. The unauthorized use of the WHO or ITU names or logos is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO) or the International Telecommunication Union (ITU). Neither WHO nor ITU are responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/mediation/rules). Suggested citation. Be he@lthy, be mobile: a handbook on how to implement mSafeListening. Geneva: World Health Organization and International Telecommunication Union, 2022. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/ bookorders. ITU Publications can be obtained from ITU Bookshop http://www.itu.int/en/ publications. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/copyright. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO or ITU concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO or ITU in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO and ITU to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO or ITU be liable for damages arising from its use. Design and layout by 400 communications. iii Contents Acknowledgements vi Executive summary viii Introduction x Hearing loss x Box 1: Hearing loss, deafness and hard of hearing xi Hearing loss caused by loud sounds xii The effect of loud sounds on ears xiii Unsafe listening xiv Hearing loss prevention through safe listening xv The WHO initiative: Make Listening Safe xvii Raising awareness to change listening behaviours xvii Safe listening devices and systems xvii Safe listening entertainment venues xix Be He@lthy, Be Mobile and digital client communication messaging for safe listening xix The Be He@lthy, Be Mobile handbook for Safe Listening xx 1. Operations management 1 1.1. Establishing programme leadership 2 1.2. Undertaking a situational assessment 3 1.3. Stakeholder analysis and engagement 4 1.4. Engaging with civil society 6 1.5. Forming strategic partnerships with the private sector 6 1.6. Funding the mSafeListening programme 7 iv BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 2. mSafeListening workplan development 10 2.1. Establish the “problem statement” 11 2.2. Determine the programme objective 11 2.3. Identify requirements of technology, infrastructure and regulations 12 2.4. Content adaptation 12 2.5. Promotion and recruitment 13 2.6. Monitoring and evaluation 14 2.7. Estimated time frames 14 2.8. Budgeting 15 3. Content development and adaptation 17 3.1. Target population(s) 19 3.2. Programme design 20 3.3. Adapting the existing message content library 22 4. Promotion, participation and retention 34 4.1. Promoting the mSafeListening programme 35 4.2. Participation in the mSafeListening programme 37 Option 1: Signing up for the programme 37 Option 2: Automatic enrolment 38 Monitoring participation 38 4.3. Retention 39 4.3.1. Ways to reduce drop-out 40 4.3.2. Opt-out option 40 5. Technology specifications 42 5.1. Selecting the appropriate technology for the context 43 5.2. Technology implementation needs 49 5.3. Software needs 50 Selecting a software provider 52 Contents v 6. Monitoring and evaluation of mSafeListening 58 6.1. Planning for M&E 59 Step 1. Determining the goal of M&E 61 Step 2. Creating and adapting an M&E framework 62 Step 3. Planning M&E human resources 64 Step 4. Selecting indicators for monitoring mSafeListening 66 Step 5. Design the outcome evaluation and prepare data collection materials 67 Step 6. Preparing the M&E budget 69 Step 7. Analysing the data 69 Step 8. Reporting and dissemination 70 References 72 Annexes 76 Annex 1. Proposed structure of a BHBM implementation team 77 Annex 2. BHBM and mSafeListening programme stakeholders and roles 79 Annex 3. Sources of sustainable funding 83 Annex 4. Questionnaires 86 Annex 5. Adapting content library for voice, messenger apps or chatbots 97 Annex 6. Information on sound levels 105 Annex 7. Important considerations for the promotion of mSafeListening 106 Annex 8. Monitoring and evaluation indicators for mSafeListening 110 Annex 9. Benefits and risks of different software models 123 Annex 10. Role of aggregators 126 Web Annex. mSafeListening message libraries vi Acknowledgements This handbook on how to implement mSafeListening was created through collaboration between the World Health Organization (WHO) Sensory Functions, Disability and Rehabilitation unit and department of Digital Health and Innovation, and the International Telecommunication Union (ITU) Telecommunication Development Bureau. WHO and ITU gratefully acknowledge the following contributors: Handbook preparation WHO Sensory Functions, Disability and Rehabilitation unit: Shelly Chadha, Alarcos Cieza, Kaloyan Kamenov. WHO/ITU Be Healthy, Be Mobile team: Roman Chestnov, Javier Elkin, Hani Eskandar, Melissa Harper Shehadeh, Surabhi Joshi, Sameer Pujari, Mariam Shokralla, Ayush Shukla. Content development for message libraries Social Marketing Gateway (SMG) team, Scotland, United Kingdom of Great Britain and Northern Ireland. Content reviewers Members of the mSafeListening Informal Expert Group: Nicola Diviani (Senior Research Fellow, Swiss Paraplegic Research; Lecturer, University of Lucerne, Switzerland); Raphael Elmiger (Federal Office of Public Health; Consumer Protection Directorate, Switzerland); Katya Feder (Research Scientist, Health Canada, Ottawa); Adrian Fuente (Honorary Fellow & Honorary Research Fellow, School of Health and Rehabilitation Sciences, University of Queensland, Australia); Adriana Lacerda (Associate Professor, School of Speech Therapy and Audiology, University of Montreal, Canada); Colleen LePrell (Professor, Department Chair, Speech, Language, and Hearing, Program Head, Speech, Language and Hearing Sciences, University of Texas, United States of America); Isabelle Naegelen (University of Luxembourg, Life Sciences Research Unit); and Sara Rubinelli (Professor, Health Sciences, University of Lucerne, Switzerland). Acknowledgements vii Guidance Members of the Be Healthy, Be Mobile Steering Committee, WHO: Ruediger Krech, Director, Health Promotion; Bernardo Mariano Junior, Director, Digital Health and Innovation; Bente Mikkelsen, Director, Noncommunicable Diseases; Ren Minghui, Assistant Director-General, Noncommunicable Diseases; Naoko Yamamoto, Assistant Director- General, Health Promotion. Members of the Be Healthy, Be Mobile Steering Committee, ITU: Stephen Bereaux, Deputy to Director, Telecommunication Development Bureau; Doreen Bogdan-Martin, Director, Telecommunication Development Bureau; Marco Obiso, Head of Cybersecurity, Digital Networks and Society Department, Telecommunication Development Bureau. Further contributions Malachi Arunda (Researcher, Lund University, Sweden); Chitra Chander (Public Health specialist, India); Zahra Habibi Babadi (Audiologist, University of Geneva, Switzerland); and Ayrton Michael Hogan (Consultant Audiologist, Australia). WHO regional offices: Jean-Marie Dangou (Regional Office for Africa); Carolina Hommes (Regional Office for the Americas); Padmaja Kankipati (Regional Office for South-East Asia); Satish Mishra (Regional Office for Europe); Elick Narayan (Regional Office for the Western Pacific); Patanjali Dev Nayar (Regional Office for South-East Asia); and Hala Sakr (Regional Office for the Eastern Mediterranean). viii “Be He@lthy, Be Mobile” (BHBM) is a global initiative led by the World Health Organization (WHO) and the International Telecommunications Union (ITU). It is based on the use of mobile technology for digital targeted client communication messaging to address diseases and health issues such as smoking, diabetes, ageing, cardiovascular diseases and chronic respiratory diseases. An estimated 430 million people live with disabling hearing loss requiring rehabilitation services. If current trends persist, this number could rise to well beyond 700 million by 2050. When it is unaddressed, hearing loss has a far-reaching impact on the individuals affected, their families, society and the economy. Across the life course, many causative and preventive factors interplay to determine a person’s hearing capacity; one of the most important of these is exposure to loud sounds. Such exposure may occur in occupational, environmental or recreational settings. This handbook focuses on the risk to hearing posed by listening to high volume sounds for prolonged periods in recreational settings. Globally, over 1 billion people in the 12–35-year age group are at risk of permanent hearing loss due to their unsafe listening practices. Such hearing loss can be completely prevented through adoption of safe listening behaviours. The mSafeListening handbook provides evidence- based message libraries for the promotion of safe listening behaviours and prevention of hearing loss. It includes guidance on how to develop, integrate, implement and evaluate a national mSafeListening programme. Executive summary An estimated 430 million people live with disabling hearing loss requiring rehabilitation services. Executive summary ix To address this issue and to prevent hearing loss caused by unsafe listening, Be He@lthy, Be Mobile has developed the mSafeListening programme. Using mobile technology to ensure access to targeted client communications, the mSafeListening programme provides health information to adolescents and young adults most at risk of hearing loss due to unsafe listening, as well as to their parents or carers. The mSafeListening handbook and programme content were prepared by an international group of experts in digital health, behavioural science and hearing health, in collaboration with WHO and ITU. All content in the handbook is based on existing research evidence, existing WHO content, and/or expert opinions. Content for the programme is available in the form of an online comprehensive message library with suggested message algorithms (web annex). The message library uses evidence- based behavioural change techniques to promote safe listening practices. All content and programming guidance described here should be considered as examples and adapted to the local context of each participating country. xHearing loss Hearing is the sense with which we perceive the sounds around us. Through hearing we engage with our environment, communicate with others, express our thoughts, and gain education. Globally, more than 1.5 billion people live with hearing loss, of whom at least 430 million require rehabilitation services (1). It is anticipated that in the coming decades these numbers could rise significantly (Figure 1). Introduction Figure 1: Projected increase in prevalence of moderate and higher grade of hearing loss 2019 2030 2040 2050 0 500 1000 1500 2000 2500 Number of people in millions 711 430 509 612 1582 1889 2206 2497 Disabling hearing lossAll hearing loss Loss of hearing (see Box 1 for commonly-used terms), if not identified and addressed, can have far-reaching consequences, adversely affecting language development, psychosocial well-being, quality of life, educational attainment, and economic independence at various stages of life (2–4). Introduction xi 1 Sound intensity is measured in decibels, represented as “dB”. Many causes of hearing loss can be prevented. Common ear diseases, ear infections, vaccine-preventable illnesses, and exposure to noise and chemicals, endanger the hearing of many people at different ages. The World Health Organization (WHO) estimates that in children, up to 60% of hearing loss can be avoided through public health measures (5). Even in adults, hearing loss can be avoided or its onset delayed through preventative and protective measures such as avoidance of ototoxic medicines or chemicals, and reduction of noise exposure. Box 1: Hearing loss, deafness and hard of hearing A person who is not able to hear as well as someone with normal hearing (i.e. with hearing thresholds of 20 decibels (dB)1 or better in both ears) is said to have hearing loss. Hearing loss may be mild, moderate, severe, or profound. It can affect one ear or both ears, and leads to difficulty in hearing conversational speech or even loud sounds. “Hard of hearing” refers to people with hearing loss ranging from mild to severe. People who are hard of hearing usually communicate through spoken language and can benefit from hearing aids, cochlear implants, and other assistive devices as well as captioning. “Deaf” people mostly have profound hearing loss, which implies very little or no hearing. They often use sign language for communication. xii BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Box 2: WHO estimates of risk due to unsafe listening practices • 1.1 billion young people worldwide could be at risk of hearing loss as a result of unsafe listening practices. • Among teenagers and young adults aged 12–35 years in middle- and high-income countries: – nearly 40% are exposed to potentially damaging sound levels in recreational venues such as nightclubs, discotheques, and bars. – nearly 50% face the risk of hearing loss due to listening at loud volumes or for prolonged time periods over their personal audio systems. 2 Sound intensity is measured in decibels, represented as “dB”. Hearing loss caused by loud sounds Exposure to loud sounds puts children and adults at risk not only of hearing loss, but other noise-induced health problems, such as insomnia and cardiovascular illnesses (6). Typically, sound intensity2 above 80 dB, heard for periods longer than 40 hours a week can lead to hearing loss (1). Loud sounds can be encountered in the workplace, in the overall living environment, and are commonly experienced as part of recreational activities. In recent years, concern has grown about the rising exposure to loud sounds in recreational settings such as nightclubs, discotheques, pubs, bars, concerts, cinemas, sporting events, and even fitness classes. With the popularization of technology, devices such as music players are often listened to at unsafe volumes and for prolonged periods of time. Regular participation in such activities poses a serious threat of irreversible hearing loss (7). Some data and facts on this are presented in Box 2. Introduction xiii The effect of loud sounds on ears Sensory cells within our ears help us to hear (8–12). Exposure to loud sounds for any length of time causes fatigue of these sensory cells (Figure 2). The result is temporary hearing loss or tinnitus. A person enjoying a loud concert, for example, may afterwards experience muffled hearing or a ringing or buzzing in their ears (tinnitus). This often improves as the sensory cells recover. However, with regular exposure, particularly to loud or prolonged noise, damage of the sensory cells and other structures can be permanent, resulting in irreversible noise- induced hearing loss, tinnitus, or both. Figure 2: Hearing mechanism and sensory cells Outer ear Temporal bone External Auditory Canal Tympanic membrane Auditory ossicles Eustachian tube Semicircular canals Vestibulocochlear nerve Cochlea Normal sensory cells Damaged sensory cells The onset of noise-induced hearing loss can be immediate (such as when exposed to a sudden burst of loud sound); however, more often the loss is gradual, it is permanent and frequently goes unnoticed or ignored until the effects become more obvious. For example, sounds may become distorted or muffled; a person may find it difficult to understand other people when they speak, especially in places where there is background noise such as in restaurants, or they may have to turn up the volume when watching the television. xiv BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Unsafe listening Unsafe listening refers to the common practices of listening to music or other audio content at loud levels or for prolonged time periods (13, 14). A sound of 80 dB is similar to the sound of a doorbell ringing (see Figure 3). Sensory cells in the ears can start to become damaged by prolonged exposure to loud sounds. For most people, listening to sounds at 80 dB for 40 hours a week is the limit of safe listening. The permissible time for safe listening decreases as sound levels increase. For example, a sound as loud as 100 dB – the level produced by a hair dryer – can only be listened to safely for less than 5 minutes each day. Music at clubs and concerts is often as loud as 110 dB, and some headphones can play music equally as loud when the volume is close to, or at, maximum level. Even a short duration of exposure to levels of such high decibels can be harmful. Habitual exposure over time almost certainly leads to tinnitus and hearing loss. Hearing loss prevention through safe listening “Safe listening” refers to a set of practices and behaviours that allow music to be listened to at an enjoyable level while lowering the risk of irreversible hearing damage. Adoption of safe listening practices will support people to continue enjoying music long into the future. The risk to hearing is dictated by a combination of the sound level and the duration of the exposure (see Box 3 on equal energy principle). Listening can be made safer by: i) reducing the sound level; and/or ii) reducing the duration of the exposure. Introduction xv Figure 3: Approximate sound level in dB* and maximum permissible time per week for safe listening Airplane taking off 140dB 0 sec Firecrackers 150dB 0 sec Jackhammer 130dB 1 sec Standing near a siren 120dB 12 sec Shouting in the ear 110dB 2.5 min Car horn at 5 meters 105dB 8 min Hair dryer 100dB 20 min Motorcycle 95dB 1h 15 min Shouted conversation 90dB 4 hours Heavy traffic (inside the car) 85dB 12h 30 min Normal conversation 60dB Unlimited Library 40dB Unlimited Soft whispering 30dB Unlimited Normal breathing 10dB Unlimited Doorbell 80dB 40 hours Box 3: The equal energy principle The equal energy principle states that the total effect of sound is proportional to the total amount of sound energy received by the ear, irrespective of the distribution of that energy over time and that the amount of energy doubles for every 3 dB increase in intensity of sound (13, 15, 16). Hence, a person may receive the same “noise dose” listening to music at 80 dB for 8 hours a day as listening to 100 dB for about 4 minutes. * This graphic is based on the 3-dB exchange rate and the WHO recommendation regarding safe listening exposure and weekly time limit. The examples of sound levels are indicative. Actual sound levels may vary. xvi BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Some simple steps that individuals can take to reduce their personal sound exposure include: • Keeping the volume down. Listening to personal audio systems (devices such as smartphones or MP3 players through which music is played, and ear/headphones, as well as headphones with music playing capabilities) at a volume level below a maximum of 60% is helpful in reducing the risk of hearing loss and tinnitus. Using carefully fitted, and if possible, noise cancelling headphones is also advisable. • Limiting time spent engaged in noisy activities. Listening on a device for a prolonged period of time can also damage hearing. When in nightclubs, discotheques, bars, sporting events and other noisy environments, taking short listening breaks away from loud noise reduces the overall duration of noise exposure. • Monitoring listening levels. Many devices, such as smartphone apps or listening software, use built-in safety features such as volume limiting and sound level monitoring. These indicate precisely the levels of sound as well as the duration of listening, and provide an assessment of the potential danger of overexposure. Apps such as HearAngel and dbTrack, or the hearing health app that is included in some smartphones, can be used for this purpose in addition to keeping track of the sound consumed and alerting the listener if the recommended limit (of 80dB for 40 hours per week) is exceeded. • Protecting ears from loud sounds. This can be achieved by wearing earplugs in noisy venues and moving away from sources of sound, such as loudspeakers. • Heeding the warning signs of hearing loss. It is critical not to ignore early signs of noise damage. Tinnitus; problems in hearing high- pitched sounds such as doorbells, phones or alarm clocks; difficulty in understanding speech especially over the telephone; and following conversations in noisy environments such as restaurants, are all signs that damage may have occurred. • Taking regular hearing checks. This is especially important for people who are at risk for hearing loss, such as those listening to music regularly or visiting loud entertainment venues. The hearWHO screening app can be used to check and track hearing status and to seek professional advice if a hearing test is failed. Introduction xvii The WHO initiative: Make Listening Safe In consideration of the facts that: • more than 1 billion young people put themselves at risk of permanent hearing loss, often unknowingly, by listening to music at loud intensity over long periods of time; and • this risk could be mitigated through public health action to promote safe listening practices; WHO, in 2015, launched the Make Listening Safe initiative. This initiative aims to promote safe listening and reduce the risk of hearing loss due to recreational loud sounds. The main areas of WHO’s work are set out below: Raising awareness to change listening behaviours WHO’s approach to safe listening centers on changing listening behaviours in the population. To facilitate this, WHO has developed evidence-based awareness tools. These include awareness materials that are available on WHO webpages. A media brief on safe listening provides accurate and up-to-date information on safe listening, along with ideas for stories that can be used by journalists to disseminate information on safe listening . In the future, WHO will be working on an educational module for safe listening for school children. mSafeListening is a part of this awareness raising effort. Safe listening devices and systems In 2019, WHO published the WHO-ITU Global standard for safe listening devices and systems (16), which offers recommendations on safe listening features on personal audio devices. The recommendations aim to make personalized information on sound consumption and risk of hearing loss available to users of personal devices such as smartphones and MP3 players. These features include: • A sound measurement function: software that tracks the level and duration of the user’s exposure to sound. • Personalized profile: an individualized listening profile, based on the user’s listening practices, which informs the user how safely (or not) they have been listening, and gives cues for action based on this information. xviii BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening • Volume limiting options: options to limit the volume, including automatic volume reduction and password-protected volume control. • General information: information and guidance to users on safe listening practices, both through personal audio devices and for other leisure activities. WHO works closely with ITU, civil society partners, private sector and governments to promote and support the implementation of this standard. See Figure 4 for example of implementation. Figure 4: Examples of safe listening features on personal audio devices Loud Today Exposure 88 dB (4h 10m) About Environmental Sound Levels Show All Filters This represents the intensity of sounds in your environment Today Exposure 85 dB (52m) About Headphones Sound Levels Show All Filters This represents the intensity of sounds in your environment OK Today Hearing 3 7-Day Exposure 7-Day Exposure Noise Notifications Headphones Sound Levels Environmental Sound Levels 3:00 PM 12:22 PM 3:59 PM OK OK Noise 72dB 30 80 120 OK Long-term exposure to sounds at this level Noise 83dB 30 80 120 Loud Repeated, long-term exposure to sounds at Loud Long-term exposure to sounds at this level should not affect your hearing. Repeated, long-term exposure to sounds at this level can damage OK Introduction xix Safe listening entertainment venues To reduce the risk of hearing loss due to loud sounds in entertainment venues and events such as concerts, discotheques, nightclubs and bars, WHO has developed an evidence-based global standard for safe listening in these settings. The standard provides six “Features” to make consumption of amplified music safer for the audience by recommending: i) an upper sound level limit; ii) ongoing monitoring of sound levels in venues and events; iii) the optimization of venue acoustics and sound system design; iv) the provision of hearing protection free of charge to audience members upon request; v) access to respite areas where attendees can rest their ears from loud sound; and vi) accessible information to the audience before and during an event to confirm safe listening measures at the venue. Governments can implement these Features through legislation or regulation, and owners and managers of venues can voluntarily adopt the standard and implement the Features as an example of good practice. Be He@lthy, Be Mobile and digital client communication messaging for safe listening Be He@lthy, Be Mobile (BHBM) is a global initiative led by WHO and the International Telecommunications Union (ITU) to encourage and enable the use of mobile technology to promote health. The aim is to help combat noncommunicable diseases, and in other health areas where change in behaviour can have a positive impact on health and well- being, through digital targeted communication approaches including text messages, apps, and chatbots. BHBM supports governments in digital health programming by providing topic-specific toolkits and technical support to countries, including guidance on programme design, fundraising and evaluation. Official partnerships are currently in place within 12 countries representing a range of income groups and disease focuses, including tobacco cessation, diabetes and cervical cancer. Through evidence-based target messages, digital health3,4 can offer a solution to reaching large audiences and to potentially change their 3 World Health Organization, “WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening,” 2019, https://www.who.int/publications-detail- redirect/9789241550505. 4 World Health Organization, “Classification of Digital Health Interventions v 1.0,” 2018, https://apps.who.int/iris/handle/10665/260480. xx BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening health-related behaviour. Systematic review evidence shows moderate effects in favour of targeted messaging to support behavioural change initiatives across numerous risk factor areas, such as quitting smoking (17), diabetes self-management (18), increasing medication adherence in people who have cardiovascular diseases (19) and increasing physical activity levels (20). The importance of effective communication and health education for safe listening has been stressed repeatedly by researchers across the world (21–26). Although limited, literature on the use of digital targeted client communication messaging based on the “health belief model” in hearing care shows promising results for improving people’s knowledge, beliefs and behaviours (27–31). The Be He@lthy, Be Mobile handbook for Safe Listening This handbook has been developed following a consultative process with a group of international experts in audiology, sound, behavioural science and digital health. It includes: • An online message library for key end-user groups, i.e. young people most at risk of hearing loss, parents, and the general population (web annex). The message library was developed through a formative process that included engagement with, and feedback from, the target user groups. • Operational guidance and resources to support national governments and programme managers in developing, implementing and monitoring mSafeListening programmes for hearing loss prevention. The handbook is intended for use by government officials, WHO staff members, academics, and in-country implementing partners who are involved in large-scale digital health messaging programmes. The mSafeListening programme should not be conducted in isolation, but rather be complimentary to existing clinical interventions, policies and awareness related to hearing loss prevention and noise reduction. The mSafeListening handbook should be read in full prior to initiating the development of an mSafeListening programme, and accompanied by other technical resources in evidence-based digital health programme planning.5 The sections and annexes provide generic resources that can be used and adapted according to country needs. The library of messages should be customized following the principles elaborated in the handbook content, in order to be locally relevant and effective. Implementers of mSafeListening are encouraged to monitor, evaluate and report on the effectiveness of the programme and provide feedback to the WHO Be He@lthy Be Mobile team. 5 World Health Organization, UNICEF, UNFA, PATH, “Digital Implementation Investment Guide (DIIG): Integrating Digital Interventions into Health Programmes,” 2020, https://www.who.int/publications-detail-redirect/9789240010567. 1. Operations management 11. Operations management Operations management 1.1. Establishing programme leadership 2 1.2. Undertaking a situational assessment 3 1.3. Stakeholder analysis and engagement 4 1.4. Engaging with civil society 6 1.5. Forming strategic partnerships with the private sector 6 1.6. Funding the mSafeListening programme 7 1 2 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening The decision to implement mSafeListening, should be followed by designation of: • a project team which may include a project manager plus 2 or 3 people to manage the overall programme design and help with day- to-day programme operations (see Box 4 for roles of a project team). • a small expert group consistent of experts in public health, hearing loss, information technology, behavioural sciences and communications specialists. The project team should work in close collaboration with the National Programme Steering Committee for digital health, National Technical Advisory Group (TAG) and if appropriate, an International Advisory Group for the host programme (BHBM or hearing care) (see Annex 1: Proposed structure of a BHBM implementation team). NOTE: In places where no digital health programme exists, the steering committee, national TAG and international advisory group may have to be established specifically for mSafeListening. Wherever possible, efforts should be made to engage with other BHBM programmes in the country (e.g. mAgeing, mTobacco or others) and technical resources shared across these different initiatives to ensure optimization of available resources and to ensure cross-learning and experience-sharing. 1.1. Establishing programme leadership The mSafeListening programmes should be led by a country’s Ministry of Health (MoH). It should be developed and implemented as part of a national or subnational digital health or digital health programme such as “Be He@lthy, Be Mobile” or be part of a national/subnational ear and hearing care plan. The parent programme (BHBM or hearing care) should have the overall responsibility for mSafeListening. 1. Operations management 3 Box 4: Responsibilities of an mSafeListening project team The SafeListening project team: • engages with the BHBM National Programme Steering Committee, National Technical Advisory Group (TAG) and International Advisory Group. • develops and maintains strong partnerships with: – other relevant programmes and government departments – academia – civil society – associations of young people, civil society groups including people with hearing loss. • runs mSafeListening to budget and to deadlines. • reports to WHO and donors. 1.2. Undertaking a situational assessment A situational assessment is crucial to understand the needs and the context within which the mSafeListening programme will be implemented. Such an assessment should be carried out at the start of planning with the aim to develop a snapshot of hearing loss, problems related to noise exposure, relevant policies and regulations, attitudes towards noise and hearing loss, and practices related to unsafe listening. Situational assessment should provide an understanding of the programme setting through: • Reviewing information and data regarding hearing loss and noise exposure in the place where this programme is to be implemented. • Mapping ongoing digital health messaging initiatives and activities for the promotion of hearing care. • Interviewing key informants and local stakeholders such as young people, their parents, people with hearing loss, and the general public. 4 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening • Exploring existing telecommunication companies, mobile network providers, statistics of mobile phone use and costs to consumers of text messages, and any industry representing body or association. • Understanding cultural issues, local attitudes and behaviours of young people about safe listening; identifying barriers and motivators; and documenting existing synergies. • Identifying existing national or subnational noise regulatory policies or safe listening programmes. • Exploring existing digital health messaging infrastructure, and researchers who can contribute to the success of the programme. • Identifying funders of health promotion interventions. The findings gathered will inform community- or country-specific development and implementation of the mSafeListening programme. The situational assessment can include specific indicators to act as a baseline measure for monitoring and evaluation (M&E). (It is thus advisable to read the M&E section of this handbook before designing the situation assessment.) The time needed to undertake a situational assessment, and the assessment priorities identified in terms of desired outcomes, may vary from country to country depending on which information already exists and at what stage are any national interventions related to hearing loss prevention, noise control and safe listening. Help with planning or undertaking a situational assessment, is available through contacting the BHBM team at bhbm@who.int. 1.3. Stakeholder analysis and engagement As with any implementation project, it will be necessary to consider a range of stakeholders; these include: • Ministry of Health and other ministries such as Finance, ICT, Telecommunication, Data Protection. • Telecommunication regulatory authority. • Digital or health service providers. • Telecommunications operators. • Local aggregator. • Data privacy commission. 1. Operations management 5 • WHO, ITU and other UN entities. • Academic institutions working in the field of digital health. • Technology providers. • National Informatics Centre • Technology Development Agency. • Local NGOs working in digital health. • Potential donors. Annex 2 provides an overview of the role of different stakeholders involved in BHBM (digital health programmes). In addition, when developing an mSafeListening programme, it is important to identify and engage with all relevant stakeholders. These include: 1. Academic institutions and experts in the field of: – hearing loss and safe listening; – behavioural science; – health communications; and – public health. Institutions and experts with specialized knowledge or research in this field are able to guide the content development and adaptation process, as well as monitoring and evaluation. 2. Representatives of the target group (i.e. young people and parents). 3. Associations of young people. 4. Civil society groups including people with hearing loss. 5. Music industry, influencers and role models for the target group. Musicians can facilitate the promotion and recruitment strategy by running campaigns on their platforms and liaising with media outlets. The groups numbered 2 to 4 can give feedback and advice on programme design and content adaptation. They can also actively support and encourage roll-out and uptake of the mSafeListening programme. The civil society group may, in many places, be able to provide financial support to the programme. 6 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 1.4. Engaging with civil society Engaging with civil society is strategic for obtaining funding and in- kind support. Civil society groups working in the field of hearing care can support the project through engaging with the target groups; undertaking media outreach; and raising awareness on safe listening and mSafeListening programmes in schools, parent associations and other relevant platforms. Civil society can also assist in development and translation of promotional materials. 1.5. Forming strategic partnerships with the private sector Strategic partnerships with the private sector can be mutually beneficial if they are a “good fit” with one or more of the long-term corporate strategies of the organization.6 Identifying private sector organizations interested in funding elements of projects or entire projects is the first step in the process. A number of factors can motivate the private sector to collaborate with country programmes, including shared missions, and the opportunity to share knowledge and extend programme reach. Engaging with the private sector can provide BHBM programmes with several opportunities to improve service delivery, creating a win-win situation. For example, private companies may benefit from exposure or association through a direct partnership and as a result may provide reduced fee, cost-free, or in-kind services. Different types of sectors may have an interest in different aspects of the programmes; for example telecommunications companies interested in providing a special product offer to clients and manufacturers of smartphones or music equipment, may engage in mSafeListening as part of their corporate social responsibility programmes. It is necessary to ensure that the partnership remains impartial, and that the organization does not pose a conflict of interest. Any contracts or written agreements should be accompanied by a process of due diligence with clear clauses regarding data ownership and intellectual property. In order to maximize chances of success, attempts should be made at selecting companies whose longer-term engagement is consistent with their corporate strategy and core business. This will ensure a sustainable and long-term partnership. BHBM has some experience in working with private partners and can provide advice and support with managing these potential partners. 6 World Health Organization, A Practical Guide for Engaging with Mobile Operators in mHealth for RMNCH (Geneva, Switzerland: World Health Organization, 2015). http://apps.who.int/iris/bitstream/10665/170275/1/9789241508766_eng.pdf?ua=1. 1. Operations management 7 1.6. Funding the mSafeListening programme Ensuring a robust funding model for mSafeListening is essential. And, while initial costs of such a programme can be high, careful implementation that maximizes input through integration with other digital health programmes and hearing care programme can make the investment more than worthwhile. At the outset, each country should commit financial and human resources, as well as political will, to ensure the programme’s success. The WHO-ITU BHBM team has specified certain funding requirements that need to be in place prior to a technical support agreement with the team. These are listed in Box 5. Box 5: Funding specifications The following funding specifications need to be met by partner countries prior to a technical support agreement with BHBM: • It is recommended that the country covers the initial investment in the platform and annual operational expenditure of running the service (including staff time). This will mean establishing a fixed budget line for basic costs of the service. • Countries can (and are encouraged to) use existing infrastructure, staff support etc. to reduce this cost. However, they must show a clear budget breakdown of the areas they are covering versus the gaps that will need to be covered. • It is recommended that, if possible, 50% of funding comes from the government. Funding must have been obtained or confirmed in some way. Identifying potential donors is not a sufficient commitment. NOTE: As well as funding specifications, the BHBM country support team will run interested potential implementers through a country readiness questionnaire. If the results show that a country is not ready in all programme-relevant areas, the team can support the country with preparatory activities. (Further information is available by contacting the BHBM team at: bhbm@who.int.) 8 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Sources of funding: BHBM programmes have found several sustainable and successful business models for scale digital health programming that broadly fall into three categories: 1. Government funding: this may come through existing or newly created budget lines. 2. Bilateral or multi-lateral support: Bilateral support is the investment in one Member State by another Member State. Multilateral support typically comes from a multilateral development bank, chartered by two or more countries, for example, the African Development Bank. 3. Third-party grants: these typically come from international health donors, national nongovernmental organizations (NGOs), health donors, philanthropists, or from the private sector (see Box 6 for tips on applying to funders). These options are not mutually exclusive, and should be explored as early on as possible in the programme’s development. Further information on funding sources is provided in Annex 3: Sources of sustainable funding. 1. Operations management 9 Box 6: Tips on applying for grants from funders Fundraising requires time and effort; well-written proposals that present the need for the programme, its goals and objectives and how these will be achieved; and a clear statement of the added value for the stakeholders involved. Often donors will have their own structured grant proposals form, but if they do not, a grant proposal should take the following broad structure: • History of your organization or ministry department, including mission statement/vision. • Project summary. • Background, context and beneficiaries. • Statement of need. • Information about the programme (including goals and objectives, strategy, scope, expected outputs. • Anticipated impact of the programme (this section can include a business model). • Project timeline. • Project budget (including any other funds or statements of in-kind support from partners). • Monitoring and evaluation and donor reporting. • Project risk identification and management (only to be included if a requirement from the donor). • Future funding, scalability and sustainability. Other fundraising tips to consider: • Appearance is important; ensure documents look professional and are copy-edited. Send PDFs (not Word documents) with your organization’s letterhead on the cover letter. • Try to get to know the donor before applying, to understand what is important to them in building the case for your programme: always think about the donor mission and agenda and how the proposal aligns with and will advance their agenda. • Include concepts such as ensuring equity to access to the programme, gender, capacity building, monitoring and evaluation and sustainability; these are important areas that are sometimes overlooked in proposals. • Be sure to advertise what is unique about your ability to carry out the programme successfully. 10 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening2. mSafeListening workplan development mSafeListening workplan development 2.1. Establish the “problem statement” 11 2.2. Determine the programme objective 11 2.3. Identify requirements of technology, infrastructure and regulations 12 2.4. Content adaptation 12 2.5. Promotion and recruitment 13 2.6. Monitoring and evaluation 14 2.7. Estimated time frames 14 2.8. Budgeting 15 2 2. mSafeListening workplan development 11 Following the designation of a project team and an expert group, situation assessment and stakeholder analysis, the following activities should be carried out. 2.1. Establish the “problem statement” Referring to the needs and context revealed through the situational assessment, the problem statement can be framed, such that it describes the problem the mSafeListening programme intends to address. For example, the problem statement could be framed as “Majority of adolescents and young adults are at risk of permanent hearing loss and tinnitus due to listening to loud sounds through personal music devices and in noisy entertainment venues.” OR “There is increase in hearing loss and tinnitus among young people due to listening to loud sounds through personal music devices and in noisy entertainment venues.” 2.2. Determine the programme objective An mSafeListening programme should aim to address the problem identified in 2.1. For example, the programme could aim to “Change listening practices among young people in order to reduce their risk of hearing loss and tinnitus.” OR “Reduce the development of permanent hearing loss or tinnitus among young people due to unsafe listening.” The problem statement and situation assessment should be developed in consultation with the identified expert group, advisory group and steering committee. 12 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 2.3. Identify requirements of technology, infrastructure and regulations This includes: • Selecting technology to be used for mSafeListening • Identifying software needs • Procuring technology • Developing a dashboard and enabling access for monitoring and reporting • Procuring a shortcode if necessary • Negotiate with telecommunications regulators, aggregators, and operators for pricing to ensure service can be used free of charge by users • User test the technology and registration process • Develop data security and interoperability standards • Technology pre-testing. Please refer to section 5 (Technology specifications) for a detailed explanation of technology specifications. 2.4. Content adaptation The steps for this can be summarized as below and are elaborated in section 3: • Translation • Local expert review • Focus group review • Ecological testing • Content verification with WHO expert group (optional) • Content development (if necessary) • Special content identification • Check existing resources • Write new key messages • Adapt content to other platforms if necessary • Adapt content e.g. write voice scripts or chatbot scripts or revise content library • Identify/produce multimedia content if necessary 2. mSafeListening workplan development 13 This mSafeListening handbook includes a library of messages that can be used by governments, partners, and other interested organizations. However, the content may have to be adapted to suit the local context. It is important that this adaptation process is user-research driven. (Further details on the process of adaptation are provided in section 3.) NOTE: For including an attribution to WHO, any adapted messages should be shared with the BHBM team prior to their finalization and use. 2.5. Promotion and recruitment In parallel with content development, it is important to outline the strategy for promoting mSafeListening in the target population and generating enrolment into this programme. The following steps are further outlined in sections 3 and 4. 1. Get to know your target audience. 2. Set up the programme enrolment procedure and service, and ensure it has been adequately tested before user recruitment. 3. Develop a promotion plan addressing the following questions: – Who will be targeted? Segment the target user group. – What are the short-term, mid-term and long-term promotion strategies? – Who will promote? For example, will there be third party marketing specialist involvement? How will workers, civil society, community leaders, and media personalities be involved? – How will they promote? For example, promotion channels should be selected according to user preferences (via mobile phone messages, social media, health centres etc.). What are the incentives for involvement? For example, health workforce training/awareness sessions. – What will be used to promote? For example, what are the key messages for different target-user segments, what other promotional materials are needed (posters, leaflets)? – When will promotion occur? For example, launch date, important dates for the calendar (World Hearing Day, national ear care or hearing day/week). Section 2 provides comprehensive guidance on how to plan promotion, and points to consider for maximizing recruitment. 14 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 2.6. Monitoring and evaluation While setting up a programme, it is important to decide how progress will be measured, i.e. what will be measured and with what frequency. The following steps listed below are detailed in Section 5, Monitoring and Evaluation. • Define Monitoring and Evaluation (M&E) outputs • Adapt an M&E framework • Plan M&E human resources • Select M&E indicators • Design data collection and select tools • Prepare budget • Monitor data collected • Evaluate data collected • Prepare reports and dissemination plans for evaluation • Plan programme review and refinement, and improvement of service provision. 2.7. Estimated time frames The following timeline is suggested based on ongoing BHBM programmes; it may vary depending on country, local process and requirements. • Planning: 3–4 months • Content adaptation: 4–6 weeks • Technology: 4–6 weeks • Development of promotion and recruitment strategy and material: 4–6 weeks • Programme implementation: 6–9 months • Monitoring and evaluation: throughout programme. 2. mSafeListening workplan development 15 2.8. Budgeting The budget required for digital health messaging programmes varies between countries. The goal is for the mSafeListening programme to be cost-free for end-users, as costs can be a major barrier to uptake. Year 1 costs will be typically higher (c. US$ 90 000 – US$ 200 000) because of the capital costs of content and software development, and higher engagement and support needs. Based on current experience, the following very rough estimates are presented: • Programme coordination: US$ 30 000 • Content adaptation: US$ 15 000 • Technology platforms and procurement: US$ 30 000 – 100 000 • Promotion: US$ 30 000 • Monitoring and evaluation (across the span of the programme): US$ 40 000 These costs will vary depending on a number of factors summarized in Box 7. Including a contingency fund of around 10% is suggested. It is often the case with technological programming that unexpected costs can arise, for example, software bug fixing or last-minute necessary changes to specifications. This buffer can then either be repaid to the donor or absorbed into financing subsequent project phases. 16 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Box 7: Common factors that influence the budget of a digital health messaging programme • The chosen delivery platform and costs of software development: These could include SMS, an existing messenger app, in-app messaging, purpose-built app, website, diagnostic tool etc. (See section 5 for details on these platforms and considerations for making the appropriate selection.) • Current and needed resources: These include human resources, content and technology/software requirements. • Need for new content: This handbook contains a library of validated messaging content. However sometimes the programme may require translation, cultural and contextual adaptation, or additional content such as e-learning content, diagnostic guidance etc.; any new content will require development which will involve additional resources and costs. • Promotion and recruitment methods: Marketing of the product or programme can be costly (as, for example, with social media advertising). • Monitoring and evaluation capacity: This can be costly, but is worth the investment and can help make the case for programme expansion and further funding. 3. Content development and adaptation 173. Content development and adaptation Content development and adaptation 3.1. Target population(s) 19 3.2. Programme design 20 3.3. Adapting the existing message content library 22 3 18 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Experience shows that despite the best efforts of an expert content development group to appropriately craft targeted health behaviour-change content for a global audience, critical improvements to programmes are realized through primary research-driven content refinement, with target population closely involved. This section of the handbook will provide guidance on: • programme design; • adapting the existing BHBM mSafeListening message library to your context; • creating additional content for an mSafeListening programme where the library content is not sufficient for local needs; and • considerations for adapting BHBM content library to other formats (e.g. voice and messenger apps and chatbots) and adding multimedia. A standard content library is available online for countries to use as a basis for an mSafeListening programme. This has been written in the format of SMS messages, but is adaptable to other digital health intervention types or delivery channels.7 It is the experience of the BHBM team that SMS is the most equitable modality to deliver health messaging in resource-limited settings (where much of the population may not have access to a smartphone). Therefore, BHBM suggests that if countries wish to deliver the programme through smartphones, SMS programming is run alongside other technology options, such as messenger apps or stand-alone apps. If the BHBM library is to be adapted or new content created, the implementation team will need to review any existing digital health or recreational noise control guidelines that may guide these processes. In both cases, the target population (e.g. young people that frequent entertainment venues or routinely use earphones or headphones for listening to music, parents and guardians) should be involved to provide essential insights and validate the finalization of programme design and content. Where feasible, these messages should be pilot-tested in the target population prior to their widespread use. NOTE: The BHBM secretariat requests to be informed of any adapted or additional messages or content (by email to: bhbm@who.int). This is to benefit further iterations of the global content libraries and the programme development of other countries. 7 World Health Organization, “Classification of Digital Health Interventions v 1.0,” 2018, https://apps.who.int/iris/handle/10665/260480. 3. Content development and adaptation 19 3.1. Target population(s) The target population for the programme (age-range, place of residence and type of recreational noise exposure) should be agreed at the outset. Given that WHO estimates that 50% of those aged 12–35 years participate in unsafe listening practices (32), the main population groups to be targeted through mSafeListening are: • Young adults (mainly aged 18–35 years): this group are most likely to listen to audio content over their personal devices at high volumes or frequently visit loud entertainment venues. • Parents of young children: studies have shown that listening behaviours can be best influenced in pre-adolescents (30, 33). The most suitable way to target this group in mSafeListening is through their parents. • Persons involved in care of young children, such as teachers or health workers who provide care. These population groups can be considered as key targets, subject to local context and regulations. For example, in some countries it may be feasible to reach even those aged 16 years directly. In others, due to limited access to personal music devices and entertainment venues, unsafe listening may only pose a challenge to those aged above 25 years. The geographical area or place of residence is also likely to be a consideration. Some countries may only wish to run mSafeListening in urban areas due to limited access to technology; or to target specific cities/towns/places where there is a known culture of people visiting loud venues or hosting high-volume events (music festivals, sporting events). 20 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 3.2. Programme design Based on experience with BHBM programmes, expert inputs and user feedback, the following suggestions are made regarding the format, frequency, and timing of messages to be delivered as part of mSafeListening. Each country/implementing partner can modify these to their local context. • Duration of programme: 6–9 months is suggested as the optimal duration of the programme for an enrolled individual, with a view that: – Complex health behavioural changes can take up to 6 months to habitually incorporate into a person´ s lifestyle (34–37). – Lengthier duration of programme may result in fatigue and thereby reduce the impact of the messages. • Message format: The mSafeListening content library consists of targeted one-way messages for both informing the target group of the risks of unsafe listening, and providing tips for safe listening. The ultimate aim is to change listening practices among the target group and mitigate their risk of hearing loss due to loud sounds. • The media and channels used: SMS format is most appropriate using available telecommunications networks. Other modalities such as WhatsApp messaging, WeChat, Signal and others can be considered based on the local context. Information provided in Section 5, Technology specifications can help in making this decision. • Timing: It is important to understand the common practices among the target group, to discover the time people are most likely to experience loud sounds through their personal devices or in the environment. To increase their effectiveness, the timing of receiving mSafeListening messages should preferably coincide with key moments when the target behaviour of a person is front-of-mind. For example, if there is a culture of visiting loud entertainment venues on a Friday evening, it may be best for a person to receive a message late on a Friday afternoon or early in the evening. Since most people in the 18–35-year age group may be engaged in study or work, it is also likely that they use their headphones more in evenings. This could vary from person to person, their personal preference, work profile and family culture. An algorithm which customizes time of message delivery based on the individual’s profile is ideal. • Frequency: At the start of the programme, sending 2 to 3 messages per week is advised. Over time, and so that messages are not perceived as intrusive, the frequency can be reduced to 1 or 2 per week (38, 39). 3. Content development and adaptation 21 • Themes: The theme of messages should be determined by the type of noise exposure. For example, people who frequent noisy entertainment venues may receive different messages from those who receive most sound through listening to music on ear/ headphones. Moreover, the theme of the message can vary according to the profile of the individual, the day or the timing. The same person can receive messages with different themes at different times. • Customization (choice and flexibility in the programme): It may be possible for some digital health messaging platforms to provide recipients the option of customizing the time of day they receive the message, or to adapt its frequency and theme according to type of exposure being faced. Such customization should be based on baseline data collected at time of enrolment and changes in listening habits during the course of the programme. • Collection and storage of baseline and accumulating research data: At the time of enrolment into the programme, it is ideal to seek baseline information on the individual’s listening habits and sources of exposure. This can be done through a brief survey questionnaire administered at the outset and is useful in terms of: – Getting a better overview of people’s listening habits, which will allow better targeting through mSafeListening messages. – Determining the optimal theme/s and timing of messages delivered, especially where the software allows customization of the type and timing of messages according to individual profiles. An example of such a survey questionnaire is provided in Annex 4. Subsequently, it may be useful to follow up with questions similar to those used in the initial survey questionnaire to determine any change in people’s listening behaviours. Responses will further refine the messaging algorithm and also assess the outcome and impact of mSafeListening. • Registration, opt-in and opt-out process and administrative communication (if any): There should be an accessible way for people to register for the programme and start receiving messages, as well as an option for people to opt out. In the event of opt-out, the programme should enquire about the reasons for quitting (see Annex 4: Questionnaires). This will provide useful information to revise and refine the programme algorithm and messages. Such feedback should be collated and discussed with the expert group, TAG and steering committee. It should also be shared with WHO. 22 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Countries must adapt the programme to their own social and cultural context; to the infrastructure and technologies available and used by the population; to available funding; and to existing health and social systems. Countries can also lengthen or shorten the programme; increase or decrease the intensity of messages; adapt the system rules; adapt registration, opt-in/out and other functions; adapt the age-limits for teenagers and young adults; and sources of exposure. It is important to engage local behavioural and communication experts along with members of the target audience at the adaptation stage to discuss the timing, frequency and time frame of the messages received. 3.3. Adapting the existing message content library The mSafeListening online message library (web annex) was developed through engagement with experts in the field of safe listening and health communication. Workshops were held to receive inputs from young people (aged 16–25 years) from diverse backgrounds, and parents. The messages were drafted and reviewed by experts; they were then pilot tested through an online platform. The content of the message library was written with the understanding it should be adapted for country use. Adaptation of messages to the local context is important because adapted content is likely to be clearer, more relevant and efficacious for the target population, and may lead to a higher retention of users. Steps involved in adaptation are described in 3.3.2 and summarized in Figure 5. 3.3.1. Important considerations in adaptation of the message library Engagement with target group and experts Local experts, young people (aged 16–25 years) and parents of pre- adolescents or adolescents should guide the adaptation process leading to a library of messages that are easy to understand, appropriate, and relevant to the target population. Inputs from these population groups can be gathered through review processes and qualitative methods, including focus groups, surveys and consumer pre-testing. Testing can be conducted by the programme organizers themselves or by a contracted specialist market research company. 3. Content development and adaptation 23 Tone of the messages Insights from the development of the existing content library for mSafeListening indicate that in order to engage young people effectively, it is critical to reflect on the style, tone and wording of the message (Box 8). For most young people, safe listening messages that are more likely to be efficacious are: • positively framed; • clearly worded and engaging; • emphasize easy changes and short-term benefits; and • coincide with key moments and events. Wording To increase their impact, messages must be understandable, acceptable and relevant to people receiving them. Words and terms used in messages should be replaced by those most appropriate to the local context. For example, in a message such as “You won’t feel so Rock ‘n’ Roll with permanent screaming in your ear. Take a 5-minute break every hour to reduce the risk of hearing loss.” If the term “Rock ‘n’ Roll” is not popular locally, it should be replaced with locally relatable wording. For example, in places where discotheques are more relatable than rock and roll music or a concert, this message can be adapted to: “You won’t feel like discoing with permanent screaming in your ear. Take a 5-minute break every hour to reduce the risk of hearing loss.” Box 8: Examples of messages Positively framed message: “Love your hearing as much as you love your family? Damage to your hearing makes it difficult to communicate. Lower your volume and stay in contact forever.” Negatively framed message (to be avoided): “Avoid listening to loud music or lose your hearing permanently” Message coinciding with key events: “You don’t want to hear bells chime before Christmas. Constant ringing in your ears may be an early sign of hearing loss. Keep volume levels low to prevent it.” (This can be customized to be relevant based on cultural context.) 24 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Framing of messages Research undertaken during the development of the mSafelistening message library suggests that messages should begin with a “hook”, i.e. a statement which draws the attention of the target audience. This needs to be something that is relevant and understandable to them and will engage them immediately. The hook statement should be followed by a “fact” i.e. information with which to educate the audience. The message then ends with a “call to action” – an alternative, acceptable safe listening practice that the target audience can undertake as soon as they know the information. Examples of message framing is provided in Box 9. Given the profile of the target group, highlighting short-term benefits of adopting safe listening behaviour is more efficacious than pointing out long-term gain. Box 9: Examples of framed messages Hook Fact Call to action 1. Example of a message pointing out hook, fact and call to action: Hook: “Certain songs are timeless, Fact: but your hearing isn’t. Hearing loss is slow to develop and permanent. Call to action: Turn the volume down today to save your hearing for tomorrow.” 2. Example of a message with short term gain: “Would you consider not wearing sunscreen at the beach? Your ears need everyday protection as well. Lower the volume on your phone to 70% to let them relax.” 3. Example of a statement signifying long term gain: “Love listening to your favourite music at maximum volume? Turn it down to 70% of the device maximum volume to enjoy your favourites even when you’re 70.” 3. Content development and adaptation 25 Scientific accuracy It is essential to maintain the scientific accuracy of a message while making it relevant to local context. All messages during and after adaptation should thus be reviewed by the expert group prior to finalization. Figure 5: Overview of the suggested adaptation process 1. Translation 2a. Local expert review 2b. Focus group review 2c. Finalizing the content 2d. Ecological testing 3. Optional WHO content verification 4. Evaluate and update • Small group test • Real world pilot 26 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 3.3.2. Steps in adaptation of message library The following process can provide a user-centred, cost- and time- balanced approach to adaptation. Step 1. Translation The first step of adaptation is translation. Sensitive translation is important for the messages to be clear and engaging. The following should be considered when translating programme content: Which languages the programme will be available in? Experience from running BHBM programmes and feedback from users shows that some users did not engage because the programme was not delivered in a language accessible to them. For example, implementors in Tunisia quickly realized that for the programme to be taken up it must be available in Arabic as well as French. In India, many languages are spoken, and the team learned that having at least Hindi in addition to English was crucial to obtain more subscriptions. In countries where multiple dialects are used, it is important to ask the following questions: • Which dialect will be acceptable to the most people? • Is it necessary to have the programme provided in more than one dialect? • Is it necessary to have the programme provided in voice only (through interactive voice response or voice notes, because some popular languages are spoken and not written)? • Consider spoken versus written or classical forms of the language – for example, would writing in the spoken style of Arabic be more engaging for users than classical, written Arabic? Verification of translation: blind back translation Once the content library has been translated from English into the local language, the translation should then be back-translated into English by a different translator (one who has not seen the original English content) to check its accuracy. This does not need to be a professional translator, but preferably someone who is bilingual. If there are discrepancies between the original and the back-translated English versions, the two translators and the implementing team can discuss the best alternatives to give a sensitive and accurate result. 3. Content development and adaptation 27 Step 2a. Local expert review Local experts in ear and hearing care, including academics, public health experts, and health communications specialists or behavioural scientists, should be invited to send to review the translated materials. By acknowledging these experts in the programme reporting and dissemination materials, they may complete the review cost-free. If resources are available, host an initial adaptation workshop in person or online for experts to meet and discuss their review. These specialists should have a background in behavioural change or health promotion in the field of hearing loss, or have experience in working with people who have, or are at risk of, hearing loss. During this review it is important to ask if additional content should be developed. The review, once collated and incorporated, should provide a draft local content library. The next step is to validate the expert-crafted content to the target population. Step 2b. Focus group review Using focus groups for testing is resource intensive, but many BHBM countries have valued this step highly and found that it merits the investment. This testing phase may include a small number of in-person focus groups with target-users and parents. If necessary, some focus groups could be substituted by telephone interviews. The material for testing with target-users should: • be selected based on the findings of the expert review (in cases of discrepancies or content highlighted for user adaptation); • comprise a mixture of messages (e.g. messages that target different behaviours such as listening to loud music over headphones, or frequenting entertainment venues which play music, sign up and evaluation messages); • reflect likely cultural preferences (e.g. substituting for local preferences of listening and genres of music); and • include key messages most likely to lead to user-desired outcomes. (If needed, more guidance can be obtained by contacting the BHBM team.) 28 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Establishing focus groups for testing Focus groups can be recruited through community message boards, school bulletins or announcements, local community events, or social media outreach. Remunerating the time of the participants is a consideration if possible. For reasons of ethics, informed consent must be sought prior to participation. Depending on the length of the programme, the proposed content can be discussed in one or two two- hour sittings. Content of focus group testing During the focus group, the participants should be asked to comment on the messages with respect to their: • Understandability – are the concepts and terminology used in the programme understood by users? • Acceptability – are the messages received by users with approval and acceptance; i.e. are they inoffensive and respectful of, and sensitive to, the local context? • Relevance – are all the messages necessary or applicable to the environment and context of the user? Will people engage with the messages? Are there themes that are missing and that should be included? It is important to take advantage of the convened focus groups to ask other questions regarding: • Timing of the messages – e.g. which day and what time of day would they prefer to receive messages. • Frequency of messages – whether the planned frequency (or expected engagement in the case of apps or chatbots) is appropriate. • Proposed channels – e.g. the choice of SMS or other messaging platforms. 3. Content development and adaptation 29 Step 2c. Finalizing the content After gathering the data and suggestions from the focus groups, it will be necessary to decide what should or should not be adapted. This can be done by convening the project team and expert group in a small workshop where adaptations can be discussed. It is important to document what has been changed and why; this recorded information may prove very useful for future adaptations and be of significance to the BHBM secretariat in helping improve the mSafeListening content library and understanding more about its global relevance and use. Additional Content development The expert group review and focus group testing may indicate the need for developing content in addition to what is contained in the WHO mSafeListening message library. Questions that may be considered by the expert group to determine if additional content is needed include: 1. Are there any groups or subpopulations in your country that are not served by adapting existing BHBM content? 2. Are there any misconceptions or false beliefs around hearing loss or listening practices that are specific to your target population and may warrant additional programme content? 3. Is hearing loss highly stigmatized in your country? If additional content is needed, explore if potential content exists from other existing health communications campaigns. If not, try to gather as much information as possible – including the opinions of specialists in the particular topic – to create the new content. It will be necessary to gather opinions of specialists in the health topic area to write new content. The BHBM team can also help with additional content development (email: bhbm@who.int). 30 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Points to consider while adapting or developing content Any additional content or adaptation of existing messages must be evidence-based and have the agreement of experts and users. Moreover, when adapting content or creating new content it is important to write with the following in mind so that messages, app content or chatbot scripts are understandable, acceptable and relevant to the users: • Ensure that the language, tone and clarity of the health messages are appropriate. • Consider the health literacy and technological literacy level of target- users. • Ensure the provision of evidence-based information; concrete instruction for self-managed behaviour change; reminders and motivational content. • Consider and potentially tailor content for specific groups (e.g. healthy populations; rural or urban populations; socioeconomic status; ethnic group age; gender) • Avoid an alarmist tone or negative-framing to behavioural change messages. If a negative frame is used, be sure to provide also an instruction, solution or hopeful statement to avoid causing anxiety and a feeling of disempowerment in the user • Include an “active” component or an “ask” (e.g. women are more likely to act on messages when given a concisely written true statement and then asked to act on it; examples include: “Remember to use your earplugs”; or “Check your hearing now”). • Consider the number of characters per message allowed in each country, or the data implications of sending the content (especially applicable to images and videos). “Dos” and “Don’ts” for developing mSafeListening messages were identified during workshops held with the target population and are listed in Box 10. 3. Content development and adaptation 31 Box 10: Adaptation or development of mSafeListening messages: “Dos” and “Don’ts” DOs • Adopt a light-hearted, positive and warm tone. • Use informal/colloquial language. This feels warmer and less clinical and helps the audience connect with the message; but it should not seem patronising and overuse should be avoided. • Use emojis ONLY if they are relevant/relative to the message – e.g. an emoji of weights in a message focusing on music at the gym. The emoji can help with a positive tone but must be relevant. • Use relatable comparisons for noise levels (e.g. traffic noise, household items etc.). • Give receivers autonomy. For example, messages that say, “Try this” are better received than messages that shock, scare and/or give orders to users. The message: “Turn the volume down today or lose your hearing tomorrow” is better framed as “Turn the volume down today to save your hearing for tomorrow”. • Use search term suggestions rather than links – e.g. “try searching safe listening”. • Start with a question to “hook” the receiver (see 3.3.1). DON’Ts • Don’t use humour –humour is very personal; it is difficult to get it right and at worst it can disengage or alienate receivers if it is viewed as inappropriate (e.g. receivers being annoyed to get a message with a flippant tone about something so serious). • Don’t use a patronizing tone. • Don’t use slang words – these are felt to be “trying too hard” • Don’t use unnecessary emojis. • Don’t use unrelatable comparisons – for example, humorous comparisons that a receiver would not have actually encountered in daily life, and which therefore would not be helpful. A message such as “Would you consider not wearing sunscreen at the beach? Your ears need everyday protection as well. Lower the volume on your phone to 70% to let them relax” would not be relevant in a place where people are unlikely to visit a beach or do not have access to sunscreen. • Don’t use technical or medical language (e.g. tinnitus) as this can be alienating. • Don’t over-use questions/rhetorical questions; this is perceived as too “gimmicky”. • Don’t use hyperlinks, as these appear to be a scam message. • Avoid a commercial/sales-like tone as this can feel like an advertisement, and some people would ignore a message given in this way. 32 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Adapting content library for voice, messenger apps or chatbots The mSafeListening content library is in an SMS format as this is the most equitable mode of disseminating mobile messaging. This can easily be adapted to free-phone voice messages, which are a good way to communicate messages to low-literacy or visually impaired populations. However, if the situational assessment and inputs from target-users of the programme suggests that it may be impactful to provide the programme through smartphone apps in addition to SMS, the content library can be adapted to messenger app (as normal instant messages or in chatbot format) or as a stand-alone app. (See Annex 5 for guidelines and considerations on adapting the content library for different messaging formats.) Step 2d. Ecological testing Ecological testing involves testing the programme content in a way that mimics the conditions in which the programme will ultimately be received. For example, it is important to note that receiving a message on a mobile phone during a normal day is different from sitting down to read a list of messages on paper or in a presentation in a focus group. While focus groups are necessary prior to the launch, they may not be sufficient: additional small group and “real-world” consumer testing – although time- and resource-consuming – is important and must be conducted, if available resources permit. Small group testing It is recommended to further test the sign-up process and the programme messages (or app or bot) for 1–2 weeks with a small group of participants (approximately 15), and ask them to rate each message (or session) immediately as they receive it by providing feedback on the acceptability and helpfulness of the message. Examples include: “How did you find the sign-up process on a scale of 1 (being very easy) to 5 (being very difficult)?”. “Please estimate how long (in minutes) it took you to sign up (approximately). Please reply with the number of minutes”. “How much did you like the message (this exercise or exchange in the case of apps or bots)?”; “How helpful was this on a scale of 1 to 5?”; or “How likely would you be to implement the suggested advice or instruction in the message, on a scale of 1 to 5?”. 3. Content development and adaptation 33 Real-world pilot testing This involves “real-world” consumer testing. Once the messages are adapted and programme parameters defined, the programme is administered live to a group of participants as part of a soft launch or pilot study. This looks very similar to the scale programme, with the same dose and frequency of messages (or interactions); however if resources are restricted, duration will be a shorter time period, for example the first 2 months of the programme instead of the full 6 months. Participants are surveyed periodically over the telephone during the course of the pilot, mainly to determine how understandable, acceptable and relevant the messages are, but also how likely they would be to engage in the suggested behavioural change. The results are used to further refine the programme. Step 3. Optional fidelity-checking the adapted content At BHBM, members of the expert group who contributed to the original WHO content library are available for reviewing the (back translated, English language version) of the content library before widespread use in individual countries. Content library can be sent to bhbm@who.int and the team will arrange for a scientific review by the expert group. A three-week time frame is needed for this review. Step 4. Evaluation and updating Many countries choose to carry out a small soft launch/pilot of the programme as part of real-world consumer-testing. This process can also test the appropriateness of the messages as well as the evaluation mechanism and indicators. This feedback can be used to further adapt the content library, even after implementation has begun. Finally, the national TAG must finalize and agree on a plan (including the person responsible) for maintaining the database and programme content (messages or chatbot scripts for example) for future updates. 34 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 4. Promotion, participation and retention Promotion, participation and retention 4.1. Promoting the mSafeListening programme 35 4.2. Participation in the mSafeListening programme 37 4.3. Retention 39 4 4. Promotion, participation and retention 35 4.1. Promoting the mSafeListening programme It is essential to promote the mSafeListening programme so that potential users know about it and can subscribe easily and conveniently. Without users signing up, the programme will be obsolete. A nationwide or population-specific strategy to promote outreach and recruitment into the programme can be a potentially expensive component and should be considered carefully and early on in the planning stages. 4.1.1. Key messages for promoting mSafeListening It is important that the key promotional messages of the programme are underscored by the overall vision of the WHO “Make Listening Safe” initiative. Promotion should clearly recognize that the aim of the programme is to promote safe listening and not to stop music listening. It is essential to highlight that the mSafeListening programme does not interfere with fundamental rights to listen to loud music, but rather enhances the freedom to enjoy the music and other sounds across the life course, without facing the risk of noise-induced hearing loss. Annex 6 provides further information on how sounds effect ears and levels of common sounds, and on how to download an infographic comparing the decibel levels, duration and common sounds. WHO initiative “Make Listening Safe”: the vision “All people, irrespective of age, should be able to enjoy listening (to whatever content they wish to listen to) without putting their hearing at risk.” 36 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 4.1.2. Considerations for promotion of mSafeListening Planning, and designating financial resources, for effective promotion at the outset is essential. Most BHBM programme implementors fed back that they should have used multiple engagement channels, and campaigned more regularly to promote the programme. They suggested using multiple media for promotion activities prior to launch, and that social media was a useful channel, as were SMS and posters, (advertising via radio or television was less useful). Important factors to consider include: • Key promoters that can help reach the target population: these may include telecommunications companies, NGOs, social media influencers, teachers, parents, and any other end-user facing groups with an interest in health promotion. • The most suitable channels to reach the target population: these include television, radio, social media advertising, messenger apps. • Involvement of notable personalities in the promotion drive: these may be well-known personalities who are involved in hearing health related activities. It may be suitable to seek out music personalities, especially those who have experienced hearing loss. • Established events/days that can be leveraged: it may be suitable to launch the recruitment drive on World Hearing Day (3 March), International Day of Music (1 October) or national days of similar relevance. • Clarity of the promotional materials: a potential user should know who the programme is for, have all relevant instructions for signing up and know how to do this having seen the promotional materials. • Source of the promotional messages: messages are most effective when they are sent from a trusted authority such as the Ministry of Health or other well-respected entities. Box 11 provides some insight into sources that are considered reliable by the target population. • Campaign website/webpage: should provide all background information and ideally be hosted on the Ministry of Health website. • User-testing: it is advisable to pre-test the promotional messages and materials prior to their launch. • Compliance with local regulations: this must be ensured. Annex 7 summarizes the important considerations, learning points and suggestions for mSafeListening based on experiences from other BHBM programmes. 4. Promotion, participation and retention 37 Box 11: Sources of reliable information Inputs gathered from the target population indicate that the following would be acceptable as sources of mSafeListening promotional information and messages: • A messenger that is perceived as relevant to the topic such as health services (e.g. Ministry of Health or national health service, doctor’s surgery, private health care, WHO) or a non-medical health and fitness related organization (e.g. a gym, community centre or sports club) • Non-health related messengers can also be appropriate, as long as they are trusted by participants; for example, a club that a participant belongs to, such as a football association, girl guides, etc. would be seen as credible, even if not explicitly related to health. • Messengers that are perceived as being hypocritical are to be avoided at all costs; these annoy participants and potentially motivate them to ignore advice. For example, young people do not want to be receiving messages from concert venues informing that the high volume inside the venue could be dangerous; they perceive this to be responsibility of the venue itself. 4.2. Participation in the mSafeListening programme Option 1: Signing up for the programme By running an effective promotion campaign, several interested users will be ready to interact with the digital messaging platform and sign up. This sign-up process must be: • clear, easy and brief; • balanced, to get all data which is most relevant for the programme, but without being excessively long and tiresome. Ideally, a sign-up process should not take more than a few minutes. • cost-free for users, whenever possible, and should include any associated downloads or replies. • tested by potential users prior to implementation. 38 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Two major barriers to uptake of digital health solutions are costs associated with signing up, and difficulties in signing up. Ideally the programmes should be cost-free, including for replies or for downloading content etc. It is also important to note that BHBM programme evaluations have shown that long or complicated sign- up processes can lose up to 30% of interested persons. The more burdensome or tedious the sign-up, the more likely users will lose interest before they have even started. User-testing can be a good way to gather feedback from target users and revise the process to make it user-friendly. Option 2: Automatic enrolment In some countries it may be possible to automatically enrol certain groups – for example, users of a telecommunications service. In these cases, users can be automatically enrolled, but there should be a cost- free and easy opt-out mechanism if they do not wish to stay in the programme. This approach has been used in some earlier BHBM programmes. In Zambia, for example, all customers of one telecommunications carrier received messages. In India and Sudan, as part of a national inclusive screening programme, users were automatically enrolled in an mDiabetes programme through health services or door-to-door visits when they screened positive for being at risk of diabetes. The legality of such an approach should be carefully considered. Monitoring participation Once interested users have enrolled in the programme, implementors can gauge their continued participation and interest periodically. This can be achieved through messages including questions designed for the purpose of checking participation and monitoring health behavioural change (e.g. “Did your wear your earplugs when going to the club?”. Reply “1” for Yes; “2” for No. Or: “Have you checked your hearing lately? What’s the score?” Reply “1” for 80–100; “2” for 50–80; “3” for below 50; “4” for not checked. Users may be asked to reply to such messages through a reply text message, or respond through an app, or by clicking a link. 4. Promotion, participation and retention 39 4.3. Retention Drop-out rates in many health behaviour change programmes is high, and mobile health programmes are no exception. If users leave and 2-way messaging is cost-free or app users can be contacted, it is useful to ask why they are leaving (consent should be established for contacting participants about drop-out). This information can also be collected via telephone surveys. Some of the common reasons for dropping out of BHBM programmes are summarized in Box 12. Box 12: Known reasons for dropping out of BHBM programmes BHBM programmes have surveyed drop-outs with interesting findings. In Tunisia, for example, 47% of those surveyed left because the programme did not meet their expectation (from what had been advertised); 53% said they left because the guidance in the programme was hard for them to follow; 40% suggested more tailoring of the programme was needed. Another important factor was the messages themselves, with users in India stating messages were not sufficiently motivating. Drop-outs in India and Tunisia fed back that using videos and images would be more effective, and further that combining the mTobaccoCessation messaging programme with other quit services and support would have been beneficial. 40 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 4.3.1. Ways to reduce drop-out In order to reduce the number of people leaving the programme it is important that: • the programme is tailored to the target population so that the messages are most relevant to them; • participants have a smooth user experience. This can be facilitated through careful selection of technology platform and rigorous user- testing of the sign-up process; appropriate programme design (see 3.2); and systematic adaptation and pre-testing of messages (as described in 3.3). • people have access to support for technology-related issues. • two-way messaging is undertaken. This can be to seek user feedback or check on user progress, as mentioned above. • user feedback is acted upon: feedback received from users should be reviewed during the course of the programme implementation, along with reasons for dropping out. Learnings from these should be used to review and refine the programme design and message library. 4.3.2. Opt-out option “STOP” messages are often included in SMS or other text-based programmes to enable the user to stop receiving messages (e.g. “Reply ‘STOP’ if you wish to stop receiving mSafeListening messages”). It is an ethical imperative that users can unsubscribe or stop receiving messages if they wish, and it is important in any BHBM programme that they know how to do this. However, literature suggests that programmes with more frequent “STOP” messages have higher drop- out rates (40). Sending two stop messages is recommended: one near the start of the programme and another mid-programme. Figure 6 provides an overview of the process of gaining, monitoring and evaluating participation in the programme. 4. Promotion, participation and retention 41 Figure 6: Participation, retention, monitoring and evaluation process Enrolment Sign up or auntomatic enrolment Customization of messages Based on individual’s preferences and usage profile Regular monitoring Through messages that require a response and help determine user’s listening behaviour In programme feedback If feasible, administration of a questionnaire to get feedback on programme design and content, to modify approach Evaluation End of programme: evaluation and feedback for future review and programme revision In case of premature exit: to understand reasons for quitting and consider these during future revisions 42 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening5. Technology specifications Technology specifications 5.1. Selecting the appropriate technology for the context 43 5.2 Technology implementation needs 49 5.3 Software needs 50 5 5. Technology specifications 43 This section looks at how to select and implement the best technology to deliver an effective mSafeListening programme. When developing a technology plan for mSafeListening, it is helpful to onboard all available ICT and software expertise from the Ministry of ICT partner and any digital health and IT specialists in the MoH and other relevant ministries. If the mSafeListening programme is to be embedded within a wider, existing digital health platform it is important to understand the platform and how the programme will fit within it. If there is no digital health platform in the country, gaining information and advocating to set one up may be worthwhile. For further details, see the Digital health platform handbook for health: building a digital information infrastructure (infostructure) for health. 5.1. Selecting the appropriate technology for the context To define and a digital health messaging programme as appropriate to the context and target users, the following aspects must be considered by the national TAG and any other advisors involved from the outset, in collaboration with local partners. Many of these aspects feature in the considerations for a needs assessment, and are marked with an asterisk (*). 44 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 1. Key functions that the technology needs to perform, focusing on the experience of the end-user (e.g. using a health (text) message delivery system to send automated messages according to an algorithm with a pre-defined frequency). It is important to consider how users access the programme and how often they will interact with it. 2. Current and predicted use and uptake of mobile technologies and communications* for health in the target population both from the public and private sectors. It is important to consider the preferences and user ratings of the target population and the availability and sustainability of these technology options within the public sector. (This information may have been gathered in the needs assessment.) 3. Outcomes of market research: for example which telecommunications system is most appropriate in the country, based on reach (subscribers), coverage, costs, security, and sustainability. 4. Equity of access to different technologies* or, for example, technologically-disadvantaged groups (SMS or IVRS are likely be most equitable), reach and access to different language versions. 5. Messaging formats within the parameters of the chosen technology. For example, should voice messages, video messages, images, GIFs, interactive messaging be used? What is the capacity for reach of these features and content and their cost in the country and on different platforms? 6. Ensuring that the programme is cost-free and available to all consumers regardless of their device, carrier, network or location. Could the data cost of initial download of an app or the receipt of Whatsapp messages be waived by the telecommunications provider, for example? 7. Ethical and regulatory data considerations and specifications to ensure that data are handled sensitively, to protect human rights and personal safety. Using a need-to-know principle, what data collection is necessary? Who owns the data? Where are they hosted and how robust is the security of the host? What are the privacy regulations and how will data be protected and kept secure? What are the considerations for data protection and how should a central database best be maintained? 5. Technology specifications 45 8. Conducting monitoring and evaluation: for example, how can user data be used to report on the key performance indicators (see blue indicators in Annex 8: Monitoring and evaluation indicators)? What frequency and which indicators should be used? How will the system produce reports and present data from users? How can the design of a dashboard enable data presentation? Can these data be made interoperable with existing health information systems? Is periodic reporting of aggregated analytics data possible? 9. Interoperability considerations: for example, are there existing health system technologies with which the programme must communicate? If there is another health messaging programme running, can some or all of the same infrastructure be used? If health workers are “prescribing” the programme, can this be recorded in the health record system, patient data system or health insurance billing mechanism? Can the health record system receive data such as on behaviour change that the user is reporting to the messaging platform? 10. Sustainability factors: for example, ongoing operating costs of the programme maintenance, costs for users such as per message or unit of data, and how will these affect the scale of the programme? 11. Contractual arrangements with partners: for example the considerations regarding intellectual property, security and privacy of mobile phone numbers, testing, expectations of involvement in monitoring and evaluation and new and outstanding service agreements. Who will hold the contractual arrangements, and what support will be given for maintenance and any other problems? These considerations and the list of simplified strengths and weakness for different technology channels described in Table 1 are designed to help with selecting the ideal technology or channel for the mSafeListening programme. It is essential to consider the technology literacy and accessibility levels of the target population when selecting the technology channel; for example, older users may be less familiar with, or not have as much access to, newer technologies. 46 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Table 1: Technology options and applications: strengths and weaknesses Description Strengths Weaknesses IVR (Interactive Voice Response) IVR is an automated phone system that interacts with users using prerecorded voice responses. Users can respond using touch tone keypad selection or via preset/ valid vocal responses (e.g. “yes”, “no”). • Voice- and phone- enabled access. • Fast time-to-market. • Supports natural language. • Ease of integration. • Accessible to those with feature phones. • Limited capability and development tools. • Inability to pause, resume, forward and rewind. • 2-way communication can be prone to malfunction due to misinterpretation of voice inputs. SMS (Short Message Service) SMS allows short text messages to be exchanged between mobile phones. • Simple, easy and convenient. • Can negotiate cost– effective delivery. • Private communications. • Fast communications. • Accessible to those with feature phones. • Some security vulnerabilities. • Fake SMS (spoofing) can result in trust issues. • 2-way messaging limited to simple interactions. • May be costly if cost– effective delivery cannot be negotiated. USSD (Unstructured Supplementary Service Data) USSD uses alphanumeric codes to exchange information with a server in real- time (e.g. user can use a code to check account balance and add money to account without need of an internet connection. • Simple and logical. • Real-time, fast and responsive. • Inexpensive. • Interactive navigation. • Session-based timeouts. • Codes more difficult to remember than Common Short Codes. 5. Technology specifications 47 Description Strengths Weaknesses MMS (Multimedia Messaging Service) MMS extends SMS technology by allowing the exchange of a variety of media (images, audio, etc.). • Direct and personal. • Messages can be stored and forwarded. • Interactivity through multimedia. • Not compatible with basic phones. • More expensive than SMS. • Content adaptation limited by screen size and resolution variations. • Read and response rates lower than SMS. Existing messenger services These include apps and platforms that enable instant messaging via an internet connection (e.g. WhatsApp, Facebook, Messenger). • Low cost. • High usage. • Increased interactivity and engagement. • Maintained by the app provider. • Allows sending of graphics and videos. • Can deploy conversational agents or chatbots (where responses are tailored to users’ inputs). • Can be simple or elaborate (natural language processing and artificial intelligence). • Can carry an avatar or visual identity. • Third party private sector involvement or deployment software may be necessary (e.g. to set up and manage back-end functions). • Potential data costs for end user to receive content. • With conversational agents (if using artificial intelligence capabilities), can require data and training before launch. Also, bugs or bot miscomprehension of inputs can be dissatisfying and potentially risky. 48 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Description Strengths Weaknesses Smartphone applications Software/programme that runs on a mobile device that commonly needs to be downloaded and installed before use. • Self-contained experience. • Graphics and videos easily integrated. • User-generated content and data input. • Automatic updates and read content offline. • Leverages device- native capabilities (camera, GPS, step counter). • Can deploy conversational agents (for tailored 2-way messaging). • Need to build for multiple platforms, involving time and high cost. • Managing multiple releases/updates. • Sensitive to users’ device changes or operations. • Need to submit to app stores for approval. • High user drop-out rates. • Initial data required for download can be costly for end user. • Often requires 3G or 4G coverage. • Only compatible with two platforms (iOS and Android). Mobile website A mobile website/ webpage designed specifically for mobile device access. • Cheaper to develop and maintain. • Supports mobile phones and smartphones. • Mobility for content and services. • Videos and graphics easily integrated. • Less functionality, unable to use advanced phone features such as camera, GPS. • Small display size • Low bandwidth affects functionality. 5. Technology specifications 49 5.2. Technology implementation needs After selecting the technology to be used, it will be necessary to identify further technology needs for implementation; these include: • Identification of process for procurement, adaptation and maintenance of the selected technology. • Dashboard design and development and access needs (consider M&E indicators for dashboard development). (Questions to be considered are: What monitoring and success indicators should the dashboard present? Who should have access to the dashboard.) • Procurement of a short code (if using SMS or telephone networks for IVRS). • Data security needs. • Pre-testing and scale-up needs. Some of these needs may be apparent from the situational assessment; if not, further research will be necessary. This extra research is worth the investment: revising developed software due to unforeseen needs can be very costly. These considerations and the following list of simplified pros and cons will help in selecting the technology or channels to be used, for example: SMS; MMS; voice over internet (VOIP)/interactive voice response (IVRS); existing messenger apps (e.g. Whatsapp, Facebook Messenger or local providers); or purpose-built apps. 50 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 5.3. Software needs A service delivery platform is necessary for running a digital health messaging programme to ensure that the programme works smoothly for users and is seamlessly integrated with the mobile telecommunications network. Such a platform will have different capabilities and features depending on the technology chosen to deliver the content (SMS, messenger services, standalone smartphone app). The choice of service delivery platform may depend on the wider digital health landscape in the country and what other platforms are already being used. Based on requirements, it will be necessary to identify whether an existing or “off-the-shelf” platform is appropriate, or whether a customized solution (designed and built for the programme) is best. It is also important to make sure the content shared does not violate any data sharing policies (some social media platforms are very strict on what data are collected or shared in the medical field on their platform). The first step is to prepare a list of things the system is required to do in order to run the programme. This is not a technical list, simply what is needed for the system to do. As an example, imagine that the research with target users (along with the consideration of the strengths and weaknesses listed in Table 4) has shown that a messenger app delivery mechanism using a simple bot will be most appropriate, and that the target-users mainly use Facebook. This means the service delivery platform may have the following requirements: • A simple Messenger for Facebook conversational agent. • The capability to reach anyone with a Facebook account. • The capacity for millions of users as part of the national programme. • The need to run in three different languages. • An easy sign-up procedure. • The ability to deliver messages according to an algorithm. • The capability to allow the user to interact using buttons only (no artificial intelligence or language processing involved). • A system able to notify users of new content. • The ability to allow data to be stored by the programme servers. • The need for technical and maintenance support with the aim to transfer this to the Ministry of Health IT team after Year 2. 5. Technology specifications 51 Once the requirements have been outlined, the type of software needed will require consideration. Looking to other service providers or countries running similar digital client communication messaging programmes may be helpful to see what software they are using, the challenges faced and lessons learned. See Annex 9 (taken from WHO’s toolkit for planning an information system (41)) for assistance in choosing whether to use an existing solution or to develop one that is customized. Consider the pros and cons of each option, mapping the technology requirements to the capabilities of the platforms. It may be important, furthermore, to investigate the programming needs (should any adaptation or integration with other systems be needed), the level of ongoing support available for each option, and the costs. In some cases the following considerations will be necessary when making a decision on software: • How will the software integrate with the mobile telecom environment. Will it work across different mobile network operators (MNOs)? • What partnerships will need to be developed to activate the service (for example, partnerships with aggregators, MNOs, mobile gateway providers)? • How will the platform be able to adapt to changes and advances in technology? • Based on the chosen platform, will the project need new or additional hardware (computers or a server to run the programme)? • Interoperability and licensing issues (e.g. if the programme is to be “prescribed” to health service users; and how the use of the programme – and patient outcomes associated with that use – may be integrated and recorded in the health information system). 52 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Selecting a software provider Existing processes and procedures for the procurement of services may well be in place in your organization and a request for proposals (RFP) is likely to be issued. It is important to start by inputting the background information compiled (see checklist in Box 13) to the RFP, ensuring the goals, values, and desired outcomes for the programme are set out. Working with service providers whose values align with those of the programme will help with relationship management. Researching service providers and having dialogue or interviews with these prior to selection can help to know if values are aligned. The next step is to use the list of requirements created to build the RFP. If there is no software specialist on your team, software providers can suggest the more technical specifications to meet your requirements. The BHBM country support team can also assist with writing the RFP. It is important to consider software providers that have implemented similar solutions at scale (ask to see audit results). Ensure providers can deal with demand for the programme and maintenance of their service (e.g. ask what maintenance activities are included in the fee). It is essential that there is a clear understanding of which components of the software will be proprietary (including a license to the mSafeListening Programme owner if possible); aim for open-source components wherever possible. When selecting the company, the desired option may be to design or adapt an existing scoring matrix to help standardize any contracting decisions made. (See Annex 6 of the WHO planning an information systems project toolkit (41) for a comprehensive scoring matrix that can be adapted for purpose.) 5. Technology specifications 53 The role of telecommunications operators It is important to note that mobile communications network environments differ from country to country. The specificities of end-user access to SMS, calls or mobile data (for stand-alone or messenger apps) should be considered in the planning stage by the inclusion of technical experts in the TAG (such as representatives of telecommunication companies; operators; telecommunications regulatory authorities; government departments responsible for information; communications, technology, and cellular associations) or individuals knowledgeable about the communications network of the country. Network operators, telecommunications companies or industry organizations can provide assistance in setting up the programme and advising on its suitability and sustainability. Certain providers may view supporting such a programme as good publicity or a useful addition to the services they offer. This can be beneficial in negotiations. Before inviting provider involvement in technology specification, it is worth considering what sort of arrangement with telecommunications companies will best suit the long-term implementation of the programme. Other considerations may include what other partnerships could be useful or necessary; or what the parameters of negotiation are with telecommunications regulators, aggregators and operators for the pricing of message dissemination. Box 13: A checklist for considerations for technology specification • Identify programme infrastructure and regulation considerations. • Select technology. • Identify software needs. • Procure technology. • Develop dashboard and enable access for monitoring and reporting. • Procure a shortcode if necessary. • Negotiate with telecommunications regulators, aggregators and operators for pricing to ensure service can be used free of charge by users. • User test the technology and registration process. • Develop data security and interoperability standards. • Pre-test technology. 54 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Negotiating with telecommunications operators Engaging in and maintaining strong partnerships with telecommunications operators is critical for mSafeListening programme implementation, and for any eventual reduction in the cost of the programme; BHBM has found that enrolment tends to be very low when consumers have to pay for it. A major barrier to two- way messaging programmes is the reply cost for participants; one programme had a 30% reply rate only due to costs for replying. The more the cost of engagement can be reduced for users, the more successful a two-way programme will be in terms of user activity. (It is important to note that mSafeListening includes mainly one-way messaging; two-way messaging is to be used mainly for monitoring purposes). If using SMS, there should be no fee to receive or reply to programme messages; with messenger apps, it will be necessary to waive data costs associated with receipt and reply to messages. If the choice is to build a standalone mSafeListening app, the initial download must be cost-free (both price of the app and the data required for its download); subsequent information exchange should also be cost-free. In all cases, user data must be private and secure. To this end it may be helpful for teams to include members with experience in operator engagement and negotiation; if necessary the BHBM country support team can help with this. The ITU (regional office) can act as a bridge to facilitate the negotiations between the Ministry of Health, Telecommunications Authority and the Telecommunication Companies. (A template for a telecommunications operator agreement is available on request from bhbm@who.int. WHO has also released a comprehensive guide to negotiating with mobile operators (this guide relates to digital health messaging programmes for reproductive, maternal, newborn and child health, but its principles are relevant across health issues). The objectives of the negotiations are to reduce or cut costs associated with the programmes, especially the costs that fall on the intended user of the programme; make the programme as problem-free as possible for the end-users; and ensure protection and privacy of their data. 5. Technology specifications 55 Tips for negotiating include: • Involve an IT expert in the consultation, who has technical knowledge of the platform and software being used to respond to technical questions or discussions. • Share the values and vision with those of the telecommunications company, highlighting where values match. • Estimate the programme’s intended user numbers before commencing negotiations with telecommunications operators – this will help assess the scale of contribution they need to make. • The negotiators must be aware of the current costs of services, costs of packages and sliding rates which are vital for the negotiations. • Ensure the project is jointly shared by both the Ministry of Health and the Ministry of ICT: in some countries, the Ministry of ICT may have an established relationship with several mobile network operators and may be in a stronger position to negotiate (especially in the case of national public network agencies). • Ahead of negotiations, hold consultations with relevant authorities (e.g. telecommunications authority, national ministries and market regulators) to identify and understand what benefits/privileges can be granted to telecommunications operators in return for their collaboration (see Box 14). 56 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Box 14: Ideas for incentivizing telecommunication operators’ buy-in • Direct benefits (e.g. a small tax reduction) to the operators from the Ministry of Information Communications Technology (MoICT). • Offering an access point into a new market by understanding their service structure and user experiences. • Within the broader programme timeline, operators can use their growing experience to develop their independent text messaging for health portfolios, nationally or internationally, driven by rising national demand. • Showing effectiveness of text messaging for health services could offer operators a new source of future revenue in value-added health services. • Given that in most countries the telecommunications market consists of two or three major providers who have more or less similar subscription plans, a telecommunications operator can distinguish itself significantly from others by demonstrating its ability to offer additional benefits to its customers. • Early-mover advantage: knowledge transfer. Operators need to learn how to run large-scale public health programmes. Participation in the programme will maximize quantity and quality of knowledge in comparison to competition. • Good public relations: Good visibility as a socially responsible company can showcase the company’s contribution to public well-being. The Ministry of Health (MoH) must ensure that it offers telecommunicatins companies the options for this visibility, including: promotion in mobile stores with MoH logo, mobile operator office, website and public campaigns. • Good working relationship with the MoH (and possibly with the Telecommunications Authority). • Operators may need support with their own interests in the digital health, mobile money or mobile health insurance fields and could be working on common areas with the MoICT or MoH. Telecommunication companies can be invited to identify priority areas where the government may consider providing them support in the future. 5. Technology specifications 57 In the absence of telecommunications provider support and in the case of an SMS or IVR programme, the mSafeListening programme can be delivered through a contractual arrangement with an “aggregator” or “gateway” company that has established relations with all telecommunications companies and networks. This can be a cost– effective way to deliver messages to many participants, regardless of their mobile carrier or location, without establishing these interfaces individually. Although the aggregator adds a further cost, this cost decreases as the scale of the programme increases. Using an aggregator can therefore be more cost–effective than attempting these activities “in-house” unless capacity and infrastructure already exist (see Annex 10 for a further explanation of the role of the aggregator). A checklist for considerations for technology specifications can be found in Box 13 under “Selecting a software provider”. 58 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening6. Monitoring and evaluation of mSafeListening Monitoring and evaluation of mSafeListening 6.1. Planning for M&E 59 6 6. Monitoring and evaluation of mSafeListening 59 Monitoring and evaluation (M&E) of an mSafeListening programme is crucial to assess the impact and effectiveness of the programme. Appropriate monitoring and evaluation can help maximize resources and inform future improvements and expansion. Monitoring is the routine tracking of an intervention’s performance using data collected internally on a regular and ongoing basis on specified indicators. This information is used to assess the extent to which an intervention is achieving its intended targets on time and on budget. Evaluation is an episodic assessment of either a completed or ongoing programme or intervention, to determine the extent to which its stated objectives were achieved efficiently and effectively. M&E also facilitate implementation and up-scaling by providing lessons learned that will enable the introduction of other digital health messaging programmes in the country and inform stakeholders, including those in other countries, about barriers, enablers and effectiveness of the programme. Being able to provide measurable outcomes in terms of quantifiable arguments and demonstrable impacts is an advantage for fundraising. 6.1. Planning for M&E M&E needs should be established, i.e. planned, resourced and integrated, at the very onset of the mSafeListening programme. M&E efforts should focus on: 1. the ability of the programme to reach its target group; 2. the effectiveness of the programme content to generate the desired health behaviour change for safe listening and hearing loss prevention; and 3. assessment of impact of the programme (while this may be considered worthwhile, it is difficult and expensive to make this assessment) This section of the handbook represents a compact guide. Further information and more comprehensive guidance is available in the WHO guide for digital health monitoring and evaluation: https://www.who.int/ reproductivehealth/publications/mhealth/digital-health-interventions/en/ 60 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening An overview of the necessary steps of M&E covered in this section is in Figure 7. Figure 7: Overview of necessary steps for M&E of mSafeListening programme 1. Define outputs and questions to be answered through M&E 3. Plan M&E human resources 5. Design data collection: (sampling requirements, create/adapt data collection tools [surveys, interview/focus group guides) 7. Collect data 2. Develop or adapt an M&E framework 4. Select/adapt M&E indicators and their frequency 6. Prepare M&E budget for HR, data collection and analysis and dissemination costs 8. Analyse and report as per plan and incorporate findings into future activities 6. Monitoring and evaluation of mSafeListening 61 Step 1. Determining the goal of M&E The goal of M&E, including information to be collected and questions to be answered, must be determined at the outset, and must be guided by the goal and objectives of the programme (see 2.1 and 2.2). Questions can relate to: • ease of use and operation of the mSafeListening platform • accessibility and understandability of content for users • effectiveness of marketing campaign in reaching new users • number of people reached and their demographic profile • percentage of completion rate or number of drop-outs • improvement in knowledge and change in behaviour Examples of questions are listed in Box 15. Box 15: Example of questions to be answered through monitoring and evaluation • How many people voluntarily signed up for the programme. What was the motivational factor? • What percentage of those who started the registration process completed it? • What practical issues were identified by users in access to safe listening messages? • Were the messages clear and understandable? Did users find them relevant to their own context? • How many people joined the programme? What is their distribution across age groups and geographical areas? • How many people completed the programme? What is their distribution across age groups and geographical areas? • What were the reasons for non-completion? • What percentage of people reported a desirable change in listening behaviour in terms of: – listening at a lower volume; – a reduction of time spent in pastimes that expose the person to loud sounds; – regular use of earplugs in noisy places? • What percentage of people have checked their hearing? • Is there any change in the hearing status of the participants as compared to a control group? (Impact related question) 62 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Identifying and agreeing upon the questions will help align the frequency and effort of data collection with its importance and cost. More expensive independent evaluations can then be scheduled less frequently to respond to specific questions about impact, as required. Step 2. Creating and adapting an M&E framework After the goals of M&E have been defined, the next step is to create a framework of how to achieve these goals. Further information is available in “Developing an M&E framework” in the WHO guide for digital health monitoring and evaluation, and described in Box 16. A results chain or logic model identifies how resources (or inputs) make it possible to carry out programme activities, which in turn produce a series of results (or outputs and outcomes) and move the programme towards achieving its stated vision (or impact). A logical model framework for mSafeListening is proposed in Figure 8. Box 16: A logic model framework for mSafeListening A logic model is a framework that illustrates the relationship between the inputs, outputs, outcomes and impact of mSafeListening, and enables easy mapping of the programme’s aims. It is a visual concept of how the elements of the digital health intervention influence each other. Inputs are defined as the financial, human, material and intellectual resources used to develop and implement an intervention. Processes or activities are defined as the activities undertaken in the delivery of an intervention and may include training courses or other capacity-building, software or hardware development, adapting content, partnership/negotiation meetings, programme promotion activities etc. Outputs are defined as the direct products/deliverables of process activities in an intervention. Outcomes refer to the intermediate changes that emerge as a result of inputs and processes. Impact can be defined as the medium- to long-term effects produced by an intervention on population health, health systems or other benefits. 6. Monitoring and evaluation of mSafeListening 63 Figure 8: A logical model framework for mSafeListening Input Programme development resources: • Staff • Time • Funding; IT architecture; linking with health services; partnerships • Governance; policy change (if necessary) • Operations and stakeholder management. Impact • Reduction in proportion of target group practicing unsafe listening behaviours • Decrease in the incidence of hearing loss within the target population. Activities • Situational assessment • Content adaptation (incl. user testing) • Programme design • M&E plan and implementation • Technology development (incl. testing) • Negotiation for free/ low-cost messaging • Individuals trained for recruitment • Outreach and promotion. Output • Baseline and needs data • Quality adapted content • Programme specifications • Functioning and integrated message platform • Low-cost/free 2-way messaging • No. of trained individuals for recruitment • No. of and type of promotion activities • Analytics on reach of promotion • No. of users enrolled (reach) • No. of messages sent and received. Outcome • Baseline vs post-data • User satisfaction re: promotion activities • User satisfaction with message programme • User engagement • Technology performance • Knowledge gains • Behaviour change and adoption of safe listening practices • Systemic change in policy. Process monitoring and evaluation Outcome monitoring and evaluation 64 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening A note on impact The overall impact of a digital health messaging programme at scale is its contribution towards the achievement of the Sustainable Development Goals (SDGs), in particular SDG 3: “Ensure healthy lives and promote well-being for all at all ages”, and SDG 9: “Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation”. Measuring the impact of a programme at scale may not be feasible for certain settings. Impact indicators pose difficulty because it is hard to ascertain the effect of the programme on the impact indicator (owing to design options of scaled implementation evaluation), often termed attribution challenges. Impact indicators also entail a significant investment of funds and time. For this reason this handbook focuses on process (activities) and outcome indicators. Step 3. Planning M&E human resources The next step is to assign roles for the completion of M&E based on the skills of the team. If there is no one in the implementing team with experience in M&E, it may be necessary to hire or train someone in M&E design and data collection and analysis. There are two parts to the M&E of a digital health messaging programme that will likely require different skills to manage: process M&E; and outcome evaluation. Process M&E refers to the assessment of the activities and processes related to the implementation of the programme (e.g. IT, resources) and will predominantly require internal data collection activities and may be manageable with existing programme human resources. Outcome evaluation refers to the assessment of the ability of the digital health messaging programme to achieve its target health outcomes (e.g. behaviour change) and will more likely require external human resources or internal human resources who can be field-based or able to reach participants (e.g. run surveys with programme users. 6. Monitoring and evaluation of mSafeListening 65 M&E processes Process monitoring provides information for planning and for feedback about the progress of the project. Inputs and processes are the critical resources that go into developing and implementing an mSafeListening programme. These should be carried out internally and regularly (monthly or quarterly). Monitoring should start at the programme’s inception and a routine reporting mechanism will need to be set up to monitor the core indicators and key deliverables. Monitoring reports and dashboards are helpful in providing a quick overview to see whether a programme is on track to reach its objectives. Process evaluation is the periodic assessment of the implementation of a programme in relation to planned activities and their overall objectives. It identifies the constraints that hinder the programme in achieving its objectives and can help to provide solutions that can then be implemented. A process evaluation tends to be carried out annually and uses a range of data collection methods; these include: • recording of the completion of key activities (e.g. number of users registered or those that dropped out prior to completion of the programme). • service analytics from telecommunications company reporting (e.g. number of messages sent, number of questions posed and percentage of responses received); • conducting focus groups and interviews for in-depth exploration of experiences, attitudes and ideas (e.g. feedback on frequency of messages, user experience etc.). Outcome evaluation refers to the assessment of the ability of the mSafeListening programme to achieve its target health outcomes (i.e. behavioural change). It can be an important investment to analyse and communicate the effectiveness of the programme. Relevant data will be collected mostly through surveys or questions to reach participants to understand how their listening behaviours have changed during and after participation in the programme. Survey questionnaires can include questions such as those initially posed during the recruitment process (see Annex 4: Questionnaires). Outcome evaluation should be undertaken at the end of the programme and can be followed up after a 6- or 12-month interval. The follow-up assessment, if feasible, will help ascertain how the behaviour change has sustained in the target population. 66 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Step 4. Selecting indicators for monitoring mSafeListening The selection of indicators is a crucial part of planning (see Box 17) that evaluates the elements in the framework (logic model) developed in Step 2, and most suitably capture the progress and outcomes of the mSafeListening programme. Indicators should include input, output and outcome indicators. Impact indicators pose difficulty because it is hard to ascertain the effect of the programme alone on the impact indicator (such as prevalence of hearing loss). They also entail a significant investment of funds and time. Annex 8 includes a list of proposed indicators that can provide a useful guide when developing the mSafeListening programme. These are segregated by core and optional indicators. Every mSafeListening programme should routinely collect and report on these core indicators (including to the BHBM secretariat if the programme is supported by BHBM). Each indicator in Annex 8 is presented with comments and a suggested data collection and reporting frequency. This matrix serves as a template and should be adapted to context. In some cases, M&E may require a review and clearance by an ethical review committee. Some lessons learnt from implementation of other digital health messaging programmes in countries are noted in Box 18. Box 17: What is an indicator? WHO defines an indicator as “a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention or to help assess the performance of a development actor” (42). 6. Monitoring and evaluation of mSafeListening 67 Step 5. Design the outcome evaluation and prepare data collection materials While programme monitoring is conducted routinely, programme evaluation is conducted periodically (e.g. every 6 or 12 months) to gauge whether the programme is achieving its objectives and to decide if any adjustments need to be made. Box 18: Lessons learned from BHBM implementing countries • Having a phase 1 implementation on a smaller scale with good evaluation and user feedback is suggested. • It is very important to plan M&E from the outset. • Most BHBM implementors reflected that they should have collected data on more indicators and that all indicators should have been clearly linked to programme aims. • Indicators should be defined before data collection; for example what are the numerator and denominator to use in a tobacco quit rate calculation, or the difference between the total quit rate and the effective quit rate. • Include the reporting dashboard and data for M&E indicators as a deliverable in the terms of reference with any third-party service provider contract. Be specific about what data are needed and when. • Ensure that a mechanism is in place for reporting service outages; plan on how these will be dealt with and potentially compensated for by third party service providers. • SMS response rates are low if replies are not cost-free for the user. 68 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Data collection tools for this evaluation commonly include: • Questionnaires/surveys: A questionnaire-based survey is suggested to assess the self-reported perceptions, behaviours, knowledge and attitudes of registered users. Pre- and post-programme evaluation design is recommended, where the same questionnaire is completed by the participants at the start of the programme and at completion. This includes questions about their listening practices (see Annex 4). Where this is not possible or where resources are limited, a single question sent during or after the programme (e.g. “Since the beginning of the programme, have you reduced the listening volume on your device”; or “Since the beginning of the programme, have you reduced the number of hours spent in listening to loud sounds?”) may provide some information. Surveys can be conducted online over the internet, or on a mobile device; or administered by an interviewer either in person, or by telephone. Since survey fatigue is a common challenge, surveys should be as short as possible and employ multiple-choice questions wherever possible. These can be undertaken during the period of the programme or after the programme is completed. • Response to messages: Questions with multiple response options may be sent to participants during or after the programme. For example: “Do your wear your earplugs when going to the club? Reply ‘1’ for always; ‘2’ for sometimes; ‘3’; for occasionally; ‘4’ for never.” • Focus group discussions: these provide an opportunity for participatory discussions and brainstorming ideas for a programme’s improvement. • Interviews with users can provide more qualitative feedback on their experience, perceptions and satisfaction with the mSafeListening programme. Questions can be similar to the above-mentioned survey questions (Annex 4). 6. Monitoring and evaluation of mSafeListening 69 Step 6. Preparing the M&E budget It is important to dedicate financial resources to M&E. This budget allocation will depend greatly on the scope of the M&E activities and the evaluation design selected. Budget lines will be necessary for the following overarching areas: • Human resources: How many full-time equivalent staff will be needed – and at what pay grade – to carry out M&E activities from design to reporting? • Translation, adaptation and/or development of tools: Existing tools may suffice, or tools may need to be adapted or validated to the specific setting, or new questionnaires created that incorporate questions on all the selected indicators. • Data collection and analysis: This covers all data collection activities, and could include travel required for project staff to collect data; telephone bills to conduct telephone surveys; incentives for respondents; hiring spaces to meet with respondents; and data analysis software licensing if necessary. • Dissemination: This covers publishing costs if findings are to be published, or incorporated into future promotion campaigns. Step 7. Analysing the data Data analysis will be necessary for reporting purposes as well as to inform the viability, impact, continuity and scalability of the programme. It should be planned at the outset and be tailored to respond to the questions determined in step 1. Analysis could be simple percentage calculations (e.g. the percentage of users who have adopted safer listening practices since the start of the programme); or if the quantitative data are sufficiently robust, analysis may include running statistical tests on the data (e.g. to discover if there is a statistical difference in the listening practices of users who completed the programme based on age, gender or other sociodemographic information). 70 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening It may be necessary to speak with someone in the data division of your organization if there are problems in analysing the data gathered (alternatively the BHBM country support team can assist). Analysing the data will allow questions developed in step 1 to be answered. Data analysis should be used to drive adaptations in the programme itself and also be reported to the relevant stakeholders in user-friendly formats. Step 8. Reporting and dissemination Data for M&E should be consistently reported to inform programme implementation in an iterative manner. They should be used to support collaboration and decision-making among stakeholders regarding ongoing resource allocation and processes for the programme’s sustainability and scalability. For example, if the data show irregularities in registration or delivery of messages, or user responses, such issues must be brought to the attention of the concerned decision-makers through regular review meetings. In addition, these data should inform annual process and outcome evaluation reporting to show progress and lessons learned. Data should be presented in a format that is concise and user-friendly and be relevant to the target audience. If there are multiple audiences, such as programme implementers and policy-makers, data need to be presented in line with their respective priorities. Evaluators should generate a list of all relevant stakeholders, such as policy-makers, donors, programme staff, etc. and consider who is most likely to use the data collected from the evaluation, how they might use the information, and the necessary communication style of the report. 6. Monitoring and evaluation of mSafeListening 71 Evaluation findings should be disseminated in a format that is easy to read and in a manner that is accessible and timely manner. Dissemination can be achieved through: • formal and informal networks via meetings; • newsletters and other forums; • professional conferences via discussion papers or posters; • journals (scientific or lay); • electronic media, such as web pages, social media and e-mail; • briefings with policy-makers; and/or • media channels for key stakeholders, e.g. health and social care workers and the general public. For detailed information on M&E of digital health interventions, see Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment (43). This guide provides an introduction to the approaches and methods to support countries in strengthening their digital health deployments, develop robust evaluations, and scale up their activities nationally and regionally. 72 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening References 1. World report on hearing. Geneva: World Health Organization; 2021 (https://apps.who.int/iris/handle/10665/339913, accessed 16 December 2021). 2. Lemke U, Scherpiet S. Oral communication in individuals with hearing impairment considerations regarding attentional, cognitive and social resources. Front Psychol. 2015;6:998. 3. Thiyagarajan JA, Araujo de Carvalho I, Peña-Rosas JP, Chadha S, Mariotti SP, Dua T et al. Redesigning care for older people to preserve physical and mental capacity: WHO guidelines on community-level interventions in integrated care. PLoS Med. 2019;16(10):e1002948. 4. Russ SA, Tremblay K, Halfon N, Davis A. A life course approach to hearing health. Handbook of life course health development. Springer, Cham; 2018. p.349–73. 5. Childhood hearing loss: strategies for prevention and care. Geneva: World Health Organization; 2016 (https://apps.who.int/iris/ handle/10665/204632, accessed 16 December 2021). 6. Daniel E. Noise and hearing loss: a review. J Sch Health. 2007;77(5):225–31. 7. Make Listening Safe. World Health Organization; 2015 (https://cdn.who.int/media/docs/default-source/documents/health- topics/deafness-and-hearing-loss/mls-brochure-english-2021. pdf?sfvrsn=bf19b448_5, accessed 21 December 2021). 8. Liberman M. Noise-induced and age-related hearing loss: new perspectives and potential therapies. F1000Research. 2017;16;6:927. 9. Le TN, Straatman LV, Lea J, Westerberg B. Current insights in noise-induced hearing loss: a literature review of the underlying mechanism, pathophysiology, asymmetry, and management options. J Otolaryngol Head Neck Surg. 2017;46(1):41. 10. Alberti PW. The anatomy and physiology of the ear and hearing. In: Occupational exposure to noise: evaluation, prevention, and control. 2001;53–62. References 73 11. Zheng Y, Guan J. Cochlear synaptopathy: a review of hidden hearing loss. J Otorhinolaryngol Disord Treat. 2018;1(1). 12. Baguley D, McFerran D, Hall D. Tinnitus. Lancet. 2013;382(9904):1600–7. 13. Environmental noise guidelines for the European Region. World Health Organization: Regional Office for Europe; 2018 (https://www. euro.who.int/__data/assets/pdf_file/0008/383921/noise-guidelines- eng.pdf, accessed 16 December 2021). 14. Berglund B LT, Schwela DH. Guidelines for community noise. Geneva: World Health Organization; 1999. 15. Occupational noise exposure: revised criteria 1998. Criteria for a recommended standard. National Institute of Occupational Safety and Health (NIOSH); Centres for Disease Control and Prevention: U.S. Department of Health and Human Services (https://www.cdc.gov/ niosh/docs/98-126/pdfs/98-126.pdf, accessed 23 December 2021). 16. Safe listening devices and systems: a WHO-ITU standard. Geneva: World Health Organization; 2019 (https://apps.who.int/iris/handle/ 10665/280085, accessed 15 December 2021). 17. Whittaker R, McRobbie H, Bullen C, Rodgers A, Gu Y, Dobson R et al. Mobile phone text messaging and app-based interventions for smoking cessation. Cochrane Database Syst Rev. 2019. doi: 10.1002/14651858.CD006611.pub5. 18. Sahin C, Courtney KL, Naylor PJ, Rhodes RE. Tailored mobile text messaging interventions targeting type 2 diabetes self- management: a systematic review and a meta-analysis. Digit Health. 2019. doi: 10.1177/2055207619845279. 19. Zhao YY, Dang FP, Zhai TT, Li HJ, Wang RJ, Ren JJ. The effect of text message reminders on medication adherence among patients with coronary heart disease: a systematic review and meta-analysis. Medicine (United States). 2019. doi: 10.1097/MD.0000000000018353. 20. Smith D, Duque L, Huffman JC, Healy BC, Celano CM. Text message interventions for physical activity: a systematic review and meta- analysis. Am J Prev Med. 2020;58:142–151. 21. Punch JL, Elfenbein JL, James RR. Targeting hearing health messages for users of personal listening devices. Am J Audiol. 2011;20(1):69–82. doi:10.1044/1059-0889(2011/10-0039). 22. Jiang W, Zhao F, Guderley N, Manchaiah V. Daily music exposure dose and hearing problems using personal listening devices in adolescents and young adults: a systematic review. Int J Audiol. 2016;55(4):197–205. doi:10.3109/14992027.2015.1122237. 74 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 23. Degeest S, Keppler H, Vinck B. Leisure noise exposure and associated health-risk behavior in adolescents: an explanatory study among two different educational programs in Flanders. Int J Environ Res Public Health. 2021;18(15):8033. doi:10.3390/ijerph18158033. 24. You S, Kwak C, Han W. Use of personal listening devices and knowledge/attitude for greater hearing conservation in college students: data analysis and regression model based on 1009 respondents. Int J Environ Res Public Health. 2020;17(8):2934. doi:10.3390/ijerph17082934 25. Rawool VW, Colligon-Wayne LA. Auditory lifestyles and beliefs related to hearing loss among college students in the USA. Noise Health. 2008;10(38):1–10. doi:10.4103/1463-1741.39002. 26. Diviani N, Zanini C, Amann J, Chadha S, Cieza A, Rubinelli S. Awareness, attitudes, and beliefs about music-induced hearing loss: towards the development of a health communication strategy to promote safe listening. Patient Educ Couns. 2019;102(8):1506–1512. doi:10.1016/j.pec.2019.03.013. 27. Wang D, Li C, Wang Y, Wang S, Wu S, Zhang S et al. Health education intervention on hearing health risk behaviors in college students. Int J Environ Res Public Health. 2021;18(4):1560. doi:10.3390/ ijerph18041560. 28. Dell SM, Holmes AE. The effect of a hearing conservation program on adolescents’ attitudes towards noise. Noise Health. 2012;14(56):39–44. 29. Loughran MT, Lyons S, Plack CJ, Armitage CJ. Which interventions increase hearing protection behaviors during noisy recreational activities? A systematic review. BMC Public Health. 2020 Dec;20(1):1–3. 30. Griest SE, Folmer RL, Martin WH. Effectiveness of “Dangerous Decibels,” a school-based hearing loss prevention program. Am J Audiol. 2007;16(2):S165–181. 31. Knobel KA, Lima MC. Effectiveness of the Brazilian version of the Dangerous Decibels(®) educational program. Int J Audiol. 2014;53 Suppl 2:S35–42. 32. Hearing loss due to recreational exposure to loud sounds, a review. Geneva: World Health Organization; 2015 (https://apps.who.int/iris/ handle/10665/154589, accessed 16 December 2021). 33. Partnership between health and education in early childhood. Lancet Child Adolesc Health. 2019;3(6):365. References 75 34. Fjeldsoe B, Neuhaus M, Winkler E, Eakin E. Systematic review of maintenance of behavior change following physical activity and dietary interventions. Health Psychol. 2011;30(1):99–109. 35. Lally P CH, Jaarsveld V et al. How are habits formed: modelling habit formation in the real world. Eur J Soc Psychol. 2009;40(6). 36. Buchholz SW, Wilbur J, Ingram D, Fogg L. Physical activity text messaging interventions in adults: a systematic review. Worldviews Evid Based Nurs. 2013;10(3):163–73. 37. Free C, Knight R, Robertson S, Whittaker R, Edwards P, Zhou W, et al. Smoking cessation support delivered via mobile phone text messaging (txt2stop): a single-blind, randomised trial. Lancet. 2011;378(9785):49–55. 38. Head KJ, Noar SM, Iannarino NT, Grant Harrington N. Efficacy of text messaging-based interventions for health promotion: a meta- analysis. Soc Sci Med. 2013;97:41–8. 39. Buil-Cosiales P, Roger-Loppacher O, Marimon F. The impact of SMS messages on young people’s participation in recycling campaigns. Communication & Society. 2014;27(1). 40. Grutzmacher SK, Munger AL, Speirs KE, Vafai Y, Hilberg E, Braunscheidel Duru E, et al. Predicting attrition in a text-based nutrition education program: survival analysis of Text2BHealthy. JMIR Mhealth Uhealth. 2019;7(1):e9967. 41. Planning an information systems project: a toolkit for public health managers. Geneva: World Health Organization; Seattle: PATH; 2013. 42. WHO evaluation practice handbook. Geneva: World Health Organization; 2013 (http://apps.who.int/iris/bitstream/10665/ 96311/1/9789241548687_eng.pdf, accessed 19 December 2021). 43. Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Geneva: World Health Organization; 2016 (http://apps.who.int/iris/bitstre am/10665/252183/1/9789241511766-eng.pdf, accessed 19 December 2021). 76 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Annexes Annex 1. Proposed structure of a BHBM implementation team 77 Annex 2. BHBM and mSafeListening programme stakeholders and roles 79 Annex 3. Sources of sustainable funding 83 Annex 4. Questionnaires 86 Annex 5. Adapting content library for voice, messenger apps or chatbots 97 Annex 6. Information on sound levels 105 Annex 7. Important considerations for the promotion of mSafeListening 106 Annex 8. Monitoring and evaluation indicators for mSafeListening 110 Annex 9. Benefits and risks of different software models 123 Annex 10. Role of aggregators 126 77 Annex 1. Proposed structure of a BHBM implementation team The BHBM implementation team is made up of four main bodies of experts: 1. National Programme Steering Committee The national Programme Steering Committee ensures agreement on the general direction of the programme and assists in decision-making. It includes representatives from the Telecommunications Ministry or equivalent, Ministry of Health (MoH) and/or other ministries mandated to support hearing health or disability-related activities (e.g. Ministry of Social Affairs, Department of Disability Affairs). The committee puts in place mechanisms to formalize a clear governance structure and functions (terms of reference, meeting frequency, roles and accountability), and to set out programme responsibilities. 2. National Technical Advisory Group (TAG) The Technical Advisory Group consists of high-level experts to support in-country project teams. TAG should comprise people vital for decision-making on funding and planning; people who would support implementation, promotion and evaluation of the programme, and who can advise the operations team. Expert advisors from government sectors should be included (e.g. health, business, social security, treasury and planning), along with those in telecommunications; the technology and software industry; local telecommunications/mobile network providers; regulatory and privacy experts; nongovernmental organizations (NGOs); health professionals; health and social care professionals; health economists; academic and research organizations; health service providers; civil society groups; opinion leaders; people with, or at risk of, noise-induced hearing loss; young people; guardians and mentors; and the media. 78 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening 3. International Advisory Group The International Advisory Group advises on technical and legal issues, the choice of platforms for scaling up, and sustainability and feasibility issues. It includes people from WHO (country, regional and headquarters offices), ITU, and the Informal Expert Group who produced the BHBM handbook. 4. mSafeListening operations/project team The mSafelistening team ensures that the programme runs to budget and to deadline, and reports to WHO and donors. The team includes a project manager plus 2–3 people to help with day-to-day programme operations. It engages with and maintains strong programme partnerships. Additional technical advisors (permanent or ad hoc) can include IT specialists; public health and disease topic specialists; communications specialists (for content adaptation and marketing and recruitment); young people, parents and mentors. Monitoring and evaluation officers can advise the team from the outset. Note: The WHO BHBM team is available for providing technical support. Even if a country does not opt to receive BHBM technical assistance, their status as a stakeholder should remain to ensure coherence to BHBM and for collecting and disseminating lessons learned. 79 Annex 2. BHBM and mSafeListening programme stakeholders and roles Stakeholder Role Ministry of Health and/ or other ministries mandated to support digital health related activities (e.g. Finance, ICT, Telecommunications, Data Protection etc.) • Owns and acts as the custodian of the programme; is part of the national steering committee and a key part of the governance function. • Assesses and identifies needs; develops and validates content. • Contracts service providers or builds in-house infrastructure/platform. • Signs cooperation agreements with all operators and/or service provider. • Funds, or partially funds, the programme. • May host the digital health messaging platform/database and own the short code. • Manages the promotion and marketing campaigns. • Works with other ministries and departments involved in the relevant digital health policy and campaigns. Telecommunications ministry or equivalent eGovernment entity (if applicable) • Generates framework to enable mServices (i.e. regulations and policies). • Funds, or partially funds, the programme, and acts as part of the governing body. • Provides technical expertise to the Ministry of Health. • May host the platform. • Facilitates dialogue between Ministry of Health and ICT stakeholders. • Supports the negotiation of preferential prices for mServices. 80 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Stakeholder Role Telecommunication Regulatory Authority • Verifies eligibility for short code acquisition. • Allocates short code • Facilitates dialogue between Ministry of Health and ICT stakeholders. • Funds, or partially funds, the programme. Digital health service providers (if Ministry of Health or eGov does not have a platform) • Provides management of the application/ platform. • Manages the platform and runs the programme. • Provides 24/7 technical support. • Deals with telecommunication operators; manages the short code where necessary. Telecommunication operator • Delivers targeted messages. • Sets the cost of messages using 3G or 4G data; agrees special tariffs with Ministry of Health if possible. • Facilitates interface with service providers and/ or local aggregators. • Supports promotion of the digital health messaging service. Local aggregator • Provides interface with all operators and manages relationship and invoicing process. • Provides reporting on services delivered/failed. • Possibly owns and manages the short code in the case of SMS. Data Privacy Commission • Sets the rules for data protection. • Enforces the application of data protection regulations. • Authorizes digital health messaging services providing they respect data privacy. • Authorize data storage outside the country, if necessary. WHO and ITU • Provide technical expertise and share knowledge from Member States. • Help convene stakeholders. Annex 2. BHBM and mSafeListening programme stakeholders and roles 81 Stakeholder Role Technical experts from various academic Institutions • Design the algorithm for mSafeListening. • Prepare contents for interventions. • Monitor the programme. • Evaluate the programme. Other relevant private sector parties • Provide technical expertise and resources with careful consideration of conflicts of interest. Communications and promotions actors • Design and facilitate the promotion and recruitment strategy. • Liaise with media outlets and service providers. • Run campaigns on their platforms (e.g. social media, television, radio) Target group (i.e. young people and parents); public health professionals; ear and hearing care specialists; associations of music industry; and people with hearing loss • Give feedback and advice on programme design and content, and usability and adaptation of content, ensuring human- centred design. Public health and health-care professionals in ear and hearing care • Are involved in design and development, including adapting content, providing human-centred design. • Promote and recruit for the programme. Academic institutions • Carry out reviews of efficacy of regional or other relevant digital health programming ahead of mSafeListening programme design. • Advise on monitoring and evaluation methods or carry out monitoring and evaluation. 82 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Stakeholder Role Technology providers (private software and application developers or experts; hardware companies; global goods/open software communities • Assist with technical specifications or software development. National Informatics Centre/Centre for Health Informatics • May be able to assist with hosting and maintaining a database of participants from the national portal. Ministry of Technology Development Agency • Can help to implement the promotion strategy. • Can help to build or maintain infrastructure nationwide. Local NGOs; international NGOs; UN (WHO, ITU, UNOICT, UNICC, UNICEF etc.) • Encourage roll-out and uptake of mSafeListening programmes. • Provide feedback and advice on programme design and content. Nongovernmental funders (start-ups, donors, partners, insurers, investors) • Help fund the mSafeListening programme in the short and long term. Abbreviations: ICT: Information and Communications Technology; ITU: International Telecommunication Union; NGOs: nongovernmental organizations; UNICC: United Nations International Computing Centre; UNICEF: United Nations Childrens Fund; UNOICT: United Nations Office of Information and Communication Technology; WHO: World Health Organization. 83 Annex 3. Sources of sustainable funding BHBM programmes have found several sustainable and successful business models for scale digital health programming that broadly fall into three categories: government funding; bilateral or multilateral support; or third-party grants. These options are not mutually exclusive and should be explored as early on as possible in the programme’s development. Where possible, any opportunities to streamline the approach to these funding sources should be found, as pursuing funding can be time-consuming. National government funding Digital health programmes are most sustainable when owned and operated by countries, as political will and financial commitment from governments maximize the programme’s chances of success (see Box A3.1 for examples). This requires the mSafeListening programme to be fully covered by national budgets. Digital health grants have been found to fit successfully within the priorities of several national strategies, so a good starting point is to explore the possibility of integrating digital health grants within existing funding mechanisms. Box A3.1: Government funding of Change to digital health programmes, India and Egypt The government of India has demonstrated important political commitment towards the scaling of the mTobaccoCessation and mDiabetes programmes, which gained traction in part due to the Prime Minister’s digital health initiative. Other government bodies were engaged in the programme to provide technical support, including the Ministry of Health and Family Welfare, the Ministry of Communication and Information Technology, the office of the Prime Minister’s MyGov platform, and the National Informatics Centre. In Egypt, in 2014, the Ministry of Health and Population established a central NCD unit to accelerate the implementation of the digital health programme. The following year, this Ministry and two others – the Ministry of Communication and Information Technology and the Ministry of Scientific Research – collaborated with BHBM and three local mobile network operators to support the implementation of mDiabetes in Egypt. 84 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Example: Government funding can come from existing budget lines. Securing funding for mSafeListening could come from a national health strategy if a pillar is dedicated to hearing care or noncommunicable diseases (NCDs). Alternatively, because hearing loss is part of the sensory disability, rehabilitation and can also be caused by injuries, it may be of interest to explore funding within these areas of health. Tax levied or mandatory contribution funds can also finance digital health programmes. Example: Universal service funds (also known as universal access or obligation funds) are collected from telecommunications companies in some countries, and although these types of funds do not commonly fund digital health programmes, this option could be explored. Other examples are funds raised from tobacco excise or sugar taxes (in the case of smoking cessation or diabetes digital health programmes). It may be worthwhile finding out if such budget lines exist and how to approach them for funding. As BHBM is a partnership between WHO and the ITU, another approach to secure funding is via technology (ministries of telecommunications commonly have larger budgets than ministries of health, and a collaboration between the two is the best way to ensure long-term sustainability of digital health projects). Example: As BHBM digital health programmes rely on technological infrastructure that can contribute toward the digitalization of a health system, they can be nested under digital transformation budgets which are often larger in size and broader in scope than budgets for hearing health. Digital health programmes should be considered part of a larger national digital health platform which may, in turn, be part of a broader national digital ecosystem. Integration within the larger factors, also ensures that the programme is sustainable and scalable and can expand into other areas. Annex 3. Sources of sustainable funding 85 Bilateral and multilateral support Bilateral support is the investment in one Member State by another Member State. Multilateral support typically comes from a multilateral development bank, chartered by two or more countries. Example: An example of bilateral and multilateral financing is in Sudan, where the African Development Bank, the Italian Agency for Development Cooperation and the Federal Ministry of Health are collectively investing more than US$ 1 million in BHBM programmes. Third-party grants Grants typically come from international health donors, national NGOs, philanthropists, or from the private sector (see Box 4 for tips on applying to funders). Normally, funds are secured from these organizations through careful outreach, cultivating relationships, sending unsolicited proposals, or responding to short or topic-specific calls for proposals. These may be focused on specific disease areas (e.g. hearing loss or NCDs) or on processes and systems (e.g. eHealth, mobile, technology-specific). The search identification strategy should expand not only to other multilateral funding institutions, but also to specific charitable organizations (e.g. Welcome Trust), that may be interested in funding parts of the research, or M&E components within each programme. Ideally, funding should be secured for the longer term (4+ years) as relying on donor-based funding may impact sustainability. Nevertheless, third- party grants can be powerful tools to demonstrate impact and results, and for building a strong case for investment from national funds. 86 Annex 4. Questionnaires Annex 4 sets out sample questions for the pre- and post-programme questionnaires and includes the following types: 1. Pre-screening questionnaire: to gather demographic information and segment the participants according to type of messages to be delivered. 2. Pre-programme questionnaire: to deliver the most suitable time of delivery of messages to participants, and obtain baseline measures of key behaviours targeted by the initiative. 3. Post-programme questionnaire: to document user experience, seek feedback from users and assess if the intended objective of promoting safe listening behaviours was achieved. The pre-screening and pre-programme questionnaires for mSafeListening can be used to gather demographic and baseline measures of the key behaviours to be targeted by the initiative. The pre-screening questions are necessary to select the relevant message library for each participant based on the participant’s characteristics. The questions can be asked over the telephone, delivered through a hardcopy or web-based survey and/or adapted for SMS delivery at enrolment into the programme. Note: The questions included in the questionnaires are samples only, and are intended as a guide to be adapted and translated as required. Not all of the questions will be applicable to each end-user group. Annex 4. Questionnaires 87 * It may be deemed important to also gather data on other sociodemographic variables (e.g. highest level of education and current professional status). Sample pre-screening questionnaire The pre-screening questionnaire is intended to gather demographic information and direct the participant to the appropriate mSafeListening message library. What is your date of birth? What is your gender?* 1. Male 2. Female 3. Other A. Which of the following best describes you? 1. I am an adolescent who often listens to music over my headphones or visits venues such as clubs, fitness classes and concerts where amplified music is played. 2. I am an adult who often listens to music over my headphones or visits venues such as clubs, fitness classes and concerts where amplified music is played. 3. I am a parent and would like to better understand how loud sounds affect hearing and how I can reduce my child’s risk of developing hearing loss when listening to music. 4. I am involved in the care of children (e.g. general health worker or educator) and I would like to better understand how loud sounds affect hearing and how I can help to prevent or identify hearing loss in those who I care for or work with. 5. None of the above (Please specify). B. Why do you want to sign up for the mSafeListening initiative? (Tick all that apply) 1. I want to understand more about hearing loss prevention and safe listening. 2. I want tips on how to listen to my music without risk of hearing loss. 3. I want to be able to guide those around me or in my care about hearing loss prevention. 4. I want to learn more about warning signs that may indicate that I (or my child/children in my care) have hearing loss. 5. Other (Please specify). 88 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening For example, a participant who responds to question A with option 3: “I am a parent and would like to better understand how loud sounds affect my hearing and how I can reduce my child’s risk of developing hearing loss when listening to music” Followed by a response to question B with option 1: “I want to understand more about hearing loss prevention and safe listening.” Will receive messages from the mSafeListening message library tailored towards parents and care-providers. For more, explore the mSafeListening message library. Sample pre-programme questionnaire The pre-programme questionnaire comprises: Section A: to determine the most suitable day of the week and the time of the day to send safe listening messages). Section B: to obtain baseline measures of key behaviours which the initiative intends to target. Options selected Question A 1 2 3 4 5 Options selected Question B and types of messages 1, 2: Messages for adolescents 1, 2: Messages for adults 1, 2: Parent/care- provider messages 1, 2: Parent/care- provider messages 1, 2, 3, 4: Messages for general population 3, 4: Messages for general population 3, 4: Messages for general population 3, 4: Messages for general population 3, 4: Messages for general population Based on the responses to the above questions A and B, participants are segmented to receive tailored messages from the mSafeListening message library. Segmentation may occur as follows: Annex 4. Questionnaires 89 Section A. General Questions What time of the day do you (or your child/children in your care) commonly listen to music (or other audio content) through headphones? (Tick all that apply) 1. Early morning 2. Late morning 3. Early afternoon 4. Late afternoon 5. Early evening 6. Late evening 7. Late night Which day of the week do you (or your child/children in your care) normally visit places that play amplified music (e.g. fitness classes/ gym, clubs, bars, concerts)? (Tick all that apply) 1. Monday 2. Tuesday 3. Wednesday 4. Thursday 5. Friday 6. Saturday 7. Sunday At what time of the day do you (or your child/children in your care) normally visit places that play amplified music (e.g. fitness classes/ gym, clubs, bars, concerts)? (Tick all that apply) 1. Early morning 2. Late morning 3. Early afternoon 4. Late afternoon 5. Early evening 6. Late evening 7. Late night Based on the information gathered from the questionnaire, the day and the time of day that the messages are delivered should correspond to what has been indicated in the response. For example if an individual visits a noisy place in late evening on Fridays, then it is most suitable to send them a message about use of earplugs during the early evening of Friday, to serve as a reminder to carry earplugs when they leave. 90 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Section B How much do you feel you know about hearing loss and unsafe listening? 1. I know nothing about hearing loss and unsafe listening. 2. I know very little about hearing loss and unsafe listening. 3. I know some information about hearing loss and unsafe listening. 4. I know a fair bit about hearing loss and unsafe listening. 5. I know a lot about hearing loss and unsafe listening. Please select how much you agree with the following statement: Behaviours can lead to a higher risk of development and progression of hearing loss: 1. Strongly disagree 2. Disagree 3. Neither agree or disagree 4. Agree 5. Strongly agree Do you know what warning signs to look for that may indicate that you (or your child/children in your care) may have hearing loss? 1. I know all of the signs to look for 2. I know many of the signs to look for 3. I know one or two of the signs to look for 4. I don’t know any of the signs to look for 5. I am not aware that there are signs to look for Have you (or your child/children in your care) ever checked your (or your child’s/children in your care’s) hearing using a digital application? 1. Yes 2. No How long ago did you last have your (or your child’s/children in your care’s) hearing tested by a doctor or other health-care provider? 1. Less than 1 year ago 2. 1–2 years ago 3. 2–5 years ago 4. More than 5 years ago 5. Never Annex 4. Questionnaires 91 On average, how much time do you (or your child/children in your care) spend in listening to music (or other audio content) over headphones? 1. No time 2. Less than 1 hour/day 3. 1–3 hours/day 4. 3–5 hours/day 5. 5–8 hours/day 6. More than 8 hours/day On a scale of 0–10 (where “0” = no sound, and “10” = the highest volume level on your device), what volume level do you (or your child/children in your care) normally prefer while listening to music (or other audio content) over headphones? 1. 1–6 2. 7–8 3. 9–10 On average, how often do you (or your child/children in your care) visit places (e.g. fitness classes/gym, clubs, bars, concerts, noisy sports arena) that play amplified music/sounds? 1. Never or rarely 2. Several times a year 3. Once a month 4. Once a week 5. Several times a week On average, how many hours (in a week) do you (or your child/children in your care) spend in places (e.g. fitness classes/gym, clubs, bars, concerts, noisy sports arena) that play amplified music/sounds? 1. Less than 1 hour/week 2. 1–3 hours/week 3. 3–5 hours/week 4. 5–8 hours/week 5. More than 8 hours/week When visiting places that play amplified music/sounds (e.g. fitness classes/ gym, clubs, bars, concerts, noisy sports arena), how often do you (or your child/children in your care) use hearing protection (such as earplugs)? 1. Never 2. Rarely 3. Sometimes 4. Most of the time 5. Always 92 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Have you (or your child/children in your care) experienced a ringing or buzzing sound in the ear (tinnitus)? 1. Never 2. Sometimes 3. Often 4. All the time Which of the following best describes your (or your child’s/children in your care’s) current hearing status? 1. I (or, my child/children in my care) have no difficulty in hearing. 2. I (or, my child/children in my care) find it difficult to hear what is being said when in a noisy situation like a bar or restaurant. 3. I (or, my child/children in my care) find it difficult to hear what is being said even when in a quiet place. 4. I (or, my child/children in my care) can only hear words when they are spoken loudly. 5. I (or, my child/children in my care) have a hearing loss and use hearing technology (such as hearing aids or implants). 6. I (or, my child/children in my care) have a hearing loss, but do not use any hearing technology. 7. I (or, my child/children in my care) am Deaf and use sign language. Sample post-programme questionnaire The post-programme questionnaire comprises: Section A: to document user experience and receive feedback. Section B: to evaluate the effectiveness of the programme in changing behaviour and impacting health outcomes. The post-programme questionnaire can be delivered in a similar format to the pre-programme questionnaire (e.g. over the telephone, hardcopy/ web-based survey, or adapted to be delivered through messaging at completion of the initiative). The number of questions and format can be adapted depending on how the questionnaire is delivered. The questions should closely match those included in the pre-programme questionnaire for comparative purposes. Annex 4. Questionnaires 93 Section A. General questions How many of the text messages did you read? (Select 1 only) 1. None, I didn’t receive any 2. None, I didn’t read any 3. Some (fewer than half) 4. Most (more than half) 5. All or nearly all of the messages Did you share any of the messages with others? (e.g. friends, partner, family members, etc.) 1. Yes 2. No If yes, kindly provide any additional details on how you shared the messages with others: What did you think about the number of messages we sent? 1. Too few, I would have liked more 2. The right amount 3. Too many messages What did you think about the length of the programme? 1. Too short 2. The right length 3. Too long Did you have any technical problems with the programme? (e.g. could not sign up easily, or could not read messages) 1. Yes 2. No If yes, kindly provide any additional details on the nature of the technical problems: Would you recommend the programme to others? 1. Yes 2. No Kindly provide any additional details that you feel may be relevant: 94 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Section B. Programme specific questions After completing the programme, do you feel that you have more knowledge about hearing loss and unsafe listening? 1. Yes 2. No 3. Not sure After completing the programme, do you feel that you have more knowledge about preventing the development or progression of hearing loss caused by listening to loud music/sounds? 1. Yes 2. No 3. Not sure After completing the programme, do you feel that you have more knowledge regarding warning signs that may indicate that you (or your child/children in your care) may have hearing loss? 1. Yes 2. No 3. Not sure Since starting the programme, the time you (or your child/children in your care) spend in listening to music (or other audio content) over headphones has: 1. Increased 2. Decreased 3. Not changed 4. I am not sure On average, how much time do you (or your child/children in your care) spend in listening to music (or other audio content) over headphones? 1. No time 2. Less than 1 hour/day 3. 1–3 hours/day 4. 3–5 hours/day 5. 5–8 hours/day 6. More than 8 hours/day Since starting the programme, the average volume level that you (or your child/children in your care) prefer to listen to music (or other audio content) over headphones has: 1. Increased 2. Decreased 3. Not changed 4. I am not sure On a scale of 0–10 (where “0” = no sound, and “10” = the highest volume level on your device), what volume level do you (or your child/ children in your care) normally prefer to listen to music (or other audio content) over headphones? 1. 1–6 2. 7–8 3. 9–10 Annex 4. Questionnaires 95 Since starting the programme, the number of times that you (or you child/children in your care) visit places that play amplified music/ sounds (such as fitness classes/gym, clubs, bars, concerts, noisy sports arena has: 1. Increased 2. Decreased 3. Not changed 4. I am not sure On average, how many hours (in a week) do you (or your child/children in your care) spend in places (e.g. fitness classes/gym, clubs, bars, concerts, noisy sports arena) that play amplified music/sounds? 1. Less than 1 hour/week 2. 1–3 hours/week 3. 3–5 hours/week 4. 5–8 hours/week 5. More than 8 hours/week Since starting the programme, the number of times you (or your child/ children in your care) use hearing protection (such as earplugs) when visiting places that play amplified music/sounds (e.g. fitness classes/ gym, clubs, bars, concerts, noisy sports arena) has: 1. Increased 2. Decreased 3. Not changed 4. I am not sure When visiting places that play amplified music/sounds (e.g. fitness classes/ gym, clubs, bars, concerts, noisy sports arena), how often do you (or your child/children in your care) use hearing protection (such as earplugs)? 1. Never 2. Rarely 3. Sometimes 4. Most of the time 5. Always Since starting the programme, have you (or your child/children in your care) checked your hearing using a digital application? 1. Yes 2. No 3. Not sure Since starting the programme, have you (or your child/children in your care) had a hearing test by visiting a doctor or other health-care provider? 1. Yes 2. No 3. Not sure Have you (or your child//children in your care) experienced a ringing or buzzing sound in the ear (tinnitus)? 1. Never 2. Sometimes 3. Often 4. All the time 96 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Which of the following best describes your (or your child’s/children in your care’s) current hearing status? 1. I (or my child/children in my care) have no difficulty in hearing. 2. I (or my child/children in my care) find it difficult to hear what is being said when in a noisy situation such as a bar or restaurant. 3. I (or my child/children in my care) find it difficult to hear what is being said even when in a quiet place. 4. I (or my child/children in my care) can only hear words when they are spoken loudly. 5. I (or my child/children in my care) have a hearing loss and use hearing technology (such as hearing aids or implants). 6. I (or my child/children in my care) have a hearing loss, but do not use any hearing technology. 7. I (or my child/children in my care) am Deaf and use sign language. 97 Annex 5. Adapting content library for voice, messenger apps or chatbots Adapting to voice Interactive voice response (IVR) enables reaching those who may not be able to interact with text content. An actor can record the adapted messages in the BHBM content library which can then be delivered by inbound or outbound calls to a smartphone or a basic phone. The call should be cost-free. With IVR, the user can input a response using key words (e.g. programme question: “Have you reached your walking goal today? Answer: ‘YES’ or ‘NO’” and depending on the response, an appropriate pre-recorded message is given by the programme to the user). This feature may be particularly useful if there are literacy or disability (e.g. visual) concerns within the target population. Voice messages (if appropriate) can be more creative in engaging target- users (again, users should be asked their preferences). For example, a series of short 1–3 minute audio plays or stories could be used to deliver the BHBM content library via the telephone. Using a number of scenarios, actors could be recorded as being engaged in a discussion (e.g. a doctor and a patient receiving information, advice or instruction; or other trusted community member providing information or strategies on behavioural change). However, when adapting content, it is important to maintain the original intent of the message in the BHBM content library. Start by using messages from the BHBM content library and add more details where you think the user could benefit from more clarity, preserving the scientific fidelity of the programme to the original library. We suggest that you ask target users what they would likely engage with in terms of multimedia. 98 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Adapting to messenger apps A recent market research survey found that people check their phones for messages and notifications up to an average of 96 times a day.8 Messages sent through widely used messenger apps (e.g. Messenger from Facebook, WhatsApp, WeChat) therefore provide an opportunity for messages to be noticed and read. Messenger apps also provide more freedom in terms of the length of messages that can be sent, and the different media used for engaging people (e.g. audio files, images, GIFs and videos, external web links, or localization based suggestions or content) (see Table A5.1). Messenger apps can further be used to provide conversational agents, or used for lower level interactivity, equal to that of SMS messaging; they are, however, restricted to reaching only users with higher income levels who have smartphones. Table A5.1: Table A5.1 Multimedia “Dos” and “Don’ts” Multimedia content “Dos” Multimedia content “Don’ts” Do ensure equal representation of men and women and different ethnic groups within your target population in all visual content. Don’t use stereotyped images of particular social groups or age groups. Do consider the file size and data usage costs for users, avoiding “heavy” files and compressing image files and video files where possible. Don’t use colours or gestures associated with a particular political or social group. Do use a variety of relevant regional accents if possible in audio materials. Don’t use complicated infographics, graphs or other visual representations of information. Keep them simple. Do try using quizzes: these can be a fun way to engage, reward and collect data about knowledge-gains and behavioural change. Be aware of accidental product placement in photographs or videos (e.g. a branded good in the background). Do be creative with content; try to use visuals alongside educational messages (e.g. illustration demonstrating how to correctly wear earplugs alongside information regarding the benefits of earplugs in reducing sound dosage, and how to use them) 8 Asurion-sponsored survey by Market Research Firm Solidea Solutions conducted 18–20 August 2019 of 1998 smartphone users in the USA, compared to an Asurion- sponsored survey conducted by market research company OnePoll between 11–19 September 2017 of 2000 adults in the USA with a smartphone. Annex 5. Adapting content library for voice, messenger apps or chatbots 99 Conversational agents and conversationalizing content If a more interactive and tailored experience is preferred, a chatbot can provide this. It will be necessary to consider the user experience of a chatbot when designing the conversational scripts for the bot. Using a natural language processing (NLP) chatbot is costly, can be imprecise and less feasible because NLP incorporates an artificial intelligence system that has to be trained and maintained using substantial data. Nonetheless, it is evident through experience with some WHO bots that users have more conversational exchanges using an NLP bot. A conversational interface chatbot (CIC) is far easier to set up than an NLP chatbot. A CIC chatbot presents limited input options for users to select using buttons, emojis or typing the corresponding number or key word from a list of information topics. This results in the user navigating the bot and getting the tailored information they desire with less likelihood of the bot misunderstanding free-text inputs. Alternatively, there are hybrid bots that are predominantly CICs but have some basic NLP capabilities, which may lead to a better user experience. Figure A5.1 provides an example of a CIC (the WHO Health Alert service provided through WhatsApp). In addition, bots have the capability to provide links to external websites for further information, to send audio or image files, GIFs or videos; so it is possible to be more creative with the content provided and potentially retain more users. Nonetheless, the mobile data costs for users that such content carries must be considered. An important limitation of bots on certain platforms is that the user has to start the conversation each time. Messenger by Facebook is currently the only bot platform where the first message of a conversation (aside from initial enrolment), or a notification, originates from the app, potentially leading to higher retention rates. 100 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Figure A5.1: Sample navigation of the WHO COVID-19 health alert chatbot CICs require a conversational script to be entered into the bot management software, the content of which can be adapted from the BHBM content library. It is, however, a good idea to start with the programme goals and aims for knowledge-gain and behavioural change, then use the BHBM content library as a basis for relevant messages (see Box A5.1 for key steps in creating a chatbot). Annex 5. Adapting content library for voice, messenger apps or chatbots 101 Box A5.1: Steps for creating a CIC chatbot The first step in creating a CIC chatbot is to prepare the thematic outline of the CIC. This is an outline of the themes of each conversation that segmented groups should receive (i.e. information, questions, behaviours). The group for segmenting will depend on the target- users and the aims of the programme. Users may be segmented into age-related groups (e.g. youth, middle-adulthood, older people); gender-based groups; or groups related to disease-risk status (general population, at risk or diagnosed). For less sophisticated bots, this will likely be two or three questions at the beginning of each session, the responses to which will place the user into a particular segment or group to follow a given algorithm. (Certain more costly and sophisticated bots can “remember” and tag a user as being in a specific group for all future sessions; however, this creates a complicated back-end structure, similar to having several separate bots, and may not be feasible.) The thematic bot outline could take the form of a table or a diagram, whichever is suitable. It may be informed by the messages in the BHBM content library, or it can be started from scratch and the key messages from the BHBM library mapped back to it later. It is possible to select which key message a user is to receive and in which week; alternatively users themselves can be given the choice of what they wish to learn in each session within the chat. The second step involves creating a bot diagram and writing the messages according to the thematic bot structure. This diagram maps out the messages that are sent and received by the bot and forms a pictorial representation of the algorithm for each chat session. Users may become fatigued after 3–7 reply-bot exchanges, so chat sessions should be short and to the point. When creating a diagram, the actual messages could be entered, or a code corresponding to another document with the full messages written (according to the need of the bot content management system). Careful maintenance of the script management spreadsheet is very important, as is interaction with the service providers, as this may have implications for cost. For example, if messages can be managed in advance using a spreadsheet, this can be used to obtain feedback on the project. Well-organized and structured content can then be presented to a company and this may result in a lower quote than if the company had had to enter and provide a content management system themselves. BHBM can provide assistance with this. This guidance is also relevant to NLP bots, however a third-party company will likely have to be used to set up the bot and programme it to understand and process natural language input. 102 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening The design of the possible topics and conversations can be followed by adapting accompanying audio files, images, videos or links to user preferences. Table A5.2 lists “Dos” and “Don’ts” for creating bot conversations. Table A5.2: Chatbot conversation “Dos” and “Don’ts” Chatbot conversation “Dos” Chatbot conversation “Don’ts” Do keep messages short, simple and to the point. Don’t use casual fillers as in normal speech as these can be misinterpreted. Do explain clearly how the user navigates the conversation (e.g. “select from the buttons below”; “type the number of your desired response”). Don’t be too “chatty”. Do include the function for saving the user’s progress so the user can return to the same point if they have to leave the conversation. Don’t overwhelm with too much information or cumbersome processes. Keep the goal of the bot in mind when writing the script. Don’t use humour unless it can be understood by all. It could create confusion. Be consistent with voice and tone throughout the script, and with tenses used. Don’t request clarification on every input, only for important questions (such as screening questions for tailoring purposes). Do proofread and test the script for errors in the algorithm. It is essential that all content works smoothly. Avoid using emojis if multiple messenger apps are to be used, as emojis cannot be recognized across all platforms. Make sure it is clear when a chat session is finished and how and when the user can next engage. Personalize messages either by writing in the first person or including a name or a mascot for the service (characterizing the speaker). Annex 5. Adapting content library for voice, messenger apps or chatbots 103 Making the content library into an app Little analysis of components is available which show that health app features increase app effectiveness. However, literature is emerging that tries to determine which apps are effective, including comment on features. Apps should have an engaging design, and be very easy to use. All instructions should be clear and navigation should be effortless. User- testing is vital in creating an app that will be used and that will retain its users. Table A5.3 contains suggestions for building a successful health app. 104 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Table A5.3: App characteristics linked to positive user-ratings and app engagement Features and content that have a positive impact on user-ratings and/or result in increased use Content: includes reference to internal drivers of behavioural change (e.g. motivation, self-efficacy, illness understanding and attribution.9 Content: includes reference to external drivers of behavioural change (e.g. availability of information; the beliefs of peers and family; and the role of social networks.9 Content: Strong evidence base and behavioural change theory. Service user and professional input: especially at the design stage – ensure user compatibility and acceptability of app.9 Clarity: factors such as ease and simplicity of use; specific instructions; features that save time; accessibility; relevant functions and clear security features which are also important and have high ratings in app stores.10 Rewards: Tangible and intangible rewards provided by the health app. Social competition: seeing other people using the app and sharing behavioural data that could be compared to others on social networking sites; the ability to share personal information (sharing information with family or friends, leads to informational and emotional social support). Entertainment factors: a gaming element or other entertaining feature. Trackers: tracking for awareness and progress (built-in feature to track user activity, including diet, exercise, sensor-based automatic tracking). Goal-setting: features help users discipline themselves and slowly change their behaviours. 9 Fitzgerald M, McClelland T. What makes a mobile app successful in supporting health behaviour change? Health Education Journal. 2017;76(3):373–381. 10 Mendiola MF, Kalnicki M, Lindenauer S. Valuable features in mobile health apps for patients and consumers: content analysis of apps and user ratings. JMIR mHealth and uHealth. 2015;3(2):e40. 105 Annex 6. Information on sound levels Approximate sound level in dB* and maximum permissible time per week for safe listening Airplane taking off 140dB 0 sec Firecrackers 150dB 0 sec Jackhammer 130dB 1 sec Standing near a siren 120dB 12 sec Shouting in the ear 110dB 2.5 min Car horn at 5 meters 105dB 8 min Hair dryer 100dB 20 min Motorcycle 95dB 1h 15 min Shouted conversation 90dB 4 hours Heavy traffic (inside the car) 85dB 12h 30 min Normal conversation 60dB Unlimited Library 40dB Unlimited Soft whispering 30dB Unlimited Normal breathing 10dB Unlimited Doorbell 80dB 40 hours * This graphic is based on the 3-dB exchange rate and the WHO recommendation regarding safe listening exposure and weekly time limit. The examples of sound levels are indicative. Actual sound levels may vary. 106 Annex 7. Important considerations for the promotion of mSafeListening Table A7.1: Considerations for mSafeListening promotion Target audience for promotion Getting to know the target audience is advisable (if this has not been done already), through focus groups, interviews and surveys. The target audience for promotion will include: • the defined target population for mSafeListening (see 3.1); and • key promoters – such as teachers, parents, telecommunications companies, NGOs, social media influencers and any other end-user facing groups with an interest in health promotion. The more that promotion and recruitment strategies are tailored to the target population, the more effective they will be at encouraging people to subscribe, and the wider the reach of the programme. Segmenting the target population into smaller groups based on key characteristics (e.g. age, gender, values) and understanding the motivations of each group (e.g. by leveraging motivation for change in the recruitment campaign and tailoring promotion materials for different segmented groups), can help make recruitment more successful. One suggestion is to run focus groups, with people representative of the target audience, to inform the design of the programme and specifically to gather ideas and recommendations about recruitment methods to form a basis for the strategy. The table below lists important learning points and suggestions for promoting mSafeListening. These are based on experiences from other Be Healthy Be Mobile (BHBM) programmes. Annex 7. Important considerations for the promotion of mSafeListening 107 Cost of the campaign Find out from target-users which channels they will engage with, and invest in these for programme success (e.g. social media advertising, radio and television). An initial underestimation of promotional costs is common and can be difficult to remedy later. The principles of negotiation can also be useful when approaching telecommunications, broadcasting and social media companies (see Annex 10). Strategies and synergies, and leveraging other campaigns The strategy for the programme should be based on actual research with the target audience that will allow identification of which channel is the most suitable. Suggested pointers are given below: • Which organizations/notable personalities are currently involved in successful mass media campaigns for raising awareness on health- related issues? Can those campaigns be linked or leveraged? • Which media or music personalities have experienced hearing loss and are likely to support efforts at hearing loss prevention? • Which digital health programmes have previously been implemented in the area? Can lessons be learned about which promotional techniques are effective in the country? • Can existing marketing or health promotional campaigns of programme stakeholders or partner agencies (such as telecommunications companies) be leveraged to allow for cost savings? • Can telecommunications companies or vendors advertise the programme on SIM-card packaging, or run announcements when users are “on hold” when calling for technical support or customer services contacts? • Is the country already running a campaign on hearing loss, where promotion of mSafeListening can be an add-on? For example, mSafeListening can be promoted on the World Hearing Day, which is observed on 3 March each year. 108 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Accessibility of promotion materials Consider the target audience and if/how they access certain media channels. What is the media channel they will most likely see and engage with? How can you make accessing recruitment materials more equitable to minority populations or people with disabilities? Content of the campaign A valuable lesson learned by one BHBM programme was to ensure that all information on how to access the programme is very concise and clear. The potential user should know who the programme is for, have all relevant instructions, and know how to sign up having seen the promotion materials. A BHBM programme in Tunisia found that users reported signing up because it was convenient via mobile phone and not because they thought the programme would work. Although it may seem obvious, campaigns should state who the intended user is. One BHBM implementor stated that they did not specify that the programme was for tobacco users; many non- tobacco users signed up out of interest or to learn about addictology. The “owner” or perceived messenger of the programme BHBM evaluations have shown that users trust and value the messages if the source is the Ministry of Health. Therefore, if possible, ensure that the user sees that the message originates from the Ministry of Health or other trusted health authority. Consider also promoting through government health services (e.g. on appointment reminder cards, in waiting rooms etc). Identify other trusted authorities such as local partners and stakeholders who can help with promotion and recruitment; and partners, such as schools, universities, centres of music learning (e.g. schools of music), cultural activities groups, civil society (e.g. national hearing loss associations). Campaign website It is highly recommended to set up a campaign webpage, ideally on the MoH website, where users can find information on the mSafeListening programme and also subscribe to it. Annex 7. Important considerations for the promotion of mSafeListening 109 Using marketing specialists or learning from the private sector The MoH or other implementing agency may not have the in-house expertise to plan and deliver an effective promotional campaign. Contracting out to a marketing agency may seem costly, but if the appropriate agency is selected (i.e. one with a good track-record in health marketing), it may boost programme numbers substantially and thereby justify the outlay. Testing the recruitment strategy through soft launch Consider a test-run or a “soft launch” prior to starting the promotional campaign to ensure that all processes are working well in advance of large numbers of participants signing up. This may involve running focus groups with users comparing differently worded or presented promotional campaign materials; or asking users what messages about the programme and marketing materials would encourage them to want to sign up. Pre-intervention information session Launching information sessions about the digital health programme at places frequently visited by the target audience can enhance the visibility of programme and encourage the participation, and become part of the promotion and campaign strategy. Likewise putting flyers in places such as schools, university notice boards, clinics, community centre, can also increase the accessibility of materials. The local mobile communications environment Check whether the sending of unsolicited messages is permitted (in some countries this contravenes the codes of conduct of telecommunications companies). Consider whether a population that often receives unsolicited health- related messages will be likely to read and respond to messages from the programme. Also consider the issue of message receipt versus message engagement. 11 0 A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y O p er at io n s 1. N um b er o f f ul l- tim e eq ui va le nt p er so n s w or ki n g a t le ad in g a g en cy o n m Sa fe Li st en in g In p u ts Th is g au g es th e co m m itm en t to h um an re so ur ce s of th e le ad in g im p le m en ta tio n ag en cy . Te rm s of re fe re nc e fo r in vo lv ed p ro gr am m e em pl oy ee s; or in fo rm at io n so ug ht ve rb al ly fr om te am le ad er A n n u al ly 2. N at io n al te ch n ic al ad vi so ry g ro up s et u p In p u t Th is re co rd s w h et h er th e te am is fu n ct io n in g b y M on th 3 . C ar rie d o u t: Ye s/ N o; a n d co m p os iti on Ye ar 1; M on th 3 3. C om m itm en ts fo r fu n di n g (i n U S$ ) a cr os s co nt rib ut or s an d th e du ra tio n o f e ac h co m m itm en t In p u t Th is re co rd s th e so ur ce o f t h e fu n d in g ; h ow m u ch th e fu n d in g w ill b e, a n d it s d ur at io n . I t al so h el p s to e st ab lis h w h et h er m or e fu n d in g m ay b e n ee d ed an d to p la n a cc or d in g ly . Q u al ita tiv e d es cr ip tio n . M ay h av e to s p ea k w ith ot h er s re sp on si b le fo r re so ur ce m ob ili ty A n n u al ly 4 . N um b er o f f un dr ai si n g ac tiv iti es fo r su st ai na b ili ty P ro ce ss Th is re co rd s ef fo rt s m ad e in su st ai n ab ili ty fr om th e ou ts et a n d p ro m ot es th e n ur tu rin g o f re la tio n sh ip s th ro u g h ou t. C ar rie d o u t: Ye s/ N o; a n d d es cr ip tio n A n n u al ly N O TE : I n d ic at or s in p in k ar e K ey P er fo rm an ce In d ic at or s (i. e. in d ic at or s co n si d er ed “c or e” o r “e ss en tia l” to B H B M p ro g ra m m in g) . A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g 11 1 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 5. B u d g et ed p la n p ro d u ce d fo r cu rr en t an d fo llo w in g y ea r P ro ce ss Th e p la n m ay e vo lv e or c h an g e ov er tim e, b u t it s h ou ld b e d ev el op ed . C ar rie d o u t: Ye s/ N o A n n u al ly 6. B u d g et s p en t In p u t A b u d g et re p or t, ei th er a s an E xc el sp re ad sh ee t or a w rit te n re p or t, is u se fu l to m on ito r sp en d in g a n d to p la n a h ea d . M ai n b u d g et ar y ite m s (in U S$ ) Q ua rt er ly (a n d cu m ul at iv e) St ak eh ol d er e n g ag em en t 7. N um b er o f p ar tn er sh ip s, an d n am e of p ar tn er or g an iz at io n s w ith s up p or tin g d oc um en ta tio n , s u ch as a m em or an du m o f un d er st an di n g o r t er m s of re fe re n ce . In p u ts Th is w ill e n ab le m ea su re m en t of th e in te re st a n d g ro w th o f t h e p ro g ra m m e in te rm s of it s p ar tn er sh ip s su p p or t fr om g ov er n m en t ag en ci es , h ea lth s er vi ce s, th e p riv at e se ct or a n d c iv il so ci et y. R ec or d s ke p t on th e n um b er o f p ar tn er sh ip s th at in cl u d e su p p or tin g d oc um en ta tio n A n n u al ly 8 . N um b er o f m ee tin g s w ith e xt er n al p ar tn er s. In p u ts Th e n um b er o f m ee tin g s h el d b et w ee n o rg an iz at io n s ex te rio r to th e ev er yd ay ru n n in g o f t h e p ro g ra m m e. Su ch o rg an iz at io n s m ay in cl u d e te le co m m un ic at io n s au th or iti es , c iv il so ci et y, o th er re le va nt m in is tr ie s, an d p riv at e se ct or p ar tn er s. S u g g es t m ee tin g s w ith s u ch p ar tn er s to t ak e p la ce a s re q ui re d , b u t at a m in im um q u ar te rly o r 6- m on th ly . M ai nt ai n a c ou nt o f su ch m ee tin g s Q u ar te rly B E H E @ LT H Y B E M O B IL E A h an d b oo k on h ow to im p le m en t m Sa fe Li st en in g 11 2 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y P ol ic y en g ag em en t an d s ys te m ic c h an g e 9. D es cr ip tio n o f p ol ic y en g ag em en t In p u ts / p ro ce ss / ou tp u ts D es cr ib e th e p ol ic y in te ra ct io n w ith th e p ro g ra m m e du rin g th e p as t y ea r, e. g . p ol ic y co n su lta tio n s h el d , p ol ic y- m ak er s in vo lv ed , “ sp ea k- in ” p ro g ra m m e m ee tin g s, n um b er o f p ol ic y b rie fs p re p ar ed , n ew sl et te rs s en t, ac tu al p ol ic y ch an g e et c. M ai nt ai n a re co rd o f su ch in te ra ct io n s an d ev en ts A n n u al ly 10 . D es cr ip tio n o f s ys te m ic ch an g e at tr ib u ta b le to p ro g ra m m e ac tiv iti es P ro ce ss / ou tp u ts Th is q ua lit at iv e in di ca to r c an d es cr ib e an y ch an g es to th e h ea lth s ys te m th at re su lt fr om p ro g ra m m e ac tiv iti es d ur in g th e p as t y ea r. Th is m ay in cl u d e ot h er u se s of a cq ui re d s of tw ar e or th e co nt en t d el iv er y p la tf or m ; i n st itu tio na liz at io n of th e p ro g ra m m e; c ha n g es in re fe rr al m ec ha ni sm s; c ha n g es in p ub lic aw ar en es s of th e h ea lth to p ic ; n ew sy n er g ie s b et w ee n p ar tn er s or b et w ee n m in is tr ie s; o r c ha n g es in p ro ce ss o r p ro ce du re s am on g p ar tn er s du e to th e p ro g ra m m e et c. Th is in di ca to r c an a ls o d oc um en t t h e ad d ed v al u es o f t h e p ro g ra m m e. A n n u al ly P ro g ra m m e co n te n t 11 . N um b er o f d es ig n co n su lta tio n s or m ee tin g s P ro ce ss Th is in cl u d es th e n um b er o f m ee tin g s w ith te ch n ic al e xp er ts , t ar g et -u se rs a n d ot h er s ta ke h ol d er s on b oa rd ed to w or k on c on te nt . M ai nt ai n a c ou nt o f su ch m ee tin g s En d Y ea r 1 A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g 11 3 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 12 . E xi st en ce o f p ro g ra m m e sp ec ifi ca tio n s In p u ts Th is in cl u d es is su es s u ch a s w h et h er th e p ro g ra m m e d es ig n s p ec ifi ca tio n s h av e b ee n s et , e .g . v er ify in g th e ai m s; ad ap tin g th e lo g ic m od el w h er e n ec es sa ry ; d es ig n in g th e le n g th o f p ro g ra m m e an d it s ru le s. C ar rie d o u t: Ye s/ N o En d 2 n d q u ar te r 13 . N um b er o f n ew m es sa g es d ev el op ed / n ew a p p c on te nt o r fe at ur es P ro ce ss N um er ic if m es sa g es ; d es cr ip tio n if a p p . A ny n ew c on te nt s h ou ld b e se nt to th e B H B M s ec re ta ria t (b h b m @ w h o. in t) . M ee tin g re co rd s an d m es sa g e lib ra ry A n n u al ly 14 . N um b er o f c on te nt d ev el op m en t or ad ap ta tio n fo cu s g ro up s/ u se r- te st in g se ss io n s P ro ce ss Th e nu m b er o f u se r- te st in g s es si on s w ith di ff er en t i nf or m an ts , e .g . f oc us g ro up s w ith ta rg et g ro up o r e xp er ts , a ca de m ic s. E xa m pl e: R ec or d s of fo cu s gr ou p d is cu ss io ns a nd m ee tin g re co rd s A nn ua lly (if n ew co nt en t is p ro du ce d af te r Ye ar  1) In fo rm an ts N o. o f gr ou ps To ta l n o. o f in fo rm an ts E xp er ts 1 6 Ta rg et gr ou p 3 24 15 . M es sa g es a d ap te d P ro ce ss Th is m ay b e re le va nt o n ly to Y ea r 1 C ar rie d o u t: Ye s/ N o En d 2 n d q u ar te r 16 . Ve rifi ca tio n o f fi d el it y of m es sa g es P ro ce ss Th is m ay b e re le va nt o n ly to Y ea r 1 C ar rie d o u t: Ye s/ N o En d 2 n d q u ar te r B E H E @ LT H Y B E M O B IL E A h an d b oo k on h ow to im p le m en t m Sa fe Li st en in g 11 4 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 17 . C on te nt m an ag em en t sy st em s et u p a n d /o r m ai nt ai n ed P ro ce ss Th e co nt en t m an ag em en t sy st em (C M S) m ay b e as s im p le a s an E xc el sp re ad sh ee t or W or d d oc um en t (a n d p as se d to te le co m m un ic at io n s co m p an ie s or b ot p ro vi d er s) ; o r as ad va n ce d a s an in -h ou se s of tw ar e so lu tio n w ith in te ro p er ab le p ro g ra m m in g la n g u ag e. T h e im p or ta nt fa ct or is th e ex is te n ce a n d m ai nt en an ce of a C M S. C ar rie d o u t: Ye s/ N o A n n u al ly 18 . % o f u se rs w h o sh ar ed th e m es sa g e co nt en t w ith o th er s n ot e n ro lle d in th e p ro g ra m m e O u tp u t Th is w ill a ss es s th e ad d iti on al re ac h of m es sa g es a n d c an b e u se d a s an in d ic at io n o f u se r- sa tis fa ct io n w ith th e co nt en t. Su rv ey s/ m es sa g e su rv ey re p lie s 19 . % o f u se rs w h o re p or te d sa ti sf ac ti on w it h t h e co n te n t th ey re ce iv ed ; e. g .: • Ea se o f u nd er st an di ng th e m es sa g es . • Ea sy to o p er at io na liz e ad vi ce o r i ns tr uc tio n . • C on te nt ap pr op ria te ne ss . • C on te nt re le va nc e. • P ro gr am m e le ng th . • Li ke lih oo d to s ug g es t pr og ra m m e to a fr ie nd . O u tp u t Th es e su g g es te d in d ic at or s ca n b e ra te d o n a L ik er t sc al e of 1 –5 . U se r- te st in g /s u rv ey s/ m es sa g e su rv ey re p lie s U se r- te st in g ph as e an d / or p os t- pr og ra m m e su rv ey A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g 11 5 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 20 . E st im at io n o f t h e p ro p or tio n o f m es sa g es re ad b y u se rs O u tp u t M es sa g es b ei n g s en t an d r ec ei ve d d oe s n ot m ea n g u ar an te e en g ag em en t. Th e fo llo w in g q u es ti on m ay b e h el p fu l: “H ow m an y of t h e te xt m es sa g es d id yo u r ea d ?” (S el ec t on e re sp on se o n ly ) 1 = N on e, I di dn ’t re ce iv e an y. 2 = N on e, I di dn ’t re ad a ny . 3 = So m e (le ss th an h al f) . 4 = M os t ( m or e th an h al f) . 5 = A ll, o r n ea rly a ll, o f t h e m es sa g es . Su rv ey s/ m es sa g e su rv ey re p lie s P ro m ot io n 21 . P ro m ot io n s tr at eg y co m p ile d P ro ce ss A p ro m ot io n s tr at eg y is a p la n la yi n g ou t th e p ro m ot io n a ct iv iti es th at w ill b e co m p le te d a n d th e tim e fr am e of co m p le tio n . C ar rie d o u t: Ye s/ N o A n n u al ly 22 . N u m b er o f p ro m ot io n ca m p ai g n s an d t yp e O u tp u t M ar ke tin g m ed iu m Ta rg et p op ul at io n N o. o f ev en ts R ea ch (n o. o f p eo pl e) M ar ke tin g m ed iu m pr ov id er a na ly tic s (r ad io pr og ra m m e/ br oa dc as tin g c om pa ny (T V )/s oc ia l m ed ia pl at fo rm , s ho ul d b e ab le to p ro vi de a na ly tic s) 6 m on th ly Fa ce b oo k ad ve rt G en er al p op ul at io n 3 A pp ro x. 30 0  00 0 R ad io ad ve rt G en er al p op ul at io n 4 A pp ro x. 80 0  00 0 P os te rs in un iv er sit ie s an d sc ho ol s St ud en ts 20 A pp ro x. 20 0 0 0 B E H E @ LT H Y B E M O B IL E A h an d b oo k on h ow to im p le m en t m Sa fe Li st en in g 11 6 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 23 . N um b er o f u se rs w ho w er e m ad e aw ar e of th e m Sa fe Li st en in g b y a gi ve n m ar ke tin g c ha nn el O u tp u t H ow d id y ou h ea r ab ou t th e p ro g ra m m e? Su rv ey th ro u g h m es sa g e ch an n el o r te le p h on e su rv ey A n n u al ly 24 . U se r- sa tis fa ct io n w ith p ro m ot io n c am p ai g n O u tc om e U se r- sa tis fa ct io n q u es tio n s co ul d in cl u d e: • W er e th e p ro m ot io n m at er ia ls e as y to un d er st an d ? • W er e th e m at er ia ls a p p ro p ria te fo r yo u an d y ou r co m m un it y? • W er e yo u a b le to s ig n u p u si n g th e in fo rm at io n p ro vi d ed b y th e p ro m ot io n ca m p ai g n? Su rv ey A n n u al ly 25 . % o f s ur ve ye d h ea lth -c ar e w or ke rs w ho k no w a b ou t (o r u se ) t he p ro gr am m e O u tp u t H av e yo u h ea rd o f m Sa fe Li st en in g? 1 = Y es 2 = N o 3 = I h av e su b sc rib ed to th e p ro g ra m m e A n n u al ly 26 . % o f s ur ve ye d u se rs w ho en co ur ag ed th ei r p ee rs to u se th e pr og ra m m e O u tc om e Th is in d ic at or a tt em p ts to u n d er st an d th e en g ag em en t of u se rs w ith th e p ro g ra m m e th ro u g h a q u es tio n s u ch a s: H av e yo u in fo rm ed a n d e n co ur ag ed an yo n e el se to p ar tic ip at e in m Sa fe Li st en in g? 1 = N o 2 = In fo rm ed , b u t n ot e n co ur ag ed 3 = In fo rm ed a n d e n co ur ag ed Su rv ey 6 m on th ly A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g 11 7 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y Fu n ct io n al it y an d t ec h n ol og y p er fo rm an ce 27 . Fu nc tio ni ng d as hb oa rd In p u t Th e av ai la b ili ty o f a fu n ct io n in g d as h b oa rd s h ou ld b e re co rd ed h er e. Ye s/ N o A n n u al ly 28 . E as e of s ig n in g u p : o n a sc al e of 1 –5 (“ 1” b ei n g ve ry d iffi cu lt ; “ 5” b ei n g ve ry e as y) O u tp u t A ny b ar rie rs to s ig n -u p m ay h av e a se rio u s ef fe ct o n th e n um b er o f su b sc rib er s. T h e p ro ce ss s h ou ld b e cl ea r an d n ot to o b ur d en so m e. T h is in d ic at or ca n c h ec k th is . I f u se rs re p or t th at th e p ro ce ss w as n ot e as y, re vi si tin g th e d es ig n o f t h e si g n -u p p ro ce d ur e m ay b e  ca lle d fo r. M es sa g e or te le p h on e su rv ey re sp on se s U se r- te st in g a n d 1s t q u ar te r 29 . N um b er o f s ys te m er ro rs O u tp u t A n e rr or in th e op er at in g s ys te m th at m ay o r m ay n ot im p ac t th e d el iv er y of co nt en t. Se rv ic e an al yt ic s1 1 an d /o r m es sa g e or te le p h on e su rv ey re sp on se s Q u ar te rly 30 . N um b er o f d ay s/ w ee ks of “s ys te m d ow n” O u tp u t Th is re co rd s th e am ou nt o f t im e th at co nt en t d id n ot re ac h u se rs d u e to a sy st em e rr or . Se rv ic e an al yt ic s M on th ly 31 . R at e of s u cc es sf u l d el iv er y of m es sa g es O u tp u t Th e n u m er at or is t h e m es sa g es d el iv er ed a n d t h e d en om in at or is t h e n u m b er o f m es sa g e at te m p ts . Se rv ic e an al yt ic s M on th ly 32 . N o. o f b u g s re p or te d an d  fi xe d O u tp u t R ec or d s sh ou ld b e ke p t on a ny te ch n ic al is su es w ith th e ap p . Te ch n ic al re p or t M on th ly 11 Se rv ic e an al yt ic s = ro u tin el y co lle ct ed b ac k- en d d at a fr om a p la tf or m t h at c an b e p ro vi d ed b y th e se rv ic e op er at or (e .g . t el ec om m u n ic at io n s p ro vi d er , c h at b ot p ro vi d er ). B E H E @ LT H Y B E M O B IL E A h an d b oo k on h ow to im p le m en t m Sa fe Li st en in g 11 8 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y R ea ch a n d r et en ti on 33 . N um b er o f p eo pl e w ho h av e su b sc rib ed / re gi st er ed O u tp u t Th e n u m b er o f t h os e w h o re g is te re d su cc es sf u lly . Se rv ic e an al yt ic s M on th ly 34 . % o f t ar g et p op u la ti on re g is te re d O u tp u t R ep or tin g th is a s a % o f t he to ta l n um b er of p eo pl e th at th e pr om ot io n c am pa ig ns w er e es tim at ed to re ac h , m ay b e he lp fu l to g au g e su cc es s. A lte rn at iv el y, th is co ul d b e ex pr es se d a s a % o f t he ta rg et p op ul at io n w ho a re a bl e to a cc es s th e pr og ra m m e (e .g . w ho h av e ac ce ss to a ph on e an d p ow er s ou rc e) (i f t he n um b er is k no w n) , e .g . ( (t ot al re gi st er ed u se rs / pr om ot io n re ac h) x1 0 0) Se rv ic e an al yt ic s an d p ro m ot io n a n al yt ic s 6 m on th ly 35 . N um b er o f n ew su b sc rib er s p er m on th O u tp u t Th is h el p s to fi n d o u t if th e in te re st in a n d n ee d fo r th e p ro g ra m m e is m ai nt ai n ed , a n d c an h el p to e st ab lis h w h et h er p ro m ot io n al s tr at eg ie s ar e ac h ie vi n g th ei r ai m o f r ec ru itm en t. Se rv ic e an al yt ic s M on th ly 36 . D em og ra p h ic in fo rm at io n a b ou t u se rs O u tp u t Th is c an h el p a ss es s w h ic h g ro up s ar e ac ce ss in g th e p ro g ra m m e an d w h ic h ta rg et -u se r g ro up s m ay n ot b e ac ce ss in g th e p ro g ra m m e eq ui ta b ly . Te le p h on e su rv ey s/ m es sa g e su rv ey re p lie s Q u ar te rly 37 . D ro p o u ts : *S TO P * r ep lie s (w ith u se of m es sa g in g) o r op te d ou t O u tc om e Th is c an b e u se d a p ro xy fo r u se r- sa tis fa ct io n . Se rv ic e an al yt ic s M on th ly A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g 11 9 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 38 . U se r en g ag em en t: % o f u se rs re ta in ed fo r: • 1 w ee k of t h e p ro g ra m m e • 1 m on th o f t h e p ro g ra m m e • 3 m on th s of t h e p ro g ra m m e • En tir e p ro g ra m m e O u tc om e Th is c an b e ca pt ur ed b y a re pr es en ta tiv e te le ph on e su rv ey , a m es sa g e su rv ey or th ro ug h a na ly tic s of *S TO P * r ep lie s/ op t- ou ts . D ep en di ng o n h ow th is d at a is c ap tu re d , t he d efi ni tio n o f r et en tio n w ill c ha ng e an d th e de no m in at or u se d in th e % c al cu la tio n w ill c ha ng e (n um b er su rv ey ed v s nu m b er e nr ol le d in th e pr og ra m m e) . Su rv ey /s er vi ce an al yt ic s M on th ly 39 . % m es sa g es /s u rv ey s se n t fr om p ro g ra m m e th at a re re sp on d ed to b y u se r O u tp u t Th is in di ca to r m ea su re s ac tiv e en g ag em en t an d in di ca te s w h et h er u se rs a re c om p ly in g w ith in st ru ct io n s or  m es sa g in g a s di re ct ed . Se rv ic e an al yt ic s 6 m on th ly B eh av io u r ch an g e 40 . % o f s ur ve ye d u se rs w h o re du ce d th ei r av er ag e lis te ni ng v ol um e O u tc om e E xa m p le q ue st io n s: • O n a s ca le o f 1 –1 0 , w h at is y ou r p re fe rr ed li st en in g v ol u m e on y ou r p er so n al a u d io d ev ic e (e .g . y ou r sm ar tp h on e) w h en y ou a re a t h om e? • O n a s ca le o f 1 –1 0 , w h at is y ou r p re fe rr ed li st en in g v ol u m e on y ou r p er so n al a u d io d ev ic e (e .g . y ou r sm ar tp h on e) w h en y ou a re t ra ve lli n g on p u b lic t ra n sp or t? Su rv ey A n n u al ly B E H E @ LT H Y B E M O B IL E A h an d b oo k on h ow to im p le m en t m Sa fe Li st en in g 12 0 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 41 . % o f s u rv ey ed u se rs w h o re d u ce d t h e am ou n t of t im e sp en t lis te n in g o n t h ei r p er so n al a u d io d ev ic e O u tc om e E xa m p le q u es ti on s: In t h e p as t w ee k, h ow m an y h ou rs e ac h d ay (o n a ve ra g e) d id y ou li st en to m u si c u si n g y ou r p er so n al a u d io d ev ic e (e .g . yo u r sm ar tp h on e) ? 1 = L es s th an 1 h ou r/ d ay 2 = 1– 3 h ou rs /d ay 3 = 3– 5 h ou rs /d ay 4 = 5 –8 h ou rs /d ay 5 = M or e th an 8 h ou rs /d ay Su rv ey A n n u al ly 42 . % o f s u rv ey ed u se rs w h o re d u ce d t h e am ou n t of t im e sp en t in n oi sy e n te rt ai n m en t ve n u es O u tc om e In t h e p as t w ee k, h ow m an y h ou rs d id yo u s p en d a t p la ce s th at p la y am p lifi ed m u si c (e .g . fi tn es s cl as se s, g ym s, c lu b s, b ar s, c on ce rt s, n oi sy s p or ts a re n as )? 1 = L es s th an 1 h ou r 2 = 1– 3 h ou rs 3 = 3– 5 h ou rs 4 = 5 –8 h ou rs 5 = M or e th an 8 h ou rs Su rv ey A n n u al ly 4 3. % o f s u rv ey ed u se rs w h o in cr ea se d t h ei r u se of h ea rin g p ro te ct io n (e .g . e ar p lu g s) w h en v is it in g n oi sy en te rt ai n m en t ve n u es O u tc om e W h en v is it in g p la ce s th at p la y am p lifi ed m u si c (e .g . fi tn es s cl as se s, g ym s, c lu b s, b ar s, c on ce rt s, n oi sy s p or ts a re n as ), h ow of te n d id y ou u se h ea rin g p ro te ct io n (s u ch a s ea rp lu g s) ? 1 = N ev er 2 = R ar el y 3 = So m et im es 4 = M os t of t h e tim e 5 = A lw ay s Su rv ey A n n u al ly A n n ex 8 . M on it or in g a n d e va lu at io n in d ic at or s fo r m Sa fe Li st en in g 12 1 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 4 4. % o f s ur ve ye d u se rs w h o re p or te d re d u ce d h ea rin g O u tc om e W h ic h o f t h e fo llo w in g b es t d es cr ib es yo ur (o r, yo ur c h ild ’s) c ur re nt h ea rin g st at u s? 1. I (o r, m y ch ild ) h av e n o d iffi cu lt y in h ea rin g 2. I (o r, m y ch ild ) fi n d it d iffi cu lt to h ea r w h at is b ei n g s ai d w h en I am in a n oi sy s itu at io n s u ch a s a b ar o r re st au ra nt . 3. I (o r, m y ch ild ) fi n d it d iffi cu lt to h ea r w h at is b ei n g s ai d e ve n w h en I am in a q ui et p la ce . 4 . I (o r, m y ch ild ) c an o n ly h ea r w or d s w h en th ey a re s p ok en lo u d ly . 5. I (o r, m y ch ild ) h av e a h ea rin g lo ss an d u se h ea rin g te ch n ol og y (s u ch a s h ea rin g a id s or im p la nt s) . 6. I (o r, m y ch ild ) h av e a h ea rin g lo ss , b u t I d o n ot u se a ny h ea rin g te ch n ol og y. 7. I (o r, m y ch ild ) a m D ea f a n d u se s ig n la n g u ag e. Su rv ey A n n u al ly 45 . % o f s ur ve ye d u se rs w h o re p or te d o n se t of  ti n n itu s O u tc om e H av e yo u e xp er ie n ce d a ri n g in g o r b uz zi n g s ou n d in th e ea r (t in n itu s) ? 1 = N ev er 2 = So m et im es 3 = O ft en 4 = A ll of th e tim e Su rv ey A n n u al ly B E H E @ LT H Y B E M O B IL E A h an d b oo k on h ow to im p le m en t m Sa fe Li st en in g 12 2 N o. In di ca to r Ty p e C om m en t D at a co lle ct io n m et ho d Fr eq ue nc y 46 . % o f s ur ve ye d u se rs w h o ch ec ke d th ei r h ea rin g O u tc om e In th e p as t ye ar , h av e yo u c h ec ke d y ou r h ea rin g e ith er u si n g a d ig ita l a p p lic at io n or in a c lin ic ? Su rv ey A n n u al ly K n ow le d g e g ai n s 47 . % o f u se rs w h o in cr ea se d th ei r q ui z sc or es p re - an d p os t- p ro g ra m m e in k n ow le d g e q ui zz es (c h at b ot s an d a p p s if ap p lic ab le ) O u tc om e Th e q ui z ca n in cl u d e q u es tio n s su ch a s: • W h at c an y ou d o to p ro te ct y ou r ea rs in a n oi sy p la ce ? • A t w h at le ve l o f v ol um e is th e so un d fr om y ou r lis te n in g d ev ic e un lik el y to h ar m y ou r ea rs ? Te le p h on e su rv ey s/ m es sa g e su rv ey re p lie s 6 m on th ly 4 8. % o f s ur ve ye d u se rs w ho re p or te d h av in g im pr ov ed k no w le dg e re ga rd in g s af e lis te ni ng O u tc om e • A ft er c om p le tin g t h e p ro g ra m m e, d o yo u fe el t h at y ou u n d er st an d m or e ab ou t th e ri sk s as so ci at ed w it h lis te n in g to lo u d s ou n d s? • A ft er c om p le tin g t h e p ro g ra m m e, d o yo u fe el t h at y ou u n d er st an d m or e ab ou t h ow y ou c an re d u ce y ou r ri sk o f h ea rin g lo ss a n d t in n itu s? Te le p h on e su rv ey s/ m es sa g e su rv ey re p lie s A n n u al ly 123 Annex 9. Benefits and risks of different software models Examples Benefits Risks Custom-developed software (Build a software system from scratch) Project Optimize demonstration projects in Albania, Guatemala, Senegal, and Vietnam. • Control over technology, functionality, and design. • The development experience creates ownership and improves sustainability. • It is possible to engage the local IT industry. • Custom development tends to be difficult to manage within time and budget. • Satisfaction is not guaranteed as the end product depends on the capabilities of the technical team. • Long-term support depends on the continued availability of individuals with the capability to maintain the software set-up. 124 BE HE@LTHY BE MOBILE A handbook on how to implement mSafeListening Examples Benefits Risks Commercial “off-the-shelf” software Buy a commercially-available product. Sage Enterprise Resource Planning (This is in used in many countries in Francophone Africa for essential medicines). • The lead time from selection to implementation is generally shorter. • The product can be evaluated prior to buying. • The product is maintained and upgraded (at a cost). • The product has usually been tested and refined during use in other implementations. • Often expensive and sold with unclear and complex fee structures (as with, for example, a fee-per-server processor). • Commercial off- the-shelf software is not often designed for implementation in low-resource settings. Free packaged software Software developed by a donor organization or technical agency. Alternatively, a system developed by another country with similar technical standards. USAID/John Snow, Inc.: • PipeLine • Supply Chain Manager World Health Organization: • Vaccination Supplies Stock Management tool. • District Vaccine Data Management tool. • Shorter lead time. • Possibility to evaluate prior to purchasing. • No upfront cost (but maintaining or customizing the product may require investment). • There is often no contract; servicing and warranty for bug-fixing therefore depends on the goodwill of one or two technical experts and there is no institutional support. • Many implementation and running costs are hidden. Annex 9. Benefits and risks of different software models 125 Examples Benefits Risks Open-source software The source code as well as the software product is freely available. Often, a community has been formed to support the open-source software. OpenLMIS.org OpenMRS.org DHIS2.org OpenXData.org • It is legally possible to make changes to the software. • Local IT industry can be engaged. • It is possible to benefit from communities and share development costs with other organizations • Can end up with a poorly supported product. • A loosely knit community might not be able to provide the business relationship you need. • Some of the implementation and running costs are hidden. Software as a service (SaaS) Database and application hosted on remote servers; software is sold (or offered cost-free) as a service that can be contracted per user and per month or year. Logistimo Magpi • Highly feasible to implement and maintain. • Clear information on the costs to implement and run the application. • Investment in improved software can easily be shared among customers. • Data hosted on remote servers (not always aligned withnational policy). • Ministries of health are not often well positioned to pay a regular service fee. 126 Annex 10. Role of aggregators Figure A10.1: Role of aggregators flow chart Cost for service provider Phone # SMS 1 SMS Management Application 3 Short Code Cost for Short Code Could be negotiated if for social purposes with Telecom Regulator Cost for aggregator per SMS (aggregators pay directly to operators) The deal can include the short code as well. Need to establish agreement with aggregator. Service provider (could be cloud hosted: local or international) Locally hosted Local aggregator 2 SMS Aggregator/ Gateway Company 4 Mobile Network Operators (MNOs) Annex 10. Role of aggregators 127 1. SMS Management Application: An SMS management application is used to write and read messages, manage contacts, and analyse data. This software can take the form of a website, a server, a desktop application, or a small piece of software that is installed on a phone. 2. SMS aggregator/gateway company: Once messages have been written on the software platform, the messages need to reach the global phone networks. The gateway that establishes this connection can either be a SIM card that is physically inserted into a device such as a phone or GSM modem, or an SMS aggregator. Aggregators are web-based services that specialize in sending messages to mobile phones globally. In many countries they also provide their users with virtual local phone numbers to receive messages. Many SMS management applications ask their users to choose an aggregator depending on the country in which they operate; others have standing agreements with specific aggregators. In most countries, all platforms can be connected with national mobile network operators, although this can be more labour intensive. 3. Short codes: Contacts need to know where to send their messages. To make this as easy as possible it can be helpful to use a memorable numerical or alphanumerical short code, such as “1234” or “REFAID” instead of a long telephone number such as +232-123-444-1122. The process to obtain short codes differs from country to country and involves substantial additional costs (i.e. in the range of hundreds of US$ per month). Using a short code also always involves direct negotiations with a mobile network operator, which can delay the start of the programme. 4. Mobile Network Operator (MNO): A mobile network operator or MNO (also known as a wireless service provider, wireless carrier, cellular company, or mobile network carrier), is a provider of wireless communications services that owns or controls all the elements necessary to deliver services to end-users, including radio spectrum allocation, wireless network infrastructure and other necessary components. World Health Organization 20 Avenue Appia 1211 Geneva 27 Switzerland website: www.who.int/initiatives/behealthy e-mail: bhbm@who.in