Be he@lthy, be mobile: a handbook on how to implement mDementia

A handbook on how to implement mDementia Be He@lthy, Be Mobile: a handbook on how to implement mDementia ISBN (WHO) 978-92-4-001996-6 (electronic version) ISBN (WHO) 978-92-4-001997-3 (print version) ISBN (ITU) 978-92-61-33161-0 (print version) ISBN (ITU) 978-92-61-33171-9 (electronic version) ISBN (ITU) 978-92-61-33181-8 (epub version) ISBN (ITU) 978-92-61-33191-7 (mobi version) © World Health Organization and International Telecommunication Union, 2021 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial- ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https:// creativecommons.org/licenses/by-nc-sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that WHO or ITU endorses any specific organization, products or services. The unauthorized use of the WHO or ITU names or logos is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization (WHO) or the International Telecommunication Union (ITU). Neither WHO nor ITU are responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/ mediation/rules). Suggested citation. Be He@lthy, Be Mobile: a handbook on how to implement mDementia. Geneva: World Health Organization and International Telecommunication Union, 2021. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris Sales, rights and licensing. To purchase WHO publications, see http://apps.who.int/bookorders. ITU Publications can be obtained from ITU Bookshop http:// www.itu.int/en/publications. To submit requests for commercial use and queries on rights and licensing, see http://www.who.int/about/licensing. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party- owned component in the work rests solely with the user. General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO or ITU concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO or ITU in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO and ITU to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO or ITU be liable for damages arising from its use. Cover Photo credit: Istockphoto/ kate_sept2004 Layout and design: Optima Graphic Design Consultants Ltd. CONTENTS Operations Management 12 Undertaking a situational assessment 13 Establishing programme leadership 14 Performing a stakeholder analysis 16 Engaging with civil society 19 Forming strategic partnerships with the private sector 19 Developing and managing your workplan 21 Funding your mDementia programme 24 Budgeting a scalable mHealth programme 24 Sources of sustainable funding 25 Technology specifications 27 Selecting the appropriate technology for your context 29 Technology implementation needs 32 Software needs 32 Selecting a software provider 33 The role of telecoms operators 33 Content development and adaptation 36 Adapting mDementia programme design 38 Adapting the existing mDementia content library 38 Translation 39 Adapting messages 39 Local expert review 40 Selecting content for focus group testing 40 Ecological testing 41 Additional content development 43 Adapting content library for voice, messenger apps or chatbots 44 Promotion, participation and retention 45 Promoting the mHealth programme 46 Specific considerations for the mDementia programme 46 Participation in the mHealth programme 52 Retention 53 Acknowledgements 3 Executive summary 4 Introduction 5 1 CLICK TABS TO JUMP TO SECTION 2 3 4 1 Monitoring and evaluation of mDementia 55 Planning M&E 56 Process M&E 59 Outcome M&E 60 Selecting indicators for M&E 60 Designing evaluation 62 Monitoring and evaluation data analysis, reporting and dissemination 62 Annexes 66 Annex 1 Pre- and post-programme questionnaires 67 Annex 2 M&E Indicators 77 Annex 3 Benefits and risks of different software models 89 Annex 4 The role of aggregators 90 Annex 5 Main themes and operationalizing strategies for message development 91 Annex 6 Adapting content library for voice, messenger apps or chatbots 92 Annex 7 Design of past evaluation studies of BHBM programmes 98 Annex 8 Programme monitoring questions 99 5 A CLICK TABS TO JUMP TO SECTION 2 Acknowledgements This handbook was created through a collaboration between The World Health Organization (WHO) Department of Digital Health and Innovation, under the leadership of the Chief Scientist’s Office and Dr Soumya Swaminathan, and the WHO Department of Mental Health and Substance Use jointly with the Development Bureau of International Telecommunications Union (ITU). WHO and ITU gratefully acknowledge the following contributors to this handbook: Handbook preparation WHO/ITU Be He@lthy, Be Mobile: Melissa Harper Shehadeh, Roman Chestnov, Javier Elkin, Hani Eskandar, Surabhi Joshi, Sameer Pujari, Mariam Shokralla, Ayush Shukla. Brain Health Unit, Department of Mental Health and Substance Use, WHO: Devora Kestel, Director and the Brain health Unit - Neerja Chowdhary, Tarun Dua, Stéfanie Freel and Katrin Seeher. Content development Lidan Zheng (Neuroscience Research Australia, University of New South Wales (UNSW), Australia). Members of the Be He@lthy, Be Mobile mDementia Informal Expert Group: Kaarin Anstey (Ageing Futures Institute, UNSW, Australia), Paola Barbarino (Alzheimer’s Disease International), Henry Brodaty (Dementia Centre for Research Collaboration, UNSW, Australia), John Mark Bwanika (Uganda Healthcare Federation, Uganda), Babafunke Fagbemi (Centre for Communication and Social Impact, Nigeria), Krishnan Ganapathy (Apollo Telemedicine Services, India), Riadh Goudier (Tunisian Alzheimer’s Disease Association, Tunisia), Miia Kivipelto (Karolinska Institute, Sweden), Déborah Oliviera (Federal University of Sao Paulo, Brazil), Anne-Margriet Pot (Ministry of Health, the Netherlands), Chris Roberts and Jayne Goodrick (Alzheimer Europe). Guidance Be He@lthy, Be Mobile Steering Committee members, from WHO: Mr Bernardo Mariano Jr, Director, Digital Health and Innovations; Dr Ren Minghui, Assistant Director General / Dr Bente Mikkelsen, Director, Non-communicable diseases; Dr Naoko Yamamoto, Assistant Director General / Dr Ruediger Krech, Director, Health Promotion. From ITU: Doreen Bogdan-Martin, Director, Telecommunication Development Bureau; Stephen Bereaux, Deputy to Director, Telecommunication Development Bureau; Marco Obiso, Chief, A.I. Digital Network and Society Department, Telecommunication Development Bureau. Further contributions Orsolya Lhasz (University of Cranfield, UK). WHO Regional Offices: Marcelo D’Agostino (WHO Americas Regional Office [AMRO]), Nazneen Anwar (WHO South East Asian Regional Office [SEARO]), Florence Baingana (WHO African Regional Office [AFRO]), Claudina Cayetano (WHO AMRO), Daniel Chisholm (WHO European Regional Office [EURO]), Patricia Codyre (WHO SEARO), Jun Gao (Western Pacific Regional Office [WPRO]), Clayton Hamilton (WHO EURO), Ba Housseynou (WHO AFRO), Jagdish Kaur (WHO SEARO), Carmen Martinez (WHO AMRO), Ahmed Mandil (WHO Eastern Mediterranean Office [EMRO]), Mohamed Hasan Nour (WHO EMRO, Renato Oliveira e Souza (WHO PAHO), Khalid Saeed (WHO EMRO) and Martin Vandendyck (WHO WPRO). WHO Headquarters: Elena Altieri, Virginia Arnold, Fiona Bull, Maria De Las Nieves Garcia Kazal, Taskeen Khan, Dzmitry Krupchanka and Yuka Sumi. Administrative support Noha Gamal El-Din, Isabel Hall, Igy Mohamed. Editing Angela Burton. 3 Executive summary The mDementia programme will complement existing health and social care offered by informal carers, health care professionals and civil society service providers. This handbook provides guidance on how to develop, integrate, implement and evaluate a national mDementia programme in five key areas: operations management; content development and adaptation; promotion participation and retention; technology specifications; and monitoring and evaluation. “Be He@lthy, Be Mobile” (BHBM) is a global initiative led by the World Health Organization (WHO) and the International Telecommunications Union (ITU). It is based on the use of mobile technology for health (mHealth) to address diseases and health issues such as smoking, diabetes, ageing, cardiovascular diseases and chronic respiratory diseases. An estimated 50 million people live with dementia worldwide, and with approximately 10 million new cases every year, dementia is projected to affect 152 million people by 2050. Dementia is a major cause of disability and care dependency among older adults, and impacts every aspect of a person’s life. There also continues to be much stigma and discrimination against people with dementia and their carers, heightening the already significant psychological, social, emotional and financial impacts of dementia on families and communities. To help address these issues, Be He@lthy, Be Mobile has developed the mDementia programme that uses mHealth (whereby devices such as mobile phones, patient monitoring devices, personal digital assistants and other wireless devices are used to support medical and public health practice) to provide health information to those at risk of developing dementia and to support carers of people living with dementia. The reach of dementia prevention campaigns and caregiver support activities may be drastically enhanced with the use mHealth. The mDementia handbook and programme content was prepared by an international group of experts in mHealth, behavioural science and dementia, in collaboration with WHO and ITU. All content in this handbook is based on WHO guidelines, existing research evidence, existing WHO e-health content, and/or expert opinions. Content for the programme in the form of a comprehensive message library with suggested message algorithms is available on request from bhbm@who.int. The message library uses evidence- based behaviour change techniques to help persons at risk of dementia and their carers. All content and programming guidance described here should be considered as examples and adapted to the local context of each participating country. 4 Introduction THE GLOBAL CHALLENGE OF DEMENTIA: ACTION TO DATE An estimated 50 million people live with dementia worldwide, 60% of whom reside in low- and middle- income countries. With approximately 10 million new cases every year, dementia is projected to affect 152 million people by 20501. Dementia is a major cause of disability and dependency among older adults and impacts every aspect of a person’s life (see Box 1). There is also much stigma and discrimination against people with dementia and their carers, heightening the already significant psychological, social, emotional and financial impacts of dementia on families and communities. The financial cost of dementia to countries is also staggering. In 2015, the total global societal cost of dementia was estimated to be US$ 818 billion – equivalent to 1.1% of global gross domestic product. This figure is expected to reach US$2 trillion by 20302. To address dementia as a global challenge, the 70th World Health Assembly adopted the Global Action Plan on the Public Health Response to Dementia 2017–20253 in May 2017. This plan represents an international commitment by Member States to improve the lives of people with dementia, their carers and families, and to create a world where they can live well and receive the care and support they need to fulfill their potential with dignity, respect, autonomy and equality. The plan includes seven action areas, each underpinned by a global target to be achieved by countries together with partners such as civil society, academia and the private sector by 2025: 1. Dementia as a public health priority 2. Dementia awareness and friendliness 3. Dementia risk reduction 4. Dementia treatment, care, and support 5. Support for dementia carers 6. Information systems for dementia 7. Dementia research and innovation This handbook is especially relevant to action area 3 – dementia risk reduction; and action area 5 – support for dementia carers. BOX 1: DEFINING DEMENTIA Dementia is the deterioration in memory, thinking, behaviour and the ability to perform everyday activities. Although dementia mainly affects older people, it is not a normal part of ageing. Alzheimer’s disease is the most common form of dementia and may contribute to 60–70% of cases. Dementia is one of the major causes of disability and dependency among older people worldwide, having a physical, psychological, social, and economic impact, not only on people with dementia, but also on their carers, families and society. Dementia risk reduction While age is the strongest known risk factor for cognitive decline, dementia (where two or more functions such as memory or language are impaired, thereby interfering with daily living) is not an inevitable consequence of ageing. The actions to reduce dementia risk in the Global Action Plan specifically aim to encourage the proactive management of modifiable risk factors to delay or slow the onset or progression of the disease. These risk factors include low physical activity, tobacco use, unhealthy diets and harmful use of alcohol. Certain medical conditions are also associated with an increased risk of dementia, including hypertension, diabetes, dyslipidaemia, obesity and depression. In 2019, WHO launched Guidelines on risk reduction of cognitive decline and dementia4 to support health care providers as well as governments, policy- makers and other stakeholders to strengthen their efforts to reduce the risks of cognitive decline and dementia through a public health approach. As many of the risk factors for dementia are shared with those of other noncommunicable diseases (NCDs), the key recommendations can be effectively integrated into programmes for tobacco cessation, cardiovascular disease (CVD) risk reduction, and improved nutrition. 5 Support for dementia carers And it is not just the needs of dementia patients that need addressing: tailored support and services to meet the physical, mental and social demands of those caring for people with dementia are also essential to prevent a decline in carers’ physical and mental health and social well-being. For this reason, in 2017 WHO launched iSupport, an online knowledge and skills training programme to help prevent and/or decrease the mental and physical health problems associated with caregiving, and to improve carers’ quality of life. The programme includes five interactive modules focused on: (i) general information about dementia; (ii) information related to being a carer; (iii) self-care; (iv) providing care for the person with dementia; and (v) addressing symptoms of dementia. Each module is organized into lessons which include tips and interactive exercises that carers can complete at their own pace. BE HE@LTHY, BE MOBILE AND mHEALTH FOR DEMENTIA Be He@lthy, Be Mobile uses mHealth (see Box 2) to provide health information to those at risk of developing dementia, and to support carers of people living with dementia. BHBM programmes demonstrate that the reach of dementia prevention campaigns and caregiver support activities may be drastically enhanced with the use of mHealth, especially when delivered as part of core health services. Systematic review evidence also shows moderate effects in favour of mHealth messaging to support behaviour change initiatives in a complementary manner across a number of risk factor areas, e.g., for quitting smoking5, for diabetes self-management6, to increase medication adherence in people who have cardiovascular diseases7 and to increase physical activity levels8. This encouraging evidence and the success of other BHBM programmes (e.g. the mTobaccoCessation and mDiabetes programmes), coupled with the recent release of the dementia risk reduction guidelines4, provides a strong foundation for delivering dementia risk reduction through mobile phones. As outlined in the WHO Global Strategy on Digital Health 2020–2025, it is important to involve stakeholders in planning and implementation of mHealth programmes, including communities, professional associations, and patient and family organizations and other stakeholders . BOX 2: ABOUT BE HE@LTHY, BE MOBILE Be He@lthy, Be Mobile is a global initiative led by the World Health Organization (WHO) and the International Telecommunications Union (ITU) to encourage and enable the use of mobile technology for health (mHealth). The aim is to help combat noncommunicable diseases such as diabetes, cancer, cardiovascular diseases and chronic respiratory diseases, and enable healthy ageing, through means such as text messages and apps. Be He@lthy, Be Mobile (BHBM) also helps Member States integrate mHealth behaviour change programmes into their national health systems. BHBM identifies concrete actions that can be undertaken by Member States to help them meet the targets set out in the Global Action Plan for the Prevention and Control of NCDs 2013–20209. And with the emergence of COVID-19 and other pressing public health issues, BHBM is now expanding to other health areas where behaviour change can have a positive impact on health and well-being, in line with the Sustainable Development Goals (e.g. SDG 3)10. 6 The mDementia programme BHBM’s mDementia programme complements existing health and social care offered by informal carers, health care professionals and civil society service providers. It does this by providing information on specific and actionable behaviour change strategies11 for dementia risk reduction and caregiving through mobile technology. It is recommended that mDementia is not conducted in isolation, but rather as part of existing policies and awareness campaigns that support dementia risk reduction and carer well-being. The programme is currently divided into two modules: mDementiaPrevention and mDementiaSupport, with the flexibility to add further content based on identified needs. Table 1 provides an overview of the mDementia programme. TABLE 1: OVERVIEW OF THE mDEMENTIA PROGRAMME mDEMENTIA Modules mDementia Prevention mDementia Support Source Developed based on WHO Guidelines for Risk Reduction of Cognitive Decline and Dementia Based on iSupport Aim To stimulate behavior change that contributes towards risk reduction of cognitive decline and dementia To improve the mental health and wellbeing of those caring for people with dementia Target Audience • People at risk of developing dementia • General population • Carers of people with dementia Target Areas • Managing weight and cardiovascular disease (e.g. dyslipidaemia, hypertension, diabetes) • Reducing alcohol and tobacco consumption • Increasing physical and cognitive activity • Consuming a healthier diet • Increasing dementia and dementia care literacy (e.g. what it means to care for someone with dementia) • Equipping carers with skills and techniques to better manage the stress associated with caring (e.g. a more positive perception of their role) • Tips and information on everyday care for a person with dementia and what to expect as dementia progresses 7 mDementiaPrevention The mDementiaPrevention programme provides complementary information to generate behaviour change that helps reduce the risk of cognitive decline and dementia. The programme uses positive behaviour change techniques to support the following: 1) health promotion among healthy people who are middle-aged or older; 2) prevention among those with identified risk factors for dementia and cognitive decline; and 3) prevention of further decline among those with mild cognitive impairment (MCI). As with many digital health interventions, the programme is designed to be implemented alongside other dementia prevention activities such as awareness campaigning or NCD prevention and healthy lifestyle services. The message content for this programme was designed based on WHO’s Guidelines for Risk Reduction of Cognitive Decline and Dementia4. The guidelines include recommendations on interventions for physical activity, tobacco cessation, nutrition, reducing harmful use of alcohol, cognitive training, weight management and management of hypertension, diabetes, and dyslipidaemia to reduce the risk of cognitive decline and dementia. (Management of depression, hearing loss and social activity were also included as interventions in the guidelines but the resultant evidence was not sufficient to provide a recommendation. Thus these risk factors are not included as part of the mDementiaPrevention programme.) Additionally, it was concluded that conducting cognitive training in message format was beyond the scope of the programme. For this reason, the message content related to cognitive training instead includes messages that encourage ways to increase lifelong cognitive activity. Within the mDementiaPrevention programme, content is tailored to different risk categories, e.g.: • People who have cardiovascular disease (dyslipidaemia, hypertension, diabetes) • People who are overweight and obese • People who consume excessive amounts of alcohol • People who use tobacco products • People with mild cognitive impairment All users should receive messages on strategies to increase their cognitive and physical activity, and improve their daily nutrition. Additional messages for other risk factors can be included on an individualized basis depending on the user’s risk profile. Information on individual risk factors for programme tailoring can be obtained through a pre-programme screening questionnaire (see Annex 1 Pre- and post-programme questionnaires on page 67 for an example) or by targeting known at-risk groups in the population. In addition, it is recommended that the weight loss and dyslipidaemia messages are restricted to those in mid-life (and not late-life) following WHO guideline recommendations. The mDementiaPrevention programme also contains messages that were not part of the WHO guidelines on medication use for people who take medication for cardiovascular disease and memory strategies for users who have memory problems. Table 2 (on page 9) contains a breakdown of the messages by target audience. 8 TABLE 2: BREAKDOWN OF mDEMENTIAPREVENTION MESSAGES BY TARGET AUDIENCE TARGET AUDIENCE AGE GROUP MESSAGE CONTENT All users All Increasing cognitive activity Increasing physical activity Eating a healthy diet People who are overweight/obese Mid-life Weight management People who have hypertension All Hypertension management People who have high cholesterol/dyslipidaemia Mid-life Cholesterol management People who have diabetes All Diabetes management People who consume alcohol excessively All Reducing excessive alcohol consumption People who use tobacco All Quitting tobacco People who use medication for cardiovascular disease All Adhering to prescribed medication use People who have reported memory problems All Memory strategies The BHBM mDementiaPrevention programme also includes a short stream of messages aimed at primary health care workers to raise awareness that healthy lifestyle advice is effective in reducing cardiovascular disease and reducing the risk of developing cognitive decline. mDementiaSupport The mDementiaSupport programme aims to improve the mental health and well-being of those caring for people with dementia. It focuses on preventing and/or reducing the mental health problems associated with caregiving, and improving the quality of life of those caring for people with dementia by equipping them with the skills and knowledge to manage the stress associated with caregiving. The mDementiaSupport programme is based on the iSupport programme, and offers practical, easy-to- read tips for carers of people with dementia that can be drawn upon when needed. For carers who have already completed the comprehensive iSupport programme, mDementiaSupport acts as a refresher, reinforcing previously acquired caregiving skills and knowledge. The mDementiaSupport programme mirrors the iSupport module structure (see Figure 1 on page 10). The only difference is that a separate message library has been created for the communication aspects of the “Being a Carer” module due to the quantity of messages related to this topic. The mDementiaSupport programme thus contains message libraries on six topics: introduction to dementia, communication, being a carer, caring for me, providing everyday care, and dealing with behaviour changes. 9 FIGURE 1: MODULE STRUCTURE OF THE iSUPPORT PROGRAMME1 1 The mDementiaSupport programme is closely based on the existing iSupport module structure Module 1 Introduction to Dementia • Introduction to dementia Module 4 Providing everyday care • Eating and drinking – more pleasant mealtimes • Eating, drinking and preventing health problems • Toileting and continence care • Personal care • An enjoyable day Module 2 Being a carer • The journey together • Improving communication • Supported decision-making • Involving others Module 5 Dealing with behaviour changes • Introduction to behaviour changes • Memory loss • Aggression • Depression, anxiety and apathy • Difficulty sleeping • Delusions and halluciniations • Repetitive behaviour • Walking and getting lost • Changes in judgement • Putting it all together Module 3 Caring for me • Reducing stress in everyday life • Making time for pleasant activities • Thinking differently 10 ABOUT THIS HANDBOOK This handbook is one of a series of practical BHBM resources that help Member States devise, plan and roll out mHealth programmes. It provides guidance on how to develop, integrate, implement and evaluate a national mDementia programme in five key areas: operations management; content development and adaptation, promotion participation and retention, technology specifications; and monitoring and evaluation.2 2 All behaviour change content in these handbooks is derived from evidence-based WHO guidelines. The handbook was prepared by an international group of experts from WHO and ITU. It supports the newly approved WHO Global Strategy on Digital Health 2020– 2025 and the Global Action Plan on the Public Health Response to Dementia 2017–2025. It acts as a catalyst for capacity building and knowledge transfer in relation to dementia, and providing guidance on the governance and roll out of sustainable, person-centred mHealth dementia programmes. It is targeted at government officials, WHO staff members, academics, and implementing partners in-country who are involved in large-scale mHealth programmes. The following chapters describe how an mHealth programme can be used to strengthen existing dementia prevention and support programmes, sets out the steps for successful mHealth dementia programme implementation, and outlines the implementation considerations when planning for a national mHealth dementia programme. The content can be adapted to support Member Countries’ own national guidelines and existing health system interventions. The handbook is predominantly aimed at government implementers of scale mHealth for dementia programming and other stakeholders involved in the process. Other potential users could include dementia promotion and prevention campaign planners and those running scale carer support services. Activities in the handbook can be undertaken with or without the support of BHBM’s experienced technical officers (the country implementation team), which is on hand to provide advice. Resources such as checklists, templates and project management and implementation tools can be found in each section. Electronic tools such as spreadsheets and Word documents can be provided on request from the BHBM country support team at bhbm@who.int. Using this handbook Readers are encouraged to read the whole handbook prior to implementing an mDementia programme, as many of the activities outlined across the five chapters are to be undertaken simultaneously. For example, though monitoring and evaluation is the last guidance section, this activity should be considered from the outset of your programme; similarly, promotional activities will be ongoing to maintain subscribers. Additional tools and resources to support these processes can be found on the BHBM website, and the WHO mDementia content library is available on request through the website or via email to bhbm@who.int. It is currently formatted as SMS messages (as SMS is the most equitable way to provide mHealth to the majority of people), but these messages can be adapted for dissemination through messenger apps, chatbots or stand-alone apps. The content library is free for use by any government health department or provider with the request that users share progress and evaluation reports with BHBM. For-profit companies can only use the content for commercial purposes after entering into a direct agreement with BHBM. To find out more about country assistance or use of WHO content, contact bhbm@who.int. 11 1 Operations Management Undertaking a situational assessment 13 Establishing programme leadership 14 Performing a stakeholder analysis 16 Engaging with civil society 19 Forming strategic partnerships with the private sector 19 Developing and managing your workplan 21 Funding your mDementia programme 24 Budgeting a scalable mHealth programme 24 Sources of sustainable funding 25 12 1 Operations management This section outlines the key steps for project planning, organizing and management activities both on a strategic level and for the day-to-day running of the mDementia programme. It explores programme leadership and partnerships, work plan development for mDementia, and securing funding. Adopting a systems approach to integrating mHealth programmes with existing health programmes can help deliver these programmes holistically – from raising awareness of health issues, to prevention, diagnosis, treatment, and support for patients and their families. This is because, for example, prevention cannot take place without awareness raising amongst the population. Awareness raising increases the demand for diagnosis, and diagnosis cannot be considered in isolation from treatment and support access and opportunities. UNDERTAKING A SITUATIONAL ASSESSMENT A situational assessment consolidates information for fundraising, planning, identifying knowledge gaps and helping with decision-making by determining current national capacity to support an mDementia programme’s implementation and scale up. It will highlight where it is necessary to invest in care and mHealth infrastructure for the programme to be a success, alongside opportunities to integrate mDementia into existing programming (dementia, mHealth or both). It is important that the mDementia programme runs in parallel with other activities (e.g., health promotion, prevention services, and awareness raising campaigns), as it is less likely to have impact if delivered as a standalone activity. The situational assessment will provide an understanding of the programme setting through visiting, observing and interviewing key informants and stakeholders (e.g. telecommunication companies, health care workers, the general public, people living with dementia and their carers), and documenting existing resources (e.g. existing dementia policies, other ongoing mHealth programmes etc.). Data gathered will inform the community- and country-specific development and implementation of the mDementia programme. This situational assessment can include indicators to act as a baseline measure for monitoring and evaluation (M&E), so it is a good idea to read through the M&E section of this handbook before the situational assessment is designed. The time needed to conduct the situational assessment, and the assessment priorities identified in the situational assessment in terms of desired outcomes, will vary from country to country depending on which data already exist and the stage © Alzheimer’s Disease International and Leah Beach 13 BE HE@LTHY, BE MOBILE: A handbook on how to implement mDementia of development of any national dementia prevention or management programme. A situational assessment template that covers the following topics is available on request from BHBM: • Assessment of any current national dementia strategy, plan or policies • Digital health policy and legislation (including privacy and security) • Assessment of current dementia services situation and capacity (see WHO’s Global Dementia Observatory Reference Guide for more on this) • Technology assessment: current state of mobile technologies and communications • Assessment of target users (demographics, values, motivations, health and digital literacy etc.) • Assessment of contextual influences • Assessment of health promotion activities and target user preferences in terms of materials • Assessment of available content (see Section 3 on page 37 on content development and adaptation ) Please contact the BHBM team at bhbm@who.int if you need help with planning or undertaking a situational assessment. ESTABLISHING PROGRAMME LEADERSHIP To facilitate planning, implementation and monitoring of the mDementia programme, a leadership team should be established with clear responsibilities and accountability for the programme (see Figure 2 on page 15). This should consist of enthusiastic and motivated people who are committed to the goals of the programme and who can work collaboratively. The leadership team should be intersectoral (e.g. include government, academia, civil society) and include different types of health and social worker disciplines. People with dementia and their carers should also be meaningfully involved. Successful programmes have had inclusive steering committees, technical advisory groups (TAGs) and in- country operations teams that meet regularly through a transparent process of input, welcoming contributions from all. 14 FIGURE 2: PROPOSED STRUCTURE OF A mDEMENTIA IMPLEMENTATION TEAM National Programme Steering Committee This committee ensures agreement on the general direction of the programme and assist in decision-making. • Includes representatives from the Telecommunications Ministry or equivalent, Ministry of Health (MoH) and/or other ministries that provide dementia care, support or activities (e.g. Ministry of Social Affairs, Ministry of Family Affairs). • Puts mechanisms in place to formalize a clear governance structure and functions (terms of reference, meeting frequency, roles and accountability) and to set out programme responsibilities. Operations Team Runs the programme to budget and to deadlines, reports to WHO and donors. • Includes a project manager plus 2–3 people to help with day-to-day programme operations. • The team engages with and maintains strong programme partnerships. • Additional technical advisors (permanent or ad hoc.) can include IT specialists, public health and disease topic specialists as well as communications specialists (for content adaptation and marketing and recruitment), and people with dementia and their carers. Monitoring and evaluation officers can advise the team from the outset. National Technical Advisory Group (TAG) Comprises high-level experts to support in- country project teams. • Includes expert advisors from government sectors (including health, business, social security, treasury and planning), telecommunications, the technology and software industry, local telecommunications/ mobile network providers, regulatory and privacy experts, NGOs, health professionals, health and social care professionals, health economists, academic and research organizations, health insurance groups, health service providers, civil society groups, opinion leaders, people with dementia and their carers, and the media. • Supports programme implementation and promotion by assisting and advising the operational team (see next group). International Advisory Group Advises on technical and legal issues, the choice of platforms for scaling up, and sustainability and feasibility issues. • Includes WHO (country, regional and headquarters offices), ITU, and the Informal Expert Group who produced the BHBM handbook. (Note: even if a country does not opt to receive BHBM technical assistance, their status as a stakeholder should remain to ensure coherence to BHBM and for collecting and disseminating lessons learned). 15 PERFORMING A STAKEHOLDER ANALYSIS As with any implementation project, there will be a range of stakeholders to consider. Table 3 provides an overview of the role of different stakeholders when planning the technological aspects of mDementia. TABLE 3: mDEMENTIA PROGRAMME STAKEHOLDERS AND ROLES STAKEHOLDER ROLE Ministry of Health (MoH) and/or other ministries or departments involved in dementia care, support, or activities (e.g. Finance, ICT, Telecomms, Data protection etc.) • Own and act as the custodian of the programme; be part of the national steering committee and a key part of the governance function • Assess and identify needs; develop and validate content • Contract service providers or build in-house infrastructure/platform • Sign cooperation agreements with all operators and/or service provider • Fund or partially fund the programme • May host the mHealth platform/database and own the shortcode • Manage the promotion and marketing campaigns • Work with other ministries and departments involved in dementia policy and campaigns Telecommunications Ministry or equivalent eGovernment entity (if applicable) • Generate framework to enable mServices (i.e. regulations and policies) • Fund (or partially fund) the programme, and act as part of the governing body • Provide technical expertise to the Ministry of Health • May host the platform • Facilitate dialogue between Ministry of Health and ICT stakeholders • Support the negotiation of preferential prices for mServices Telecommunication Regulatory Authority • Verify eligibility for shortcode acquisition • Allocate shortcode • Facilitate dialogue between Ministry of Health and ICT stakeholders • Fund or partially fund the programme Digital health/ mHealth service providers (if Ministry of Health or eGov does not have a platform) • Provide management of the application/platform • Manage the platform and run the programme • Provide 24/7 technical support • Deal with telecom operators; manage the shortcode where necessary 16 STAKEHOLDER ROLE Telecom operators • Deliver messages to end-users • Set the cost of messages, using 3G or 4G data, and agree special tariffs with Ministry of Health if possible • Facilitate interface with service providers and/or local aggregators • Support promotion of the mDementia service Local aggregator • Provide interface with all operators and manage relationship and invoicing process • Provide reporting on services delivered/failed • Possibly own and manage the shortcode in the case of SMS Data Privacy Commission • Set the rules for data protection • Enforce the application of data protection regulations • Authorize mHealth services providing they respect data privacy • Authorize data storage outside the country, if necessary WHO and ITU • Provide technical expertise and share knowledge from Member Countries • Help convene all stakeholders Technical experts from various academic Institutions • Design the algorithm • Prepare contents for interventions • Monitor the programme • Evaluate the programme Other relevant private sector parties • Provide technical expertise and resources with careful consideration of conflicts of interest Communications and promotions actors • Design and facilitate the promotion and recruitment strategy • Liaise with media outlets and service providers • Run campaigns on their platforms (e.g., social media, TV, radio) End users and their representatives (health care professionals, dementia associations, carers and people with dementia) • Give feedback and advice on programme design and content, and usability and adaptation of content, ensuring human-centred design • Engage with policy-makers to strengthen policy Health care professionals • Public and private health care provision, including pharmaceutical companies and insurance companies • Be involved in design and development, including adapting content, providing human-centred design • Promote and recruit for the programme 17 STAKEHOLDER ROLE Academic institutions • Carry out reviews of efficacy of regional or other relevant mHealth programming ahead of mDementia programme design • Advise on M&E methods or carry out M&E Technology providers (private software and application developers or experts; hardware companies; global goods/open software communities) • Assist with technical specifications or software development National Informatics Center/ Center for Health Informatics • May be able to assist with hosting and maintaining a database of participants from the national portal Ministry of Industry/Technology Development Agency • Can help to implement the promotion strategy • Can help to build or maintain infrastructure nationwide Local nongovernmental organizations (NGOs), international NGOs, UN (WHO, ITU, UNOICT, UNICC, UNICEF etc.) • Encourage roll out and uptake of mDementia programmes • Provide feedback and advice on programme design and content Nongovernmental funders (start ups, donors, partners, insurers, investors) • Help fund the mDementia programme in the short and long term Each of these stakeholders should be assessed according to their level of support for the programme and their power to influence its success (are they supportive, resistant, apathetic, interested?). An onboarding and engagement strategy may needed, especially for stakeholders who may inhibit the programme’s success, or who are instrumental to its success. We suggest expanding Table 3 (on page 16) with a further two columns, one for engagement and support level and one for the communications plan for that stakeholder group. For further guidance on stakeholder analysis in digital health, see the WHO and ITU Digital Health Platform Handbook12, p.28. 18 ENGAGING WITH CIVIL SOCIETY Civil society comprises interested parties who are not commercial entities or governmental actors. This group will likely include people with (or at risk of developing) dementia, and their carers or family members. It can also include faith-based groups, community groups or identity-based groups (e.g. based on ethnicity or sexuality etc.). Engaging with civil society is a strategic option for obtaining funding and in-kind support, as those groups that comprise it will have established credibility and access to patient groups and wider networks that may be beneficial to understanding patient needs. Another potential benefit of partnering with civil society is sharing ownership and tapping into their network for translation or promotional activities to widen the reach of the project (see Case study on page 20). FORMING STRATEGIC PARTNERSHIPS WITH THE PRIVATE SECTOR A number of driving forces – including demographics, health care spending and technological advances – make digital health increasingly relevant in the private health care sector. Strategic partnerships can be mutually beneficial if they are a good fit with one or more organizations’ long-term corporate strategies. Identifying private sector organizations interested in funding elements or whole projects is the first step in the process. Through consultation with private companies, BHBM has identified a number of factors that motivate the private sector to collaborate with country programmes, including shared missions, and the opportunity to share knowledge and extend programme reach, among others. In addition, the private sector provides BHBM programmes with several opportunities to improve service delivery. For example, private companies may benefit from exposure or association through a direct partnership and as a result may provide reduced fee, free, or in-kind services. Different types of sectors may have an interest in different aspects of the programmes: • Telecommunications companies: Partnering with telecommunications companies strengthens the capacity of countries to implement BHBM programmes by leveraging their technological, outreach and promotional expertise. Senegal’s Ministry of Health was one of the earliest partners implementing the mDiabetes SMS programme and, alongside ITU, has negotiated a business model involving all three national telecom operators. Telecoms may be interested in providing a special product offer to clients, and these programmes may help provide a competitive angle and help project a caring image. • Health insurance companies: Health insurance companies may be interested in funding mDementia because it is a prevention/disease management programme that helps keep their client base healthier. There may also be interest in building common data infrastructures as many companies may already have companion apps or solutions for their clients that may incentivize participation in these programmes. • Technology companies: Technology companies may also want to show that they can operate across a range of sectors, and national scale projects would further demonstrate product applicability. Additional benefits to companies marketing health products include obtaining insights into target users’ preferred digital tool of engagement. In addition, if the project includes technology with artificial intelligence elements (e.g. conversational chatbots) or has a strong social media presence, technology companies could analyse trends and conversations about the programme to strengthen programme promotion and retention. Research and academic institutions may also want to use the data to gain a deeper understanding of dementia management and implementation science and in turn, may be able to help with data collection and M&E. It is necessary to ensure that the partnership remains impartial and that the organization does not pose a conflict of interest. Any contracts or written agreements should be accompanied by a due diligence process with clear clauses regarding data ownership and intellectual property. In order to maximize chances of success, attempts should be made at selecting companies whose longer-term engagement is consistent with their corporate strategy and core business. This will ensure a sustainable and long-term partnership. BHBM has some experience in working with private partners and can provide advice and support with managing these potential partners. 19 CASE STUDY: THE BENEFITS OF BHBM PARTNERSHIPS A detailed study of BHBM programmes in India, Senegal and Sudan shows that while a wide variety of approaches can be taken to implementing mHealth programmes, they often result in the same outcomes: advanced technical capabilities, improved local infrastructure, and dialogue between the private- public and third sector to build a stronger partnership to tackle NCDs. BHBM was introduced to each country as a means to tackle NCDs and to improve the overall health care system, with a specific focus on NCD prevention. Enabled by digital technology, BHBM has brought unexpected benefits to each country’s health care system, which in turn has contributed to the long- term sustainability of the programme. Seeing the value created by the exchange of knowledge and expertise between all sectors involved in BHBM has strengthened partners’ views of BHBM mHealth initiatives, and encouraged them to adopt long-term, strategic views of the partnership, which could lead to financial sustainability. Different approaches, shared goals While India adopted a traditional, government-led approach to mHealth implementation, Senegal and Sudan took a bottom-up approach, encouraging patient groups to take an active role in BHBM mHealth initiatives. Implementation of the BHBM initiative in Senegal was kickstarted by an established, local civil society organization that represented diabetes patient groups. The organization was looking for a way to help their members manage diabetes and the MoH considered the BHBM programme an appropriate solution. Once WHO, the MoH and civil society organizations were united in their common vision, a large pharmaceutical company became interested in funding the programme, which has since reached over 100,000 people. A number of best practices and lessons learned have emerged from BHBM’s multisectoral partnerships, and engagement with the private sector: Multisectoral dialogue: In Senegal and Sudan, BHBM became the first health care initiative that, through an ITU-supported digital platform, increased the country’s technical capacity and brought the public and private sector together by inviting all telecommunication companies to join forces from the start. In addition, direct benefits from BHBM have resulted in a population-wide outreach on NCDs, especially on diabetes, and enabled benefits beyond cost-benefits, by forming an innovative platform and environment to improve overall access to health care. Alignment with public value: Public value refers to how wealth is created through a partnership between private, public and third sector to meet everyone’s basic needs. To design future strategic partnerships, members of BHBM need to align themselves with a larger societal purpose such as the improved well- being of society. In Senegal, alignment with a social purpose resulted in a flat partnership structure where all mobile providers joined forces – regardless of market competition, power position or previous market share – resulting in an increased level of trust between all sectors. Empower civil society: In Senegal and Sudan, civil society, especially patients’ groups and diabetes associations, demonstrated a creative approach to solve a social problem by actively becoming involved in innovation and bringing partners together through an open dialogue. Patients’ groups now have more power and could be key to establishing future strategic partnerships. Service-based innovation: Developing a long-term view on partnership and placing a digital platform at the heart of BHBM has led to some spill-over benefits for the wider health service. Best practices from India, Senegal and Sudan highlight how a digital platform has the potential to enable an array of new services to emerge and influence existing health services, which could help to sustain BHBM on a country level. Communication and planning: BHBM has learned that communication and careful and inclusive planning of any joint activities is essential for success. By aligning intentions and values at the beginning of partnerships and developing an agreed-upon workplan, all parties feel comfortable in knowing what to expect and when. All parties should also remain transparent about the resources and processes that each has at its disposal. In this way, any differing approaches or procedural delays can be explained and accounted for without harming the project or the partnership. 20 DEVELOPING AND MANAGING YOUR WORKPLAN This section provides a template for developing a project workplan that can be used and adapted by any country intending to implement an mHealth programme. People with dementia and their carers should be included in developing and carrying out this workplan. The template includes a checklist (see Figure 3) covering all five sections outlined in this handbook (operations management, technology specifications, content development and adaptation, promotion, participation and retention, monitoring and evaluation). It also sets out some key initial steps to creating an enabling environment for carrying out your workplan. FIGURE 3: WORKPLAN TEMPLATE AND CHECKLIST FOR DEVELOPING AN mDEMENTIA PROGRAMME Create an enabling environment: The activities in this section use information gleaned from the situational assessment and are designed to run alongside operations management activities. p Establish a “problem statement” – a description of the problems related to dementia that the programme is intended to address p Develop a snapshot of the dementia situation and context in your country, including dementia prevention and care p Establish national or government commitment to help address this situation through establishing an mDementia programme p Present the overall project objective: for example: “to create a dementia prevention mHealth programme for the population of [country], particularly targeting persons at risk of developing dementia, and to implement it as a free national service” and/or “to create a mHealth programme for carers of people with dementia to improve their well-being and to implement it as a free national service” p Clearly state that you will engage individuals in this programme. Clearly state that your programme will engage all concerned with dementia – from grassroots to government p Document the process of programme identification or formulation (including information used and stakeholders involved) p Explore potential relationships to previous and current programmes or activities related to dementia both locally and internationally (e.g. mAgeing) Operations management: Based on clear planning decisions, develop an operations management plan, including identification of those responsible for implementing the project and for service provision. p Set up the national steering committee, TAG and international advisory group, and their terms of reference p Establish roles and responsibilities of the operations team: team leader assigned, implementation team members identified, responsibilities and accountability for main activities clearly communicated p Complete situational assessment p Complete stakeholder assessment and engagement plan 21 p Specify the desired outcomes of the programme e.g., dementia risk reduction, health behaviour change/improve the well-being of carers of persons with dementia p Engage with civil society p Set the strategy for operationalizing the programme p Set the sustainability plan: how will the initiative evolve with progress in technology and society; be extended to cover other NCDs; and become integrated into a comprehensive mNCD model? Technology: Identify programme infrastructure and regulation considerations. p Select technology p Identify software needs p Procure technology p Develop dashboard and enable access for monitoring and reporting p Procure a shortcode if necessary p Negotiate with telecommunications regulators, aggregators and operators for pricing to ensure service can be used free of charge by users p User test the technology and registration process p Develop data security and interoperability standards p Technology pre-testing Content development and adaptation: This should be based on a user research-driven content refinement process. Adaptation: p Translation p Local expert review p Focus group review p Ecological testing p Content verification with WHO expert group (optional) Content development (if necessary) p Special content identification p Check existing resources p Write new key messages Adapt content to other platforms if necessary p Adapt content e.g., write voice scripts or chatbot scripts or revise content library p Identify/produce multimedia content if necessary 22 Promotion and recruitment: Decide how you will market and generate enrolment in the programme. p Get to know your target audience p Set up the programme enrolment procedure and service, and ensure it has been adequately tested before user recruitment Promotion plan: p Who will be targeted? Segment the target user group p Who will promote? E.g., will there be third party marketing specialist involvement? Identify and involve promoters such as health workers, civil society and community leaders, media personalities p How will they promote? Recruitment methods e.g., select promotion channels according to user preferences (phone messages, social media, health centres etc.). Incentivization? Health workforce training/ awareness sessions p What will be used to promote? E.g., what are the key messages for different target user segments, what other promotional materials do you need (posters, leaflets?) p When will promotion occur? E.g., launch date, important dates for the calendar (national dementia day, international awareness days etc.) short-term, mid-term and long-term promotion strategies Monitoring and evaluation: Decide what the programme will measure, and with what frequency. p Define M&E outputs p Adapt an M&E framework p Plan M&E human resources p Select M&E indicators p Design data collection and select tools p Prepare budget p Monitoring data collected p Evaluation data collected p Reports and dissemination plans for evaluation, refinement, and improvement of service provision Estimated time frames: Planning: 3–4 months Content adaptation: 4–6 weeks Technology: 4–6 weeks Development of promotion and recruitment strategy and material: 4–6 weeks Programme implementation: should start after 6 months Monitoring and evaluation: throughout 23 FUNDING YOUR mDEMENTIA PROGRAMME Ensuring a robust funding model for mDementia programming is essential. And, while initial costs of such a programme can be high, careful implementation that maximizes input through integration with other digital health programmes can make that investment more than worthwhile. When implementing a BHBM mHealth programme such as mDementia, each country should commit financial and human resources, as well as political will, to ensure the programme’s success through ensuring a supportive policy environment for implementation. BHBM has several funding specifications that partner countries should meet prior to a technical support agreement: • It is recommended that the country covers the initial investment in the platform and annual operational expenditure of running the service (including staff time). This will mean establishing a fixed budget line for basic costs of the service. • Countries can (and are encouraged to) use existing infrastructure, staff support etc. to reduce this cost. However, they must show a clear budget breakdown of the areas they are covering versus the gaps that will need to be covered. • It is recommended that 50% of funding comes from the government if possible. • Funding must have been obtained or confirmed in some way. Identifying potential donors is not sufficient commitment. As well as funding specifications, the BHBM country support team will run interested potential implementers through a country readiness questionnaire. If the results show that a country is not ready in all programme- relevant areas, the team can support the country with preparatory activities. For more information, contact bhbm@who.int. BUDGETING A SCALABLE mHEALTH PROGRAMME The budget required for mHealth programming varies between countries. Year one costs will be typically higher (c. US$ 90 000 – US$ 200 000) because of the capital costs of content and software development, and higher engagement and support needs. We tend to suggest to countries the following very rough estimates (in US dollars): • Programme coordination: US$ 30 000 • Content adaptation: US$ 15 000 • Technology platforms and procurement: US$ 30 000 – 100 000 • Promotion: US$ 30 000 • Monitoring and evaluation (across the span of the programme): US$ 40 000 These costs will vary depending on a number of factors: • The chosen delivery platform and costs of software development: These could include SMS, an existing messenger app, in-app messaging, purpose-built app, website, diagnostic tool etc. A technology assessment and understanding the needs and preferences of your end-users will help you choose your delivery platform. See Situational assessment (on page 13) for information about carrying out a technology assessment. • Current and needed resources: These include human resources, content and technology/software requirements. • Need for new content: This handbook contains some messaging content, but if your programme requires translation and cultural and contextual adaptation, or additional content such as e-learning content, diagnostic guidance etc., this could require development. Again, an assessment of current relevant resources is necessary to understand the necessity and cost of new materials. • Promotion and recruitment methods: Marketing your product or programme can be costly, for example, social media advertising. Involve end-users in designing your marketing strategy to increase relevance and chances of success. • Monitoring and evaluation capacity: This can be costly, but it is worth investing in because reliable data and dissemination of that data can help make the case for programme expansion and further funding. The goal is for the mDementia programme to be free of charge for end-users, as costs can be a major barrier to uptake. Creative or collaborative solutions may be necessary to ensure that there are no data-charges for the download of an app for example, or no cost incurred for signing up or sending an SMS as part of a two-way 24 messaging programme. For advice on negotiating with telecommunications companies, see on page 33 in Section 2. We suggest including a contingency fund of around 10%, as it is often the case with technological programming that unexpected costs can arise, for example, software bug fixing or last-minute necessary changes to specifications. This buffer can then either be repaid to the donor or absorbed into financing subsequent project phases. SOURCES OF SUSTAINABLE FUNDING BHBM programmes have found several sustainable and successful business models for scale digital health programming that broadly fall into three categories: government funding, bilateral or multi-lateral support, or third-party grants. These options are not mutually exclusive, and they should be explored as early on in the programme’s development as possible. Any opportunities to streamline the approach to these funding sources should be found where possible, as they can be time-consuming. National government funding mHealth programmes are most sustainable when owned and operated by countries, as political will and financial commitment from governments maximize the programme’s chances of success (see Box 3 for BHBM examples). This requires the mDementia programme to be fully covered by national budgets. mHealth grants have been found to successfully fit within several national strategies’ priorities, so a good starting point is to explore the possibility of integrating it within existing funding mechanisms. Example: Government funding can come from existing budget lines, for example, securing funding for mDementia could come from a national health strategy if a pillar is dedicated to dementia or NCDs. Alternatively, because part of mDementiaPrevention focuses on tobacco cessation, it may be interesting to explore funding within dedicated Tobacco Control units (perhaps with those more concerned with cardiovascular disease or cancer or the national initiative for the prevention of NCDs). Tax levied or mandatory contribution funds are another way of financing mHealth. Example: Universal service funds (also known as universal access or obligation funds) are collected from telecoms companies in some countries and though these types of funds do not commonly fund mHealth, this option could be explored. Another example are funds raised from tobacco excise or sugar taxes (in the case of smoking cessation or diabetes mHealth programmes). It is worth finding out if such budget lines exist and how to approach them for funding. BOX 3: GOVERNMENT FUNDING OF mHEALTH, INDIA AND EGYPT The government of India has demonstrated important political commitment towards the scaling of the mTobaccoCessation and mDiabetes programmes, which gained traction in part due to the Prime Minister’s digital health initiative. Other government bodies were engaged in the programme to provide technical support, including the Ministry of Health and Family Welfare, the Ministry of Communication and Information Technology, the Prime Minister’s office’s MyGov platform and the National Informatics Centre. In Egypt, in 2014, the Ministry of Health and Population established a central NCD unit to accelerate the implementation of the mHealth programme. The following year, this Ministry and two others – the Ministry of Communication and Information Technology and the Ministry of Scientific Research – collaborated with BHBM and three local mobile network operators to support the implementation of mDiabetes in Egypt. As BHBM is a partnership between WHO and the ITU, the technology angle is another avenue to secure funding (it is common that Ministries of Telecommunications have larger budgets than Ministries of Health and a collaboration between the two is the best way to ensure long term sustainability of mHealth projects). Example: As BHBM mHealth programmes rely on technological infrastructure that can contribute toward the digitalization of a health system, they can be nested under digital transformation budgets which are often larger in size and broader in scope. mHealth programmes should be considered part of a larger national digital health platform which may, in turn, be part of a broader national digital ecosystem. By integrating the programme within the larger pieces, it also ensures it is sustainable and scalable to expand in other areas. 25 Bilateral and multilateral support Bilateral support is the investment in one Member State by another Member State. Multilateral support typically comes from a multilateral development bank, chartered by two or more countries, for example, the African Development Bank. Example: An instance of bilateral and multilateral financing is in Sudan, where the African Development Bank, the Italian Agency for Development Cooperation and the Federal Ministry of Health are investing over US$ 1 million in BHBM programmes. Third-party grants Grants typically come from international health donors, from national nongovernmental organizations (NGOs) or health donors, philanthropists or from the private sector (see Box 4 for tips on applying to funders). Normally, funds are secured from these organizations through careful outreach and relationship cultivation, sending unsolicited proposals, or responding to short or topic- specific calls for proposals. These may be focused on specific disease areas (e.g. dementia, mental health) or on processes and systems (e.g. eHealth, mobile, technology specific). The search identification strategy should expand to other multilateral funding institutions but also to specific charitable organizations that may be interested in funding parts of the research or monitoring and evaluation components within each programme (e.g. Wellcome Trust). Ideally, secured funding should be obtained for the longer term (4+ years) as relying on donor-based funding may impact sustainability. Nevertheless, this may be a powerful tool to demonstrate impact, results, and build a strong case for investment from national funds. BOX 4: TIPS ON APPLYING FOR GRANTS FROM FUNDERS Fundraising requires time and effort and well-written proposals that lay out the need for the programme, the goals and objectives and how they will be achieved, and a clear statement of the added value for the various stakeholders involved. Often donors will have their own structured grant proposals form, but if they do not, a grant proposal should take the following broad structure: 1. History of your organization or ministry department, including mission statement/vision 2. Project summary 3. Background, context and beneficiaries 4. Statement of need 5. About the programme (including goals and objectives, strategy, scope, expected outputs and anticipated impact. This section can include a business model) 6. Project timeline 7. Project budget (including any other funds or statements of in-kind support from partners) 8. Monitoring and evaluation and donor reporting 9. Project risk identification and management (only include if it is a requirement from the donor) 10. Future funding, scalability and sustainability Other fundraising tips: • Appearance is important, so try to make documents look professional and remember to copy-edit them. Send PDFs (not Word documents) with your organization’s letterhead on the cover letter. • Try to get to know your donor before you apply, to understand what is important to them in building the case for your programme: always think about the donor mission and agenda and how the proposal aligns with and will advance their agenda. • Include concepts like ensuring equity to access to the programme, gender, capacity building, monitoring and evaluation and sustainability, as these are important areas that are sometimes overlooked in proposals. • Be sure to advertise what is unique about your ability to carry out the programme successfully. 26 2 Technology specifications Selecting the appropriate technology for your context 29 Technology implementation needs 32 Software needs 32 Selecting a software provider 33 The role of telecoms operators 33 27 If you wish to embed mDementia within a wider, existing digital health platform it is important to understand that platform, and how your programme will fit within it. If your country does not have a digital health platform, it may be worth learning about and advocating to set one up. For more on this, see the Digital Health Platform Handbook for Health: Building a Digital Information Infrastructure (Infostructure) for Health. © WHO / NOOR / Sebastian Liste 2 Technology specifications This section looks at how to select and implement the best technology to deliver an effective mDementia programme. When developing your technology plan for mDementia, try to onboard all of the ICT and software expertise you have from the Ministry of ICT partner and any digital health and IT specialists in the MoH and other relevant ministries. ISBN: 978-92-61-26091-0 9 789261 260910 Digital Health Platform: Building a Digital Information Infrastructure (Infostructure) for Health Published in Switzerland Geneva, 2020 International Telecommunication Union Telecommunication Development Bureau Place des Nations CH-1211 Geneva 20 Switzerland www.itu.int Handbook D ig ita l H ea lth P la tf o rm : B ui ld in g a D ig ita l I nf o rm at io n In fr as tr uc tu re (I nf o st ru ct ur e) fo r H ea lth 2 02 0 28 BE HE@LTHY, BE MOBILE: A handbook on how to implement mDementia SELECTING THE APPROPRIATE TECHNOLOGY FOR YOUR CONTEXT To define and design the technological aspects of your programme appropriate to your context and your target users, the following aspects of the mDementia programme must be considered from the start by the national TAG and any other advisors involved, in collaboration with local partners (aspects that feature in the considerations for a situational assessment are marked with an asterisk*). 1. Key functions that the technology needs to perform, e.g. using a health (text) message delivery system to send automated messages according to an algorithm with a pre-defined frequency. Consider how users access the programme and how often they will interact with it. 2. Current and predicted use and uptake of mobile technologies and communications,* e.g. consider the preferences and user ratings of the target population and the availability and sustainability of these technology options within the public sector. (You may have gathered this information in your situational assessment.) 3. Outcomes from market research, e.g. which telecom system is most appropriate in the country, based on their reach (subscribers), coverage, costs, data security, and sustainability? 4. Equity of access to different technologies,* e.g. for technologically disadvantaged groups (SMS or Interactive Voice Response Services – IVRS – are likely to be most equitable), reach and access to alternative language versions. 5. Messaging formats within the parameters of your chosen technology, e.g. should voice messages, video messages, images, GIFs, interactive messaging etc. be used? What is the capacity for reach of these features and content, and their cost in the country and on different platforms? 6. Ensuring that the programme is free and available to all consumers, e.g. regardless of their device, carrier, network or location. Could the data cost of the initial download of an app or the receipt of WhatsApp messages be waived by the telecoms provider, for example? 7. Ethical and regulatory data considerations and specifications, e.g. to ensure that data are handled sensitively, to protect human rights and personal safety. Using a need-to-know principle, what data collection is necessary? Who owns the data? Where is it hosted and how robust is the security of the host? What are the privacy regulations and how will data be protected, kept secure? What are the considerations for data protection and how should a central database best be maintained? 8. Conducting monitoring and evaluation, e.g. how can user data be used to report on the key performance indicators? (see blue indicators in Annex 2 Monitoring and evaluation indicators on page 77) What frequency and which indicators should be used? How will the system produce reports and present data from users? How can the design of a dashboard enable data presentation? Can this data be made interoperable with existing health information systems? Is periodic reporting of aggregated analytics data possible? 9. Interoperability considerations, e.g. are there existing health system technologies with which the programme must communicate? For example, if there is another health messaging programme running, can some or all of the same infrastructure be used? If health workers are “prescribing” the programme, can this be recorded in the health record system, patient data system or health insurance billing mechanism? Can the health record system receive data such as the behaviour change that the user is reporting to the messaging platform? 10. Sustainability, e.g. ongoing operating costs of the programme maintenance, costs for users such as per-message or unit of data, and how these affect the scale of the programme? 11. Contractual arrangements with partners, e.g. considerations regarding intellectual property, security and privacy of mobile phone numbers, testing, expectations of involvement in monitoring and evaluation and new and outstanding service agreements. Who will hold the contractual arrangements, and what support will be given for maintenance and any other problems? These considerations and the list of simplified strengths and weakness for different technology channels in Table 4 (on page 30) are designed to help you to select the ideal technology or channel for your mDementia programme. It is essential to consider the technology literacy and accessibility levels of your target population when selecting your technology channel. For example, older users may be less familiar with, or not have as much access to, newer technologies. 29 TABLE 4: TECHNOLOGY OPTIONS AND APPLICATIONS: STRENGTHS AND WEAKNESSES CHANNEL DESCRIPTION STRENGTHS WEAKNESSES IVR (Interactive Voice Response) • IVR is an automated phone system that interacts with users using pre-recorded voice responses. Users can respond using touch tone keypad selection or via pre-set/valid vocal responses (e.g. “yes”, “no”) • Voice- and phone- enabled access • Fast time-to-market • Supports natural language • Ease of integration • Accessible to those with basic phones and low literacy • Limited capability and development tools • Inability to pause, resume, forward and rewind • Two-way communication can be prone to malfunction due to misinterpretation of voice inputs SMS (Short Message Service) • SMS allows short text messages to be exchanged between mobile phones • Simple, easy and convenient • Can negotiate cost- effective delivery • Enables private communication • Enables fast communication • Accessible to those with basic phones • Some security vulnerabilities • Fake SMS (spoofing) can result in trust issues • Two-way messaging is limited to simple interactions • May be costly if cost- effective delivery cannot be negotiated USSD (Unstructured Supplementary Service Data) • USSD uses alphanumeric codes to exchange information with a server in real- time (e.g. user can use a code to check account balance and add money to account) without need of an internet connection • Simple and logical • Real-time, fast and responsive • Inexpensive • Interactive navigation • Session-based timeouts • Codes and instructions to users more difficult to remember than Common Short Codes MMS (Multimedia Messaging Service) • MMS extends SMS technology by allowing the exchange of a variety of media (e.g. images, audio etc.) • Direct and personal • Messages can be stored and forwarded • Interactivity through multimedia • Not compatible with basic phones • More expensive than SMS • Content adaptation limited by screen size and resolution variations • Read and response rates lower than SMS 30 CHANNEL DESCRIPTION STRENGTHS WEAKNESSES Existing Messenger Services • These include apps and platforms that enable instant messaging via an Internet connection, e.g., WhatsApp, Facebook, Messenger • Low cost • High usage • Increased interactivity and engagement • Maintained by the app provider • Allows graphics and videos to be sent • Can deploy conversational agents or chatbots (where responses are tailored to users’ inputs) • Can be simple or elaborate (natural language processing and artificial intelligence) • Can carry an avatar or visual identity • Third-party private sector involvement or deployment software may be necessary (e.g., to set up and manage back-end functions) • Potential data costs for end-user to receive content • In the case of conversational agents, if using Artificial Intelligence capabilities, can require data and training before launch. Also, bugs or bot miscomprehension of inputs can be dissatisfying and potentially risky Smartphone Applications • Software/programme that runs on a mobile device that commonly needs to be downloaded and installed before use • Self-contained experience • Graphics and videos easily integrated • User-generated content and data input • Automatic updates and ability to read content offline • Leverages device-native capabilities (camera, GPS, step counter) • Can deploy conversational agents (for tailored two- way messaging) • Need to build for multiple platforms, with time and high cost • Managing multiple releases/updates • Sensitive to users’ device changes or operations • Need to submit to app stores for approval • High user drop-out rates • Initial data required for download can be costly for end-user • Often requires 3G or 4G coverage • Only compatible with two platforms (iOS and Android) 31 CHANNEL DESCRIPTION STRENGTHS WEAKNESSES Mobile Website • A mobile website/ webpage designed specifically for mobile device access • Cheaper to develop and maintain • Supports mobile phones and smartphones • Mobility for content and services • Videos and graphics easily integrated • Less functionality, unable to use advanced phone features such as camera, GPS • Small display size • Low bandwidth affects functionality Technology implementation needs Having identified the technology that you will use, there may be additional technology needs for implementation, such as: • Identification of process for procurement, adaptation and maintenance of the selected technology • Dashboard design, development and access needs (consider M&E indicators for dashboard development: what monitoring and success indicators should the dashboard present? Who should have access to the dashboard?) • Procurement of a short code (if using SMS or telephone networks for IVRS) • Data security needs • Pre-testing and scale-up needs Some of these needs may be apparent from your situational assessment, but if not, further research will be necessary to define these needs. This extra research is worth investing in, because revising developed software because of unforeseen needs can be very costly. Software needs A service delivery platform is necessary for running an mHealth programme to ensure that the programme works smoothly for users and is seamlessly integrated with the mobile telecoms network. Such a platform will have different capabilities and features depending on the technology you have chosen to deliver the content (SMS, messenger services, standalone smartphone app). The choice of service delivery platform may depend on the wider digital health landscape in your country and what other platforms are already being used. Based on your requirements, you will need to identify whether an existing or off-the-shelf platform is appropriate, or whether a customized solution (designed and built for the programme) is best. It is also important to make sure the content shared does not violate any data sharing policies (e.g. some social media platforms are very strict on what data is collected or shared in the medical field on their platform). The first step is to prepare a list of things you need the system to do in order to run the programme. This is not a technical list, but is simply what you need the system to do. For example, imagine that your research with target users (and your consideration of the strengths and weaknesses listed in Table 4 on page 30) has shown that a messenger app delivery mechanism using a simple bot will be most appropriate, and that your target users mainly use Facebook. This means your service delivery platform may have the following requirements: • A simple Messenger for Facebook conversational agent • Be able to reach anyone with a Facebook account • Have the capacity for millions of users as part of this national programme • Need to run in three different languages • Have an easy sign-up procedure • Be able to deliver messages according to an algorithm • Enable the user to interact using buttons only (no artificial intelligence or language processing involved) • A system able to notify users of new content • Allow data to be stored by the programme servers • Need technical and maintenance support with the aim to transfer this to the ministry of health IT team after year two 32 Once you have outlined your requirements, start to think of software you may need for this. It may help to look to other service providers or countries running similar mHealth programmes to see what software they are using, the challenges they faced and what they have learned. See Annex 3, Benefits and risks of different software models on page 89 (taken from WHO’s toolkit for planning an information system13) to help you to choose whether to use an existing solution or to develop a customized one. Consider the pros and cons of each option, mapping your technology requirements to the capabilities of the platforms. Also, investigate the programming needs (should any adaptation or integration with other systems be needed), the level of ongoing support available for each option, and the costs. In some cases the following considerations will be necessary when making your software decision: • How will the software integrate with the mobile telecom environment. Will it work across different mobile network operators (MNOs)? • What partnerships will need to be developed to activate the service (for example, partnerships with aggregators, MNOs, mobile gateway providers)? • How will the platform be able to adapt to changes and advances in technology? • Based on the chosen platform, will the project need new or additional hardware (computers or a server to run the programme)? • Interoperability and licensing issues (e.g., if you would like to “prescribe” the programme to health service users; and how you might integrate and record the use of the programme – and patient outcomes associated with that use – in the health information system) Selecting a software provider It is likely that your organization will have existing processes and procedures for the procurement of services and will likely issue a request for proposals (RFP). You can start by inputting to this RFP the background information you compiled (see checklist in Section 1 on page 21) and make sure the goals, values, and desired outcomes for the programme are set out. Working with service providers whose values align with those of your programme will help with relationship management. Researching your service providers and having dialogue or interviews with service providers prior to selecting them can help to know if your values are aligned. Next, use the list of requirements you have created to build the RFP. If you do not have a software specialist on your team, leave it to the software providers to suggest the more technical specifications that can meet your requirements. The BHBM country support team can also help you with writing the RFP. Consider software providers that have implemented similar solutions at scale (and feel free to ask to see audit results). Ensure they can cope with the demand for the programme and maintain their service (e.g. ask what maintenance activities are included in the fee). Make sure that you understand what components of the software will be proprietary (and license them to the mDementia Programme owner if possible) but try to aim for open- source components wherever possible. When selecting the company, you may wish to design or adapt an existing scoring matrix to help standardize any contracting decisions made. See Annex 6 (on page 92) of the WHO planning an information systems project toolkit for a comprehensive scoring matrix that can be adapted for your purpose. The role of telecoms operators It is important to note that mobile communications network environments differ from country to country. The specificities of end-user access to SMS, calls or mobile data (for stand-alone or messenger apps) should be considered in the planning stage by including technical experts in the TAG (such as representatives of telecom companies; operators; telecom regulatory authorities; government departments responsible for information; communications, technology, and cellular associations) or individuals knowledgeable about the communications network in the country. Network operators, telecoms companies or industry organizations can provide help in setting up the programme and advising on its suitability and sustainability. Some providers may view supporting such a programme as good publicity or a useful addition to the services they offer. This can work in your favour when negotiating with them. Before you invite their involvement in technology specification try to consider what sort of arrangement with telecoms companies will best suit the long-term implementation of the programme. What other partnerships can be useful or necessary? What are your parameters of negotiation with telecom regulators, aggregators and operators for pricing of message dissemination? 33 Negotiating with telecoms operators Engaging in and maintaining strong partnerships with telecoms operators is critical for mDementia programme implementation, and any eventual reduction in the cost of the programme. BHBM has found that enrolment tends to be very low when consumers have to pay for it. And a major barrier to two-way messaging programmes is the reply cost for participants, with the costs of replying meaning that one programme had just a 30% reply rate. The more you can reduce the cost of engagement to users, the more successful a two-way programme will be in terms of user activity (note, mDementia includes both one-way and two-way messaging). In the case of SMS, there should be no fee to receive or reply to programme messages and in the case of messenger apps, it will be necessary to waive data costs associated with receipt and reply to messages. If you chose to build a standalone mDementia app, the initial download must be free (both price of the app and the data required for its download) and subsequent information exchange should also be free. In all cases, user data should be private and secure. To this end it may be helpful for teams to include members with experience in operator engagement and negotiation, and if necessary the BHBM country support team can help with this. The ITU (regional office) can act as a bridge to facilitate the negotiations between the Ministry of Health, Telecommunications Authority and the Telecommunication Companies. A template for a telecoms operator agreement is available on request from bhbm@ who.int. WHO has also released a very comprehensive guide to negotiating with mobile operators (this guide relates to mHealth for reproductive, maternal, newborn and child health, but its principles are relevant across health issues). The objectives of the negotiations are to reduce or cut costs associated with the programmes, especially the cost that falls on the intended user of the programme; make the programme hassle-free for the end users, and ensure data protection and privacy of the end users. Tips for negotiating • Bring an IT expert with technical knowledge of the platform and software you are using along with you to the consultation to respond to technical questions or discussions. • Share your values and vision and discuss those of the telecoms company also, highlighting where values match. • Estimate the programme’s intended user numbers before commencing negotiations with telecom operators – this will help assess the scale of contribution they need to make. • The negotiators must be aware of the current costs of services, costs of packages and sliding rates which are vital for the negotiations. • Ensure the project is jointly shared by both the Ministry of Health and the Ministry of ICT, as, in some countries, the Ministry of ICT may have an established relationship with several mobile network operators and may be in a stronger position to negotiate (especially in the case of national public network agencies). • Ahead of negotiations, hold consultations with relevant authorities (e.g. telecom authority, national ministries and market regulators) to identify and understand what benefits / privileges can be granted to telecom operators in return for their collaboration (see Box 5 (on page 35) for tips on how to do this). In the absence of telecoms provider support and in the case of an SMS or IVR programme, the programme can be delivered through a contractual arrangement with an “aggregator” or “gateway” company that has established relations with all telecommunications companies and networks. This can be a cost-effective way to deliver messages to many participants, regardless of their mobile carrier or location, without establishing these interfaces individually. Although the aggregator adds a further cost, this cost decreases as the scale of the programme increases. Using an aggregator can therefore be more cost-effective than attempting these activities “in-house” unless capacity and infrastructure already exist. See Annex 4 The role of aggregators (on page 90) for further explanation of the role of an aggregator. A checklist for considerations for technology specifications can be found in Section 1, Developing and managing your workplan, (on page 21). 34 BOX 5: IDEAS FOR INCENTIVIZING TELECOM OPERATORS’ BUY-IN • Direct benefits to the operators from the Ministry of ICT, e.g. a small tax reduction. • Offering an access point into a new market by understanding mHealth service structure and user experiences. • Within the broader programme timeline, operators can use their growing experience to develop their independent mHealth portfolios, nationally or internationally, driven by rising national demand. • Showing effectiveness of mHealth services could offer operators a new source of future revenue in value-added health services. • Given that in most countries the telecom market consists of two or three major providers who have more or less similar subscription plans, a telecom operator can really distinguish itself from the rest by demonstrating its ability to offer additional benefits to its customers. • Early-mover advantage: knowledge transfer. Operators need to learn how to run large- scale public health programmes. Participation in the programme will maximize quantity and quality of knowledge in comparison to competition. • Good public relations: Good visibility as a socially responsible company can showcase the company’s contribution to public well-being. The MoH needs to ensure that it offers telecoms companies the options for this visibility, including: promotion in mobile stores with MoH logo, mobile operator office, website and public campaigns. • Good working relationship with the Ministry of Health (and possibly with the Telecommunications Authority). • Operators may need support with their own interests in the mHealth, mobile money or mobile health insurance fields and could be working on common areas with MoICT or MoH. Telecom companies can be invited to identify priority areas where the government may consider providing them with support in the future. 35 3 Content development and adaptation Adapting mDementia programme design 38 Adapting the existing mDementia content library 38 Translation 39 Adapting messages 39 Local expert review 40 Selecting content for focus group testing 40 Ecological testing 41 Additional content development 43 Adapting content library for voice, messenger apps or chatbots 44 36 3 Content development and adaptation Experience shows that despite the best efforts of an expert content development group to appropriately craft health behaviour-change content for a global audience, programmes are most often improved through research that places target users at the heart of the process. A standard content library for mDementiaPrevention and mDementiaSupport is available on request for countries to use as a basis for their mDementia programme. This has been written in the format of SMS messages, but is adaptable to other mHealth technologies. SMS is the most equitable way to deliver health messaging in resource- restricted settings (where much of the population may not have access to a smartphone). Therefore, BHBM suggests that countries run SMS programming alongside other technology options such as messenger apps or stand-alone apps if countries wish to deliver the programme through smartphones. This section of the handbook will provide guidance on: • adapting programme design; • adapting the existing BHBM mDementia content library to your context; • creating additional content for a mDementia programme where the content we have provided is not sufficient for local needs; • considerations for adapting BHBM’s content library to other formats (e.g., voice and messenger apps and chatbots), and adding multimedia Whether you are adapting an existing BHBM library, or creating new content, the implementation team should review any existing mHealth or dementia prevention/ risk reduction and care programmes and/or guidelines that may guide the adaptation or creation of programme content. In both cases, target users (e.g. people at risk of developing dementia, carers of people with dementia) should be involved to provide essential insights and validate the finalization of programme design and content. We ask that you please share any additional messages or content with the BHBM secretariat (bhbm@who.int) to inform further iterations of the global content libraries and for sharing with other countries for their benefit. © WHO SEARO 37 BE HE@LTHY, BE MOBILE: A handbook on how to implement mDementia ADAPTING mDEMENTIA PROGRAMME DESIGN The BHBM mDementia content library includes one- and two-way messages for mDementiaPrevention and mDementiaSupport, and makes suggestions for programme length and frequency of messages (the algorithm). The suggested duration for each module is 6 months. This duration was chosen based on evidence that shows that complex health behaviour change takes 6 months to habitually incorporate into one’s lifestyle14,15. The suggested algorithm is such that messages start at a higher frequency and diminish over time16. Messages can be sent simultaneously or consecutively for different behaviours, and there is no evidence for whether one is better than the other. The mDementiaPrevention programme segments the population based on the risk factors present (gathered from replies to pre-programme screening questions, see Annex 1 Pre- and post- programme questionnaires on page 67) to tailor the content better to individuals. Tailoring can also be done for the mDementiaSupport programme if needed. There is flexibility for countries to adapt the programme to the social and cultural context; to infrastructure and technologies available and used by the population; to available funding; and to existing health and social systems. Countries can also lengthen or shorten the programme; increase or decrease the intensity of messages; adapt the system rules; and adapt registration, opt-in/out and other functions. When adapting the programme, including its rules and logistics, the following should be considered and guided by feedback from target users of the programme: • The objectives of the programme • The media and channels used • The timing, frequency and duration of the programme • The collection and storage of baseline and accumulating research data • Registration, opt-in and opt-out process and administrative communication (if any) • Whether there will be two-way interaction • Degree of choice and flexibility in the programme • Timing of messaging regarding the possibility of stopping the programme • The extent of interaction with clinicians or the health service system ADAPTING THE EXISTING mDEMENTIA CONTENT LIBRARY The mDementia content library was created by gathering world experts in dementia, mHealth and health communications. Messages were drafted based on WHO guidelines, iSupport and the latest scientific evidence. The content has been written with the understanding that the messages should be adapted for country use. Adaptation is important because it can make information clear and more relevant to the target population. Some literature suggests that adapting mental health interventions can increase their efficacy17,18,19. Adapted content will enable users to relate to and implement the behaviour change strategies, and may lead to higher user retention. The messages must be understandable, acceptable and relevant to people using them to increase their impact. See Figure 4 (on page 39) for the steps involved in mDementia message content adaptation. 38 FIGURE 4: OVERVIEW OF SUGGESTED ADAPTATION PROCESS Local expert review Focus group review Ecological testing – Small group test – Real world pilot Optional WHO content verification Scale the programme and update if necessary End user involvement Translation The first step of adaptation is translation. Sensitive translation is important, so the messages are clear and engaging. The following considerations are important when translating programme content. 1. Which languages will the programme be available in? Experience from running BHBM programmes and feedback from users shows that some users did not engage because some of the programme was not delivered in their country’s major and most accessible languages. For example, in Tunisia implementers quickly realized that the programme must be available in Arabic as well as French for it to be taken up. In India, many languages are spoken, and the team learned that having at least Hindi in addition to English was crucial to obtain more subscriptions. 2. What dialect should be used? In countries where multiple dialects are used, it is important to ask the following questions: • Which dialect will be acceptable to the most people? • Is it necessary to have the programme provided in more than one dialect? • Is it necessary to have the programme provided in voice only (through interactive voice response or voice notes, because some popular languages are only spoken, not written)? • What about spoken versus written or classical forms of the language – for example, would writing in the spoken style of Arabic be more engaging for users than classical, written Arabic? 3. Verification of translation: blind back translation Once you have translated from the content library from English into the local language, this translation should be back-translated into English again by a different translator (one who has not seen the original English content) to check its accuracy. This does not need to be a professional translator, but preferably a bilingual person. If there are discrepancies between the original and the back-translated English versions, the two translators and the implementing team can discuss the best alternatives to give a sensitive and accurate translation. Adapting messages Local experts and target users should guide the adaptation of messages (or other content such as chatbot scripts or app content) to make them easy to understand, appropriate and relevant to target users. Their inputs can be gathered through review processes and qualitative methods, including focus groups, surveys and consumer pre-testing. You could run this yourself or contract a specialist market research company to do it for you. In the provided mDementia content library, some words are written in red text. These depict where surface or superficial adaptations will likely be necessary (e.g., replacing food items with local foods or activities that are context or culture specific). Other parts of the message Translation 39 need adaptation too. The challenge is to maintain the scientific accuracy of the message while making it relevant to locals. For example, fish is suggested in some programmes as a source of protein and Omega-3. If fish is very expensive or unavailable in the local context, it will be necessary to adapt fish to another appropriate food containing Omega-3 and protein e.g., nuts and seeds. Another example, in the case that it is unsafe or too hot to walk outside, it may be appropriate to replace walking exercise with exercise in the home. Local expert review Local experts in dementia, including academics and expert and primary care health workers, and health communications specialists or behavioural scientists, can be invited to review the translated materials. By acknowledging them in the programme reporting and dissemination materials, they may perform the review for free. If you have the resources, you can host an initial adaptation workshop in person or online for experts to meet and discuss their review. It is important to ask them during this review if additional content should be developed (see Additional content development on page 43). This review, once collated and incorporated, should give you a draft local content library. The next step is to validate the expert-crafted content to the target population. Selecting content for focus group testing Using focus groups for testing is resource intensive, but many BHBM countries have valued this step tremendously, finding that it is worth the investment. This testing phase may be a small number of in-person focus groups with target users and health and social care workers and – if necessary – some focus groups could be substituted by telephone interviews. The material for testing with target users should: • be selected based on the findings of the expert review (in case there were discrepancies or content highlighted for user adaptation); • comprise a mixture of sign up, one and two way messages and evaluation messages; • reflect likely cultural preferences and adaptation needs (e.g., substituting for local foods or activities but also message framing and tone); and • include key messages most likely to lead to user desired outcomes. Please contact BHBM if you need more guidance. CASE STUDY: ADAPTATION OF THE TAKORE I TE KAI ‘AVA’AVA (“STOP SMOKING”) PROGRAMME, NEW ZEALAND A New Zealand smoking cessation text message programme (STOMP)20 was adapted for use in the Cook Islands by the Ministry of Health and the University of Auckland in New Zealand (financed by WHO). The STOMP programme, which is theory-based and shown to be effective in supporting smoking cessation, had previously been adapted for the United Kingdom21, Argentina22, and Samoa23. The adaptation process involved making the linguistic and cultural nuances necessary for the new context while maintaining the integrity of the original, evidence-based programme. The messages needed to be adapted to be more focused on the direct health effects of smoking rather than be general motivation messages to support behaviour change. This was due to the low level of previous population-based education on the harms of smoking in the Cook Islands. Another key feature of the adaptation was the need for formal language to be used in both the English and Cook Island Māori versions without abbreviations or text language. The final version of the free text message programme was named “Takore i te Kai Ava’ava”, and a shortened version of the name (“TextTakore”) appeared at the start of all text messages to distinguish it from unsolicited messages common in the Cook Islands. As the local telecommunications provider was unable to provide two-way message functionality, different language versions, as well as personalization and tailoring within messages, of the programme had to be delivered from New Zealand through a gateway company direct to users in the Cook Islands. A pilot study of Takore i te Kai Ava’ava found the programme to be highly acceptable and demonstrated potential to provide motivational support to smokers in the Cook Islands wanting to quit. 40 Ensure that focus groups are representative of the target population, with geographic and cultural diversity (e.g., rural and urban, including different districts or national regions), linguistic diversity (regional dialects), diverse socioeconomic statuses and different cadres of health and social workers (if they constitute the target population for promotion or receipt of the messages). You may also need to consider group dynamics and cultural norms when composing your focus groups. The goal of the focus groups is to verify whether the programme content is: Understandable - Are the concepts and terminology used in the programme understood by users? Acceptable – Are the messages received with approval and acceptance, meaning that they are respectful of and sensitive to the local context? Relevant – Are all messages necessary and applicable to the environment and context of the user? Will people engage with the messages? Take advantage of the fact that you have convened focus groups to ask them other questions if you have them, e.g. what time of day they would prefer to receive messages and whether the dose of messages you are planning (or expected engagement in the case of apps or chatbots) is appropriate. In one BHBM country programme, implementors had highlighted that they needed to change the time of day the messages were sent and that two messages a day (at the start of the programme) was too much according to feedback from users. Once the data and suggestions from the focus groups are gathered, you need to decide what you adapt and what you do not adapt. This can be done by convening the project team in a small adaptation workshop, where adaptations can be discussed if necessary. Document what you have changed and why. This information will be very useful to record for the BHBM secretariat in improving the mDementia content library and understanding more about its global relevance and use. Ecological testing Ecological testing means testing the programme content in a way that mimics the conditions in which the programme will ultimately be received. For example, it is important to note that receiving a message on a mobile phone during a normal day is different from sitting down to read a list of messages on paper or in a presentation in a focus group. While focus groups are therefore necessary prior to the launch, they may not be sufficient: additional small group and “real world” consumer testing – although time- and resource-consuming – is important if feasible. Focus group discussion in India for mTobaccoCessation ©WHO Be He@lthy Be Mobile 41 In one BHBM country annual report, the team deemed user testing and piloting as “essential”. Small-group testing Test the sign-up process and the programme messages (or app or bot) for 1–2 weeks with a small group of participants (approximately 15), and ask them to rate each message (or session) immediately as they receive it by providing feedback on the acceptability and helpfulness of the message (e.g., “How did you find the sign-up process on a scale of 1 (being very easy) to 5 (being very difficult)?”; “Please estimate how long in minutes it took you to sign up (approximately)”; “How much did you like the message/exercise (or exchange in the case of apps or bots)?”; “How helpful was this on a scale of 1 to 5?”; “How likely would you be to implement the suggested advice or instruction in the message on a scale of 1 to 5?”). Real world pilot testing In conducting “real world” consumer testing, the adapted programme is sent live to a group of participants as part of a soft launch or pilot study. This looks very similar to a national/regional programme, with the same dose and frequency of messages (or interactions). However, if resources are restricted, this can be conducted over a shorter period of time e.g., the first 2 months of the programme instead of the whole 6 months. The group is surveyed over the telephone periodically during the course of the pilot, mainly to determine how understandable, acceptable and relevant the messages are, but also how likely they would be to engage in the suggested behaviour change. The results are used to refine the programme. Optional fidelity-checking of adapted content At BHBM, the expert group who contributed to the original WHO content library are available for reviewing the back-translated, English language version of the content library before widespread use in your country. You can send your content library to bhbm@who.int and the team will arrange for a scientific review by the expert group. Allow a three week timeframe for this review. Phase 1 evaluation Many countries choose to carry out a small soft launch/ pilot of the programme as part of real-world consumer testing. This process can also test the appropriateness of the evaluation mechanism and indicators. This phase 1 programme evaluation will likely gather qualitative feedback from subscribers that can be used to further adapt the content library, even after Phase 1 implementation has begun. As a final stage, the national TAG must finalize and agree on a plan (including who is responsible) for maintaining the database of programme content (messages or chatbot scripts for example) for updates (e.g., to incorporate updates from BHBM or updated national clinical guidelines etc.). CASE STUDY: mTOBACCOCESSATION PHASE ONE TESTING, TUNISIA In Tunisia, ahead of scale roll-out of the mTobaccoCessation programme, implementers adapted message content with experts and then carried out a soft-launch with 1086 subscribers to test the programme ahead of national programme implementation. The team gathered user feedback through telephone surveys with 413 completers and 349 drop-outs. This very valuable information was then used to improve messages and the programme as a whole. Feedback included: • Improving the enrolment process so it was less burdensome • French and Arabic versions were necessary • Provide more clear and practical advice and avoid stereotyped advice • Provide more real-time 2-way messaging (not just scheduled messages) when users need support • Adapt the messages to the users’ social, physical and environmental situation 42 ADDITIONAL CONTENT DEVELOPMENT While the BHBM mDementia content libraries are comprehensive, you may find that additional material is required for your target population and this addition of content is an adaptation in itself. This was the case for the Sudan mDiabetes programme, where it was necessary to create particular messages on prevention and management of eye and retinal complications, a major problem in the country. To assess whether you need additional content for your programme or whether the BHBM mDementia library is sufficient, you will need to consider issues such as (some of these may have been already included in your situational assessment): • Are there any groups or sub-populations in your country that are not served by adapting existing BHBM content? • Are there any misconceptions or false beliefs around dementia that are specific to your target population and that warrant additional programme content? • Is dementia highly stigmatized in your country? • Are there special informational or management needs in your country resulting from high-prevalence of related conditions or syndromes (e.g., high rates of cardiovascular disease) that are not already covered in the BHBM library? • Is it more appropriate to run the programme through a conversational agent (in which case, conversational scripts and additional content my be necessary)? • Is there an area beyond dementia risk reduction or carer support that has been identified as a high need which requires additional content to be developed (e.g. dementia awareness raising, dementia services/care)? If you find that additional content is needed, explore if potential content exists from other health communications campaigns (e.g. dementia or cardiovascular disease campaigns). If not, try to gather as much information as you can – including the opinions of specialists in the topic at hand – in order to create the new content yourself. The BHBM team can also help you with additional content development – email bhbm@who.int. If you are creating new content it is important to write with the following in mind so that messages, app content or chatbot scripts are understandable, acceptable and relevant to the users: • Ensure that the language, tone and clarity of the health messages are appropriate • Consider the health literacy and technological literacy level of target users • Ensure the provision of information, concrete instruction for self-managed behaviour change, reminders and motivational content • Consider and potentially tailor content for specific groups (e.g. for gender, healthy populations, rural or urban populations, socioeconomic status, ethnic group, age, gender); • Avoid an alarmist tone or negative framing to behaviour change messages. If a negative frame is used, be sure to provide also an instruction, solution or hopeful statement to avoid causing anxiety and a feeling of disempowerment in the user • Include an “active” component or an “ask” (e.g., women were more likely to act on messages that gave a concisely written true statement and then asked them to act on it, such as “Get screened now”) • Consider if the information should be static or dynamic, depending on whether the programme is uni- or bi-directional or a fully interactive via a chatbot or an app (based on input from the user) • Consider the number of characters per message allowed in each country or the data implications of sending the content (especially applicable to images and videos). New content should then be user tested with target users as outlined above in the adaptation section. Any information incorporated should be evidence based and agreed upon by experts and health and social care providers, as well as end users. Annex 5 Main themes and operationalizing strategies for message development (on page 91), has more detailed information on creating new content. 43 ADAPTING CONTENT LIBRARY FOR VOICE, MESSENGER APPS OR CHATBOTS The mDementia content library is in SMS format, as this is the most equitable mode of dissemination of mobile messaging. This can easily be adapted to free- phone voice messages, which are a good way to get messages to low-literacy or visually impaired populations. However, if your situational assessment and inputs from target users of the programme suggests that it may be impactful to provide the programme through smartphone apps in addition to SMS, you may adapt the content library to messenger app (as normal instant messages or in chatbot format) or as a stand-alone app. See Annex 6 Adapting content library for voice, messenger apps or chatbots for guidelines and considerations on adapting the content library for different messaging formats on page 92 and Table 4 in Section 2 for strengths and weaknesses of each different technology (on page 30). A checklist for considerations for technology specifications can be found in Section 1, Developing and managing your workplan, (on page 21). © Alzheimer’s Disease International and Leah Beach 44 4 Promotion, participation and retention Promoting the mHealth programme 46 Specific considerations for the mDementia programme 46 Participation in the mHealth programme 52 Retention 53 45 4 Promotion, participation and retention © WHO/ Yoshi Shimizu 2019 Promoting the mHealth programme It is essential to promote the mHealth programme, so that potential users know about it and can subscribe easily and conveniently. Without users signing up, the programme will be obsolete. A nationwide or population- specific strategy to promote outreach and recruitment into the programme can be a potentially expensive component and should be considered carefully and early in the planning stages. Most BHBM programme implementers fed back that they should have done more promotion, used multiple engagement channels, and campaigned more regularly to promote the programme. They suggested to use multiple media for promotion activities prior to launch, and that social media was a useful channel, as were SMS messages and posters, though radio and TV advertising were less useful. See Table 5 on page 47 for important considerations, learning points and suggestions from other BHBM programmes. Specific considerations for the mDementia programme Dementia is highly stigmatized in some countries. It is important to consider that stigma surrounding dementia may prevent users from signing up to the programme. It may be useful to conduct an anti-stigma campaign just before and/or overlapping with this promotion campaign to prepare the population for acceptance of the programme. It may also be beneficial during promotion to detail what dementia risk factors are (e.g. cardiovascular risk factors) as well as the early signs of dementia to increase dementia literacy and raise awareness of the need for the programme among potential end-users. With regards to promotion for mDementiaSupport, it may be beneficial to explain the value of carer support (i.e. carer support can improve mental and physical health as well as helping carers to provide care for longer). In the context of great socioeconomic upheaval (e.g. COVID-19), this kind of support may be particularly relevant due to issues such as disruption of services, increased social isolation and carer burden that may occur in conjunction with work obligations or child care. 46 BE HE@LTHY, BE MOBILE: A handbook on how to implement mDementia TABLE 5: CONSIDERATIONS FOR mHEALTH PROMOTION CONSIDERATION DESCRIPTION Target audience for promotion Once defined, get to know your target audience if you have not done so already, through focus groups, interviews and surveys. There will be a number of target audiences for promotion: 1. End users who can be reached directly (e.g. middle-aged and older adults in the general public); 2. Key promoters – local and international dementia organizations and associations, health workers, social workers, pharmacists, telecoms companies and health insurance companies, NGOs, social media influencers and any other end-user facing groups with an interest in health promotion The more that promotion and recruitment strategies are tailored to the target population, the more effective they will be at encouraging people to subscribe, and the wider the reach of your programme. Segmenting them into smaller groups based on key characteristics (e.g., gender, or socioeconomic status for example) and understanding their values and motivations can help make recruitment more successful (e.g., leveraging motivation for change in the recruitment campaign and tailoring promotion materials for different segmented groups). We suggest running some focus groups with people representative of the target audience to inform the design of the programme and to gather their ideas and recommendations about recruitment methods to use as a basis for your strategy. Cost of the campaign Find out from your target users what channels they will engage with and invest in them for programme success, e.g., social media advertising, radio and TV. Initial underestimation of promotional costs is common and can be difficult to remedy later. The principles of negotiation with telecommunications companies can also be useful when negotiating with broadcasting and social media companies (see Section 2, Negotiating with telecommunication operators on page 34). Strategies and synergies and leveraging other campaigns The strategy should be based on actual research with the target audience which will identify what channel is the best. Which organizations/notable personalities are currently involved in successful mass media campaigns for dementia, and can those campaigns be linked or leveraged? Which mHealth programmes have previously been implemented in the area? Can lessons be learned about which promotional techniques are effective in the country? Leveraging existing marketing and health promotional or marketing campaigns of programme stakeholders or partner agencies, such as telecommunications companies, can cut costs. Telecoms companies or vendors could advertise the programme on SIM card packaging, or run announcements when users are on hold for technical support or customer services. 47 CONSIDERATION DESCRIPTION Strategies and synergies and leveraging other campaigns Countries can also leverage from or integrate into other programmes’ campaign strategies, e.g. NCD, mental health, ageing. If the country is already running a campaign on dementia, try integrating the mDementia programme as an add-on. For example, if your country has a national dementia day or an event related to dementia, announce the mDementia programme and how to subscribe to it on the day of the event. Accessibility of promotion materials Consider the target audience and if/how they access certain media channels. What is the media channel they will most likely see and engage with? How can you make accessing recruitment materials more equitable to minority populations or people with disabilities? Some BHBM programmes found that males of higher socioeconomic status were more engaged (with some programmes having just 11% female users). Promotion should be targeted at difficult-to-access groups. Other BHBM programmes have found social media and the messaging media itself (e.g., SMS messaging) most effective for recruitment. Campaign content A valuable lesson learned by one BHBM programme was to make sure that all information on how to access the programme is concise and clear. The potential user should know who the programme is for, have all relevant instructions and know-how to sign up once they have seen the promotion materials. A BHBM programme in Tunisia found that users reported signing up because it was convenient via mobile phone and not because they thought the programme would work. So, Tunisia then included information on how effective such programmes are in the promotion materials. Though it seems obvious, campaigns should state who the intended user is. One BHBM implementer stated that they did not specify that the programme was for tobacco users, so they got many non-tobacco users signing up out of interest or to learn about addictology. Who “owns” or is the perceived messenger for the programme BHBM evaluations have shown that users trust the messages and value the messages if they come from the Ministry of Health (MoH). For example, in one BHBM programme users said that they now believed cervical cancer to be very important because the MoH would not be sending messages about it otherwise. If possible, ensure that users see that the message comes from the MoH or other trusted health authority. Consider setting up a page within their website for users to access information about the programme, which can be added to promotional materials. Consider promoting through government health services (on appointment reminder cards, in waiting rooms etc.). Identify other trusted authority e.g., local partners and stakeholders who can help with promotion and recruitment. These can include partners such as faith-based or social organizations, cultural activity groups, civil society (e.g., national dementia association, private clinic and hospitals (if applicable) etc.). 48 CONSIDERATION DESCRIPTION Using marketing specialists It may be that the MoH or other implementing agency does not have the in-house expertise to plan and deliver an effective promotion campaign. Contracting out to a marketing agency may sound expensive, but if the appropriate agency is chosen (one with a good track-record in health marketing), it may substantially boost programme numbers. Creating a call for proposals with the aim of the promotion campaign and disseminating it to companies will solicit proposals with a range of methodologies. Selecting a marketing company will depend on the suitability of their proposed methods for your target group, their success and experience with other health campaigns, and a competitive price. Testing the recruitment strategy through a soft launch Consider a test-run or a “soft launch” prior to starting the promotional campaign to ensure that all processes are working well before a large number of participants sign up. This may be running focus groups with users comparing some differently worded or presented promotion campaign materials or asking them what messages about the programme and marketing materials would make them want to sign up. Pre-intervention information session Launching information sessions about the mDementia programme at places where carers of people with dementia or people at risk of developing dementia might frequently visit can also enhance the visibility of programme and can encourage the participation in the programme. This can be part of the promotion and campaign strategy. Putting flyers in high density areas (targeting members of the general public) or where specific target audiences (e.g. carers of people with dementia) will visit frequently such as religious places, clinics, community centre, can also increase the accessibility of materials. The local mobile communications environment Check whether sending unsolicited messages is allowed (this contravenes telecoms codes of conduct in some countries). Consider whether a population that often receives unsolicited health-related messages will be likely to read and respond to messages from the programme. Also consider the issue of message receipt versus message engagement. 49 PROMOTION WITHIN EXISTING HEALTH SERVICES OR INSURANCE SCHEMES Integrating the programme into existing health and social services and structures is a cost- effective way to encourage the use of the programme. Health care and social workers are in a prime position to encourage patients who have dementia risk factors, in particular cardiovascular risk factors, to enrol in the mDementiaPrevention programme. Health care and social workers and/or dementia associations who are aware of a dementia diagnosis can also refer the mDementiaSupport programme to family members or carers of the person with dementia. In order to achieve this, a health worker-targeted promotion plan may be needed to ensure that health care and social workers are aware of the programme and are actively encouraging their patients to register. When engaging health services for promotion of your programme, consider: • What is the benefit of the programme to the health and social service and the health and social worker personally? • Whether health and social workers could be resistant to change and the use of mHealth (as was the case in one BHBM programme). If so, why? How may you manage their apprehension and get their support to promote it to their patients? • The level of community engagement and number of volunteers or health and social care workers who may assist in promoting the mHealth programme. • How users of the health and social service will register with or sign up to the programme: directly by text message, online, by telephone, in person, or through the health and worker? • Whether the use of incentives to encourage participation is possible in the health service • Current or historic use of mobile messaging for outreach and promotion within the service Information about the best ways to engage with health services as key promotors of the programme should be collected from health care workers themselves (e.g. through focus groups). Having a healthy population is also in the interest of health insurance companies. Getting them involved early in the implementation process could help, so they can disseminate promotional materials to their clients or even incentivize the use of the programme. Alternatively, they may have important knowledge to share if they have run mHealth programmes themselves. 50 CASE STUDY: mDIABETES PROMOTION IN EGYPT In February 2016, the Ministry of Health in Egypt launched a national mDiabetes programme at a media event with the aim of increasing people’s access to information on diabetes prevention and management. With support from WHO, the SMS and WhatsApp messaging programme was promoted at health care facilities in Cairo, and at public places such as train and metro stations. A Facebook page called “Your Health in a Message” was set up for targeted recruitment. Banners, posters and brochures were distributed to patients along with their prescriptions, and in outpatient waiting areas, pharmacies and various government offices. The ministry also partnered with popular radio and television channels and journalists to promote the programme, and in partnership with WHO conducted a media workshop to inform journalists about the programme, which resulted in widespread media coverage. A group of social change agents was trained to promote the mDiabetes programme during their home visits in the catchment areas of primary health care centres in Giza and Qalyobia, giving diabetes patients the option to register by giving their mobile numbers. A database of diabetes patients was also obtained from public hospital and primary health care centre records, diabetes national institutes, and from a database of government-sponsored patients enrolled in the National Programme of Treatment. These patients formed the database for the mDiabetes programme. Self-registration was encouraged through two mobile numbers listed on all promotional materials. Diabetes patients had the option to register either by sending a text or WhatsApp message, by a missed call to any of the two numbers, or by registering through the Ministry of Health website. Promotion materials from mDiabetes in Egypt sms ةردابم ةيحص ةايلح ليبولما ايجولونكت 51 Participation in the mHealth programme An effective promotion campaign should result in a number of interested users being ready to sign up interact and with the mHealth programme. This sign up and interact with process must be user-tested to ensure it is clear, easy and brief. Two major barriers to uptake of digital health solutions are costs associated with difficulties in signing up. BHBM programme evaluations have shown that long or complicated sign-up processes can lose up to 30% of interested persons. In one survey of interested non-starters, the majority who did not complete the process reported the sign-up fee as the main deterrent. Another programme reported a large proportion of unintentional users signing up, thereby wasting resources. Programmes should be free for users wherever possible, including replies to messages (see Section 2, Negotiating with telecommunication operators (on page 28))or downloads associated with use of the programme where smartphones are involved. CASE STUDY: PROMOTING AN mDIABETES PROGRAMME, SUDAN Sudan’s Ministry of Health launched its SMS mDiabetes programme in February 2020. A private company was contracted to run the campaign for the first 3 weeks, which included a video and audio advert in addition to posters, brochures and stickers. In the first week the advert was aired on different media channels (social media, TV and radio), and in the second week, text message adverts were sent to all mobile phone subscribers in Sudan through the country’s three main operators. The message reached more than 11 million mobile phone users. In the third week the programme reached out to the general population in public places, encouraging them to participate and distributing printed materials in different states. The campaign successfully enrolled over 75 000 participants in under 3 weeks. Initially, the programme was run as part of the country’s 6-week mDiabetes campaign, following which a mRamadan programme, designed to promote good health and share diabetes-related information during the holy month of Ramadan, was added to the same promotion campaign. In total, 71 242 participants successfully completed the programme. :ىلع دمتعي (لولأا عونلا يركس) لافطلأا يركس جلاع ،ةمظتنلما ةعباتلما ،ةضايرلا ةسرامم ،ةيحصلا ةيذغتلا ،ينلوسنلأا داشرلإاو معدلا رفوت hehyehnoonאjeemmeemlamא lamyeh_hamza_aboveאseenrehlamא hehlammeemhah yehrehכseenlamאbeh hehyehainwawtehlamlam tehmarbutayehmeemlamאainlamא tehmarbutahahsadlamא tehmarbutameemzahnoonmeem waw tehmarbutayehdalאhahtehlam_alefא tehmarbutahahsadlamא hehrehאzainwaw yehlamאtahyehlam_alefא noonwawainlamא tehmarbutameemzahnoonmeemwaw tehmarbutayehnoonאdalwawseenlamא yehrehכseenlamא tehאyehainmeemjeem ainmeem noonwawאaintehlamאbeh yehrehכseenlamאbeh hehyehainwawtehlamlam hehyehnoonאjeemmeemlamא lamyeh_hamza_aboveאseenrehlamא hehlammeemhah Promotion materials from mDiabetes in Sudan 52 CASE STUDY: A “MISSED CALL” SIGN-UP MECHANISM FOR mTOBACCOCESSATION PROGRAMME, INDIA India’s Ministry of Health and Family Welfare (MOHFW) and Ministry of Communication and Information Technology partnered with BHBM to introduce innovative technologies for strengthening tobacco cessation services through a two-way mTobacco cessation text message delivery system. Through this multistakeholder collaboration, a nationwide mCessation programme (QuitNow) was launched in January 2016 as part of the Prime Minister’s Digital India initiative. Registration with the QuitNow programme followed the steps adjacent: 1. User to call a toll free number from a mobile and hang up within two rings (a “missed call”), or to register at the website: http://www.nhp.gov.in/ quit-tobacco. 2. Text message sent to the subscriber confirming registration. 3. Series of text messages sent to the user providing information on: • The importance of quitting and encouragement to quit • Setting up a quit date • Tips to support the quit attempt • Follow-up support. User testing can be a good way to get feedback from target users and make the sign-up process more user friendly. A balance will need to be struck between getting all the data from the user that you need (in order to place them in a segment that receives tailored content or to act as a baseline for key evaluation indicators) and not tiring or boring them so they give up. Another option is automatic enrolment for particular groups of health service or telecoms service users. This approach has been used in BHBM programmes. In Zambia, all customers of one telecoms carrier received messages. In India and Sudan, users were automatically enrolled through health services or door-to-door when they screened positive for being at risk of diabetes (as part of a national inclusive screening programme) and there was a free and easy opt-out mechanism. If appropriate, and in partnership with health or screening services, this option could be a possibility, though the local context should be considered (e.g., legality of sending unsolicited messages or acceptability of such an approach). Once users have enrolled in the programme, two-way messages or gathering data in an app can help to gauge the ongoing participation of users (providing that responses are free). These messages or requests can be part of the basic content package such as motivational messages, or designed for the purpose of checking participation and monitoring health behaviour change (e.g., “Did you do 30 minutes of physical activity today? Reply 1 for yes or 2 for no”) and some of the BHBM message libraries include such messages. Information on participation can also be captured at the evaluation stage, for example, BHBM has collected useful data in surveys whereby users were asked to estimate the proportion of messages that they a) saw and b) read, and whether they sustained behaviour change throughout the programme. Retention Drop-out in many health behaviour change programmes is high, and mHealth programmes are no exception. For example, in an mRamadan tobacco cessation programme, 30% dropped out after the first week, 56% had dropped out after a month, and 68% had dropped out by month four. If users drop out and two-way messaging is free (or app users can be contacted), it is useful to ask why they are leaving (consent should be gathered for contacting participants about drop-out). Telephone surveys are another way to gather this information. BHBM programmes have surveyed drop-outs with interesting findings, such as in Tunisia, 47% of those surveyed left because the programme didn’t meet their expectation [from what was advertised]. 53% said they left because the guidance in the programme was hard for them to follow, with 40% suggesting more tailoring. Another important factor is the messages themselves, with users 53 in India stating they were not motivating enough. Drop- outs in India and Tunisia fed back that using videos and images would be more effective and also combining the mTobaccoCessation messaging programme with other quit services and support would have been beneficial. Ways to reduce drop-out include: • Interaction and two-way messaging16: this means giving the user dynamic and focused support (e.g., in a messaging programme, sending the word CRAVE to get support when craving) or messages/tasks about goal-setting and follow-up. If messages are received through an existing messenger app, ensure users can be notified of new content, because not all messenger apps have this feature (so you may have to send an SMS to alert them to new content). • Programme tailoring: The more users feel the programme is relevant to them and their goals, the more likely they will be to progress towards their goals and be motivated to complete the programme. Therefore, user testing the programme content is vital to get it right for your users. BHBM programme implementers have also suggested tailoring programmes to different user groups for this reason. For example, tailoring diabetes management messages according to whether the user has been diagnosed with diabetes is an obvious thing to do, but consider also tailoring according to age, gender, or whether the user is a rural or urban dweller. Tailoring will mean that screening questions will be needed to place users in a given group. The reply to these questions should be free, but if for any reason – other than cost – a user cannot reply, they should still be enrolled in a generic version of the programme. • Smooth user experience: The programme should run smoothly and responsively on the technology platform of choice. Technical problems, especially if there is no technical support for users, can be very frustrating and is a major reason for drop-out in BHBM programming. Two BHBM programmes had catastrophic issues with mobile communications carriers having technical difficulties, with 60% of messages not being sent and another instance where no messages were sent for a period of 6 weeks. Ensure, through pre-testing the platform, that messages or content will be sent or accessible and timely. For example, if the programme features two-way messaging, the response to the users’ reply should be instantaneous, otherwise, the user will become frustrated and may leave the programme. An app should be free of bugs and be responsive. • Option to opt-out: STOP messages are often included in SMS or other text-based programmes to enable the user to stop receiving messages (e.g., “Reply STOP if you wish to stop receiving mDementia messages!). It is an ethical imperative that users can unsubscribe or stop receiving messages if they wish, and it is important in any BHBM programme that they know how to do this. However, the literature suggests that programmes with more frequent STOP messages have higher drop-out rates24. We suggest sending two stop messages, one near the start and one mid- programme. For the mDementia programme, connecting with local dementia organizations and/or care or support services may help with retention. Some of the messages in mDementiaSupport provide links and contact details for local and national services in order to give the end-user more holistic support beyond what the programme can provide as well as making the programme feel more personalized. Offering a network of support may also help encourage end-users to stay in the programme (e.g. helping users connect with other people in similar positions through local dementia organizations, or having a human operator check in over the phone periodically to make sure they are on track). Gamification of the programme (e.g. earning points by completing activities) or providing a small reward or incentive (e.g. certificate of completion) at the end of the programme may also give users the motivation they need to finish the programme. A checklist for considerations for technology specifications can be found in Section 1, Developing and managing your workplan, (on page 21). 54 5 Monitoring and evaluation of mDementia Planning M&E 56 Process M&E 59 Outcome M&E 60 Selecting indicators for M&E 60 Designing evaluation 62 Monitoring and evaluation data analysis, reporting and dissemination 62 55 5 Monitoring and evaluation of mDementia Monitoring and evaluation (M&E) of the mDementia programme is intended to assess whether the programme improves population-level health outcomes, is cost-effective and/or warrants scaling and further investment. M&E also facilitates implementation and scaling by informing stakeholders, including those in other countries, about barriers, enablers and effectiveness of a given programme, as well as generating information that will enable the introduction of other mHealth and/or dementia programmes in the country. In addition to this, being able to show measurable results for any programme is an asset when fundraising for new or future programmes as these provide quantifiable arguments and demonstrate impact. M&E should be integrated into routine programme management functions as an ongoing activity from the start of implementation (see Figure 5 on page 57). It should be led by mHealth focal points in collaboration with the concerned unit responsible for the disease or risk factor specific area. Data should be collected on an ongoing basis, ideally establishing a baseline, continuously feed into the implementation process and measure change at the end. This helps assess impact, maximize resources and inform scale-up and year-on- year improvements. This section of the handbook represents a short guide to M&E. If you require further information and more comprehensive guidance, see the WHO guide for digital health monitoring and evaluation: https://www.who. int/reproductivehealth/publications/mhealth/digital- health-interventions/en/ PLANNING M&E Step 1: Define the goals of M&E Stakeholders should determine the desired outcomes of the mDementia programme at the early planning stage. These should include, for example, the ability of the programme to reach target groups (e.g. carers of people with dementia, individuals at risk of developing dementia and the general public) and the effectiveness of the programme content to generate health behaviour change. From these broad goals, questions to be answered through M&E can be developed. Such questions may include: • Does the mDementia platform operate as expected? • Is the content easy to understand and accessible to users? • How effective is the marketing/promotion campaign in reaching new users? 56 BE HE@LTHY, BE MOBILE: A handbook on how to implement mDementia Define goals of M&E Create/adapt an M&E framework Plan M&E human resources Select/adapt M&E indicatorsDesign studyPrepare budget Collect data and analyse data Reporting • How many people of the target population did the programme reach? What are the socioeconomic and demographic profiles of these individuals? • How many completed the programme in its entirety? • Are users improving their knowledge or modifying their behaviour as a result of the programme? It is important to have clear goals and questions in mind in order to complete the next steps of creating an M&E plan. Step 2: Create or adapt an M&E framework After the goals of M&E have been defined, the next step is to create a framework of how to achieve these goals. For more information, see “Developing an M&E framework” (chapter 2, part 2b of the WHO guide for digital health monitoring and evaluation) and see Box 6 (on page 57) for a “logic model” framework. A results chain or logic model identifies how resources (or inputs) make it possible to carry out programme activities, which in turn produce a series of results (or outputs and outcomes) and move the programme towards achieving its stated vision (or impact). See Figure 6 (on page 58) for an example logic model for mDementia (an editable version can be made available on request from bhbm@who.int). BOX 6: A LOGIC MODEL FRAMEWORK A logic model is a framework that illustrates the relationship between its inputs, outputs, outcomes and impact, and is easy to map to your programme’s aims. This is a visual concept of how the elements of the mHealth intervention influence each other. • Inputs are defined as the financial, human, material and intellectual resources used to develop and implement an intervention. • Processes or activities are defined as the activities undertaken in the delivery of an intervention and may include training courses or other capacity building, software or hardware development, adapting content, partnership/ negotiation meetings, programme promotion activities etc. • Outputs are defined as the direct products/ deliverables of an intervention’s processes or activities. • Outcomes refer to the short- or intermediate- term changes that emerge as a result of inputs and processes; and • Impact can be defined as the medium- to long-term effects produced by an intervention on population health, health systems or other benefits. FIGURE 5 OVERVIEW OF STEPS FOR mDEMENTIA M&E PROGRAMME 57 FIGURE 6. LOGIC MODEL FOR mDEMENTIA Input • Programme development resources • staff; • time; • funding; IT architecture; linking with health services; partnerships; • governance; policy change (if necessary); • operations and stakeholder management Activities • Situational assessment • Content adaptation (incl. user testing) • Programme design • M&E plan and implementation • Technology development (incl. testing) • Negotiation for free/low- cost messaging • Health workers (HCWs) trained for recruitment • Outreach and promotion Output • Baseline and needs data • Quality adapted content • Programme specifications • Functioning and integrated message platform • Low-cost/free 2-way messaging • No. of trained HCWs • No. of and type of promotion activities • Analytics on reach of promotion • No. of users enrolled (reach) • No. of messages sent and received Outcome • Baseline Vs post- data • User satisfaction re: promotion activities • User satisfaction with message programme • User engagement • Technology performance • Knowledge gains • Increased health literacy • Behaviour change • Medication adherence • Caregiver wellbeing • HCWs programme engagement and satisfaction • Systemic change in policy or health service delivery Impact • Reduction in dementia risk factors (e.g., increased physical activity, reduced blood pressure and cholesterol, decreased alcohol and tobacco consumption) • Slow in cognitive decline among those with mild cognitive impairment • Prevention of dementia among those at risk • Increased well-being of caregivers PROCESS MONITORING AND EVALUATION OUTCOME EVALUATION 58 A note on impact The overall impact of an mHealth programme at scale is its contribution towards the achievement of the Sustainable Development Goals (SDGs), in particular, SDG 3: “Ensure healthy lives and promote well-being for all at all ages”; and SDG 9: “Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation”. The impact of the mDementia programme also aligns with the vision and aim of the Global Dementia Action Plan3 (see Box 7) as well as the global targets for action areas 3 (risk reduction) and 5 (carer support). Measuring the impact of a programme at scale may not be feasible for certain settings. Impact indicators pose difficulty because it is hard to ascertain the effect of the programme on the impact indicator (owing to design options of scaled implementation evaluation), often termed attribution challenges. They also entail a significant investment of funds and time. It is for this reason that this handbook focuses on process (activities) and outcome indicators. Step 3: Plan M&E human resources The next step is to assign roles and responsibilities to staff to help carry out the M&E plan. These should be assigned based on the skills of the team. If there is no- one in the implementing team with experience in M&E, you may have to hire or train someone on M&E design, data collection and analysis. There are two parts to M&E of an mHealth programme that will likely require different skills to manage: process M&E and outcome M&E. Process M&E refers to the assessment of the activities and processes related to the implementation of the programme (e.g. IT, resources) and will predominantly require internal data collection activities and may be manageable with existing programme human resources. Outcome M&E refers to the assessment of the ability of the mHealth programme to achieve its target health outcomes (e.g. behaviour change) and will more likely require external human resources or internal human resources who can be field- based or able to reach participants (e.g. run surveys with programme users). BOX 7: GLOBAL ACTION PLAN ON THE PUBLIC HEALTH RESPONSE TO DEMENTIA 2017–2025 Vision: The vision of the global action plan on the public health response to dementia is a world in which dementia is prevented and people with dementia and their carers live well and receive the care and support they need to fulfil their potential with dignity, respect, autonomy and equality. Aim: The goal of the global action plan is to improve the lives of people with dementia, their carers and families, while decreasing the impact of dementia on them as well as on communities and countries. Action Area 3: Dementia risk reduction Global target: The relevant global targets defined in, and in keeping with, the Global Action Plan For Prevention and Control of Noncommunicable Diseases 2013–2020, and any future revisions are achieved. Action Area 5: Support for dementia carers Global target: 75% of countries provide support and training programmes for carers and families of people with dementia by 2025. PROCESS M&E Process monitoring provides information for planning and for feedback about the progress of the project. Inputs and processes are the critical resources that go into developing and implementing an mHealth programme. Implementers of all mHealth programmes should conduct routine process monitoring for the purposes of good programme management. This should be carried out internally and regularly (monthly or quarterly). Monitoring should start at the programme’s inception and a routine reporting mechanism will need to be set up to monitor the core indicators and key deliverables. In some countries, with more sophisticated web-based platforms, the monitoring report might be presented as a data dashboard. Monitoring reports and dashboards are helpful in providing a quick overview to see whether a programme is on track to reach its objectives. 59 Process evaluation is the periodic assessment (e.g. yearly) of the implementation of a programme in relation to planned activities and their overall objectives. It identifies the constraints that hinder the programme in achieving its objectives and can help to provide solutions that can then be implemented. A process evaluation tends to be carried out annually and utilizes a range of data collection methods. These can include the simple recording of the completion of key activities; the use of service analytics from telecoms company reporting (e.g., to understand the flow of content between the programme and user); or conducting focus groups and interviews for in-depth exploration of experiences, attitudes and ideas (e.g. feedback on frequency of messages, user experience etc.). OUTCOME M&E Outcome M&E is a type of assessment concerned with determining if, and by how much, a programme’s activities helped meet programme targets. It tracks information directly related to a programme’s users, such as changes in biological markers, knowledge or behaviours. It can be an important investment to analyse and communicate the effectiveness of the programme. In terms of designing outcome M&E, the two major considerations are data collection methodology and study design. These should be selected based on the goals of the programme and M&E questions set out in step 1. Suggested methods of data collection for a robust evaluation of mDementia would include surveys and biological tests such as blood sugar levels, cholesterol levels or blood pressure readings (see Annex 1 Pre- and post-programme questionnaires on page 67). A survey is suggested to assess the self-reported perceptions, behaviours, knowledge and attitudes of registered users. The best and simplest way to measure behaviour change is to deploy the same survey before and after an intervention. Surveys can be done online over the Internet, on a mobile device, or administered in-person by an interviewer over the phone or face-to- face. Since survey fatigue is a common challenge, surveys should be as short as possible and employ multiple- choice questions wherever possible. SELECTING INDICATORS FOR M&E Steps 4: Select or adapt M&E indicators The next step is to develop the indicators that will be used to evaluate the elements in the framework (logic model) developed in step 2. WHO defines an indicator as “a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention or to help assess the performance of a development actor”25. This handbook features a set of recommended indicators for the mDementia programme presented in Annex 7 Design of past evaluation studies of BHBM programmes (on page 98). This includes process indicators for all mHealth programmes and outcome indicators specific to the mDementiaPrevention and mDementiaSupport programmes. Data collection is expensive, so it is important to strike a balance between feasibility of data collection and utility of data to inform and improve the programme. For this reason, core indicators have been marked in blue in the indicator matrix in Annex 2 Monitoring and evaluation indicators (on page 77). Each indicator is presented with comments and a suggested data collection and reporting frequency. These core indicators are indicators that every mHealth programme should routinely collect and report on (including to the BHBM secretariat if the programme is supported by BHBM). This matrix serves as a template and is accompanied by an Excel file for inputting the data and may need to be adapted to context. In some cases, M&E may require a review and clearance by an ethical review committee, but it is important to check a specific country’s laws and regulations. The government or the agency responsible may identify the need to secure any necessary ethics approval for the process. This may be from national or local ethics committees, or from other stakeholder institutions with formal ethics approval systems. Specific indicators for mDementiaPrevention The relevant outcome indicators for the mDementiaPrevention programme should measure reductions in risk factors or unhealthy behaviours related to these risk factors (e.g. tobacco use, excessive alcohol consumption, obesity, hypertension, high cholesterol) and increases in protective factors (e.g. physical activity, 60 healthy diet, cognitive activity). Self-reported changes in health indicators can be captured via messages during the programme as part of programme monitoring or through surveys conducted post-programme as part of programme evaluation. Objective measures of cardiovascular risk factors and weight can also be collected in clinical settings by linking in with health services (e.g. blood pressure, cholesterol measures, blood sugar levels, weight). See Annex 1 Pre- and post- programme questionnaires (on page 67) and Annex 7 message monitoring questions (on page 98). Specific indicators for mDementiaSupport The relevant outcome indicators for the mDementiaSupport programme should relate to improvements in carer well-being and knowledge of dementia/dementia care. Self-reported measures of well- being, understanding, perceptions and attitudes towards care can be measured via two-way messages sent during the programme. We also recommend using psychological scales to assess outcomes for the mDementiaSupport programme. The relevant scales should assess the domains of carer burden, well-being, quality of life, depression, stress and/or self-efficacy (see Table 6 ). TABLE 6: PSYCHOLOGICAL MEASURES TO ASSESS CARER OUTCOMES DOMAIN DESCRIPTION Carer burden Carer burden is defined as the physical, psychological, emotional, social and financial stressors associated with caregiving26 Well-being Well-being is defined as the minimization of stress, burden, depression and/or anxiety27 Quality of life Quality of life is defined by WHO as an individual’s perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns28 Depression Depression is characterized by WHO as the persistent sadness and a lack of interest or pleasure in previously rewarding or enjoyable activities29 Stress Stress is a response to demands and pressures that challenge a person’s ability to cope30 Self-efficacy Self-efficacy is a person’s belief or confidence that they can successfully perform and achieve their goals in a given situation31 The most suitable scales will differ between countries but it is recommend that they: • have been validated for the country/context. This means that the scale has been translated, adapted and evaluated within the target population to ensure it captures the domains it was set out to measure. • have good psychometric properties, meaning the scales are both reliable and valid. • are brief, in order to minimize survey fatigue. • have been used as a population screening tool rather than in a clinical context. This means they can be administered on a large scale and are suitable for low-literacy populations. Ideally, the scales should also be self-reportable, meaning that they can be administered without a trained clinician. 61 For more information on psychology scales for carers click on the links for each domain in Table 6.27,32,33,34,35,36 The BHBM team can also help you choose the most appropriate scales for your context, please email bhbm@who.int. WHO can provide translated versions of the WHOQOL-BREF37 to assess quality of life and WHO-538 to assess depression when requested. DESIGNING EVALUATION Step 5: Design the evaluation and prepare data collection materials While programme monitoring is conducted routinely, programme evaluation is conducted periodically (e.g. every 6 or 12 months) to gauge whether the programme is achieving its objectives and to decide if any adjustments need to be made. Annex 7 Design of past evaluation studies of BHBM programmes provides a (non-exhaustive) on page 98 list of examples of M&E designs. All BHBM programmes use surveys, the design of which depends on available resources. In the case of mDementia, example surveys for capturing self- reported behaviour change for mDementiaPrevention and mDementiaSupport can be found in Annex 1 Pre- and-post programme questionnaires (see Figure 7 on page 63, for one example). A pre-post evaluation design is recommended for both modules. However, if this is not possible or resources are limited, some message programme monitoring questions that are sent during the programme (e.g. “Since the beginning of the programme, have you increased X behaviour?”) may be able to provide similar information (see Annex 7 SMS monitoring questions on page 99). Once you have completed steps 1-5, you should have a good idea of what information you will collect, when, how, and by whom. This will enable you to carry out step 6, Prepare M&E budget. Step 6: Prepare M&E budget It is important that financial resources be dedicated for M&E. This budgeting will depend greatly on the scope of the M&E activities and the evaluation design you have selected. It will be necessary to have budget lines for the following overarching areas: Human resources – How many full-time equivalent staff will you need, and at what pay grade, to carry out M&E activities from design to reporting? • Translation, adaptation and/or development of tools – Existing tools may suffice, or you may need to adapt or validate tools to your setting, or even create new questionnaires that incorporate questions on all your selected indicators. • Data collection and analysis – This covers all data collection activities, and could include travel required for project staff to collect data, telephone bills to conduct telephone surveys, incentives for respondents, hiring spaces to meet with respondents, data analysis software licensing if necessary. • Dissemination – This covers publishing costs if you wish to publish findings. Some scientific journals charge US$ 5000 to publish a paper via open access, or you may want to incorporate findings into future promotion campaigns or updating materials, for example. MONITORING AND EVALUATION DATA ANALYSIS, REPORTING AND DISSEMINATION Step 7: Collect and analyse data Data analysis will be necessary to make sense of the data for reporting purposes as well as to inform the viability, impact, continuity and scalability of the programme. Analysis could be simple percentage calculations (e.g. % of users that have quit tobacco since the start of the programme), collating qualitative data into themes (e.g. reasons why some participants did not/could not change their behaviour), or if your quantitative data is robust enough, this may include running statistical tests on the data (e.g. is there a statistical difference in the number of users who completed the programme based on age, gender or other sociodemographic information?). Speak with someone in the data division of your organization if you are not sure how to analyse the data you have gathered, or the BHBM country support team can help. Analysing the data will allow you to answer the questions you developed in step 1. 62 FIGURE 7: EXAMPLE OF PRE-POST EVALUATION DESIGN FOR mDEMENTIAPREVENTION Step 8: Report and disseminate findings M&E data should be consistently reported to inform programme implementation in an iterative manner. Data should be presented in a concise, user-friendly format and be relevant to the target audience. It should be used to support collaboration and decision-making among stakeholders regarding ongoing resource allocation and processes for the programme’s sustainability and scalability. For example, if the data shows irregularities in registration or delivery of messages, or user responses, then such issues must be brought up to the concerned decision-makers through regular review meetings. In addition, this data should inform annual process and outcome evaluation reporting to show progress and lessons learned. If there are multiple audiences, such as programme implementers and policy-makers, the data needs to be presented in line with their respective priorities. Evaluators should generate a list of all relevant stakeholders, such as policy-makers, donors, programme staff, etc., and consider who is most likely to use the data collected from the evaluation, how they might utilize that information, and the necessary communication style of the report. Evaluation findings should be disseminated in an easy-to-read format and in an accessible and timely manner through: • formal and informal networks via meetings, newsletters and other forums; • professional conferences via discussion papers or posters; • journals (scientific or lay); • electronic media, such as web pages, social media and e-mail; • briefings with policy-makers; • media channels for key stakeholders, e.g. health and social care workers and the general public. For detailed information on M&E for digital health interventions, see WHO, 2016. Monitoring and evaluating digital health interventions - A practical guide to conducting research and assessment. This guide provides an introduction to the approaches and methods to support countries in strengthening their digital health deployments, develop robust evaluations, and scale up their activities nationally and regionally. A checklist for considerations for technology specifications can be found in Section 1, Developing and managing your workplan, (on page 21). On average, how much physical activity do you do per week? Post-programme survey question On average, how much physical activity do you do per week? Pre-programme survey question “Remember to do you exercise. Aim for 2.5 hours per week, which can be done by exercising 30 minutes on most days of the week.” User receives mDementia message “Did you do your 30 min of exercise today?” SMS Monitoring question 63 References 1 Dementia: number of people affected to triple in next 30 years. In: World Health Organization [website]. Geneva: World Health Organization; 2017 (https:// www.who.int/news/item/07-12-2017-dementia- number-of-people-affected-to-triple-in-next-30- years, accessed 19 January 2021). 2 Dementia. In: World Health Organization [website]. Geneva: World Health Organization; 2020 (https:// www.who.int/news-room/fact-sheets/detail/ dementia#:~:text=Social%20and%20economic%20 impact&text=In%202015%2C%20the%20total%20 global,%25%20in%20high%2Dincome%20 countries, accessed 19 January 2021). 3 Global Action Plan on the Public Health Response to Dementia 2017–2025. Geneva: World Health Organization; 2017. 4 Guidelines for risk reduction of cognitive decline and dementia. Geneva: World Health Organization; 2020. 5 Whittaker R et al. Mobile phone text messaging and app-based interventions for smoking cessation. Cochrane Database of Systematic Reviews. 2019(10). 6 Sahin C et al. Tailored mobile text messaging interventions targeting type 2 diabetes self- management: A systematic review and a meta- analysis. Digital Health. 2019;5:2055207619845279. 7 Zhao Y-Y et al. The effect of text message reminders on medication adherence among patients with coronary heart disease: A systematic review and meta-analysis. Medicine, 2019;98(52). 8 Smith DM et al. Text message interventions for physical activity: a systematic review and meta- analysis. American Journal of Preventive Medicine. 2020;58(1)142–151. 9 Global action Plan for the Prevention and Control of Noncommunicable Diseases 2013-2020. Geneva: World Health Organization; 2013. 10 Transforming our world: the 2030 Agenda for Sustainable Development. New York, NY: United Nations Division for Sustainable Development Goals; 2015. 11 Michie S et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine. 2013;46(1): 81–95. 12 Digital health platform handbook: Building a digital health information infrastructure (infostructure) for health. Geneva: International Telecommunications Union; 2020. 13 Planning an information systems project: a toolkit for public health managers. Seattle: PATH; 2013. 14 Buchholz SW et al. Physical activity text messaging interventions in adults: a systematic review. Worldviews on evidence-based nursing. 2013;10(3):163–173. 15 Free C et al. Smoking cessation support delivered via mobile phone text messaging (txt2stop): a single- blind, randomised trial. Lancet. 2011;378(9785): 49–55. 16 Head KJ et al. Efficacy of text messaging-based interventions for health promotion: a meta-analysis. Social science & medicine. 2013;97: 41–48. 17 Steinka-Fry KT et al. Culturally sensitive substance use treatment for racial/ethnic minority youth: A meta-analytic review. Journal of Substance Abuse Treatment. 2017;75:22–37. 18 Benish SG, Quintana S, Wampold BE. Culturally adapted psychotherapy and the legitimacy of myth: a direct-comparison meta-analysis. Journal of Counseling Psychology. 2011;58(3):279. 19 Shehadeh MH et al. Cultural adaptation of minimally guided interventions for common mental disorders: a systematic review and meta-analysis. JMIR Mental Health. 2016;3(3): p. e44. 20 Rodgers A et al. Do u smoke after txt? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tobacco control, 2005;14(4):255–261. 21 Free C et al. Txt2stop: a pilot randomised controlled trial of mobile phone-based smoking cessation support. Tobacco Control. 2009;18(2): 88–91. 22 Colantonio LD et al. Cross-cultural adaptation of a text message-based program for smoking cessation in Buenos Aires, Argentina. Nicotine & Tobacco Research. 2015;18(3):314–320. 23 McCool J et al. Assessing the cross-cultural adaptation and translation of a text-based mobile smoking cessation program in samoa (TXTTaofiTapaa): pilot study. JMIR mHealth and uHealth 2018;6(8):e173. 64 24 Grutzmacher SK et al. Predicting attrition in a text- based nutrition education program: survival analysis of Text2BHealthy. JMIR mHealth and uHealth. 2019;7(1): e9967. 25 WHO evaluation practice handbook. Geneva: World Health Organization; 2013. 26 George LK, Gwyther LP. Caregiver well-being: a multidimensional examination of family caregivers of demented adults. The Gerontologist. 1986;26(3):253–259. 27 Cunningham NA, Cunningham TR, Roberston JM. Understanding and measuring the wellbeing of carers of people with dementia. The Gerontologist. 2019;59(5):e552–e564. 28 Measuring quality of life: The World Health Organization quality of life instruments (the WHOQOL-100 and the WHOQOL-BREF). WHOQOL- measuring quality of life 1997. 29 WHO. Depression [website]. Geneva: World Health Organization; 2020 (https://www.who.int/ health-topics/depression#tab=tab_1, accessed 19 January 2021). 30 Leka S et al. Work organization and stress: systematic problem approaches for employers, managers and trade union representatives. Geneva: World Health Organization; 2003. 31 Crellin NE et al. Self-efficacy and health-related quality of life in family carers of people with dementia: a systematic review. Aging & Mental Health. 2014;18(8):954–969. 32 Deeken JF et al. Care for the caregivers: a review of self-report instruments developed to measure the burden, needs, and quality of life of informal caregivers. Journal of Pain and Symptom Management. 2003;26(4):922–953. 33 Dow J et al. How best to assess quality of life in informal carers of people with dementia; A systematic review of existing outcome measures. PloS One. 2018;13(3):e0193398. 34 Van Durme T et al. Tools for measuring the impact of informal caregiving of the elderly: a literature review. International Journal of Nursing Studies. 2012;49(4):490–504. 35 Whalen KJ, Buchholz SW. The reliability, validity and feasibility of tools used to screen for caregiver burden: a systematic review. JBI Database of Systematic Reviews and Implementation Reports. 2009;7(32):1373–1430. 36 Breedvelt JJ et al. A systematic review of mental health measurement scales for evaluating the effects of mental health prevention interventions. European Journal of Public Health. 2020;30(3):539–545. 37 Development of the World Health Organization WHOQOL–BREF quality of life assessment. Psychological medicine. 1998;28(3):551–558. 38 WHO Five Well-Being Index. In: Child Outcomes Research Consortium [website]. London: CORC; n.d. (https://www.corc.uk.net/outcome-experience- measures/the-world-health-organisation-five- well-being-index-who-5/, accessed 19 January 2021). 65 Annexes Annex 1 Pre- and post-programme questionnaires 67 Annex 2 M&E Indicators 77 Annex 3 Benefits and risks of different software models 89 Annex 4 The role of aggregators 90 Annex 5 Main themes and operationalizing strategies for message development 91 Annex 6 Adapting content library for voice, messenger apps or chatbots 92 Annex 7 Design of past evaluation studies of BHBM programmes 98 Annex 8 Programme monitoring questions 99 66 ANNEX 1 PRE- AND POST-PROGRAMME QUESTIONNAIRES This annex sets out some sample questions you can use for your pre- and post-programme surveys. In cases where implementers have to create their own survey questions, Figure A1.1. depicts how – working back from your M&E goal – you can formulate appropriate M&E questions, indicators, and survey questions for both mDementiaPrevention and mDementiaSupport programmes. FIGURE A1.1: Sample steps to formulate survey questions to meet mDementia M&E goals (for mDementiaSupport and mDementiaPrevention). How many people quit tobacco since the start of the programme Did the programme help people improve their knowledge about dementia? % of surveyed users that report having improved knowledge of dementia Compared to how you felt at the beginning of the programme, do you feel like you understand more about dementia? % surveyed users who used tobacco at the start of the programme who quit tobacco Have you stopped consuming tobacco since commencing the programme? Evaluate behaviour change Evaluate knowledge gain Goal of mDementia Prevention M&E Goal of mDementia Support M&E M&E Question M&E Question Indicator Indicator Survey Question Survey Question 67 mDementiaPrevention The pre-screening questionnaire (see Table A1.1) for mDementiaPrevention is used to gather demographic and baseline measures. It can also be used to help select the relevant messages for each participant based on which risk factors are present (see Figure A1.2 for a breakdown of messages by target audience). The questions can be asked over the phone, delivered through a paper or web-based survey and/or adapted for SMS delivery at enrolment into the programme. (Note: the sections in blue text need to be adapted to cultural and/or country customs and norms.) TABLE A1.1: SAMPLE PRE-PROGRAMME QUESTIONNAIRE QUESTION ADD-ON MESSAGES What is your date of birth? What is your gender? 1 Male 2 Female 3 Other Highest level of education completed: 1 Primary [additional information or definition relevant to grade] 2 Secondary/high school [additional information or definition relevant to grade] 3 Tertiary/Bachelor Degree [additional information or definition relevant to grade] 4 Higher degree/Postgraduate [additional information or definition relevant to grade] Do you use tobacco products (e.g. cigarettes, shisha etc.)? 1 Yes 2 No If yes ‣ add on tobacco messages How much alcohol do you drink on average? [Insert link to local resource on number of alcoholic drinks w/ pics] 1 Don’t drink alcohol 2 Less than one drink a day 3 1 to 5 drinks a day for men or 1 to 3 drinks a day for women 4 More than 5 drinks a day for men or more than 3 drinks a day for women If 3 OR 4 ‣ add on alcohol messages Do you currently have diabetes? 1 Yes 2 No 3 I don’t know If yes ‣ add on diabetes messages Do you currently have high blood pressure? 1 Yes 2 No 3 I don’t know If yes ‣ add on hypertension messages Do you currently have high cholesterol? 1 Yes 2 No 3 I don’t know If yes AND age is between 40 and 65 ‣ add on cholesterol messages 68 QUESTION ADD-ON MESSAGES Has a doctor prescribed you with medication for diabetes, hypertension or high cholesterol? 1 Yes 2 No If yes ‣ add on medication use messages Are you overweight or obese? 1 Yes 2 No 3 I don’t know If yes AND age is between 40 and 65 ‣ add on weight messages Do you currently have memory problems? 1 Yes 2 No If yes ‣ add on memory messages On average, how much physical activity (e.g. walking, running, sports) do you do per week? 1 less than 1 hour 4 3–4 hours 2 1–2 hours 5 4–5 hours 3 2–3 hours 6 more than 5 hours FIGURE A1.2: BREAKDOWN OF THE MESSAGES BY TARGET AUDIENCE Physical Activity, Diet/Nutrition, Cognitive Activity TARGET AUDIENCE RISK FACTORS mDementiaPrevention Mid-life (40-65) Weight High Cholesterol Hypertension Diabetes Tobacco Alcohol Memory Problems Medication Use Late-life (65+) Healthy / No Risk Factors Healthy / No Risk FactorsAt Risk At Risk Self-reported Memory Problems 69 The post-programme questionnaire (see Table A1.2) is used to evaluate effectiveness of programme in changing behaviour and impact on health outcomes, in addition to user experience. It can be delivered in a similar format to the pre-programme questionnaire (e.g. over the phone, paper/web survey or adapted to be delivered through messaging at completion of the programme). The number of questions and format can be adapted depending on how the questionnaire is delivered. The questions should match what was included on the pre-screening questionnaire for comparative purposes. TABLE A1.2: EXAMPLE POST-PROGRAMME QUESTIONNAIRE After completing the programme, do you feel like you understand more about how to reduce your risk for dementia? 1 Yes 2 No If you were a tobacco user when you started the programme, have you reduced the amount of tobacco you use since commencing the programme? 1 Yes 2 No 3 I didn’t use tobacco [If yes] ‣ Have you quit smoking or using tobacco? 1 Yes 2 No How much alcohol do you drink on average? [Insert link to local resource on number of alcoholic drinks w/ pics] 1 Don’t drink alcohol 2 Less than one drink a day 3 1 to 5 drinks a day for men or 1 to 3 drinks a day for women 4 More than 5 drinks a day for men or more than 3 drinks a day for women Since the programme started, have you reduced the amount of alcohol you consume? 1 Yes 2 No 3 Not applicable, I don’t drink alcohol On average, how much physical activity (e.g. walking, running, sports) do you do per week? 1 less than 1 hour 4 3–4 hours 2 1–2 hours 5 4–5 hours 3 2–3 hours 6 more than 5 hours Nutrition Since the programme started, have you increased the amount of vegetables you eat? 1 Yes 2 No Since the programme started, have you reduced the amount of red meat and meat products you eat? 1 Yes 2 No 3 Not applicable Since the programme started, have you reduced the amount of fried and sugary foods and drinks you consume? 1 Yes 2 No Since the programme started, have you reduced the amount of salt you consume? 1 Yes 2 No 70 Since the programme started, have you learnt a new skill (e.g. a new language, musical instrument)? 1 Yes 2 No Since starting the programme, have you increased the amount of social activities you do? 1 Yes 2 No Since the programme started, have you lost weight? 1 Yes 2 No Since the programme started have you increased your use of health services (e.g. seeking health check-ups or advice) 1 Yes 2 No TABLE A1.3: PROGRAMME EVALUATION How many of the text messages did you read? (Select 1 only) 1 None, I didn’t receive any 2 None, I didn’t read any 3 Some (less than half) 4 Most (more than half) 5 All or nearly all of the messages Did you share any of the messages with others? (Friends, partner, family members, etc.) 1 Yes 2 No What did you think about the number of messages we sent? 1 Too few, I would have liked more 2 The right amount 3 Too many messages What did you think about the length of the programme? 1 Too short 2 The right length 3 Too long Did you have any technical problems with the programme? (e.g. could not sign up easily or could not read messages) 1 Yes 2 No Would you recommend the programme to others? 1 Yes 2 No Did taking part in the programme help you stay well or improve your health? 1 Yes 2 No 71 If resources permit, clinical/health measures should be collected to determine the effectiveness of the mDementiaPrevention programme in reducing dementia risk factors. Table A1.4 includes examples of clinical measures that could be collected. If used, these need to be conducted both pre- and post-programme for comparative purposes. These measures can be administered through a health care worker or medical professional or as part of a clinical study. Data collection of this sort is resource intensive and should only be used if one has the necessary resources to do so. TABLE A1.4: SAMPLE CLINICAL DATA COLLECTION FORM Hypertension is a risk factor for dementia. Measure and record the patient/participant’s blood pressure before and after the programme. Blood pressure before programme Blood pressure after programme Is the patient/participant’s blood pressure less than 140/90 mmHg? Yes No Diabetes is a risk factor for dementia. If the patient/participant has diabetes, measure and record the patient’s fasting blood sugar levels before and after the programme. Blood sugar level before programme Blood sugar level after programme Is the patient/participant’s fasting blood sugar less than 7 mmol/L or 126 mg/dl? Yes No High cholesterol is a risk factor for dementia. Measure and record the patient/participant’s cholesterol levels before and after the programme. Cholesterol level before programme Cholesterol level after programme Is the patient/participant’s total cholesterol reading less than 5.0 mmol/l or 190 mg/dl? Yes No Being overweight or obese is a risk factor for dementia. Measure and record the patient/participant’s body mass index (BMI) and waist circumference before and after the programme. BMI before programme Waist circumference before programme BMI after programme Waist circumference after programme Was the patient/participant’s BMI less than 25 kg/m2? Yes No Is the patient/participant’s waist circumference <90 cm if you are male or <80 cm if they are female? Yes No 72 Measuring cognitive decline and dementia risk • If possible, obtain a measure of cognition for the user before and after the programme to measure cognitive decline over time. The WHO iCOPE handbook provides examples of cognitive measures. • Several dementia risk scores are available3 and offer a way to more directly measure dementia risk. Some risk scores can be completed via self report. Measurement through risk scores (if used) should be provided both before and after the programme to evaluate the effect of the programme on dementia risk. • The effects of the programme on incident dementia can be measured through followed cognitive assessments (e.g. at 2 years, 5 years, 10 years post-programme) or obtainable through health records. 3 Stephan BC et al. Prediction of dementia risk in low-income and middle-income countries (the 10/66 Study): an independent external validation of existing models. The Lancet Global Health. 2020;8(4):e524–e535. mDementiaSupport • In addition to psychological scales used to measure carer outcomes outlined on page 60, which should be conducted both pre- and post-programme, an additional pre-screening questionnaire for mDementiaSupport can be used to gather demographic information, baseline information and/ or information used for tailoring the programme. An additional post-programme questionnaire can also be used to gather information about programme evaluation. Tables A1.5 and A1.6 show examples of questions and information that countries can gather. The questions can be asked over the phone, delivered through a paper or web-based survey and/or adapted to be delivered through SMS or instant message at enrolment into the programme. Note: The sections in blue text will need to be adapted to cultural and/or country customs and norms. TABLE A1.5: SAMPLE PRE-PROGRAMME DEMOGRAPHIC QUESTIONS What is your date of birth? Gender Male Female Other What is your relationship to the person you care for? Are you his/her: Spouse/Partner Daughter or Son-in-law Other Sibling From another family Child Friend/Neighbour What is the gender of the person you care for? Male Female Other How much time do you dedicate to care? Full-time voluntary carer Part-time voluntary carer Other Do you have any disabilities or conditions that impact your day to day life? If yes, tick all that apply Physical disability Sensory disability Mental health condition Brain injury Cognitive condition (e.g. intellectual disability, dementia, autism spectrum disorder etc.) Other 73 Are you caring for anybody else? If yes, tick all that apply Children Parents Siblings Other family members or friends Other, please detail What is your current income level? [If relevant to country/region context] [Insert list of normative income brackets] What is your ethnicity? [If relevant to country/region context] [Insert list of ethnicities common to region] TABLE A1.6: SAMPLE PRE-PROGRAMME QUESTIONS FOR PROGRAMME TAILORING Why did you sign up for mDementiaSupport? (Tick all that apply) I want to understand more about dementia I want tips on how to care for the person with dementia I want tips on how to maintain my own well-being while being a carer I want to learn more about how to obtain help and support I want to learn more about communicating with the person with dementia I want to learn more about responding to behaviour changes in the person with dementia Other, please detail Does the person you care for experience any of the following? (Tick all that apply) Problems with eating and drinking Problems with toileting and continence Problems with personal care (e.g. brushing teeth, bathing etc.) Memory loss Aggression Depression, anxiety, apathy Difficulty sleeping Delusions and hallucinations Repetitive behaviour Walking and getting lost Changes in judgement Communication difficulties Sensory issues Other, please detail 74 EXAMPLE PRE-PROGRAMME BASELINE QUESTIONS How much do you feel like you know about dementia? 1 I know nothing about dementia 2 I know very little about dementia 3 I know some information about dementia 4 I know a fair bit about dementia 5 I know a lot about dementia Please select how much you agree with the following statement: I know what it means to be a carer for someone with dementia 1 Strongly disagree 2 Disagree 3 Neither agree or disagree 4 Agree 5 Strongly agree Please select how much you agree with the following statement: I know what changes to expect in the person with dementia as the disease progresses 1 Strongly disagree 2 Disagree 3 Neither agree or disagree 4 Agree 5 Strongly agree POST-PROGRAMME QUESTIONS FOR PROGRAMME EVALUATION How many of the text messages did you read? (Select 1 only) 1 None 2 Some 3 All or nearly all What did you think about the number of messages we sent? 1 Too few, I would like more 2 The right amount 3 Too many messages What did you think about the length of the programme? 1 Too short 2 The right length 3 Too long Did you have any technical problems with the programme? (E.g. could not read texts) 1 Yes 2 No Would you recommend the programme to others? 1 Yes 2 No 75 Did taking part in the programme help you stay well or improve your health? 1 Yes 2 No Since joining the programme, do you feel that you have more knowledge of the health and support services available to you? 1 Yes 2 No How much do you feel like you know about dementia? 1 I know nothing about dementia 2 I know very little about dementia 3 I know something about dementia 4 I know a fair bit about dementia 5 I know a lot about dementia Please select how much you agree with the following statement: I know what it means to be a carer for someone with dementia 1 Strongly disagree 2 Disagree 3 Neither agree or disagree 4 Agree 5 Strongly agree Please select how much you agree with the following statement: I know what changes to expect in the person with dementia as the disease progresses 1 Strongly disagree 2 Disagree 3 Neither agree or disagree 4 Agree 5 Strongly agree 76 ANNEX 2 M&E INDICATORS Indicators in blue are Key Performance Indicators – i.e. those seen as core or essential to BHBM programming. # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY OPERATIONS 1 Number of full-time equivalent persons working on programme at leading agency Input This gauges the human resources commitment from the leading implementation agency. Terms of reference of involved employees or verbally from team lead Annually 2 Number of full-time equivalent persons working on programme at supporting agencies Input This gauges the human resources commitment from supporting agencies such as WHO or ITU regional or country offices. Terms of reference of involved employees or verbally from team lead Annually 3 National technical advisory group set up Input This records whether the team is functioning by month 3. Carried out: Yes/no, and composition Year 1, month 3 4 Commitments for funding (US dollars) across contributors and the duration of each commitment Input The funding source, level and duration will help implementers to gauge whether more funding is needed, and plan accordingly. Qualitative description. May have to speak with others responsible for resource mobility Annually 5 Number of fundraising activities for sustainability Activities Suggest to think about sustainability from the outset and nurturing funding relationships throughout. Carried out: Yes/no and description Annually 6 Budgeted plan produced for current and following year Activities While this plan may evolve as it develops, there should nevertheless be a plan. Carried out: Yes/no Annually 7 Budget spent Input A budget report, be it an Excel spreadsheet or a written report, is useful to monitor spending and plan ahead. Main budgetary items in US dollar amounts Quarterly (and cumulative) 77 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY STAKEHOLDER ENGAGEMENT 8 Number of partnerships and name of partner organizations with supporting documentation such as a memorandum of understanding or terms of reference Input This will enable the programme to measure the interest and growth of the programme in terms of its partnerships’ support from other government agencies, health services, private sector and civil society. Records on number of partnerships that include supporting documentation Annually 9 Number of meetings with external partners Input This is the number of meetings held between organizations not involved in the everyday running of the programme. Such organizations would include telecommunications authorities, civil society, other relevant ministries, and private sector partners. Suggest this to take place as required, but at minimum there should be quarterly meetings with such partners. Maintain a count of such meetings 6 monthly POLICY ENGAGEMENT AND SYSTEMIC CHANGE 10 Description of policy engagement Input/ activities/ output This qualitative indicator can describe the policy level support to the programme, or any change resulting the programme. Describe the policy interaction with the programme over the last year, e.g., policy consultations held, policy-makers involved in programme meetings, number of policy briefs prepared, newsletters sent, actual policy change etc. Maintain a record of such interactions and events Annually 11 Description of systemic change attributable to programme activities Activities/ output This qualitative indicator can describe any changes to the health system as a result of programme activities over the past year. This may include other uses of acquired software or the content delivery platform, institutionalization of the programme, changes in referral mechanisms, changes in public awareness of the health topic, new synergies between partners or between ministries, changes in process or procedures among partners due to the programme etc. This attempts to document other added value of the programme. Conduct focus groups or survey to measure these changes Annually 78 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY PROGRAMME CONTENT 12 Number of design consultations or meetings Activities This would include the number of meetings with technical experts, target users and other stakeholders onboarded to work on content. Maintain a count of such meetings Year 1 end 13 Programme specifications have been drawn up Input Have the programme design specifications been set, such as verifying the aims, adapting the logic model if necessary, designing the length of the programme and its rules? Carried out: Yes/no End 2nd quarter 14 Number of new messages developed/new app content or features Activities Numeric if new messages have been created or description if app content has been created. Please send any new content to BHBM secretariat (bhbm@who.int). Records of new content sent to BHBM Annually 15 Number of content development or adaptation focus groups/user testing sessions Activities The number of user testing sessions with different informants, e.g., focus groups with community members, health workers, specialist clinicians, academics. For example: Academics and specialists: Number of groups 1, Total no. of informants 6 Community members: Number of groups 3, Total no. of informants 24 Health workers: Number of groups 1, Total no. of informants 7 Carried out: Yes/no Annually (if new content is produced after Year 1) 16 Messages adapted Activities This is an important project milestone but may be relevant only to year 1. Carried out: Yes/no End 2nd quarter 17 Verification of message fidelity Activities This may be relevant only to year 1. Carried out: Yes/no End 2nd quarter 79 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 18 Content management system set up and/or maintained Activities The content management system (CMS) may be as simple as an Excel spreadsheet or word document (and passed to telecoms companies or bot providers), or as advanced as an in-house software solution with interoperable programming language. What is important is whether the CMS exists and is maintained. Carried out: Yes/no Annually 19 % of users sharing the message content with others not enrolled in the programme Output This will assess the additional reach of messages and can be used as an indication of the satisfaction with content. Surveys/message survey replies Post-programme 20 % of users reporting satisfaction with the content they received e.g.: • Ease of understanding the messages • Easy to operationalize advice or instructions • Content appropriateness • Content relevance • Programme length • Likelihood to suggest programme to a friend Output These suggested indicators can be rated on a Likert scale 1–5. User testing/surveys/ message survey replies User testing phase and/or post- programme 80 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 21 Estimation of the number of messages read by users Output Though messages are sent, receiving doesn’t mean engaging, so it could be helpful to ask “how many of the text messages did you read?” (select one option only): 1 None, I received no messages; 2 None, I read no messages; 3 Some (less than half); 4 Most (more than half); 5 All or nearly all of the messages. Surveys/message survey replies During and post- programme PROMOTION 22 Promotion strategy compiled Activities A promotion strategy is a plan laying out the promotion activities that will be completed and their timeframe. Carried out: Yes/no Annually 23 Number of promotion campaigns and type Output FACEBOOK ADVERT Target population: General population Number of events: 3 Reach (no. of people): 300 000 RADIO ADVERT Target population: General population Number of events: 4 Reach (no. of people): Approx. 800 000 POSTERS IN CLINICS Target population: Health service users Number of events: 38 clinics Reach (no. of people): Approx. 900 000 Marketing medium provider analytics (radio programme or broadcasting company (TV) or social media platform should be able to provide analytics) 6 monthly 81 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 24 Number of users who were made aware of the mHealth intervention by a given marketing channel Output How did you hear about the programme? Survey through message channel or telephone survey Annually 25 User satisfaction with promotion campaign Outcome User satisfaction questions could include: • Were the promotion materials easy to understand? • Were they appropriate for you and your community? • Were you able to sign up with the information provided by the promotion campaign? Survey Annually 26 % of surveyed health care workers who know about (or use) the programme Output Have you heard of mDementia? Survey Annually 27 % of surveyed health care workers who encourage their patients to use the programme Outcome This indicator attempts to understand the health care worker’s programme engagement. Survey 6 monthly FUNCTIONALITY AND TECHNOLOGY PERFORMANCE 28 Ease of sign up: Was it easy to subscribe on a scale of 1–5? (1 being very difficult, 5 being very easy) Output Any barriers to sign-up may have a serious effect on the number of subscribers. The process should be clear, easy and not too burdensome. This indicator can check this. If users say it was not easy, you may need to revisit the design of your sign-up procedure. Message or telephone survey responses User testing and first Quarterly 29 Number of system errors Output An error in the operating system or the dashboard that may or may not impact the delivery of content. Service analytics4 and/ or message or telephone survey responses Quarterly 4 Service analytics are routinely collected, back-end data from a platform that can be provided by the service operator (e.g., telecoms provider, chatbot provider). 82 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 30 Number of days/weeks of system downtime Output This records any amount of time that content did not reach users due to a system error. Service analytics Monthly 31 Messages: • Number of messages delivered • Rate of successful delivery of messages Output The numerator is the messages delivered and the denominator is the number of message attempts. Service analytics and/or message survey responses Monthly 32 Number of bugs reported and fixed (apps) Output Records should be kept on any technical issues with the app. Technical report Monthly REACH AND RETENTION 33 Number of people who have subscribed/registered Output A number is requested. Service analytics Monthly 34 % of target population registered Output It may be helpful to report this as a % of the total number of people that your promotion campaigns were estimated to reach, to gauge its success. Or this could be expressed as a % of the target population who are able to access the programme (e.g., who have access to a phone and power source) (if the number is known), e.g., ((total registered users/promotion reach)x100). Service analytics and promotion analytics 6 monthly 35 Number of new subscribers per month Output This helps to see if appetite for the programme is maintained and can help you to work out if promotion strategies are meeting recruitment targets. Service analytics Monthly 36 Demographic information about users Output This can help you to assess what groups are equitably accessing/ not equitably accessing the programme. Telephone surveys/ message survey replies Quarterly 83 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 37 User engagement: % of users retained for • 1 week of programme • 1 month of programme • 3 months of programme • Complete programme Outcome This can be captured by a representative telephone survey, a message survey or through analytics of STOP replies/opt-outs. Depending on how you capture this data, the definition of retention will change and the denominator used in the % calculation will change (number surveyed vs number enrolled in programme). Survey/service analytics Monthly 38 Number of messages/inputs received from users Output If a 2-way messaging programme or other 2-way communication platform is being used, on average, how many responses were received from participants? This could be reported as a %, with the total number of prompts for reply as the denominator for the % calculation ((replies/number of prompts)x100). Service analytics Quarterly 39 % messages sent from programme that are responded to appropriately by user Output This indicator measures active engagement and indicates whether users are complying with instructions or messaging as directed, e.g. number of valid responses to two-way messaging. Service analytics 6 monthly BEHAVIOUR CHANGE mDementiaPrevention 40 % of surveyed users that increased their physical activity to at least 30 minutes a day for most days of the week Outcome Example question: • In the past week, on how many days did you do physical activity for 30 minutes? Reply with a number from 0 to 7. Telephone surveys/ message survey replies During and post- programme 84 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 41 % surveyed users who changed their dietary behaviours to healthier ones since the programme started Outcome Example questions: • Since the programme started, have you increased the amount of vegetables you eat? • Since the programme started, have you reduced the amount of red meat and meat products you eat? • Since the programme started, have you reduced the amount of fried and sugary foods and drinks you consume? • Since the programme started, have you reduced the amount of salt you consume? Telephone surveys/ message survey replies During and post- programme 42 % surveyed users who were smokers at the start of the programme who quit or reduced their intake of tobacco Outcome Example question: • Have you reduced the amount of tobacco you consume since commencing the programme? Telephone surveys/ message survey replies During and post- programme 43 % of surveyed users that reduced their consumption of alcohol over the course/since the beginning of the programme Outcome Example question: • Have you reduced the amount of alcohol you consume since commencing the programme? Telephone surveys/ message survey replies During and post- programme 44 % of surveyed users that report increasing their cognitive/social activity since the beginning of the programme Outcome Example questions: • Have you increased the amount of mentally challenging activities you have done this week (e.g. puzzles, learning to do something new)? • Since starting the programme, have you increased your social activities? Telephone surveys/ message survey replies During and post- programme 45 % of overweight users that report having lost weight since the beginning of the programme Outcome Example questions: • Since the programme started, have you lost weight? Telephone surveys/ message survey replies During and post- programme 85 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 46 % of surveyed users that report adhering to their medications (as prescribed) the majority of the time Outcome Example questions: • Did you take all your medications on time last week? Telephone surveys/ message survey replies During and post- programme 47 % of surveyed users who no longer meet the criteria for CVD risk factors Outcome/ impact This can be collected by the health facilities based on pre- post measurements of blood pressure, cholesterol and blood sugar levels. HCW Post- programme WELL-BEING AND HEALTH (mDementiaSupport) 48 % of surveyed carers who report they feel better supported to give care to the person with dementia Outcome Example questions: • Compared to the beginning of the programme/last month, do you feel more connected to the person you care for? • Compared to the beginning of the programme/last month do you feel more prepared to deal with the future, and any changes that may occur as the dementia progresses? • Compared to how you felt at the beginning of the programme/ last month, do you feel that you are better able to cope with the behaviour changes of the person you care for? Telephone surveys/ message survey replies During and post- programme 49 Improvement in scores on validated health and well- being scales Outcome The scales should cover the domains of carer burden, well-being, quality of life, depression, stress and/or self-efficacy. Telephone surveys/ message survey replies Post- programme BEHAVIOUR CHANGE mDementiaSupport 50 % of surveyed carers that changed their behaviours to maintain or improve their well-being Outcome Example questions: • Did you reach out to anyone for help in the past month? • Did you allow yourself some time to do pleasant activities in the past month? Reply 1 for YES or 2 for NO. Telephone surveys/ message survey replies During and post- programme 86 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY CARER PERCEPTION CHANGE 51 % of surveyed carers who have changed their views and perceptions about caregiving for the better since the beginning of the programme Outcome Example questions: • Compared to the beginning of the programme/last month, do you feel more prepared to support the person you care for in decision making? Reply 1 for YES or 2 for NO. • Compared to how you felt at the beginning of the programme/ last month, do you feel that you are better able to change your unhelpful thoughts into helpful ones? Telephone surveys/ message survey replies During and post- programme KNOWLEDGE GAINS 52 % of users who increased their quiz scores pre- and post- programme in knowledge quizzes (chatbots and apps) Outcome Example questions could include: • What is one way you can help prevent dementia? Telephone surveys/ message survey replies During and post- programme 53 % of surveyed mDementiaPrevention users that report having improved knowledge of dementia risk factors Outcome Example survey question: • After completing the programme you feel like you understand more about the factors associated with increased dementia risk? Telephone surveys/ message survey replies During and post- programme 54 % of surveyed mDementiaPrevention users that report having improved knowledge of strategies for dementia risk reduction Outcome Example survey question: • After completing the programme do you feel like you understand more about how to reduce your risk for dementia? Telephone surveys/ message survey replies During and post- programme 87 # INDICATOR LOGIC MODEL COMPONENT COMMENT DATA COLLECTION METHOD DATA COLLECTION FREQUENCY 55 % of surveyed mDementiaSupport users that report having improved knowledge of dementia Outcome Example survey question: • Compared to how you felt at the beginning of the programme/last month, do you feel like you understand more about dementia? Telephone surveys/ message survey replies During and post- programme 56 % of surveyed mDementiaSupport users that report having improved knowledge of what it means to be a carer Outcome Example survey question: • Compared to how you felt at the beginning of the programme, do you feel like you understand more about what it means to be a carer? Telephone surveys/ message survey replies During and post- programme 57 % of surveyed mDementiaSupport users that report having improved knowledge of what to expect in the person with dementia as the condition progresses Outcome Example survey question: • Compared to how you felt at the beginning of the programme, do you feel like you understand more about what to expect in the person with dementia as the condition progresses? Telephone surveys/ message survey replies During and post- programme 58 % of surveyed users that report having improved knowledge of available health services they can access Outcome Example survey question: • Since joining the programme, do you feel that you have more knowledge of the health and support services available to you? Telephone surveys/ message survey replies During and post- programme 88 ANNEX 3 BENEFITS AND RISKS OF DIFFERENT SOFTWARE MODELS5 MODEL mHEALTH EXAMPLE BENEFITS RISKS Custom- developed software (Build a software system from scratch) Custom-developed software: The extraordinary situation related to COVID-19 has led to numerous mobile phone apps being developed for it. • You have control over technology, functionality and design • Development creates ownership and improves sustainability • Local IT engagement • Custom development tends to be difficult to manage within timelines and budgets • Satisfaction is not guaranteed as the end product depends on the capabilities of the technical team • Long-term support depends on the continued availability of individuals Commercial off-the- shelf software (Buy a commercially available product) BlueStar, by Welldoc. The first mobile health product to secure reimbursement as a diabetes therapy, WellDoc's BlueStar is an FDA-approved tool for chronic disease management, particularly for Type 1 and Type 2 diabetes management. • Lead time from selection to implementation is shorter • You can evaluate it before buying • The product is maintained and upgraded (at a cost) • It has normally been tested and refined in other implementations • Often expensive and sold with unclear fee structures (e.g., fee per server processor) • Commercial off-the- shelf software is not often designed for implementation in low- resource settings Open-source software (The source code as well as the product is freely available, often with a supporting community) RapidPro SMS: UNICEF Innovation and Nyaruka created RapidPro in 2014, which is a suite of software designed to send and receive data using basic mobile phones, manage complex workflows, automate analysis and present data in real-time. RapidPro allows users to easily design, pilot, and scale SMS and IVR services that connect directly with a mobile phone user, without the help of a software developer. • You have the right to make changes to the software • You can engage the local IT industry • Benefit from communities and share development costs with other organizations • Can end up with a poorly supported product • A loosely knit community might not be able to provide the business relationship you need • Some of the implementation and running costs are not clear 5 This table, taken from PATH/WHO’s toolkit for planning an information system [please add a hyperlink: https://path.azureedge.net/media/documents/TS_opt_ict_toolkit. pdf] can help you to choose whether to use an existing solution, or to develop a customized one. 89 MODEL mHEALTH EXAMPLE BENEFITS RISKS Software as a Service (SaaS) (Database and application are hosted on remote servers and software is sold or offered as a contracted service) Software as a Service (SaaS): Flatiron Health supplies cancer-treatment centres with tools used by doctors for tracking patients’ progress and finding out what heals and what does not. Flatiron’s software allows medical cases to be shared with researchers to provide supplementary data. • Highly feasible to implement and maintain • Clarity about the cost to implement and run a SaaS application • Investment in improved software can easily be shared among customers • Data hosted on remote servers is not always permitted by national policy • Ministries of health are not often well positioned to pay a regular service fee ANNEX 4 THE ROLE OF AGGREGATORS Aggregators are a key part of the process for getting SMS messages to users. The process of which they are a part is set out in Figure A4.1. FIGURE A4.1: THE ROLE OF AGGREGATORS IN DELIVERING SMS MESSAGES mDementiaPrevention Physical Activity, Diet/Nutrition, Cognitive Activity Mid-life (40-65) Late-life (65+) Self-reported Memory ProblemsAt Risk High Cholesterol Memory ProblemsWeight SMS Management Aplication SMS Aggregator/ Gateway Company Mobile Network Operators (MNOs) Short Code Cost for Short code Could be negociated if for social purposes with Telecom Regulator Cost for aggregator per SMS (aggregators pay directly to operators) The deal can include the short code as well Need to establish agreement with aggregator Locally hosted Local aggregator Service provider (could be cloud hosted: local or international) Cost for service provider SMS Phone # Hypertension Diabetes Medication Use Tobacco Alcohol Healthy / No Risk Factors Healthy / No Risk FactorsAt Risk 1 2 4 3 90 1. SMS Management Application: An SMS management application is used to write and read messages, manage contacts and analyse data. This software can take the form of a website, a server, a desktop application or a small piece of software that is installed on a phone. 2. SMS aggregator/gateway company: Once messages have been written on the software platform, the messages need to reach the global phone network. The gateway that establishes that connection can either be a SIM card that is physically inserted into a device such as a phone or GSM modem, or an SMS aggregator. Aggregators are web-based services that specialize in sending messages to mobile phones globally. In many countries, they also provide their users with virtual local phone numbers to receive messages. Many SMS management applications ask their users to choose an aggregator depending on the country in which they operate, others have standing agreements with specific aggregators. All platforms can be connected with national mobile network operators in most countries, though that can be more labour intensive. 6 Maar MA et al. Unpacking the black box: a formative research approach to the development of theory-driven, evidence-based, and culturally safe text messages in mobile health interventions. JMIR mHealth and uHealth. 2016;4(1):e10. 3. Shortcodes: Contacts need to know where to send their messages. To make this as easy as possible it can be helpful to use a memorable numeric or alphanumeric shortcode such as “1234” or “REFAID” instead of a long phone number such as +232-123- 444 1122. The process to obtain shortcodes differs from country to country and involves a substantial additional cost in the range of hundreds of US dollars per month. Using a shortcode also always involves direct negotiations with a mobile network operator, which can delay the start of the programme. Mobile Network Operator (MNO): A mobile network operator or MNO, also known as a wireless service provider, wireless carrier, cellular company, or mobile network carrier, is a provider of wireless communications services that owns or controls all the elements necessary to deliver services to end-users, including radio spectrum allocation, wireless network infrastructure and other necessary components. ANNEX 5 MAIN THEMES AND OPERATIONALIZING STRATEGIES FOR MESSAGE DEVELOPMENT This annex shows the relationship between the main strategies, operationalizing strategies, and the affected behaviour change conditions that are helpful to keep in mind when composing the messages. Research shows that there are “mediators of meaning” when developing written messages, which go beyond the basic tailoring of messages to different cultural settings. These mediators help explain the hidden assumptions related to culture, health, and the health system, which may be interpreted through health communication messages. One study6 found that there are six main themes or factors that influence the level of congruence between the message content that researchers perceive to send versus the content that is actually perceived by the recipients. MAIN STRATEGIES BASED ON THEMES OPERATIONALIZING STRATEGIES FOR MESSAGE DEVELOPMENT BASED ON SUB-THEMES BEHAVIOUR CHANGE CONDITION AFFECTED Use positively framed advocacies (these are more persuasive); avoid negative or non-affirming framing of advocacies Empower and ease stress by pointing to successes. Inspire and show respect for receivers. Show compatibility with positive indigenous views of health as “living a good life”. Motivation 91 MAIN STRATEGIES BASED ON THEMES OPERATIONALIZING STRATEGIES FOR MESSAGE DEVELOPMENT BASED ON SUB-THEMES BEHAVIOUR CHANGE CONDITION AFFECTED Avoid fear- or stress-inducing messages Do not exacerbate people’s stressful lives (e.g., experience of low income or racism). Motivation Avoid oppressive or authoritarian messages Show respect for autonomy. Authoritarian messages may be perceived as lacking in respect; invoke historic distrust issues with colonial/medical system, and may cause defiant response. Provide healthy lifestyle education messages. Motivation Build on healthy cultural and traditional practices whenever possible; avoid incongruity with cultural and traditional practices Empower with a strengths-based approach to local culture. Show respect for culture. Capability, opportunity, motivation Recognize social determinants of health as drivers of ability to adopt behaviours; avoid disconnect with the reality of social determinants of health and the diversity of cultures within a population Consider cultural settings and cultural norms related to lifestyle. Understand affordability and accessibility of foods and medications. Consider access to providers and/or medications in the health care system. Capability, opportunity Ensure pragmatic content within the local setting; avoid lack of clarity and lack of practicality of content Preference for practical tips over higher level advice. Avoid ambiguity in wording and assumptions. Consider and check the local dialect in translation. Capability ANNEX 6 ADAPTING CONTENT LIBRARY FOR VOICE, MESSENGER APPS OR CHATBOTS Adapting to Voice Interactive voice response (IVR) enables you to reach those who may not be able to interact with text content. An actor can record the adapted messages in the BHBM content library and they can be delivered by in- or outbound phone calls to a smartphone or a basic phone. The call should be free. With IVR, the user can input a response using key words (Programme: “Have reached your walking goal today? Say YES or NO” and depending on the user’s answer, an appropriate pre-recorded response is given by the programme). This feature may be particularly useful if there are literacy or disability (e.g. visual) concerns within the target population. You can be more creative with voice messages (if appropriate) to engage the target users (again, they should be asked their preferences). For example, you could make a series of short 1–3 minute audio plays or stories in order to deliver the BHBM content library via the telephone. Actors could be recorded to be engaged in a discussion, using a number of scenarios (e.g., a doctor and a patient receiving information, advice or instruction, or other trusted community member 92 providing information or behaviour change strategies). However, when adapting content, make sure you always maintain the original intent of the message in the BHBM content library. Adapting to messenger apps A recent market research survey found that people check their phones for messages and notifications up to an average of 96 times a day.7 Messages through messenger apps (e.g. Messenger from Facebook, WhatsApp, WeChat) therefore provide an opportunity to get your messages read and noticed, as these apps are often used. Messenger apps also provide more freedom with the length of messages you can send and with the different media with which you can engage people, such as audio 7 Asurion-sponsored survey by Market Research Firm Solidea Solutions conducted 18–20 August 2019 of 1998 smartphone users in the USA, compared to an Asurion-spon- sored survey conducted by market research company OnePoll between 11–19 September 2017 of 2000 adults in the USA with a smartphone . files, images, GIFs and videos, or external web links or localization based suggestions or content (see Table A6.1 for multimedia dos and don’ts). They can be used to provide conversational agents, or used for lower level interactivity, on a par of that of SMS messaging. They are, however, restricted to reach users with higher income levels who have smartphones. Start from the BHBM content library and add more details to messages where you think the user could benefit from more clarity, preserving the scientific fidelity of the programme to the original library. We suggest that you ask target users what they would likely engage with in terms of multimedia. TABLE A6.1: MULTIMEDIA DOs AND DON’Ts MULTIMEDIA CONTENT DOs MULTIMEDIA CONTENT DON’Ts  Do ensure equal representation of men and women and different ethnic groups within your target population in all visual content  Don’t use stereotyped images of particular social groups or age groups  Do consider the file size and data usage costs for users, avoiding “heavy” files and compressing image files and video files if possible  Don’t use colours or gestures associated with a particular political or social group  Do use a variety of relevant regional accents if possible in audio materials  Don’t use complicated infographics, graphs or other visual representations of information. Keep them simple  Do try quizzes: they can be a fun way to engage, reward and collect data about knowledge gains and behaviour change  Be aware of accidental product placement in photographs or videos (e.g., a branded good in the background)  Do be creative with content, try to use visuals alongside education messages (e.g. illustration of a culturally appropriate healthy meal alongside a nutrition instruction about healthy meals) 93 Conversational agents and conversationalizing content If you wish to create a more interactive and tailored experience, a chatbot is one way to achieve this. It is necessary to consider the user experience of a chatbot as you design the conversational scripts for the bot. It is expensive, can be imprecise and less feasible to use a natural language processing (NLP) chatbot, because NLP incorporates an artificial intelligence system that has to be trained and maintained using lots of data. Having said this, experience with some WHO bots is that users have more conversational exchanges with an NLP bot. A conversational interface chatbot (CIC) is much easier to set up than an NLP chatbot. A CIC chatbot presents limited input options for users to select using buttons, emojis or typing the corresponding number or key word from a list of information topics. This results in the user navigating the bot and getting the tailored information they desire with less likelihood of the bot misunderstanding free-text inputs. Alternatively, there are hybrid bots that are predominantly CIC but have some basic NLP capabilities, which may lead to a better user experience. See Figure A5.1 for an example of a CIC (the WHO Health Alert service provided through Messenger for Facebook). Bots also carry the capability to provide links to external websites for further information, to send audio or image files, GIFs or videos, so you can be more creative with the content you provide and potentially retain more users. You should, however, be aware of the mobile data costs that such content carries for users. An important limitation of bots on certain platforms is that the user has to start the conversation each time. Messenger by Facebook is currently the only bot platform where the first message of a conversation (aside from initial enrolment) or a notification comes from the app, potentially leading to higher retention rates. CICs require a conversational script to be entered into the bot management software, the content of which can be adapted from the BHBM content library. It is however a good idea to start with the programme goals and knowledge-gain and behaviour change aims, then use the BHBM content library to base relevant messages on (see Box A5.1 for key steps in creating a chatbot). After you have designed the possible topics and conversations, you can make accompanying audio files, images, videos or links according to user preferences. Table A6.2 lists some dos and don’ts for creating bot conversations. FIGURE A6.1: SAMPLE NAVIGATION OF THE WHO COVID-19 HEALTH ALERT CHATBOT 94 BOX A6.1: STEPS FOR CIC CHATBOT CREATION The first step is to prepare the thematic outline of the CIC. This is an outline of the themes of each conversation that segmented groups should receive (information, questions, behaviours). The groups for segmenting will depend on your target users and the aims of the programme. It may be that you will segment users into youth, middle-adulthood and older people’s groups and by gender and disease-risk status (general population, at risk or diagnosed). For less sophisticated bots, this will likely be two or three questions at the beginning of each session, the responses to which will place the user into a particular segment or group to follow a given algorithm. (Some more expensive and sophisticated bots can remember and tag a user as in a specific group for all future sessions, but this creates a complicated back-end structure, rather like having many separate bots in fact, and may not be feasible.) The thematic bot outline could be in a table form or a diagram, whichever suits you and your team. It may be informed by the messages in BHBM content library, or you can start from scratch and map the key messages from the BHBM library back to it later. You can outline the different key messages you want users to receive in which week if you wish, or you may give users the choice what they wish to learn each session, so have all available at each chat. Secondly, create a bot diagram and write the messages according to the thematic bot structure. This diagram maps out the messages that are sent and received by the bot and is a pictorial representation of the algorithm for each chat session. Users may get fatigued with after 3–7 reply-bot exchanges, so keep chat sessions short and to the point. When creating a diagram, you could enter the actual messages or a code corresponding to another document with the full messages written (according to the need of the bot content management system). Careful maintenance of the script management spreadsheet will be very important. This will also be key for interaction with service providers and may have implications for cost. For example, if messages can be managed using a spreadsheet in advance, this can be used to obtain feedback on the project and approach a company with well-organized and structured content, and may result in them providing a lower quote than if they had to enter and provide a content management system. BHBM can provide assistance with this. This guidance is also relevant to NLP bots, however, a third party company will likely have to be used to set up the bot and train it to understand and process natural language inputs. 95 TABLE A6.2: CHATBOT CONVERSATION DOs AND DON’Ts. CHATBOT CONVERSATION DOs CHATBOT CONVERSATION DON’Ts  Do keep messages short and simple and to the point  Don’t use casual fillers like in normal speech as these can be misinterpreted  Do explain clearly how the user navigates (“select from the buttons below”, “type the number of your desired response”)  Don’t be too chatty  Do have the function to save the user’s progress so they can come back to the same point if they have to leave the conversation  Don’t overwhelm  Keep the goal of the bot in mind when writing the script  Don’t use humour unless you are sure it will be understood by all. It could create confusion  Be consistent with your voice and tone throughout the script and with tenses  Don’t request clarification on every input, just clarify for important questions such as screening questions for tailoring purposes  Do proofread and test your script for errors in the algorithm. It is essential that all content works smoothly  Avoid using emojis if multiple messenger apps will be used, as they can’t be recognized over all platforms  Make sure it is obvious when a chat session is over and how and when the user can next engage  Personalize either by writing all messages in first person or including a name or a mascot for the service (characterizing the speaker) Making the content library into an app There appears to be little component analysis of which health app features increase app effectiveness. However, there is some literature emerging that tries to determine what apps are effective, with some comment on their features. Apps should have an engaging design, and be very easy to use. All instructions should be clear and navigation should be effortless. User testing is vital to creating an app that will be used and that will retain its users. Table A5.3 contains suggestions for building a successful health app. 96 TABLE A6.3: APP CHARACTERISTICS LINKED TO POSITIVE USER RATINGS AND APP ENGAGEMENT FEATURES AND CONTENT THAT HAS A POSITIVE IMPACT ON USER RATINGS AND/OR INCREASED USE Content: includes reference to internal behaviour change drivers (such as motivation, self-efficacy, illness understanding and attribution (internal drivers)8 Content: includes reference to external behaviour change drivers (such as availability of information, the beliefs of peers and family, and the role of social networks)8 Strong evidence base and behaviour change theory Service user and professional input, especially at the design stage: ensures user compatibility and acceptability of app8 Clarity: factors such as ease and simplicity of use, specific instructions, features that save time, accessibility, relevant functions and clear security features are also important with high ratings in app stores9 Tangible and intangible rewards provided by the health app Social competition: seeing other people using the app and sharing behavioural data that could be compared to others on social networking sites and the ability to share personal information (sharing information with family or friends, leads to informational and emotional social support) Entertainment factors: a gaming element or other entertaining feature Trackers: tracking for awareness and progress (built-in feature to track the user’s activity, including diet, exercise, sensor-based automatic tracking) Goal-setting features: help users discipline themselves and slowly change their behaviours 8 Fitzgerald M, McClelland T. What makes a mobile app successful in supporting health behaviour change? Health Education Journal. 2017;76(3):373–381. 9 Mendiola MF, Kalnicki M, Lindenauer S. Valuable features in mobile health apps for patients and consumers: content analysis of apps and user ratings. JMIR mHealth and uHealth. 2015;3(2):e40. 97 ANNEX 7 DESIGN OF PAST EVALUATION STUDIES OF BHBM PROGRAMMES TYPE OF DESIGN DESCRIPTION Desk review Desk reviews are done to mainly evaluate the processes of the programme. Data can be collected from meeting reports, memoranda of understanding (MOUs) or terms of references (TORs) signed, or official records of implementation meetings. Focus group discussions or interviews with users Conducting focus group interviews with participants after the programme allows for in-depth exploration of experiences/satisfaction of, and knowledge gained from, the programme. Focus groups can be also be conducted among programme implementers to explore different interpretations of survey findings and brainstorming ideas for a programme’s improvement. The responses will be likely to be more detailed and thus more informative. Post-intervention surveys This is a survey that is carried out either over the telephone in a structured interview, online as a self-report, or as a series of questions and responses using the programme medium (SMS, messenger app etc.). One questionnaire is administered after the user has finished the programme. Without baseline data however, it is difficult to determine causality for outcome measures, which impacts negatively on the study’s quality, as there is no pre-programme information with which to make a comparison. This is likely, however, to be the cheapest way to gather evaluation data from users. Pre-post-studies (sometimes called repeated measures design) This is where a survey or other measurement (such as weight or blood pressure) is taken before a user starts the programme and then completes the same survey or measure after the programme. Pre-programme measures (baseline measures) are taken after registration but before the programme starts. This enables comparison between the baseline measures and post-measure in the same participant, meaning that evaluators are a bit more likely to be able to infer causality than via a post-intervention survey, for example. Prospective cohort studies This is where a study population is split into groups, for example, a group who receive messages (the intervention group) and a group that do not. A survey or other measurement is taken among both groups before (the baseline measure) and after the programme is completed by the intervention group, then the results from both groups are compared. The two groups can be matched on specific characteristics that may have an effect on the outcomes being measured, for example, recruiting persons in the same age range or those who are non-smokers, in order to reduce biasing factors. This is called cohort study matching. This leads to a more accurate interpretation of cause and effect (i.e. that it is the programme that gives rise to differences between the groups). The participants are usually followed up for a substantial amount of time to investigate the long-term effects of the intervention. Randomized controlled trials (RCTs) This research design is where participants are placed randomly into groups (e.g., programme and no programme) and measures are taken before and after the programme. The random group allocation should mean, if the population is large enough, that biasing characteristics of the participants should be equally present in both groups. Also, elements of the study are controlled, such as participant selection characteristics (for example, only participants who do not take medicines for cardiovascular diseases). This way, the differences measured pre- and post-programme are more likely to be attributable to the programme and not other factors. This, however, is very expensive to run and sometimes is not realistic when running scale programmes. While RCT designs are often considered the gold standard of scientific robustness, they may not be the most appropriate for evaluating scale programmes. They are expensive and their coverage will be limited. In past BHBM programmes they have been used to evaluate a pilot programme, providing robust proof of a concept, for example. 98 ANNEX 8 PROGRAMME MONITORING QUESTIONS mDementiaPrevention SMS or messenger app monitoring questions should be sent regularly during the programme (e.g. monthly). An example of how these questions can be integrated into the mDementiaPrevention programme is available on request as part of the message content/algorithm package. If needed, these questions can replace the post-programme questions, in particular in cases where drop-out may be high and participants may not reach the end of the programme. Countries can choose to provide a motivating two-way response message for user replies (optional two-way messaging response) if they choose (see Table A8.1). TABLE A8.1: SAMPLE TWO-WAY MONITORING MESSAGES FOR mDEMENTIAPREVENTION TOPIC EXAMPLE MESSAGES OPTIONAL TWO-WAY MESSAGING RESPONSE Medication use • Did you take your medications yesterday? Reply with 1 for Yes, 2 for No or 3 for not applicable • Did you take all your medications on time last week? Reply with 1 for Yes or 2 for No  If Yes – Congratulations, you are right on track. Make sure you take your medication exactly the way your doctor prescribed it.  If No – It is important to take your medication exactly the way your doctor prescribed it. Putting reminders on your phone can make it easier to remember to take your medication. Hypertension • Have you reduced the amount of salt you have eaten in the past week? Reply with 1 for Yes or 2 for No  If Yes – Great job, if possible, check your blood pressure regularly to make sure it is at a safe level.  If No – It is important to monitor your salt intake to protect your heart and health. Diabetes • Have you reduced the amount of sugary drinks and sweets you consume in the past week? Reply with 1 for Yes or 2 for No  If Yes – Great work. You are on your way to a healthier lifestyle.  If No – Changes aren’t easy. Keep going and stay motivated. Healthy eating habits are key to good health. High cholesterol • Have you reduced the amount of fried and fast food in your diet? Reply with 1 for Yes or 2 for No  If Yes – Great accomplishment. Keep up your healthy eating habits and exercise. Use our tips to help you maintain your cholesterol at a safe level.  If No – Step-by-step you will get there. It may help to identify behaviours that trigger unhealthy habits and avoid them. 99 TOPIC EXAMPLE MESSAGES OPTIONAL TWO-WAY MESSAGING RESPONSE Weight • Have you lost weight in the past month? Reply with 1 for Yes or 2 for No • Have you been regularly monitoring your weight over the past month? Reply with 1 for Yes or 2 for No  If Yes – This is a fantastic accomplishment. Well done! Keep up your good habits and make them routine.  If No – Changing habits is difficult. Congratulate yourself for even small changes. Take some time and reflect on the changes you made to improve your health. Healthy eating • Have you reduced your intake of red meat and meat products? Reply with 1 for Yes or 2 for No or 3 for not applicable • Have you increased your intake of green, leafy vegetables? Reply with 1 for Yes or 2 for No  If Yes – Well done. You are on the right track. Remember to keep healthy eating part of your daily routine.  If No – Try to find creative ways to overcome barriers that stop you from eating healthily. Why not talk to someone about it and see if they have ideas to help? Try out strategies that fit with you and your daily routines. Physical activity • Did you do at least 2.5 hrs of exercise last week? Reply with 1 for Yes or 2 for No • In the past week, on how many days did you do physical activity for at least 30 minutes? Reply with a number from 0 to 7  If Yes (or replied with 4–7) – You should feel proud. Keep it up! Physical activity is good for your heart and mind.  If No (or replied with 0–3) – Sometimes it takes a few tries to reach a goal. If you are having trouble with 30 minutes a day, try starting with 10 minutes of exercise a day. Tobacco cessation • Have you reduced the amount of tobacco you consume this week? Reply with 1 for Yes or 2 for No • Have you quit smoking or using tobacco? Reply with 1 for Yes or 2 for No • Have you been smoke- or tobacco-free for the past 7 days? Reply with 1 for Yes or 2 for No • Have you been smoke or tobacco-free for the past 30 days? Reply with 1 for Yes or 2 for No  If Yes – It feels great to achieve your goals. Keep it up! If you feel like using tobacco, go to the gym, take a jog, or walk the dog instead.  If No - Concentrate on the positives you receive from the new lifestyle; do not dwell on what you are giving up. You can achieve your goal Harmful use of alcohol • Have you reduced the amount of alcohol you consume over the past week? Reply with 1 for Yes or 2 for No • Have you limited your alcohol intake to 1 drink or less per day? Reply with 1 for Yes or 2 for No  If Yes – This is a great accomplishment. Give yourself a pat on the back. Keep it up and reap the benefits of a healthy lifestyle.  If No – It often takes a few tries to reach your goals. It may help to avoid situations that trigger a change in your healthy lifestyle. 100 TOPIC EXAMPLE MESSAGES OPTIONAL TWO-WAY MESSAGING RESPONSE Cognitive activity • Have you increased the amount of mentally challenging activities you have done this week (e.g. puzzles, learning how to do something new)? • Have you read a book, newspaper or magazine this week? • Have you solved a puzzle, or played a card or board game this week? • Have you attempted to learn a new skill this week?  If yes – Great work! Keep doing new activities and keep that mind active  If no – It can be hard to find time or motivation to take up a new activity. Try working together with a friend. You can challenge and support each other while having fun! mDEMENTIASUPPORT Monitoring questions should be sent regularly during the programme (e.g. monthly). An example of how these questions can be integrated into the mDementiaSupport programme is available upon request as part of the message content/algorithm package. Countries can choose to provide an open-ended response (optional two- way messaging response) to gather more information about why the programme may not be working (see table on page 102). Follow-up phone calls (with participant consent) may replace message responses if preferred. Likert scales (a scale of 1–5) could also be used in the place of yes/no response options to obtain more detailed feedback (see Table A7.2 for sample messages). 101 TABLE A8.2: SAMPLE TWO-WAY MONITORING MESSAGES FOR mDEMENTIASUPPORT TOPIC EXAMPLE MESSAGES OPTIONAL TWO-WAY MESSAGING RESPONSE Introduction to dementia • Compared to how you felt at the beginning of the programme/last month, do you feel like you understand more about dementia? Reply 1 for YES or 2 for NO  If Yes – That’s great, if you want to find out more information you can look for [local resources].  If No –To help us improve the programme, please tell us why you answered no by responding to this text. Being a carer • Did you reach out to anyone for help in the past month? Reply 1 for YES or 2 for NO • Did you receive any help in the past month? Reply 1 for YES or 2 for NO  If Yes – It is very good that you are involving others. Caregiving needn’t be lonely.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. • Compared to the beginning of the programme/last month, do you feel more connected to the person you care for? Reply 1 for YES or 2 for NO  If Yes – That’s great, it’s important to keep communicating with each other as the disease progresses.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. • Compared to the beginning of the programme/last month do you feel more prepared to deal with the future and changes that may occur as the disease progresses? Reply 1 for YES or 2 for NO  If Yes – We’re glad to hear that. Don’t forget to reach out to family and friends for help if you need it.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. • Compared to the beginning of the programme/last month, do you feel more prepared to support the person you care for in decision-making? Reply 1 for YES or 2 for NO  If Yes – That’s great to hear. As the disease progresses, keep communicating and be prepared to adapt the amount of support you provide.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. Caring for me • Did you allow yourself some time to do pleasant activities in the past month? Reply 1 for YES or 2 for NO  If Yes – Great work. You are on your way to a healthier lifestyle.  If No – Changes aren’t easy. Keep going and stay motivated. Healthy eating habits are key to good health. 102 TOPIC EXAMPLE MESSAGES OPTIONAL TWO-WAY MESSAGING RESPONSE Caring for me • Compared to how you felt at the beginning of the programme/last month do you feel more, less or about the same levels of stress? Text MORE, LESS or SAME  MORE – You may feel more stressed as the disease progresses. Remember to ask for help and take time to take care of yourself. Text SELFCARE for tips.  LESS – That’s wonderful to hear. If you ever need tips on how to relax text SELFCARE.  SAME – Some days are better than others. If you ever need tips on how to relax text SELFCARE. • Compared to how you felt at the beginning of the programme/last month, do you feel that you are better able to change your unhelpful thoughts into helpful ones? Reply 1 for YES or 2 for NO  If Yes – That is excellent news. Always remember, helpful thoughts are important to feel better and will assist you in dealing with situations in a more useful way.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. Providing everyday care • Compared to how you felt at the beginning of the programme/last month, do you feel like you can communicate better with the person with dementia? Reply 1 for YES or 2 for NO  If Yes – That’s fantastic. Dementia can often make communication difficult. As the disease progresses, keep in mind you may need to adapt your communication style.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. • Compared to how you felt at the beginning of the programme/last month, do you feel more prepared to support the person in everyday activities? Reply 1 for YES or 2 for NO  If Yes – That’s great to hear. Text PERSONAL if you ever need tips on how to better support the person you care for with everyday activities.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. • Compared to how you felt at the beginning of the programme/last month, do you feel more prepared to adapt your routine as the disease progresses? Reply 1 for YES or 2 for NO  If Yes – Well done. It is important to adapt routines as the abilities of the person living with dementia change.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. Dealing with behaviour changes • Compared to how you felt at the beginning of the programme/last month do you feel that you are better able to cope with the behaviour changes of the person you care for? Reply 1 for YES or 2 for NO  If Yes – That’s great! Remember different approaches work at different times and some days may be better or worse than others.  If No – To help us improve the programme, please tell us why you answered no by responding to this text. 103 World Health Organization 20 Avenue Appia 1211 Genève 27 Switzerland International Telecommunication Union Place des Nations CH-1211 Geneva 20 Switzerland website: www.who.int/initiatives/behealthy e-mail: bhbm@who.in e-mail: mHealth@itu.int