A GUIDE TO IMPLEMENTATION RESEARCH ON IMMUNIZATION FOR PROGRAMME MANAGERS IN THE WHO AFRICAN REGION
2
CONTENTS
CONTENTS ............................................................................................................................................... 2 LIST OF TABLES ......................................................................................................................................... 4 LIST OF FIGURES ....................................................................................................................................... 4 ABBREVIATIONS ....................................................................................................................................... 5 FOREWORD .............................................................................................................................................. 7 1. INTRODUCTION AND CONTEXT........................................................................................................ 9 1.1 1.2 1.3 1.4 1.5 1.6 BACKGROUND ................................................................................................................................... 9 WHAT IS IMPLEMENTATION RESEARCH? ............................................................................................... 11 WHY IS IMPLEMENTATION RESEARCH NEEDED? ...................................................................................... 13 WHO SHOULD BE INVOLVED IN IMPLEMENTATION RESEARCH? .................................................................. 14 HOW IS IMPLEMENTATION RESEARCH USED? ......................................................................................... 15 TARGET AUDIENCE ........................................................................................................................... 16
2. GETTING INVOLVED IN IMPLEMENTATION RESEARCH ON IMMUNIZATION ....................................... 17 2.1 2.2 2.3 2.4 2.5 SELECTING A PRIORITY IMPLEMENTATION RESEARCH TOPIC IN IMMUNIZATION .............................................. 17 SPECIFIC PRIORITY RESEARCH FROM THE REGIONAL PERSPECTIVE ................................................................ 21 REGIONAL OVERVIEW OF PREVIOUS IMPLEMENTATION RESEARCH ON IMMUNIZATION .................................... 25 CENTRES CONDUCTING RESEARCH ....................................................................................................... 26 FUNDING OF IMPLEMENTATION RESEARCH ............................................................................................ 27
3. DOING IT YOURSELF: THE IMPLEMENTATION RESEARCH PROCESSES ................................................. 28 3.1 3.2 3.3 3.4 3.5 3.6 3.7 CONTEXTUALIZING IMPLEMENTATION RESEARCH ISSUES ........................................................................... 29 DEVELOPING AN IMPLEMENTATION RESEARCH PROPOSAL ......................................................................... 30 PLANNING TO CONDUCT IMPLEMENTATION RESEARCH ............................................................................. 36 DATA ANALYSIS AND PRESENTATION .................................................................................................... 37 COMMUNICATING THE RESEARCH FINDINGS TO THE HEALTH SYSTEM .......................................................... 38 MONITORING AND EVALUATING THE PROJECT ........................................................................................ 39 ADDITIONAL ACTIVITIES ...................................................................................................................... 39
REFERENCES ........................................................................................................................................... 41
3
List of tables
TABLE 1: AVAILABILITY OF VACCINES IN THE COUNTRIES OF THE WHO AFRICAN REGION FROM THE START OF THE EXPANDED PROGRAMME ON IMMUNIZATION TO 2010 ............................................................................................. 11 TABLE 2: POTENTIAL TOPICS FOR IMPLEMENTATION RESEARCH ON IMMUNIZATION IN THE AFRICAN REGION ....................... 23 TABLE 3: STEPS IN THE DEVELOPMENT OF AN IMPLEMENTATION RESEARCH PROPOSAL .................................................... 31 TABLE 4: FEATURES OF SOME RESEARCH TOOLS ....................................................................................................... 33
List of Figures
FIG. 1: THE SIX PROCESSES OF IMPLEMENTATION RESEARCH ........................................................................ 29
4
ABBREVIATIONS
AII BCG CDC
Accelerated Immunization Initiatives bacillus calmette-guerin Centers for Disease Control and Prevention
Conj. Men A meningococcal a meningitis conjugate vaccine DPT3 DPT EPI ESA FG GAVI GRIPP GVAP HCW HepB Hib HPV IM IR IST IVE JRF MCV1 MCV2 third dose of diphtheria, pertussis and tetanus vaccine diphtheria, pertussis, tetanus Expanded Programme of Immunization East and Southern Africa Focus Group Global Alliance for Vaccines Initiative Getting Research into Policy and Practice Global Vaccine Action Plan Health Care Worker hepatitis B vaccine Haemophilus influenzae type B vaccine Human papillomavirus Independent Monitoring Implementation Research Inter-country Support Team Immunization, Vaccines and Emergencies Joint Reporting Form first dose in routine immunization of measles-containing vaccine second dose in routine immunization of measles-containing vaccine 5
MNT NIPs OPV PCV PEI TFI RI RIN SAGE SIA SOP TDR TT TAG YF
Maternal and Neonatal Tetanus National Immunization Programmes Oral Polio Vaccine Pneumococcal Conjugate Vaccine Polio Eradication Initiative Task Force on Immunization Routine Immunization Routine Immunization and New vaccines Strategic Group of Experts Supplementary Immunization Activity Standard Operating Procedure Tropical Diseases Research Tetanus Toxoid Technical Advisory Group yellow fever vaccine
6
Foreword Immunization is considered as one of the most cost-effective public health interventions. Yet morbidity and mortality from vaccine-preventable diseases still remain unacceptably high in the African Region owing to suboptimal immunization services delivery. The factors responsible for the inability to utilize proven and effective vaccines still pose considerable challenges to both those delivering vaccination services and policy-makers. Interest in implementation research on immunization is growing, largely in recognition of the contribution the research can make in maximizing the beneficial impacts of immunization services. Immunization implementation research should identify the challenges that countries confront in immunization to enable the development and use of evidence-based policies and practices. Unfortunately, the essential skills to conduct this type of research are limited in the African setting, and public health academic curricula rarely provide courses in this area. There is a need, therefore, for clarity about what implementation research exactly is and what it can offer, in order to stimulate its usage in enhancing the delivery of immunization services. The starting point for implementation research is the identification of a challenge or question requiring solutions. Such questions very often come from those who face problems in the field, the programme managers. These are the people for whom this guide is primarily intended, to provide them clarity on the application of implementation research to immunization. The aim of this guide is not to create front-line investigators out of programme managers, rather it is envisaged that their interest will be stimulated to use and support the use of implementation research as a tool to generate solutions to the challenges they face in delivering immunization services. Given that this guide’s intended primary audience range widely in qualifications and operate in a variety of environments, it is purposefully designed to be simple. It has three parts. The first section provides the background on what implementation research is, why it is needed, how it can be used to strengthen immunization service delivery, how it differs from the regular research, and its basic concepts. The second section is an 7
overview with examples on how priority topics for implementation research have been selected in other settings, but highlighting the fact that priorities differ according to setting. It also deals with how priority research topics could be derived from specific challenges in a setting. The third section describes the methodological approaches for selecting an implementation research question and developing a proposal, as well as how to conduct the research and disseminate the findings.
8
1. Introduction and context
1.1 Background The Immunization, Vaccines and Emergencies (IVE) Cluster of the World Health Organization’s (WHO) Regional Office for Africa aims to provide policy and programmatic direction for Member States to optimize immunization services, including in emergencies, while accelerating efforts for the eradication of polio and elimination of measles, maternal and neonatal tetanus and epidemics of meningococcal A meningitis. The activities of the Cluster also aim to strengthen surveillance for vaccine preventable diseases (VPDs), promote integration of services to maximize access to vaccines, and strengthen immunization systems in the countries, including vaccine regulation and safety. These objectives are defined by the Cluster’s strategic plan, the African Region’s strategic objectives and core capacities and the Global Immunization Vision and Strategy (GIVS) 2006–2015. In addition, recommendations from the Strategic Advisory Group of Experts (SAGE), the Task Force on Immunization (TFI) and various technical advisory groups (TAGs) guide IVE Cluster’s strategic plans and implementation, monitoring and evaluation activities. The IVE Cluster runs four major programmes: the Polio Programme, the Routine Immunization Programme, the Outbreak and Disaster Management Programme and the Accelerated Immunization Initiative. The Polio Programme works closely with the global Polio Eradication Initiative (PEI) to help the countries to interrupt the transmission of wild poliovirus and to eradicate the disease. The Routine Immunization Programme supports the countries to ensure that the regular vaccines reach every child to achieve the desired high immunization coverage and herd immunity. It also provides guidance on the introduction of new vaccines into the existing vaccination schedule. The Accelerated Immunization Initiative ensures that the countries receive the technical support required to enable them to eliminate measles, maternal and neonatal tetanus and yellow fever. The Outbreak and Disaster Management Programme facilitates provision of support and guidance to countries during humanitarian emergencies.
9
Immunization coverage is a major indicator of the IVE Cluster’s performance. Traditionally, coverage of the third dose of diphtheria, pertussis and tetanus (DPT3) vaccine has been used as a surrogate for the overall immunization coverage. DPT3 coverage has risen steadily over the years in the WHO African Region, reaching a national average of about 80%. However, maintaining the vaccination coverage at the GIVS target levels of 90% at the national level and at least 80% at the level of the district or the equivalent administrative unit has remained elusive for many countries, while for others even reaching these targets has not been possible. The low vaccine coverage in some countries has seen large numbers of children remain unimmunized, with the attendant negative impact on the goals for vaccine-preventable diseases, in particular polio eradication and elimination of measles and maternal and neonatal tetanus. The rich pipeline of new immunization products and technologies has not had a big impact in the African Region. The past biennium alone has witnessed an unprecedented introduction of four new vaccines, that is the vaccines against pneumococcal pneumonia, meningococcal A meningitis, rotavirus diarrhoea and cervical cancer due to human papillomavirus. These have been tested in the Region, licensed and prequalified by WHO and are now being registered and increasingly being introduced in the African Region (Table 1). In addition, the countries have frequently undertaken supplementary immunization activities (SIAs) in line with the global and regional declarations and resolutions they have endorsed for the eradication of polio, elimination of measles and maternal and neonatal tetanus, and control of epidemic meningitis and yellow fever. These activities mean that the immunization programmes in the countries now also have to cope with the challenges posed by the introduction of new vaccines and conducting of SIAs as well as trying to attain and maintain wide coverage with the existing vaccines for routine immunization. This places a considerable burden on the health systems in general and the immunization systems in particular and poses not only technical challenges but also managerial, systems, socio-behavioural, financial and communication bottlenecks. Implementation research on immunization has the capacity to identify these challenges to enable the development and use of evidence-based policies and practices, be it to introduce a new vaccine or to scale up coverage of existing vaccines in national immunization programmes. 10
Table 1: Availability of vaccines in the countries of the WHO African Region from the start of the Expanded Programme on Immunization to 2010
Vaccine Year available BCG, DPT, OPV, TT (traditional vaccines) 1974 Measles 1974 YF 2000 (before 2000 in few countries) HepB 2000 (before 2000 in few countries) Hib 2000 *HPV 2008 *PCV 2009 *Rotavirus 2009 *Conjugate meningococcal A 2010 *Became available to immunization programmes of the WHO African Region within the last 4–5 years.
At its meeting in Windhoek, Namibia, in 2011, TFI recognized that (1) the immunization coverage rates in the Region were generally suboptimal, (2) the multiplicity of simultaneous immunization activities and goals in an environment of weak health systems may pose problems that hinder the efficiency of vaccine delivery, and (3) research for health had been and will continue to be the tool that provides solutions that improve delivery of health care services, including immunization. TFI recommended that the IVE Cluster provide guidelines on implementation research to support the countries to find solutions that will improve the efficiency of immunization services delivery. 1.2 What is implementation research? Implementation research is a growing field, and several definitions have been put forward for it. A very simple and broad definition terms implementation research as the scientific inquiry into questions concerning implementation.2 It has also been described as a scientific study producing practically usable knowledge in the form of evidence, findings, information etc. that can improve programme implementation outcomes such as effectiveness, efficiency, quality, access, scaling up and sustainability regardless of the type of research design or methodology used. In this guide implementation research is defined as that area of research devoted to understanding the processes for implementing health initiatives and the bottlenecks in introducing and scaling up proven public health interventions and finding practical solutions to overcome these barriers. Implementation research identifies implementation barriers, develops optimal approaches for 11
implementing interventions and feeds back the research findings into the health system in a particular setting to improve health care delivery . The basic intent of implementation research is to understand not only what is or is not working but also how and why implementation is going right or wrong and modifying the approaches in order to improve it. The focus of implementation research is the context and the interaction between the real world and the intervention being studied, which is what sets it apart from, say, monitoring and evaluation of regular research. The goal is to make a difference that is to improve the effectiveness, quality, efficiency and equity of policies, programmes and services. It requires partnerships and multidisciplinary teams to be able to tap into various resources and bring to bear multiple perspectives and offer multisectoral insights into the research. Implementation research involves more than just scientists and includes policy-makers, the media and community members in identifying operational problems, planning research to address the problems and utilizing the research outcomes.
Implementation research differs from conventional research in regard to the origination of the research problem and the involvement of end-users in the research process. It provides information that is easily generalizable so that it can be applied across various settings, it assesses efforts by health care providers to improve service delivery and it provides practical solutions that facilitate changes in policies so that they are easily taken up by the end-users.
The need to address implementation barriers and improve delivery of health services is often greatest in settings where health systems are the weakest or nonexistent such as many locations in the African Region. Despite the importance of implementation research, it continues to be a neglected field of study, partly because of the lack of understanding of what it is and what it can offer and partly due to the lack of investment in implementation research activities. Moreover, the essential skills to conduct this type of research are limited in the African setting, and public health curricula in academic institutions rarely offer courses in this area. This guide aims to address these gaps so that 12
programme managers get a better understanding of the subject and its capacity to facilitate the delivery of immunization services. It is hoped that they will take greater interest in the subject, recognizing it as an integral part of programme planning and execution. A key message is that contributions to implementation research can come from people in or out of academia. Indeed, very often it is the person in the field who is facing particular problems who asks the questions that become the starting point for new thinking. Making sure that those questions are heard and that the appropriate research is undertaken that is directed at finding answers to them rather than the topics that researchers themselves may find interesting are crucial characteristics of implementation research. 1.3 Why is implementation research needed? Research in health is an activity that is undertaken to generate new knowledge that will eventually be used to reduce the disease burden or other health-related problem in the human population.1 Research can have one or several goals such as creation of new knowledge, validation of knowledge, transformation of knowledge into best practices, development of indicators to measure impact on health status or identification of implementation gaps and development of interventions to fill them. It is prudent to always consider how knowledge from your programme’s activities, available data and implementation barriers can be analysed to provide valuable evidence for public health use. One of the greatest challenges facing the global health community and particularly the field of immunization is how to implement proven interventions in the real-world situation. The science of implementation and dissemination comprises a set of multidisciplinary methods aimed at improving this process of translating research evidence into everyday health-related practices. Implementation research is part of implementation science. It particularly examines how interventions can be best integrated into diverse practice settings and emphasizes direct engagement with the institutions and communities where the health interventions will be introduced. The concept of implementation science becomes extremely relevant in efforts to develop immunization
13
strategies, including the introduction for new vaccines, integration of other services into immunization programmes or vice versa, and undertaking of the frequent supplementary immunization campaigns that often face a range of challenges that limit their impact. Implementation research can play a pivotal role in improving understanding of these challenges, which will improve the delivery of the immunization services. Without implementation research and the knowledge it generates, we are at best hoping that the valuable resources we commit will help things work out instead of applying research knowledge for evidence-base decision-making. 1.4 Who should be involved in implementation research? Implementation research is predicated on the fact that very often health interventions, technologies, programmes or strategies that have proved efficacious in well-structured settings may not be as effective within routine health systems or real-life settings. It seeks answers to questions such as Why do people refuse vaccination but not other health interventions? Why has immunization coverage in the African Region plateaued at about 70% over the past two years? and How can immunization services reach mobile populations such as nomadic communities? Answers to these questions could lie in factors related to the environment, socioeconomic or cultural issues, or health systems and user characteristics. Implementation research provides a systematic way of identifying these factors and adapting implementation processes to optimize the outcomes from the strategies or interventions. Implementation researchers, therefore, need to bear in mind the needs and limitations of their intended audience. This means that in most cases the audience for this research is not another researcher but a non-specialist in need of clear, evidence-based analysis of field data that might form the basis of a decision. Implementation research is most likely going to be useful where the implementers of immunization services have contributed to the identification, design and conduct of the research undertaken. It is important, therefore, for researchers and implementers to recognize the value of coming together in this process whereby the implementers generate research questions from the field while the researchers provide the expertise in research methods needed for trustworthy studies.
14
1.5 How is implementation research used?
1.5.1 Understanding the research context
Implementation research helps in understanding of the contextual factors that can influence the outcomes of interventions. This is because the same intervention may be implemented in several ways with different outcomes. Implementation research is required to identify the sociocultural, behavioural or other factors that are detrimental to the implementation of interventions and to device approaches to circumvent these barriers while the intervention is still being delivered.
1.5.2 Assessing the performance of health interventions
Implementation research can be used to gauge the performance of immunization services over time and to serve as the basis for projections into the future. Where appropriate, it can also be used to look at the way similar interventions or programmes have performed elsewhere, noting the disparities in the outcomes.
5.1.3 Supporting quality improvement and health systems strengthening
The core concern in implementation research is to ask questions that are relevant to the challenge faced. Implementation research can yield many benefits but those benefits are maximized only where research is answering the questions that decision-makers and practitioners are asking or should be asking. This document aims to:
Promote implementation research as an important tool for strengthening immunization systems in particular and health systems general;
Highlight identified priority immunization research areas and suggest the related types of studies that could be conducted; and
Provide the guiding principles and general guidance on implementation research in support of immunization.
15
1.6
Target audience
This document provides a simplified overview of implementation research on immunization, and it is intended for policy-makers and programme managers. It is hoped that knowledge on implementation research will stimulate interest in its use as a tool to provide solutions to the challenges in the delivery of immunization services, and subsequently in strengthening health systems in general. Furthermore, a good understanding of this overview might allow programme managers to conduct simple implementation research projects directly or to follow the processes when independent researchers are engaged to address more complex research questions. This document is intentionally simplified. Only the key concepts of implementation research are highlighted, but the technical details of processes that are common to both implementation and regular research, which can be found in standard research textbooks, are not addressed fully in this document.
16
2. Getting involved in implementation research on immunization
2.1 Selecting a priority implementation research topic in immunization The goals in selecting a priority implementation research topic immunization must be to drive immunization policy, maximize the impact of vaccines and immunization, and help define and implement a process to facilitate efforts to generate relevant and credible evidence to support decision-making for national immunization programmes. Two key parameters are useful in selecting candidate research issues, the first of which is that the research will improve the coverage of recommended vaccines and facilitate introduction of newly licensed or near-to-licensing vaccines instead of vaccines in early development, and the second of which is that there should be a focus on research that identifies immunization barriers as well as that tests attractive strategies to overcome the barriers. Some of the guiding principles in the selection of research topics include: Research that strives for equity in the immunization process and the dissemination of results within the research setting and beyond; Topics addressing integration of immunization with other health services; Research that finds new ways not only of doing things but also of improving the current ways in which things are done; and Research questions that are addressable within a 5-year time frame.
2.1.1 Domains of implementation research on immunization
In general, immunization implementation research topics can be categorized into eight domains: Immunization and coverage Social determinants of vaccination Health and immunization systems
17
Cold chain and logistics management Vaccine product profiles Programme management Programme monitoring and impact assessment Research capacity and application
2.1.2 Potential research questions and methods for immunization implementation research according to domain
Immunization and vaccine coverage
Research question: How should vaccination services be adapted to increase coverage in general and for specific or special groups such as mobile populations or special circumstances such as administration of the second dose of the oral cholera vaccine in epidemics? Method: Concurrent, non-randomized cluster trials of various interventions implemented in certain locations but not in others, according to the specifics of the population. Research study example: Home intake of the second dose of the cholera vaccine with the vaccine distributed when the first dose is administered under the supervision of a health facility or private clinics. Involvement of other government departments such as the ministry of agriculture in the planning of immunization is required in order to track and vaccinate nomadic populations. Social determinants of vaccination
Research question: What are the profiles of the high risk communities such as their gender, socioeconomic factors etc., how do these profiles influence vaccine uptake and what are the best strategies to identify and vaccinate such groups?
18
Methods: Qualitative methods to define the profiles and identify the reasons that individuals and communities refuse to be vaccinated. This will require a deviance study using case studies, focus group discussions and key informant interviews. Health and immunization systems
Research question: What are the specific barriers to immunization among children not reached by immunization services and what are the service delivery strategies to respond effectively to these barriers? Methods: Qualitative methods comparing areas in which children are reached with those where they are not reached case studies, key informant interviews and focus group discussions. Cold chain and logistics management
Research question: What would be an effective “common operating platform” for the integrated logistics management of all immunization services activities at the district level, including the certification of a trained logistics manager to lead the activity? Methods: Mixed methods of both qualitative and quantitative inquiry for the various activities with convergence of data and analysis. Repeated measuring of the variables relating the context, actors and depth and breadth of intervention implementation level across the subunits will be needed followed by harmonization/integration of the logistic factors. Vaccine product profile
Research question: What is the feasibility of establishing dynamic and linked subnational and national vaccine databases on to promote vaccine safety? Methods: A mixture of methods of qualitative and quantitative inquiry with convergence of data and analysis. Assessment of the awareness and routine and regular reporting of vaccine safety issues will be needed. An assessment of the existence of the required technical and structural infrastructure for collection and collation of data locally and
19
nationally will be undertaken. Key informant interviews of relevant personnel in the ministry of health will be required. Programme management
Research question: What tools do health workers need to effectively assess and correct missed opportunities for children whose immunization is delayed or whose schedules have been interrupted? Methods: A mixture of qualitative and quantitative inquiry. Quantitative assessment will determine the number and proportions of children not vaccinated or whose vaccination is delayed, while structured questionnaires, in-depth and key informant interviews and focus group discussions will be used to identify the reasons for missed opportunities. Programme monitoring and impact assessment
Research question: Can improvements be made to the methods for estimating the size of the target population at national and subnational levels in order to obtain reliable data? Methods: Quantitative methods using modelling, simulation, forecasting modelling, data extrapolation and sensitivity analysis. Research capacity and application
Research questions: How frequently is evidence-based information used to support recommendations from NITAGs or similar bodies? What are the barriers to the effective functioning of NITAGs and how can they be addressed to strengthen these institutions and vaccine policy-making at the country level? Methods: A mixture of qualitative and quantitative inquiry. This will entail a retrospective and quantitative assessment of the evidence base on previous recommendations and policies. Qualitative methods will involve key informant interviews of major actors on issues relating to the decisions gleaned from the evidence base. Given that it will not be possible to conduct all the research at one time, there will need for prioritization or research based on criteria such as these: 20
Appropriateness – should we do it? Relevance – why should we do it? Chances of success – can we do it? Impact of the research outcomes – what benefit will be achieved?
2.2 Specific priority research from the regional perspective In addition to the processes of identifying implementation research on immunization priorities by domain, a number of research questions that have been posed by some strategic bodies and immunization services providers are relevant. The examples provided here are not exhaustive and might not be relevant to your setting or country, but they provide an overview of the diversity of research problems. The emphasis remains that countries should identify research topics and priorities that are peculiar to their needs even if those are not captured in the list below. 2.2.1 Recommendations from advisory groups
TAGs and SAGE in their various meetings have identified important implementation research topics or questions some of which are yet to be addressed. These include: What are the causes of failure to be vaccinated in the African Region?3 How can progress towards the goals for hepatitis B control be monitored through serological surveys of the prevalence of hepatitis B surface antigen (HBsAg) in a population?3 How can data from subnational-level analysis of the reasons for the failure to be vaccinated or for under-vaccination in the countries with low routine immunization coverage be assessed?4
How can the good examples from health systems strengthening by immunization programmes and research of vaccine hesitancy be best documented?5
21
2.2.2 Research topics from survey of staff in regional and global immunization programmes
Contributions on topics for potential implementation research were solicited also from immunization staff from WHO headquarters, regional and sub-regional offices and immunization programmes in some countries. The commonly suggested areas for implementation research included the use of high quality and reproducible data on immunization for planning implementation of immunization activities; identification of a set of parameters that best defines an effective and efficient national immunization programme; sociobehavioural factors that affect immunization, including community perceptions and attitudes; and the major barriers to attaining of vaccine coverage levels higher than 90% during SIAs. Other suggestions were on assessment of supervisory practices and capacity for immunization; vaccine refusal by parents, communities or faith-based organizations and the effect on routine immunization service delivery; identification of the main incentives or disincentives for vaccinators during SIAs; identification of social mobilization activities in immunization; the impact of new vaccine introduction on government expenditure as spending is shifted from other areas to meet the cost of these more expensive vaccines; and the quality of interactions between service providers and parents or guardians and how this affects community involvement in immunization and vaccination coverage. These research topics or questions are broad and their importance will vary by country and setting, so it is important for the countries to carry out their own situational analysis to arrive at their priority topics.
22
Table 2: Potential topics for implementation research on immunization in the African Region Title Investigating awareness on acute flaccid paralysis (AFP) of health workers and retraining them Brief justification and objective Aside from non-vaccination of children, the persistence of polio has partly been attributed to gaps in surveillance. Sensitive AFP surveillance is necessary to guide the interventions. A number of health care works especially in private hospitals are not aware of the AFP surveillance procedures. Given that a significant proportion of patients go to such clinics, the inability to detect, investigate or report AFP on time will be a major fault in PEI. This study aims to assess AFP awareness and to follow that with training of health care workers in public and private hospitals on AFP reporting. The ultimate aim of both routine and supplementary immunization activities is to induce protective anti-polio antibody levels in a high proportion of the population. The failure to break polio transmission might be due to either low vaccination coverage or inability to induce lasting protective antibody levels in a community. Given the issues associated with coverage data, a sero-survey would be the authentic and objective way of assessing herd immunity as well as validating the coverage data. Therefore, a cross sectional sero-survey among children both under and above 5 years of age is proposed. The factors that affect routine and supplementary immunization are likely to be related, as coverage rates for both are suboptimal. To be effective, communitybased interventions such as immunization require community participation and committed health workers. Rewarded competition of communities to raise coverage levels has been successful elsewhere. The aim is to stimulate massive community motivation and participation leading to immunization of every eligible member during both routine immunization activities and SIAs through agreement on expectations and rewards with community leaders from poor performing areas. Despite the fact that vaccinators have clearly defined areas to cover during SIAs, they do not reach a significant proportion of children. Understanding why vaccinators do not cover the entire area assigned to them and the factors that would motivate them to do so will be helpful in guiding vaccination practice. The reasons are likely to vary according to setting. The aim of this research is to interview vaccinators in areas with low vaccination coverage. There has been a consistent blame game between health workers and users of their services regarding their expectations in improving immunization service delivery. A mutual understanding of these expectations will help to improve programmatic activities, attitudes of health care workers, community participation and ultimately immunization coverage. This study aims to evaluate the impact on immunization implementation of agreements on mutual expectations between health care workers and the communities on both routine and SIAs immunization services delivery. The Endgame Plan calls for the introduction of at least one dose of IPV in addition to OPV into routine immunization by the end of 2015. The
Cross-sectional survey of polio antibodies
Impact of community reward/competition on immunization coverage
Identification of incentives and disincentives of vaccinators during SIAs
Mutual expectations of health care workers and the community for improved coverage
IPV knowledge of vaccinators in preparation for introduction of IPV
23
Title
Brief justification and objective simultaneous administration of OPV and IPV to a child as recommended is unusual and is likely to encounter challenges from both health care providers and parents if the purpose is not well known particularly by health care providers. Health care workers have a critical role to play in the implementation of this intervention but their understanding and concerns about IPV introduction need to be addressed. This study aims to evaluate the knowledge and perceptions of health care workers on IPV introduction. This will help in guiding training and defining the methods of information dissemination for the IPV rollout.
Validation of routine immunization coverage data
Administrative data tend to be the primary source of estimates on national and global vaccination coverage and immunization systems performance for WHO and UNICEF. However, the validity of administrative data has been questioned repeatedly, particularly in the African Region. Therefore, this study aims to assess the validity of administrative-based immunization estimates compared with health facility records and household survey data. Some areas persistently have poor routine and supplementary immunization coverage. Ignorance of the etiology, transmission, pathogenesis and prevention of VPDs could be responsible for poor community motivation and demand for vaccines in such areas. The approach in such a community will not be to push more SIAs but to educate them on the causes of the common childhood diseases and the role of immunization in preventing them. This study aims to assess the knowledge of parents on polio and other VDPs in settings with low and high immunization coverage. Immunization coverage with the first dose of DPT3 is about 90% in the Region and that for the three doses has plateaued at about 70%. The reasons for children dropping out from the schedule or delaying in getting vaccines need to be identified and addressed to improve coverage. This research aims to study the degree of timeliness of routine immunization and the reasons associated with vaccination delay.
Knowledge of parents on polio and other VPDs in good and poor immunization coverage areas
Risk factors for delays in age-appropriate vaccination
24
2.3
Regional overview of previous implementation research on immunization
An review was undertaken of recent publications on implementation research paying particular attention to (1) the countries where the studies were conducted; (2) the population size studied, that is whether the sample was a small community, a district, the nation etc.; (3) the subject of the research, that is whether diseases, vaccines, infrastructure, human resources etc. were studied; and (4) the specific recommendations and outcomes from the research. The literature search was limited to the PUBMED database and the keywords used were “implementation research”, “immunization” and “vaccination coverage”. Only the recent literature was reviewed. A significant proportion of the publications were from Nigeria, South Africa, Ethiopia, Benin, Burkina Faso, Kenya and Mozambique.6-10 A few others were from the Democratic Republic of Congo and Angola. Most of the studies focussed on specific regions or parts of a country, which made it difficult to generalize the findings to other parts of the same country or to other countries. These studies identified the factors in poor immunization to include the lack of strong, committed leadership in some countries to effectively mobilize immunization teams,5 ignoring of older groups during polio campaigns,11 the lack of access to some remote communities,9 the lack of infrastructure including cold chain storage especially for new vaccines,6 tactlessness of the vaccinators,12 and negative parental attitude.13 Parental attitude is a very important factor as a recent study in Australia has shown, where children of immigrants from East Africa were found to have poor immunization coverage.10 Although no reason was suggested for the observation, it is likely to be linked to the behaviour and practices acquired in the countries of origin. Other factors highlighted from the review of the literature were poverty;14 poor utilization of services despite their good access;15 religious reasons; occupation, with low rates being prominent among young female traders; illiteracy; older paternal age; food insecurity; and having had a stillbirth in the family.7
25
The extent to which all these factors negatively affect immunization coverage will vary according to the setting and country and will be influenced by the sociocultural context and other aspects. But some of the factors may apply across many settings. A number of publications were on the current strategy of integrated childhood interventions. A recent study showed that intermittent preventive treatment in infants (IPTi) with antimalarial drugs is well accepted by both the service providers and the communities. Some of the publications made specific recommendations to address the low immunization coverage, but most were not designed for that purpose. Other studies had relatively small sample sizes, limiting the type of statistical analysis that could be conducted and the inferences that could be made, especially in relation to their applicability to the wider population. Most of the studies reviewed focussed on routine immunization services delivery and only a few were on SIAs. Many of the studies dealt with easily accessible populations and just a few were directed at mobile populations such as nomadic communities and other hard-to-reach populations, although most often are the groups with low vaccination coverage from SIAs. These limitations underline the need to obtain Region-specific data, which this guide aims to facilitate.
2.4 Centres conducting research Traditionally, research in the countries in the Region is conducted by a variety of institutions, which may be classified as the ministry of health and its health research units, medical schools and other university units, schools of public health, independent research institutions funded by external donors, private foundations, and international organizations. Some organizations such as the African Malaria Vaccine Network Trust have been conducting surveys on the research institutions of the Region. Research in immunization could be conducted by all of these institutions depending on its nature, the capacity of the institution and whether institutional capacity building initiatives are part of the project. A major resource that has not been adequately utilized for implementation research are the schools of public health. Students of such schools need to conduct research as part of 26
their academic requirements for graduation. Since these students are often health professionals undergoing in-service training with the aim of returning to work in the health sector, they are the best placed to use their experiences in generating research questions and subsequently upon completion of their programmes to utilize their research findings to facilitate the implementation of their programmes. By involving all the stakeholders, including academic institutions, the countries can undertake implementation research and use the outcomes to strengthen their immunization systems. The countries are encouraged to collaborate with the local centres or institutions actively engaged in research so as to ensure that there is no duplication of research work and that the limited resources are put to efficient use. 2.5 Funding of implementation research Currently implementation research in the Region is financed through both internal and external sources, as well as by public and private stakeholders. The major sources of funding identified through the review of published literature included the Centres for Disease Control and Prevention of the United States of America (CDC), WHO, the Bill & Melinda Gates Foundation, and various governments such as the USA, United Kingdom, Australia, Belgium, Canada and others. In some cases the partners have funds earmarked for implementation research in specific areas. The local sources of research funding include the ministries of health, universities and research institutions, and local and international NGOs. The countries are encouraged to work with both local and international immunization partners to fund implementation research in support of immunization. Governments should invest in research in accordance with the international and regional commitments they are party to such as the Algiers and the Abuja declarations, recognizing that the information that is generated will provide evidence that will enhance their immunization services delivery. The funds used to support implementation research by the countries could come also from the general immunization budget and they should be readily deployed when needed so that research to provide solutions to the questions arising from implementation of interventions is not delayed.
27
3. DOING IT YOURSELF: The implementation research processes The aim of this guide is primarily not to turn programme managers into front-line investigators. Rather, it is intended to stimulate their interest in supporting the use of implementation research as a tool for generating solutions to the challenges they face in the delivery of immunization services. But we recognize that some programme managers are trained in research or have even conducted research to some extent. A number of such managers may require refreshing of their knowledge to renew their interest and confidence to actively undertake research or to be able to follow research activities related to their programme activities. Simple implementation research projects can be conducted directly but the more complex research questions should be addressed by independent researchers. This section of the document provides an overview of the processes involved in implementation research. As previously mentioned, only the key concepts of implementation research are presented, not the technical details of the processes that are common to both implementation research and other research, which can be found in standard research textbooks. The overall goal of implementation research is to obtain evidence of barriers to the implementation of an immunization initiative and identify solutions to the barriers that can be applied quickly and effectively to ensure significant improvement in the immunization services in the short and medium terms. Like all research, implementation research is governed by two broad principles: that its findings should be warranted, and that its methods should be transparent. In implementation research, the “research question is king” and it is what determines the method used. Questions usually arise from implementers and/or policy-makers of immunization services following anticipated or encountered challenges. The generic principles of implementation research as outlined in the previous section can be applied to guide the countries in undertaking implementation research in support of their immunization programmes. Such principles include identifying the factors responsible for underutilization of existing vaccines; the barriers to the introduction of new vaccines; the challenges in eradication of polio, measles, maternal and neonatal tetanus, yellow fever and meningitis; the ways to improve
28
efficiency in the delivery of immunization services; and the approaches to integrate immunization with other childhood health services. When these challenges are presented as needing answers from research, the six processes of implementation research as depicted in Fig. 1 should guide the investigator to arrive at a scientifically and technically sound investigation that will produce useable answers to facilitate immunization activities and policy decisions. Fig. 1: The six processes of implementation research
Source: TDR implementation research tool kit18
3.1 Contextualizing implementation research issues This pre-implementation phase entails carefully evaluating the contextual aspects in the setting in which the research is to be conducted. Physical, socioeconomic, cultural, health systems, stakeholder and institutional contexts of a setting contribute to and affect the planning, implementation, monitoring and outcomes of implementation research. Therefore, consideration should be given to issues such as whether the location is rural or urban, the physical barriers to access of immunization services, availability of relevant infrastructure, general standards of living, level of inequality, cultural beliefs related to health, gender equity, literacy rates, and government capacity to provide the immunization services. 29
The health system is made up of multiple subsystems that include the public health care system, the formal and informal private health care systems and the household health care system. Implementation research often involves many stakeholders and institutions, and these should be identified in the early stages of the research in a systematic and comprehensive way. How these stakeholders are likely to be affected by the research and how they will respond to the research outcomes should be assessed as crucial determinants of the relative success or failure of an implementation project. This assessment can be achieved through a SWOT (strengths, weaknesses, opportunities and threats) analysis. In the contextualizing phase the investigator should consider the cultural beliefs and practices, the political structure, the way the health system is organized and the whole range of stakeholders in the environment. These contextual factors must be considered throughout the research processes because they constitute the real-life setting for implementing public health interventions such as immunization. It is because of its capacity to illuminate contextual issues that implementation research is such a powerful tool for implementers during the planning stage of interventions for effective policymaking and programme design. 3.2 Developing an implementation research proposal A research proposal is a document that describes the suggested research, why it is being conducted and its design, expected results and impact. The proposal forces the investigators to clarify their thoughts and to think about all aspects of the study, it provides the necessary guide and reference for the team working on the research, and it is an essential requirement for obtaining institutional ethical approval. A methodological approach is required to develop a comprehensive proposal (see Table 3).
30
Table 3: Steps in the development of an implementation research proposal Research questions What is the problem and why must it be studied?
Parts of the proposal Selection, analysis and statement of the problem or research issue
Elements of the parts Problem identification Prioritization of the problem Analysis Problem justification Literature search Analysis of previous data Pilot or survey Other information
What background information is available?
Review of literature and data
Why conduct the study? What are the expected results? What additional information is required to meet the objectives? How will the information be collected?
Formulation of objectives Research design and methodology
General and specific objectives Variables Data collection techniques Population sampling Plan for data collection Plan for data processing and analysis Ethical considerations Pre-testing and piloting Human and other resources Time line and activity plan
Assignment of roles What resources are available? How will the study be administered? How will the results be used?
Work plan Budget Project administration plan Utilization of results
Material support and equipment Actual funds Administration or project management Monitoring Identification of users Partner briefing sessions Lobbying
How will the proposal be presented to funders and partners?
Project brief
A proposal should be judged by three main criteria: its ability to answer the research question and achieve the objectives, the feasibility of its particular set-up for the study, and the presentation of its specifics in enough detail to allow another investigator to undertake a similar study and arrive at the same or a comparable conclusion. The components of a proposal include the introduction, a description of the research design, the project plan, a statement of the expected impact, the ethical justification and the dissemination plan for the research outcomes.
3.2.1 Introduction
The introduction aims to provide the foundation and context of the research (overview of the problem); highlight the background information from research on similar topics (systematic analysis of the data and succinct review of the literature), and state why the 31
proposed research should be undertaken (rationale), what it is hoped to achieve (objectives) and the expected results (outcomes). The introduction should include a statement of the problem; a literature review or synthesis of existing knowledge; the rationale for the research, which is an outline of what is going to be studied and why; an argument for the study; the research questions; and the general and specific objectives. It should also summarize the expected outcomes, including the anticipated impact, and provide a succinct rationale for the reason the project should be funded. The research question should be of interest to the research community, policy-makers, decision-makers and other relevant stakeholders. It should be shaped by the problem and it, in turn, should shape the research design. The research question should also be clear and specific, answerable and able to generate important information. The research objectives should be SMART (specific, measurable, achievable, realistic and time-bound).
3.2.2 Research design
The research design section includes descriptions of the study participants, research methods, data collection tools, data analysis approaches, quality management methods, and an account of how the ethics concerns will be addressed. The details on the study participants should include the method of and criteria for their selection such as age, gender, ethnicity and income and how the study site and sample size were determined.
The research design determines the methods to be used, that is whether qualitative, quantitative, between- or within-subjects, experimental, correlational, individual or collective case studies, or mixed methods etc. will be the most appropriate. There are no fixed rules in the selection of research methods for implementation research provided that the method used is appropriate to address the research question. However, qualitative and quantitative inquiries are the two prominent types. Qualitative methods involve collecting data in the form of words from a small number of purposefully selected participants with the data being used to explore the values, attitudes, opinions, feelings and behaviour of the individuals. The results are descriptive rather than predictive.19 Quantitative methods, 32
in contrast, involve collection and analysis of objective and often numerical data and are used to determine the relationship between variables or to explore differences between two or more groups. Quantitative research designs and methods are decided on before the start of data collection and cannot be modified during the study. They use questionnaires and their data are presented in numerical form and analysed using descriptive or inferential statistics.
Most implementation research projects draw on a wide variety of information and use mixed-method research approaches. In addition, implementation of their outcomes typically is influenced by a wide array of contextual factors. Often these factors change over time, producing unpredictable effects that require continual adaptation by the implementers. This has profound implications for the research methods used, particularly in regard to the need for flexibility in order to be responsive to and capable of capturing changes at multiple points in time.
3.2.3 Data collection and analysis
Closely linked to the choice of the research design is the selection of the tools for data collection. In this regard, once the most efficient design has been chosen, the resources available and the level of rigor or depth intended for the study are the key determinants. Table 3 summarizes some of the commonly used tools and the considerations that should guide their selection. Table 4: Features of some research tools Method Level of rigor Financial Costly Normal Very costly Normal Costly Costly Very costly Extremely costly Very costly Resources required Human High Normal High Normal High High High Very high Very high Time Long Short Long Short Long Short Long Very long Very long
Focus group discussions Key informant interviews Exit interview and mystery client surveys Direct observation Rapid assessment methods Case studies Participatory evaluations Population-based surveys Quasi-experimental or pragmatic trials
++ ++ ++ ++ + + ++ +++ +++
33
The research approach chosen also has implications on the budget and scheduling. Some methods are more demanding in terms of resources than others, depending on the depth and rigor of the research required to achieve the objectives and answer the research question. The countries are encouraged to focus on qualitative studies that are also multisite and multi-country and include gender, social-cultural, economic and political aspects, in line with the recommendations of TFI. The choice of methods should be guided by the resources and expertise available. The proposal should include information on how the data will be managed, including how they will be coded, cleaned and verified. Information on the computer capacity available and the statistical methods to be used for data analysis should be provided. 3.2.4 Ethical considerations
All research involving human subjects should be conducted in accordance with the ethical principles contained in the World Medical Association Declaration of Helsinki. All individuals involved in the conduct of the study must be fully informed of and comply with ethical principles including those related to (1) beneficence – meaning that the proposal must be scientifically and technically sound and capable of providing answers to the research question; (2) non-maleficence – implying that the potential risks to participants are properly balanced with the benefits and that adequate treatment will be given to the participants if adverse events arise from the interventions; and (3) respect – meaning that it will be ensured that the participants are informed, their consent is given voluntarily and confidentiality is adhered to. 3.2.5 The project plan
The project plan section helps to demonstrate the feasibility of the project in a visible way. There are three phases of implementation research: planning, implementation and follow-through. The plan should present a clear indication of the time frame of the project, the tasks involved, the time when each activity in the project will be implemented and the member of the team responsible for the tasks involved. The project plan facilitates definition of the project focus, clarifies responsibilities and roles and defines the tools for project monitoring and evaluation, and review. Time lines need to be realistic and to cover the duration of the entire project. They should be depicted using the
34
most appropriate graphic display such as graphs, tables, spreadsheets, Gantt charts or other visuals. A description of the research team members’ qualifications should be provided showing their skills and expected contribution to the project. The management structure should be depicted graphically in an organogram or another type of visual representation. The team should be multidisciplinary and diverse where necessary, as this helps to prove to reviewers that there is enough expertise to effectively conduct the proposed research.
The budget and justification section gives details on the funds needed to conduct the research. It needs to be realistic and should align with the activities proposed in the research design. The budget should cover salaries, training, equipment, consultants, supplies, communication and travel among other items. Each budget item should be justified.
3.2.6 The impact statement
This section should define the process for determining that the project objectives and deliverables have been satisfied and monitoring of resource use and adherence to the research protocol. It should also describe how the research findings will be used in the health system to improve health outcomes.
The capacity building plan should show how the project will improve the research capacity of the national and local institutions involved via training, mentorship etc. It should also indicate how the project will help to build the capacity to conduct and demand for implementation research within the health system.
The dissemination plan should specify what the research results could be and the tools and avenues that will be used to get the key message from the research to the stakeholders. The messages should be tailored to be accessible, relevant and applicable to all stakeholders. Details on the medium through which the research findings will be
35
communicated, how the stakeholders will be involved in the communication process and the strategies to promote the stakeholders’ research uptake should be provided.
The proposal should include a project summary, a table of contents, a references list, biographical résumés of the investigators and a 20-minute presentation summarizing the proposal. Although the project summary and table of contents come at the beginning of the proposal, they are prepared after the other sections are completed. Some agencies or funders may require an initial letter of intent as a preliminary screening step before requesting for full the proposal to ensure that the research area aligns with their goals. The letter of intent should include the same components as a research proposal but in much less detail.
3.3 Planning to conduct implementation research Before starting the project, ethical clearance for the research must be sought, project implementation mechanisms set up and steps taken to ensure that good practices in research will be observed during the project. 3.3.1 Obtaining ethical approval
Any research involving human subjects must strictly adhere to the relevant ethical requirements and processes. The documents typically required by the ethical committee include a cover letter and a research proposal/protocol with details on the participant consenting process, risks and benefits to participants, participant recruitment process, target population, methods for returning research results to stakeholders, post-study obligations of the parties to the project, biographical résumés of the principal investigator and the research team members, and the dissemination plan. 3.3.2 Project implementation
Action must be taken before commencing the project to assemble the research team, sort out logistical issues, allocate activities and tasks to research team members and establish research sites, as well to define the procedures for reporting on the progress, closing and evaluating the project and disseminating the research findings.
36
3.3.3 Good practices in research
Good practices in research aim for optimization of the procedures and to obtain data of good quality. They include ensuring that every team member feels as part of the team and feels responsible for the outcomes of the project. This might entail training the investigators; pre-testing the instruments; making accommodation for data management processes including data back-up, protection and sharing; and starting the research implementation process with a launch function during which the entire team is presented to the community.
3.4 Data analysis and presentation The data analysis and presentation plans for implementation research should aim to show understanding of the implementation processes and to communicate to the stakeholders on how those processes will be undertaken. Data accumulated during the research might be voluminous and might need to be summarized and presented in a concise format. For this, the tools of descriptive statistics will be used, including tabulations, figures and correlations. Stakeholders’ requirements vary in regard to the type of information needed, so there should be emphasis on simplicity and interpretability. 3.4.1 Quantitative data analysis
Quantitative data analysis involves generating descriptive statistics, distributions and summary tables etc. The statistics could be group comparisons, associations and causality and measures of risks. It is important to review the results and ascertain that the outcomes of the study can be generalized beyond the small sample studied. Also, it is essential to question whether the differences or associations found could be attributed to chance and so may not be real. Data presentation could be in both textual and visual formats such as tables, diagrams, graphs and maps. 3.4.2 Qualitative data analysis
The analysis of qualitative data entails verbatim transcribing/translating, coding or annotating of comments. This is followed by identification of the main issues raised by the interviewees, that is the themes; grouping the topics within each of these themes to construct a taxonomy of subcategories; specifying what was actually said, that is the 37
components within each subcategory; and exploring the interrelationships of the various themes. Qualitative analysis can be done manually or with software, and the ensuing data may be presented as text, graphics or cartoons. 3.5 Communicating the research findings to the health system Communication of the research findings and feeding them into the health system is done continuously as the study is implemented instead of waiting to do so at its close. The reporting of the findings as they are generated to the health system to guide programmes and the practice is one of the important features that distinguish implementation research from regular research. The key components in the process of communicating research findings include knowledge translation, dissemination tools and the dissemination strategy.
3.5.1 Knowledge translation
Knowledge translation is the activities involved in moving research findings into the hands of the people and organizations that can put them into practical use. The central paradigm of knowledge management is to capture and make available information or evidence in understandable and practical ways so that it can be used by others. 3.5.2 Dissemination tools
The tools to be used to disseminate the outcomes of the project will include research reports, peer-reviewed papers, press releases, policy briefs, photo stories and graphic presentations. 3.5.3 Dissemination strategy
The dissemination strategy should be a two-way communication process in which it is ensured that the language of the dissemination materials is appropriate, the content is sensitive to the context and culture, and the dissemination channels and times are carefully determined.
38
3.6 Monitoring and evaluating the project This step entails developing and implementing an M&E plan. The M&E plan specifies how the project will measure its achievements. It documents the stakeholders’ consensus on the project, thereby fostering transparency and accountability; serves as a record in the institutional memory; tracks progress against set plans; checks for compliance with established standards; identifies trends and patterns in the programme; and helps to adapt strategies and inform decisions for project management. 3.6.1 Developing an M&E plan
To develop an M&E requires knowing what the project wants to change and how, what the specific objectives of the project are to achieve this change, what the major indicators are and how they will be measured, and how the data for M&E will be collected and analysed. Once these issues have been clarified, the responsibilities for the project should be assigned to the team members, the targets set, the reporting system defined and the methods for dissemination and utilization of the results decided on. 3.6.2 Implementing the M&E plan
The M&E plan is implemented in three stages: checking and measuring progress, analysing the situation, and reacting to new events, opportunities and issues. 3.7 Additional activities Additional activities may be integrated into the implementation research processes to enhance the quality and efficiency of the research. These include:
Research protocol review and approval Documentation of the standard operating procedures Validation of the research instruments Training of the project team on the research procedures Quality control of monitoring processes Evaluation of the services provided 39
Evaluation of the performance of the service providers Review of reports
40
References 1. Rudan I, Chopra M, Kapiriri L, Gibson J, et al. Setting priorities in global child health research investments: universal challenges and conceptual framework. Croat Med J. 2008;49:307–17. Peters DH, Tran NT, Adam T. Implementation research in health: a practical guide. 2013. Geneva: World Health Organization (who.int/alliancehpsr/alliancehpsr_irpguide.pdf). Clements W, de Quadros, Biellik, Hadler et al. Researching routine immunization – do we know what we don’t know? Vaccine 2011; 29:8477–82. SAGE Meeting, Apr 2008, Weekly Epidemiological Records, (22), 2008, 83, 193– 208. SAGE Meeting, Apr 2011, Weekly Epidemiological Records, (21), 2011 86, 205– 220. Haddad S, Bicaba A, Feletto M, Taminy E, Kabore M, Ouédraogo B, Contreras G, Larocque R, Fournier P. System-level determinants of immunization coverage disparities among health districts in Burkina Faso: a multiple case study. BMC International Health & Human Rights. 2009; 9(Suppl 1):S15 doi:10.1186/1472-698X9-S1-S15
2.
3.
4.
5.
6.
7.
Le Menach A, Llosa AE, Mouniaman-Nara I, Kouassi F, Ngala J, Boxall N, Porten K, Grais RF. Poliomyelitis outbreak, Pointe-Noire, Republic of the Congo, September 2010–February 2011. Emerging Infectious Diseases 2011;17(8):1506–9. Kamara L, Lydon P, Bilous J, Vandelaer J, Eggers R, Gacic-Dobo M, Meaney W, Okwo-Bele JM. Global Immunization Vision and Strategy (GIVS): a mid-term analysis of progress in 50 countries. Health Policy Plann 2013, 28:11-19. Fatiregun AA and Okoro AO. Maternal determinants of complete child immunization among children aged 12–23 months in a southern district of Nigeria. Vaccine. 2012;30(4):730–6.
8.
9.
10. Paxton GA, Rice J, Davie G, Carapetis JR, Skull SA. East African immigrant children in Australia have poor immunisation coverage. Journal of Pediatrics and Child Health. 2011;47(12):888–92. 11. TDR/GFATM/WHO. 2008. Framework for Operations and Implementation Research in Health and Disease Control Programmes (www.who.int). 41
12. Schwarz NG, Gysels M, Pell C, Gabor J, Schlie M, Issifou S, Lell B, Kremsner PG, Grobusch MP, Pool R. Reasons for non-adherence to vaccination at mother and child care clinics (MCCs) in Lambaréné, Gabon. Vaccine. 2009;27(39):5371–5. 13. Fourn L, Haddad S, Fournier P, Gansey R. Determinants of parents’ reticence toward vaccination in urban areas in Benin (West Africa). BMC International Health and Human Rights. 2009;9 Supplement 1:S14. 14. Makoutode M, Mohamed S, Paraïso NM, Guevart E, Akpaka Nago MR, Bessaoud K, Oladokun RE, Lawoyin TO, Adedokun BO. Impact of parental attitudes on infant vaccinal coverage in Benin. Med Trop (Mars). 2009;69(3):267–71. 15. Sia D, Fournier P, Kobiané JF, Sondo BK. Rates of coverage and determinants of complete vaccination of children in rural areas of Burkina Faso (1998–2003). BMC Public Health 2009; 9:416. doi: 10.1186/1471-2458-9-416. 16. Owino LO, Irimu G, Olenja J, Meme JS. Factors influencing immunisation coverage in Mathare Valley, Nairobi. East Africa Medical Journal. 2009;86(7):323–9. 17. de Sousa A, Rabarijaona LP, Ndiaye JL, Sow D, Ndyiae M, Hassan J, Lambo N, Adovohekpe P, Guidetti F, Recht J, Affo A. Acceptability of coupling intermittent preventive treatment in infants with the Expanded Programme on Immunization in three francophone countries in Africa. Trop Med Int Health. 2012 Mar;17(3):308-15. doi: 10.1111/j.1365-3156.2011.02915.x. Epub 2011 Nov 29.
18. TDR Implementation Research Tool Kit, 2014. (http://www.who.int/tdr/publications/year/2014/ir-toolkit-manual/en/) 19.
Mays N, Pope C. Qualitative research: observational methods in health care settings. BMJ 1995;311:182–4.
42