Assessing country-level efforts to link research to action.

620 Bulletin of the World Health Organization | August 2006, 84 (8) Abstract We developed a framework for assessing country-level efforts to link research to action. The framework has four elements. The first element assesses the general climate (how those who fund research, universities, researchers and users of research support or place value on efforts to link research to action). The second element addresses the production of research (how priority setting ensures that users’ needs are identified and how scoping reviews, systematic reviews and single studies are undertaken to address these needs). The third element addresses the mix of four clusters of activities used to link research to action. These include push efforts (how strategies are used to support action based on the messages arising from research), efforts to facilitate “user pull” (how “one-stop shopping” is provided for optimally packaged high-quality reviews either alone or as part of a national electronic library for health, how these reviews are profiled during “teachable moments” such as intense media coverage, and how rapid-response units meet users’ needs for the best research), “user pull” efforts undertaken by those who use research (how users assess their capacity to use research and how structures and processes are changed to support the use of research) and exchange efforts (how meaningful partnerships between researchers and users help them to jointly ask and answer relevant questions). The fourth element addresses approaches to evaluation (how support is provided for rigorous evaluations of efforts to link research to action). Bulletin of the World Health Organization 2006;84:620-628. Voir page 626 le résumé en français. En la página 627 figura un resumen en español. Assessing country-level efforts to link research to action John N Lavis,a Jonathan Lomas,b Maimunah Hamid,c & Nelson K Sewankambo d a McMaster University, HSC-2D3, 1200 Main Street West, Hamilton, Ontario, Canada L8N 3Z5. Correspondence to this author (email: lavisj@mcmaster.ca). b Canadian Health Services Research Foundation, Ottawa, Ontario, Canada. c Institute for Health Systems Research, Ministry of Health, 59000 Kuala Lumpur, Malaysia. d Faculty of Medicine, Makerere University, Kampala, Uganda. Ref. No. 06-030312 (Submitted: 22 January 2006 – Final revised version received: 17 April 2006 – Accepted: 11 May 2006) Introduction The idea of linking research to action in the health sector has captured a great deal of international attention. In late 2004 WHO issued the World report on knowledge for better health, with a chapter devoted to linking research to action.1 Shortly thereafter WHO convened the Ministerial Summit on Health Research in Mexico City, and the resulting Mexico statement on health research called on “all major stakeholders to strengthen or to establish activities to communicate, imp prove access to, and promote the use of reliable, relevant, unbiased and timely health information.”2 In May 2005 the World Health Assembly called on WHO Member States to “establish or strengthen mechanisms to transfer knowledge in support of evidencepbased public health and healthcare delivery systems, and evidencepbased healthprelated policies”; it also called on WHO’s DirectorpGeneral to “assist in the development of more effective mechanisms to bridge the dip vide between ways in which knowledge is generated and ways in which it is used, including the transformation of .627 ةحفص في ةيبرعلاب صخللما لىع علاطلاا نكيم healthpresearch findings into policy and practice.” 3 But statements and resolutions are easier made than acted on. Those who want to take meaningful steps to link rep search to action would ideally be able to draw on high quality, locally applicable research to inform their efforts. Those interested in linking research to action in clinical environments can draw on an overview of systematic reviews of ranp domized trials of interventions designed to better align healthpcare professionals’ practise with research 4 as well as a sysp tematic review of randomized controlled trials of strategies for guideline dissemip nation and implementation.5 Most of the studies included in these reviews were conducted in highpincome countries. However, these interventions should also be evaluated in lowp and middlep income countries.6 Those interested in linking research to action in the areas of health management and policypmaking have to deduce the attributes of potential interventions from systematic reviews of observational studies that examine the factors that influence the use of research.7,8 For example, interactions between researchers and healthpcare policypmakers and the timing or timep liness of research being made available appear to increase the likelihood that research will be used by policypmakers. Hence, interventions such as interactive workshops that bring together researchp ers and healthpcare policypmakers and web sites that provide “onepstop shopp ping” for systematic reviews have been promoted (but not yet evaluated). Healthpcare professionals, managp ers and policypmakers are not the only people who may use research. The full range of potential users (outside the research community itself ) includes the general public, patients, healthpcare prop fessionals, health managers, executives of biomedical companies and public policypmakers. (While being someone who uses research may be a small role for members of these groups, we use the term “research user” throughout as shorthand for these groups.) Some intermediary groups — by which we mean the media, civil society groups, professional associap tions and other groups that work at the interface between researchers and users of research — may also have critically 621Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health John N Lavis et al. Country-level efforts to link research to action important roles in linking research to action. In this paper we develop a framep work for assessing countryplevel efforts to link research to action. The main purpose of the framework is to inform countryp level dialogues about the domains to which attention could be directed in orp der to link research to action. Countries provide a natural unit for assessment given that there may be a division of labour within a country (for example, among research funders). A country’s capacities and constraints will affect the initial focus of their efforts (for example, creating a demand for research may be one of the first steps for some countries whereas integrating and building on exp isting efforts may be among the first steps for others). Capacities will also affect a country’s requirements for partnerships beyond its borders (for example, regional initiatives may provide economies of scale and global initiatives may provide support). When discussing efforts to link research to action within a country, the term “action” includes maintaining the status quo, which can sometimes take as much effort or more than bringing about change. The framework does not include action in the form of commercialization, however, because the different context brought about by the profit motive warp rants separate treatment. Given the state of the research in this field, many elements of the framework are based only indirectly on research. If countries select and implement one or more options from the framework among particular groups and evaluate them rigorously then 5–10 years from now we should be in a much better posip tion to refine the framework and ensure that its elements are solidly supported by research. Had such evaluations been undertaken for the frameworks that have already been developed for policyp makers,9–13 we would have a more rigorp ous research base from which to draw now. The main purpose of the examples offered here is to highlight how the framework is grounded in promising innovations that are being implemented for one or more user groups. These inp novations warrant wider consideration and rigorous evaluation. Key concepts Talk of linking research to action often puts fear into those who prefer research to be focused on discovery rather than application. But these two goals need not be in conflict. Discoveries made today are essential to the applications of tomorrow, and making efforts to forge better links between discovery and application is essential to reaping the benefits of investment in discovery.14,15 Many researchers develop innovations in basic science, theory and methodology. Researchers draw on these innovations to conduct studies and write articles and reports. And then later, researchers draw on individual contributions to produce systematic reviews. A fraction of these systematic reviews (and, occasionp ally, single studies) will yield actionable messages for one or more categories of potential research users; at other times the reviews will yield calls for more or different types of research. Talk of linkp ing research to action is then, in part, simply talk about focusing our efforts to link research to action at the apex of the knowledge pyramid while continuing to build a solid base for the pyramid (Fig. 1). Discovery and application are interp dependent points along a continuum; they are not competing objectives for the role of research in society. The funders and producers of disp coveryporiented research, who function in an environment where others have taken the lead on linking research to acp tion outside the research community, can presumably remain focused on discovery and, when appropriate, on linking rep search to action by researchers.16 These researchers will then be better positioned to explore applications of basic science or create derivative theoretical and methp odological innovations. The funders and producers of discoveryporiented research, who function in an environment where others have not taken the lead on linking research to action, face a greater chalp lenge. Those who disburse public funds to these funders and researchers may well ask them why at least some of the disp bursements are not allocated to linking research to action outside the research community. Systematic reviews For those interested in applying research, the challenge is different: there is a need to collectively create music, not noise, for the select ears of research users. Faced with a desire to link research to action, the first impulse of many funders and researchers is to confuse marketing with targeted assistance by promoting all research, regardless of its contribution to the global pool of knowledge. But funders and researchers who market single studies, articles and reports can do harm. Take, for example, the issue of whether to allow private forpprofit hospitals to compete with private notp forpprofit hospitals. A systematic review of studies comparing mortality rates in private forpprofit hospitals with those in private notpforpprofit hospitals found 15 studies that met the eligibility criteria; all but one study found a survival advanp tage for being treated in notpforpprofit hospitals.17 Funders and researchers who promoted the introduction of forpprofit hospitals on the basis of the single study with a different result were potentially doing harm. For potential research users, systemp atic reviews offer four advantages over single studies; the first two advantages apply primarily to metapanalyses that address the question “what works?” First, the likelihood of being misled by research is lower with a systematic review than with an individual study (that is, bias is reduced).18 Second, confidence in what can be expected from an intervenp tion is higher with a systematic review than with an individual study (that is, precision is increased).18 Third, drawing on an existing systematic review constip tutes a more efficient use of time because the research literature has already been identified, selected, appraised and synp thesized in a systematic and transparent way;8 potential research users can thus focus on assessing the local applicability of a review and on collecting and synthep sizing other types of information, such as routine health information. Fourth, a systematic review can be more construcp tively contested than an individual study because debate will focus on appraisal and synthesis rather than on why one study was identified and selected over others.8 Systematic review methodologies inp creasingly lend themselves to providing responses to the many different types of questions asked by different potential research users. For example, health manp agers and policypmakers are interested in finding the most effective solutions to the most burdensome health probp lems, the most effective ways to fit these solutions into complex health systems (or the most effective ways to organize health systems) and the most effective ways to bring about desired changes in health systems.19 But in addition to askp ing questions about effectiveness (does changing X change Y?), they also ask questions about cost effectiveness (is X1 more cost effective than X2 in achieving 622 Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health Country-level efforts to link research to action John N Lavis et al. Fig. 1. Knowledge pyramid1,26 WHO 06.105 Actionable messages Systematic reviews of research Single studies, articles and reports Basic science, theoretical and methodological innovations a onepunit change in Y?), relationships (is X associated with Y?), mechanisms (how are X and Y linked or why does changing X change Y?), and meaning (how have individuals viewed or experienced X or Y?). A variety of new approaches have been developed to conduct and update syntheses of research to address this broad array of questions.20–23 Although not all developers or users of these new approaches agree with labelling them systematic reviews, for convenience we use that label here as a generic term for all these forms. However, there remain legitimate and important differences in perspectives about a number of these approaches and these warrant further debate.24,25 Approaches to link research to action While actionable messages arising from systematic reviews may be the natural unit of research to consider when atp tempting to link research to action, people are still needed to make these links. Four approaches can be employed, either singly or in combination, to link research to action (Fig. 2).1,26 “Push” efp forts are led by researchers, intermediary groups and other purveyors of research (such as communications staff).27 Such efforts are well suited to situations where the potential research users are unp aware that they should be considering a particular message (or in some cases would prefer to continue to disregard it). “Userppull” efforts involve patients, healthpcare professionals, civil servants and others “reaching in” to the research world to extract information for a decip sion that they face.28 Researchers and intermediary groups can facilitate these efforts by improving access to optimally packaged research that is of high quality and relevance. Userppull efforts are well suited to situations where potential users have identified an information gap and want to address the gap in a timely way. Exchange (or “linkage and exchange”) efforts occur when the producers or purveyors of research develop a partnerp ship with a group who uses research.29 Such partnerships are well suited to situations where the two groups can establish a shared understanding about the questions to ask, how to answer them through a systematic review or as part of a research project or programme, and how to weight the research and other types of information that each group brings to the table. A fourth approach, which integrates efforts through largepscale knowledgep translation platforms, includes elep ments of the push, pull and exchange approaches. For example, a proposal for the Regional East African Commup nity Health (REACH)pPolicy initiative includes: • a governing board comprising reprep sentatives from groups of producers, purveyors and users (that is, an exp change approach); • a clear goal (that is, improving people’s health and health equity in east Africa through the more effective use and app plication of knowledge to strengthen health policy and practice); • regular prioritypsetting processes to ensure that systematic reviews and efforts to link research to action are highly relevant to the needs of potenp tial research users; • push efforts in areas where actionable messages have been identified; and • a range of efforts to facilitate user pull (such as onepstop shopping for opp timally packaged systematic reviews of high quality and relevance, and a rapidpresponse unit that provides written summaries, telephone conp sultations or inpperson consultations about the best research in a timely way). There is more than one approach to linking research to action, and not every approach will work in all situations. For example, a healthpcare professional with a patient in her office or a senior civil servant who has to brief the health minp ister in 5 minutes cannot wait for a push effort or a partnership. Similarly, the five researchers who study a particular issue in a country cannot respond to all phone Model C — Exchange efforts Fig. 2. Models for linking research to action WHO 06.106 Producers or purveyors of research Research users Model A — Push efforts by producers or purveyors Producers or purveyors of research Research users Model B — User pull efforts Producers or purveyors of research One group of research users Producers or purveyors of research Research users Model D — Integrated efforts Knowledge translation platforms 623Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health John N Lavis et al. Country-level efforts to link research to action calls from, or develop partnerships with, every clinic or hospital. A community health centre that wants to undertake communitypbased research to inform its strategic direction while developing lop cal capacity to produce and use research may not be well served by research from other communities that have similar sop cioeconomic and ethnocultural profiles. Moreover, a single knowledgeptranslation platform will still leave many actionable messages unused and many user groups without service. Given the failings of any one approach operating in isolation, there is great value in using all four app proaches simultaneously. Framework for assessing country-level efforts The proposed framework has four elep ments for assessing countryplevel efforts to link research to action: the general climate for research use, the production of research that is both highly relevant to and appropriately synthesized for rep search users, the mix of clusters of activip ties used to link research to action, and the evaluation of efforts to link research to action (Table 1). When discussing the mix of activity clusters, elements of the integrated approach are discussed in the context of the other three approaches. Additionally, the userppull approach is separated into those activities that can be undertaken by the producers or purp veyors of research (for example, efforts to facilitate pull by appropriately packaging key messages of the research) and those activities that can be undertaken only by research users (for example, revising decisionpmaking processes to include explicit consideration of research). In the first element of the framep work we posit that the general climate is conducive to linking research to action when the following conditions are met: • at least some funders have a mandate to support efforts to link research to action and they support these efforts in several ways; • universities and other research instip tutions consider such efforts to be in their tenure and promotion processes and work to remove disincentives to link research to action; • some researchers place value both on promoting the use of research and — if they are to establish mutually respectful partnerships with research users — on the other types of inp formation on which research users regularly draw (for example, public policypmakers draw on legal evidence about institutional constraints, civil servants’ assessments of stakeholders’ interests and public opinion polls); and • intermediary groups and research usp ers place value on the use of research. The funding environment poses particup lar challenges in many lowp and middlep income countries because funding may come not only from national bodies but also from international development aid, research institutes in highpincome countries or international foundations and agencies, all of which may have broad development, health or internap tional goals rather than nationally focused health research goals. The second element of the framep work focuses on the production of research. We posit that the research enp terprise is conducive to linking research to action when the following conditions are met: • some funders periodically engage potential research users in priorityp setting processes,30 commission or fund scoping reviews to fill informap tion gaps in areas identified as priorip ties by users, support the production and regular updating of systematic reviews and, when appropriate, fund single studies; • funders and ethics review boards place value on systematic reviews to justify additional research on a topic; and • some researchers participate in conp tinuing education programmes to develop their capacity to conduct systematic reviews and respond to the calls for research in priority areas as well as committing to updating regularly systematic reviews they are funded to produce. The emphasis on production is parp ticularly important in lowp and middlep income countries where, apart from a few fields and especially in applied fields,31 there is a dearth of high quality research that can be linked to action.32,33 Our emphasis on “some” funders and researchers, rather than on “all”, is to enp sure that gains can be achieved without jeopardizing the innovations in basic scip ence, theory and methodology that form the base of the knowledge pyramid. The third element in the framework addresses the mix of activity clusters used to link research to action. Push efforts are also likely to be conducive to linking research to action when some funders, researchers or intermediary groups, or some combination of these, engage in the following components of a systematic push effort: • periodically identify actionable mesp sages arising from systematic reviews (or occasionally from single studies when a strong case can be made for their unique contributions); • fineptune messages and related rep sources for different user group; • work with and through the most credp ible messengers for each user group; • use researchpinformed strategies to encourage and support action based on the messages; and • evaluate their impact against achievp able objectives.27 As emphasized in the Introduction, the degree to which strategies to encourp age and support action can be based on research varies by user group. Push efforts are also likely to be conducive to linking research to action when some funders, researchers or intermediary groups, or some combination of these, develop mep dia releases for systematic reviews (rather than only for articles and reports based on single studies) as is now being done by the United Statespbased Center for the Advancement of Health, when some researchers employ selfpassessment tools to evaluate their capacity to develop and execute researchpinformed push efforts, and when some researchers participate in continuing education programmes to develop these capacities. We posit that efforts to facilitate user pull are likely to be more conducive to linking research to action when some funders, researchers or intermediary groups, or some combination of these: • provide onepstop shopping for opp timally packaged reviews that are of high quality and relevance (either in standpalone format as is done through initiatives like the Health Evidence Network of WHO’s Rep gional Office for Europe and WHO’s Reproductive Health Library or as part of a national electronic library for health as has been done in the United Kingdom); • profile these reviews during “teachable moments” (for example, as is done for physicians in the United Kingdom through the “Hitting the Headlines” service of the National Library for Health); 624 Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health Country-level efforts to link research to action John N Lavis et al. Table 1. Framework for assessing country-level efforts to link research to action Domain Elements General climate • Some funders have a mandate to support efforts to link research to action as well as to support excellence in research • Some funders support efforts to link research to action by providing funding for such efforts and by considering such efforts in research-assessment exercises at the level of institutions and (when appropriate) in peer-review processes at the level of research proposals • Universities and other research institutions support faculty members’ efforts to link research to action by considering such efforts (when appropriate) as part of tenure and promotion processes • Some researchers place value on promoting the use of research and on recognizing the importance of other types of information on which research users regularly draw to inform their own or their organization’s decision-making • Intermediary groups (including the media, civil society groups and professional associations) place value on the use of research to inform their target audiences, constituencies or memberships • Users of research (including funders) place value on the use of research to inform their own or their organizations’ decision-making • Some researchers and research users (as well as intermediary groups including the media, civil society groups and professional associations) exhibit a commitment to work within a model of ongoing “linkage and exchange” that guides their efforts to produce research and other types of information to link research to action Production of • Some research funders periodically engage potential users of research in priority-setting processes to identify priority research areas both for research and for efforts to link research to action • Some funders commission or fund scoping reviews to identify the state of research in priority areas before undertaking efforts to support systematic reviews or additional research on a topic, or both • Some funders commission or fund the production and regular updating of systematic reviews that address the full range of questions asked by research users in particular priority areas (as well as the documents that assess an intervention, such as health technology assessments, and guidance documents, such as clinical practice guidelines or system-oriented syntheses, that build on these systematic reviews) • Funders and ethics review boards place value on the use of systematic reviews to justify additional research on a topic and on the registration of systematic reviews to avoid unnecessary duplication • Some researchers respond to funding calls for scoping reviews, systematic reviews and original research (as well as to funding calls for the assessment and guidance documents that build on these systematic reviews) in high-priority areas and commit to regularly update the systematic reviews • Some researchers periodically participate in skill-development programmes to develop their capacity to conduct systematic reviews (as well as to produce the assessment and guidance documents that build on these systematic reviews) that address one or more of the types of questions asked by potential research users Push efforts • Some funders, researchers or intermediary groups, or some combination of these, periodically engage in all or some (as one cluster of elements of a systematic push effort by identifying actionable messages arising from systematic reviews (and from activities among single studies when a strong case can be made for their unique contributions), fine-tuning messages and related a mix of four resources for different user groups, working with and through the most credible messengers for each user group, clusters) using research-informed strategies to encourage and support action based on the messages and evaluating their impact against achievable objectives – For clinicians, research-informed strategies may be drawn from an overview of systematic reviews of randomized trials of interventions designed to better align health-care professionals’ practise with the research,4 as well as from a systematic review of randomized controlled trials of all guideline dissemination and implementation strategies 5 and a summary of this research from the perspective of low- and middle-income countries 6 – For public policy-makers, the attributes of potential interventions have to be deduced from systematic reviews of observational studies that examine the factors influencing the use of research in health-care management and policy- making 7,8 — for example, interventions that encourage interactions between researchers and health-care policy- makers and that ensure the best available research is made available in a timely way are addressing two factors that increase the likelihood that research will be used by policy-makers • Some funders, researchers or intermediary groups, or a combination of these, develop media releases for systematic reviews and (when appropriate) profile and place in context locally conducted studies • Some researchers periodically employ self-assessment tools to evaluate their capacity to develop and execute research-informed push efforts • Some researchers periodically participate in skill-development programmes to enhance their capacity to develop and execute research-informed push efforts Efforts to facilitate • Some funders, researchers or intermediary groups, or a combination of these, maintain user-group specific web sites user pull (as one (or CD-ROMs) that provide one-stop shopping for systematic reviews that are optimally packaged and of high quality cluster of activities and relevance (also, as part of a national electronic library for health, provide access to health statistics, performance among a mix of indicators and other locally relevant data, as well as names and contact information for people who are familiar with four clusters) the reviews and data) 625Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health John N Lavis et al. Country-level efforts to link research to action Domain Elements Efforts to facilitate • Some funders, researchers or intermediary groups, or a combination of these, maintain user-group specific web sites user pull (as one that profile systematic reviews that are optimally packaged and of high quality and relevance as well as being locally cluster of activities applicable during “teachable moments”, such as during a period of intense media coverage of an issue among a mix of • Some funders, researchers or intermediary groups, or a combination of these, maintain user-group specific rapid- four clusters) response units that provide written summaries, telephone consultations or in-person consultations about the best available research • Some funders, researchers or intermediary groups, or a combination of these, periodically participate in skill- development programmes to enhance their capacity to develop and execute efforts to facilitate user pull User-pull efforts • Research users periodically employ user-group specific self-assessment tools to evaluate their capacity to acquire, (as one cluster of assess, adapt and apply research activities among • Research users develop structures and processes to help them to acquire, assess, adapt and apply research; to a mix of four combine research with other types of information as inputs to decision-making; and to promote the use of research in clusters) decision-making • Research users periodically participate in skill-development programmes to enhance their capacity to acquire, assess, adapt and apply research Exchange efforts • Some trusted individuals (sometimes called “knowledge brokers”) build relationships among researchers and research (as one cluster of users who have shared interests activities among • Some researchers and research users build partnerships around single studies, programmes of research, or systematic a mix of four reviews to enable them to collectively ask and answer locally relevant questions (for example, to co-produce research clusters) and other types of information) • Some researchers and research users build partnerships around efforts to link research to action, specifically push efforts, efforts to facilitate user pull, and user-pull efforts undertaken by those who use research • Some researchers and research users develop partnerships outside the context of the co-production of research and efforts to link research to action to enable them to discuss a broad range of issues of mutual interest • Some researchers and research users periodically participate in skill-development programmes to enhance their capacity to engage in mutually beneficial partnerships Evaluation • Some funders provide funding for rigorous evaluations of efforts to link research to action • Funders, researchers, intermediary groups and user groups participate in rigorous evaluations of efforts to link research to action (Table 1, cont.) • administer rapidpresponse units of the type described above (as is done for policypmakers through the European Observatory on Health Systems and Policies); and • participate in programmes to enhance their capacity to develop and execute efforts to facilitate user pull. Because studies conducted within particular health systems or particular populations in countries may have limp ited applicability to other health systems or populations, we posit that efforts both to push and to facilitate user pull will need to highlight the factors that influp ence the local applicability of systematic reviews.19,34 Userppull efforts are likely to be conp ducive to linking research to action when some research users: • employ selfpassessment tools to evaluate their capacity to acquire, assess, adapt and apply research (such as the tool developed by the Canadian Health Services Research Foundation);28 • develop structures and processes to help them use and promote research (for example, policypmakers may rep quire that submissions make explicit the research and routine health inforp mation underlying any assessments of options); and • participate in skillpdevelopment prop grammes to enhance their capacity to use and promote research (such as the Executive Training for Research Application programme developed by the Canadian Health Services Research Foundation). Exchange efforts are likely to be more conducive to linking research to action when they are: • personal and ongoing, thus creating a window onto the research world for the potential research user (beyond the specific study, research prop gramme or systematic review around which the exchange is occurring) and a window onto the health system for the researcher (facilitating an underp standing of the cultural and other differences between the contexts of researchers and research users); • based on a meaningful partnership where the relative roles and expertise of researchers and research users are recognized; and • supported by skillpdevelopment programmes that allow participatp ing researchers and research users to develop their capacity to engage in mutually beneficial partnerships. These exchange relationships may be developed and nurtured by trusted inp dividuals or organizations acting in the role of what have been called “knowledge brokers”.35 The USpbased Agency for Healthcare Research and Quality has paid particular attention to developp ing partnerships focused on systematic reviews,36 whereas other funders have been more focused on building partnerp ships around single studies or research programmes.29 For the last element in the framep work we posit that evaluations are likely 626 Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health Country-level efforts to link research to action John N Lavis et al. to support future efforts to link research to action when: • funding is available for evaluations of largepscale natural or planned innovap tions to link research to action; and • funders, researchers, intermediary groups and user groups participate in rigorous evaluations of efforts to link research to action. Weak evaluation designs that do not examine both the intended and uninp tended consequences of these innovap tions will not provide the research needed to inform future efforts to link research to action. Applying the framework No country can or should undertake every component of the four elements in the framework, especially in the domains of producing research and facilitating user pull. The production of systematic reviews, for example, is a global responp sibility even if their interpretation and translation into actionable messages is best done at the local level. For example, a systematic review conducted in Malaysia may need little adaptation beyond transp lation before it can be used in deliberative forums and decisionpmaking venues in Thailand. Some forms of facilitating user pull are also a global responsibility. The Cochrane Library, for example, provides onepstop shopping for high quality sysp tematic reviews addressing the question of “what works?” National electronic libraries for health can provide links to this global resource, which can often be accessed for free by individuals in lowp and middlepincome countries through, for example, the Latin American and Caribbean Center on Health Sciences Information (BIREME), the Health InterNetwork Access to Research Initiap tive (HINARI), and the International Network for the Availability of Scientific Publications (INASP). A number of initiatives have atp tempted to address one or two elements of the framework. For example, the Applied Diarrhoeal Disease Project and the Joint Health Research Systems Project for southern Africa were initiatives that focused primarily on producing highly relevant research and undertaking push efforts. Both emphasized the creation of studies relevant to health policy and systems through the inclusion of policypmakers, managers or healthpcare professionals in the researchpplanning process and the dissemination of results, which was achieved through targeting presentations of findings towards those who could act on them.37 The Internap tional Clinical Epidemiology Network’s Knowledge “Plus” Program focuses prip marily on push efforts involving clinical practise guidelines. The International Health Policy Program focused primarp ily on exchange efforts by pairing young researchers with policypmakers in order to strengthen linkages and inform health policy. The programme relied less on universities than on standpalone organip zations with research capacity.38 As part of its new strategic plan, the Alliance for Health Policy and Systems Research has articulated its intent to focus on all elements of the framework.39 Three regional initiatives that ofp fer great promise are in development. The Regional East African Community HealthpPolicy initiative proposes to esp tablish a unit located within the East African Health Research Council to adp dress all elements of the framework for Kenya, Uganda and the United Republic of Tanzania. The Western Pacific Region of WHO has launched the planning stages of a project known as Evidence Informed Policy Networks (EVIPNet Asia) to address many elements of the framework for one municipality in China (Beijing), two Chinese provp inces (Shandong and Sichuan), the Lao People’s Democratic Republic, Malaysia, and Viet Nam.40 The African Region of WHO has launched a similar prop gramme, known as EVIPNet Africa. The speed with which these regional initiatives are moving suggests they will pioneer new integrated knowledgep translation platforms well before other parts of the world. Conclusions The framework for assessing countryp level efforts was designed to inform countryplevel dialogues about the opp tions for linking research to action for different groups of users. Many elep ments of the framework are based only indirectly on research. To push forward our understanding of these elements and their interrelationships, we should rigorously evaluate innovative countryp level efforts targeted at particular user groups. An important first step in any country will be to identify for specific user groups the supportive elements that are already in place (and that warrant evaluation) as well as the unsupportive elements that should be addressed and the aspyetpunassessed elements that should be examined. Initiatives that are under development in subpSaharan Africa and the western Pacific may prop vide opportunities to test the framework in a comprehensive way. O Funding: John Lavis receives salary supp port as the Canada Research Chair in Knowledge Transfer and Exchange. Competing interests: none declared. Résumé Évaluation à l’échelle d’un pays des efforts pour lier recherche et action Un cadre permettant d’évaluer à l’échelle d’un pays les efforts pour lier recherche et action a été mis au point. Ce cadre comporte quatre composantes. La première évalue le contexte général (quel appui ou quelle valeur les acteurs qui financent la recherche, les universités, les chercheurs et les utilisateurs des données de recherche accordent-ils aux efforts pour relier recherche et action). La deuxième composante s’intéresse à la production de la recherche (dans quelle mesure la définition des priorités garantit-elle l’identification des besoins des utilisateurs et dans quelle mesure également des analyses de portée, des revues systématiques et des études simples sont-elles entreprises pour répondre à ces besoins). La troisième composante traite de l’utilisation d’une combinaison d’activités relevant de quatre groupes pour établir un lien entre recherche et action. Ces activités comprennent les efforts de type «push» (comment mettre en œuvre des stratégies pour étayer les interventions par des messages en provenance de la recherche), ceux visant à faciliter l’extraction d’informations par les utilisateurs («user pull») [comment s’opère l’approvisionnement à une source unique à partir de revues de haute qualité, présentées de manière optimale, isolément ou dans le cadre d’une librairie électronique nationale en faveur de la santé, comment l’attention est attirée sur ces revues pendant les «fenêtres 627Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health John N Lavis et al. Country-level efforts to link research to action Resumen Evaluar los esfuerzos realizados en los países para vincular las investigaciones a la acción Elaboramos un marco para evaluar los esfuerzos desplegados a nivel de país para vincular las investigaciones a la acción. El marco consta de cuatro elementos. El primero evalúa el clima general (de qué manera quienes financian las investigaciones, las universidades, los investigadores y los usuarios de las investigaciones respaldan o valoran los esfuerzos realizados para vincular las investigaciones a la acción). El segundo aborda la producción de investigaciones (cómo mediante el establecimiento de prioridades se asegura que se identifiquen las necesidades de los usuarios, y cómo llevar a cabo revisiones sintéticas, revisiones sistemáticas y estudios independientes para responder a esas necesidades). El tercer elemento aborda la combinación de cuatro tipos de actividades orientadas a vincular las investigaciones a la acción: actividades impulsoras (formas de aplicación de estrategias para apoyar las medidas basadas en los mensajes derivados de las investigaciones), actividades de fomento de la «atracción de usuarios» (por ejemplo una «ventanilla única» para conseguir combinaciones óptimas de revisiones de calidad, ya sea de forma independiente o como parte de una biblioteca electrónica nacional de salud; la reseña de esas revisiones durante las «oportunidades didácticas», como una intensa cobertura mediática; y la actuación de unidades de respuesta rápida en respuesta a las necesidades de los usuarios para optimizar las investigaciones), las actividades de «atracción de usuarios» emprendidas por quienes utilizan las investigaciones (análisis de cómo los usuarios evalúan su capacidad para usar las investigaciones, y de la manera de modificar las estructuras y los procedimientos para fomentar el uso de las mismas), y las actividades de intercambio (determinación de cómo las fórmulas de colaboración más eficaces entre investigadores y usuarios les ayudan a dar forma y responder conjuntamente a los interrogantes pertinentes). El cuarto elemento aborda los métodos de evaluación (la prestación de apoyo para la realización de evaluaciones rigurosas de las actividades de vinculación de las investigaciones a la acción). References 1. World Health Organization. World report on knowledge for better health. Geneva: WHO; 2004. 2. World Health Organization. The Mexico statement on health research, 2004. Available from: http://www.who.int/rpc/summit/agenda/Mexico_Statement- English.pdf 3. World Health Assembly. Resolution on health research, 2005. Available from: http://www.who.int/rpc/meetings/58th_wha_resolution.pdf 4. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero LA, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care 2001;39 Suppl 2:S2-45. 5. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:1-72. 6. Haines A, Kuruvilla S, Borchert M. Bridging the implementation gap between knowledge and action for health. Bull World Health Organ 2004;82:724-33. 7. Innvaer S, Vist GE, Trommald M, Oxman AD. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy 2002; 7:239-44. d’enseignement» (couverture médiatique intense, par exemple) et comment les unités de réponse rapide répondent aux besoins des utilisateurs en matière de recherche de qualité optimale], les efforts de type «user pull» entrepris par les utilisateurs de la recherche (comment ces utilisateurs évaluent-ils leur capacité à exploiter les données de recherche et comment modifier les structures et les processus pour appuyer l’utilisation de la recherche), ainsi que les efforts d’échange (comment des partenariats judicieusement organisés entre chercheurs et utilisateurs aident les uns et les autres à poser des questions pertinentes et à y répondre). La quatrième composante concerne les méthodes d’évaluation (comment apporter un soutien en faveur d’évaluations rigoureuses des efforts pour lier recherche et intervention). صخلم لمعلاب ثوحبلا طبرل يرطقلا ديعصلا لىع ةلوذبلما دوهجلا ميـيقت طبرل يرطقلاا ديعصلا لىع ةلوذبلما دوهجلا ميـيقتل لمع راطإ انركتبا لولأا صرنعلا مِّيقي ؛صرانع ةعبرأ لمعلا راطإ ن َّمضتيو .لمعلاب ثوحبلا تاجاحلاو ثوحبلا نولويم نيذلا نم معدلا ميدقت ةيفيك( ماعلا خانلما دوهجلا لىع ريدقتلا ءافضإ ةيفيك وأ ثوحبلا نم نيديفتسلماو ينثحابلاو ثوحبلا جاتنلإ ى َّدصتيف نياثلا صرنعلا امأ .)لمعلاب ثوحبلا طبرل ةلوذبلما تحبصأ دق نيديفتسلما تاجاح نأ تايولولأا دادعإ ةيلمع نمضت فيك( تاعجارلماو ،تاجايتحلاا قاطنل تاعجارلما تيرجأ فيكو ،ةددحمو ةفورعم ثلاثلا صرنعلا امأ .)تاجايتحلاا هذه ةيبلتل ةدرفلما تاساردلاو ةيجهنلما ثوحبلا طبرل مدختست يتلا ةطشنلأا نم تاعومجم عبرأ رفاضت جلاعيف ةيفيك( ةعفاد دوهج لىع دنتست هذه ةطشنلأا تاعومجم لوأو ؛لمعلاب لىإ دانتسلااب ذ َّفنت يتلا ةمعادلا ةطشنلأا معدل تايجيتارـتسلاا نم ةدافتسلاا لذبت يتلا دوهجلا ةطشنلأا تاعومجم نياثو ،)ثوحبلا نم ةدافتسم تلااسر تارايتخلاا نم ةدحاو ةئماق ميدقت نكيم فيك( نيديفتسلما رارجتسا ليهستل ةئماقلا هذهل نكيم ثيحب ،ةيلاثلما فورظلا في ةقئاف ةدوجب عتمتت تاعجارلم ،ةيحصلا ةينطولا ةينوتركللإا ةبتكلما نم ءزجك م َّدقت وأ اهدحول م َّدقت نأ هعم نكيم لكشب ،ةعيسر تاظحل في تاعجارلما هذه فينصت نكيم فيكو نكيم ثيحبو .ةيملاعلإا طئاسولاب ةفيثك ةيطغت عم ،سوردلا صلاختسا ،)ثوحبلا لضفأ نم نيديفتسلما ةجاح يبلت نأ ةعيسرلا ةباجتسلاا تادحول نم نيديفتسلما رارجتسلا لذبت يتلا دوهجلا ةطشنلأا تاعومجم ثلاثو ثوحبلل ينمدختسلما مييقت ةيفيك( ثوحبلا نومدختسي نيذلا كئلوأ لَبِق يرغتت نأ تايلمعللو ىنبلل نكيم فيكو ،ثوحبلا نم ةدافتسلاا لىع مهتاردقل لىإ( دوهجلا لدابت ةطشنلأا تاعومجم عبارو ،)ثوحبلا مادختسا معدل ًايخوت لىع مهدعاست نأ ينمدختسلماو ينثحابلا ينب ةميمحلا ةكاشرلل نكيم ىدم يأ جلاعيف عبارلا صرنعلا امأ .)كترشم لكشب اهيلع ةباجلإا لىعو ةلئسلأا حرط لذبت يتلا ةينضلما دوهجلا مييقتل معدلا ميدقت نكيم فيك( مييقتلا بيلاسأ .)لمعلاب ثوحبلا طبرل 628 Bulletin of the World Health Organization | August 2006, 84 (8) Special Theme – Knowledge Translation in Global Health Country-level efforts to link research to action John N Lavis et al. 8. Lavis JN, Davies HTO, Oxman AD, Denis J-L, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy 2005;10 Suppl 1:S35-48. 9. Frenk J. Balancing relevance and excellence: organizational responses to link research with decision making. Soc Sci Med 1992;35:1397-404. 10. Sauerborn R, Nitayarumphong S, Gerhardus A. Strategies to enhance the use of health systems research for health sector reform. Trop Med Int Health 1999;4:827-35. 11. Chunharas S. Linking research to policy and action. In: Neufeld V, Johnson N, editors. Forging links for health research: perspectives from the Council on Health Research for Development. Ottawa: International Development Research Centre; 2001. p. 109-39. 12. Alliance for Health Policy and Systems Research. Getting research into policy and practice. In: Strengthening health systems: the role and promise of policy and systems research. Geneva: Alliance for Health Policy and Systems Research; 2004. p. 51-70. 13. Court J, Young J. Bridging research and policy in international development: context, evidence and links. In: Stone D, Maxwell S, editors. Global knowledge networks and international development. Routledge; 2004. p. 1-24. 14. International Organizing Committee. International conference on health research for development. Bangkok: International Organizing Committee; 2000. 15. International Working Party to Promote and Revitalise Academic Medicine. ICRAM (the International Campaign to Revitalize Academic Medicine): agenda setting. BMJ 2004;329:787-9. 16. Ravensbergen J, Lomas J. Creating a culture of research implementation: ZonMw in the Netherlands. Global Forum Update on Research for Health 2005;2:64-6. 17. Devereaux PJ, Choi PTL, Lacchetti C, Weaver B, Schunemann HJ, Haines T, et al. A systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals. CMAJ 2002; 166:1399-406. 18. Egger M, Smith GD, O’Rourke K. Rationale, potentials, and promise of systematic reviews. In: Egger M, Smith GD, Altman DG, editors. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Books; 2001. p. 3-19. 19. Lavis JN, Posada FB, Haines A, Osei E. Use of research to inform public policymaking. Lancet 2004;364:1615-21. 20. Bravata DM, McDonald KM, Shojania KG, Sundaram V, Owens DK. Challenges in systematic reviews: synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142 (Part 2):1056-65. 21. Pignone M, Saha S, Hoerger T, Lohr KN, Teutsch S, Mandelblatt J. Challenges in systematic reviews of economic analyses. Ann Intern Med 2005;142 (Part 2):1073-9. 22. Chou R, Helfand M. Challenges in systematic reviews that assess treatment harms. Ann Intern Med 2005;142 (Part 2):1090-9. 23. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005;10:45-53. 24. Lomas J. Using research to inform healthcare managers’ and policy makers’ questions: from summative to interpretive synthesis. Healthcare Policy 2005; 1:55-71. 25. Lavis JN, Davies HTO, Gruen RL, Walshe K, Farquhar CM. Working within and beyond the Cochrane Collaboration to make systematic reviews more useful to healthcare managers and policymakers. Healthcare Policy 2006;1:21-33. 26. Lavis JN. Bridging the know-do gap. Hamilton: McMaster University Centre for Health Economics and Policy Analysis; 2004. 27. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003;81:221-48. 28. Canadian Health Services Research Foundation. Is research working for you? A self-assessment tool and discussion guide for health services and policy organizations. Ottawa: Canadian Health Services Research Foundation; 2005. 29. Lomas J. Using ‘linkage and exchange’ to move research into policy at a Canadian foundation: encouraging partnerships between researchers and policymakers is the goal of a promising new Canadian initiative. Health Aff 2000;19:236-40. 30. Lomas J, Fulop N, Gagnon D, Allen P. On being a good listener: setting priorities for applied health services research. Milbank Q 2003;81:363-88. 31. Haines A, Victora C, Horton R. The Lancet’s series on health-systems research: a call for papers. Lancet 2004;363:261-2. 32. Horton R. Medical journals: evidence of bias against the diseases of poverty. Lancet 2003;361:712-3. 33. Rochon PA, Mashari A, Cohen A, Misra R, Laxer D, Streiner DL, et al. Relation between randomized controlled trials published in leading general medical journals and the global burden of disease. CMAJ 2004;170:1673-7. 34. Gruen RL, Morris PS, McDonald EL, Bailie RS. Making systematic reviews more useful for policy-makers. Bul World Health Organ 2005;83:480-1. 35. Canadian Health Services Research Foundation. The theory and practice of knowledge brokering in Canada’s health system. Ottawa: Canadian Health Services Research Foundation; 2003. 36. Atkins D, Fink K, Slutsky J. Better information for better health care: the evidence-based practice center program and the Agency for Healthcare Research and Quality. Ann Intern Med 2005;142 (Part 2):1035-41. 37. Varkevisser CM, Mwaluko GMP, Le Grand A. Research in action: the training approach of the Joint Health Systems Research Project for the southern african region. Health Policy Plan 2001;16:281-91. 38. Andreano R. The international health policy program: an internal assessment. Madison (WI): University of Wisconsin Press; 2001. 39. Alliance for Health Policy and Systems Research. Knowledge for better health systems and better health. The alliance strategic plan: ten year outlook and 2006–2008 plan. Geneva: Alliance for Health Policy and Systems Research; 2006. 40. Hamid M, Bustamante-Manaog T, Viet Dung T, Akkhavong K, Fu H, Ma Y, et al. EVIPNet: translating the spirit of Mexico. Lancet 2005;366:1758-60.