Priorities for pharmaceutical policies in developing countries: results of a Delphi survey.

Priorities for pharmaceutical policies in developing countries: results of a Delphi survey J.-D. Rainhorn,' P. Brudon-Jakobowicz,2 & M.R. Reich3 The use of the Delphi method as a systematic and logical approach to establishing consensus among international experts on the priorities for interventions in national drug policies in developing countries is described. The Delphi survey showed a high degree of reliability, as evidenced by the high response rate, the quality of respondents, and the high standard for consensus. In addition to creating consensus on key issues and key components for priority intervention, the study identified six components that could constitute a basic framework for designing drug policy in developing countries. The study's con- clusions have important implications for decision-makers within international development agencies and national governments. Introduction In the 1970s the lack of basic drugs at prices that the poor could afford became a major concern for health officials in the newly independent states of the devel- oping world. In response, the World Health Assem- bly passed a resolution (WHA28.66) in 1975 that mandated the World Health Organization "to devel- op means by which the Organization can be of great- er direct assistance to Member States in advising on the selection and procurement, at reasonable cost, of essential drugs of established quality corresponding to their national needs". WHO published the first model list of essential drugs in 1977 (1). Based on another World Health Assembly resolution in 1979 (WHA32.4 1), the Organization established the Action Programme on Essential Drugs and Vaccines in 1981 to develop a strategy which embraced all aspects of national drug policies. In the 1980s, a large number of countries adopt- ed essential drug lists with support from develop- ment agencies and nongovernmental organizations (NGOs) and began operating active programmes. Sev- eral countries, including Bangladesh (2), Philippines (3), and Nigeria (4), developed national policies and carried out major changes in the pharmaceutical sector. I Director, Centre de Recherche et d'Etudes sur le Developpe- ment de la Sante (CREDES), Paris, France. 2 Scientist, Action Programme on Essential Drugs, World Health Organization, 1211 Geneva 27, Switzerland. Requests for reprints should be sent to this address. 3 Professor of International Health Policy and Director of the Takemi Program in International Health, Harvard School of Pub- lic Health, Harvard University, Boston, MA, USA. Reprint No. 5476 Despite these efforts, however, serious problems remained in many developing countries, due in part to ad hoc, ineffective, and contradictory pharmaceu- tical policies. In 1988, according to WHO, more than 1300 million people had little or no regular access to the most essential drugs (5). At the international level, the pharmaceutical industry and many developed country governments voiced strong opposition to WHO's efforts to pro- mote policies on essential drugs. At the national level, conflicts arose over the design of pharmaceuti- cal policy and such issues as the role of the private sector and the priority for local production. Conflicts also emerged among international development agencies over where to intervene first in the pharma- ceutical sector and over which strategies were likely to be cost-effective (6). Numerous efforts to resolve these controversies have achieved only limited success. The Conference of Experts on the Rational Use of Drugs, held in Nai- robi in 1985, was a concerted attempt by WHO to create agreement on what should be done (7). Since then, conflicts of opinion have persisted on priorities for action, reflecting the divergent views and inter- ests of the different actors (8). A new approach is therefore required to help formulate drug policies that will meet the challenges of the next decade. This article reports the use of the Delphi technique as a systematic and logical approach to establish consen- sus among international experts on the priorities for interventions in national drug policies. Methods The Delphi technique is a method for structuring communication in a process that allows a group of individuals to deal with a complex problem and Bulletin of the World Health Organization, 1994, 72 (2): 257-264 © World Health Organization 1994 257 J.-D. Rainhorn et al. reach consensus (9). The process involves the use of a series of questionnaires designed by a monitor group and then sent by mail in several rounds to a respondent group of experts who remain anonymous (10). After each round, the results are summarized and assessed by the monitor team and used to devel- op a questionnaire for the next round. The assess- ment document and new questionnaire are then sent to all members who responded. A Delphi survey is considered complete when a convergence of opinion occurs or when a point of diminishing retums is reached (11). A major advantage of the Delphi technique is that it avoids problems commonly encountered in face-to-face group meetings. These problems include the influence of key persons on the responses of other panel members as well as the geographical constraints and costs of bringing together a group of experts. The anonymity of answers allows Delphi participants to express their personal views freely. The method is particularly useful for a subject with strong differences of opinion or high levels of uncer- tainty. The reliability of the Delphi method depends largely on the selection of panel members, the size of the group, and the number of rounds (11). The Delphi method was developed as a forecast- ing tool at the RAND Corporation in 1948 (10), and has been widely used in defence studies and business strategy. The first studies in the health field were published at the end of the 1960s (12). In the phar- maceutical field, the Delphi technique has been used for conceptualizing the future of the pharmacist pro- fession (13), for standardizing terminology (14) and prescription practices (15), and for exploring policy options (16). The pharmaceutical industry has applied the Delphi technique to forecast the evolu- tion of important markets (17), although most of these studies are proprietary and not publicly avail- able. Our review of the literature did not identify any use of the Delphi technique to assess or design drug policies in developing countries. Study design. The study was designed by a monitor group, set up at the Harvard School of Public Health in Boston, USA, who received support from aca- demic specialists and public health experts originat- ing from developing countries. The study was based on a three-step logical approach and was completed over a six-month period. The first step sought to define the main problems faced by developing coun- tries in the pharmaceutical sector, which have been called "key issues". The second step was to identify for each of the key issues those elements of the pharmaceutical system that have a major impact on performance; these have been called "key compo- nents". The third step was to rank both the issues and the components, in terms of importance for intervention, to establish priorities. Respondent group A list of 54 persons with substantial expertise in pharmaceutical policy in developing countries was prepared to serve as the respondent group. As shown in Table 1, the group included persons from different types of institutions: multilateral donors, such as the World Bank and European Economic Community; the United Nations system, such as WHO and UNICEF; nongovernmental organizations; research and consulting groups; pharmaceutical companies; universities; and consultants. Half were pharmacists or physicians, and half were economists, managers, policy analysts, anthropologists, or statisticians. The group of experts represented people from 12 coun- tries on four continents. The questionnaires As shown in Fig. 1, the first questionnaire was devel- oped after a review of the literature related to phar- maceutical policy in developing countries, including unpublished reports and documents from more than 50 countries. The monitor group proposed a prelimi- nary set of five key issues and 40 key components. Table 1: Evolution of the Delphi respondent group by categories: initial and final compositionsa Development UN system Drug Nongovernmental agencies organizations industry organizations Consultants Academics Total Physicians, (5)/4 (5)/4 (1)/1 (5)/5 (6)/5 (3)/3 (25)/22 pharmacists Other (e.g., economists, managers, policy- analysts, statisticians, anthropologists) Total of each subgroupb (3)/2 (8)/6 (3)/2 (8)/6 (7)/4 (8)/5 (4)/i (9)/6 (3)/2 (9)/7 (6)/5 (26)/16 (9)/8 (51)/38 a The initial composition is shown in parentheses. The final composition of respondents in Round No. 3 is shown to the right. b The size of each initial subgroup was nine for a total of 54. Three experts declined the invitation to participate, giving a total of 51 in the survey. WHO Bulletin OMS. Vol 72 1994258 Pharmaceutical policies In developing countries Respondents were asked to review the preliminary lists, indicate their agreement or disagreement, rephrase the proposed formulations, and propose additional issues and components. Three members of the initial group declined to participate in the study. In the Round No. 2 questionnaire, respondents were asked to select the formulation that best defined each key issue, review additional key issues pro- posed in Round No. 1, rank the key issues according to feasibility of intervention and likely impact on the expected outcomes, and select five priority compo- nents for each key issue. In the Round No. 3 questionnaire, respondents were requested to select the best formulation of two key issues not yet decided, rank the major structural constraints confronted by efforts to improve drug sector performance, and rank the key components according to the priority of intervention for solving each key issue. Results Key issues A strong consensus was obtained in Round No. 1 on the five key issues proposed by the monitor team: "Structural constraints in the country" (91.1% of respondents), "Lack of government commitment" (86.7%), "Structural weakness of the drug public sec- tor" (84.4%), "Limited drug affordability" (82.2%), and "Irrational use of drugs" (100%). The initial for- mulation of two key issues was accepted with strong agreement: "Structural constraints in the country" (61.9%) and "Irrational use of drugs" (73.3%). New formulations were proposed in Round -No. 2 for the remaining three, taking into account the suggested rephrasing and comments from respondents. A total of 40 additional key issues were pro- posed by the respondents, along with 10 comments regarding alternative key issues. Those most fre- quently mentioned were: the behaviour of the private sector (17.8%), the role of the multinational pharma- ceutical industry (13.3%), the underestimated role of traditional medicine (8.9%), the lack of foreign exchange (6.7%), and the poor efficiency of inter- national aid (6.7%). These five were proposed in Round No. 2 as possible new key issues or compo- nents. In Round No. 2, 59.0% of respondents accepted the formulation "Limited affordability of essential drugs in both public and private sectors". New for- mulations, reflecting comments from respondents, were proposed for the remaining two key issues in Round No. 3. The proposal to add a sixth key issue was rejected because none of the proposed additional key issues received more than 20.5% agreement. A majority (62.5%) responded that the first key issue to address for improving drug sector performance is the problem of government commitment (with a mean rating of 5.38 out of 6). Respondents ranked the weakness of the public drug sector as the second pri- ority issue (with a mean rating of 4.23) (Table 2). In Round No. 3 a consensus emerged on the for- mulations that best defined the remaining two key issues: "The lack of government commitment and capacity to design and implement a rational drug policy" (97.4%), and "The structural weakness and poor performance of the public drug sector" (57.9%). Key components In Round No. 1, the respondent group added another 119 items to the preliminary list of 40 components prepared by the monitor team. Those mentioned at least three times were included on a revised list, resulting in a total of 46 key components. In Round No. 2, respondents were asked to select five priority components for each key issue. Across key issue categories, six components were mentioned as priorities by more than 70% of the respondents: the establishment of appropriate legisla- tion and regulation, the selection of essential drugs, the allocation of sufficient funds for drugs in the health budget, the improvement of procurement pro- cedures, the establishment of a drug pricing policy, and the organization of continuing education pro- grammes in drug use (Table 2). In Round No.3, a consensus emerged that the major constraints for achieving the objectives of availability and affordability of essential drugs for the majority of the population are the public sector's capability, the health human resources availability, and the country's socioeconomic level. A consensus also emerged for each key issue on the three top- ranked key components for intervention (Table 2). Discussion The study's results are discussed below from three perspectives: the reliability in establishing consen- sus, the priority issues for policy, and the priority components for action. Reliability The high rate of agreement obtained on most ques- tions in this survey indicates that a strong consensus emerged on the priorities for intervention in national drug policies. Systematic application of the Delphi WHO Bulletin OMS. Vol 72 1994 259 J.-D. Rainhorn et al. Fig. 1. The Delphi method: summary of major actions taken by the monitor group and the Delphi participants. ACTIONS BY MONITOR GROUP ACTIONS BY DELPHI PARTICIPANTS Round No. 1: Take position on key issues, propose additional issues and write comments Propose additional components Round No. 2: Analyse responses Prepare assessment document No. 1 Prepare second questionnaire Round No. 3: [45/51 = 88.2% response] Select key issues Rephrase and rank key issues Select key components [40/45 = 88.9% response] Analyse responses Prepare assessment document No. 2 Prepare third questionnaire Rank major structural constraints Rank key components for each issue Select priority components for action [38/40 = 95% response] (overall 38/51 = 74.5% response) Final review: Analyse responses for three rounds Prepare final paper --O- Comment on final paper WHO Bulletin OMS. Vol 72 1994 Propose key issues and key components Prepare first questionnaire and background papers on Delphi method I mi~~~- op -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 260 Pharmaceutical policies in developing countries Table 2: Selection and ranking of key components In Round No. 2 and Round No. 3. All components mentioned as a priority in Round No. 2 by more than 50% of the respondents are listed for each key issue. The top three components are ranked according to the results of Round No. 3. Key issues! Mean rating for ranking % of respondents in agreement with key components: key components of each key issuea <50% 50-59% 60-69% >70% A. "Lack of government commitment and capacity to design and implement a rational drug policy" [5.38] 1. Legislation/regulation 2. Essential drug selection 3. Drug allocation in the health budget -Drug control authority + B. "Structural weakness and poor performance of the public drug sector" [4.23] 1. Procurement procedures 2. Essential drug selection + 3. Health human resources development + -Distribution/logistics + C. "Irrational use of drugs" [3.61] 1. Continuous education (drug use) 2. Prescribing practices + 3. Essential drug selection + -Pharmaceutical firms marketing + -Public information/education + D. "Limited affordability for essential drugs in both public and private sector" [3.23] 1. Essential drug selection + 2. Drug pricing policy 3. Procurement procedures + -Drug financing policy + -Generic drugs + E. "Structural constraints in the country" [2.97] 1. Public sector capability + 2. Health human resources availability + 3. Country socioeconomic level + -Foreign exchange + - Transport facilities + a In Round No. 2 respondents ranked the key issues on a scale of 1 to 6, with 6 representing the key issue considered most important according to feasibility of intervention and likely impact on the expected outcomes. ** The six key components (that received more than 70% agreement and were top-ranked for key issues A, B, C, and D) could consti- tute a basic framework for designing pharmaceutical policy. method by the monitor group circumvented contro- versies that have long affected this field. A review of several factors suggests that the consensus obtained is reliable. The survey was designed to meet the criteria of the classical Delphi method (9). The monitor team carried out three rounds of questionnaires, which is standard in most surveys and considered to be suffi- cient (9). Members of the respondent group were all well informed on the subject, and the size of the group reached acceptable standards for a Delphi sur- vey (18). The literature reports that once an adequate size is reached (generally around 30 well-informed and motivated experts), few new ideas result from increasing the group's size (19). The response rate for the three rounds was very high (74.5% of the initial group), compared with figures reported in other Delphi surveys. Finally the survey used a high cutoff rate, compared with other Delphi surveys, to signify the achievement of consensus. The respondent group was selected to represent diverse disciplines, various countries and social envi- ronments, and constrasting views on pharmaceutical policy, and the proportions remained reasonably con- WHO Bulletin OMS. Vol 72 1994 261 J.-D. Rainhorn et al. sistent through all three rounds. The group remaining at the survey's end (after removing the 25.5% of dropouts) had approximately equal representation among the subgroups, which strengthens the survey's reliability. The higher rate of responses among physicians and pharmacists could reflect a greater involvement in pharmaceutical questions (Table 1). The three rounds of questionnaires allowed respondents to propose their own key issues and components, which were used by the monitor team for proposing alternative choices. The comments from respondents and their proposals for 40 addition- al key issues and 119 additional components suggest that the group used the questionnaires to express divergent opinions. These factors, combined with the high rate of agreement, suggest that overspecifica- tion of the initial issues was not a serious problem. The monitor team used the assessment docu- ments to present disagreements and minority views expressed by respondents, and these documents were sent to respondents in Rounds No. 2 and No. 3. The continuing high rate of responses, which reflects the lack of discouraged dissenters who would otherwise have dropped out, also contributes to the reliability of the consensus. Priority issues The key issues identified by the Delphi survey have been previously described in the literature on phar- maceutical policy as important problems faced by developing countries. This study makes an important contribution in establishing the following five key issues as priorities. (1) Lack of government commitment and capacity to design and implement a rational drug policy. A majority of respondents ranked this key issue as top priority for attention. This result emphasizes the crit- ical role of government in defining a drug policy, setting objectives, and implementing strategies. Agreement on this key issue supports the idea that a comprehensive drug policy is necessary to achieve significant progress. A WHO situation analysis of more than 30 African countries in 1990 reached a similar conclusion.a (2) Structural weakness and poor performance of the public drug sector. In the past ten years, many aid projects have attempted to improve the perfor- mance of the public drug sector. But repeated fail- ures have convinced some agencies to decrease the public sector's role and to advocate a greater role for a Bilan de la situation pharmaceutique des pays africains. Geneva, WHO Action Programme on Essential Drugs, 1990 (unpublished report, in French only). the private sector in the poorest countries (20). This study's consensus suggests that the public sector still needs to be strengthened in order to ensure drug availability. (3) Irrational use of drugs The survey's 100% agreement on this key issue reflects growing aware- ness that the impact of cost-effective procurement and logistics systems will be significantly under- mined if drugs are not prescribed and used correctly. To date, only limited action has been taken in a few countries to address problems of irrational use. (4) Limited affordability for essential drugs in both public and private sectors. Consensus on this key issue may reflect two broader concems in the inter- national community: first, that drugs in the private sector are often unaffordable for poor people in developing countries; and second, that recent efforts to promote cost-recovery schemes in the public sec- tor through sales of drugs could similarly restrict affordability of essential drugs for many poor people and could have adverse distributional effects. (5) Structural constraints. A strong consensus emerged that broader structural constraints in the country need to be considered, even though pharma- ceutical policy cannot resolve such constraints. This consensus reflects increasing recognition that struc- tural constraints contribute to the failure of many intemational aid projects (21). For those poor coun- tries that are too weak to sustain an effective drug policy on their own, interventions are necessary not only on pharmaceutical matters but also on national conditions outside the drug sector. Priority components Six key components were mentioned as priorities for action by more than 70% of the respondents in Round No. 2 and were top-ranked as interventions for solving each key issue in Round No. 3. In the past, some countries and donor agencies have addressed several of these components but rarely in a comprehensive policy package. This study suggests that future efforts to design drug policy and improve pharmaceutical conditions need to include all six components as a basic framework (with other com- ponents added according to each country's specific conditions). (1) The establishment of appropriate legislation and regulation was ranked as the first priority for gov- emnment action. Most developing countries and the donor community have underestimated the problems associated with implementation of national policies. By ranking this component first, the respondents acknowledged the importance of translating policy into legislation and regulatory structures. 262 WHO Bulletin OMS. Vol 72 1994 Pharmaceutical policies in developing countries (2) The selection of essential drugs was identified as a priority component for four key issues, reflecting strong support from the respondent group. Essential drug lists have been recognized since 1977 as a major tool for improving the drug situation in devel- oping countries, especially when combined with other policy elements. The results of this survey con- firm that the use of such lists is still a basic element of appropriate drug policy in developing countries. (3) The importance of maintaining a significant drug allocation in the health budget was also empha- sized by the respondent group. In recent years, the combined consequences of economic crisis and structural adjustment programmes have frequently led to a decrease in drug budgets. The survey sug- gests that the state should remain financially involved in pharmaceutical provision. (4) The improvement of procurement procedures was ranked as an important priority for addressing the poor performance of the public drug sector and for improving drug affordability. Small additional efforts in the area of procurement procedures can yield substantial improvements and savings.b But most developing countries have few pharmacists with an expertise in purchasing low-cost drugs of good quality in the international market, and interna- tional aid projects have not generally focused on this problem. (5) The establishment of a drug pricing policy was selected as a priority component for improving the limited affordability of essential drugs. This agree- ment suggests that pricing mechanisms should be introduced to ensure that everybody - even the poorest - has access in both private and public sec- tors to the drugs they need. (6) Continuing education programmes have been recognized as an important intervention for promot- ing effective drug use (22), although they are often difficult to implement. The respondents' selection of this component reflects a need for greater efforts to understand prescribing behaviour and identify meas- ures to alter it in developing countries. The strong consensus reached in this survey argues that decision-makers in both international development agencies and national governments should adhere to the six priority components in designing and evaluating projects concerned with pharmaceutical policy. It is important to note, how- ever, that this Delphi study did not produce agree- ment on the policy content of each component. b Financing essential drugs: report of a WHO Workshop, Harare, 1988. Unpublished WHO document, WHO/DAP/88.10, 1988. These often difficult choices need to be made at the national level. Within individual countries, the Del- phi technique could be used to specify the contents of national drug policy and create consensus among local interest groups (including consumer associa- tions) involved with the pharmaceutical sector. Acknowledgements The survey was sponsored by the Centre de Recherche et d'Etudes pour le DNveloppement de la Sant6 (CREDES), Paris, France, the WHO Action Programme on Essential Drugs, and the Takemi Programme in International Health, Harvard School of Public Health, Boston, MA, USA. The authors thank the participants of the Delphi group, and also the late U. Brinkmann, L. Chen, M. Garenne, D. Ross- Degnan, and R. Vaurs for comments on the manuscript. Resume Etablissement des priorites pharmaceutiques dans les pays en developpement: r6sultats d'une enquete Delphi Dans les ann6es 70, le manque de medicaments de base a des prix abordables pour les popula- tions les plus d6munies est devenu un grave sujet de pr6occupation pour les nouveaux pays ind6- pendants du monde en developpement. En d6pit des progres realis6s par certains d'entre eux et du developpement de I'aide internationale, I'accessi- bilite aux medicaments essentiels continue de poser un probleme, notamment dans les pays les plus pauvres. Les d6saccords qui ont persist6 sur ces questions au cours des annees 80 et I'absen- ce de consensus international sur les interven- tions prioritaires ont contribu6 a freiner la concep- tion et la mise en ceuvre de politiques pharma- ceutiques appropri6es. Cet article rend compte de l'utilisation syst6- matique et logique de la technique Delphi pour d6gager un consensus parmi les experts interna- tionaux sur les objectifs prioritaires des politiques pharmaceutiques nationales dans les pays en developpement. Le groupe interrog6 initialement comprenait 54 experts internationaux appartenant a divers types d'institutions et ayant des vues dif- f6rentes sur la politique pharmaceutique. L'enque- te a ete menee selon une approche logique en trois etapes. Elle visait a identifier les principaux problemes auxquels sont confrontes les pays en developpement dans le secteur pharmaceutique ("questions cl6s"), a d6terminer, pour chaque question cle, les 6l6ments du secteur pharmaceu- WHO Bulletin OMS. Vol 72 1994 263 J.-D. Rainhorn et al. tique qui ont une incidence majeure sur ses per- formances ("el6ments cles") et a classer ces questions et ces 6l6ments en fonction de l'impor- tance des interventions, de fagon a etablir des priorit6s. Le taux de r6ponse eleve, la qualite des parti- cipants et le degre de consensus exige ont confe- re aux resultats de l'enquete un haut degre de fia- bilite. L'ampleur de I'accord atteint sur la plupart des questions montre qu'un consensus tres fort s'est degag6 sur les priorites des politiques phar- maceutiques nationales. L'application syst6ma- tique de la technique Delphi a permis d'eviter les controverses que l'on observe depuis longtemps dans ce domaine. L'6tude a apport6 une contribu- tion importante en demontrant le caractere priori- taire des cinq questions cl6s suivantes: absence de volont6 et incapacit6 des gouvernements a concevoir et a mettre en ceuvre une politique pharmaceutique rationnelle; utilisation irrationnelle des medicaments; difficult6s pour se procurer les medicaments essentiels, tant dans le secteur public que dans le secteur prive; contraintes struc- turelles. Plus de 70% des participants ont egale- ment mentionn6 six 6l6ments cl6s qui, selon eux, justifiaient une intervention prioritaire: 6tablisse- ment d'une legislation et d'une r6glementation appropri6es; choix des m6dicaments essentiels; maintien d'un poste "m6dicaments" important dans le budget de la sante; am6lioration des pro- cedures d'achat des m6dicaments; 6tablissement d'une politique de fixation des prix; poursuite des programmes d'education. Les conclusions de l'6tude ont des implica- tions importantes pour les d6cideurs au sein des agences de d6veloppement internationales et des administrations nationales. Cette 6tude offre en effet un cadre g6neral pour la conception d'une politique pharmaceutique dans les pays en deve- loppement. 11 importe toutefois d'observer qu'elle n'a pas permis d'arriver a un accord sur la place de chaque element dans la politique g6nerale. 11 s'agit la d'un choix souvent difficile qui doit etre fait au niveau national. Dans chaque pays, la technique Delphi pourrait etre utilisee pour speci- fier le contenu de la politique pharmaceutique nationale et cr6er un consensus parmi les groupes d'interets locaux (y compris les associa- tions de consommateurs) qui oeuvrent dans le secteur pharmaceutique. References 1. The selection of essential drugs. Report of a WHO Expert Committee. Geneva, World Health Organiza- tion, 1977 (WHO Technical Report Series, No. 615). 2. Report of the Expert Committee for Drugs - Ban- gladesh. World development, 1983, 11: 251-257. 3. The Philippine national drug policy: a primer. Mani- la, Department of Health, 1987. 4. National drug policy for Nigeria. Lagos, Federal Min- istry of Health, 1990. 5. The world drug situation. Geneva, World Health Organization, 1988. 6. Reich MR. Essential drugs: economics and politics in international health. Health policy, 1987, 8: 39-57. 7. The rational use of drugs: Report of the Conference of Experts, Nairobi. Geneva, World Health Organ- ization, 1987. 8. Chetley A. A healthy business? World health and the pharmaceutical industry. London, Zed Books, 1990. 9. Lindstone HA, Turoff M, eds. Introduction. In: The Delphi method: techniques and applications. Read- ing, MA, Addison-Wesley, 1975: 3-12. 10 Lindeman CA. Priorities within the health care system: a Delphi survey. American Nurse's Associa- tion, 1981, 5: 1-49. 11. Fink A et al. Consensus methods: characteristics and guidelines for use. Am. j. public health, 1984, 74: 979-983. 12. Farrell P, Scherer K. The Delphi technique as a method for selecting criteria to evaluate nursing care. Nursing papers, 1983, 15(1): 51-60. 13. Gourley RD et al. ASHP members' concepts of institutional pharmacy in the year 2000. Am. j. hosp. pharm., 1985, 42: 96-101. 14. Rinaldi RC et al. Clarification and standardization of substance abuse terminology. J. Am. Med. Assoc., 1988, 259: 555-557. 15. Thompson DF, Heflin NR. Frequency and appropri- ateness of drug prescribing for unlabeled uses in pediatric patients. Am. j hosp. pharm., 1987, 44: 792-794. 16. Jillson IA. The national drug-abuse policy Delphi: progress report and findings to date. In: Lindstone HA, Turuff M, eds. The Delphi method: techniques and applications. Reading, MA, Addison-Wesley, 1975: 124-159. 17. Rohatgi K, Rohatgi PK. A Delphi study on health in future India. Journal of scientific and industrial research, 1980, 39: 359-363. 18. Pineault R, Daveluy C. L'approche par recherche du consensus. In: La planification de la sante: con- cepts, m6thodes et strat6gies. Montreal, Agence d'ARC, 1986: 226-246. 19. Delbecq AL, Van de Ven A, Gustafson DH. Group techniques for program planning. Glenview, IL, Scott, Foresman, 1975. 20. The World Bank. World development report 1991. The challenge for development. New York, Oxford University Press, 1991. 21. Reich MR, Marui E, eds. International cooperation for health: problems, prospects and priorities. Dover, MA, Auburn House, 1989. 22. Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: a critical review of the experimental literature. Milbank Memorial Fund quarterly, 1990, 67: 268-317. 264 WHO Bulletin OMS. Vol 72 1994