Evaluation of health financing reforms : report on a consultation 6-8 June 1995, Geneva, Switzerland

WHO/SHSINHP/95.10 Distr. LIMITED English only

EVALUATION OF HEALTH FINANCING REFORMS

REPORT ON A CONSULTATION 6-8 June 1995 Geneva

WORLD

HEALTH

ORGANIZATION

DIVISION OF STRENGTHENING OF HEALTH SERVICES NATIONAL HEALTH SYSTEMS AND POLICIES UNIT 1995

Table of Contents

INTRODUCTION .............................................. . SUM.MAR Y OF DISCUSSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. NEXT STEPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. ANNEXES 1. Evaluation of Health Financing Reforms: Abolition of User Fees in the Rural Health Centres of Zimbabwe Evaluation of User Fee Schemes in Two Districts in Uganda Financing Reforms for the Health Care Scheme in Thailand Mexico: Use of Charges in the Tertiary Level of Attention Evaluation of Health Care Financing Reform in Kyrgyzstan Ghana Health Financing Reform Evaluation Health Financing Reform: A Framework for Evaluation Methods for Evaluating the Effects of Health Financing Reforms Methods for Evaluating the Effects of Finance Reforms and Strategies for Institutionalizing Evaluation in the Policy Making Process: the Case of Niger (copies of transparencies used for presentation) Stakeholder Analysis: A Vital Tool for Strategic Managers Agenda for the Consultation List of Participants in the Consultation '1

7

2. 3. 4. 5. 6. 7. 8. 9.

10. 11. 12.

2

SUMMARY OF DISCUSSIONS Framework for Evaluation of Health Financing Reforms 1. Concerns were raised that the framework might be too narrow in terms of the reforms to which it may be applied. Health financing should be interpreted broadly to include community financing, internal markets, and policies to increase the role of the private sector (e.g., contracting out) in addition to user fees and health insurance. 'W"hile several country representatives said that the framework is relevant and useful for evaluating their reforms, the 12-18 month time frame proposed in the document might be too short to evaluate the full effects of reform. It was noted that this is a somewhat arbitrary time frame chosen as the period during which evaluations would be supported by this activity. However, it is hoped and expected that evaluative work will continue and indeed, be fully integrated into the routine policy making processes of ministries of health by the end of this period.

2. There is a need for a periodic low-cost approach to evaluation rather than a reliance on a series of ad hoc activities, such as occasional high-cost, donor-funded household surveys. 3. The framework should not lead to an assessment of health financing reforms in isolation from measures to reform public finance more generally. 4. It was suggested that there is a need for new tools to be used to set national priorities for reform. These tools should measure the burden of disease, incorporating assessments of morbidity, mortality, and disability. A system of national health accounts is needed to assess financial flows, bu!> the nc;eded data are often lacking. ~ew strategies are also needed to integrate the ideas of an essential p'ackage of clinical interventions with a comprehensive primary health care approach. 5. It is important that the framework and evaluation work supported under this activity serve to strengthen existing information and management systems instead of encouraging the establishment of parallel systems.

Indicators, Data Sources, Methodologies, Capacity Building. and Institutionalization 6. In the presence of mUltiple reforms, a decision must be made to either evaluate the combined effects of the package of reforms or to attempt to disentangle the separate effects of each specific measure. The choice depends on the proposed use of the evaluation and the methodological feasibility of identifying separate effects. It may not be necessary to fully dissect the observed relationships, but the evaluation should enable an understanding of the process by which change has occurred. To the extent possible, however, evaluators should attempt to minimize the consequences of potential confounding variables by identifying the key indicators and using simple and specific means to control for third factors.

INTRODUCTION A consultation was held on 6-8 June, 1995, at WHO Headquarters in Geneva, to support national capacities to evaluate the effects of health financing change on sectoral objectives. Financing issues are often at the core of national health sector reform agendas, and many multilateral and bilateral assistance agencies are encouraging Ministries of Health to implement user charges, to make changes in the public/private mix of financing and provision, health insurance and different provider payment options. There is however, relatively little empirical evidence showing the effects of specific measures on equity, efficiency, sustainability, quality, and acceptability to clients and providers. The purpose of this initiative is to provide a means for health sector decision-makers to monitor. analyse and improve health financing policies. This process should also strengthen national capacities to monitor and evaluate ongoing reforms. and to generate demand from decision-makers to incorporate this type of analytical input into the policy process. Internationally, the findings of the evaluations in each country will contribute to the growing body of li~erature on health care financing and, by doing so. to improved policies and outcomes in a number of countries. The consultation included policy makers and researchers from six countries. plus consultants, representatives of donor agencies, and WHO technical staff. Each country participating in the consultation has recently, or will imminently, implement health financing reforms on a national, regional, or pilot basis. The country participants included one policymaker and, in most cases, one other person, either a researcher or policymaker. A participant from each of the countries is expected to be the principal investigator for the evaluations to commence in each country based on the protocols developed during the consultation. ,t is anticipated that these evaluations will take place over a 12 to 18 month period following the introduction of the reform. . The meeting began with a presentation and discussion of a draft framework for the evaluation of health financing reforms. This was followed by presentations by each of the six countries on the reform to be evaluated and initial ideas on data sources and methods. The first day concluded with technical presentations on indicators. data sources, methodologies, and strategies for integrating evaluation into the n... ,lOnal policy making process. The second day and the beginning of the third day were used by each country team to develop its initial ideas for evaluation into an interim proposal. Each of the six national proposals were presented and discussed in plenary during the third and final day of the consultation. The meeting concluded with an open discussion of the process of WHO support and ideas for next steps to ensure that the work continues In each country. This report consists of a summary of the discussions during the consultation and the proposed next steps to finalize the proposals to evaluate financing reforms in each of the six countries. The interim proposals are included as annexes, as are the background papers on the framework for evaluation of financing reforms and on indicators, data sources, evaluation methods, and strategies for institutionalizing evaluation in the policy making process. The agenda for the consultation and the list of participants are also included as annexes to this report.

.., )

7. The evaluations should emphasize the use of existing information systems and should consider how targeted research activities could strengthen these systems. It was noted that some of these systems are not within the jurisdiction of ministries of health. and attempts should be made to coordinate evaluative work with other relevant agencies (e.g .. central statistics offices and ministries of planning). 8. It was noted that the discussions focused on methods and data sources for assessing outcomes, yet it is important to also develop approaches to assessing the process of reform and the role of key stakeholders in that process. Much of the analysis being conducted currently as part of the inter-country study of decentralization managed by NHP focuses on process and stakeholders, and methods developed for that study might also prove useful for similar analyses needed for the evaluation of financing reforms. A description of stakeholder analysis is included as Annex 10.

Zimbabwe Proposal to Evaluate Reform 9. The Zimbabwe team plans to evaluate two recently implemented measures designed to increase use of primary level health facilities: the elimination of user fees at rural health centers and a waiver of fees in public hospitals for patients referred from lower levels of the health system. Several questions were raised. What indicators will be used to evaluate these specific measures? Will any other measures be implemented to increase utilization of rural health centers (e.g., quality improvements)? 10. To assess the whether the fee waiver has any impact, the team suggested that the extent to which people still jump the queue and self-refer to hospitals could be measured. Other indicators will be developed during a national planning workshop. Complementary measures are being implemented to improve quality and thus increase demand for care at rural health centers. These include improving management capacity at facility and district level and strengthening management and transportation systems for drugs and other supplies.

Uganda Proposal to Evaluate Reform 11. The Uganda team plans to study the effects of user fees in a range of facilities in two districts with a view to the development of a national health care financing strategy. Because of the extent of decentralization in Uganda, districts may have very different systems of user charges, indicating that much can be learned from a comparative analysis. Facility-based surveys of patient origin were suggested as one method to generate information on the relationship between distance/time costs and facility utilization. The following questions were asked: How will the study districts be selected? What indicators of health improvement will be used? What criteria should be used to identify tracer conditions that might be used to assess the health effects of the financing policies? How will potentially confounding factors that might cause changes in indicators but be unrelated to user fees be controlled for? 12. Districts in different regions and with different user fee policies will be selected. Selection will also depend on the commitment of the district management team to support evaluation. The team will consider whether they wish to include indicators of

4 potential health effects of reform. These might be broad indicators (e.g .. maternal or infant mortality rates) or measures relating to specific tracer conditions likely to be affected differently by different policy with regard to user charges.

Thailand Proposal to Evaluate Reform 13. The Thailand team proposes to evaluate a package of recent reforms intended to expand the population coverage and improve the financial sustainability of its voluntary "health card" insurance scheme. It was noted that the team's proposal contains a number of specific research components under a broad umbrella of activities. and that there may be a need to prioritize these specific components. The need to assess the incremental administrative costs of this reform and its evaluation (i.e., the management and information systems needed for monitoring and evaluation) was also noted. 14. The team gave two reasons for the broad scope of the proposed activities. First. a broad approach was needed to incorporate the evaluation intQ the routine information system of the government and to strengthen that information system. Second. a broad national study might prove very valuable for other countries and for WHO. Indeed. it was noted during the discussion that the Thai health card scheme has had considerable influence and that, for example, several African countries have tried to introduce similar schemes. Administrative costs will be addressed in the evaluation. The cost-recovery ratios to be calculated will include such costs, for example. However, there is a need to define mme specifically the elements to be included in an assessment of incremental administrative systems and costs.

ilvfexico Proposal to Evaluate Reform 15. The Mexican team plans to evaluate the effects of a demonstration project designed to reduce government expenditure on the tertiary level facilities of the National Institutes of Health (NIH), enabling this spending to be reallocated to more costeffective services. The pilot project will introduce a package of measures to increase the share of costs funded by users of the National Institute of Cardiology, a 5.500 bed facility. The measures will allow public doctors to practice privately at the Institute during the afternoon, when the ambulatory services are currently under-utilized. By charging wealthier patients in the afternoon, it is hoped that some of the funds generated can cross-subsidize care for poorer patients in the morning. A series of management and quality assurance interventions is also proposed to improve the performance of the facility. 16. A number of questions were raised about this planned reform and its evaluation. How will the number of non-paying attenders in the mornings be limited so that they do not spill over into the afternoon? How will all of the staff be given an incentive to work during the afternoons? Might the mechanism proposed for paying providers (i.e .. fee-for-service) lead to cost increases, so that efficiency may worsen even if government spending is reduced? Relatedly, might the increase in price in a prestigious and high quality public facility lead to price increases in private facilities that provide some similar services?

5 17. The team noted that most doctors in Mexico work in the public sector in the morning and conduct their own private practice in the afternoon. The plan is to provide high quality private services at the Institute with prices 20 percent below those of the private sector. An incentive system must be created from the revenues raised that rewards all levels of staff, not just the doctors. It was noted that in Thailand. establishment of private practice in public hospitals had the salutary effects of stopping/slowing the brain drain of skilled staff from the public sector and increasing patient satisfaction. However, utilization by the insured population increased considerably, as did total health expenditures, raising questions about the overall efficiency effects of this reform. It also provided people with a greater opportunity to bypass intended referral channels. The Mexican team will try to address these concerns in its reform· and evaluation plans.

Kyrgyzstan Proposal to Evaluate Reform 18. The Kyrgyz team plans to evaluate one pilot project and two national level reforms. The pilot project will be implemented in the oblast (i.e., province) of IssykKul and involves a combination of social health insurance, provider payment reform. • and a new primary care system of small group practices. The collection of contributions is scheduled "to 'begin in August 1995. At the national level, the two reforms to be introduced in 1996 will be ~ formal system of user charges and new mechanisms for allocating government resources to health services. The discussion focused on the demonstration project in Issyk-Kul. Questions included: What will government's role be in funding health services after insurance is implemented? Will there be other reforms to improve quality o~ otherwise encourage people to contribute to the scheme? Is the interpretation of some of the efficiency indicators, such as referral rates and average length of stay, ambiguous? It is envisioned that the government will continue to play a major role in funding 19. health services by contributing to the Basic Health Insurance Fund on behalf of the nonemployed population. Many concurrent reforms are planned in order to improve quality and efficiency. For example, the new primary care centers (APTKs) will be staffed by of a gynaecologist, pediatrician, internist. several practice nurses, and support staff to enable the provision of high quality first contact care in a country characterized by a very high physician:population ration but a near absence of general practitioners. For purposes of the evaluation, it will be important to consider the reforms being put into place in Issyk-Kul as an integrated whole rather than as separate components. In Kyrgyzstan, indicators such as referral rates, admission rates, and average length of stay are so high that they clearly need to be reduced. Thus, downward movement in any of these indicators will reflect unambiguous improvements in efficiency. One concern, however, is the accuracy of the reported historical data. Thus, it will be useful to validate the centrally reported data with some on-site reviews.

Ghana Proposal to Evaluate Reform 20. The Ghana team plans to evaluate planned changes to its national user fee system. The changes include an increase in fee levels for the first time since 1985 and a revision of exemption categories to limit the number on non-poor persons who are

6 exempted (especially Ministry of Health staff). Several questions relating to the reforms and their evaluation were raised. Will the new fee system be more dynamic than that put into place in 1985 so that prices and exemptions can readily adjust to changes in inflation? How can this be done, so that fees can be a more effective policy instrument? What is the relationship between work that has been done on costing services and the new fee schedule? How can the proposed indicators be made more operational? How can the evaluation of process highlight the role of discretionary decisions. which might give rise to inefficiencies or corruption? Can an organizational framework be created to assess why some facilities are collecting and using revenues, while others are not? What incentives to patients have been introduced to encourage appropriate use of the referral network? 21. The new fee schedule is related to a costing exercise that was recently undertaken. The new fees are based on the assumption that government will cover the costs of capital and of personnel. For other operating costs, the percentage covered by the fee ranges from 100 percent (e.g., for drugs) to 15 percent. Regular costing work could prompt more regular adjustment of fees, but there is no guarantee that this will happen, as future fee increases will still require Parliamentary action. The general indicators identified in the interim proposal will be operationalized in specific terms during the next few months, when the proposal will be finalized. Government is now more active pursuing accountability for the collection and use of revenues. The new fee schedule should also reduce the potential for discretionary practices, because it was the failure to increase prices in the face of inflation that left room for informal charges to be expanded. The fee schedule encourages appropriate use of the referral system by charging lower fees in rural than in urban health centers, and the level of fees increases by level of facility. An additional fee is charged for persons who self-refer to higher level hospitals. Summary Discussion 22. The framework was considered a useful tool for thinking about reform and developing prQPosals for evaluation. While much remains to be done in terms of developing appropriate indicators, the proposals demonstrate the extent to which routine data sources can often used to generate needed information. It was suggested that the evaluation studies will contribute to policy making and health information systems. and will build capacity for monitoring and evaluation. 23. It is useful to have a general framework of national reform efforts and then to locate the specific reform to be evaluated within that mapping. This might enable us to learn from both the national experience as well as from specific activities. 24. It will be important for the evaluations to identify, as specifically as possible. what worked (in terms of the reform), and why, and what failed to work. and why. Comparative research may help to identify the enabling set of circumstances that condition the performance of specific reforms. This relates to the sixth component of the framework that focuses on synthesizing the conditions, in addition to the specifics of the reform, that effect the outcomes of policy change. However, it was pointed out that identifying these conditions may not always be possible.

7

NEXT STEPS 25. A number of participants raised questions about the activities that would follow upon the consultation to ensure that the evaluation work proceeded in each country. First of all. steps will be taken to have the framework translated into Arabic. French. Russian, and Spanish. Most of the tCJIl1S wanted additional time to prepare their final proposal. NHP guaranteed to provide financial or technical support to ensure the development of fully fledged proposals for each country. When the proposals are finalized, they will be shared among resource people and the other participants in the consultation for review and comment. 26. NHP intends to support the evaluative work in each country but cannot guarantee to provide full financial support for implementing each of the six country proposals. In some cases, NHP will be able to provide full support for one or more pieces of country work. In other cases, NHP will provide partial support and work with the country to identify donors who would be interested in providing funds for the evaluations. This has already been done for some of the countries. In the revised proposals. it will be helpful to indicate other potential sources of funding. Ghana

27. Conduct a national workshop to assemble data requirements, plan data collection procedures, establish a timetable for implementation of studies, prepare training plans. and finalize protocols. NHP will provide financial support for the workshop and finalization ofc the proposal. Previous assessments supported by NHP of the effects of user fees in Ghana will be drawn upon to assist with the finalization of the evaluation protocols. It is planned to complete the proposal by July 1995, with the national workshop to follow in August. Kyrgyzstan

28. A workshop on health care reform is scheduled for the end of June 1995. and the evaluation team will use this occasion to disseminate the proposed evaluation of financing reforms in Kyrgyzstan. NIiP staff will be present at this workshop and will work with the Kyrgyz team to finalize the evaluation protocols by early July. The final proposal will be presented to the Kollegya, the decision making of the Ministry of Health, before the end of July. Mexico

29. The evaluation proposal will be completed by July/August 1995. This will be followed by a startup workshop to build consensus for the planned evaluation of the reform. The evaluation will commence with two months of activities prior to the implementation of the reform to allow for the definition and measurement of key variables during a baseline period.

8

Thailand 30. The proposal to evaluate reform will be completed during August 1995. This will occur just after a national workshop is held to build consensus for the planned evaluation and to finalize the proposed methodology for evaluation. NHP will provide financial support for the national workshop.

Uganda 31. Prior to finalizing the proposal, the evaluation team will review numerous documents that have been prepared on user fees in Uganda from the ministries of health and local government, by national experts. and by donor agencies. Co-principal investigators have been identified, one medically qualified, the other a health economist. The Child Health and Development Centre, an interdisciplinary research unit of Makerere University. will provide technical backup. NHP will support the document review and proposal preparation, the latter to include a one-week national workshop to finalize the draft evaluation protocols. NHP will send an international consultant to Uganda in September 1995 to review the final draft proposal. Zimbabwe

32. The evaluation team will be comprised of a principal investigator from the Ministry of Health's Department of Finance, Planning, and Administration, assisted by a core group of four people representing the Ministry's health information system. Maternal and Child Health Department, Finance Department, and Planning Department. NHP will support the process of revising the evaluation proposal, which is scheduled to be completed by the end of July 1995. Before that time, the districts and facilities to be included in the evaluation will be identified.

Annex 1

~~ALCA:!CS OF HEALTH F:SANCrS~ REFaR~S Abolltlon of user Fees: Rural Health Cent:e - ZI~EAB~~

1.

Statement of the problem

\Vhy reforms being implemented: referral system needed improvemer:t why did it need to be improved: primary health facilities are underutili:,ed population perceive primary health facilties as being of poor quality congestion of district and provincial hospitals as these instiutions replicated the functions of the primary health facilities increase the productivity of rural health facilit~~s contain costs at district, provincial and central hospitals What reforms: description of the policy change financial incentives to use primary health facilities: abolish financial barrier fee waiver for attendance at the appropriate level for referrals Why reforms being evaluated: assessment of the consequences of policy measures What are the policy concerns: Improve allocative efficiency Improve quality Promote appropriate utilization of health services see 1. 2. 3, 4.

2.

Objectives

See 4.

by removing financial barriers and improving quality at primary health facilities which service the poor, the policy would improve equity; improving efficiency through better utilization of resources at different levels of the health system; strengthening primary health capacities to extend the coverage of preventive and promotive health activities;

3.

Institutional Framework of Policy Evaluation Polk"J'making

Internal ~10H meetings ~10H level: planning pool meeting

fortnightly -

Provincial level: provincial health team - Support-Implementation District Health team: Implementation-operational level

Workshops planning: Investigators: national. province and district expected outcomes: process and outcome indicators defined and da:a source identified, and data collection process and calendar of reports defined. process evaluation: evaluate quality, evaluate administration of fee waivers for referrals, etc .. outcome evaluation 4. Methods for Data Collection and Analysis

Size of the investigation: 7 provincial hospitals, 8 district hospitals. 80 health centers Quantitative Assessment of effects of Policy change on utilization and referrais at different levels of the system: Use existing health information system data on utilization and referrals before and after the introduction of policy change: number of patients plus referrals at different levels Evaluation of quality of rural health services: structure, process, and outcome (consumer satisfaction and perception) availability of drugs and other key process indicators of the administrative mesaures Assessment of the Policy CHange Implementation Administrative level Operational level

s.

Reporting and dissemination

calendar of reports (to be produced at the planning meeting) midterm workshop: focus on process final workshop: focus on outcome and policy implications (national planni:-:g forum)

6.

\\'ork Action Plan Princical Investigator Assisted bv a core group of four . Health information system Maternal and Child Health Finance department Planning department June-August: Selection of Districts and Identification of District Teams Final Constitution of Investigation Team Finalize proposal: july 31 final proposal at \VHO

7.

Summary of input needs Technical Assistance Immediate needs Finalizing the proposal (planning < detailed> meeting)

Annex 2

EVALUATION OF USER FEE SCHEMES IN MO DISTRICTS IN UGANDA STATEMENT OF THE PROBLEM

The health sector in Uganda suffers from three major problems:

a

severe under-funding of the existing services insufficient equipment demotivated and poorly trained staff and insufficient running expenses

b

curative oriented health system

C

inefficiencies in the use of heatth resources

EXPECTATIONS FROM REFORMS 1.

Low health expenditure per capita - funds from user charges small but significant additional resource

2.

Full retention of all fund collected by the health units, with proper management of the funds will alleviate some of the proolems

3.

By providing an alternative fund for curative services, government funds will be focused on promotive and preventive services I PHC leveL Providing quality service at lower level will minimize by passing to higher and expensive levels Involving beneficiaries in the management of the health services will : - promote a feeling of ownership at local levels - create cost conscienceness to seMices provided e.g. reduction in overprescribing. - may dissuade unnecessary consultations . Devolution of powers to local authorities to recruit, remunerate and discipUne professional staff is intended inefllciencies

4.

s.

6.

OBJECTIVES OF THE STUDY OVERALL OBJECTIVE

The overall objective is to study user fees in two districts with a view to contnbute to ongoing national effort on development of a national health care financing strategy. SPECIFIC OBJECTIVES

To analyse the impact and process of user charges scheme wrth reference to equity,efficiency and sustainability. To generate data for developing standards and guidelines for user charges int he country. To generate information for improved decision making and managerial process at all levels of health care finanCing.

Allocative efficiency

% Expenditure of hospitals, Hie Administration costs Share of expenditure on curativel preventive services Attendance at first contact level Outreach (Existence, services & attendances) Self medication

MOLG, MOH & distncts MOLG, distnct & faCility at district level H/C, Dispensaries H/C, Dispensanes

Communities

Technical efficiency No. of drugs prescribed per OPO visits registers % OPO cases receiving antibiotics

Surveys & patient

"

0/0 days stockouts of specific drugs

Store records & users Inventory/physical inspection

Level of provision of diagnostic equipment

FINANCING REFORMS FOR THE HEALTHC.-\RD THAILAND 1. Statement of problems

SCHE~fE

L'f

The health card scheme in Thailand was launched as a community financing programme in 1983. It has been constantly modified over the 12 year period. The latest modification was in 1993. The price of the card was set at 500 Baht a card in which the government would allocate another 500 Baht to the health card fund because the rationale has changed from community financing to health insurance. The health card scheme is purely voluntary health insurance scheme. However, there are other 2 varieties of health cards designed to accommodate the need of providing medical coverage to those who work for the government at village level; ie community leaders and village health volunteers. These two cards are issued free of prepaid premium to provide coverage to them and their family members, hence these variants are a kind of public welfare scheme. The coverage of the health card scheme was 1.7% of the total population in 1991 by the national survey of the National Statistical Office (NSO), while the figure of the Ministry of Public Health (MOPH) was as high as 5%. Since 1994, the Agriculture and Cooperative Bank (ACB) has been approached to be a panner of the scheme to collect money from the village level. The coverage figure from the ACB, which is considered to be the most reliable, was only 3.3% of the total population. The low coverage figure may be either because the cash from card sales was not put into the ACB, or because the competing public welfare scheme has expanded dramatically from 20% coverage to almost 50% of the population. Now, perhaps less than 10 million people were uninsured In 1992, the Office for Health Insurance was set up in the MOPH. This office was expected to oversee the activities of all health benefit and health insurance schemes within the MOPH leverage. The office has tried to establish a 'reinsurance' policy to provide a safety margin to small health card funds against high medical expenses. However, this policy has not yet been implemented because the Ministry of Finance withheld the MOPH's power to use this money since the beginning of 1995 fiscal year The MOPH appointed the Health Card Executive Board in 1995 to mark a new era of the health card scheme. The Board members include representatives from the MOPH. the Ministry of Finance, the Budget Bureau, the ACB and expens in health insurance. The Board is proposed because the management of the health card has turned to be a revolving fund, ie the matching budget from the government need not be spent within the same financial year. Funhermore, the Board agreed that the fund management of every province should be standardised, so that providers are willing to panicipate. In 1994, the scheme still operated with a 16% deficit as of the marginal cost of providing care. The high utilisation rate (4 visits/person/year and 0.2 admission/person/year) may be the result of selection bias; ie the high risk group tended to buy health cards while the healthy not. This could be proved by the lower utilisation rates amongst the health cards issued to community leaders and volunteers

Low number of enrollment

r

Questionable sustainability of

~'SCbem\ Under-flnanced

Low cost recovery

/\

Provider - - - + Consumer ...- - Limited dissatisfaction dissatisfaction access

/ r

\

Not realistic matching funds

i Management problems - high cost sharing in certain treatments - payments to different providers - problems on information and monitoring system

Budgetary constraints

Figure 1 Problems in the health card scheme and their relationships

High coverage

i ~ Properly financed

,'Cheme", \

Sustainability of

Different Revolving pr.:es, funds Price indexation

/\

Provitier satisfaction

Consumer ...- - Increase satisfaction access

/

i

i

Management innovations Expand - centrally monitoril1g system servIces - gUidelines for payment network - reinsurance policy - Health Card Executive Board - Ballking support

Figure 2 Changes in the health card scheme and expected effects .\

-r:. \

.

'> -

" '"'fi.J~

,..

-'

METHOOOLOGY 1 Reviewing documents ( reports. studies from MOLG. MOH. institutions) Field work

2

RecordS, interviews. & surveys

3

Design of monitoring tools and mechanisms

4 5 6

Dissemination and feedback at district and national levels Implementation of monitoring tools Ongoing feedback .

Annex 3

In summary, the health card scheme has faced the urgent problems as follows (see figure I): • • • • • Questionable sustainability of the scheme. Under-financing as oflow cost recovery ratio and not realistic matching budget. Disincentives for providers especially referral hospitals. Lack of centrally managed, monitoring and supponing system. Low coverage of the scheme.

From the above problems, the main aim of the reforms is to increase sustainability of the scheme, the changes that have been and to be undenaken are (see figure 2)' • Change the management of the scheme to be a revolving fund. • Establish a centrally monitoring system to advise on fund management. • Set up rules and guidelines of paying for hospitals and health centres on a case basis • Establish a reinsurance policy, the central office plays the reinsurer role. • Appoint a Health Card Executive Board to oversees and issues policies. • Establish a network of financial administration with the commercial bank. • To establish a mechanism of price amendment by indexation to inflation. • To create new types of health cards that provide higher benefits with higher price. • To expand the network of service providers to include all public and private sectors. • To increase the cost recovery ratio, either by increase the price to consumers or increase the proponion of matching fund from the government or both. 2. Objectives

As the health card scheme is going to 'rethink' and 'retool' to provide a higher coverage rate of the publicly-managed voluntary health insurance scheme, it is vital to set up a comprehensive evaluation framework staning from the primary concern in the Ministry - sustainability - to higher societal objectives - equity and efficiency. • To assess the sustainability of the scheme in terms of expansion of the scheme, cost recovery at provincial funds, political and budgetary suppon including other external suppon. • To assess the equity of health in terms of equity of finance, access to health care and equity of utilisation to quality of care. • To assess the efficiency in terms of allocation of funds to each level of care, fund management and care provided to referred patients • To assess the acceptability of the scheme both consumer and provider sides • To assess the role of the Health Card Board as of policy decision body and a new corporate management image. • To set up information system necessary for programme evaluation and ')olicy development on health insurance issues.

3.

~Iethods

Five groups of methods will be used to answer all the stated ohjectives One method usually serves more than one objectives, and one objective may be answered by vari-'.lS research methods. The relationship between the objectives, methods and the polic' concerns is presented in table 1. 3A. Action research on the development of information system As there are conflicting figures from various sources of information on the samt activities, there should be a mechanism to validate the reliability of the inform a ,I sources. Moreover, the reforms being applied need more sensitive data on cara holder characteristics, utilisation, clinical information, resources used, financial management. cross-boundary and referrals etc. This action research will lay out the information needs, how to collect the reliable data. verification of the obtained data and dissemination of information to the policy makers. This action research is an ongoing pro·:ess, however, it needs to be undertaken at the earliest. 3 B. Utilisation anti financial study This part of the study will focus on the merit of the information system including some special surveys on the financial flows, financial management of the funds, reinsurance policy implementation, cross-boundary flows and patient referrals. These substudies will answer some of the equity questions and most of the efficiency.

3C Qualitative research on policy processes The setup of the Health Card Executive Board is so new and come at the period as the fund modified to be a revolving funds, ie the scheme turns to be a state-enterprise insurance business. The policy making process of this body should be strengthened and monitored. Qualitative policy research also reviews the 12 year experience of the scheme and the experiences of 'not-for-profit health insurance scheme, to be an input for policy development as well as to document the changes of the policies within the next 18 months. 3D. Household sun.eys Household survey will be a crucial part if the National Health and Welfare and the Socio-economic Surveys of the Natio,nal Statistical Office cannot answer some questions on equity of finance (eg Kakwani index of progressivity) and equity of utilisation. However, the surveys will shed the light on the comprehensive health insurance reforms rather than limited to health card reforms. 3E. ,'Jarket surveys The strengths of the market survey should contribute more to the reforms of the health card scheme. The questions on who are the customers of the scheme and who will be the potential costumes will direct the policy on product design to target at equity Equity here is defined as the target group (farmers, the poor and the semi-poor) is enrolling in the scheme, not only the chronically deceased. 3 F. Policy brief and workshop This is the final process to summarise what have been learnt from the reforms International peer review is needed.

I I I

i.

~

~ j a •

~

~

.. ~ a 0

1 :'

""

!

a

; i I-

g

i l

. I !

e ~

~

I eo ~ 3

~ 3

'0

""

t .

t • 5 i 3 '3

~

i

!:

2 ~ !I

.2 ~

t

~ ~ 1 2 i a

i

t '"

~

~

• 1 I

~

50

~ ;:

i

, f I

...

a ~ .~

~

= e a ~ I ~ '0

I!

90 ...

! i:~

i

~ 2 3 ~

1 5 e a ~

~

'0 !

~ ~

'8 ~ oj

g

! ~

j 1 ~

a ! !I ~

~

~

3

~ i:~

i

t

l

~

t ~ .a

:c

~

•

e

l '5

• i

ie ;;-

~

'0

j

J ~

~

a

g

!

""

.!

• t ~ i

~ a

i

i:-

.. ;

~

= i

5

.5 3 E' ~ l ill

f I ~

3 a ... e " f ~ ~ !

I j ~

3 j

I ~ !

I a r .; a

. ! i

c

3 l ~

j

0

•

• I e e

~

•i t 9 i : ...

. ].

• ,

a

~

i , I I

I

I

. I I 2 1:

j

0 !

I!

~

a

a ~

I 11 ... r I . J

!

3

a • I I• I 1 J • I ~ '! ~

ls

e

... j

!i

I ~

) f '!

~

.i

i:-

1i l . f ."

... i l ~ t ... j

~

I

j 1

]1t a

f

• I I

to

I

e i

~ ! 2

1 1 ~

~

1 .! ':J ~

I j '3

i

a

l•

...

!

i j

i

. ~ ~ ~

~ .! ~

~

= ~ c~

. , a ~

I

I

;;

•

r II

•

• J • II

~ ~

.x ~

.i :l

• J

!

J iE' 9

; i e a ! = " i i l

.. ;

I ,

c·

r a

i" ~ ..

• 1 ;

"3 c

$ a .3 ... ." ! ~ ~

~

::

!

. a ~

-; ~

t ;f

i

i

• ~

1

. ! 90 i

l

i

· ~ • i I ~

2

l-

I!

! .II

~

I t l

i•

1 3 -=

1 ! l

r

'0

'!

:

I i

I I 3 ';

i I

~

s '3

I ~

J!

f ! 1 i Ji 1 !. 8

• to • it

I f

l• '!

!I a

. ... a

'0

l

J

r a • j

C' 2

r

0

-=

3 3

1 -r !

...

a

~

I

I

I

J I:

1 1

• t

I I

' . .,.ellYn

Melb,", .......KIAI .. ~. 0.- 01 I............

l"ccboteal,ndtallun

Pul• ., ('Olllr,1U

AACa d'ltv",) to &emu of rnuurcn ....JCalaJ 10 "lUCK,"

.Ja,.

M.,.,...I b .....~' 01, ... ....~b caRl .1 ......... '0 ..n .... 'Y. 1CWDd''Y ..... Icnaa'Y QR.

To IIKTCUC' .'Iuu'1W dlKVIILJ

...... , ... _ _ OI........... n.I ..."on

Inclolcarc

OJIIC.

AaKN

~"..oc ....y

01 lu ....

"'_ _ I"~ 01 lu.... at """""",~I ....~I

It.... IbanAI. ut .......on ..... e. K,uur(C o.JruoumiMtOn by PRNult."wl 'l.I'kb

To ItValua&c lbe eRe.., uf Uae: ~MQ'c lu tbe K'wU~&n'

.... ulcmco.

lu ....

A6KU

""uuntACe' ,lIldq .All U . . . .'K ....

SpcalllC llanq' <Un

110. referred CUCI a...

( 'lUi'

,ut. .........un Q.IoC

r~.c

'u•• IIWU

To pruv_ bR8Ik'W1 .1Mt ..aeb.... ., 10 ' .... 11

"k..-..

_b I"ab ..........1 ..p~_ ..,....

fundi,

Incnry and rnouu.:c

'u"'" 10

OIudy 01 lu........ n.ac_ ...

UJn..UmptMJA .un&l a.-I) ul .be "'CHe'"

.ad ....waN_ ,dCIR' .. n'R

peyn~ ...

<aK.

AaKN ..,.,.p'ab...., 01 , ... Kbraw .IAOA, Q)AlullWn aAd

M._ aurver. """" ""......ooc.... oun

('o"'UIDe,

wata5IaaKtA aAd pRWMk,

S ..............., 01 , ... "" .......

....'laalOft IOU." Q)mpared With Iuw lAa!1hC

aad pualbk c.,.u.,. 10 ,.,~

............

card IIw*kn and

"'be.

_ ...... ,....ectacmc

ICKIRC'I,

compare "''''''100 UII

en. ..............111 lu lbe C. . . III,

pRNklcr lAOtOlIWS. PrUidua ..... 'or

oocw

'wn 01 ....k~ ca,.... 10, polK} «'-""toD,

0. ....._'" ,he ...10<)' ..nocnon

Q .......,ove pobo:y

,~.un·h

U~ 0' ...lonaahoA

loacuc cn-.-.cQl ma....clDCnI ltnlOtM,

....

In be.lIb .:aId "MInC. tbe ruk

.be "-'rpor...w

.......«' ol.he IIcaflb

oJ 1I~.I.b '- '•• d bC',,''''''vC'

,·.RI_nJ

lOuod pult,} lur

,,,,,.u1ahlhl), aJk.l o.ft'CntlC'

--

4. Reporting and dissemination The results of each substudy will be presented at the Health Card Board meetings. Policy options relevant to the study results will be drawn up so that the Health Card Board can make a good infonned decision. The results of the whole package of this research will be summarised and published and distributed to health-related policy bodies, the public and academics of health insurance. 5. Work action plan and input needs Table 2 Time table

Activities 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Starting date: July 1995 PI and team selection x Start-up workshop x (consensus building and development of instrument) Finalise proposal x Assignment x Review literature x x Set up infonnation system x x x x Utilisation, financial data analysis x x x x x x x' x x x Household survey x .x x x 9 X X x Market survey x- ,,,··;'X. X X· X . X X x Qualitative study Policy brief and workshop x Country repor~Jng x

6. Summary or input needs

This evaluation package of the health card financing refonns requires a total budget of 4.8 million Baht (USS 192.000, see table 3) to accomplish within the first 18 months Thirty percent of this will be borne by the MOPH budget. Another 50% will be seek from other independent agencies~ eg the Health Systems Research Institute (HSRI), the Thailand Health Research Institute (THRl) or other international donor agencies Twenty percent is expected from the WHO Headquarters, not inclusive of the technical suppon. T a bl e 3 Estlmate . db udliet b ).Y se ected activitIes

Activities Development of infonnation system Utilisation and financial studies Qualitative research on policy processes Household surveys Market surveys Policvobrief and workshop

Estimated budget (BahtJ. 1,000,000 800,000 800,000 1.200.000 800.000 200.000

Possible lunders MOPH HSRl, WHO MOPH, WHO, THRl MOPH. others HSRl, THRl MOPH, WHO

Annex 4

..

MEXICO USE OF CHARGES IN THE TERTIARY LEVEL OF ATI'ENTION ~.

1. STATE~fENT OF PROBLEM

In Mexico, the National Health System is organized by levels of care. The tertiary level of attention includes 10 National Institutes of Health. Those institutions provide three types of services: research, training, and health provision of complex treatments. 3% of the total health demands are solved in these level. One major problem that is facing by the National Institutes of Health (NIH) is the reduction of the public expenditure and the need to increase the sources of finance to support their activities. The two major sources of finance of the NIH are the public expenditure and the income collected by the use of charges. At the moment, the 80 % of the expenses are obtained by federal ta1;;ation and the 20% remaining are from user charges. the amount of money that is needed for the functioning of the NIH is huge and there is a need to find alternative ways for the financing of this type of services in order to liberate some money to support primary health care activities. The NIH are working only 8 hours a day in the ambulatory services and there is capacity to increase the utilization of surgical rooms in the afternoon. Also those institutions have a strong prestige in the country and there is a possibility to attend more people with strong capacity to pay. The purpose of this pilot study is to extend the private practice to the afternoon and to allow public doctors to do their private practice on site. That means that a doctor could attend the poor people during the morning remunerated by the institutional salary and in the afternoon could attend the private patients and be remunerated by the charges established. The strategy will imply the move of the wealthy people form the morning to the afternoon. ~urthermore,

.

It has to be consider that the User Charges System is designed on a differential basis, that means that the charges are established ccording to the ability to pay of the patients. There are 10 different SOl ) economIC classifications and the wealthy patients will be classified in .e 'lpper levels. The cross subsidy policies will be pursued, so with part of the lone' raised from the wealthy people will be subsidized the de mane of the poor patients. In the design of the system there will be four types of funds: - 30 % of revenue collected to support an economic incentive program for the staff of the NHI. - 30 % to support a quality assurance program. - 15 % to support administrative expenses. - 25 % to provide specific inputs to benefit the poor patients. For example, to provide pacemakers to the patients, or to buy expensive drugs, etc, A specific component to achieve technical efficiency will be integrated to the pilot study. This component will include the establishment of cost centers, social markets studies, and a quality assurance committee. 2. OBJECTIVES

2.1 OBJEcrIVES OF HEALTH FINANCING REFORM

- To reduce goverment's subsidy to tertiary health care. - To increase alternative sources of financing for the NIH. - To design and establish economic incentives for performance of health staff. - To create a fund for quality assurance. - To develop a comprehensive management strategy to achieve technical

2.2 OBJECflVES OF THE PILOT STUDY - To monitor the changes in the different sources of financing. -To monitor the changes in the patterns of utilisation -To monitor change in quality - To integrate the different systems of information into a costing system. - To generate an efficiency culture inside the NIH with specific benefits for the staff, the institution and the patients. - To design a specific social marketing assessment of the services provided by the NIH in relation to similar services provided by the private sector.

3. METHODS FOR ANALYSIS AND DATA COLLECTION The methodology of this study will be a "Demostration - Evaluation Project". This methodology will imply the definition of three periods: a) Activities before the intervention. This implied the definiton of a set of activities to develop before the intervention and the measurement of key variables in the baseline period. b) The implementation of the intervention, c) The monitoring of the key variables and indicators during the period stated, and the assessment of the feasibility of transferring the whole intervention or some specific components to other tertiary level institucions.The pilot study will be conducted in The National Insitute of Cardiology of Mexico. In order to achieve a systematization of this work, the information will be presented clustered by different areas:

• Activities before the intervention: 1) Financing Issues: - Design of the financing issues of the intervention. Establishment of fees and honoraries for the staff. - Establishment of the legal framework. - Establishment of the organizational framework. - Identification of the key variables, determination of key indicators, and collected of baseline information. Key Variables and indicators: - % and absolute levels of public and private expenditure

- Number of patients by socioeconomic classification per month - Number of patients by fee paid per month - Design of an emergency or contingency plan.

2) Management Improvement Issues This component will be integrated by five different segments: a) Economic incentives for performance. b) Quality assurance program. c) Cost centers. d) Efficiency improvement e) Social Marketing assessment.

For each of the segments above it is necessary to develop the following activities · Design of the especific segments of intervention. · Establishment of the legal framework. · Establishment of the organizational framework. Organization of the quality assurance committee, the performance assessment for economic incentives committee, and the administrative additional sta.! required . . Identification of the key variables, determination of key indicato~, and collected of baseline information. Key Variables and indicators: Economic incentives: - % change in monthly salary by type of personnel.

• Number of patient treated per month by socioeconomic classification . . Number of surgeries treated per month by socioeconomic classification. Quality assurance . • Number of quality issues solved per month. -Change in average waiting time -Change in length of waiting lists Cost -Number of DRGs or tracers supported by cost information

Efficiencv . - Cost per case ( tracer or ORG) by socioeconomic classification. - Average cost of curative outpatient fisrt visit by socioeconomic classification. - Average cost of other units of activity by socioeconomic dassification. Social marketing. - % of public fees below private fees in the upper socioeconomic

level.

- Identification of comparative advantages of the services provided by the public institution in relation with the private. - Design of an emergency or contingency plan . • The implementation of the intervention and its monitoring and evaluation.

For the first month after the intervention is started, there is need for the daily monitoring of the indicators used to evaluate the new type of serv'ices provided. Special attention will be focused in the change in earnings of health staff and the Union attitude toward the new program. After the first month, all the set of variables and indicators will be monthly assessed in order to evaluate the performance of the new program in terms of equity, efficiency, and quality. In relation to management improvement there is a need to apply the economic incentives program according to priority groups and to encourage productivity in crucial activities. Furthermore, in the quality assurance program the monitoring of the different levels of quality by socioeconomic classification will devote special analysis.

~.

4. REPORTING AND DISSEl\<IINATION At the end of the pilot study an assessement of the feasibility of transferring the new program to other tertiary level institutes must be addressed.

S. WORK ACTION PLAN

Finalize proposal Start up workshop: instruments, assignments, consensu.s building Activities before the intervention Starting point of the intervention First assessment Second assessment Third assessment Final report

1 - 2 months. 3 - 4 months.

5 - 6 months. 7 months. 9 months 12 months. 15 months. 18 months.

,.

Annex 5 o

EVALUATION OF HEALTH CARE FINANCING REFOIt.\1 IN KYRGYZSTAN l-STA TE~E:\TT OF PROBLEM The Kyrgyz Republic experiences a transition period since the independence in 1991. [t is a transition from centrally controlled public economy to market economy. DurIng this period country faces severe economical problems. The Gross Domestic Product fell over the period of 1990 to 1994 by accumulative 50% and this decline has been accomparued by high intlation rate. This macroeconomic background intluenced the health sector l[ varIOUS points. I) The funds for health care financing has showed a severe decline in real terms and the out-of pocket expenditure is assumed to be considerably increasing. The health sector suffers from underfunding and nearly half of the available funds is spent on salaries. therefore the need for drugs and medical supplies can not be met and the quality of services declines. 2) There is a geographical variation in health expenditure that is not correlated with the need. Furthermore. the expenditures on different type of services does not corresponu to the priorities of the country. 3) There is excess supply in terms oI staffing, facilities. beds and the services are utIiized inefficiently with unnecessary referrals. hospitalization and long stays. .. ) Ctilization of services also shows geographical variations as resource allocation v. ruch implies inequity in access.

11. HEALTH CARE REFORM All these forced the Ministry of Health to take immediate action. Therefore. the ~l\..\lAS Health Care Reform Programme was developed and is being undertaken by the support of WHO. UNDP, nCA, WORLD BANK, ODA. DANIDA. The programme has got two immediate objectives as: 1. development of a 10-year master plan 2. strengthening the management capacity Development of the master plan has been undenaken in three phases as: I. Situation analysis 2. Development of strategic policy options 3. Refinement of the selected option First !\VO phases have already been completed and the main directions of the reform policy have been identified.

Objectives Of Health Care Reform In Kyrgyzstan 1) To improve in the health status of population: ::) To ensure equity which aims at reduction and elimination differences in health indicators in different regions of the Republic and between urban and rural populations: 3 ) To ensure guaranteed access of the population to the health care services: -+) To ensure effectiveness and quality of care.

At the beginning of the reform process. the current structures will be preserved. however. efficiency and equity gains should be ensured - this implies the prioritization strate~·. The separation of provision and financing functions, however. would be the next step for a more efficient system - this implies the c :1tracting strategy. The timetable of this approach is shown below.

Health Care Reform Policy in the Kyrgyz Republic ,. '991 1917 ,. '991 2000 2001 2002 2003 2004 2005 Dle

PRIORITIZATlON BASED ON THE EXIST1NG SYSTEM p,.,.,..tlon PIloting Piloting

CONTRACTING SOCIAL INSUAANC&

The details of this policy will be discussed at a workshop at the end of June 1995.

Health care financiDI

"

The tax financing will continue with an introduction of some user charges. .--\fter the economy is stabilized, development of social insurance will be considered. In the short term the reform in health care financing will look for immediate and implementable solutions for the problems mentioned earlier with three main objectives: 1. Increasing the available resources 2. ensuring the better allocation of resources according to need 3. ensuring the more efficient use of available resources

Increasing (he funding: Besides attempts for increasing the resources allocated from the State budget. the he:1lth budget will be supplemented with the introduction of user fees. The rate of user fees and exemptions will be identified in the next couple of months. The possibility of introducing earmarked taxes will also be explored to ensure some additional funds.

Resource allocation At the first instance. financing principles and resource allocation will be changed and :he main element of the new system will be per capita resource allocation which makes it necessary to determine the minimum volume of medical services at different ~evels to which the population has a free and guaranteed access. It will be necessarY to determme a number of indicators characterizing the requirements of the population with regard to health care such as sex and age protile of the population: climatic and geographical conditions: general mortality; infant mortality; and other demographic and epidemiological characteristics of the population. Furthermore, the possibility of moving resources for health sector from the ~1inistry of Finance to Ministry of Health will also be considered at the workshop. This will en...'W'e the better allocation of the resources to respond the priority health problems of the country.

Efficient use of resources Payment systems to providers will be revised. New mechanisms of remuneration and provision of other benefits and incentives will be developed to ensW'e the effective performance of health persoMel at the primary health care level. For hospitals. there may be a move from itemized budget to global budgets at the tirst instance that will be discussed dW'ing the forthcoming workshop. This will accompanleO with structural changes in hospital management to ensW'e better use of available resources. Furthermore. the service delivery will be reorganized as mentioned below to ensure the provision of better quality and efficient services.

Primary health care Another key element of health care reform in the Kyrgyz Republic will be strengthening the primary health care. The FAPs and rural outpatient units. which are currently lacking in both technical and manpower resources will be replaced by much stronger primary health care centres which will be done through the unification and transformation of ineffective rural hospitals. This will help to achieve • intensification of preventive activities conducted by primary health care centres and aimed at reducing the number of patients requiring hospital treatment • reduction in the demand for hospital care thus permitting the channeling of more resoW'ces to primary health care.

Secondary and tertiary care The secondary and specialized services will be reduced. The priority areas in refonning hospital care include: • increasing the number of pediatric beds at the expense of other types of specialized beds • • • • reducing the number of specialized beds at all type of hospitals ana closing some hospitals reducing the average length of stays standardizing methods of patients' examination and treatment. as well as the list of conditions requiring hospitalization development of new management models for hospitals. including new management structures and management information systems.

Demonstration Project in Issyk-Kul: The purpose of GSAID- funded demonstration project was to ensure the pilot implementation of the health insurance legislation that had passed the Kyrgyz Parliament in 199:. The demonstration project consists of the implementation of three components: 1. A Basic Health Insurance (BHI) fund complemented by cost recovery to improve the financing of the health sector; 2. The implementation of a new primary care system comprised of small group practices each consisting of a gynecologist. pediatrician, internist, several practice nurses and support staff (APTKs) 3. The implementation of new incentive-based provider payment systems consisting of fundholding for APTKs and case-based payment system for hospital care.

Improving Health Financing A Basic Health Insurance Fund will be quasi-public. It will operate independently from the Ministrv of Health and have an elected board of directors. The BHI fund will collect a 6% payroll" tax from employees including both state and private employers. The budget will provide a capitated payment for children, students. disabled, unemployed and pensioners. The ~finistry of Health will maintain the responsibilities for public health (sanitary Epidemiological Service. health prevention. health promotion. and training). All health service delivery will be managed through the BHI pooling at the levels of oblast and three experimental rayons A basic package of benefits will be established by the fund . This package will be provided to the population free of charge. services not covered by the package will be subject to user fees which will be kept by the health facility.

Improving primary health care

APTKs will be fonned throughout the demonstration area to form basis for tirst contact primary care. Patients will have right to enroll in any !rcensed APTK. Enrollment would be based on 1500 to 2000 per physician. APTKs will have the right to refer patier:·..s to any licensed specialist, laboratory, radiological centre and health facility. Development of new payment methods

A.PTKs will be paid on a capitated payment rate consisting of the full costs of outpatient care and a percentage of hospital care. For the outpatient care not provided by APTKs. a fee schedule will be set. The prices will constitute a ceiling, but specialists and facilities could charge less than the schedule. For inpatient services. APTKs would pay a percentage of the hospital costs. The percentage will be set at 80% of hospital care. A cased-based payment system will be implemented that will pay hospitals a predetermined amount for each case treated. The initial classification will be based on hospital departments and will move to diagnosis and procedures over time. The new payment system will create a perverse incentives that must be counteracted \\ith a system of quality assurance. The quality assurance system will consist of licensing and accreditation will be carried out by a joint commission composed of the BHI and Oblast Health Department. The commission will license hospitals. APTKs. laboratories. independent specialty practices and radiological centres. The quality assurance department will monitor the referrals, admissions and discharges.

III. EVALUATION OF HEALTH CARE FINANCING REFOR..i\1S 1. Objectives For Evaluation Of Financing Reforms 1) To evaluate the impacts of the financing reforms that have been introduced: 2) To provide information for the policy -makers J) To develop capacity in monitoring and health services research at both central and local

levels in Kyrgyzstan 4) To integrate evaluation into the regular function of the Ministry of Health

Reforms To Be Evaluated A- At Issyk-Kul Demonstration site 1) Social Health Insurance 2) Provider payments 3) Performance of APTKs B- At National Level

1) Gser Fees 2) ~ew resource allocation mechanisms and Provider payments

2. Indicators And ~Iethods .-\- Issyk-kul Demonstration Site Specific Objectives For Introduction Of Health Insurance To raise additional funds 2) To increase efficiency in utilization of funds by strengthening the primary health care. reducing referrals and hospitalization 3) To contain cost of hospital services .. ) To increase patient choice and satisfaction 5) To increase the quality of care by instituting quality assurance mechanisms I)

Indicato" For lssyk·Kul Health Reform Demonstration Project Objective To raise additional funds Indicator a) Number of contributors (employers and employees) b) amount collected through premium collections c) amount of government subsidy to the scheme a) Referral rates b) Hospital admissions per 100 c) Hospital readmission rates d) Description of contracts e) share of health expenditure for primary health care f) Proportion of health expenditure on salaries g)number of APTKs a) number of beds per capita b) ALOS c) ALOS for certain conditions d) throughput e) turnover interval Data Source a) BHI Fund b) BHI Fund. tax inspection dept.. c) Oblast Finance Dept., Tax inspection dept. a) Medical Information Centre (MIC) b) MIC c) MIC d) Independent Medical Commission. QA Dept. e) ObFin f) ObFin g)ObZdrav a)MIC b)MIC c)MIC (?) d)MIC e)MIC a)'b),c) ~iJl be compared with the other oblasts for 3 years \1ethods

!

I

To increase efficiency in utilization of funds by strengthening the primary health care. reducing referrals and hospitalization

\

To contain cost of hospital services

,

all will be compared \\;th the other oblasts for 3 years. c) will be compared \\ith another oblast for certain conditions.

To increase patient choice and satisfaction

a) number of contributors to private insurance companies b) proportion of enrolees who change their PHC provider in a year c) reported patient satisfaction d) number of reported complaints

a)Gosstrahnadzor b)BHI Fund. MIC c)Facility based patient survey (twice a year) d)BHI

To increase the quality of care by instituting quality assurance mechanisms

to be developed

J)

Quality dept.

!

I I I j

I

I I

B. ~ ational Level In case the implementation process starts in the beginning of 1996. impacts of user charges and new resource allocation mechanisms will be evaluated at national level. Spec~tic Objectives For Introduction Of l..:ser Fees

1) To raise additional funds for maintaining services 2) To ensure cost-consciousness among the consumers and more rational use of ser.;ces by consumers 3) To ensure access for vulnerable groups through exemption mechanisms Indicato" For Evaluation of User Fees

Objectives To raise additional funds for maintaining servIces

Indicator a) Revenue from user fees b) Proportion of revenue from user fees in total health expenditure c) Distribution of spending from user fees d)Qualitati ve evaluation of chief doctors e)Proportion of expenditure on drugs a) description of the fee structure b) self-referrals or direct admissrvns c)number of outpatient visits per capita a) description of exemptions and processes b) % of population exempted c) characteristics of the population that do not use services

Data Source a) MoH b) MoH. MoF c) MoH, MoF d) sample interviews e) MoH

~ethods

To ensure costconsciousness among the consumers and more rational use of services by consumers To ensure access for vulnerable groups through exemption mechanisms

a) MoH b) MIC c) MIC

a) MoH b) MoH c) Household survey

Specific Objectives For Changing Resource Allocation Jlechamsm 1) To improve the equity in resource allocation 2) To increase efficiency in service provision by reducing excess capacity 3 ) To improve the qualiry of care for vulnerable groups

Indicaton for Evaluating ~ew Resource Allocation ~Iechanisms

Objectives To improve the equity in resource allocation

Indicator a) Description of the basis for allocating the resources b) distribution of taxation burden c) Per capita government health expenditure by oblasts d)Per capita outpatient visits by oblasts e) Hospital admissions by oblasts a)nwnber of hospitals b)nwnber of beds c)nwnber of health personnel d)ALOS e)nwnber of hospital admissions per 100 a) proportion of patients satisfied b) percentage of practices comply with standard treatment protocols

Data source a) yloH b)MoF c)MoF d)MIC e)MIC

\-lethod

\ I ,

To increase efficiency in service provision by reducing excess capacity

a)MIC b)MIC c)MIC d)MIC e)MIC

To improve the quality of care for vulnerable groups

a) Focus group survey b) Hospital reports

3. Reporting And Dissemination The health care reform programme pursues a participatory approach. This approach \\ill be followed during the evaluation study. The Kollegya. the decision-making body of the \finistry of Health. will be the target group to inform about the results. There will be regular reporting to Kollegya. Formal reporting will be held at the following stages as: • • • finalization of proposal interim report final country report

Before finalizing the country report. the findings will be discussed with all relevant officials and parties at a workshop. The final report will be distributed to the high officials in the Ministry of Health as well as all oblast health administrations.

FINPROPXlS

T ACTIVITIES 1. PI and team slll;l:t;o.1 2. Finalize proposal 1 2 3 4 5 6 1

MONTHS 8

9

10

11

12

13

14

15

16

17

1B I

-T -

~~,::::n!~r~!' !~ 4 Baseline data collucllon&analys 5 Repoll on contellt and type of re'orm 6 A~sess"'r~ ~. plOcess 1 Interim dala collecllon & analys 8 Interim report 9. Assessment Workshop 10.Final data collecllon and analys 11. FlRal drah rep. 1'J Workshop 13 Final report

.

1

'r'

~

P'I~IC I

Annex 6

7 June 1995

Ghana Health Fmancing Reform Evaluation

1.

STATEMENT OF THE PROBLEM

To address the problem of insufficient revenue generation for the health sector. the Government of Ghana Ministry of Health is planning to introduce several related reforms in health financlDg and management. User fees collected from patients at government health facilities do not provide eno~ revenue to permit improvements in quality or expanded access to services needed by the populatlon.. Out of pocket payments by users of government facilities now amounts to 3 ~ of total expenditures. or 8 ~ of recurrent expenditures. Several reasons lie behind this small and declining amount of user fee revenue: fees were changed last in 1985 so inflation has diminished their value today; salaries of health workers were increased and therefore the government health budget has tnCreased in recent years; there is leakage of revenue. fees collected informally at facilities are not captured by the formal government revenue system. and there are other problems of financial perfonnance; large amounts of the population were exempted from paying fees for health services. inc1udJ..ng: (a) (b) (c) all health workers and their families (who over-used free government health a large number of disease categories. and paupers; and to 5em.;:::),

SO 9Ii of fees collected at facilities were remitted to the treasury, so there was little incentlve! collect fees.

At this time. very little fee income is collected in the official revenue system plus locaUy collected l.alOfficlal fees are not available to use in improving quality of services in the facilities. After careful study. several reforms to address these problems were developed: adoption or health financing refonns; creation of a Ghana national health service (making the health service a parastata.l. decentralized body); changing tertiary hospitals to autonomous bodies with their own board: and decentralization of financial control to sub-district level. The health financing reforms being considered or implemented are intended to address many of problems facing the health sector. and these financing reforms will be the focus of the Ghana c:vallWlon study" Five specific elements are included in the health financing reform package:

2 1. 2. 3. Increase in health care fees; Revision of the exemption categories. reducing the number of people who are exempted; Retention of revenue at facility level. to be used for service Improvement. which hopefully Wl11 provide an incentive to collect required fees; On-aoing proaram to improve and strenathen financial management at all levels. from hea.Qquarters down to the institutional level. Preparations for a national social insurance scheme which would cover the formal sector. IJDtially. including: public sector employeea. employeea of industrial and commercial companies. pnvate road transport workers. and cocoa farmers and cocoa farm caretakers.

4.

5.

Objectives of these reforms are focused on efficiency, quality and equity concerns. including: Health care fee revenues would be increased and be retained at the facilities. which will translate IDto improved quality of care; Accountability and other aspects of financial management would improve. providing additlooal efficiency in the revenue collection system; Patients are given incentives under the structured fee schedule to follow the referral pattern of primary, secondary, and tertiary care facilities. thereby improving efficiency; and Vulnerable groups within the population are protected by the fee exemption categones and by having lower fees in rural health service facilities. Revenue generated from the health insurance scheme should ultimately take care of curative health care costs. releasing government funds (inputs) for primary health care and preventive ser'ICe5.

2.

OBJECTIVES OF EVALUATION STUDY a. Technical and Analytical Determine effects of reforms on key variables: revenue generation (expected versus actual collections. accountability. effectiveness in expenditures of the revenue) utilization (by income groups. location. age) efficiency (allocative and teclmical)

3

quality (availability of inputs for health ~re procedures, time spent with panents. perceived quality) equity (relative expenditurea on health by households, performance of fee exemptions and waivers) Document process for introducing the reform: Planning and background study procedures; Implementation, monitoring and follow-up processes; Training and system development activities; Supervision and management processes

b.

Policy systems and processes At headquarters, incorporating results into the policy process Frequent feedback of analyzed data to: PPME, regional directors. distnCl health management teams, Parliamentary Select Committee and Parliament

c.

Institutional development Create c::apacity to analyze routine accounting and revenue generation data a.t facilities, districts. and regions Establish in MOH a capability for using social science and economics tools examine economic and financial implications of reforms on health sector performance (such as: equity, efficiency, quality) to

4

Ghana llealth Financing Refonn Evaluation Aclivilics Finalize propolAl Develop inlilNmcllla Deli!!!! i!!!l!aci jodiealou: revenue aencnlion, ulilizalion, efficiency, qualilY, equilY ~UI!!£I11 eroccs1

Key Elcmcnls July 1995

Tinung

Participanls MOIIIPPME MOIIIPPME, Ghlna SUilillli.al Service, Minilliry uf Finame

Auaull 1995

(or il!!roducinl !be

refonn: Plannina; moniloring lod follow-up; Tninina and aylllCma developmclll; Supervilioa and manaaemelll Conduci Saari-Up Wortabop Aup_ I99S SeplCmbcr I99S Baaclinc-Augull I99S COIlIinuina-Sep 9S lu Mar 97 PolI--April 1997 MOH/PPME CeRler for Heallb Saalillicl, MOH A,,;ounling Omcen, MOil Di"n,1 alld Regional omc ilia, MiniSiry of Finance !

MOH/PPME Ind OIhen

> > lnlroduce Refonn in Feea Coneci dlaa

> > &Cgin

Feedble~

10

PolicYRII~en

January 1996 Scplcmbcr 1996 Mar.h 1996 h' April 1997 MOH/PPME MOIlfPI'ME. Ghana Slal,.I;':1I1 S<rvi.:c

AallCll e"alulli"n process An.IYLe d.la

P,cl,ar< I'ulo. y w ... ~.h'fl'.

Uroc(

1",I>Io':lli"" a"d

January I.. 1.111;1 1,}97

MOII/PI'ME

--

---MlY 1997 MOII/PPME

Cu"dlld Cuunlry I{cpUI1,,,g

5

Objectives of Ghana Health Financing Reforms • • Health care fee revenues would be increased and be retained at the facilities, which will translate into improved quality of care; Accountability and other aspects of financial management would improve, providing additional efficiency in the revenue collection system; Patients are given incentiv~s under the structured fee schedule to follow the referral pallern of primary, secondary, and tertiary care facilities, thereby improving efficiency; and Vulnerable groups within the population are protected by the fee exemption categories and by having lower fees in rural health service facilities. Revenue generated from the health insurance scheme should ultimately take care of curative health care costs, releasing government funds (inputs) for primary health care and preventive services.

•

•

•

6

OBJECTIVES OF EVALUATION STUDY a. Technical and Analytical Impacts of reforms on: revenue generation (expected versus actual collections, accountability, effectiveness in expenditures of the revenue) utilization (by income groups, location, age) efficiency (allocative and technical) quality (availability of inputs for health care procedures, time spent with patients, perceived quality) equity (relative expenditures on health by households, performance of fee exemptions and waivers) Document process for introducing the reform: Planning and background study procedures; Implementation, monitoring and follow-up processes; Training and system development activities; Supervision and management processes

7

b.

Policy systems and processes At headquarters, incorporating results into the policy process Frequent feedback of analyzed data to: PPME, regional directors, district health management teams, Parliamentary Select Committee and Parliament

c.

Institutional development Create capacity to analyze routine accounting and revenue generation data at facilities, districts, and regions Establish in MOH a capability for using social science and economics tools to examine economic and financial implications of reforms on health sector performance (such as: equity. efficiency, quality)

~

..

".? ·:·· .•

;.f.:~~"t.~:·· .. :·

Annex 7

HEALTH F~ANCING REFOR.\;I: A FRAMEWORK FOR EVALUATION

Revised \Vorking Document 1995

This document has been prepared by Joseph Kutzin. Inputs. comments, and suggestions provided by Andrew Creese. Katja Janovsky. Miloud Kaddar. Barbara McPake, Jeffrey Muschell. Sanguan Nitayarurnphong, and Robert PJ. Zegers are gratefully acknowledged.

TABLE OF CO~TE~TS

Introduction.

. ................... . of this Activity .. . Ov~rvi~w and Purpose of the Framework for Evaluation ... . Criteria for Country Participation ..................... . Obj~ctives

Part I. 1.1 1.2 1.3 1.~

1.5

Context for Health Financing Reform Existing health tinance institutions. financial flows and incentives Epidemiological and demographic protile and service utilization patt~rns \-lacroeconomic and other extra-sectoral conditions .......... . Systems for policy making. policy analysis. and use of information . Initial assessment and major perceived problems in the health sector Type of Health Financing Reform .................... . Description of the specitlc reform(s) being implemented ... . Intended changes in health tinance institutions, tinancial flows and incentives ..................................... . ... . Description of other relevant reforms being implemented ........ . Expectation of how the reform(s) will address identitied health sector problems ...................................... . Process of Implementation .......................... . Description of the actual process of implementation of the reform( s) . 0l'ew health finance institutions. financial flows and incentives . 0l'ew systems for policy making, policy analysis, and use of information Assessment of the Effects of the Health Financing Reform .... Equity ....................................... Efficiency ..................................... Sustainability ................................... Acceptability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...

h

h

Y

1') 11 12 12 12 12 1~ 1-+ 1-+

Part 2. 2.1

., .,

2.3

2.4 Part 3. 3.1

1-+ 1-+

Part

~. ~.1

~.2 ~.3 ~.4

Part 5. 5.1 5.2

Policy Feedback ....... ' .......................... . Systems and processes for transmitting evaluative information to policy makers ........................................ . Integration of evaluation into the policy making process ..... . Synthesis of Conditions with Consequences for the Effects of Reform The financing reform(s) ............................ . Other health policies .............................. . Institutional conditions in the health sector ............. . ytanagerial capacity in the health sector ............... . Extra-sectoral factors ........................... .

19

Part 6. 6.1

6.2 6.3

:0 :0 :0 :1 "' ,l _ "'1

6.4 6.5

Timing of Country Activi::es ............................. .

Introduction Financing issues have dominated approaches to health reform in many countries. In lov,income countries. an objective of policy has often been to rectify an overall shortfall of health sector funding. This has led to numerous measures designed to mobilize supplemental resources. such as community tinancing. expanded social insurance. and ne\\ fee systems for publicly provided health services. In general. these measures ha\e placed additional financial demands on households. The objectives of equity. and. to a lesser extent. efficiency. have received lower priority relative to the aim of increasing revenues. \;liddle- and higher-income countries have .... given more attention to changing on2:amz.:ltloml structures and incentives that intluence service providers. rather than just consumers. In these countries. more attention has been paid to the objectives of cost-effecti\·eness. quality and acceptability. and cost containment.

-

- -

Although understanding of health tinancing concepts and issues has expanded in recent years. there is a conspicuous shortage of evaluations of measures that have been implemented in developing countries. To improve the outcomes of health tinancing policies. countries must have the ability to monitor and evaluate the effects of reform measures and use this information to modify policies periodically as needed. l'nfortunately. this ability and related systems are \veak in most developing countries. Yet reforms continue to be implemented in the absence of this capacity, and thus the policy making process often lacks an adequate analytical base. Therefore, many countries need to strengthen their policy making process by improving systems and human resource capacities to analyze information to evaluate reforms. Relatively little is known about the conditions that facilitate progress toward sectoral goals. In addition. there has been no systematic attempt to generate information about the institutional and managerial conditions needed to facilitate positive outcomes of tinancing policy changes. There is, therefore, a clear need for country-based assessments of the effects of specific financing measures and the conditions supportive of effective implementation. Such research can improve understanding of the effects of financing policies and the role of supporting conditions for national policy makers. international agencies. and consultants advising on health finance policy. Because of these needs, the National Health Systems and Policies (NHP) unit of the Division of Strengthening of Health Systems (SHS) of the World Health Organization l\VHO) is conducting a multi-country activity to support national capacities to evaluate the effects of health financing reforms. This framework has been prepared to provide a common approach to reporting the effects of financing changes. It is not. however. a strict protocol describing a methodological approach. Countries participating in this activity will define the specific reform(s) that they are implementing and that they wish to evaluate. They will also develop specific protocols to evaluate the effects of these reforms. including identification of relevant outcome and process indicators, data needs. data collection systems, and analytical methodology. This will include also mechanisms and systems to feed this information into the national policy making process. I WHO ana other sources of expertise will provide technical support as needed and requested by the participating

I A consultation was held on 6-8 June. 1995. at WHO/Geneva. With the purpose of generating In country protocols for evaluation based on the framework.

Inttlal ,<!t or

countries. It is expected that the country-based evaluations will be supported over a 18 month period follov"ing implementation of the reform. Oh/t!crives of" rhis .i.cri\·iry·

I:

[0

The evaluations of health tinancing reforms to be supported under this activity are not intended to be part of an academic exercise. The success of this activity depends. in large part. on the extent to which the evaluations are used to improve existing policies. Plans to evaluate policy change in each participating country. therefore. should also describe a process for transmitting tindings periodically to the country" s health sector decisionmakers. It is hoped that the tindings '.\jill be used to improve the implementation of he:.llth tinance policies and by so doing. lead ministries of health to incorporate evaluative work into their normal activities. This is important because the activities that can be supported under this work program are time-limited. whereas the process of reform is ongoing. and the effects occur over many years. \Vith this in mind. a critical objective of the evaluation activities to be supported in each country is to strengthen capacity for policy monitorir.g and assessment. In addition. it is hoped that the tindings of the evaluations in each country will contribute to the growing body of literature on health care tinancing and. by doing so. to improved policy recommendations and outcomes. With these broad goals in mind. this activity has the following objectives: • To promote greater use of policy analysis in ministries of health by demonstrating to national decision makers that evaluation of reforms can provide practical information useful for improving policies and implementation. To improve understanding of the effects of specific health financing changes on the sectoral objectives of efficiency, equity, sustainability, acceptability to clients and providers. and quality. To identify specific policy. institutional, managerial, and extra-sectoral conditions necessary for reforms to achieve their intended effects. To disseminate the findings of the country studies to a broad audience of policy makers and researchers from governments and international agencies.

•

•

•

Overview and Purpose of the Framework for Evaluation The proposed framework suggests a common approach to reporting the consequences of health finance reforms that can be applied in a variety of country settings. An outline of the framework is presented in Table 1. It begins with the context for the reform. i.e .. the existing or pre-existing health system, health situation, and other relevant issues prior to the implementation of the reform. This is followed by a description of the specitic nature. or type. of the policy change that has been (or is to be) put in place. Following the description of the intended policy change. the process of implementation is described. These first three parts of the framework for evaluation are largely descriptive. The fourth part consists of an assessment of the effect of the policy change on sectoral objectives and institutions. The assessment will be based on protocols for data collection and analysls developed by the participating countries. This is followed by what is likely to be the mO::it

Table 1. Framework for Evaluation/Outline for Country Reports 1.

Context for Reform !.l

1..2 1..3 1.-+ 1.5 2.

(Pre-JExisting health finance institutions. financial t10ws and incentl\eS Epidemiological and demographic protile and service utilization panerns \facroeconomic and other extra-sectoral conditions Systems for policy making. policy analysis. and use of information Initial assessment and major perceived problems in the health sector

Type of Reform

2.1 .., ..,

2.3 2.-+

Description of the specific reform(s) being implemented Intended changes in health finance institutions. financial t10ws and incentives Description of other relevant reforms being implemented Expectation of how the reform(s) will address identitied health sector problems

3.

Process of Implementation .3 . 1 ., ., .) .)

.

Description of the actual process of implementation of the reform( s) ~ew health finance institutions. tinancial t10ws and incentives :-';ew systems for policy making, policy analysis. and use of information

~.

Assessment of the Effects of the Reform

4.1 4.2 4.3 4.4 5.

Equity Efficiency Sustainability Acceptability

Policy Feedback

5.1 .., ).-

-

Systems and processes' for transmitting evaluative information to policy makers Integration of evaluation into the policy making process

6.

Synthesis of Conditions with Consequences for the Effects of Reform

6.1 6.2 6.3 6.4 6.5

The tinancing reform(s) Other health policies Institutional conditions in the health sector yfanagerial capacity in the health sector Extra-sectoral factors

important phase of the work to be carried out: feedback of policy anal}'sis to decisionmakers. preferably based on data generated by the existing or upgraded health information system. At the end of the evaluation process. each element of the health financing change is .:malyzed with the aim of synthesizing the conditions that have the most Important impacts on the outcome of the policy change. These refer to concurrent policy reforms. institutional and managerial features. and extra-sectoral factors that either facilitate or hinder the success of the tinancing change. The framework provides a common approach to reporting the effects of policy ..:hange. but it is not a methodology for evaluation. Because countries will determine the speciti..: reforms that they will evaluate. because data availability and methodological possibilitles varv to countrv. - considerablv . from countrv " . . and because thev . mav .. be at!~tferent sta.res - of the reform process. it is not realistic to propose that the same analytical method be used m each country evaluation. Therefore. the commonality expected to be achieved in the different country evaluations is with regard to the organization of country reports (i.e .. discussion of context. type. implementation. assessment of outcomes. policy feedback. and synthesis of conditions) and the criteria against which the financing changes \.vill be evaluated. The suggested criteria relate directly to the perceived objectives of health policy and are as follows: a. b. c. d. equity in the utilization and financing of health care; allocational and technical efficiency; financial and institutional sustainability; and acceptability to clients and providers.

Each of these is discussed in greater detail below. It is not expected that it will be possible or desirable for each country to assess specific reforms with respect to every criterion. but it is important to establish the range of criteria against which reforms ..:ould be evaluated. In practice. reform is a process rather than a discreet event. and different countries are likely to be in different stages of the process of health finance reform. Furthermore. changes in health financing policy are likely to be part of an overall package of reforms that may include. for example. dec:tntralization or budget reform. This may cause difficulty in attributing observed changes in output indicators to specific reform measures. In addition. as stated above. data availability varies greatly across countries. and it lS unlikely that the evaluative work in a single country will lead to robust conclusions about the effects of specific reforms on each sectoral objective. Because of the great variation in country circumstances. lists of possible output indicators and questions to answer are presented in the frame\vork with the understanding that no single country will be able to address the full range.

Criteria for Country Participation Since the country-based activities are intended to provide an assessment of the effects of health financing change and feed those results back to policymakers. the principal ..:nterion for participation in this activity is that a country has either recently or will imminently implement changes in its health financing mechanisms on either a national or pdot baSIS.

In other \vords. there must exist the potential for a concurrent evaluation of the policy chan!2e. The health tinancin!2 reforms to be considered for inclusion in this activitv . are llser charges. health insurance. or a combination of the two. However. these may be interpreted broadly to include reforms within these mechanisms or reforms that have the express purpose of changing the way service providers are paid or shifting the relative share of health tinancing derived from general taxation. social insurance contributions. private insurance premiums. or out-of-pocket payments by users. Thus. for example. reforms that could be analyzed as part of this activity include:

-

-

l"ser Fees

• •

•

changes in the level or structure of prices charged in government health facilities: changes in policy or implementation with regard to exemption from charges: changes in policy with regard to fee retention or the management o~ ee revenues:

Health Insurance • • • • expansion of coverage under social insurance or publicly mandated and subsidized Insurance: changes in provider payment mechanisms and/or the role of the fund holding institution(s); changes in the package of services covered by insurance; or changes in beneticiary cost sharing.

6

Part 1.

Context for Health Financing Reform

This sl!ction of I!ach country report should identify the key issues. both inside and outsidl! of the health sector. that led to the decision to implement reform. This \vill include an assessment of who (institutionally) are the key actors andor interest groups moti\ating reform. and. conversely, which groups are resistant to reform. The context should also provide an initial assessment of the equity and efficiency of health sector tinancing and delivery. plus a description of the major perceived problems and issues facing thl! Sl!ctor. To evaluate the effects of reform. it is necessary to define a baseline situation to \\ hich a post-reform situation \vill be compared. An understanding of context is needed to allow for a reasonable understanding of the changes in process and outcomes that occur as a consequence of the tinancing change. The description of context should inform judgml!nt as to the changes that are attributable to the policy change and those that retlect preexisting underlying trends. 1.1 Existing ii . . Jith finance institutions. financial flows and incentives

The institutions and organization of the health sector prior to the implementation of reform should be described because these factors condition responses to specific reform measures, The description should include both the physical and managerial structure of the Sl!nlCcs. with emphasis given to the key institutional actors with control over resource-related decisions. This need not be a long description but should focus on those institutional :md organizational issues of relevance to the reforms being implemented. ft should also allow for an initial assessment of health sector performance (i.e .. equity. efficiency. accl!ptability. sustainability).

Sources of finance. How are health services financed? \Vhat is the role of gO\'ernment (including various levels and entities in addition to the ministry of health). private citizens. employers. donors. and any other groups in paying for health services? \\ nat are the relative amounts contributed by each? To what extent are payments from each source made via taxation, social insurance contributions. private insurance premiums. and out-ofpocket payments? To what extent does government;: 'vvide indirect financing of particular sources through the tax system (tax credits or deductions for private hl!alth insurance premiums paid by employers. for example)? Have these patterns of financing been changing in recent years (i.e .. what are the trends )': An understanding of trends is useful for putting observed changes in context. ft might be useful to present global information on trends in such indicators as the share of public and total health expenditure in GDP. per capita health spending (public and private). and the share of health in total government expenditures. This may also contribute to an understanding of the sustainability of sectoral tlnancing. Try to determine the extent to which each of these sources of finance is progresslvl! or regressive relative to individual or household ability to pay. This should involve :m examination of the ways in which revenues from each source are generated and from whom. fn most developing countries. unfortunately. the availability of reliable sun I!: .lata

needed to estimate. for example. the incidence of taxation falling on different income groups is limited. :';~vertheless. a descriptive analysis of sources of government funds cJ.n enable investigators to make a reasonable" guesstimate" as to whether public tinancing is progressive. \\ nere household survey data exist. comparisons of out-of-pocket spending with household income levels can be made to assess equity in private tinance. 2

"anagement of finance. With the exception of direct out-of-pocket payment by individuals to providers. funding for health services is managed by an intermediJ.te entity between the original source of funds and the service provider. In publicly managed systems. this manager of tinance. or fund holder. is usually the ministry of health. Other examples of fund holding institutions are social insurance funds. private insurers. entitles that combine management of tinance with service delivery (such as health maintenance organizations), or any other institution responsible for purchasing health care services. Within the public delivery system. at what level of the health system are decisions on resource allocation and use made? For what types of decisions? In other \vords. how decentralized is health sector decision making, and what is the specific nature of this decentralization. \\nat are the roles and functions of (either public or private) fund holding institutions in the sector? Are they purely financial intermediaries. or are they actively involved in setting/enforcing rules for service use (e.g .. referral requirements) and monitoring provider performance? In the public sector. are there health fund holders in addition to a central ministry of health? Are these local government entities or government health insurance bodies (e.g., social security health insurance funds)': In the private sector, are the fund holders (insurers) for-profit or not-for-protit? Is there J. single fund holder or are there many? If there :'.:-e multiple fund holders. do they compete with each other. or are their members (the Sl .es of the contributions they recei\'e) detined by law (e.g., by location of residence or employment category)? \Vhere fund holders compete for beneficiaries. it will be important to describe the nature of this competition. In particular, the manner in which government regulates this competition, if it does so at all. is of vital importance for understanding incentive structures. Service providers (recipients of finance). Fundamentally, health care financing is concerned with paying providers. Understanding the composition and distribution of service providers is thus central to understanding the implications of alternati\'e methods or' financing. For example. are services Jelivered mainly from public or private sources: How effective is government at regulating. or at least monitoring, private providers) To what extent is the sector characterized by competition or monopoly in delivery) \'ery often. but not always. the distribution of providers leads to competition in urban areas and monopoly (or the absence of providers) in rural areas. This is a separate issue from the question of the "public/private mix" because publicly-organized systems can have . competitive processes at work (e.g .. as with the reforms to the National Health Ser\'ice of the U.K.), and privately-oriented systems can have monopolistic systems of delivery (e.g .. a private mission hospital in a rural area with no other providers of similar services J.

: For examples of assessments of the progresslvlty of health care finanCing. see Baker. Judy L :!nd JacJL:C:s \ In ~~, Gaag. 1993. "E4ulty In Health Care and Health Care Financing: E~ Idence from Five Developing Clluntr:c:s' ; G E~-:'. Van Doorslaer. Adam Wagstaff. and Frans Rutten. Eds. Equity In (he Finance and Deln·er;,·o) Heailh C..;rt! it! Internal/onal PerspeC(lve. Oxiord Cniverslty Press.

8 Health finance flows and provider payment. The description of the institutional sources. management. and recipients of health sector tinance allows for a depiction of the tlow of funds in the sector. such as the example shown in Figure l.' The tigure presents the tlO\V of funds under a health system \vhere public health services. training. and research are fully funded by government and provided by the ministry of health. Personal health services provided in public facilities are also fullv . funded bv .. l20vernment throul2h the ministry of health. Private providers are paid directly by individuals and by employers on behalf of their employees. There is no insurance fund holder in this example. The figure does not. of course. give the only possibility for the organization of tinancial tlows. but it provides a model for describing financing mechanisms and institutional arrangements in specific countries. A.s is shovin in Parts 2 and 3 below. it can also provide a pomt of comparison of financial tlows before and after reforms are implemented.

-

Sources of Finance

l\ianacement of Finance

Providers and Services Training PubHc Health Research

I

IGov..........t

K

Employers

I 'I I I

MInistry of Health

PubHc Hospitals. Ambulatory Care Providers, and Pharmacies

Individual Citizens

Private Hospitals, Ambulatory Care ~-r----------------1~ Providers, and Pharmacies

Figure I: Depiction of Possible Flow of Funds in the Health Sector Prior to Reform Make an assessment of the efficiency with which government and. where possible. private resources are allocated. Useful data would include levels and trends in expenditures by type of facility or program (e.g., tertiary hospitals. secondary hospitals. health centers.

) This figure (and Figure:! below) and the associated discussion are dertved from a lecture on health tin.ln~:~.~ reform given by Howard Barnum at the Second Workshop on Health in the ~ewly Independent States IS , .n th~ World Bank on 15 December 1993.

I"

vertical programs. central administration. etc.) and by type of expenditure (e.g .. personnel. pharmaceuticals. other consumables. etc.). Have costs been rising rapidly? Do resource allocation patterns give sufticient support to cost-effective public health and clinical interventions: Does there appear to be an excess supply and use of high-cost medical technologies 1

Incentins. The discussion of finance policy and institutional issues should lead to an assessment of the incentives in the health sector prior to the implementation of reform. The incentives to three categories of actors in the sector--providers. consumers. and (where they exist) fund holders--should be described. A number of questions could be addressed to get an understanding of these incentives and their effects. How do the existing organizational arrangements and provider payment methods affect provider behavior with respect to the quantity of cases. intensity of services provided per case. choice of diagnostic and therapeutic technologies. avoidance of patients likely to require extensive care. and quality? \\inat are the incentives generated to consumers by these arrangements and cost sharing obligations for access to care and the type of service and facility selected? 'Nnat are the incentives for fund holders to be prudent purchasers of services for their beneficiary popUlation. induding their incentives to monitor closely the behavior of both providers and consumers? Do fund holders have any incentive (and capability) to avoid covering persons who are likely to consume a large amount of health care resources (i.e .. incentives and ability for risk selection)? To what extent is eJch of these actors at financial risk for health service costs?

1.2 Epidemiological and demographic profile and sen'ice utilization patrerns The epidemiological and demographic stage of a country may go a long way toward explaining the motivation and specific nature of reform. Information on levels and trends in dependency ratios and disease profiles could help put reforms in context. In particular. trends in diseases with potentially important tlnancial implications for the health sector. such as HIV / AIDS. should be identified where relevant. In addition. the demographic features of a country may facilitate or hinder the success of specific financing reforms. Thus, for example, it may be important to provide information on variations in population density within a country, especially when considering financing alternatives in sparsely populated areas. Try to determine the use of health services by different popUlation groups (e.g .. income. age. gender. urban/rural) relative to their need for such care. Ideally. this assessment would be based on survey data on illness prevalence and service utilization for different population groups. Survey data comparing self-reported illness and service utilization andlor treatment costs for different population groups would also be useful. 4 In the absence of survey data, information on the distribution of health professionals. sen·ices.

" Wagstaff. Adam and Eddy Van Doorslaer. 1993. "Equity in the Delivery of Health Care \lethods anJ F'.nu·.n,,> In Eddy Van Doorslaer. Adam Wagstaff. ~lnd Frans Rutten. Eds. Equity In the Finance and DelIver:- o.f H.QIli! C·.lN .in International Perspective. Oxford L'nlver51ty Press.

10 ilnd expenditures (by region and facility type compared to the distribution of the ~0pulation. for example) could give an indication of equity in service availability. I J .\!dcrot:(':onomic and other extra-sectoral conditions Conditions outside the health sector can have important implications not only as J motivation for reform but also as part of the enabling environment for reform. Evaluations should identify those conditions in the macro environment that have such implications and indicate the extent to which these conditions can be affected by policy (health or otherwise).

:\Iacroeconomic conditions. This includes. where data permit. information on employment and the labor market that might be relevant to the ability of people to pay user charges or to participate in formal insurance systems. This could include informattOn on the extent of the population working in the formal sector. the extent of unemployment. labor migration patterns. the seasonality of cash incomes. gender patterns in employmc:~t. and levels of absolute and relative poverty and income inequality. Broad economic conditions with implications for health sector financing also Include intlation (both general and within the health secton. exchange rate changes. trade and budget deticits and the debt burden. and structural adjustment polie c. if any. Other extra-sectoral conditions. Specific institutions must be available for certain tinancing measures to function effectively. For example. rural facilities that charge fees need a means to store collected revenues safely and to ensure that the potential benetits arising from use of these revenues are not lost due to int1ation. Thus. there is a need for J rural banking and investment structure, and the availability of such institutions should be described in evaluations of financial reforms in rural areas. Other factors that may be critical to the success of reforms, especially in rural areas. are the quality and cost of transportation and communications systems. Assessment should also be made of the general capacity of the population to understand the changes being implemented. Thus. J. measure of the education level of the population. such as literacy rates. can help establish the context for what is feasible. Another important contextual factor is a country's political system and the level of development of its political institutions. Local level organizational development may be '.t ..:ritical input into reforms aimed at expanding community management of user fee revenue. for example. Moreover. the extent to which a government can introduce reform without fear of political consequences can strongly int1uence the type of reform or the speed of its implementation. 1. -+ S},:stems for policy making. policy analysis. and use of information

The objective of this activity is to improve policy decisions by injecting analytical information into the policy process. Therefore. it is important to have a clear understanding of how policy is made and the basis for policy decisions. Is information on utilization. expenditures. and cov'erage routinely generated for both public and pri \ate health services'? 'Nnat institutions are involved in generating this information? Ho\.. accurate is the information believed to be? How is this information organized and presented to decision makers in the ministry of health? To what extent does such information serve as a basis for health financing policy? To what extent is this

1 ,

( :

information used for routine monitoring and for management decisions at various levels of the health services?

-

-

!. 5 Initial assessment and major percei\'ed problems in the health lector

Based on all of this contextual information. the evaluation should include an initial assessment of equity. efficiency. acceptability. and sustainability in the health sector. Based on this assessment and pronouncements made by national decision makers. the evaluation should identify the major problems facing the health sector and the principal motivations for implementing health tinancing reform.

12

Part 2.

Type of Health Financing Reform

~.l Descriprion

0/ rhe

specific re/ormfs) being implemenTed

The evaluation must include a detailed description of the policy change that has been or lS about to be implemented. Two broad categories of reforms will be considered in this activity: . user charlZes and health insurance. Does the reform involve user charlZes. - health insurance. or both? If user charges. what has changed? Is it just a change in pnce levels. or are there other elements of the change. such as exemption criteria. retention and use/management of fee revenues. procedures for changing prices. the types of services or facilities to which fees apply. etc.': Similarly. if insurance. what has changed'l Do the changes involve the role of the insuring institution(s). mechanisms for provider payment. services covered by insurance. the extent of the population covered by insurance. cost sharing requirements. fund holding arrangements. merging of different public insurance schemes. etc.: Does the reform involve the changing the mix of public and private tinancing and delivery of care?

-

2.: InTended changes in health finance institutions. finanCial flows and incenTi .... es

~ext. the planned reforms to the institutional structure of the health sector should be described. This includes planned changes in existing institutions and any new institutions to be created as part of the reform that relate to the sources of finance. management of finance. and provider institutions. Based on this. a depiction of the intended financial nows in a reformed health system can be presented. such as the example shov>n in Figure 2. In comparison to the baseline tinancial nows shown in Figure 1. the new tlows shO\\TI in Figure 2 will arise from a reform to establish a social insurance fund. The ministry of health will continue to finance public health. training, and research. but it \vill no longer pay for personal health services directly. Personal health services will continue to be provided by a mix of public and private providers. but these will be paid primarily by the new social insurance fund. The fund will be tinanced by contributions from employers and employees (the arrow from "individual citizens" to the fund), as well as receiving a subsidy from central government. Individuals will face copayments when seeking care from private providers, and they may continue to purchase services directly from the private sector. The description of the planned reforms in institutions and financial nows should allow for an assessment of the incentives to be created by the new arrangements. The same questions raised in the corresponding part of the Context section should be addressed here. For example. how will provider payment methods change, and what are the expected consequences of this change: How will the incentives facing providers. consumers. and fund holders described in the Context section change?

2. 3 De"cription of other relevanT reforms being implemenTed The financing change may be implemented by itself. or it may be part of a package of reforms being introduced simultaneously. If the latter holds true. it will be important to understand the potential effects of the other reforms. These concurrent reforms could ha\ e

Sources of Finance

Management of Finance

Providers and Services

Govenunent Ministry of Health

\ i

Employers

;\ I·r-----I

Public Hospitals. Ambulatory Care· Providers, and Phannacies

Social Insurance Fund

Individual Citizens

Private Hospitals, Ambulatory Care I--+-------------~Providers, and Phannacies

Figure 2: Depiction of Possible Flow of Funds in a Reformed Health Sector important consequences for the success of the health financing reform being assessed. In addition. the degree to which changes in outcome and process indicators are attrIbutable to the tinancing reform vis-a-vis other reforms may be difficult to determine without a good understanding of the likely effects of these other measures. Therefore. concurrent reforms that are likely to have important implications for incentives and health sector performance should be described.

2. -I Expectation of how the reJorm(s) will address identified health sector problems The Context section concluded with an identification of the major problems and issues facing the health sector and the motivations for health financing reform. Here. the objectives of the reform should be described. What are the explicit. and to the extent that it is possible to determine, implicit objectives of the change? Objectives might be detined in general terms. such as increasing revenues, improving quality and efficiency. or improving equity. Alternatively, they may be specified in quantified terms, such as expanding insurance coverage to 20 percent of the population or increasing cost recovery in public facilities to 15 percent. Based on the information presented in both the Context section and the Type section to date. describe how the reform will address the problems that have been identitied in the health sector.

14

Part 3.

Process of Implementation

J. / Description of the actual process of implementation of {he re{orm( s}

The previous section dealt with the intended form of policy change. This section should describe the process of how the tinancing reform is actually implemented. Therefore. It must be prepared after the reform has been implemented and address whether the changes described in Part 2 have been implemented as intended. Because it takes time to fully implement reforms. and because reforms are subject to modifications. this description of the reform process should be revised periodically as needed. This analysis of the process of reform implementation should indicate the extent to \\ hll.:h the objectives of the reformers have been realized. This should include a discussion of the roles played by different stakeholders (e.g .. professional groups. private insurers. civil servants. politicians. consumer groups) in the sector and in the reform process.

J 2 Sew health finance institutions, financial flows and incentives The same issues raised in Part 2.2 of the framework should be addressed here. Have the intended institutional reforms in the sources and management of health tinance and in the provision of services actually occurred? Are the new financial flows as depicted in Figure 2. or has something different evolved (if so. a new tigure may be useful for descripti\e purposes)? .A.re incentives to providers. consumers. and fund holders functioning as expected? It is often the case that health policy reform leads to changes and consequences that were not intended by the reformers. What unexpected changes or unanticipated consequences of the reform process have occurred, if any? \Vhat problems are these likely to cause?

J 3 .Vew systems for policy making, policy analysis. and use of information Have these systems changed from those described in Part 1.4? What is new in the policy making process? Has government's capacity to analyze policies been strengthened? How' Have incentives to use information changed? Have systematic measures been implemented to feed health information to policy makers and managers at different levels of the system) \Vhat kinds of health information (e.g., utilization. expenditure. or cost recovery data) have proven to be most useful for influencing policy and managerial decisions? \\nat have been the initial costs of setting up these new systems. and what will be needed to sustain them over time?

i5

Part 4.

Assessment of the Effects of the Health Financing Reform

Rdorms in health financing are a means to achieve sectoral objectives. These objectives were described above in the Introduction section. and it \vas suggested that they can be aperatlonalized as cnteria by \vhich tinancing reforms could be assessed. The remainder af this section suggests possible indicators for each of these criteria. This is not meant to be an exhaustive listing. and evaluators are encouraged to develop additional or substltute indicators based on the availability and limitations of data. \\ nile focusing on the specific objectives of health policy discussed below. the evaluations should address a number of broad questions. What changes have occurred \vith regard to health sector performance since the implementation of the reform? To what extent are changes in health sector performance attributable to the reform? To what extent has the reform achieved its objectives and addressed the perceived problems of the health sector: What are the incremental administrative costs of implementing the reform and monitoring its effects? "Wnat new problems have arisen that may be attributable to the reform? Evaluation of the effects of reform requires identification of performance indicators. systems for data collection. and a methodology to attribute changes in indicators to changes in policy. To improve confidence in the attribution of changes to specific policy measures. methodological approaches should be employed. where feasible. that take advantage of "natural experiments" or are otherwise able to use control groups where no policy change has occurred. 5

-1.1 Equity Equity in finance. This aspect of equity reflects a comparison between an individual's or household's level of income (ability to pay) and its actual direct and indirect payments for health services. Indirect payments include taxes paid to government. some of which are used to fund health services. As suggested in the discussion of the sources of tinance in Part 1.1. an attempt should be made to determine the progressivity of different funding sources and whether finance has become more or less progressive as a consequence of the reform. In general, has the distribution of contributions to sources of finance by income levels changed? How much, and in \;hat direction? For tax-funded services. an assessment should be made of any changes that may have occurred in the progressivity of taxation. preferably with the assistance of the ministry of finance or a national taxation authority. This would help to answer the question of who is paying for government health services. Other methods might be employed to try and answer the question. "who is paying what for health care?" To examine part of the equity impact of exemption

; .-\ good example of this is an evaluation of the introduction of user charges with revenue retention In [he health centers of one province of Cameroon. The change was implemented gradually. which enabled the researchers co compare utilization In three health centers before and after the policy was Introduced v.lth utilization dUring the ,am~ time periods In two other health centers in v.hlch the policy had not yet been phased In. Such an .lDprOaCh ';0n(~rJi, :0, (he effects of underlvIng factors affect In I! utilIzation In all centers so that the different experiences In (he' ;!\per:men:l. and "control" health'ce;ters can be more- clearly attrIbuted to the policy change. See Lltvack. jennie [ .lna L':.lude Bodar!. [993. "Cser Fees Plus Quality Equals Improved Access to Health Care: Results of a Fteid E. . perIme~[ :n Cameroon." SOCial SCience and lledlclne 37(3):369·383.

16 mechanisms. for example. a facility-based survey might help to determine if and the ~xtent to which. the income and asset levels of paying patients are greater than for non-

paying patients. Equity in utilization relative to need. Assessment of this aspect of equity seeks to determine the extent to which utilization of health services renects the need for those services rather than the income level of patients. For government-funded services. the evaluation should seek to determine the impact of the reform on the capture of governm~nt health subsidies by different population subgroups. .-\ comparison should be made with the information reported in Part 1.2 to see if there have been changes with respect to equity in utilization since the implementation of the health tinancing reform. So;ervice utilization data can make an important contribution here. How has the distribution of morbidity (disease burden) by population subgroup. such as income class. gender. age. employment status. insurance status. urban/rural residence. etc .. changed since the reform was implemented? To what extent does the distribution of service use ret1ect the disease burden of these different subgroups? This will require population-based data on disea:;e incidence and prevalence by subgroup in addition to service utilization figures. Other potentially useful indicators of geographic equity in service availability and use are measures of the distribution of health sector personnel. facilities. budget. and utilization per capita for different regions of the country. Patient origin surveys can be another source of data on equity within a geographic area by providing information on the percentage of visits/admissions to persons living within a given radius of a health facility.

·l2 Efficiency Allocational efficiency. This aspect of efficiency concerns the extent to \vhich health sector resources are distributed to their most cost-effective uses. Health systems are often characterized by inefficient patterns of resource allocation, and the evaluation should assess whether the reform has either improved or worsened these patterns. The description of the allocative efficiency of the sector prior to reform (as per the discussion of health finance nows. provider payment, and incentives in Part 1.1) should serve as a point of comparison for the period after reform. Several issues indicative of allocative efficiency could be addressed here. For example. has the share of tertiary hospitals in total health expenditure increased or decreased? What has happened to the shares of government spending on preventive vs. curative services, or. if possible. to the shares of spending on services of high vs. low cost-effectiveness. Have there been changes in the rates of use of speciEc medical technologies? One indicator of allocational inefficiency is a pO<lrly functioning referral system wherein patients crowd hospitals for services that could be provided in a less expensive setting. To get a sense of whether this pattern has changed over time. a comparison could be made of the case mix of patients in health centers and hospital outpatient departments. Similarly. changes in facility bypass and referral rates should also be tracked. not only for purposes of broad policy evaluation. but also because this information can be potentially useful for resource allocation decisions facing district or regional health sector managers. Technical efficiency. . As with allocative efficiency. . there should be an attempt to assess the impact of changes in tinancing institutions and provider payment incentives on technical efficiency. This aspect of efficiency concerns the management and use of

resources that have been allocated \vithin the health sector. Technical inefficiency exists when the costs of providing a unit of a given quality of service are higher than necessary. \\ bile an absolute level of technical efficiency can not be easily detined. assessments of relative efficiency can be made by comparing different facilities or the same facility at different periods in time. Thus. one way to examine changes in technical efficiency is to compare the unit costs of service provision. For such comparisons to be valid. however. they must account for any possible differences in quality or the mix of patients treated. For inpatient care. hospital service statistics (bed turnover rate. bed occupancy rate. and average length of stay). within and across various types of hospitals and specialties. are useful indicators of the relative technical efficiency of patient management. .-\s with unit cost comparisons. the validity of conclusions dra\Vtl from this analysis depends on the extent to which variation in quality and case mix has been incorporated.

·l3 Sustainability Financial sustainability. This refers to the capacity of the reformed health system to provide a sufficient level of finance to enable it to function effectively over time without needing a substantial injection of external support. This implies that the mix of funding sources provide stability in tinance over time and that there are adequate mechanisms in place to contain the growth of costs within the availability of finance. Possible indicators for evaluating the impact of the reform on financial sustainability include: changes in cost-recovery ratios in government health facilities. changes in the mix of funding sources for health services and patterns of expenditure of revenues from these funding sources. comparison of a health/medical price index with the general consumer price index. the share of total health spending in GOP and of government health spending in total government expenditure. and per capita utilization rates for specific services. such as hightechnology diagnostic and therapeutic services and drugs. Institutional sustainability. This aspect of sustainability relates to the capacity of health system management to develop and implement measures to support effective reform and is thus related to the managerial capacity of the ministry of health. An important aspect of the institutional sustainability of a reform is the cost of administering it effectively. Thus. for example, a relevant indicator would be the financial costs or staff-hours used to administer a fee or insurance reform at both the facility and ministry levels. Similarly. the costs of monitoring and evaluation of the reform should be assessed as well. [1' the capacity to implement a reform effectively does not exist. then cost is not the relevant factor, at least in the short run. Thus. the evaluation should assess not only the administrative costs of implementing and managing a reform but also the skills and systems needed for the reform to be effective.

·l.J Acceptability Quality. The acceptability of health services to both clients and providers of care is closely related to the quality of services available. Evaluation should seek to determine whether the reform has led to improvements in indicators of quality. Possible indicators include the availability of drugs (measured. perhaps, by the frequency of stockouts l. assessments of prescribing practices or other evaluations of service protocols. and changes in staffing patterns.

18 Consumer satisfaction and choice. Improving service quality is an important factor In improving client satisfaction with the health services. but there may be other aspects of satisfaction that quality indicators may not reveal. One factor that is likely to contribute to satisfaction is expanded opportunities for choice by consumers. Thus. reforms that generate expanded choice are likely to be more acceptable. Consumer satisfaction can be assessed directly by surveys or focus group studies of user perceptions. \{ore objecti\e indirect measures of satisfaction include rates of facility bypass. rates of participation in voluntary insurance schemes. and. in fund holding schemes. rates at which patients chango;! their principal purchaser. general practitioner, or health center. Acceptability of reforms to providers. This can be critical to the political viability of reform. Evaluators should review the role of professional provider organizations in tho;! reform process and whether this role has changed over time. In addition. providers at different levels of the health system can be surveyed periodically to determine their lo;!\ds of satisfaction with the reforms.

Part 5.

Policy Feedback

The principal objective of the evaluations is to improve national policies. Therefore. this component of the e\'aluation process is the most important. and the success of this activit: hinges on whether periodic evaluation is integrated into the policy making process. If policy makers perceive that their decisions are improved through the information generated by evaluation and that it can provide answers of importance to them. the process of evaluation is more likely to be incorporated into the routine operations of ministries of health rather than being tied to specific external projects or studies. This incorporation is important because the full effects of any tinancing change are unlikely to be observed during the planned period of WHO support. Thus. the evaluation process needs to be institutionalized within each country so that tlndings can be used to periodically tine-tune policy implementation. In addition. because refonn is a process rather than a discreet event. continuous evaluation will be needed as financing policies evolve.

5.1 Systems and processes for transmitting evaluative information to policy makers The evaluations should report on the methods that have been used to infonn policy makers of relevant findings. Were special meetings or workshops arranged. or were pre-existing ministry meetings used for this new purpose? This section of the evaluation should .date the infonnation presented in earlier parts of the framework (1.4 and 3.3) that described infonnation systems. the policy process. and the use of infonnation for policy purposes.

5.2 Integration of evaluation into the policy making process The importance of fully integrating and institutionalizing evaluation into the policy process was discussed above. An assessment should be made regarding the extent to which the evaluation has been integrated to date. and the likely prospects for this integration to continue. The way in which the evaluations were used as well as the process by which this use occurred should also be described. Was analytical infonnation provided by the evaluation used to change policies? If so, how? Do policy makers perceive evaluation to be useful. and are they willing to commit human and financial resources to continue and strengthen this process?

20 Part 6. Synthesis of Conditions with Consequences for the Effects of Reform

It is hoped that each evaluation will suggest reforms that will lead to improved outcomes. Howe\·er. many of the conclusions reached will involve considerable judgment because tinanclnl.! be difficult to clearlv - chanl.!es do not occur in a vacuum. and thus it mav . attribute changes in outcomes to changes in health tinancing policy. Indeed. organiz:.uionai reforms that occur as part of a financial reform (or vice versa) should be considered as elements of the financing change being studied. There may also be other factors. such as overall trends in government budget allocations. epidemiological shifts. or macroeconomlC changes. that int1uence health sector performance. Because of the difficulty of controlling for each of these factors in a systematic manner. investigators will need to rely on thelr judgment to attribute changes in outcomes to changes in policy and implementation.

-

Based on an analysis of the process and outcomes of the change in health tinancing potic:. investigators should attempt to identify the conditions associated with the success or failure of the reform relative to the achievement of sectoral objectives. These conditions include the characteristics of the specitic reform. other health policies or reforms. institutional features of the health sector. managerial capacity within the health sector. and factors outside of the health sector.

6.1 The financing re!orm(s) First is the nature of the specific policy itself. For example. experience to date has shown that for user charges to lead to improved quality and thus to the potential for improvements in efficiency and access to better quality services. some or all of the revenues collected must be retained and used by the collecting facility to improve quality. Experience with health insurance indicates that unregulated fee-for-service reimbursement of providers leads to rapid escalation of health sector costs and excessive use of protitably priced technologies. However, less is known about the effects of other features of policy in different contexts, and each evaluation should identify the critical aspects of the reform that affected health system performance. Empirical issues related to the specific reform should also be addressed here. For example. it is known that it is possib;e for an increase in fees to improve access to care of acceptable quality if the revenues are ;etained to improve quality in local facilities. enabling those who previously did not seek care because of high transport costs to do so. However, the objective circumstances under which such a policy will succeed are not knO\\TI. At what point does the level of user charges create the same barriers to access previously generated by the time and transport costs to the more distant facility v. here the same level of quality care was available? Empirical questions such as these need to be addressed. even in a qualitative manner. because the same policy measure is unlikely to yield the same results in different country settings.

6.2 Other health policies It is cornmon for reforms to user charges or health insurance to be implemented In conjunction with other sectoral change~. Important examples of these other reforms

21 include decentralization. the creation of tinancially or managerially autonomous hospitals. and the prioritization of go\'ernment health spending on an "essential package" of clinical and public health services. The implementation of reforms concurrent with tinancing change complicates the attribution of the effects of specitic reform measures. As \\'ith tinancing reforms. these other policies are also intended to affect sectoral objectives. y{oreover. they are likely to interact \\'ith tinancing reform. such that the effects of the tinancing change may be quite different depending on whether concurrent reforms are present. The evaluation should attempt to show the effects of these other policies on the tinancing reforms and its impacts on sectoral objectives.

6,3 Institutional conditions in the health sector These conditions refer to the structure of the health sector and the role of the ministry of health. It includes such elements as administrative arrangements within the ministry. procedures for accountability. the existence of specialized regulatory bodies. the magnitude and composition of the private health sector. and the quality and coverage of health management information systems. Little is known about the forms of sectoral organization that are most supportive of specitic financing reforms. Evaluation could suggest. for example. whether the success of an increase in user fees in improving sustainability is enhanced by a decentralized decision making structure. Similarly, each evaluation could suggest the types of organizational structures that facilitate efficiency in insurance systems. Information systems are another important element of the institutional composition of the health sector. Therefore, the analysis of institutional conditions should indicate the kinds of information needed on a routine basis to support the reform (e.g .. per capita utilization and cost data :~'r insurance schemes) and the systems needed to generate this information. 6. oJ J..Ianagerial capacity in the health sector

Because it is difficult to quantify managerial capacity. these conditions may be the hardest to specify. However. evaluators should try to make a qualitative assessment of the managerial capacity needed to successfully implement change. For example. what skills are needed at the facility level to convert additional fee revenues into quality improvements? What skills and systems are needed to manage social insurance schemes? Answers to these kinds of questions are critical because managerial capability provides the link between defined policies and the effectiveness of their implementation.

6.5 Extra-sectoral factors A final category of conditions that can determine the feasibility and outcome of specitic health financing reforms are factors that are external to the health sector. These were described in Part 1.3 of the Context section and include a number of economic. institutional. and cultural forces that are largely outside the influence of health sector decision makers. For example. a country's macro-economic situation is of critical importance. Levels and trends in economic growth. formal sector employment. and inflation may determine the impact of changes in user charges or attempts to expand insurance coverage. Each evaluation should indicate the importance of, where relevant. the pre-existing extra-sectoral factors described in Part 1.3 and changes in rhese conditions during the period of reform implementation.

Timing of Country Activities Each evaluation plan should include a schedule indicating when activities will be implemented and various parts of the country reports completed. The example provided m Table .2 suggests that certain activities can begin immediately. that is. once the reform to be implemented has been identitied and the decision to evaluate it has been taken. Three main activities can begin at this time: preparation of the report section on the Context for reform. preparation of the report section on the Type of reform. and preparation of the protocol for evaluation of the reform. The protocol should indicate the process;:.d outcome indicators to be monitored. so collection of baseline data should begin \\hen the protocol is complete. The Context and Type sections of the report do not depend on this data collection. and so they can be completed within the tirst few months after the decision to evaluate reform has been taken. Description and assessment of the Process of Implementation should begin shortly after the reform is implemented and be revised periodically as needed. given that the reform process is likely to change over time. with modifications to the current reform or the introduction of new reforms. Table 2. Indicative Schedule for Health Financing Evaluations Timing Immediate Activity Reports

Draft: Context. Type. Context; Type; development of evaluation and evaluation protocol protocol. data collection systems. and reporting processes: collection of baseline data Final: Context. Type. and evaluation protocol Draft: Process Revised draft: Process Draft: Assessment of Effects, Policy Feedback Final: Process. Assessment of Effects. Policy Feedback. Synthesis of Conditions

Shortly after Process: establishment of systems for data implementation collection and reporting of tlndings to policy of reform makers 6-12 months after implementation of reform 12-18 months after implementation of reform Ongoing ongoing assessment of Process; data collection. analysis. and feedback to policy makers (initial Assessment of the Effects of Reform and Policy Feedback) ongoing assessment of Process; ongoing Assessment of the Effects of Reform; ongoing Policy Feedback; Synthesis of Conditions process assessment. data collection. analysis. and feedback to policy makers

The protocol should include plans for process and outcome indicators. data requirements. new data collection tools (if any). analytical methodology, and mechanisms to transmit findings of the evaluation to policy makers. After the protocol is complete. work should begin on developing any needed data collection systems and on establishing channels to transmit tlndings to policy makers. Baseline data should also be collected at this time. .-\

few months after the reform has been implemented. collection of new data should begIn. and a tirst draft report on the Process of Implementation should be completed. Enough data should be collected by the period between 6 and 12 months after the implementation of reform to allow for an initial assessment of the effects of the reform. ,-'r at least the directIOn of these effects. The evaluators should have reported to jeclslon makers on their rindings to date and recommended if any changes in the reform process appear to be indicated. By the end of the tirst year after the implementation of reform. J. draft report on the Assessment of Effects and the methods and consequences of Policy Feedback should be completed. Over the next 6 months. evaluation of process and outcomes should continue. as should the feedback of tindings to policy makers. Based on this further analysis. the evaluators should attempt to Synthesize the Conditions with the most important implications for the effects of reform. This synthesis should be mcluded 10 a tinal report. along ..\lith revised versions of the sections on the Process of Evaluation. Assessment of Effects. and Policy Feedback. The evaluation work will be supported by \VHO for a 12-18 month period following the implementation of reform. but it is hoped that evaluation will have become 10tegrated into ·the national policy making process by that time. and that countries will continue to evaluate reforms beyond this period. Therefore. collection of outcome and process data. analysis of these data. and the transmission of analytical findings to decision makers should become ongoing activities within ministries of health.

Annex 8

Methods for evaluating the effects of health financing reforms Barbara McPake, Health Economics and Financing Programme l , Health Policy Cnit. London School of Hygiene and Tropical Medicine

L

Introduction

This paper aims to identify the range of methods which are available to evaluate health financing reforms. and to discuss ilieir relative usefulness in different situations. It focuses on less resource intensive methods. and less sophisticated analytic approaches, with the intention that it \\ill guide relatively small scale evaluative studies which can be conducted with limited specialist input. Two main sections follow. The first discusses indicators which might be used to measure the effects of health financing reforms on the following objectives:

* * *

* * * *

allocative efficiency; technical efficiency; equity in utilization and access to care for disadvantaged groups; equity in finance; financial 3ustainability; institutional sustainability; and administrative costs.

Alternative data collection methods which might be used to measure these indicators are considered, and the accuracy with which each indicator is likely to reflect the achievement of each objective is discussed. In the second section, analytical approaches which aim to judge the extent to which trends in indicators and differences in indicators emerging from comparison can be attributed to financing refonn are discussed. Health financing refonn potentially covers a wide policy area. In this paper, discussion will focus on changes to and introduction of user charge systems; and changes to and introduction of insurance systems. These currently dominate health financing refonn agendas and themselves contain widely differing policy changes. A distinction within these two categories which is important for selecting methods of study is the level of the system at which the change is made and is to be evaluated. Thus, methods required for evaluating the introduction of a national health insurance system are likely to be quite different from those required to evaluate a village le\'el operated prepayment system. After consultation of a selection of literature which evaluates health financing changes the following list of indicators of the success of changes with respect to each of outcomes \\as produced. Clearly, not all these indicators relate to all levels and types of change. Although listed

by objective, they are not all exclusively indicative of a single objective. In particular. there are major overlaps between indicators of allocative and technical efficiency, and of equity of utilisation. Although the remainder of the paper is divided by objective, points of overlap are noted as they arise.

1.

allocative efficiency

[a] [b] [c]

[d] [e]

[f] [g] [h] [i]

[j] [k] [1] [m] [n]

% expenditure on all hospitals % expenditure on tertiary and quaternary hospitals distribution of resources among referral levels % expenditure on 'public health' shares of expenditure on identified high and low priority activities shares of staff time on identified high and low priority activities changes in patients' utilisatior. of different sectors (traditional, public, mission, private) level of fust contact with a PHC unit levels of self-prescribing utilisation trends by type of utilisation utilisation rate of insured relative to uninsured populations

unit expenditures in insurance facilities relative to MoH facilities resource availability in insurance facilities relative to MoH facilities unification of social security funds number of drugs prescribed per OP visit % ofOP cases receiving antibiotics % of OP cases receiving injections % of children under five receiving ORS relative to anti-diarrhoeals cost per case for drugs per OP visit % days stock outs of specific drugs level of provision of diagnostic equipment average cost of a curative outpatient (first) visit average costs of other units of activity

2.

technical efficiency

[a]

[b] [c]

[d] [e]

[f] [g] [h] [i] .., .).

equity in utilization and access to care for disadvantaged groups

[a]

[b] [c] rd] r -.::

i

utilisation by facility utilisation of services by specific population groups urban/rural differentials in utilisation source of treatment before death by income J.nd demographic groups '), e"\ernrti()n~. rric~ redllctl()n~. dl'rl:- ,!I i, ".\ _',;

[f] [g]

[h] [i]

U1 [k]

[1]

and free insurance entitlements correlation between [e] and poverty or income levels by geographical area social and economic characteristics of those recelvlng concessions price comparisons between public and other sectors distribution of providers and facilities insurance coverage levels of disadvantaged groups economic, social and geographical characteristics of insured and uninsured popUlations total membership of locally based health insurance schemes

4.

equity in finance

[a] [b]

[c]

[d] [e]

[f]

[g) [h] [i]

[j] [k]

distribution of taxation burden breakdown of sources of fmance for insurance programmes by government, employer, employee and patient (co-payments) breakdown of sources of fmance for individual facilities by government, insurance, direct payment and other. breakdown of sources of finance for health sector overall geographical distribution of government health expenditures sectoral distribution of government health expenditures (between direct public sector expenditures, and support to insurance and private sectors) geographical distribution of insurance agencies' expenditure distribution of utilisation of different sectors by socio-economic group extent of cross-subsidy within a facility through stepped charges (eg. private wards) extent of cross-subsidy between different insurance funds comparison of contributions. income levels and benefits across different insurance funds

tinancial ~L1q~lin~lhilit\

[a 1 [hi

cost recovery ratios I) ,\

()t' SJp

h~l\\l?en re\~nlle Jf1rll'()~r t'i!',;,-'j

l"

[c]

donors. government and other sources relative gro\\l1h rates of health sector expenditure. health sector price index. GNP and specific incomes of main paying groups (government, insurance agencies and direct payers). adequacy of organizational and admini~trative structures adequacy of human resource fonnation administrative cost of a user charge system in relation to revenue raised administrative costs of insurance agencies In relation to their expenditure

6.

institutional sustainability

[a] [b]

7.

administrative costs

[aJ [b]

2.

Indicators of the objectives. data sources and data collection methods

2.1

Allocative efficiency

Allocative efficiency requires that resources are allocated to the activities in which they have the highest value. Using resources for low priority activities which have a small impact on health (such as high technology interventions of dubious efficacy) while higher priority activities which have a major impact on health (such as immunisation programmes) go unfunded is an example of allocative inefficiency. A number of indicators ([a] to [c]) aim to measure the extent of allocation of expenditure by level of care. The assumption underpinning the use of these indicators is that allocative efficiency is improved by the reallocation of expenditures to lower levels. As noted above, these indicators could equally be included under the objective 'equity of utilisation' under the assumption that this too is improved by the reallocation of expenditures to lower levels. There are few countries in which an analysis of distribution of expenditures by level does not indicate substantial allocative inefficiency and inequity of utilisation. However. it does not necessarily follow that reallocation towards lower levels represents improvement. Using the share of expenditures allocated to all hospitals [a] as a measure of allocative efficiency or equity of access implies that all hospital expenditure is low priority. In practice, hospital activity includes supervision of primary health care and sometimes covers outreach clinics which offer high priority activities. The role of the district hospital in quality assurance of activities throughout the district is now well recognised (for example, World Bank, 1994). Reduced expenditure on district hospitals could affect high priority services and if expenditure reallocated elsewhere is affected (for example because the supervision system breaks dovm). allocative efficiency could worsen. The indicator therefore needs to be used with caution. I: may be sati!r to tocus on the share of expenditure allocated to tertiary and quaternary ho-r 1tals fhi 111 111<.:':,,-,. hi~11 rrinrIry ~lc:i\iti\?s ~lr(' certainly te\\er If a rnC\r CNtntn Inc1uck>, .,,1\

interventions with a cost per DAL Y of less than $300 in an essential package which identities highest priority interventions, the estimates of Jamieson et al. (1993) suggest very few interventions otTered by referral hospitals will be included. This suggests that the appropriate level of public expenditure on referral hospital services is almost nil. While this is oversimplistic, both politically and in the context of identifying the technically appropriate balance of expenditures within a national health system, it may be safe to assume that any reallocation of resources from this to other levels of care is an improvement in allocative efficiency. Nevertheless, choice of this indicator ignores improvements arising from changes in expenditure allocations between other levels of care and an overall assessment of the distribution of resources among referral levels [c] in conjunction with infonnation about the uses of expenditures \ViII often be the most appropriate course. An alternative to the share of hospitals in expenditure is a direct estimate of the share of public health in expenditure (indicator [d]). To use this indicator 'public health' must be defined in such a way that existing budget categories can be used to gain an indication of its share in the total. Sometimes this implies a broad a definition For example, a given level of care alongside any public health related programmed activity financed by the institution in question, such as occupational health or environmental sanitation might have to be equated to 'public health'. Other budgeting systems may allow a more detailed breakdown of expenditures within levels of care. Clearly, the more specific the breakdown possible, the more meaningful the indicator.

The base of these indicators could be total national health expenditure, expenditurt:' from the Ministry of Health budget, expenditure by insurance institutions, or any other source which the health financing refonn is expected to affect. Total national health expenditure is not usually readily available, unless a specific study has been done, since it requires some estimate of the size and distribution of private sector expenditures not only in large private and mission hospitals but in small private clinics and the traditional and informal sectors. Estimation of this is unlikely to be within the scope of small scale studies. since household surveys are almost the only way to get comprehensive infonnation about such a wide range of sources of care. An existing study of the extent and nature of private expenditure. even if only carried out once, could indicate the likely trend in distribution of total expenditure if it is thought that a refonn has changed the balance between expenditures in the public and private sectors, or between insurance covered service provision and unorganised individual use of health services. The data required to estimate hospital share in financing institutions' expenditures will usually be fairly readily available from the institutions themselves. At local level, for example within a specific facility, indicators of allocative efficiency can be more specific ([e] and [f]). These are unlikely to mislead but are not sensitive to changes in utilisation patterns between facilities, levels of care or sectors [g]. Indicators [e] and [t1 may also only imperfectly reflect changes in utilisation patterns of the facility itself - for example from more to less serious cases. or from curative to preventive service use. The level of tirst contacts with a PHC unit [h] and the extent of self-piescribing [i] are indicators of allocative efficiency l 1l) the e.\tcnt that they CJn he asslImed to rerre~ent senices of high or lcm pril,riry respec:i\ e :,

These indicators might also retlect technical efficiency if it is believed that sectors differ in this respect too. Data to measure [e] to [f] could be derived from a sample of facilities and the use of the "Cost. Resource Use and Financing Methodology" (Hanson and Gilson. 1992) or similar. Indicators [g] to [i] may be based on facility utilisation data or household survey where available. A number of indicators are specifically relevant to an analysis of the introduction of insurance. or reforms to insurance mechanisms ([k] to [n]). Since most of these ([k] to [m]) measure relative expenditure and resource concentrations in insured and uninsured populations and facilities. they are also likely to reflect both allocative efficiency and equity of utilisation (since in most countries insured populations are both richer and in better health). However, these indicators are quite difficult to interpret. The introduction or reform of insurance may, for example, add to the total revenues available to the health sector, and an increasing proportion of resources allocated to the insured does not preclude an increasing total resource allocation to the uninsured. It is often argued (but has rarely been demonstrated) that provision of separate facilities for the insured leads to better targeting of public health expenditures, for example. This argument suggest one limitation to indicator en] also, which assumes that unification of social security funds is equivalent to efficiency improvement (technical as well as allocative) without assessing the effect of this on total resource availability and targeting of public sector expenditure. Multiple unintegrated social security funds are usually perceived as inequitable (utilisation), allocatively inefficient and technically inefficient. There is assumed to be an inverse relationship between the income and health status of fund members and the resources allocated, and facilities and administrative resources are assumed to be wastefully duplicated. Assessing whether integration produces anticipated improvements requires evidence from other indicators, however. Data relating to the insured population, expenditures of insurance funds and resource availability in insurance facilities can often be obtained from the routinely collected data of the insurance funds themselves. Some public sector comparisons are equally easily available (for example total public expenditure and the number of the uninsured). Utilisation and facility based data may require specific studies in a sample of public facilities. Care must always be taken to establish the extent of comparability of data taken from different sources by establishing the methods of compilation and collection of data.

2.2 Technical efficiency Technical efficiency requires that once the activity or output mix has been determined (tor example the mix of curative and preventive services to be offered), the activities are carried out without wastage of inputs and at the minimum possible cost. Usually, the most important cost components for health service delivery are salaries and drugs (Hanson and Gilson 1992). In the systems of some middle and most high income countries. technological interventions start to assume a more important role. Theretore indicJt<Jrs (It' :lr(1n)f1ri~lte dru~ use ([ a1 to [tl) are imr()rtzlnr indic3tors oftechnic3] eftici<...'!K: Th_"'\..· _,rc ; -" - .'

from the International Network for the Rational Use of Drugs (INRUD) guidelines (fNRUD. 1991). Although there is no consensus as to the appropriate absolute levels of each of these indicators, in most countries it can safely be asswned that a smaller nwnber of drugs per OP visit. lower proportions of outpatient cases receiving antibiotics and injections and higher proportions of under fives receiving ORS relative to anti-diarrhoeals represent technical efficiency improvements. Cost per outpatient case for drugs is more ambiguous since drugs may be overprescribed when available but often absent altogether. The indicator therefore needs to be used in conjunction with an analysis of existing problems. These indicators may be sensitive to some financing policy changes if these imply different incentives to prescribers. For example a change from cost per service reimbursement to cost per case may result in reductions of drug cost per outpatient case. Although salary costs are important it is not clear that higher or lower levels represent greater efficiency. Indicators based on salary costs are theretore not recommended. In middle income countries, it may be useful to assess trends or make comparisons of levels of provision of diagnostic equipment [g]. Again, it is not easy to identify the appropriate level in absolute terms. If rapid growth is noted following financing change or a major difference is revealed between insured and uninsured facilities, for example, there is likely to be cause for concern. Even if appropriate levels of provision have not yet been surpassed, the experience of many countries suggests that such cost elements have a tendency to evade control, eventually representing serious divergence from technical efficiency. Finally average (or unit) cost comparisons ([h] and riD might be used to assess technical efficiency but there are important limitations which indicate the need for caution. In those countries in which health services are believed to be under funded, increases in unit cost (which might arise because drugs are now available when not before, for example) may indicate technical efficiency improvement. In other countries in which cost containment is believed to be a problem, such indicators may be more straightforward. In both cases, ensuring that 'like with like' is being compared should ensure that a comparison of Wlit cost is more meaningful. In these circumstances, higher unit costs may be explained by under-utilised staff, equipment or facilities which might not be noticed by the other indicators discussed. In circwnstances where unit cost differences are explained by drug use or diagnostic equipment use differences. it is better to measure this directly, giving more opportunity to consider what the appropriate levels might be. Data for the measurement of these indicators mostly require facility level studies. INRUD (INRUD, 1991) have recommended methods for the drug use indicators. Hanson and Gilson ( 1993) recommend methods for measuring unit cost indicators which take into account the limitations discussed above. Some countries have data at national level regarding provision of diagnostic equipment, and identification of nwnbers of specific items available at lower level such as facility or district is a simple matter. 2.3 Equity in utilization and access to care of disadvantaged groups utilisation indicators ([a] to [d]) have been used most to assess equity of utilisation and access tl1 cue (1f Jisad\Jntaged groups. {'-;e of Jggreg::ne t,;tilisati(\n data raj for e\almti"'l ,': ,-', .

assumes that utilization is most sensitive to policy change among the poorest groups. There is evidence from a few countries that this assumption is justified (Gertler and van der Gaag, 1993). The problem with aggregate utilisation as an indicator is that it may hide offsetting utilisation changes. For example, the introduction of a user charge and simultaneous improvement in quality, might increase the utilisation of those who had previously used more expensive providers and still reduce the utilisation of those who had previously been unable to afford those providers and are now unable to afford the user fee. Overall, utilisation levels could be maintained, and no problem therefore detected. A second limitation which applies to all four utilisation indicators is the possibility that a refonn in health financing may well be accompanied by other changes such as changes in accounting and recording procedures, and training of staff which could affect the quality of record keeping at a facility. Thus the utilisation indicator may reflect completeness of recording rather than actual utilisation levels. It is important to consider whether changes observed may be explaineJ in this way. Data for aggregate utilisation can be obtained from routine data collection systems. Disaggregated data [b] presents difficulties according to what type of disaggregation is attempted, There is not usually any explicit information regarding economic status which is routinely collected. Data on utilisation by women and by under 5s1under 1s, usually are available and may suggest the degree to which overall trends reflect disaggregated trends, as well as the utilisation of these priority groups. If it is possible to identify the location of patients' residence it may be possible to infer the utilisation of different economic groups from this. However. distance plays an independent role in households' health care seeking behaviour and this analysis will be more conclusive if areas of differing economic status but similar distance can be compared. Comparisons of utilisation trends in rural and urban facilities [c] enable an assessment of whether geographical inequities are widening or narrowing in response to financing reform. This indicator does not reflect the extent to which rural populations use urban facilities, however (a factor which could be influenced by a reform which increases adherence to the referral system. for example). Indicator [d] suggests an alternative to facility based data for the assessment of disaggregated utilisation trends. If deaths are registered adequately, a study can be undertaken based on the population included in the register of deaths and this indicator measured. Such a study would be much smaller than a general household health service utilisation study and would control for severity of condition and enable detailed information on household characteristics to be collected. Clearly, there is a need for great sensitivity on the part of investigators, however. The list of indicators [e] measures the extent to which policies in place to address .equity of utilisation are being implemented. These are likely to be available from facility or national records. If they are assumed to ret1ect equity of access. it has also to be assumed that th.ey are targeted appropriately. or at least more oft.en targeted appropriately than not. One rough r.est l~ f rhi) i) implii.?d by inJicHPr rtl [f r_m;<.:ring is J.pprorriat.e. rhe g<':')g::1[,hi(::' J:,r " ' 1 ' . "'

exemptions, free insurance coverage and other measures would be expected to reflect the distribution of poverty or income. Nevertheless. it is quite possible that geographical targeting is adequate whereas its application to individuals still works perversely, favouring civil servants and the more politically powerful, for example. Direct study of the characteristics of those exempted from charges or offered reductions or loans is required before indicator [g] can be measured. Examples of possible studies include exit polls at facilities or follow up of households who are recorded as having received exemption or other concession. Price comparison of public and other sectors [h], enables the concept of 'relative affordability' to be evaluated. Since disadvantaged groups are most likely to seek care where it is cheapest. the relative affordability of the public sector determines their access to that sector. Still. interpretation of the indicator relies on an assessment of the quality of public sector services in relation to competing alternatives. If the public sector aims to provide at least some minimum standard of quality to the poorest groups, its undercutting of sources which are thought not to provide at least this quality, for example unlicensed drug sellers and practitioners is an indicator of its achievement of this objective. Since the public sector is the sector in which public subsidy is concentrated, the use of this sector by the poorest groups indicates at least some share 0 f that subsidy reaching those groups. This indicator is also therefore likely to reflect equity in flnance. Although prices of services in the informal sector are not usually routinely collected, they can quite easily be established through interviews with members of the community or through informal approaches to the providers themselves. Only large scale flnancing reform at national, or at least regional level is likely to affect the distribution of providers and facilities [i]. For example, this indicator could be used to assess the success of attempts to expand insurance programmes to rural areas in achieving improved physical access to services for rural populations through increasing the demand for services there and attracting new providers. In most countries, registration records will enable the mapping of registered practitioners. Failing this, or if unregistered practitioners are also considered to be important, maps can be constructed for sampled areas through interviews with members of the community and physical search. There are also a number of indicators which specifically relate to evaluation of insurance reforms (fj] to [1]). Coverage levels of disadvantaged groups 01 give a partial picture of their access to services since it is often the case that despite coverage, poor physical access to services results in continued low utilisation or utilisation of inferior quality services. The same limitation applies to indicator [k] since across groups which are disadvantaged for these reasons, there are other barriers to access than flnancial ones. Nevertheless, flnancing reforms can be judged most directly by their effects on financial barriers to access and both indicators are sensitive to important changes which may be taking place. Since locally based health insurance schemes are usually intended to expand utilisation of services in a speciflc locality which has previously been characterised by low utilisation. and thus to improve equity in utilisation in relation to other areas, the total membership of such a scheme [I] is an indicator of its success in this respect.

Equity in finance requires that subsidies should flow from the rich to the poor and not from the poor to the rich. Subsidy flow is determined by the distribution of the financial burden imposed by health services. and the distribution of the utilisation of services. Since finances flow through a number of institutions (including the exchequer. insurance agencies and facilities themselves), determining the distribution of the financial burden involves a breakdown of the sources of tinance for each of these (indicators (aJ to (dJ). Each of these agencies then finances a number of different sectors and institutions and the distribution of the expenditures of each has to be analysed separately ([eJ to (gJ). Utilisation patterns of each facility or sector have to be analysed ((hJ), and even within institutions and sectors, cross-subsidies may take place. for example between private and general wards (iJ or between social security funds OJ resulting in inequities when funds are compared [k]. These have to be assessed through analysis of the distribution of financial burden and utilisation in facility departments. Therefore each of the listed indicators of financial equity provides only a component of the analysis needed. Central government accounts provide a certain amount of the information needed to assess the distribution of the taxation burden (aJ. They should provide information on proportionate shares of each different type of tax and for income tax, the distribution of the burden between income bands. Understanding the distribution of the burden imposed by sales and import tax requires separate study, however, which may be considered beyond the scope ofa health financing study. In some countries an analysis of the distribution of the burden of taxation may already have been undertaken. Insurance agencies' own records should enable a breakdown of sources of finance for insurance programmes (bJ. Nevertheless there are ambiguities in separating the employer and employee contribution. It is not clear to what extent voluntary and compulsory employer contributions are offset against wages, and without analysis of the labour market, and even the macro-economic effects of contributions, these ambiguities can only be noted. For individual facilities [cJ, the contribution of different sources offinance is not usually known and requires separate study. Again Hanson and Gilson (1993) offer a set of simple guidelines which could be followed. For a comprehensive assessment of the sources of finance for the whole health sector [d], a large household survey and wide ranging facility assessments are probably required. More modest studies might simply combine the measurement of indicators raj and (bJ with some sample facility assessments, noting what health sub-sectors are unlikely to have been properly accounted for in this process. The geographical distribution of both government [e] and insurance agencies' [g) expenditures may be straightforward if a decentralised budgetary system is used by which funds are first allocated to regions or districts. In more centralised systems. allocations to individual facilities will have to be collated by geographical areas. and budgetary headings such as personnel or capital equipment will have to be apportioned 3ccording to information about the distribution of these items in as much detail as practicable. How easy it is to disentangle government expenditures on different sectors (direct expenditure on the public sector and support to insurance ,!nd ;'ri\ ::rc ~'-'l':,'r' J rtllki'ends on rt'cnnJ keering syq~ms fmputi'1g th~ \alue (\ft:1'\ (,'nl"'>". ''', , 1'1 ,\

will in most cases be feasible at least approximately using tax records. The work involved in reaching greater degrees of precision than, for example. assuming all those whose insurance contributions are tax exempt have the average level of income and average insurance package for that group. mayor may not be deemed worthwhile, depending on the record keeping system. The extent of cross-subsidy within facilities [i] requires facility level studies in which the costs of services provided within departments and the direct payments received for the services offered by those departments are compared. Where only a proportion of costs are covered through direct payment in the facility as a whole, this indicator will instead pennit a sub-facility analysis of the channelling of outside subsidies. Cross-subsidy takes place between insurance funds OJ when they purchase services from the same facilities and either negotiate different prices, or predominantly utilise different services between which there is some cross-subsidy. Both facilities' and insurance agencies' accounts sho~ld identify prices paid. Facility level analysis of costs and income by department (as for [i]) would enable analysis of the second type of cross-subsidy. Indicator [k] will nonnally be capable of assessment through a comparison of the packages offered by alternative funds, perhaps combined with quality and cost assessment of the benefits offered, and further investigation of the income levels of the population covered as discussed above. Where contributions and benefits are not proportionate in comparisons of insurance funds, previous indicators may have identified disproportionate subsidy as the cause. Other causes such as differences in efficiency between funds, or excessive profit in some funds may reflect other issues than equity of finance. Where there is an association with income level of those covered, some combination of relevance to equity of finance and equity of utilization is implied. This might be an issue where a separate insurance fund is set up to cover poorer or rural groups and cannot take advantage of the economies of scale available to larger urban based funds. Identification of consequent inequity of finance or utilization might suggest a rationale for targeting of public subsidy towards such funds.

2.5 Financial sustainability Financial sustainability can be narrowly defined as the extent to which national or local budgets are funded from national resources, or more flexibly as the medium to long tenn stability of a mix of funding sources. Health financing refonns may have specific objectives relating to fmancial sustainability. For example, it might be intended that particular facilities become selffinancing in which case cost recovery ratios [a] are an adequate measure of success in achieving the objective. In comparing cost recovery ratios, it is important to ensure comparable numerators and denominators and in identifying self-financing as an objective of policy, to identify the intended definition of 'self-financing' - whether, for example, it is intended th:::.t salary costs or capital depreciation should be included in the denominator. If donor and government finance is intended to play a longer tenn role. documenting how significant these roles are and the trends in them [b) may suggest whether or not sustainability is likely to be an iSSue. \Vhere a substantial role for donors is measured. an assessment of the I,'ng tcml clr:h.:il: Jild \\ illil1gl1c" "f the ,i"m'r to cClntinue to contrihut\? tl' the he,ll:.h ~C(· ':. '" , ,

, I

the country concerned may be needed. \\tbere donor support takes the fonn of loans - wholly or partly - the contribution to the country's overall debt burden should be considered. At national level, the contributions of donors should largely be documented. although some donor finance. for example church contributions to mission health facilities may not be channelled through the Central Government and may be more difficult to ascertain. This infonnation can be supplemented with infonnation gleaned from facility based studies. The longer tenn capacity of nationally based resources to maintain their role in financing the health sector might be assessed using indicator [c] alongside more qualitative approaches. Disproportionate gro\\-th in health sector expenditure relative to GNP andlor the incomes of paying groups suggests an impending problem of cost containment which poses a severe threat to financial sustainability. If no health sector price index has been calculated a consideration of the growth rate of the salary bill and the pharmaceutical import bill (unless the country concerned is a major manufacturer of pharmaceuticals) should give an indication of the relationship bern:een health sector :':1d general inflation.

2.6 Institutional sustainability Institutional sustainability requires that the capacity of institutions is sufficient to manage the process of change and administer new systems which are to be put in place. There are no single dimensional indicators of what can only be a qualitative assessment. The indicators listed indicate the need for assessment of two aspects: the adequacy of organizational and administrative structures [a] and the adequacy of human resource formation [b]. Assessment of adequacy involves comparison of the roles expected of an organisation or individual, the structural and skills requirements of each role and an assessment of whether these exist or not. For example, if a district health office has been charged with supervising the accounting of collected revenues, institutional sustainability requires that the structure of the office is amended to allow for that (which may only mean the amendment of ajob description or may require the creation of a sub-unit charged with that responsibility), and that individuals who are given that role have the skills and training needed. If gaps between roles expected and skills and training are identified, assessment of institutional sustainability requires assessment of the capacity of training institutions to fill the gap. 2. 7 Administrative costs Methods for measuring administrative costs are the same as those for measuring any other kind of cost and existing methodologies can be used. The indicators suggested express these as proportions of the revenues raised or expenditures made by the financing institution. 3. Analytical methods

The previous section discussed the extent to which each of the indicators identified accurately retlected the intended objective of policy. and the sources ofinfonnation that would be required to measure each indicator. This section anempts to review the methods which can be used to \..'\:llu::lte tht: Jc~re~ \1( clss(1ciJtion ret\\een the indicators and the policy chan:;t:

Some of the indicators simply describe features of financing mechanisms and their implementation. If policy refonn has involved the introduction of the mechanism where it was previously not used, there is no question that these indicators reflect something associated with the policy refonn. For example. if user charges have been introduced, the percentage exempted cannot but be associated with the policy change. This may seem a frivolous point, yet such indicators already enable some analysis of the effects of policy by describing salient features of the policy and its implementation. Some evaluative reports of health financing refonn go no further than to describe such features but still are able to identify many of the policy's strengths and weaknesses and even suggest measures to improve on the policy's perfonnance. Yang (1991) is able to identify problems of cost inflation, inequity, and inefficiency 0 f administration mainly from descriptive indicators (although some other approaches are also employed) in an evaluation of the national health insurance system in Korea. Table 1 shows the indicators used to assess each of these issues.

Table 1: Use of descriptive indicators to evaluate a national health insurance system Problem Descriptive indicator

Cost inflation Inequity

Re-imbursement mechanism (retrospective cost reimbursement on cost-plus basis) High co-insurance rate 'Two-class' health care system Identification of inequitable risk pooling Unaccountable management duplicated in each society Proportion of administrative costs to total revenue

Inefficiency of administration Source: Yang (1991)

Moens (1990) also used a number of descriptive indicators to assess equity of access and financial sustainability in an evaluation of a local prepaid health plan system a Zai'rian health zone (Table 2).

Table 2: Use of descriptive indicators to evaluate a local prepaid health plan Problem Descriptive indicator

Equity of access Financial sustainability Source: Moens. 1990

Membership rates and distribution % cost recovery

Beyond use of descriptive indicators. there are only two aporoaches to associat-: chanc;'C.?-.; indicaw['s \\ ith ..:hanges in pl)lic: • ciw,~-,;,-,..:ti\)11JI ..li1d longitudinal. 1 -;

III

Cross-sectional approaches rely on there being the opportunity to compare different observations (for example health facilities, areas. individuals) among which there is a difference in policy. at the same time. For example, health facilities in which user fees have been introduced can be compared to health facilities in which they have not. Analysis attempts to assess the extent to which differences in the indicators between the two groups of health facilities can be ascribed to the policy. Longitudinal approaches compare the same observations over a period in which policy changes. For example, the same health facilities might be compared before and after the introduction of a user fee. Analysis attempts to assess t..::! extent to which changes in indicators bet\veen the preand post-policy introduction periods, can be ascribed to the policy. Clearly, there are many variations within each of these approaches. The extent of implementation of policy can be a continuous rather than discrete binary variable. For example. instead of crosssectional or longitudinal comparisons of the absence or presence of fees. a range of observations with different levels of fee might be used. In some cases a combination of longitudinal and crosssectional approaches are used. trends being compared between observations where the extent of policy implementation differs. Cross-sectional comparisons can result from controlled experiments in which a policy is introduced on a selective basis deliberately so that its effects can be measured, or they can result from 'natural experiments' in which there is an external reason for partial application of a policy. Longitudinal comparisons can be prospective (when it is possible to start measuring indicators before implementation of a policy), or retrospective (when routinely collected data relating to past experience are an~lysed after policy implementation). Difficulties in conclusively associating policy change with change in indicators are mainly common to both approaches and result in the use of sophisticated analytical approaches. The underlying problem is that most available indicators are not uniquely affected by the policy under consideration and as a result, analysis has to be multi-variate: it must consider the full range of variables which might affect the indicator of interest and ensure that the policy variable can be isolated as the causative factor. For example, if utilisation levels are found to be lower in facilities with user fees than without. it must be established that the explanation is not really a difference in the size or dispersement of the catchment population, in disease profile. or in other factors such as perceived quality of services which affect the popularity of the facilities and which also differ between the two groups. It is not the purpose of this paper to describe the range of techniques available to address this problem, or the limitations and potential modifications which have been developed. Instead, a few simple and easily understood methods ""ill be outlined and some examples of their use described. Nevertheless, in some cases these methods will not be capable of ensuring conclusive attribution of effects to policy and further progress can only be made by employing a statistical expert. If a policy change is discrete (for example a new programme is introduced on a specific date). a very simple method of attempting to relate longitudinal data to a policy change is to identify the date of introduction and look for sudden discontinuities or reversals in trends which are then highly likely to be explained by the new programme. An example of this approach is pro\ided by \foens (1990) in the study already discussed (Table 3).

, ,

....

Table 3: Patient revenue and operating cost - trend over the period in which a prepaid health plan was introduced (it was introduced in 1986) Year Patient revenue Operating cost

1984 1985 1986 1987 1988 Source: ~oens(1990)

668,449 878,583 1,918,905 3,848,136 8,034,130

1.853.629 2.035,i35

3.141.105 4,674.026 9,909.054

Although the trends suggest revenue and operating costs were already increasing before the introduction of the plan, a doubling of revenue each year since introduction compares to an increase of only 30% the previous year. This is strong evidence of an association. Nevertheless, there is still a need to ask whether or not other changes took place at the ~ame time as the policy change which were also not discrete. For example, the change in question might be part of a package ofrefonns introduced concurrently, or might be associated with a change in government which might explain sudden changes in other trends. A less satisfactory approach to evaluation of the association of a trend with a policy change is exemplified by a study by Yoder (1989) who assessed the utilisation impact of the substantial increase in government user fees in Swaziland (Table 4). Yoder compared the pre- and postchange data without reference to general trends over a longer period. The extent of reduction in utilisation suggests a serious cause and would be unsustainable as a long term trend. Nevertheless, the argument would have been considerably strengthened by more historical information.

Table 4: Monthly average attendance before and after government fees increased to mission fee levels Pre-change attendance Sector (10/83-12/83)

Post-change attendance (10/84-12/84)

% Change

Government ~ission

817 783 805

552 862 665

-32..+ 10,1 -17.4

Totals Source: Yoder ( 1989) ",.... 1...:' .1\ 1rnnri1.r..:!1 t'l J lo

It is possible to improve on both these approaches by attempting t.o identify whether or nelt there ~lrc.:

ut 1 1~r po

1\':y

l)r

1...14 •.

,1 ... , " " ' "

,1"", ,I ~ L.=\.:s .' I,' '11\....1 I...l')L..\....

,-.,t-'

"'\-'~I";" \.-.11.

.t., ...... - ;."1 u:-, lit .... \.. • •

'....H

~

.. ,~ ....

"I" .... \

d'

~!,-

,I'

"'.'

,

..... --.

'-,,"'"

. . . -'"

" , :: , ,

financing policy in question. If reasonable hypotheses are tirst generated and then excluded. the case for linking policy and indicator change is strengthened. For example. Yang's (1991) review of the Korean health insurance system discussed above. reviews a range of possible factors explaining health spending increases (such as general price inflation in Korea and failure to implement adequate controls over technology adoption), before attributing a share of the irulation to the expansion of the insurance system and some of its specific features. Cross-sectional studies resulting from controlled experiments have the advantage that assignment of cases to intervention and control groups can be done randomly, or by a method structured to ensure that differences in results can be explained by the intervention rather than by other predictable factors. It can therefore more safely be assumed that this is the case. Litvack and Bodart (1993) selected five facilities, three 'treatment' and two 'control' to evaluate the impact of a user fee accompanied by quality improvement interventions in one province of Cameroon. Comparability with the treatment facilities was the principal criterion for selection of the control facilities. This study also had a longitudinal component (baseline information was collected) and did use complex multi-variate techniques to support the conclusions. Nevertheless, this analysis was able to confirm the validity of a simple comparison of pre- and postintervention differences in utilisation between treatment and control facilities (Table 5)

Table 5: Percentage of sick people using a health centre before after user fee introduction accompanied by quality improvement Baseline (%) Control Treatment Follow-up (%)

45 44

38 48

It is much more difficult to reach firm conclusion on the basis of cross-sectional studies where assignment of observations to intervention and non-intervention groups has not been controlled but rather has been part of the outcome of the policy development process. In these circumstances there will often be a host of factors which differ between the intervention and nonintervention situations and which may have influenced the policy process and thus explain the adoption and non-adoption of the policy. Interpretation in these situations therefore has to be extremely cautious. Two studies related to the British Nepal Medical Trust revolving drug fund ~rojects in Nepal exemplify this type of cross-sectional study (Chalker, 1992; Fryatt et al., 1993). Chalker ( 1992) compared a revolving drug fund district with a district in which there was no fund, with respect to prescribing practices. Health posts in the drug fund district had a greater tendency to\vards polypharmacy than in the non-drug fund district. In this study, it was possible to select any district for comparison purposes since the revolving drug fund only operated in one district. Comparability was one factor determining the choice of'control' district. In addition. age. sex dnd morbidity were explicitly compared between the two districts. Fryan ~t al. ( I c)93) compared tJcilitil?" u,ing t~e-rer-item \\iri1 ri1n'': u;ing ~~e-rer-rre~crirrion chargin~ r3:,c-; ~lnd ""11:,: ",'

'\

the average drug cost per prescription was only 50% in the fee-per-item facilities. The fee-peritem facilities were selected to pilot the idea of a fee-per-item charging system and comparability with other facilities was not a criterion for selection. The author concluded that it was difficult to know to what degree the communities involved in the facilities compared were comparable. Roemer and Maeda (1976) attempted to answer the question of whether or not there was an inverse relationship between expenditures on the insurance sector and expenditures on the public health sector when the unit of observation was different Latin American countries. Without using multi-variate approaches they conducted a series of analyses of correlation between pairs of variables: GDP and public health expenditure; GDP and health inswance expenditure: and public health expenditure and health insurance expenditure. They did not find significant negative correlation between social security expenditures and health insurance expenditures. However. this is a case where a more sophisticated multi-variate approach probably should not have been avoided. The positive correlation between both GDP and health insurance expenditure, and GDP and public health expenditure ensured that any underlying negative relationship between health insurance expenditure and public health expenditure would not be revealed by simple correlation analysis. A number of conclusions can be summarised from the above. First, there are only three approaches to assessing the degree of association between indicators of financing policy objectives and financing policy change. The first uses descriptive indicators which are inherently associated with the policy change such as cost recovery ratios. The second uses a longitudinal approach in which the timing of the policy change is compared with the trend in the indicator concerned. The third uses a cross-sectional approach in which observations across which policy implementation varies are compared. Second, both longitudinal and cross-sectional approaches encounter the problem that there may be a need to control for the influence of external factors. Simple methods can be used to identify whether or not the problem is likely to exist in any individual situation. These include. for longitudinal approaches, looking for discontinuities in trends where a policy change is quite discrete; looking for changes which are sufficiently large to exclude the likelihood of a long term trend; and generating hypotheses about other explanations of a trend which it is then attempted to reject. For cross-sectional approaches, experimental studies should minimise the expected influence of confounding variables in the design stage. Uncontrolled studies may still have some scope to do this if there is a wide choice of observations to select from, at least on one of the 'intervention' and 'control' sides of the comparison. Failing this, hypotheses regarding alternative explanations of differences between groups can be generated and their viability tested. as with longitudinal approaches. If the result of the use of these methods is that confounding factors cannot be ruled out as the explanation of a change in the indicator. only more sophisticated multi-variate techniques can result in greater certainty about the impact of the policy change.

,..., ,

References Chalker. 1. (1992) Does regular drug supply and a Jixed prescription fee mean beuer drug use) A. study of prescriptions from health posts in the hills of Nepal. M.Sc Community Health in

Developing Countries, London School of Hygiene and Tropical Medicine Fryan, R.J., Rai, P., Crowley, S.P .. and Gurung, Y.B. (1993) Community Financing of Health Care in rural Nepal. Field evaluation of a 'fee-per-item' drug scheme. British ~epal Medical Trust, Tonbridge, Kent Gertler, P. and van der Gaag, 1. (1990) The Willjngness to Pay for Medical Care: Evidence from two developing countries, lohns Hopkins University Press, MD Hanson and Gilson (1993) Cost. Resource Use and Financing i\1ethodology for Basic Health Services: A practical manual. Bamako Initiative Technical Report Series Number 16, lINICEF. New York. INRUD (1991) Indicators ofrational drug use: draft procedures manual, International Network for the Rational Use of Drugs, Boston Jamieson, D.T., Mosley, W.H., Measham, A.R., and Bobadilla., J.L. (1993), Disease control priorities in developing countries, Oxford Medical Publications (published for the World Bank). Oxford University Press, New York. Litvack, 1.1. and Bodart, C. (1993) User fees plus quality equals improved acces to health care: results ofa field experiment in Cameroon, Social Science and Medicine, 37, 3, 369-83 Matsaganis, M. (1991) Is health insurance in Greece in need of refonn?, Health Policy and Planning, 6, 3, 271-81 Moens, F. (1990) Design, implementation and evaluation ofa community financing scheme for hospital care in developing countries: a pre-paid health plan in the Bwamanda health zone. Zaire. Social Science and Medicine, 30, 12, 1319-27 Roemer, M.l. and Maeda, N. (1976) Does social security support for medical care weaken public health programs, International Journal of Health Services, 6, 1, 69-78 World Bank (1994) Better Health in Africa, World Bank, Washington D.C. Yang, B.M. (1991) Health Insurance in Korea: opportunities and challenges, Health Polic.v and Planning 6, 2, 119-29 Yoder. R.A. (1989) Are people willing and able to pay for health services? Social Science and Jledicine. 29. I. 35-42

Annex 9

Franl)o i s DIOP

Exhibit I

Profile of Niger Indicator Population (in millions) Urban Population (%) GNP\capita (US $) GNP\capita annual growth (%) "literacy rate - Woment (%)

Circa '91

7.9 15

300 -4.1

83 -

Sources: MSP, 1994; WDR, 1993 HFS ------"-

..

Exhibit 2 ---:-----

Limited Health Resources = > Poor Health Status I

-

- - - - -.--.

r

Indicator Government Expenditures on Healtb\capita (US $)

Circa '91 3

I II

J (,

(, (

I

Population\physician Population\nurse %Population within 5 km of PUP Under-five mortality rate (per HFS ](00)

, 34,900

~

~ I, t

1

I

700 32

t: .

. oj

... (

II I·

II

318 -- ------_._---------------------

Sources: MSP, 1994, 1993; WDR, 1993

..

Exhibit J

Policy Objectives Promote a Sustainable District Health System • • • Improve Quality .

Increase Utilization\Access Sustainability

IIFS

-

Exhibil !f

Limited Experience With Cost Recovery • Weak Implementation of User Fees in I-Iospitals

•

Prinlary Health Services Free of Charge

•

Cost Recovery Experience in PHC

• • III~S

Tibiri (User Fees) Mirriah (Additional ~Iead

Tax)

Exhibit 5" -----"-~--------- --~~

- - ~----~----

--

---------

Policy Debate and Decision • Alternative Health Finance Mechanislns • • • Private Financing (user fees) Prepayment (health cards) Social Financing (local head tax)

•

National Workshop on Cost Recovery • Decision on Pilot Tests ()f

Cost Recovery .

I I Ii'S

--~---~-----

Exhibit

6

Policy Objectives Promote a Local Health Finance System which is • • • Equitable Efficient Viable ._-----------------~-

-----------

--------------

IIFS

Exhihil 7

Policy Objective: Equity of Access Promote Equity of Access • • Improve Access for General Population Improve Access for: • • • Women and Children The Poorest Population Living Away from PHF

IIFS

Exhihil 1?

Policy Objective: Efficiency Improve Quality at Minimum Costs • Improve Quality of Health Services

•

Minimize Costs of Health Services

•

Service Delivery

•

11ousehold Expenses

IIFS

Exhibit 'I

Policy Objective: Viability Promote a Viable Local Health Finance System

-- -

-------

--

----

•

Willingness and Ability to Pay

•

Cost Recovery

• --

Resource Management

-

---

IIFS

Exhibit 10 .,----~----

.. ----

Pilot Tests of Cost Recovery in the Non-Hospital Sector A Framework for Policy Development • Consensus on Policy Reforms

•

Identification of Constraints to Policy Reforms

•

Strengthening Implementation Capacity

•

Evidence of Consequences of Course of Actions

IIFS

Exhibit "

Building Consensus on Policy Reforms Institutional Aspects • MOH and Government • • • National Steering Committee of CR Directorate of Studies and Programming

Larger Social Fora .Sectoral Workshop == > Technical and Institutional Issues • National Workshop ==

> Policy Issues

IIFS

Exhibit Ic2

Strengthening Implementation Capacity Institutional Aspects • Decision-making • • National Steering COlllmittee of CR

Execution • Directorate of Studies and Programming • Institutional Unit = Implementation

> Coordination and

•

Technical Support: Other MOH Directorates

I-IFS

Exhibit 13

Evidence of Consequences of Course of Actions Technical Aspects of Pilot Test Design Component Intervention Quality Improvements

Ilicia Control No No Yes Yes Yes --.----~---

Say Fee Yes

Boboye Tax+fee Yes Tax + fee Yes Yes Yes

Financing Alternatives Data Baseline Survey Final Survey Facility Data ~~---~

FEE Yes Yes Yes

-- . - --------------- --

J-IFS

Exhibit 1'1

Quality Improvements • Availability of Drugs

•

Standardized Diagnosis and Treatment Protocols

, , I

t

..

•

Financial and Drug Stock Management

HFS

..

Exhibit

I~

Financing Alternatives Two methods of cost recovery tested against the status quo Level

Say Control Fee Dlela No Charges .

Boboye Tax + fee No Charges

J

I

I

Population

200 FCFA\Adult ($.67) 50 FCFA\Adult ($.17) 25 FCFA\Child ($.08)

Users of Public Health Facilities --.-~--

No Charges

200 FCFA\Adult ($.67) 100 FCFA\Child ($.33)

--

HFS

•

Exhibit "

;

1·

Selected Pilot Test Findings 1. Utilization of Curative Services

,,, ,,, <I' ~.,

C>l ~.

~ ].

r .•

hI

-I

m . ~

2. Utilization of Preventive Services

~

,

It: .. (., (J. ('II t·

r:

til

~

O.

3. Financial Sustainability

IIFS I •

..

Exhibit 17. Despite the fees introduced, utilization increased when quaJity of care improved.

Changes in Utilization, Consultations General Population 80

Ii ::,

160 . --- -----.:50- 1 - -- - - --~

j

70 . - ---------,j,

i40 ~

--

-- - --------------~---------

- 1 - - - - - - - - - ------

• j20 • o

o

-1:30 -. - ----------0----

It 10 -. - - ---------

I1l6la

(Control) ~

Say (Fee)

Boboye

ffax+Fee)

~

I liS

-

Exhibit Ig. Utilization by households in more remote areas increased, particularly in the tax -plus-fee district.

Changes in Utilization Among Remote Villagers Population Beyond 2 HoulS of Walking Time 10 PHF 50 --1---- - --- ---------- --__ _ r

-2 :::, S

J is 30-----------

-

c: 40

-.. _ ... ..

I II

_-

- - - - - ---------

(, (

l,

(

~ ----------------

~20 "--

~ I,

't ---_._-. - ---

~ ~ ct

& 100

. ~. II (.

-

------------I

. (.

r·

OJ

II

1-10 -20 J

i.I:1

-_.-

(- .. -----.

--

-

III8Ia (Control)

Say (I;"ee)

Boboye (Tax+Fee)

~

~

II~

-

Exhibit 11. Utilization by children increased when 'the user fee was low.

Changes in Utilization Among Children 40 ,------- - ---------~--------

Children Under ] 5 Years of Age ------------------------

~

J

.§ 30,------- --- -----------f------

-

-

f (j

~

oS 20

.

10-.· ------

I

~ .!!

.

0

,--I

-

------Say (Fcc)

-10

-0

---

- - - -

-

--- ---- -- --- - - I -------- ---------- -- -.- ------

l/lela (Control) ~

BOboya (Tax + Foc)

~ J-FS

-

Exhibit JO. Utilization by women increased relative to the control are~ particularly where the fee was low.

D

,

Changes in Utilization Among Women Women 40 -- -- ---------~. ---~-

lu

'\1

01

<II

N .~

c'., ---.~-~

._-_._--_.

-----

1

o

J

m

I)

:)

is

~

~ 20-J

-i

lh ~

R ~ .~

~ ........

.s 0-

tJi UI

I

i-20 tS • i-40!! I -60 I

&

~

til

(I)

---

--~-

.. -

kl ~ If·

-

--~

-

----

----------

-------

-------

-

I

-- - 1

II18Ia (Control) - 4.--::tt..." _

Say (Fee)

Boboye (Tax + Fee) II

~

II~

..

Exhibit

J...

Utilization of prenatal care by the poorest 50% of households increased relative to the control district.

Preventive SelVices: Utilization of Prenatal Care Poorest 50% 10 ~.-~--

---

"--~

I

! t

j

II

0-

l: ~

I

oS

~10 -.----

~

,. ---------------------

~

t •

I,

I 1It

~

20 -

"

:i •

K

-30 --

I

, Boboye (Tax + Fee)

IIlela (Control)

Say (Fee)

~

-~

I~FS ~

Exhibit JJ. Utilization by the poor increased when tle user fee was low. .;.

Changes in Utilization Among the Poorest 12. _.

h) 1<.. ,[)

""'

(II

Poorest 25% 0-

... ~ ru J)

111D

to --i

J C::

i", &

---- --..

~ fi{

-60-------oS .-... ~ 4D - 1---

~ ::l

~ G:

2D 1

U'

~

r: ~ ...

Pi tll

I

I .!I "

• -211

. I

-4D . -- --- -

--- -r Say (Fee)

- --- I

IIlela

Boboye

(Control)

rrax+Fee) (,

~

~

IFS

..

II,

Exhibit ::13. Utilination of prenatal care increased, especially in the tax-plus-Iow-fee district.

.

Preventive Services: Utilization of Prenatal Care General Population 30

/-

/.

(I

"

r_

q ~J

r.

a5

,20 15

- 25

--J --

01

-,

I-

-r-------------------------------- ---- - - - - - - - _ o-

~ iJi lH ~ ~

I

I

f 5 -. ts -5

~ 10

S .......

~

-f-

--. -- --------

.kl

0 - .---1-----

m ~

0-0

-10 - -

- -- ---r---------

--- ----

--

-- I

I

IIlela (Control)

Say (I'~)

Boboye (rax + Fcc) "

~

IFS

..

Exhihit )5. Illness-related out-or-pocket spending declined in the pilot districls.

Sustainability Financial Benefits to Population: Changes in Spending per Episode 10

r __

u~_~

_______]

j 'tj

0

110 .!If

~

:'20

i s ~30 &

Including FeEt and Tax

!40 ~

-50

I

- - -T8X-----I r

Without

lIIala

Say

Boboye

(Control)

(Fee)

(Tax+Fee)

~

--~

I I ~~'-'

Exhihit :l6. Training and experience with diagnostic and lrealnlenl prolocols can lower drll~ costs.

Sustainability Containing Costs Through Better Compliance with Protocols: Drug Costs per New Patients 300 1250 ~ ~OO ll:: ~

.i

Q

~oo ~

~50 i

~

!

tso o Say (Fcc) Boboye (Tax

+ Fce)

~

---~

, I~~'-)

Exhibit:17. Labor productivity inlproved in the pilot test districts. I)espitc costly illlprovcnlents in quality, the unit cost of services fell in the tax-plus-fee district

Sustainability Improving Efficiency Through Cost Recovery and Quality Change: Changes in Costs per Visits 60

.-.... ~ 40· 'CD

~20

c3 ~

ttl

0

CD a:: -40

.!!

~-20

-60

I

. I

r-: ~ .~ ~

Say ~-~~,-

Boboye ----~

----

--

--.

-- -------- ----

-~--------~---

Legend Average Operating CoslsNisl1 (FCFA)

~:~

Average Labor CoslsNlsil (FCFA)

111~~

Exhibit 21l. A high percentage of operating costs were recovered in the tax-plux-fcc district.

Sustainability Cost Recovery Performance: Percentage of Costs Recovered 160

~ IX:

140

149 89

~120

!100 IX: ~ 80

160 20

52 .:::.••••••••••: ••••••~ Say (Fee) Legend

35 12 Boboye (Tax + Fee) ~

40

o • T olal Drugs

•

Drugs and AdmlnislrallOn Cos Is

Tolal Cosls

~-..;;

-~

~

I ,~~."

Exhibit

J,

Evidence of Consequences of Course of Actions And Policy Process • • Field Visits of MOH Officials: Evidence of Positive Changes Multiple -Reports on Performance and Dissemination: • • • National Steering Committee of CR Sectoral and National Workshops

Identification of Constraints to Policy Reforms

• HFS

Decision to Develop Plans for Nationwide Extension ---- --------

---------

Exhibit

30

Evidence of Consequences of Course of Actions And Policy Outcomes • Direct Policy Outcomes • • • National Policy on Financing PHC Adopted Institutional and Implementation Capacity Strengthened

Indirect Policy Outcomes • • Improvement in Training Policy Improvement in Drug Policy .

__._- -_.--- -

------~---.--

IIFS

I ' "-,

Exhibit 31

~----

C~nclusions on Policy CO! Reco1ery, Equity, and Sustainability I. I

Qual" improvaaats under (Xlst reoovery improve accessibility in poor rural ~ttings ..dean save people money

(J

f ~

2. 3.

Poor lral populatoos are willing to pay for quality improvements SomC:lnancing nethods may perfonn beUer than others: in this case, Tax-Jus-Fee per",nned better than Fee-for-Service relative to: I

i t

b

~

~

J} Promoti~ utilization of health services

II

. ~

~

E Generat1g revenues and recovering costs

(' Populatib stated preferences

~ IFS

~

..

Exhibit ~~ " I·

I

Conclusions on Policy Process Health Financing Policy Reforms Are Facilitated by: 1. Evidence that the policy reform will yield the desired result 2. Evidence that the policy reform is technically and politically feasable 3. Involvement of government leaders and commitment of key MOH decision-makers and technicians - ------ -

..

t-

f.

II

I

~

1 If 1I

~I I

(I-

~ T·

I'

..

~

~ .tI.

~!

Il·

--

HFS

...

Annex 10

Technical Notes A ublication of USAID's 1m No.2 Meren 1992

Stakeholder Analysis: A Vital Tool for Strategic Managers By Benjamin L. Crosby The recognition of the key role played by stalccholders in the determination of policy, i~ implementation. and outcomes has made stakeholder analysis a vital 1001 for sttategic managers. The term stakeholder analysis encompasses a range of different methodologies for analyzing stakeholder interes~ and is not a single tool-a fact that has led to some confusion about what it is and how one does it. The purpose of this note is to help clarify the notion of stakeholder analysis by exploring a series of alternative methods. their advantages and potential limitations. A common analytic methodology for use by the IPC team and developing countty managers in reconnaissance and other diagnostic activities will also be suggested. organi.zation~ then it should be taken into accounL For instance. the urban industtial impon substitution sector in many developing countties is opposed to reforms to facilitate a more expon-driven economy. Since this sector is often the most economically powerful sector, it is generally in a position to weaken political authority should it actively oppose the govemmenL

What is it and Why do it? The purpose of stakeholder analysis is to indicate whose interests should be taken into account when making a decision. At the same time, the analysiS ought to indicate why those interes~ should be taken into account. How do we know when a group's or actor's interest JIWS1 be given specific and serious consideration? First. if an actor or group is in a pOSition to damage or weaken the authority or political support of the decision maker or the

Second. if the group's presence and/or support provides a net benefit or sttengthens an organization and/or enhances the decision-maker's authority (and capacity to secure compliance to decisions). then it should be given close consideration. For example. if a group can bring new resources, provide entty into a new market or otherwise enhance the organ ization 's sttength. itshould be taken into accounL Third. if a group is capable of influencing the direction or mix of an organization's activities, it needs to be counted as a stakeholder. Consumers are often viewed as stakeholders in organizations charged with the delivery of public services. But since public utility organizations in LDCs are frequently monopolistic. and since most consumers are poor and have little. if any, capacity to mobilize, the decision maker can safely exclude them from the ' decisional calculus. But Wt other cases even amorphous groups can be 'powerful stakeholders, panicularly if they are large; the influence of the comparatively affluent American teenager on the music and fashion industries of the united States is a case in point.

Benjamin L. Crosby is a Director of MSI; he holds a Ph.D. from Washington University in St. Lou.is. Missouri and manages the Implementing Policy Change prOJect.

Generally, stakeholder analysis focusses on ('.110 key elements: groups or actors are analyzed 10 terms of: a) the interest they take in a particular issue and. b) the quantity and (ypes of resources they can mobilize (0 affect outcomes regarding that issue. However, the way in which and the degree to which each of these elements is analyzed varies considerably. Overly inclusive approaches run the risk of turning the analysis tedious without a great deal of added value. As a rule of thumb. one might apply the following: only those groups or actors with real and mobilizable resources that can be applied for or against the organization and its interests to the issue at hand should be included. They are the ones that have the capacity to directly influence policy outcomes.

There are a couple of characteristics of this approach and in the matrix that might limit its usefulness for cenain analysts. First. while exchanges or resources are noted. the degree to which such resources are in fact salient to issue outcomes is not easily perceived in the matrix itself. Second. neither the maai.x nor the narrative analysis indicate the degree to which the group has the capacity to mobilize the resource or exchange noted. Nevertheless. the approach does quicldy communicate who has what-imponam elements for strategy developmenL Honadle and Cooper (1989) take a slightly different and more limited approach to stakeholder analYSIS than Brinkerhoff. Their matrix arrays the primary actors or stakeholders across the horizontal axis. and on the venicallists a series of problems upon which those stakeholders might have some impact or capacity to help resolve the issue. The matrix, however, is not clear about how stakeholders can actually help in resolving the problem indicated. merely that they might be able to. Perhaps more importantly. the matrix does not really indicate the level of interest of the stakeholder in the problem nor the direction of that interesL Is the stakeholder for the policy or against it? How strongly does the actor feel about the issue? However. as a "flrst cut" mechanism for illusttating the array and range of problems and actors. Honadle and Cooper's approach is quite useful. Other interpretations of stakeholder analysis go much funher than the two approaches just described. Gamman's approach (1991) is much more descriptive and analytically is quite comprehensive. However. his approach is keyed into aiding the analyst in strategy design-and is therefore necessarily more complete in his analysis. Unlike the earlier approaches, it not only lists the important actors but also attempts to gauge their relative importance, their interests and/or objectives, how these interests are in conflict with others, and the leade~ of each group. While the main strength of Gamman's approach lies in its comprehensiveness. it is also the source of some potential problems--especially with respect to how many and which groups are or ought to be

Approaches to Stakeholder Analysis: Cenain schemes are quite limited in what they expect to acheive with stakeholder analysis, while othe~ are considerably richer both in data and analytic requirements. By applying the criteria noted in the paragraphs above regarding which groups ought to be included in a stakeholder analysis this note will explore some of the dimensions of these schemes. ranging from one of the more simple forms of stakeholder analysis to much more complex frameworks. Brinkerhoff's (1991) approach to stakeholder analysis focuses upon use of the tool for managing programs. This focus highlights identifying what a program needs from its stakeholders to be effectively implemented. These needs are framed in terms of types of exchanges between the program and its key stakeholders; e.g .• flnancing, physical inpulS, political support. approvals. policy support. technical assistance. and so on. Stakeholders are identified and classified according to the resowees they control, their interests in the program's activities and outputs, and their imponance to the different typt.-os of exchanges. Brinkerhoff then summarizes the analysis in a maxtrix in which actors concerned about a panicular issue are arrayed along a vertical axis, while the horizontal axis illusttates certain types of exchanges (or resources) the actor can bring to the issue. I

i

[PC T,cI.IfiClJi. {Vola is a publication

1

0( the UDiUd Stir. AIIGCY for Int.. nauonal De~elop_nt's ImplemenCiaa Policy Chance ProJect, wUKb is managed by the Agency's Bureau 01 R.eardl aad DeYefopment, orne. of Economic: " Institutioaai Dt ..JopmenL

The project's contract team consists of Mallatemmt S1ltelll5 Internalional (prime eontralSor); the International De .. eiopmeftl Man.~ment Cesltef' at the Vni""il, 0( Marylud; Abt ....odales lac.; and J)e"eloplIMnt Altenaali_. The IPC Project ome. is loc:ac.d at MSl, 600 Wa~r Street, S'\. Wasbinctan, D.C. 20024. Telephone: (20%) 4I4-n10; Fa:: (20%) _~1S4

Morcr.lm WPOATA\REPOms\ 1611-014\014-005 cr.p

Included in the discussion. To be complete, Gamman suggests that the analyst look beyond the range of obvious actors or groups and detennine which unmobilized and/or unorganized groups might be affected in some manner by the policy (regardless of whether they are in fact affected by the policy), and how they might feel about that policy. When the policy focus IS fairly broad, then the number of groups that could be Included under that critena mounts rapidly, and can quickly tum the analysis into a somewhat burdensome eltercise (at least from a busy manager's point of view.) Another potential difficulty with Gamman's approach is that he does not examine the nature of stakeholder resources nor their capacity to mobilize those resources. Without some clarity regarding resource levels and capacity, judging stakeholder impact on policy issues will be difficult The level of effon required to carry out the sorts of activities suggested by Gamman is substantial. Thus. before proceeding. managers should weigh the potential gains from the analysis. Also. Gamman's approach requires a degree of sophistication and familiarity with the environment that a shon-term consultant seldom has. If this type of analysis is desired. then. effon should be made to obtain assistance from local knowledgeables or informants. These caveats notwithstanding. Gamman's approach can provide a wealth and richness of information to aid both in the policy design and implementation process. A fourth approach is that utilized by Lindenberg and Crosby (1981) in conjunction with their political mapping techniques. and is that which has been suggested for use by the !PC projecL This approach develops a matrix in which infonnatioD for each group is arrayed according to the group's interesrs. the level of resources it possesses, irs capacity for mobilization of resources. and the group's position on the issue in question.

rn the first cell (Table 1) are listed those interests that will be affected by the policy or decision to be taken. What are the group's specific interests in the policy? The analyst should be careful to select only those two or three interests and/or expectations that are most imponanL In the second ceil are noted those resources that the group possesses that could be brought to bear in the decision making or implementation of the policy. Can the group offer some special knowledge or infonnation? Would the group's staDlS and presence on one side of the issue be key to its implementation or blockage? If the group appears to have resources that can be brought to bear, it is important to know whether the group is capable of mobilizing those resources quickly or only slowly. Quickly mobilizable resources are advantageous if the issue has immediacy, but less so if the impact of the issue is funher out into the future. If the group cannot mobilize or make effective use of its resources, then they are not really resources in any meaningful sense of the word. The analyst's judgement regarding mobilization capacity should be noted. Finally, the group's position regarding the issue should be examined and noted.. Judgement should be more discrete than a simple for or againsL If a group is barely in favor of an issue. a convinCing argument could be enough to change its position. In some respects this approach is similar to Gamman's, but with the difference that the analyst need not go beyond the range of obvious actors. Only those actors with a position on the issue and resources that can be brought to bear need be considered. Nevertheless. the level of effon. analysis and inclusiveness of this approach is much broader (and time consuming) than recommended by either Brinkerhoff or Honadle and Cooper. As with the Gamman approach. the analyst should carefully weigh the benefirs to be achieved from the analysis against the cosrs of carrying it out

TABLE ONE GROUP's INTEREST IN

GROUP I

ISSUE

RESOURCES

RESOURCE MOBn.IZ. CAPACITY

POSITION ON ISSUE

I I Morcn 1992 WPDATA\ REPORTS\ 1611·Q14\Q14.()Q5 CMP

I !

I I

I :~ge ~

While stakeholder analysis is certainly helpful to gain a better understanding of the interests and resources of the important players for policy decision-making and implementation, it is even more so when used U1 conjunction with other strategic management tools such as political mapping, forcefield analysis, and the environmental analysis marrix (see IPC note, "The Political Environment for Implementation of Policy Change: Tools for Analysis", 1991.). With political mapping. stakeholder analysis can help to refine the placement of political groups on the map. In the case of forcefield analysis, it helps clarify a group's poSition as well as the comparative importance or salience of the group on the forcefield. Indeed. stakeholder analysis is generally a more interesting and powerful tool if used in combination with others.

better to try to develop a falIly ample list and then begin to reduce. If sufficient information is available, the analyst might also conSider drawing a political map or working up a tentauve forcefield analysis. If possible. expens at local universities or think-tanks or from the country's embassy should be sought out for their opinions. With all thiS, the analyst ought to be able to generate some early hypodleses regarding the array of stakeholders and their relative importance. Once in the field. the analyst should seek out local knowledgeables to obtain their opinions regarding key players and their interests. The analyst is encouraged to use as wide a range of informants as possible since many informants also have panicular agendas they wish to promote. Key informants can be quite varied: journalists, top officials in the church, leaders of business groups, congressmen. leaders of political parties, international donor officials, leaders of interest groups, university professors, consultants. embassy officials (other than the US too), labor leaders, radio and TV commentators, local think tanks, management institutes, high ranking military officials or professors at the military colleges, and local and international PVO heads. Although personal interviews are the standard method of obtaining information. other techniques can be used. For instance. this writer has had success in using informal panel groups and workshops to discuss these issues and work through differences of opinions. The difficulty of this approach is that it generally requires more ~ than that enjoyed by most intermittent consultants. This technique is also a good sounding board for testing the analyst's ideas. Ideally, the developing manager should acquire a solid familiarity with the tool both to diminish reliance on outside advice and in order to gauge the effectiveness and veracity of external advice and analysis when sought. Besides the obvious review of appropriate literature, there are a couple of useful mechanisms thal this writer has found for U'ansferring the technology. One useful method is through workshops; fl!st, a brief introduction to the method and second. case studies or exercises to hone skills in using the technique. If several mdivldt;,3.ls will be expected to work with stakeholder anal:, SIS, this method is particularly effective. Care should be exerclSed to include only those who indeed ',\.ill work with the technique and who will actually find it

How to Conduct a Stakeholder Analysis for a Reconnaissance Visit: There are several reasons why an external analyst might carry out a stakeholder analysis. First. for identification of the client and where he/she sits in the environment (who he/she is, what he/she is supposed to do, the importance of the position in relation to others) and to understand some of the pressures and expectations regarding hislher role. Second, the analyst should acquire a broad understanding of the environment and how stakeholders interact with the environment and the organization in order to playa more effective role with the client_ A knowledge of who's who and why will produce a more effective interaction with the client. Third, managers can sometimes hold strong opinions about stakeholders whicb conflict with generalized perceptions in the environment. 'The external analyst can playa valuable role as an "independent auditor" of those stakeholders_ Finally, given that some approaches to stakeholder analysis can be quite time-consuming, the external analyst can provide at the very least an initial cut to be reviewed by the client. Generally, the analyst can begin the stakeholder analysis before going into the field through perusal of the literature on the case country's politics and economics. In addition to standard sources such as the CIA's annual World Factbook. and professional journal publications, World Bank, AID, and UNDP publications can be helpful. The analyst should make a list of the groups that seem most relevant to the policy issue under consideration. One should not try to be very restricuve at this stage-it would be MorCl"lim

WPOATA\RoPORTS\ 10 i i -Qi41C 14-005 c~c

useful. Those who will not actually use it will find it quae tedious. Ano!.her useful me!.hod. especially when only a very narrow group will benefit is through one on one consultation in the use of the Instrumenl. When the number of users is more than three. however. the workshop method will prove more efficient.

share !.heir own views and perhaps diminish the Importance of others. While noc perfect. the consultation process can help reduce bias. When should the manager undertake the analYSIS? There are two points at which stakeholder analysis IS critical. First. when the policy is being formulated-at the point when decisions regarding who will be favored are taken. While It IS true that they often are not taken into conslderauon at this point. managers can supply important mput regarding critical stakeholders and how they can affect policy outcomes. Since policy makers are often not in direct contaCt or have little to do with critical stakeholders. information supplied by the manager. who is in much closer and direct contact. can be critical. It is at this point that the manager can help the policy maker avoid erroneous decisions. The second point is in the formulation of a strategy • for implementation. It is at this point where the manager will have greatest input. It is also at this point where decisions become critical in terms of asSuring alliances and suppan. A solid analysis of stakeholder expectations and a keen apprec..i.atiQg,-f)j ... the relative imponance of different stakeholder groups.· ., can be. key input for the design of stralegles to handlecenain groups. knowing what pieces of me policy should be emphasized. or how t6 assure future support. Finally. and perhaps as a warning. since stakeholder eltercises can be fascinating. it can be tempting [0 devote too much time. and worse, too much credence to the analysis. The stakeholder analysis is only a tool. one that helps to understand better the field upon which policy change and the implementation of those changes will be played. It is not an end in itself.

How can the Developing Country Manager use Stakeholder Analvsis: In many respects there are parallels in the use and usefulness of stakeholder analysis for the developing country manager and the elttemal analyst/consultant. However. it can generally be assumed that local managers have the advantage of their own knowledge (bolstered by a historical view) and usually much readier access to local knowledgeables. Nevenheless. managers should be warned that famil~ty ~th names and groups does not necessarily amount to a suong analytical understanding of the different stakeholders. Often. the manager will know (perhaps intuitively) that a particular group is suong and needs to be taken into account. But he/she may ndt understand why the group needs to be taken into account. What are the groups resources? How quickly can they be mobilized? What are their interests and whY,":, For PW'pOses of strateg v construction. these are imponant pieCes 01 ,nformation. Ids imponant that the manager confmn his perceptions of stakeholders either through group discussions with associates within his organization or through eltternal analysis. It is only natural that managers will elevate the imponance of groups that

REFERENCES Brinkerhoff. Derick W .• Improvjng Development . Program Performance. Boulder. Colorado: Lynne Reinner Publishers. 1991. Bryson. John M. Smucgic Planning for Public and Non-profit Orv&lIZ?!rionS. Oxford: Jessey-Bass. 1988. Gamman. John. "Stakeholder Analysis", (unpublished memolnote for IPC program. 1991). Honadle. George and Cooper. Lauren. "Beyond Coordination and Control: An Interorgaruzational 1m Approach to Sauctural Adjustment. ServICe Delivery. and Natural Resource Management." World Development. 17, no. 10, 1989. Lindenberg. Marc and CrOSby, Benjamin. ~1anagetng Deyelopment: the Political Dimension. Hartford. Connecticut: Kumarian Press. 1981. White. Louise G.• "Implementing Policy Changes" (unpublished paper attached to US ...lJD PrOject Paper for Implementing Policy Change ProJect.) September. 1989.

V10rc:~

..,;lCATA\ RE~I7TS\ 1611 <l1~\O 14<105 c~p

Annex 11

CONSULTATION ON THE EVALUATION OF HEAL TH FINANCING REFORMS Geneva. 6-8 June. 1995 Location: WHO Headquarters. Room .\1505

AGENDA Tuesday, 6 June 1995

09.00 - 09.30 09.30 - 10.00

Introduction and welcome Presentation of the Framework for Evaluation of Health Financing Reforms Discussion of the framework

10.00 - 10.30 10.30 - 10.45 10.45-11.15

COFFEE BREAK Presentation by Ghana Financing reform to be assessed, context and objectives of the reform, and proposed data sources and methods of analysis: foUowed by .liscussion Presentation by Kyrgyzstan Financing reform to be assessed, context and objectives of the reform, and proposed data sources and methods of analysis; foUowed by discussion Presentation by Mexico Financing reform to be assessed, context and objectives of the reform, and proposed data sources and methods of analysis; followed by discussion

11.15 - 11.45

11.45 - 12.15

12.15 - 13.45 13.45 - 14.15

LUNCH Presentation by Thailand Financing reform to be assessed, context and objectives of the reform, and proposed data sources and methods of analysis; followed by discussion Presentation by Uganda Financing reform to be assessed, context and objectives of the reform, and proposed data sources and methods of analysis; followed by discussion

14.15 - 14.45

2 14.45 - 15.15 Presentation by Zimbabwe Financing reform to be assessed. context and objectives of the reform, and proposed data sources and methods of analysis~ foUowed by discussion

15.15 - 15.30 15.30 - 16.00

COFFEE BREAK Presentation on indicators, data sources, and methods for evaluating the effects of finance reforms Presentation on methods for evaluating the effects of finance refonns and strategies for institutionalizing evaluation in the policy making pro·. :) Discussion of indicators, data sources, methodologies, capacity building and institutionalization Summary of the day and discussion of arrangements for the rest of the Consultation

16.00 - 16.30

16.30 - 17.15

17.15 - 17.30

17.30

RECEPTION

Wednesdav, 7 June 1995 Each country team will convene to develop a plan (protocol) for evaluating, over a 12 to 18 month period, the health financing reform( s) being implemented in their country. This will include an estimate of budgetary requirements. Each team will be supponed by at least one additional resource person.

Thursday, 8 June 1995

09.00 - 10.30

Country groups continue to work on developing evaluation protocols. and prepare brief presentations for the plenary session

10.30 - 10.45 10.45 - 12.15

COFFEE BREAK Presentation of protocols by each country group~ followed by discussions

12.15 - 13.45 13.45 - 15.15

LUNCH Presentation of protocols by each country group~ followed by discussions (continued)

3

15.15 - 15.30 15.30 - 17.00

COFFEE BREAK ~ext steps: discussion of plans and financial arrangements for

implementing the work in each country

17.00 - 17. 15

Closure of the meeting

•• •

,

'

o

Annex 12

CONSULTATION ON THE EVALUATION OF HEAL TH FINANCING REFORMS Geneva. 6-8 June 1995 Location: WHO headquarters, Room M505

LIST OF P ARTICIP ANTS GHANA Dr Jennifer Brown-Aryee Policy, Monitoring and Evaluation Unit Ministry of Health P.O.Box M 44 Accra Tel: 233 32 665 421 ext. 4631 Fax: 233 21 667 967

KYRGYZSTAN Ms. Ajnagul Shajakmetova MANAS National Team Member, Health Care Reform Programme c/o Ministry of Health Moskovskaya Street 148 720405 Bishkek Tel: 7 3312 223295 Fax: 7 3312 262314

MEXICO Dr Carlos Cruz-Rivero General Director of Health Economics Unit Ministry of Health Londres 226, Piso 6 Col. Juarez 06600 Mexico D.F. Tel: 525 6002 Fax: 525 6202

Ing. Jose Antonio Rosas Mantecon Private Secretary to the Undersecretary of Planning Ministry of Health Lieja No.7, Piso 1 Colonia Juarez 06696 Mexico D.F. Tel: 525) 553 7292 Fax: 286 53 55 THAILAND Dr Sanguan Nitayarumphong Assistant Permanent Secretary on Health Policy and Planning Ministry of Public Health Tivanont Road Nontaburi Tel: 591 8510 Fax: 02) 5918510 Dr Supasit Pannarunothai Public Health Specialist Academic Department Buddhachinaraj Hospital Amphur Muang Phitsanulok 65000 Tel: 055 247 572 Fax: 66 55 219063 or 6655 258 813 UGANDA Dr Jesica Jitta Director Child Health & Development Centre Makerere University P.O.Box 6717 Kampala Tel: 256 41 541 684 Fax: c/o WRlUganda 25642 21164 Dr J.H. Kyabaggu Commissioner for Medical Services in charge of Primary Health Care, Planning and Training Chairman of the National Task force on Health Care Financing Ministry of Health P.O.Box 8 Entebbe Tel: 256 42 21110 Fax: 256 42 20608

., .J

ZIMBABWE Mr S.1. Chihanga Assistant Secretary Planning & Management Ministry of Health and child Welfare P.O.Box CY 1122 Causeway Harare Fax: 7291541793634 Mr T.A. Zigora Deputy Secretary (Finance, Administration and Planning) Ministry of Health and Child Welfare P.O.Box CY 1122 Causeway Harare Tel: 263 4 724715 Fax: 7291541793634

Temporary Advisers Dr Barbara McPake Lecturer in Health Economics London School of Hygiene & Tropical Medicine Keppel Street London WCIE 7HT England Tel: 0171 927 2267 Fax: 0044 71 637 5391 Dr Fran~ois Diop Associate Abt Associates Inc., Hamden Square, suite 600 4800 Montgomery Lane Bethesda, Maryland, 20814 USA Tel: (301) 913 0526 Fax: (301) 652 3916

INTERNA TIONAL AGENCIES

Dr Knut Odegaard Project Manager Swedish Institute for Health Economics Box 2127 A- 22004 Lund Sweden Tel: 046 329100 Fax: + 46 46 121604 Mr Clas Rehnberg Assistant Professor of Health Economics Stockholm School of Economics Centre for Health Economics Box 6501 113 83 Stockholm Sweden Tel: 46 8 736 9284 Fax: 46 8 3021 15

UNICEF Mr Abdelmajid Tibouti Senior Adviser in the Bamako Initiative Management Unit UNICEF 3 United Nations Plaza New York, N.Y. 10017 USA Tel: 212 326 7539 Fax: 212 326 7059

USAID Mr Robert C. Emrey Technical Adviser Division for Health Policy and Sector Refonn Office of Health and Nutrition Bureau for Global Programme, Field Support, and Research US Agency for International Development (USAID) 320 Twenty-First Street Washington D.C. 20523 USA Tel: 703 875 4566 Fax: 703 875 4633

5

WHO Regional and Country Representatives Dr Mario Boyer Regional Advisor Health Systems Development Division of Health Systems and Services WHO Regional Office for the Americas/ Pan American Sanitary Bureau Tel: 202 861 3225 Fax: 202 861 2648 Dr. Gillin Gedik WHO/EURO c/o Health Care Policies and Systems Unit Resident Technical Advisor, MANAS Health Care Reform Programme c/o Ministry of Health of Kyrgyzstan Moskovskaya 148 Bishkek 720405 Kyrgyzstan Tel: 7 3312 2232 95 Fax: 7 3312 26 23 12 Dr B. Sabri Regional Adviser/Managerial Process for National Health Development (RAIMPN) WHO Regional Office for the Eastern Mediterranean (EMRO) Tel: 20 3 482 0223/482 0224 Fax: 20 3 483 8916 Ms Sylvia Tereka WHO Country Team Economist Member of Joint WHO/Ministry of Health Working Group on Health Care Financing c/o The WHO Representative P.O.Box 6 Entebbe Uganda Tel: 256 42 20572 Fax: 2564221164 Mr Robert Zegers Associate Professional Officer Managerial Process for National Health Development (MPN) WHO Regional Office for South-East Asia (SEARO) Tel: 91 11 331 7804 Fax: 91 11 331 8607

Division of Strengthening of Health Services (SHS) Dr J.-P. Jardel, Assistant Director-General Dr. E. Tarimo, Director, Division of Strengthening of Health Services (SHS) Dr I. Tabibzadeh, Chief. District Health Systems Mr A. Creese, Chief National Health Systems and Policies (NHP) Dr K. Janovsky, NHP Mr 1. Kutzin, NHP Mr 1. Muschell, NHP Dr P. Travis, NHP

Representatives from Headquarters' Divisions Dr G. Carrin, Division of Intensified Cooperation with Countries (lCO) Dr S. Orzeszyna, Medical Officer, Division of Epidemiological Surveillance and Health Situation and Trend Assessment (HST) Dr S. Simeant, Medical Officer/Epidemiologist, Division of Epidemiological Surveillance and Health Situation and Trend Assessment (HST) Dr S. Muziki, Action Programme on Essential Drugs (DAP) Dr E.R. Villar Montesinos, Division of Intensified Cooperation with Countries (lCO)