What, if anything, does the Euro Health Consumer Index actually tell us?

Eurohealth incorporating Euro Observer — Vol.22 | No.1 | 2016 31 WHAT, IF ANYTHING, DOES THE EURO HEALTH CONSUMER INDEX ACTUALLY TELL US? By: Jonathan Cylus, Ellen Nolte, Josep Figueras and Martin McKee Since 2005, the Health Consumer Powerhouse has produced its annual Euro Health Consumer Index,[1] ranking European health systems according to their performance[2] on a host of indicators around (i) patient rights and information, (ii) accessibility, (iii) outcomes, (iv) range and reach of services, (v) prevention and (vi) pharmaceuticals. In its most recent iteration, the United Kingdom ranked only 14th of 35 countries studied. This is in stark contrast to the assessment by the Commonwealth Fund just a year before, in which the UK was rated as the best performing health system among a set of high-income countries in 2014.[3] Jonathan Cylus, Ellen Nolte, Josep Figueras and Martin McKee, European Observatory on Health Systems and Policies, Brussels and London. Email: j.d.cylus@lse.ac.uk While we understand the excitement surrounding health system rankings, we caution against over-interpreting them and, especially, the Euro Health Consumer Index which, as we will show, is especially problematic. Arbitrary scores are given to indicators The index is constructed by scoring performance in the five areas listed above as good (3), intermediary (2) or not-so-good (1), based on arbitrary cut-off points. Consequently, countries with similar performance will receive very different scores if they are just on either side of the cut-off point. For example, Poland scores not-so-good on “equity of health care systems” because only 69.6% of health care is publicly funded. Yet Slovakia receives an intermediate score as it achieves 70.0% (a whopping 0.4 percentage points more). Switzerland earns a “good” score on this measure, despite its high levels of deductibles and out-of-pocket spending as a share of total expenditure that is twice the EU-15 average, along with high levels of unmet need compared to other countries and many peer-reviewed studies concluding that the Swiss financing system is regressive. 4 The point system does not reflect what matters to citizens There is no obvious logic in how many points are allocated to each indicator. For example, all health outcomes indicators are worth a total of 250 points, while accessibility is worth 225 points. Yet there are more outcome indicators (8) than accessibility indicators (6), so that the maximum score on any given accessibility metric (e.g. waiting time) will be higher than on an outcome metric (37.5 compared to 31.25 points). Thus, not only do abortion rates and cancer survival carry the same weight (since both are considered health Acknowledgement: Reprinted by permission of The BMJ. Originally published online on 9 February 2016. O PI N IO N Eurohealth SYSTEMS AND POLICIES Eurohealth incorporating Euro Observer — Vol.22 | No.1 | 2016 32 outcomes) but an outcome indicator like cancer survival counts less than an accessibility indicator like direct access to a specialist. This seemingly indiscriminate approach to allocating points also means that countries can accumulate similar points by prioritising different issues, even if these are unlikely to be seen as equivalent by citizens, had they been asked. For example, a hypothetical country coming last on cancer survival and infant deaths earns approximately 10.4 points for each indicator (out of a possible 62.5 total points) leaving around a 41.6 point deficit. Rather than investing in improving these outcomes, it could just compensate for this simply by abolishing gatekeeping, since allowing direct access to a specialist can gain 37.5 points), which may 6 or may not 7 improve outcomes on these measures. There is no apparent basis for selecting the indicators Lastly, the indicators are a strange mix of trends over time and cross-sectional rankings. For example, heart disease and stroke deaths are measured as changes over time, whereas hospital-acquired infections use the most recent data point, penalising countries showing substantial improvements, such as the United Kingdom where the percent of hospital- acquired infections being resistant has fallen from 21.6% in 2010 to 13.8% in 2013. Others, such as Estonia, have seen this figure increase by 2.8 percentage points in the same period (from an admittedly low level, at just 3.5% in 2013) but still receive the maximum points. Conclusion: We should just ignore the findings of the Euro Health Consumer Index While many other health systems rankings that have been widely criticised, such as the 2000 World Health Report, these are far more transparent, methodologically, than the Euro Health Consumer Index. Yet, there is no “right” way to rank health systems, or any other complex system for that matter. Choices must be made regarding the indicators to be assessed and the values to attribute to them. However, the notion that we can (or should!) rank health systems based on a single measure that seemingly haphazardly combines indicators that have been “scored” what seems to be at random is debatable. Composite indices conceal what is actually going on in health systems, and offer little guidance for policymakers who want to improve the performance of their system. Although the report accepts that its results are not “dissertation quality” and must be treated “with caution” it draws inappropriate conclusions about the superiority of one system versus another one, leading to uninformed recommendations and assertions that display limited understanding of health systems. This is patently irresponsible. There is great potential for countries to learn from each other through careful comparison but the Euro Health Consumer Index’s use of poorly constructed composite indices of uncertain origin is unlikely to inform any evidence-based policy development. References 1 Euro Health Consumer Index 2015. Available at: http://www.healthpowerhouse.com/ files/EHCI_2015/EHCI_2015_report.pdf 2 Watson R. Netherlands tops European healthcare index. BMJ 2016;352:i538. Available at: http://www. bmj.com/content/352/bmj.i538 3 Davis K, Stremikis K, Squires D, Schoen C. Mirror, mirror on the wall: How the performance of the US health care system compares internationally. New York: Commonwealth Fund, 2014. Available at: http://www.commonwealthfund.org/~/media/files/ publications/fund-report/2014/jun/1755_davis_ mirror_mirror_2014.pdf 4 De Pietro C, Camenzind P, Sturny I, et al. Switzerland: Health system review. Health Systems in Transition 2015;17(4):1–288. Available at: http://www.euro.who.int/en/about-us/partners/ observatory/publications/health-system-reviews- hits/full-list-of-country-hits/switzerland-hit-2015 5 The world health report 2000 – Health systems: improving performance. Geneva: World Health Organization, 2000. Available at: http://www.who.int/ whr/2000/en/ 6 Hawkes N. The role of NHS gatekeeping in delayed diagnosis. BMJ 2014;348:g2633. Available at: http:// www.bmj.com/content/348/bmj.g2633 7 Olesen F. Re: The role of NHS gatekeeping in delayed diagnosis. 27 April 2014, Available at: http:// www.bmj.com/content/348/bmj.g2633/rr/695920 Luxembourg: Health system review in brief By: F Berthet, A Calteux, M Wolter, L Weber, E van Ginneken, A Spranger Copenhagen: World Health Organization 2015 (acting as the host organization for, and secretariat of, the European Observatory on Health Systems and Policies). Number of pages: 18 Freely available for download at: www.euro.who.int/__data/ assets/pdf_file/0006/287943/Mini-HiT_Luxembourg-rev1. pdf?ua=1 The Luxembourgish health system has the highest per capita health spending in purchasing power parity amongst European countries and lacks capacity to train health personnel, with shortages in some specialty care, which also necessitates a generous policy towards receiving care abroad. Reforms in 2008 and 2010 have targeted these challenges by establishing a single health insurance fund envisioned to play a stronger role in cost-containment and introducing e-health. There is also room for efficiency gains, especially in hospital care. Current reforms aim to implement a national structured health information system for hospital services, which is a prerequisite to further announced reforms such as the introduction of a diagnosis-related groups based payment system.