- Open Access
- Open Peer Review
Health system performance assessment landscape at the EU level: a structured synthesis of actors and actions
Archives of Public Healthvolume 75, Article number: 5 (2017)
Many policy makers and other stakeholders in the EU have expressed interest in better understanding the performance of their own health systems to identify opportunities for improvement in effectiveness, efficiency and equity. Health system performance assessment (HSPA) has received considerable attention at EU level as an instrument to improve transparency and accountability. This is equally important for population health and sustainable health spending. The goal of this paper is to synthesise and map the current state and developments in the field of HSPA relevant in the EU context and by this aid the navigation in the growing HSPA system, understand the available tools and identify opportunities for improvement.
Structured synthesis of the literature on initiatives in the field of HSPA at EU level was carried out. Key literature was identified by a focused review performed between October 2015 and June 2016 on websites of key institutions including the EU, OECD and WHO and Google engine. We used six predefined criteria for identifying key literature. Identified initiatives were classified according to analytical and conceptual output or whether a guiding or advisory role was resumed. A visual map of the relationships between the different actions and actors involved in HSPA was developed. In addition, expert opinion was sought to refine the map.
We identified a total of 64 relevant initiatives and their relationships in the field of HSPA. These include institutions such as the European Commission (73%), European Council (8%), OECD (9%) and WHO-EUR (9%). 24 initiatives produced analytical outputs, four developed conceptual outputs and six had a guiding role. The role of the EU in HSPA and collaboration with other key actors have intensified considerably since the adoption of the EU Health Strategy in 2013. The EU HSPA landscape is complex with seemingly few streamlining activities.
Knowledge transfer and exchange of expertise are key to HSPA. While cooperation between the key actors have intensified recently and clearly reflect the “Health in all Policies” (HIAP) approach, there is considerable room for improved streamlining activities to share knowledge and avoid overlapping efforts, especially within the European Commission.
Many policy makers, researchers, public health specialists and other stakeholders in the European Union (EU) have expressed interest in better understanding the performance of their health systems to identify opportunities for improvement in effectiveness, efficiency and equity. At the same time, health system performance assessment (HSPA) at EU level has received considerable attention as an instrument to improve transparency and accountability.
to set out the goals and priorities for a health system;
to act as a focus for policymaking and coordinating actions within the health system;
to measure progress towards achievement of goals;
to act as a basis for comparison with other health systems;
to promote transparency and accountability to citizens and other legitimate stakeholders for the way that money has been spent.
While policy making in many areas of EU health systems is in the responsibility of Member States (MSs), cross-country comparisons of health system performance have become increasingly important. First, structural reform in many MSs and policy recommendations from the European Commission have been increasingly targeting health care. A large and growing share (30% in 2013) of social protection spending in the EU is used for health care services . Second, the crisis including recent migrant waves led to increased pressure on various segments of social spending including health. Sustainability and efficiency of health spending is thus high on the agenda. The development of the directive aimed at making cross-border health care for the EU citizens possible is seen as another major driver behind the increased use of comparisons and benchmarking of quality of care provided [2, 4]. Also, the growing availability of comparable datasets has enhanced the technical feasibility to compare performance . These data, carefully analysed and accompanied by conceptual models of knowledge are powerful tools to influence policy-making [2, 5]. Finally, providing expertise and assistance in the area of HSPA are a fundamental role for the EU level, in setting standards, collating and disseminating experience, and disseminating standardised information. Such endeavours are a public good that can only be done effectively and comprehensively at EU level, and can contribute new resources for guiding policy and practice in individual MSs.
Health system performance is measured against multiple objectives. This calls for a strong framework covering access, equity, efficiency and quality and their interrelation in order to understand the content and the scope of the comparison [2, 6]. Although HSPA is primarily a country-specific process for which there is no single accepted template, having harmonised methodologies and tools to support health policy makers in taking decisions requires actions also at EU level [1, 7]. The importance of good practice HSPA therefore has been receiving high-level support both at national and at EU level [7, 8]. The European Commission (EC) communication on effective, accessible and resilient health systems  and the mandate from November 2014 given to the European Commissioner for Health by the EC President to develop expertise for HSPA provide evidence of this support .
Progress has been made in monitoring the health of the population and the performance of health systems in terms of the scope, nature and timeliness of performance data that have been made publicly available over the last 30 years . Nevertheless, methodological challenges remain in accomplishing the aim of health system governance improvement. These challenges include creating and unifying reporting standards of data and indicators, and establishing coherent HSPA frameworks for cross-country comparisons . This should ultimately guide the way how best to deploy performance data . The wider performance framework developed under the EuroREACH project based on the OECD HCQI initiative may be seen as one of the starting points in this context .
Clear evidence is needed on the prioritization, including the necessity and rationale for specific health system indicators. Furthermore, core indicators for different types of policy-use, (e.g. monitoring/forecasting, benchmarking, target-setting, cross-country comparison) should be identified and categorised accordingly. In recent years, some advanced cross-country approaches were established analysing the comparative performance of disease specific variables across countries (EuroHOPE)  as well as regional variations of hospital indicators (ECHO) . However, there is a lack of a European-wide coherent framework addressing data needs, quality of data and guidance in using and applying indicators of HSPA.
While the Euro Health Consumer Index (EHCI), a private initiative ranking 35 European health systems according to their performance along 38 indicators shed light on the potential of comparing important health and health system indicators , it has been criticised by some for the lack of transparency in indicator selection and scoring . More work is needed in providing standards for the quality of indicators and for a sound rational of monitoring specific indicators and their significance in cross-country comparisons. These challenges are being addressed in BRIDGE Health, a European project that aims at preparing a comprehensive, integrated and sustainable EU health information system which will incorporate know-how and technical tools to coordinate and harmonise research and surveillance for MSs in key EU health policy areas . The present work forms an integral part of BRIDGE Health research activities. In addition, the BRIDGE Health System Indicator Task Force was established to exchange expertise on the work of health system performance indicators development.
Currently, stakeholders have difficulty navigating the HSPA landscape, understanding the opportunities and tools it provides, and using the data to identify opportunities for improvement in effectiveness, efficiency and equity of their healthcare systems. To the best of our knowledge, no synthesis exists to help understand the organization of the HSPA system at the EU level. Hence, the goal of this paper is to synthesise the current state and developments in the field of HSPA relevant to the EU context by visually mapping the relationships between the different actions and actors involved in HSPA in the EU.
The rest of the paper is organised as follows. The second part presents the methods, including six predefined selection criteria of initiatives and the output classification that led to the structure of the visual map. Section 3 describes the different initiatives that were identified both at EU and joint action levels. Finally, the implications of the current structure of HSPA in the EU are discussed.
Sources and initiative selection
A focused search was conducted and documented to identify initiatives in health system performance at the EU level. For simplicity, in this initial step we considered initiatives that are either completely or mostly at EU level covering the following key institutional players: the European Commission and its relevant Directorates-General (DG), EU agencies and the European Council, the Organisation for Economic Cooperation and Development (OECD), and the World Health Organisation Regional Office Europe (WHO-EUR). Initiatives developed at the level of MSs and other international or intergovernmental initiatives that do not pertain to Europe (e.g. Commonwealth Fund, World Bank) were excluded. Due to the nature of institutional websites as the primary source of this search and the inherent linkage between findings, formal systematic review methods were not pursued for the mapping exercise. The following selection criteria for initiatives were applied of which a minimum of three had to be fulfilled:
Initiatives that aim to harmonise monitoring of health systems and health policy: initiative i) use internationally comparative data, ii) are available within the last 16 years, iii) have recurrent output available in English language.
Initiatives that pertain to HSPA frameworks.
Initiatives that seek to foster transparency, accountability and accessibility of data in the field of public health and health systems research.
Initiatives that inform a blueprint for an indicator repository of a European health information infrastructure by referring to a list of indicators.
Initiatives that provide assessment of indicators and meta-information related to the provision of i) references on the indicator selection process, and ii) description, calculation, rationale, data availability, comparability of indicators.
Initiatives that provide analytical outputs for evaluating health systems including a quantitative analysis accompanied by an interpretation of analytical outputs.
The current search was conducted on over 25 websites of the key institutions, and the top 50 search hits of the Google website. In addition, the database on completed European and international projects drawn from the Health Data Navigator (HDN) was used . The HDN was developed as part of the EuroREACH project and is continued in BRIDGE Health. The search was limited to a publication date after 2000 and was performed between October 2015 and June 2016 accompanied by three rounds of adjustments based on input from experts of the BRIDGE Health System Indicators Task Force. Included were policy and research documents, consultation and opinion papers, project reports, meeting minutes and other corresponding electronic documents. In total, over 75 websites were searched with the following terms in English to draw out a broad range of relevant initiatives: “health system performance assessment” AND (“framework” OR “initiative” OR “method”) AND (“EU” OR “European” OR “Europe” OR “European Commission”). Tthe search string (“EU” OR “European” OR “Europe” OR “European Commission”) was not used on websites related to the EC and the European Council to avoid redundancy. Reference lists of included literature were also examined. However, the majority of the initiatives included in this synthesis were identified with the search terms on the respective websites.
A visual organizational map was created which features identified initiatives. Such a map is a commonly used diagram in the field of business administration that outlines the roles, responsibilities and relationships between individuals within an organization in just one picture [19, 20]. It typically illustrates the relations and decision making power by linking the different functions depicted as boxes with lines on different levels [19, 20]. For our purpose, the map was organised by institution and the extent of their contributions by making use of graphical forms and colour schemes. The map makes use of angular rectangles to represent these institutions, their associated key directorates the supporting agencies as well as the individual initiatives. We further distinguished the type of resources produced by these initiatives:
Analytical output (yellow): provision of health data series, indicators and/or any kind of analysis or accompanied by respective country reports.
Conceptual output (green): providing literature review and/or conceptual innovation (e.g. suggesting indicators, defining frameworks).
Guiding or advisory, body (blue): mandated to create recommendations and issue opinions on specific topics related to health information and HSPA.
Initiatives for which more than one type of output applies are represented by multi-coloured rectangles reflecting their multidimensionality. Lines indicate collaborative or supportive relations between actors and initiatives. While there are of course links between Eurostat and OECD, the map expresses this collaboration via the central role of the EC. A grey line connecting EC and OECD is used to represent this increasingly important collaboration. This synthesis is complemented by a timeline perspective in the form of a flow chart were the evolvement of actors and actions between 1998 and 2016 in the field of HSPA is displayed.
Figure 1 presents the current structure of HSPA initiatives and their respective organizational actors. The synthesis of these initiatives resulted in a total of 64 activities indicated by different boxes. It consists of a total of four key institutions aligned with six DGs in the EC which are linked to seven supporting agencies. The majority of displayed initiatives is associated with the EC and the European Council amounting to 37 in total when excluding the aforementioned EU institutional players (N = 17). Five initiatives were included for both the OECD and WHO-EUR. Concerning the type of output, the map displays a total of 24 initiatives that were classified as solely producing analytical outputs, four with conceptual outputs and six with a guiding and/or advisory role. Both analytical and conceptual outputs are provided by a total of nine initiatives which are represented by multi-coloured rectangles in green and yellow. A combination of conceptual output and guiding activities could be linked to a total of three initiatives illustrated by green and blue rectangles such as the “Peer review on HSPA” initiative. In total eight initiatives including the supporting agencieswere categorised to have all three types of output. Figure 2 gives an overview of the development process in the field of HSPA initiatives over time indicating their relevant starting dates. Brief narratives to the main findings are provided below.
International agencies play an important role in promoting the comparison of health system performance across EU countries . Since the World Health Report 2000, plenty of resources have been developed to facilitate such comparison and support HSPA in general . The OECD and WHO have an instrumental role in driving international comparisons of health systems with their data repositories. The most important data are the OECD Health Statistics including the “Health Care Quality Indicators (HCQI)” [21–23] and the “WHO Health for All database” , and specific annual publications, such as “Health at a Glance”  or the “European Health Report” [26, 27]. Furthermore, we identified different HSPA related topics featured explicitly in publications resulting from the OECD’s “health policy studies” [28–31]. Analytical work on the efficiency of health care systems performed by Joumard and colleagues remains central in this area . More recently, the OECD initiated an “ad hoc reflection group” to seek comments from experts on health care efficiency indicators with the intent that comments would facilitate prioritization of these indicators .
A major milestone in the collaboration between the EC, OECD and WHO-EUR was the initiation of the joint data collection in the context of the OECD System of Health Accounts (SHA), the standard framework for producing consistent and internationally comparable health system financial data [1, 33]. Also, the biannual publication series “Health at a Glance: Europe” represents another major and vastly important collaboration output between the EC and OECD.The selection of indicators for this report is largely based on the “European Core Health Indicators (ECHI)” shortlist. This list has been created by the EC with the aim to monitor health in the EU . The “European Observatory of Health Systems and Policies (OBS)” is a further partnership between the EC, the WHO-EUR, the World Bank and certain MSs serving as a prime source for informed and comparable descriptions of health systems . The “European Health Information Initiative (EHII)”, established in March 2015, is another recent collaboration between WHO-EUR, EC, OECD, MSs and related health information networks and associations supporting the development of a single European health information system focusing on a set of targets and indicators as defined in Health 2020 . The “State of Health in the EU” initiative, launched in June 2016, is the youngest collaborative initiative with DG Santé and OECD which is aided by the OBS. Itt aims to develop individual country health profiles to support health policies in EU countries .
Main actors within the European Commission in the field of HSPA are DG Santé, DG RTD, DG ECFIN, Eurostat, DG EAC and DG EMPL. The European Commission has created the “European Core Health Indicators (ECHI)” initiative, which assembles 88 indicators relevant to HSPA, for over 50 of which data are readily available and reasonably comparable. Besides the ECHI indicators and the joint “Health at a Glance: Europe” reports, DG Santé operates the field of HSPA with further initiatives. First, the “Expert Group on HSPA (EG HSPA)” serves as a forum where MSs exchange experiences on the use of HSPA at national level, and receive support in identifying tools and methodologies to develop HSPA further . So far, one relevant report has been produced describing the different strategies MSs apply to assess quality of care. Further work of the EG HSPA will cover prioritised topics such as integrated and primary care [39, 40]. Furthermore, the independent and multidisciplinary “Expert Panel on Effective Ways of Investing in Health (EXPH)” also issued an opinion on identifying criteria for prioritizing areas for HSPA which was based on work of the EG HSPA’s predecessor, called “Sub-group 5 on measuring and monitoring the effectiveness of health investments” [12, 41]. The work of Sub-group 5 was initiated by the Council’s “Working Party on Public Health at Senior Level” in 2011. Most recently, access to health services was also subject to a detailed statement by the EXPH . The “Expert Group on Health Information (EGHI)”, a consultative MSs body, supports the implementation of national and cross-EU health strategies including health information . Core to this is the envisioned set-up of a European Research Infrastructure Consortium (ERIC) on Health Information which is currently being conceptualised within the BRIDGE Health project [43–45]. Fig. 1 lists most of the projects endowed by BRIDGE Health that collected new data and developed health information frameworks, standardisations and quality control methods. The “European Structural and Investment Funds ESIF for Health” developed a toolkit with a set of widely available indicators deemed useful for the final evaluation of actions supported from the ESIF .
Several other relevant projects have been funded under the Commission’s FP7 programme to identify and analyse health data from the perspective of cross-country comparisons including “EuroREACH” , “EuroHOPE”  and “ECHO” . DG RTD funds EU research programmes as part of Horizon2020 and the Joint Research Centre (JRC). In the recent programme JRC aims to provide science and knowledge service on health and safe environment and consumer health and safety among other non-nuclear topics . In addition, the “European Research Infrastructure Consortia (ERICs)” “SHARE”, “BBMRI” and “ESS” were established to provide resources and related services used by the scientific community to conduct top-level research in the broader field of health [50–53].
DG ECFIN has produced three relevant pieces of work focusing on the sustainability of health services. For example, in the context of the “European Semester” Country-Specific Recommendations (CSR) are made through the analysis of health system efficiency [54–57].
At EU level, the European Statistical Office, “Eurostat”, is the main data collection body and main data provider . Its role in the area of high quality health data has enhanced greatly in recent years. In light of the broadened understanding about the importance of wellbeing for economic progress, the “Quality of life” indicators were developed . Together with OECD’s “Better Life initiative”  they form a further strand of approaches where relevant indicators are being used. These initiatives are currently only loosely connected. At Eurostat level, the initiative on “Sustainable Development Indicators” is using selected health indicators at different levels of importance for monitoring the sustainable development strategy of the EU . DG EAC monitors the health of Europe’s youth with one tailored tool [62, 63].
Lastly, the “Social Protection Committee (SPC)” and its Indicators Subgroup adapted the “Joint Assessment Framework (JAF)” methodology to the area of health systems. This serves as a reference HSPA tool for supporting the monitoring and assessment of structural reforms focusing on issues related to access, quality and equity [5, 8]. Progress on the social protection policy goals is monitored with a tool called “Social Protection Performance Monitor (SPPM)” that makes use of some specific ECHI indicators . The recent “European Pillar of Social Rights” initiative led by DG EMPL also addresses the importance of timely access to ensure good quality and affordable health care and long-term care for Europeans .
This paper focused on synthesising and visualizing major HSPA activities and actions in the EU context initiated before and ongoing in June 2016. This synthesis is complemented by a timeline perspective. Fig. 2 gives an overview of the development process in the field of HSPA initiatives. It shows that activities in this area have intensified in recent years including enhanced efforts to better collaborate across key international actors. The results of this synthesis paper highlight three main points within the sphere of HSPA in the EU.
First, there is a clear reflection of the “Health in all Policies” (HIAP) approach  emphasised in the Commission’s Communication in 2014 on health systems and a growing visibility of HSPA as shown in the timeline perspective . The Commission promotes cooperation at EU level with a view to strengthen effectiveness, increase accessibility and improve resilience of the national health systems in the EU. Many DGs have activities in place to promote HSPA, such as the provision of comparable indicators to monitor health and health systems (ECHI), expert bodies for better exchange of experiences between MSs (e.g. EG HSPA) or structured HSPA frameworks to guide monitoring (JAF Health). These actions are complemented by co-funded research in this area .
Second, the map reflects the complexity of coordinating recent activities in HSPA. Different roles assumed by individual DGs and their relationships between each other within the field of HSPA are not entirely clear at this stage. For example, while DG Santé is mainly concerned with population health and health information issues in general, DG EMPL has a focus on access to health care including equity issues, and DG ECFIN concentrates on efficiency and sustainability. While this fragmentation between DGs largely reflects policy mandates under current EU governance responsibilities, it calls for improved cooperation across them to better bundle expertise on the different domains covered by HSPA. Also and as Fig. 2 indicates, HSPA only gained formal attention in 2013 with the adoption of the EU Health Strategy “Together for Health” . Even though the Commission managed to considerably increase the visibility of the role of the EU in HSPA since then, the visualisation of the current activities suggests that coordination needs improvement. For example, to achieve the goals outlined in key documents on health systems 2014 regarding e.g. resilience and sustainability likely requires more capacities across Commission services [9, 10]. Increasing cooperation within the Commission would reduce duplications of work and ensure more inclusivity.
Third, the current mapping exercise should be seen as a starting point for extension to MS-level HSPA initiatives and potential other relevant international initiatives to build the base for the development of a comprehensive indicator repository. Such initiatives will be included in the planned next wave of our HSPA research in the context of BRIDGE Health activities. The work will focus on a comprehensive synthesis of the relevant indicator landscape. Starting point is the development of the european Health System Indicators survey (euHS_I survey). The euHS_I survey aims to identify overlaps between these initiatives and gaps in the relevant availability of health care system performance indicators. Also, the importance of indicators for the different performance domains will be assessed. This should lead to the compilation of a set of high quality headline indicators with important policy relevance for HSPA.
The synthesis provided within this paper has limitations. First, it focuses on major players and excludes MS initiatives as well as other potentially relevant international initiatives (e.g. World Bank, EHCI, Commonwealth Fund). Second, the visual map shows the status quo in 2016 including also bodies that were replaced by newly created Expert groups such as the Sub-group 5 from the WPPHSL paving the way to the creation of the EG HSPA. Figure 2 was created to complement the map with a timeline. However, communication/collaboration lines between different DGs were omitted for simplification purposes. For example, there is cooperation and exchange between EG on HSPA with EGHI and SPC/ISG. Not displaying those lines should not create the impression that there is no collaboration or interaction at all.
This paper synthesises, maps and describes the current developments in HSPA at EU level. To our knowledge, this is the first piece of work to map out the relationships between different actors and actions driving the field of HSPA.
The results of this mapping exercise can be used to obtain a comprehensive overview of the current developments in the field of HSPA. In summary, our findings indicate that the European Commission’s role in HSPA is increasingly important to promote standards for good practice HSPA. Also, activities are largely consistent with goals outlined with the adoption of the EU Health Strategy in 2013. This reflects the strengthened role of the European Commission as a hub for promoting unified standards to enhance comparative analysis of welfare systems. Equally, it acknowledges the increasing importance of health system performance for European policy making. However, the coordination of various HSPA activities by different actors, especially within the European Commission leaves room for improved cooperation in order to avoid duplication of work and ensure overall efficiency of ongoing activities.
Working Group on Ageing Populations and Sustainability
Consumer, Health and Food Executive Agency
Classification of the Functions of Government
- DG EAC:
Directorate-General for Education and Culture
- DG ECFIN:
Directorate-General for Economic and Financial Affairs
- DG EMPL:
Directorate-General for Employment, Social Affairs and Inclusion
- DG RTD:
Directorate-General for Research & Innovation
- DG SANTÉ:
Directorate-General for Health and Food Safety
European Centre for Disease Prevention and Control
European Core Health Indicators
European Collaboration for Health Optimization
European Food Safety Authority
Expert Group on Health Information
European Health Information Initiative
European Health Interview Survey
European Health and Life Expectancy Information System
European Medicines Agency
European Monitoring Centre for Drugs and Drug Addiction
Environment and health indicators system
Economic Policy Committee
- ESIF/ESI Funds:
European Structural and Investment Funds
European Social Survey
European system of integrated social protection statistics
European Agency for Safety and Health at Work
Diagnosis-Related Groups in Europe, towards Efficiency and Quality
European Health Care Outcomes, Performance and Efficiency
A Handbook to Access Health Care Data for Cross country Comparisons of Efficiency and Quality
European Union Statistics on Income and Living Conditions
Expert Panel on Effective Ways of Investing in Health
Health Care Quality indicators
Health for All
Health in all Policies
European Research Infrastructure Consortium on Health Information for Research and Evidence-based Policy
Health System Performance Assessment
Joint Assessment Framework
Organisation for Economic Co-operation and Development
Quality of Life
Sustainable Development Indicators
System of Health Accounts
- SHARE- ERIC:
Survey of Health, Ageing and Retirement in Europe - European Research Infrastructure Consortium
Social Protection Committee
Social Protection Performance Monitor
World Health Organisation
Working Party on Public Health at Senior Level
Smith PC. Peer Review Belgian Health System Performance Assessment. Brussels: 2014. http://ec.europa.eu/social/BlobServlet?docId=12450&langId=en. Accessed Feb 5 2016.
Papanicolas I, Smith PC. Health system performance comparison. An agenda for policy, information and research. European Observatory on Health Systems and Policies Series. World Health Organization. 2013.
European Commission. Commission Staff Working Document: Key economic, employment and social trends behind a European Pillar of Social Rights. Brussels: European Commission. 2016;SWD(2016) 51 final.
Directive 2011/24/EU of the European Parliament and of the Council of 9 March 2011 on the application of patients’ rights in cross-border healthcare. 2011; http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX%3A32011L0024. Accessed 28 Sept 2016.
EC Social Protection Committee Indicators Sub-group. Developing an assessment framework in the area of health based on the Joint Assessment Framework methodology: final report to the SPC on the first stage of implementation. Brussels European Commission. 2013.
Hofmarcher M, Smith P. The Health Data Navigator. Your toolkit for comparative performance analysis. A EuroREACH product. 2013. http://www.healthdatanavigator.eu/HDN_Toolkit_Final.pdf. Accessed Oct 15 2015.
EC Expert group HSPA. Minutes of 1st meeting - Expert Group on Health Systems Performance Assessment (HSPA), 10 November 2014, Brussels. 2014. http://ec.europa.eu/health/systems_performance_assessment/docs/ev_20141110_min_en.pdf. Accessed Oct 5 2015.
EC Social Protection Committee Indicators Sub-group. Progress report on the review of the Joint Assessment Framework in the area of health. Brussels. 2015.
European Commission. Communication on effective, accessible and resilient health systems. Brussels: European Commission. 2014; COM(2014) 215 final. http://ec.europa.eu/health/healthcare/docs/com2014_215_final_en.pdf. Accessed 5 Oct 2015.
Juncker JC. Mission Letter to V. P. Andriukaitis, Commissioner for Health and Food Safety. Brussels: European Commission. 2014; 01 November 2014. https://ec.europa.eu/commission/sites/cwt/files/commissioner_mission_letters/andriukaitis_en.pdf. Accessed 5 Oct 2015.
Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Health Economics, Policy and Management. New York: Cambridge University Press. 2009.
EXPH - EXpert Panel on effective ways of investing in Health. Definition and Endorsement of Criteria to Identify Priority Areas When Assessing the Performance of Health Systems. Brussels: European Commission. 2014. http://ec.europa.eu/health/expert_panel/sites/expertpanel/files/002_criteriaperformancehealthsystems_en.pdf. Accessed 9 Jan 2017.
Hakkinen U, et al. Towards explaining international differences in health care performance: results of the EuroHOPE project. Health Econ. 2015;24 Suppl 2:1–4.
Bernal-Delgado E, et al. ECHO: health care performance assessment in several European health systems. Eur J Public Health. 2015;25 suppl 1:3–7.
Health Consumer Powerhouse. Euro Health Consumer Index 2015 Report. Stockholm: 2016. http://www.healthpowerhouse.com/files/EHCI_2015/EHCI_2015_report.pdf. Accessed 2 May 2016.
Cylus J, Nolte E, Figueras J, McKee M. What, if anything, does the EuroHealth consumer index actually tell us? the BMJ blog. 2016. http://blogs.bmj.com/bmj/2016/02/09/what-if-anything-does-the-eurohealth-consumer-index-actually-tell-us/. Accessed 2 Feb 2016.
Bogaert P, Robine JM, Van Oyen H. Working towards an EU health information system, Adjacent Governmen. 2016.
EuroREACH. Health Data Navigator. 2013. http://www.healthdatanavigator.eu/international-home/eu-and-international-projects. Accessed 07 Jan 2016.
Haskell AC, Breaznell JG. Graphic charts in business: How to make and use them. New York: Codex Book Company, Incorporated. 1922.
Pugh DS, Pugh D. Organization theory: Selected readings, Penguin books Harmond-sworth. 1984.
OECD. OECD Health Statistics 2016. 2016.
Carinci F, et al. Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators. International J Qual Health Care. 2015;27(2):137–46.
Arah OA, Westert GP, Hurst J, Klazinga NS. A conceptual framework for the OECD health care quality indicators project. Int J Qual Health Care. 2006;18 suppl 1:5–13.
World Health Organization Regional Office for Europe. European health for all database (HFA-DB). 2015.
OECD. Health at a Glance 2015. OECD Publishing. 2015.
World Health Organization Regional Office for Europe. Core Health Indicators in the WHO European Region 2015 - Special focus: human resources for health. 2015. http://www.euro.who.int/__data/assets/pdf_file/0019/290440/Core-Health-Indicators-European-2015-human-resources-health.pdf. Accessed 15 Nov 2015.
WHO Regional Office for Europe. European Health Report 2015. 2015. http://www.euro.who.int/__data/assets/pdf_file/0008/284750/EHR_High_EN_WEB.pdf?ua=1. Accessed 12 Oct 2016.
OECD. Health Publications. 2016. http://www.oecd.org/health/health-publications.htm. Accessed 20 Jun 2016.
Joumard I, André C, Nicq C. Health Care Systems: Efficiency and Institutions. Paris: OECD Publishing. 2010;769:133.doi:10.1787/18151973.
Hofmarcher MM, Oxley H, Rusticelli E. Improved health system performance through better care coordination. Paris: Organisation for Economic Co-operation and Development. 2007;DELSA/HEA/WD/HWP(2007)6.
OECD. Fiscal Sustainability of Health Systems. Paris: OECD Publishing. 2015;doi:10.1787/9789264233386-en.
OECD. Review of Health Statistics: Discussion of possible future strategic directions to improve health system performance assessment. Directorate for Employment Labour and Social Affairs-Health Committee, editor in OECD Conference Centre. Paris. 2014: DELSA/HEA(2014)17.
OECD, Eurostat, WHO. A System of Health Accounts. 2011. doi:10.1787/9789264116016-en.
European Commission. European Core Health Indicators 2016. http://ec.europa.eu/health/indicators/echi/list/index_en.htm. Accessed 10 Mar 2016.
European Observatory on Health Systems and Policies. European Observatory on Health Systems and Policies. 2016. http://www.euro.who.int/en/about-us/partners/observatory. Accessed 12 Oct 2015.
World Health Organization Regional Office for Europe. European Health Information Initiative (EHII). 2016. http://www.euro.who.int/en/data-and-evidence/european-health-information-initiative-ehii. Accessed 20 Jun 2016.
European Commission. State of Health in the EU. 2016. http://ec.europa.eu/health/state/summary/index_en.htm. Accessed 21 Jun 2016.
European Commission. Expert Group on Health Systems Performance Assessment (HSPA). 2016. http://ec.europa.eu/health/systems_performance_assessment/policy/expert_group/index_en.htm. Accessed 27 Jun 2016.
European Commission - Expert Group on Health Systems Performance Assessment. So What? Strategies across Europe to assess quality of care. Luxembourg: P.O.o.t.E. Union. 2016;doi:10.2875/03667.
EC Expert group HSPA. Minutes of 6th meeting - Expert Group on Health Systems Performance Assessment (HSPA), 07 April 2016, Rome. 2016. http://ec.europa.eu/health/systems_performance_assessment/docs/ev_20160407_mi_en.pdf. Accessed 27 Jun 2016.
Working Party on Public Health at Senior Level (WPPHSL). Sub-group 5 report “possible criteria for selecting prioritized areas for comparisons and assessment. 2012. http://ec.europa.eu/health/expert_panel/sites/expertpanel/files/criteriaperformancehealthsystems_report02_en.pdf. Accessed 9 Jan 2017.
EXPH - EXpert Panel on effective ways of investing in Health. Access to health services in the European Union. Brussels: European Commission. 2016. http://ec.europa.eu/health/expert_panel/sites/expertpanel/files/015_access_healthservices_en.pdf. Accessed 9 Jan 2017.
European Commission. Expert Group on Health Information (EGHI). 2016. http://ec.europa.eu/health/strategy/implementation/hic/index_en.htm. Accessed 15 Sept 2015.
Van Oyen H, Bogaert P. Bridge Health: Concept Paper - Technical Report BRIDGE Health Brussels: BRIDGE Health project. 2016;N° WP1_2016_03.
EC - Expert Group on Health Information (EGHI). A potential ERIC on Health - Scoping paper for considerations of EU Member States and EEA/EFTA countries in the Expert Group on Health Information. 2014. http://ec.europa.eu/health/strategy/docs/hi_eric_scopingpaper_en.pdf. Accessed 15 Sept 2015.
ESIF European Structural and Investment Funds. Technical Toolkit: Set of indicators useful for the final evaluation of actions, developed under the project “Provision of support for the effective use of European Structural and Investment (ESI) Funds for health investments. 2015. http://www.esifforhealth.eu/pdf/WP3%283%29_Indicators_FINAL_20150211.pdf. Accessed 5 Dec 2015.
EuroREACH. EuroREACH project - Improved access to health care data through cross-country comparisons. 2013. http://www.euroreach.net. Accessed 7 Jan 2016.
ECHO. European Collaboration for Healthcare Optimization consortium. Handbook on methodology. Zaragoza (Spain): Instituto Aragonés de Ciencias de la Salud- Instituto Investigación Sanitaria Aragón. 2014. http://www.echo-health.eu/handbook/CV_CID.html. Accessed 1 Feb 2016.
European Commission. JRC in brief. 2016. https://ec.europa.eu/jrc/en/about/jrc-in-brief. Accessed 30 Sept 2016.
European Commission. European Research Infrastructure Consortia (ERIC). 2016. https://ec.europa.eu/research/infrastructures/index_en.cfm?pg=eric. Accessed 30 Sept 2016.
European Social Survey. About the European Social Survey European Research Infrastructure – ESS ERIC. 2016. http://www.europeansocialsurvey.org/about/. Accessed 30 Sept 2016.
SHARE. SHARE - Survey of Health, Ageing and Retirement in Europe. 2016. http://www.share-project.org/. Accessed 30 Sept 2016.
BBMRI-ERIC. BBMRI, the pan-European Biobanking and Biomolecular Resources Research Infrastructure. 2016. http://www.bbmri-eric.eu/. Accessed 30 Sept 2016.
European Commission. Europe 2020 - European Semester Thematic Fiche: Health and Health Systems. Brussels: 2015. http://ec.europa.eu/europe2020/pdf/themes/2015/health-and-health-systems.pdf. Accessed 05 Dec 2016.
Medeiros J, Schwierz C. Efficiency estimates of health care systems - European Commission. Economic Papers 549. Brussels: European Economy. 2015. http://ec.europa.eu/economy_finance/publications/economic_paper/2015/ecp549_en.htm. Accessed 20 Nov 2015.
Eckefeldt P, Schwierz C, Giamboni L, Aarnout M, Carone G. Identifying fiscal sustainability challenges in the areas of pension, health care and long-term care policies - European Commission. Occacional Papers 201. Brussels: European Economy. 2014. http://ec.europa.eu/economy_finance/publications/occasional_paper/2014/op201_en.htm. Accessed 20 Oct 2015.
European Commission, Economic Policy Committee (AWG). Joint Report on Health Systems. Occasional Papers 74. Brussels: 2010. http://ec.europa.eu/economy_finance/publications/occasional_paper/2010/pdf/ocp74_en.pdf. Accessed 15 Oct 2015.
Eurostat. Health Statistics. 2016.
Eurostat. Quality of life - Facts and views. Brussels: European Commission. 2015. http://ec.europa.eu/eurostat/documents/3217494/6856423/KS-05-14-073-EN-N/742aee45-4085-4dac-9e2e-9ed7e9501f23. Accessed 15 Nov 2015.
OECD. How’s Life? 2015. OECD Publishing. 2015;doi:10.1787/23089679.
Eurostat. Sustainable development indicators. 2015. http://ec.europa.eu/eurostat/web/sdi/indicators. Accessed 15 Sept 2015.
European Commission. Commission Staff Working Document on EU indicators in the field of youth. SEC(2011) 401 final. 2011. http://ec.europa.eu/youth/library/publications/indicator-dashboard_en.pdf. Accessed 02 Feb 2016.
European Commission. Youth monitor - European Commission. 2015. http://ec.europa.eu/youth/dashboard/index_en.htm. Accessed 02 Feb 2016.
European Commission Social Protection Committee Indicators Sub-group. 2014 Social Protection Performance Monitor (SPPM) dashboard results. 2015. http://ec.europa.eu/social/main.jsp?catId=738&langId=de&pubId=7823. Accessed 2 Feb 2016.
European Commission. Health in all policies. 2016. http://ec.europa.eu/health/health_policies/policy/index_en.htm. Accessed 17 Aug 2016.
Seychell M, Hackbart B. The EU health strategy – investing in health. Public Health Reviews. 2013;35(1):1–25.
We would like to thank the following members of the BRIDGE Health HIS Task Force for their support, without whose help this work would never have been possible:
Members of the BRIDGE Health System Indicators Task Force are:
• Peter, Achterberg; Rijksinstituut voor Volksgezondheid en Milieu (RIVM), Netherlands
• Reinhard, Busse; Technische Universität Berlin, Fachgebiet Management im Gesundheitswesen, Germany
• Angela, Fehr; Robert Koch Institute, Germany
• Sabrina, Hense; Robert Koch Institute, Germany
• Zeynep, Or; Institut de Recherche et Documentation en Economie de la Santé – IRDES, France
• Peter, C, Smith; Imperial College, UK
• Thomas, Ziese; Robert Koch Institute, Germany
We are grateful to Noemi Kiss for her valuable comments in reviewing the first drafts and Agata Łaszewska for.copy-editing the final version of this paper.
This research is part of the BRIDGE Health Project, which is funded by the Health Programme of the European Union (grant 664691).
Availability of data and materials
NP carried out the review, developed the map and prepared the first draft of the paper. MMH and JS initiated and supervised the research and conceptualised and revised the paper. All authors contributed substantially to the manuscript. All authors critically reviewed the manuscript and had final approval of the submitted version.
Nataša Perić (BSc, MSc, PhD candidate), Researcher, Department of Health Economics, Centre for Public Health, Medical University of Vienna, Austria.
Maria M. Hofmarcher-Holzhacker (MSc Economics, MPH), Director of HealthSystemIntelligence and Research Associate at the Department of Health Economics, Medical University Vienna, Austria.
Judit Simon (MD, BA, BSc, MSc, DPhil, FFPH), Professor of Health Economics and Head, Department of Health Economics, Centre for Public Health, Medical University of Vienna, Austria.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
About this article
- Health system performance assessment
- Health information
- Policy making
- International comparisons
- European Union
- Public health indicators