Skip to main content

Policy-makers’ views on translating burden of disease estimates in health policies: bridging the gap through data visualization

Abstract

Background

Knowledge Translation (KT) and data visualization play a vital role in the dissemination of data and information to improve healthcare systems. A better understanding of KT and its utility requires examining its processes, and how these interact with available data tools and platforms and various users.

In this context, the aim of this paper is to understand how relevant users interact with data visualization tools, in particular Global Burden of Disease (GBD) visualizations, while also examining KT processes related to data visualization.

Methods

A qualitative case-study consisting of semi-structured interviews with eight policy officers. Interviewees were selected by the Institute for Health Metrics and Evaluation (IHME) from three countries: Canada, Kenya and New Zealand. Data were analyzed through framework coding, using qualitative analysis software.

Results

Policy officers’ responses indicated that data can prompt action by engaging users, and effective delivery and translation of data was enhanced by data visualization tools. GBD was considered valuable for use in policy (e.g., planning and prioritizing health policy; facilitating accountability; and tracking and monitoring progress and trends over time and between countries). Using GBD and data visualization tools, participants quickly and easily accessed key information and turned it into actionable messages; engaging visuals captured decision-makers’ attention while providing information in a digestible, time-saving manner. However, participants emphasized an overall disconnect between research and public health. Functionality and technical issues, e.g., absence of tool guides and tool complexity, as well as lacking buy-in and awareness of certain tools from those less familiar with GBD, were named as major barriers. In order to address this “know-do” gap, user-friendly knowledge translation tools were stated as crucial, as was the importance of collaboration and leveraging different insights from data generators and users to inform health policy.

Conclusions

Policy officers aware of KT are willing to utilize data visualization tools as they value them as an engaging and important mechanism to foster the use of GBD data in policy-making. To further facilitate policy officers’ efforts to effectively use GBD data in policy and practice, further research is required into how users perceive and interact with data visualization and other KT tools.

Peer Review reports

Background

Timely, accurate and high-quality health information is one of the key inputs for public health action, health systems strengthening and progress towards the achievement of important health goals, such as universal health coverage and the health-related Sustainable Development Goals [1]. Effective public health action and implementation of useful health policy is integral to achieving improved health. A range of tools and mechanisms are used to promote the utilization of health information in policy development and implementation. They can be summarized under the name of knowledge translation (KT) approaches [2].

The World Health Organization (WHO) defines KT as: “the exchange, synthesis, and effective communication of reliable and relevant research results” [3]. The focus of KT is on promoting interaction among the producers and users of research, removing the barriers to research use, and tailoring information to different target audiences so that effective interventions are used more widely [4]. WHO identifies four key KT approaches that, either singularly or combined, illustrate the link between health information and action [5, 6]:

  • Push approaches, where information producers ensure research processes and findings are more digestible for decision-makers by effectively translating technical information and jargon into non-technical information;

  • Pull approaches, where information users, such as policymakers, request health information based on their needs;

  • Exchange approaches emphasizing linkages between information producers and users, which can be facilitated by knowledge brokers (KBs), to work together at specific points or during the entire information generation cycle; and finally,

  • Integrated approaches, where a KT platform is established in an organization or broader health system allowing the promotion of early and sustained engagements between information producers and users and institutionally linking research to action through push, pull or exchange efforts [6].

These approaches have been used by information producers, policymakers and others in the health sector in countries across the world, engaging many different types of stakeholders in diverse areas of policy-making [5].

In the context of KT, health information includes the summary and analysis of health status and problems in populations over time, as well as quantification of associations between outcomes and risks; it also assesses effectiveness of public health interventions [5]. Data consist of raw facts of a study or work, in qualitative or quantitative form; while evidence is defined as ‘findings from research and other knowledge that serves as useful basis for decision-making in public health and health care’ [5]. Explicit knowledge consists of these three: health information, data and evidence [5]. Health data are usually presented in different formats and with different degrees of aggregation, which can make understanding it challenging. This was one of the issues that has been identified as a barrier to participants’ use in decision-making processes.

Data visualization—understood as ‘the representation and presentation of data to facilitate understanding’ – is considered an important mechanism for increasing the usefulness of health information [7, 8]. Data visualization can be considered a ‘push’ approach to KT [9] as it seeks to present data in more accessible ways, such as graphs, charts, and maps with interactive features [10], but it also facilitates interaction between multisectoral stakeholders from different areas by connecting them in the production of such tools, resulting in mutually beneficial collaboration [11]. Given this, data visualization tools can play an important role in KT processes.

The usefulness of data visualizations has been investigated and critically assessed in computer science for some time [12], but so far there is limited understanding of how well existing visualization tools help users to master the wealth of information represented by complex data. This is particularly relevant for health sector and health policy systems decision-making. Providing a base for further assessments, this paper explores how a number of professional experts in a few countries analyze and use Global Burden of Disease (GBD) data using data visualization platforms.

This study aims to contribute to a more robust understanding of the use of data visualization to support KT processes. It uses a case-study of policy officers’ engagement with one data visualization platform, GBD Compare. GBD Compare is a tool developed as part of the GBD study, led by the Institute for Health Metrics and Evaluation (IHME) in collaboration with thousands of researchers worldwide. GBD quantifies global health loss from diseases, injuries, and risk factors across age groups, sexes, countries, regions, and time, with estimates currently produced for 195 countries, by age and sex, from 1990 to the present. GBD Compare is primarily a push approach aimed at making research processes and findings more digestible for decision-makers.

The GBD study provides a rich context for exploring processes of knowledge translation, due both to the vast scope of the data available and its explicit orientation towards helping policymakers understand their countries’ health challenges. In order to support decision-making, GBD Compare has been developed as a KT mechanism for this audience, with the aim of “allow [ing] decision-makers to compare the effects of different diseases, such as malaria versus cancer, and then use that information at home” [13]. The GBD estimates are intended to be used for policy and so KT is an integral part of this effort; if people are not able to access, interact with, and understand the estimates, it is difficult to incorporate them into decision-making. GBD Compare is intended to help users explore patterns and trends, make comparisons across and between different axes such as country, demographic groups, and diseases, and create and capture relevant graphics such as maps, graphs, and tables. Given its range and intended orientation towards supporting policymaking, GBD Compare provides a useful case study for exploring the following questions:

  1. 1.

    How do policy officers perceive, interpret and utilize web-based data visualization tools?

  2. 2.

    How can these tools and platforms be optimized as catalysts for KT processes?

Methods

Study design, setting and methods

This was a qualitative case-study using semi-structured interviews with policy officers (see Additional file 1 for the interview guide). A case-study design was selected due to the descriptive and exploratory nature of the inquiry—how knowledge translation tools are perceived, interpreted, and used by policy officers—and because understanding the specific context in which these processes are occurring is important to answering the primary research questions [14]. Given this study design, rather than seeking to draw general inference about larger populations or other contexts, this work seeks instead to examine and explicate processes related to the use of data visualization for knowledge translation, which may have relevance, explanatory value, or provide transferrable lessons for other settings.

Semi-structured interviews allowed data collection around themes pertinent to the research topic while also giving respondents the space to discuss issues or areas that they deemed important [15]. The purposive selection was facilitated by IHME using a predetermined list of potential respondents with expertise and characteristics relevant to the study (policy officers who advise on public policy in government settings in a technical capacity; individuals who have prior familiarity with the GBD and its data visualization platform, GBD Compare; and English-speaking). The final sample size was determined by reaching saturation [16].

Study sample

As described above, the study sample consisted of respondents who were a) policy officers and b) familiar with the GBD tool. In terms of their involvement in policymaking, we define policy officers as people who influence policy processes in order to improve and shape implementation strategies. These advisors often act as KBs between the academic world and policymakers by providing data and evidence to governments. Working as developers and promoters of new government policy, these specialists have technical expertise in a specific field and advise colleagues (e.g., senior bureaucrats and government ministers) on policy options. Among other functions, they help evaluate and monitor policies, create dialogues, and strategize.

A total of eight persons, out of 10 invitees, agreed to participate in a semi-structured interview. Seven interviews were conducted over a period of 3 months (with one done as a joint interview with two participants). Of the seven interviews, five were Skype audio, one Skype video, and one Zoom video call. Interviews ranged from 25 min to 64 min long, although a majority were about 50 min in duration. The study participants included an epidemiologist, a research scientist and research associate, a UHC program coordinator, a Monitoring and Evaluation specialist, an NCD Division Lead, a principal advisor in epidemiology and principal policy analyst in strategy and policy. The average interview duration was 50 min. Seven interviewees were female and one male with a wide range of duration of work experience in the field of health policy and/or knowledge translation work (range 3 to 15 years). All participants were highly educated individuals holding senior positions, having either master’s degrees (n=7), or PhD (n=1), in public health and medical sciences. Three currently work in healthy policy and were previously clinicians before transitioning into policy-focused roles.

To explore potential contextual dynamics, respondents with these characteristics were also selected from different countries with varied data landscapes and policy contexts. In this case, one Low- and Middle-Income Country (LMIC) (Kenya) and two high-income countries (HICs) (Canada and New Zealand). Each participant was an individual with expertise in a particular discipline and employed by a government agency working in different ways to contribute to implementing health policy. The interviewer was located in Stockholm, Sweden, while interviewees were located in their respective countries’ locations.

Data management and framework analysis

All interviews were recorded after receiving participants’ consent. Interviews were then transcribed and uploaded to qualitative analysis platform, Dedoose, for data analysis [17]. Data analysis began using open coding as a process to ‘open up’ the text in order to discern ideas and meanings [18]. This consisted of close readings of interview transcripts and consideration of the multiple meanings found within them [19], generating and applying codes, and then looking across codes to discern key themes [20]. These themes were then grouped by adapting Lavis et al.’s framework for assessing country-level efforts to link research to action [6]. Their framework consists of four elements: general climate for research use, production of relevant and synthesized research, mix of activities linking research to action and the evaluation of efforts to link research to action [6]. This framework explicates the various directions, dynamics and inter-relationships of those involved in KT and data use. Of note, while Lavis et al’s framework was used, another theme, data use, was added to encapsulate all results.

Results

Themes

A total of 6 themes and 16 subthemes emerged from the data (see Table 1). Participants identified different advantages and weaknesses of data visualization tools for KT, as well as discussing various aspects of knowledge translation and data visualization use. Specifically, the primary themes were: production of research, exchange efforts, efforts to facilitate user pull, push efforts, general climate, and data use (see Fig. 1):

Table 1 Components of the data visualization
Fig. 1
figure 1

Themes

Exchange efforts – build relationships among researchers and research users who have shared interests

Participants from both LMIC and HICs highlighted the need to collaborate and consolidate efforts between data generators and users to produce data, foster data use and inform health policy. Participants emphasized the importance of collaboration in utilizing and incorporating different insights, perspectives, and experiences. They described the interlinkages and exchange between all involved parties as crucial in informing health policy: participants highly valued conferences, trainings or other events that brought people together so they could meet, discuss and exchange ideas and information with one another. Given the explicit intent of GBD data to be used as a decision-making tool and to inform health policy, respondents noted the imperative of a collaborative approaches to using tools, engaging in dialogue with colleagues rather than simply extracting and presenting data to them. As one respondent, a policy analyst in a high-income setting, explained about using GBD Compare with a colleague: “Both of us come from different perspectives … so I’m mainly using the visualization tools and also using the methodology behind it to create epidemiological insight … and then provide it and discuss it with [colleague], so that he can look at it from the policy perspective to see what has been done in the field before.” Other respondents noted the importance of constant communication between different loci of decision-making when engaging with health data like that from the GBD, which has implications for many sectors; for example, a participant from an LMIC highlighted the need to make connections between different areas, particularly between and amongst various Ministries, such as Transport and Finance, when thinking about audiences for the information provided in the tools. Similarly, a research scientist and an epidemiologist from HIC stressed how delivery of data needs to be done thoughtfully and, importantly, to the right people, with one explaining: “you can’t just show them the evidence like here’s the reality … it’s really a process of change and so it’s figuring out who your stakeholder, where your stakeholders or your decisionmakers are right now”. Applying the tools therefore helps in the process of fostering data-driven decision-making, but only if relevant entities and actors are identified and brought in.

With reference to collaboration, participants noted a key challenge that remains in terms of a broader disconnect between research and public health. Respondents noted that without effective communication and collaboration between relevant stakeholders, health information can remain under or unused. An epidemiologist, from LMIC, put it succinctly: “people don’t talk to each other, academicians and then the ward and the public health professionals and so there is a gap there that’s a kind of, the bridge, the connection is not there”. While the GBD study seeks to use a collaborative process in the production of the data that ends up in GBD Compare, this finding suggests that participants remain acutely aware of the underlying challenges of collaboration and cross-stakeholder communication remain when it comes facilitating effective KT processes.

Facilitating user pull – technical considerations

Participants from LMIC countries noted certain challenges in using the tool, GBD Compare. In particular they noted the complexity and intricacies of the tool, its scope and interface requiring regular use in order to understand and maximize utilization, and the way data is presented in graphics sometimes proving difficult to interpret. As one participant noted, “if I’m seated at my desk and I have no training and I want this information, I may not be able to make sense of that information”. A research scientist from HIC observed some did not utilize the tools if they did not feel as confident using them or as familiar with how to use them. More guidance and training were highlighted as necessary for increasing utilization of the tool, as without clear guidance, insufficient user understanding of GBD Compare can be quite limiting. In addition, participants from all three countries suggested several improvements that could be made to tool functionality, ranging from technical issues, such as ensuring compatibility with certain browsers, to the way in which information is displayed.

Facilitating user pull – research production

While respondents noted challenges in using the data visualizations, they all identified deeper structural issues at the local and country-level hindering GBD uptake by various policy officers and country leaders, such as lacking buy-in, or awareness, of GBD. Those from LMIC settings observed a need to increase engagement around GBD among policy and other audiences in country, suggesting that buy-in would increase once the benefits of GBD could be seen and the methodology was better understood. Participants experienced challenges garnering interest in using the tools, and commented on a certain skepticism and questioning of GBD data and its quality, particularly in the context of existing and competing information sources. One participant from LMIC explained that “the country has invested a lot in our routine data system that anytime you could bring this issue and not and mention that the routine data has not been used, it could immediately face a lot of resistance.” Another noted, “sometimes people feel that evidence that is not generated in-country is not good enough.” The same concerns arose from those in high-income settings. For instance, skepticism surrounding the estimates and how they were calculated raised some doubts. A research scientist in HIC noted these concerns: “one barrier we have with using it internally is sometimes our more senior managers are concerned that because these are modeled estimates, they’re not going to match what comes out of our national systems”.

Moreover, while participants highly valued and appreciated transparency of data sources in the GBD Compare tool and found that they themselves and their colleagues were more trusting of data if there was clarity about its source, several users discussed concerns about discrepancies found within GBD data, which is re-estimated each cycle due to new methods and underlying data sources: “the little issue that at times we have is the fact that each and every time every year when the new GBD results are released, you may find varying … indicators”. Confusion regarding modeling methodology and estimates led to distrust among users and were therefore less willing and likely to use the data. Despite these concerns, participants had hope that visualizations could help gain more trust once GBD methods were shared and explained.

In one HIC, the presence of a GBD oversight team was found to be of immense help. The oversight team consisted of deputy directors-general who provide technical and/or policy assistance to both policymakers and researchers on GBD Compare. This led to more, maximized tool use, and its value was emphasized highly by participants who worked with this team. However, more guidance from IHME was requested around explanations for global patterns and recommendations for potential policies that could be implemented. Participants hoped that examples of existing, successful policy interventions for certain diseases, e.g., increasing obesity rates, could be shared by IHME.

Push efforts – identify actionable messages

Data visualization was used to better understand data by quickly, uniquely and clearly conveying information. Visualizations were noted as a valuable method to prompt action and support policy recommendations to stakeholders by grabbing their attention and providing illustrative data. As one policy analyst from a HIC shared: “decisions are not rational things, are they? They’re about hearts and about minds and so visualizations are something that helps you to bridge the heart and the mind a little bit … images can stay with you in a way that words on a page won’t, so I think there’s something about the translation that’s hard to encapsulate but it helps to tell the story.”

Policy officers indicated that data can prompt action by engaging users, if compiled and presented in a succinct, colorful way with the help of data visualizations and related tools. Instead of handing users a bland 60-page report, policy officers could present the data in a more productive manner. One respondent from an HIC described the power of data visualizations, in particular the GBD tree map visualization: “that one is really useful for, you know, showing the overall burden and then making a comparison to resources allocated. Whenever I present that slide, I get silence in the room for 30 seconds and then discussion”.

Policy officers also noted that being able to present data on risks factors and their contribution to disease was useful for capturing audience attention, challenging preconceptions of the current situation and therefore spurring appropriate, effective action. Making the evidence more engaging was perceived as critical to incite and inspire change in policymaking, with one HIC policy analyst noting that “it’s quite a powerful way of describing the interplay of risk factors and poor outcomes for health, and just yesterday we were using this analysis for some of our senior leaders in our section to look at health loss and the factors that contribute to it. So, I’d say the visualization tools are very important part of bringing the data to life.”

Additionally, user-friendliness emerged as an important aspect of data visualization and its powerful, unique ability to convey information. Participants from LMIC discussed “scorecard” approaches to presenting information on GBD Compare as attention-drawing tools: “it’s color coded so you have like red, orange and green depending on the progress that has been made … it can be printed, but most of the time, you can even get it on your mobile phone”. In the words of another participant, at the click of a button you can have information and indicators for an entire country. Top management, or other audiences who need to know the important facts, do not need to spend a lot of time trying to access this kind of data: “they just need it at a glance, they are so busy”.

While, as discussed earlier, existing information systems could be seen as a barrier to buy-in for the data found in GBD Compare, it is also the case that participants have been able to show the utility of GBD and its visualization tools as a useful complementary resource in conjunction with existing local systems, and using the tools to identify and fill gaps in local information or data proved very valuable. For instance, a research scientist used GBD data in addition to their existing platforms. They do not have a system that comprehensively covers both morbidity and mortality for all causes. In LMIC, a participant described using GBD for subnational estimates: “for example life expectancy, or top five causes for mortality … we could easily sell that even to the leaders at the subnational level”. The GBD estimates are therefore used as a way to compare in-country data to give a better overall grasp of the situation, used in tandem with other systems.

General climate – country support and approach (es) towards KT

According to participants, in order to optimize KT, sufficient resources, awareness and time all need to be devoted to the process of KT and policymaking, and a lack of awareness and engagement with GBD and/or data visualization were identified as issues to be addressed addressed. These issues were particularly seen in the LMIC setting, where country leaders were resistant to using outside data and more skeptical of GBD estimates. Leadership attitudes set the tone for the rest of the country and trickled down into departments or divisions. In general, respondents noted that fewer resources were devoted to KT and data collection and visualization in LMIC countries, highlight the need for “more money, more funds, more capacity … especially in my field …. there’s a lot of gaps in knowledge”. In high-income contexts, however, KT was much more supported. For instance, using KT was even enforced as a part of the research and funding process in these contexts. A research scientist in HIC explained how KT is an integral part of acquiring funding and how a KT plan is mandatory and evaluated as part of the funding application process.

Data use

GBD data was used to help identify needs and prioritize resources accordingly. Participants described how GBD helps inform the future, for instance by giving direction to the country’s goals. One respondents explained that “it shows you how [things have] changed and you can change over the years, to see how both the risk and the causes of diseases are changing … you can see where you’re making progress and where we are not making progress and how, if that’s a neglected area, we should focus on it.” Policy officers found that GBD results, as presented in GBD Compare, help to discern patterns, and could therefore be used for not only monitoring, but also planning for time, funding and resource allocation:

“We have the indicators which we use to monitor the progress of the project, so each and every time we have to keep looking at the data, see how the data is behaving, look at trends over time, maybe over the months, over the years, and see whether, make decisions based on that and see if we are on track you know if we need to do, need to change our tactic for better results.”

A research scientist in HIC further explained commented on their utility for planning, explaining how in “planning, developing our five-year plans, we have to actually look back and really bringing all that still, the charts and you know, the visualizations to show the current situation—it becomes like building your case,” though also emphasized that other aspects of decision-making need to be considered, as “there are many factors that influence what an organization is going to focus on and evidence is just one thing.”

Another key aspect of GBD Compare highlighted by respondents was how the data facilitated comparison among countries, which illuminated any gaps and may trigger processes of policy changes. While GBD data in itself does not indicate what the policy responses should or would be, it helps to identify gaps, understand areas where disease burden is high or significantly increasing, particularly in this directly comparative context.

Data was also used to increase accountability and transparency. In this case, participants noted one other tool developed by IHME, Financing Global Health, a data visualization focused on tracking and monitoring government spending on health. One respondent noted that this tool was helpful in conjunction with GBD Compare:

“You can say the US government spends this much on malaria … I think that’s really good because many times even for us, being right now we are lower middle-income country but when we were low-income country, there was a lot of investment by partners, you know donors … you could have this notion that they spend this amount but that amount doesn’t really get to the ground.”

Discussion

Our findings indicate that while there were variations in the perception, interpretation and utilization of a data visualization tools by policy officers, all study participants found these tools to be extremely useful.

Data visualizations are instrumental to quickly and clearly conveying information and thus contribute to increasing the use of burden of disease data in policy. According to our findings, the process of KT requires sufficient resources, awareness, time and sense of ownership. In some cases, participants perceived unwillingness to use GBD data among their colleagues and the wider stakeholder community. Knowledge producers, such as IHME, can make more concerted efforts to increase buy-in and familiarity with various data platforms, perhaps by advertising their objectives and their data more in the health information realm, and by providing more training for current and prospective users.

A substantial body of literature confirms that KT processes build important bridges between evidence and practice, inter alia through co-creation process and the involvement of all relevant actors, from differing backgrounds [21]. These results build on existing evidence of the literature on data visualization and KT, such as assertions that data visualization’s interactive nature allows decision-makers to explore data from various perspectives [22]. Scholars have highlighted how visualizations can help users (e.g. policy officers presenting data and information) explore, analyze and communicate results from complex big data in health [22], and can be used to convey a larger story or narrative in a stimulating and more effective way than text alone [23].

However, there is a scarcity of research on using information technology as a means for knowledge dissemination in health policy-making [24]. A systematic review by Delnord et al. reports that only one previous study proposed a framework for monitoring the impact of national health information systems in regards to KT [25]. During the study process, we observed that inaccessibility of the evidence hampered evidence-informed policy; evidence will be quickly discarded by policy-makers if they struggle to access it online or if the way in which is presented is difficult to understand [26]. However, data visualizations tools represent just one example of strategies for facilitating the use of explicit knowledge (health information, data, evidence) in policy. Others include synthesis tools, such as policy briefs, which summarize and present information in a more accessible ways, as well as knowledge networks (e.g., regular meetings or advisory groups/committees) that bring stakeholders together to exchange ideas [9].

Actors’ use of available interactive tools and the creation of exchanges and exchange between data producers and users affect the extent to which health information is applied in the policy process [27]. Collaboration between multisectoral stakeholders such as evidence producers and users, funders and actors in healthcare with varying perspectives increases mutual understanding and trust which facilitates KT [26, 28], which is echoed by participants’ emphasis on the importance of collaboration and communication.

The three countries in this study provided contrasting contexts in regards to both GBD uptake and health information systems, in terms of resources, data quality, transparency as well as awareness and support for KT. This is particularly relevant for public health in LMICs, where local-level evidence might be ignored or inadequate hardware and data quality prevail [29]. In Kenya, a majority of health facilities are still operated by faith-based organizations and non-governmental organizations, data remains scarce and KT is not prioritized [30]. In contrast, policy-makers in HICs have a wealth of data, resources and they integrate KT and GBD data in their policy-making process [21, 22]. While some participants noted attributing high importance to KT and use of data visualization, they also encountered hindrances such as lacking resources, particularly in LMICs. Dedicated resources (e.g. staff, time and money) to support KT efforts are needed [31].

In order to take appropriate actions, health policy-makers require various kinds of information about health system performance and public health problems and needs [24]. Health information and data helps to identify problems, as well as aids monitoring and evaluation by measuring the magnitude of a disease and assessing progress in addressing these health problems [5]. However, data is only part of the evidence that is needed for effective evidence-informed policy-making. For the remainder of the policy cycle, other kinds of evidence and health information are needed, in particular primary and secondary research [5]. The Evidence-informed Policy Network (EVIPNet), of WHO, is looking at comprehensively addressing the evidence needs of policy-makers at all stages of the policy cycle while synthesizing the best available evidence in a user-friendly manner [5].

Methodological considerations

This was a qualitative case study that generated an in-depth, multi-faceted understanding of data visualization in real-life contexts across three countries. The case study provides valuable input from selected experts to help design further research; specifically, the usability and relevance of sophisticated visualization tools in the analysis of complex data and information like GBD. However, such an approach comes with its own methodological limitations, including selecting study participants purposively (e.g., pre-selected by IHME based on participant use of GBD Compare); also, findings may not be generalized to other contexts [32].

The research questions were exploratory in nature, seeking specifically to understand and draw out inferences from respondent perspectives and experiences. An inductive approach in this case is optimal, where data was gathered to build concepts, hypotheses, and theories, rather than deductively deriving hypotheses to be tested [33]. Semi-structured interviews allowed for participants to diverge and proved useful as relevant topics could be delved into and discussed in detail. Also, participants came from different backgrounds, professions, and countries. Interviewing policy officers from three countries with contrasting contexts regarding GBD uptake, and healthcare systems generally, yielded diverse responses and data while also giving the study a richer, more comparative perspective.

Of note, interviews and data collection were not done in situ in each country where participants were doing their work. A better grasp of the local system and context would have provided more background and better understanding of the participants’ responses and emerging themes from their insights. Additional limitations include self-reporting and potential biases, the main bias being the low number of interviews, as well as the fact that interviews may take different courses, so it is difficult to compare responses.

Conclusions

This study provides insight into the complexities of utilization and interpretation of data visualization tools. For instance, policy officers familiar with the concept of KT are more willing to utilize data visualization tools as they view them as engaging and important implements for KT and policy-making process. While future research can build on these observations to examine how users perceive and interact with data visualization and related tools, and their implications for KT, a suggestion emerging from this work is for organizations such as IHME and WHO to consider how certain KT mechanisms can be strengthened and thus make data more digestible, usable, and informative for health policy-making. Focus should be placed on finding and developing solutions that facilitate efforts in bridging the gap between evidence and policy; specifically, understanding how health metrics are used and interpreted and developing a KT framework incorporating these can help identify and overcome challenges. The results of this study also demonstrate the importance of actors across the health information system coming together to work with one another in processes of knowledge translation. Thus, adopting a system-wide approach when translating health information into policy could be useful to strengthen and enhance a collaborative research processes and encourage systematic, transparent use of GBD in the policy-making process.

Availability of data and materials

Additional data can be obtained upon request.

Abbreviations

EVIPNet:

Evidence-informed Policy Network

GBD:

Global Burden of Disease

HIC:

High-Income Countries

IHME:

Institute for Health Metrics and Evaluation

KB:

Knowledge Broker

KT:

Knowledge Translation

LMIC:

Low- and Middle-Income Countries

WHO:

World Health Organization

References

  1. Scarlett J, Köhler K, Reinap M, Ciobanu A, Tirdea M, Koikov V, et al. Evidence-informed Policy Network (EVIPNet) Europe: success stories in knowledge translation. Public Heal Panor. 2018;04(02):161–9.

    Google Scholar 

  2. World Health Organization. EVIPNet Europe strategic plan 2013–17 [Internet]. 2015. Available from: www.euro.who.int

    Google Scholar 

  3. World report on knowledge for better health: strengthening health systems. Vol. 83. Geneva; 2005.

  4. Oliver K, Cairney P. The dos and don’ts of influencing policy: a systematic review of advice to academics [Internet]. Vol. 5, Palgrave Communications. Springer US; 2019. p. 1–11. Available from: https://doi.org/10.1057/s41599-019-0232-y.

  5. EVIPNet Europe. Conceptual background and case studies: introduction to EVIPNet Europe [Internet]. Copenhagen: EVIPNet Europe; 2017. Available from: https://www.euro.who.int/__data/assets/pdf_file/0008/344762/EVIPNet-Europe-Starter-Kit_complete_ENG.pdf

    Google Scholar 

  6. Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84:620–8.

    Article  Google Scholar 

  7. Robinson I. Data visualisation: contributions to evidence-based decision-making. SciDevNet/Centre Agric Biosci Int. 2016.

  8. Rosling H, Zhang Z. Health advocacy with Gapminder animated statistics. J Epidemiol Glob Health. 2011;1(1):11–4. https://doi.org/10.1016/j.jegh.2011.07.001.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Blessing V, Davé A, Varnai P. Evidence on mechanisms and tools for use of health information for decision-making. Evid Mech tools use Heal Inf Decis [Internet]. 2017. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29300440.

  10. Allen WL. Visual brokerage: Communicating data and research through visualisation. Public Underst Sci. 2018;27(8):906–22.

    Article  Google Scholar 

  11. Arctic Institute of Community-Based Research. Knowledge translation overview; 2008. p. 1–3.

    Google Scholar 

  12. Mirel B. Visualizations for data exploration and analysis: a critical review of usability research. Tech Commun. 1998;45(4):491–509.

    Google Scholar 

  13. Institute for Health Metrics and Evaluation. About GBD [Internet]. Seattle: University of Washington; 2019. [cited 2019 May 2]. Available from: http://www.healthdata.org/gbd/about

    Google Scholar 

  14. Yin R. Applications of case study research. 3rd ed. Thousand Oaks: SAGE Publications, Inc.; 2012.

    Google Scholar 

  15. Willis K. In: Desai V, Potter R, editors. Doing development research. London: SAGE Publications, Inc.; 2006. p. 144–52.

    Chapter  Google Scholar 

  16. Ritchie J, Lewis J. Qualitative research practice: a guide for social science students and researchers. First. London: SAGE Publications Ltd; 2003.

    Google Scholar 

  17. Dedoose [Internet]. Los Angeles: SocioCultural Research Consultants, LLC; 2018. Available from: www.dedoose.com.

  18. Benaquisto L, Given LM. Open coding. In: The SAGE encyclopedia of qualitative research methods. Thousand Oaks: SAGE Publications, Inc.; 2019.

    Google Scholar 

  19. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006;27(2):237–46.

    Article  Google Scholar 

  20. Simmons N. Axial Coding. The SAGE Encyclopedia of Communication Research Methods. Thousand Oaks, CA: SAGE Publications, Inc.; 2019. p. 80–2.

  21. Davis D, Evans M, Jadad A, Perrier L, Rath D, Ryan D, et al. Learning in practice: the case for knowledge translation: shortening the journey from evidence to effect. Br Med J. 2003;327(July):33–5.

    Article  Google Scholar 

  22. Chishtie JA, Babineau J, Bielska IA, Cepoiu-Martin M, Irvine M, Koval A, et al. Visual analytic tools and techniques in population health and health services research: protocol for a scoping review. JMIR Res Protoc. 2019;8(10):e14019.

    Article  Google Scholar 

  23. Otten JJ, Cheng K, Drewnowski A. Infographics and public policy: Using data visualization to convey complex information. Health Aff. 2015;34(11):1901–7.

    Article  Google Scholar 

  24. Zakkar M, Sedig K. Interactive visualization of public health indicators to support policymaking: An exploratory study. Online J Public Health Inform [Internet]. 2017;9(2). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5630277/pdf/ojphi-09-e190.pdf.

  25. Delnord M, Tille F, Abboud LA, Ivankovic D, Van Oyen H. How can we monitor the impact of national health information systems? Results from a scoping review. Eur J Public Health. 2020;30(4):648–59.

  26. Donnelly CA. Four principles for synthesizing evidence. Nature. 2018;558:361–4.

    Article  CAS  Google Scholar 

  27. Putland AD. The respective roles of technical and generalist policy specialists in the policy process [Internet]. Southern Cross University; 2013. Available from: https://researchportal.scu.edu.au/discovery/delivery?vid=61SCU_INST:ResearchRepository&repId=1267219220002368#1367476090002368.

  28. What is knowledge translation? [Internet]. University of Waterloo. [cited 2019 May 2]. Available from: https://uwaterloo.ca/get-facts-knowledge-translation/knowledge-translation.

  29. Malla C, Aylward P, Ward P. Knowledge translation for public health in low- and middle- income countries : a critical interpretive synthesis. Glob Heal Res Policy. 2018;3(29):1–12.

    Google Scholar 

  30. Republic of Kenya Ministry of Health. Health sector human resources strategy 2014–2018. 2014;

    Google Scholar 

  31. Health Policy Project. Capacity development resource guide: knowledge translation. 2014.

    Google Scholar 

  32. Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(100).

  33. Merriam SB. Introduction to Qualitative Research. In: Qualitative research in practice: examples for discussion and analysis. First. San Francisco: Wiley; 2002:3–16.

Download references

Acknowledgements

The authors would like to thank the interviewees who participated in this study and Maja Pašović, Engagement Officer at IHME, for her review of the manuscript and comments.

Disclaimer

The authors do not report any competing interests. The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

ZEK, NK and AL conceived, planned and executed the study. AL analyzed the data and drafted the initial version of this paper that was revised by several iterations among all authors. TK, NK and ZEK provided the framework for analysis and contributed to the interpretation of the results. TP revised the manuscript for knowledge translation-specific considerations. All authors contributed to, reviewed and approved the final manuscript. The writing of the manuscript was co-led by TK and ZEK.

Corresponding author

Correspondence to Tanja Kuchenmüller.

Ethics declarations

Ethics approval and consent to participate

This study was part of a quality assurance work for the data visualization tool provided by IHME. No patient information nor sensitive personal data was used. Participants were fully informed of the study details and each participant did consent prior the participation in the study. There were no interventions affecting the participants autonomy and they had the right to refuse participation and to withdraw from the study at any time. The collected data was stored in a password-protected location. Audio files were kept private and deleted after transcribing was completed. No conflicts of interest exist as no funding were provided by IHME or other entities for this study. NK and KLK were full-time IHME staff during the time of the interviews; ZEK is a GBD collaborator.

Consent for publication

Not applicable.

Competing interests

KLK and NK are a staff member of IHME; ZEK is a GBD collaborator (non-paid).

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lundkvist, A., El-Khatib, Z., Kalra, N. et al. Policy-makers’ views on translating burden of disease estimates in health policies: bridging the gap through data visualization. Arch Public Health 79, 17 (2021). https://doi.org/10.1186/s13690-021-00537-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13690-021-00537-z

Keywords