How can the results of a qualitative process evaluation be applied in management, improvement and modification of a preventive community trial? The IHHP Study

Abstract

Background

This study reports the results of the qualitative process evaluation (PE) of the Isfahan Healthy Heart Program (IHHP), an integrated community-based trial for prevention and control of non-communicable diseases in Iran.

Methods

The study explored the overall quality of program implementation. The participants, including designers of IHHP, stakeholders and community members (n = 60) were purposefully recruited from the intervention areas. Data collected from semi-structured interviews and field notes were analyzed using a modified thematic analysis.

Results

Four main themes were identified. Our findings highlighted the key role of the resources as both facilitating and hindering factors. IHHP directors faced incompatibilities arising from negative perceptions/attitudes which resulted in decreased adherence to the program. Hence various strategies were used to motivate, strengthen and organize the human workforce implementing the program.

Conclusion

Recommendations arising from evaluation of the program were used in subsequent stages of implementation. Qualitative research is an important component of community trials which can improve their implementation.

Introduction

The high prevalence of cardiovascular diseases and their associated mortality worldwide, including in Iran, have made them a major health concern [1–3]. As a public health response, a comprehensive action-oriented integrated community-based program entitled Isfahan Healthy Heart Program (IHHP) was launched in Iran in 1999-2000.

This program has provided an opportunity to assess the effectiveness of lifestyle interventions designed for non-communicable diseases (NCDs) prevention and control in a developing country setting. The long-term objectives of IHHP were to decrease the incidence of NCDs and their related disability/mortality. Improving of the knowledge and awareness of the general population and health professionals about NCDs, and improvement of the skills of health professionals in controlling NCD risk factors constituted the short-term objectives of IHHP [4].

Study communities were two intervention counties (Isfahan and Najafabad) and a reference area (Arak), all located in Central Iran. The program had three phases. The first phase, that was conducted in intervention and reference areas in 2000 to study the existing situation of cardiovascular diseases and it's risk factors as well as the knowledge, attitude and practice of the population about the disease and healthy life style. The second-phase, that included multidisciplinary interventions, began in 2001 based on the results of the situation analysis and needs assessment in the community; this phase targeted the whole community and environmental changes in Isfahan and Najafabad [4]. The third Phase was conducted in both reference and intervention areas to the evaluate outcomes by repeating the same studies done in the first phase on independent samples [5, 6]. The program consisted of ten interventional projects, namely Healthy Food for Healthy Communities (HFHC), Isfahan Exercise Project (IEP), Women's Healthy Heart Project (WHHP), Heart Health Promotion from Childhood (HHPC), Youth Healthy Heart Project (YHHP), Worksite Intervention Project (WIP), Non-Governmental Organizations (NGOs) and Volunteer Intervention Project (NVIP), Health Professionals Education Project (HPEP), Healthy Lifestyles for High Risk Groups (HLHR) and Healthy Lifestyles for Cardiac Patients (HLCP). Each project targeted a different group and consisted of various strategies, including healthy nutrition, physical activity, tobacco control and stress management (Table 1) [6].

Table 1 Brief summary of the main activities of the ten interventional projects of Isfahan Healthy Heart Program [6]

IHHP evaluation protocol targeted the impact, process and outcome of the program with work beginning in 2002 [5]. We reported the results and describe the methods of the qualitative component of IHHP process evaluation (PE) [7]. Appreciating the value of process evaluation [8, 9], IHHP directors set out to gain in-depth understanding of how the program played out in action. Details of process evaluation, designed as a triangular study have been reported previously [7]. Unlike most evaluation studies focusing primarily on objective and subjective outcome evaluation [8, 9], we aimed to understand the quality of program implementation and explore the perceptions/experiences of program implementers/targets during the interventions. We focused on obtaining information that could help with program management and improvement.

Methods

Design

Using a qualitative approach, this study revealed the perceptions of program implementers/targets during IHHP interventions. This is an acceptable method in complex and context-dependent community-based programs.

Setting, sample and procedures

This study was conducted in intervention areas. Initially, a process evaluation committee (PEC) consisting of an independent team of researchers was assembled. The PEC members were five health professionals with previous experience in qualitative research. They became familiar with IHHP goals, objectives, strategies and interventional projects through 8 meetings in the presence of PEC members and IHHP board of directors (and some key stakeholders and program designers). The study participants consisted of key informants and others who had some experience with topics of interest to the targeted population, and included policy makers, stakeholders, projects administrators, local leaders, the community members etc. They were purposefully selected on the basis of their role in the interventional projects and information about the program areas, as well as recommendations from those already interviewed. A total of 60 participants were included in the study. Subsequently, feedback gained from the initial meetings was used to make decisions about the study's leading questions.

Data collection and Interviews

Data collection involved in-depth semi-structured interviews and obtaining field notes during completion of the research over 3 years (2004-2006). This provided a source of formative data with which to identify issues and experiences gained during the implementation stage of the IHHP. Ten open-ended questions were used (Table 2). These questions were reviewed through meetings in terms of content, format, and audience appropriateness; subsequently they were modified to suit our participants and their specific roles.

Table 2 Qualitative interview guiding questions.

Two members of the PEC accompanied by their assistants conducted the interviews. They were provided with training to administer qualitative interviews on a one-on-one basis. Two pilot interviews were completed, transcribed verbatim, and reviewed by all of the members of the PEC to ensure that domains of interest were being fully explored. Then the interviewers were introduced to the IHHP managers and participants by sending letters explaining the purpose of the study; permission to collect data was obtained by use of forms. The investigators met the selected participants and then conducted face-to-face interviews. Each interview took approximately 1 hour to complete. Interviewers explored the questions shown in Table 2. All interviews were undertaken in an agreed private place (often a workplace). The interviews continued until reaching a level of information saturation. About six participants from each of the ten IHHP interventional projects were included in the study. Data collection and analysis were conducted simultaneously, and ceased when no new themes could be extracted from the interviews. The interviews were audio-taped with the participants' consent. The interviewer made detailed notes only in one instance where an interviewee preferred not to be audio-recorded. Taped interviews were transcribed verbatim very soon after the interviews.

Research ethics

The ICRC Ethics Committee (NIH Code: FWA 0000t8578) approved the study. Informed consent was obtained from the participants prior to interviews; the participants were ensured about confidentiality and anonymity. All data were protected and were made available only to the authors; tapes of interviews were kept in a locked drawer.

Analysis

Collection and analysis of qualitative data were done simultaneously. A modified thematic analysis was used to analyze the data obtained from the transcripts, as well as relevant field notes; it was used to identify explicit and implicit themes, and essences or patterns within the text in relation to the study aims and objectives [10]. Transcripts were independently read by the two principal researchers (M.A & K.R), who subsequently coded, identified and categorized the existing themes. Another researcher (H.A) assessed the work on a subset of transcripts to ensure consistency. The analysis process was repetitive, the texts were coded and common codes were combined to identify related themes [11]. After analyzing each interview (and related notes), sections of texts coded to a particular theme were examined across the datasets and then the overall themes were developed [12]. We ensured that any emergent themes and concepts were developed from the data and not from a predetermined hypothesis. Theme choices were collectively discussed and the final interpretation was produced and agreed upon. Table 3 shows the list of themes and code definitions.

Table 3 Themes, sub-themes and their definitions/descriptions.

Quality assurance of data analysis and interpretation

The reliability and validity of the study was strengthened by including adequate participants, conducting the analytical process actively, and reaching a level of saturation. Relevance of these strategies has been supported [13]. Audio recordings and verbal transcripts were used to collect data. Text coding and data analysis were performed independently by three senior researchers and the results were discussed by the PEC members to identify any inconsistencies. Additional strategies were adopted to strengthen the study; these included investigator triangulation and peer-checking of the themes [14].

Results

There were several interventions in each project, hence a variety of themes and sub-themes emerged from this study. IHHP was performed in multiple settings (Annex 1), however, the main target groups who underwent interventions were the general population, health professional and cardiovascular patients. Here we present the results obtained from the HLCP, HPEP and WIP projects. Most of the project managers were selected from amongst policymakers, stockholders and academics. Hence, most of the participants in the current quantitative research had positive attitudes to the planned interventions, which resulted in higher commitment to the program. They felt more responsible and adjusted their performance using feedbacks. Data emerging from this qualitative study produced several themes and related sub-themes, as follows:

Resources

Accessibility of resources was emphasized by the participants; it affected the implementation of interventions. Adequate and appropriate allocation of resources yielded satisfactory results in line with program goals. Lack or shortage of resources, limitations in material and human resources, increased personnel responsibility, and changes in management, led to serious problems. Availability of human resources critically affected the implementation of IHHP interventions. The participants said they faced considerable shortage of human resources. The IHHP board of directors had relied on the participants themselves to appropriately respond to this challenge, i.e. the program aimed to make the participants self-enterprising, for instance, many of the educational materials prepared in IHHP projects were used by nurses for peer education; here the resource was not supplied directly by IHHP directors, rather by program stakeholders and the community. This reflects improvement of culture in a community-oriented health program. Educational technology was found to be of great importance second to human resources.

Attitudes to the program

We defined 'attitude' as any positive or negative belief about the program's effectiveness, feasibility and rationale, as well as feelings about program designers/directors. We found from the participants' experiences that positive attitudes, especially those of project managers strengthened their commitment to the interventions, enabling them to achieve a higher level of program acceptance. Another finding related to the lack of awareness of program goals, and negative attitude to program effectiveness among some health professionals/policy makers. Project managers spent much time and energy solving this conflict. Our analysis suggests that a key barrier to implementation was the stakeholders' negative attitude to IHHP.

Enabling factors

Project managers arranged meetings with stakeholders to draw their compliance with planned IHHP activities. Based on the participants' experiences, some of these activities were helpful and were accepted by the stakeholders as a "pleasant experience". The factors perceived as being "enabling" are:

Effective education

The trainees were pleased with their education program and found it in line with their occupational needs.

Positive reinforcement

Program planners found more incentives to encourage the acceptance/continuation of desirable behaviors. For example, one method used to encourage smoking cessation was to establish a reward system.

Creative activities

We indentified activities within the programs for prevention, treatment and rehabilitation of cardiovascular disease which increased motivation and enhanced the stakeholder's push toward program goals.

Recognition and appreciation

Gaining appreciation was a great challenge in implementing interventions, especially in hospitals. Behaviors related to program objectives need not only recognition, but also appreciation.

Sensitization

It attempts to build an acceptable atmosphere where the stakeholders learn the educational concepts underlying IHHP. We found that improving people's knowledge improved their behaviors.

Consistent participation

The implementation of interventions requires advocacy and cooperation of other organizations. Communication with stakeholders is essential to the continuum of interventions. Participation included formal and/or informal relations to the program's multidisciplinary background. In addition, the quality of relations predicted agreeable outcomes.

Discussion

Our findings supported the notion expressed as "more staff, more training, more skills and strengthened health systems" [15]; financial, technological and human resources etc were central to achieving IHHP goals and objectives. Although IHHP aimed to draw upon a spontaneous motion toward healthy lifestyle, our findings stress the importance of timely and adequate provision of resources for the initiation and continuation of the program. All interventional projects tend to suffer from resource scarcity. The importance of resource management [16, 17] and the role of health care as a dynamic sector [18] has been demonstrated. Development of appropriate approaches for analyzing the need for health/social care services and resources is important [19]. We found that a responsive atmosphere aided by creative activity-planning, providing positive feedback for healthy behaviors, and developing materials tailored to the stakeholders' needs facilitated implementation of interventions and helped overcome some practical limitations; a fertile atmosphere receptive to new programs was conceived. In order to direct human resources toward change, it is necessary to consider three relevant elements, namely attitude, expertise and skill [20]. Attitudes toward partnership in the program, as well as perceptions about the usefulness of the planned interventions and their outcomes constituted an important concern in the program. As the attitudes and values of individuals are the main factors for promoting positive behaviors and transforming them into opportunities for success [20], it is important to assess the participants' attitudes toward change [21].

Based on our participants' experiences, enabling factors were the most interesting ones. We found that readiness of stakeholders and the community to accept new changes was considerably grounded in their motivation. The various educational sessions drawn up by program designers were part of a core strategy to achieve this goal. To make these sessions more effective, social, environment and contextual factors were taken into account. The participants in IHHP i.e. program planners, stakeholders, and the community joined in a collaborative approach of both intra- and inter-sectoral nature. Some of the problems faced in the process were caused by traditional, professional, and organizational values and/or interpersonal incompatibilities. The emphasis on collaboration is consistent with the efforts encouraging a collaborative approach to health services delivery [22]. This is especially true with respect to self-managed work teams, applicable to community-based programs [22]. The results also stress the importance of administrative formalization initiatives to enhance collaboration among professional/organizational groups. The relevance of such view is well documented. Formalization of functions and processes appear to be an interesting means to further collaboration [22].

In summary, most of the participants expressed satisfaction with the program. This may have been due to IHHP's integration with existing systems/facilities at the community level. The PEC found that IHHP designers had developed strategies to provide accessible and useful services to all users. Notably, they could overcome resource shortages through appropriate management. They employed various strategies, such as training and information programs to assess the special needs/opinions of target groups, risk sensitization programs, inter-sectoral collaboration, alliance formation, and behavioral change through inductive approaches etc; this way, they tackled some of the negative attitudes to practicality/usefulness of program.

Conclusion

This study highlights the important role of resource availability, as well as facilitating/hindering factors. Acceptance of interventions and compliance were determined by attitudes/perceptions about the feasibility/effectiveness of the program. Face-to-face meetings and distribution of educational materials in the community reduce resistance to the program.

Organized planning and consistent participation are essential to success of interventions and adherence to the program. Successful strategies and interventions were officially integrated into the health system. The findings and recommendations of evaluation were used in subsequent program planning. We propose that simple IHHP-type interventions be adapted for implementation and integration in communities with limited financial and/or human resources.