How can the results of a qualitative process evaluation be applied in management, improvement and modification of a preventive community trial? The IHHP Study
© Sarrafzadegan et al; licensee BioMed Central Ltd. 2011
Received: 16 April 2011
Accepted: 5 December 2011
Published: 5 December 2011
This study reports the results of the qualitative process evaluation (PE) of the Isfahan Healthy Heart Program (IHHP), an integrated community-based trial for prevention and control of non-communicable diseases in Iran.
The study explored the overall quality of program implementation. The participants, including designers of IHHP, stakeholders and community members (n = 60) were purposefully recruited from the intervention areas. Data collected from semi-structured interviews and field notes were analyzed using a modified thematic analysis.
Four main themes were identified. Our findings highlighted the key role of the resources as both facilitating and hindering factors. IHHP directors faced incompatibilities arising from negative perceptions/attitudes which resulted in decreased adherence to the program. Hence various strategies were used to motivate, strengthen and organize the human workforce implementing the program.
Recommendations arising from evaluation of the program were used in subsequent stages of implementation. Qualitative research is an important component of community trials which can improve their implementation.
KeywordsCommunity interventions Health behavior Health Cardiovascular program evaluation
The high prevalence of cardiovascular diseases and their associated mortality worldwide, including in Iran, have made them a major health concern [1–3]. As a public health response, a comprehensive action-oriented integrated community-based program entitled Isfahan Healthy Heart Program (IHHP) was launched in Iran in 1999-2000.
This program has provided an opportunity to assess the effectiveness of lifestyle interventions designed for non-communicable diseases (NCDs) prevention and control in a developing country setting. The long-term objectives of IHHP were to decrease the incidence of NCDs and their related disability/mortality. Improving of the knowledge and awareness of the general population and health professionals about NCDs, and improvement of the skills of health professionals in controlling NCD risk factors constituted the short-term objectives of IHHP .
Brief summary of the main activities of the ten interventional projects of Isfahan Healthy Heart Program 
Healthy Food for Healthy Community
Training about healthy cooking methods, making high-fiber low-salt bread, production of healthy food products by food industries, modifying food labels, educating the public on the concept of healthy nutrition, improving the formulation of confections, introduction of healthy brands and half-portions into restaurants and fast food shops
Isfahan Exercise and Air Pollution Control Project
Providing training on exercise and physical activity through local TV, distribution of educational CDs about exercise at home and at worksite, organizing public exercise rallies, organizing automobile-free days, organizing healthy heart exhibitions, educating about air pollution control methods through local TV, advocating the development of bicycle lanes in the city
Women Healthy Heart Project
Training the young women and their family members who attended pre-marriage classes, instructors of the Literacy Campaign Movement and their students, instructors of charities, women's Basij Movement, women in different organizations, women attending health centers and health houses, female volunteer-instructors of the Red-Crescent Society, educating the public through TV and cook books, distributing a CD about methods of physical activity requiring no special facilities
Heart Health Promotion from Childhood
Population approach by training children, parents, health professionals, school and kindergarten staff about healthy lifestyle, promotion of physical activity in schools and kindergartens, introduction of healthy snacks in schools and kindergartens and establishing healthy buffets in schools, forming role model groups from volunteer students, providing practical training through TV about healthy lifestyle; and high-risk approach by screening for CVD risk factors in children of patients with premature CVDs and children with at least one risk factor such as obesity.
Youth Intervention Project
Training volunteers from the Red Crescent Society, garrison instructors, soldiers in their mandatory military service and university students, and kitchen staff of universities and garrisons about healthy lifestyle, conducting the international anti-smoking Quit & Win campaign
Worksite Intervention Project
Training occupational medicine physicians or health assistants, introducing dietary modifications into restaurants of factories, enforcing no-smoking regulations at worksite, using the existing screening system to detect high-risk groups, encouraging physical activity at worksite, publishing health messages about CVD prevention as an integrated part of official newsletters of different organizations.
Non-Governmental Organizations and Volunteers Project
Training health workers in cities and villages, forming, training and empowerment of an assembly of health volunteers, training of the community about performing physical activity in the absence of facilities, healthy nutrition and coping with stress via trained volunteers and health-related NGOs.
Establishment of educational assemblies, training GPs through continuing medical education (CME) courses, training physicians through periodical seminars, training nurses by organizing educational assemblies, and publishing and distributing books and newsletters among nurses and other health professionals in urban and rural areas, running information campaigns on various occasions
Healthy Lifestyle for High-Risk Groups
Training the personnel of the health system, high-risk individuals, retired employees, activation of clinics at hospitals, public training, distributing educational brochures to those attending pharmacies, printing health messages on laboratory report sheets
Healthy Lifestyle for Cardiovascular Patients Project
Training the patients and their families at the time of discharge, printing cards for patients to record all necessary information related to their condition, establishing rehabilitation units in all heart hospitals, improving nutrition-related and cooking procedures at hospital restaurants, distributing educational folders containing educational materials on cardiovascular disease secondary prevention and rehabilitation
IHHP evaluation protocol targeted the impact, process and outcome of the program with work beginning in 2002 . We reported the results and describe the methods of the qualitative component of IHHP process evaluation (PE) . Appreciating the value of process evaluation [8, 9], IHHP directors set out to gain in-depth understanding of how the program played out in action. Details of process evaluation, designed as a triangular study have been reported previously . Unlike most evaluation studies focusing primarily on objective and subjective outcome evaluation [8, 9], we aimed to understand the quality of program implementation and explore the perceptions/experiences of program implementers/targets during the interventions. We focused on obtaining information that could help with program management and improvement.
Using a qualitative approach, this study revealed the perceptions of program implementers/targets during IHHP interventions. This is an acceptable method in complex and context-dependent community-based programs.
Setting, sample and procedures
This study was conducted in intervention areas. Initially, a process evaluation committee (PEC) consisting of an independent team of researchers was assembled. The PEC members were five health professionals with previous experience in qualitative research. They became familiar with IHHP goals, objectives, strategies and interventional projects through 8 meetings in the presence of PEC members and IHHP board of directors (and some key stakeholders and program designers). The study participants consisted of key informants and others who had some experience with topics of interest to the targeted population, and included policy makers, stakeholders, projects administrators, local leaders, the community members etc. They were purposefully selected on the basis of their role in the interventional projects and information about the program areas, as well as recommendations from those already interviewed. A total of 60 participants were included in the study. Subsequently, feedback gained from the initial meetings was used to make decisions about the study's leading questions.
Data collection and Interviews
Qualitative interview guiding questions.
When the program has been implemented at this place, what factors do you think have affected the effective use of the interventions of this project?
Do you think it is better to change the way that work is done?
In your view, how successful was this intervention?
Do you think this intervention must be continued, modified, or discontinued? State your reasons.
Have the goals of your respective intervention been realized in the current system of your work?
What are some of the barriers or difficulties that you are encountering with the program? and which challenges are you confronting during the implementation of interventions?
Which kind of facilities/barriers are you confronting during the implementation of interventions?
State the degree of your satisfaction with this intervention.
How do you assess the future continuity of habits encouraged by the designed intervention?
Do you think this intervention can be implemented at the national level? Please state your reasons
Two members of the PEC accompanied by their assistants conducted the interviews. They were provided with training to administer qualitative interviews on a one-on-one basis. Two pilot interviews were completed, transcribed verbatim, and reviewed by all of the members of the PEC to ensure that domains of interest were being fully explored. Then the interviewers were introduced to the IHHP managers and participants by sending letters explaining the purpose of the study; permission to collect data was obtained by use of forms. The investigators met the selected participants and then conducted face-to-face interviews. Each interview took approximately 1 hour to complete. Interviewers explored the questions shown in Table 2. All interviews were undertaken in an agreed private place (often a workplace). The interviews continued until reaching a level of information saturation. About six participants from each of the ten IHHP interventional projects were included in the study. Data collection and analysis were conducted simultaneously, and ceased when no new themes could be extracted from the interviews. The interviews were audio-taped with the participants' consent. The interviewer made detailed notes only in one instance where an interviewee preferred not to be audio-recorded. Taped interviews were transcribed verbatim very soon after the interviews.
The ICRC Ethics Committee (NIH Code: FWA 0000t8578) approved the study. Informed consent was obtained from the participants prior to interviews; the participants were ensured about confidentiality and anonymity. All data were protected and were made available only to the authors; tapes of interviews were kept in a locked drawer.
Themes, sub-themes and their definitions/descriptions.
Experiences with resources
Resources (human work force and hardware) were perceived as key factors in carrying out the projects. They were both facilitating and hindering
Participant (no. 6-HLCP): "The major problem is insufficient number of nurses in the hospitals, unfortunately."
Participant (no.2-HLCP): "We can't follow up the implementation the interventions, because we have few professionals against high duty load."
Participant (no.6-HLCP): "We prepare slides as powerpoints; get pamphlets to nurses who agreed to become trained."
Participant (no.5-HLCP): "All instruments and resources were prepared by the hospital managerial team and nobody had helped them."
Participant (no.1-HLCP): "I brought my personal computer from my own shop and installed educational materials on it."
Attitude toward the program
It implies any positive or negative beliefs (barriers) about IHHP, in terms of program effectiveness and feasibility, directors' qualifications etc.
Participant (no. 6-HLCP): "Nursing supervisors in hospitals are very concerned... we printed a folder on some topics for patients' education.... It seems to be successful. Many physicians requested these educational materials for their private offices."
Participant (no. 5-WIP): "Initially, there was some resistance on the part of workers, employees, cooks, etc. Yes, resistance persists actually"
Participant (no. 2-WIP): "Among the barriers were the economic concerns of some managers at work which was obvious. Any plan, like regular daily exercise for employees, that took up some of the work time, however little, would bring about resistance"
Applicable and understandable education had encouraged learning and adherence to the program
Participant (no. 6-HPEP): "I myself, enjoyed the subject of the tobacco control program"
Participant (no. 6-HLCP): "Patients need a holistic approach. For example they might ask: When am I allowed to drive? ... Some topics in our classes cannot answer the patients' questions."
Offering positive feedback to people increased acceptance and continuation of desirable behaviors/healthy life-style
Participant (no. 1 - WIP): "We have an antismoking program named Quit and Win. It was preformed regularly."
Participant (no. 3 - WIP): "We planned some competitions like caricatures, among employees and their families to encourage smoking cessation. Consequently, all their family members were persuaded to participate."
Some activities which have increased stakeholders and other participants' incentives and adherence to the program
Participant (no.1 - HLCP): "We set up stations and an exhibition that introduced IHHP pamphlets. In fact these activities drew people's attention to communicate with us."
Recognition and appreciation
It means to acknowledge that the peoples' compliance has enhanced their participation in the program interventions
Participant (no. 1 - HPEP): "Now, education is valuable, it affects employee's evaluation and job improvement. Thus employees are attracted to education."
Participant (no. 8 - HPEP): "Nurses don't motivate, because they have lost their professional values. There is no difference between active and passive employees."
Alerting and informing the people against detrimental effects of risky behaviors has encouraged them to adopt healthy behaviors
Participant (no. 2 - HLCP): "The audiences in the seminars understood how useful the secondary prevention and rehabilitation training programs are for patients."
Participant (no. 3 - WIP): "After recent interventions in our factory, if the employees don't get vegetables and dairy diet during a week, they will complain about it to the factory health center."
Quality assurance of data analysis and interpretation
The reliability and validity of the study was strengthened by including adequate participants, conducting the analytical process actively, and reaching a level of saturation. Relevance of these strategies has been supported . Audio recordings and verbal transcripts were used to collect data. Text coding and data analysis were performed independently by three senior researchers and the results were discussed by the PEC members to identify any inconsistencies. Additional strategies were adopted to strengthen the study; these included investigator triangulation and peer-checking of the themes .
There were several interventions in each project, hence a variety of themes and sub-themes emerged from this study. IHHP was performed in multiple settings (Annex 1), however, the main target groups who underwent interventions were the general population, health professional and cardiovascular patients. Here we present the results obtained from the HLCP, HPEP and WIP projects. Most of the project managers were selected from amongst policymakers, stockholders and academics. Hence, most of the participants in the current quantitative research had positive attitudes to the planned interventions, which resulted in higher commitment to the program. They felt more responsible and adjusted their performance using feedbacks. Data emerging from this qualitative study produced several themes and related sub-themes, as follows:
Accessibility of resources was emphasized by the participants; it affected the implementation of interventions. Adequate and appropriate allocation of resources yielded satisfactory results in line with program goals. Lack or shortage of resources, limitations in material and human resources, increased personnel responsibility, and changes in management, led to serious problems. Availability of human resources critically affected the implementation of IHHP interventions. The participants said they faced considerable shortage of human resources. The IHHP board of directors had relied on the participants themselves to appropriately respond to this challenge, i.e. the program aimed to make the participants self-enterprising, for instance, many of the educational materials prepared in IHHP projects were used by nurses for peer education; here the resource was not supplied directly by IHHP directors, rather by program stakeholders and the community. This reflects improvement of culture in a community-oriented health program. Educational technology was found to be of great importance second to human resources.
Attitudes to the program
We defined 'attitude' as any positive or negative belief about the program's effectiveness, feasibility and rationale, as well as feelings about program designers/directors. We found from the participants' experiences that positive attitudes, especially those of project managers strengthened their commitment to the interventions, enabling them to achieve a higher level of program acceptance. Another finding related to the lack of awareness of program goals, and negative attitude to program effectiveness among some health professionals/policy makers. Project managers spent much time and energy solving this conflict. Our analysis suggests that a key barrier to implementation was the stakeholders' negative attitude to IHHP.
Project managers arranged meetings with stakeholders to draw their compliance with planned IHHP activities. Based on the participants' experiences, some of these activities were helpful and were accepted by the stakeholders as a "pleasant experience". The factors perceived as being "enabling" are:
The trainees were pleased with their education program and found it in line with their occupational needs.
Program planners found more incentives to encourage the acceptance/continuation of desirable behaviors. For example, one method used to encourage smoking cessation was to establish a reward system.
We indentified activities within the programs for prevention, treatment and rehabilitation of cardiovascular disease which increased motivation and enhanced the stakeholder's push toward program goals.
Recognition and appreciation
Gaining appreciation was a great challenge in implementing interventions, especially in hospitals. Behaviors related to program objectives need not only recognition, but also appreciation.
It attempts to build an acceptable atmosphere where the stakeholders learn the educational concepts underlying IHHP. We found that improving people's knowledge improved their behaviors.
The implementation of interventions requires advocacy and cooperation of other organizations. Communication with stakeholders is essential to the continuum of interventions. Participation included formal and/or informal relations to the program's multidisciplinary background. In addition, the quality of relations predicted agreeable outcomes.
Our findings supported the notion expressed as "more staff, more training, more skills and strengthened health systems" ; financial, technological and human resources etc were central to achieving IHHP goals and objectives. Although IHHP aimed to draw upon a spontaneous motion toward healthy lifestyle, our findings stress the importance of timely and adequate provision of resources for the initiation and continuation of the program. All interventional projects tend to suffer from resource scarcity. The importance of resource management [16, 17] and the role of health care as a dynamic sector  has been demonstrated. Development of appropriate approaches for analyzing the need for health/social care services and resources is important . We found that a responsive atmosphere aided by creative activity-planning, providing positive feedback for healthy behaviors, and developing materials tailored to the stakeholders' needs facilitated implementation of interventions and helped overcome some practical limitations; a fertile atmosphere receptive to new programs was conceived. In order to direct human resources toward change, it is necessary to consider three relevant elements, namely attitude, expertise and skill . Attitudes toward partnership in the program, as well as perceptions about the usefulness of the planned interventions and their outcomes constituted an important concern in the program. As the attitudes and values of individuals are the main factors for promoting positive behaviors and transforming them into opportunities for success , it is important to assess the participants' attitudes toward change .
Based on our participants' experiences, enabling factors were the most interesting ones. We found that readiness of stakeholders and the community to accept new changes was considerably grounded in their motivation. The various educational sessions drawn up by program designers were part of a core strategy to achieve this goal. To make these sessions more effective, social, environment and contextual factors were taken into account. The participants in IHHP i.e. program planners, stakeholders, and the community joined in a collaborative approach of both intra- and inter-sectoral nature. Some of the problems faced in the process were caused by traditional, professional, and organizational values and/or interpersonal incompatibilities. The emphasis on collaboration is consistent with the efforts encouraging a collaborative approach to health services delivery . This is especially true with respect to self-managed work teams, applicable to community-based programs . The results also stress the importance of administrative formalization initiatives to enhance collaboration among professional/organizational groups. The relevance of such view is well documented. Formalization of functions and processes appear to be an interesting means to further collaboration .
In summary, most of the participants expressed satisfaction with the program. This may have been due to IHHP's integration with existing systems/facilities at the community level. The PEC found that IHHP designers had developed strategies to provide accessible and useful services to all users. Notably, they could overcome resource shortages through appropriate management. They employed various strategies, such as training and information programs to assess the special needs/opinions of target groups, risk sensitization programs, inter-sectoral collaboration, alliance formation, and behavioral change through inductive approaches etc; this way, they tackled some of the negative attitudes to practicality/usefulness of program.
This study highlights the important role of resource availability, as well as facilitating/hindering factors. Acceptance of interventions and compliance were determined by attitudes/perceptions about the feasibility/effectiveness of the program. Face-to-face meetings and distribution of educational materials in the community reduce resistance to the program.
Organized planning and consistent participation are essential to success of interventions and adherence to the program. Successful strategies and interventions were officially integrated into the health system. The findings and recommendations of evaluation were used in subsequent program planning. We propose that simple IHHP-type interventions be adapted for implementation and integration in communities with limited financial and/or human resources.
- Emslie C: Women, men and coronary heart disease: a review of the qualitative literature. J Adv Nurs. 2005, 51 (4): 382-95. 10.1111/j.1365-2648.2005.03509.x.View ArticlePubMedGoogle Scholar
- Sarraf-Zadegan N, Boshtam M, Malekafzali H, Bashardoost N, Sayed-Tabatabaei FA, Rafiei M, et al: Secular trends in cardiovascular mortality in Iran, with special reference to Isfahan. Acta Cardiol. 1999, 54 (6): 327-33.PubMedGoogle Scholar
- Sarraf-Zadegan N, Sayed-Tabatabaei FA, Bashardoost N, Maleki A, Totonchi M, Habibi HR, et al: The prevalence of coronary artery disease in an urban population in Isfahan, Iran. Acta Cardiol. 1999, 54 (5): 257-63.PubMedGoogle Scholar
- Sarraf-Zadegan N, Sadri G, Malek AH, Baghaei M, Mohammadi FN, Shahrokhi S, et al: Isfahan Healthy Heart Programme: a comprehensive integrated community-based programme for cardiovascular disease prevention and control. Design, methods and initial experience. Acta Cardiol. 2003, 58 (4): 309-20. 10.2143/AC.58.4.2005288.View ArticlePubMedGoogle Scholar
- Sarrafzadegan N, Baghaei A, Sadri Gh, Kelishadi R, Malekafzali H, Boshtam M, et al: Isfahan healthy heart program: Evaluation of comprehensive, community-based interventions for non-communicable disease prevention. Prevention and Control. 2006, 2 (2): 73-84. 10.1016/j.precon.2006.10.003.View ArticleGoogle Scholar
- Sarrafzadegan N, Kelishadi R, Esmaillzadeh A, Mohammadifard N, Rabiei K, Roohafza H, et al: Do lifestyle interventions work in developing countries? Findings from the Isfahan Healthy Heart Program in the Islamic Republic of Iran. Bull World Health Organ. 2009, 87 (1): 39-50. 10.2471/BLT.07.049841.View ArticlePubMedGoogle Scholar
- Rabiei K, Kelishadi R, Sarrafzadegan N, Abedi HA, Alavi M, Heidari K, et al: Process evaluation of a community-based program for prevention and control of non-communicable disease in a developing country: The Isfahan Healthy Heart Program, Iran. BMC Public Health. 2009, 9: 57-10.1186/1471-2458-9-57.View ArticlePubMedPubMed CentralGoogle Scholar
- Burn's N, Grove K: Understanding nursing research. 2001, Philadelphia: W. B SaundersGoogle Scholar
- Shek DT, Ma HK, Lui JH, Lung DW: Process evaluation of the Tier 1 Program of the project P.A.T.H.S. ScientificWorldJournal. 2006, 6: 2264-73.View ArticlePubMedGoogle Scholar
- Taylor B, Kermode S: Evidence-Based Practice in Nursing and Healthcare: A Guide to Best Practice. 2006, Cengage Learning Australia, 3Google Scholar
- Miles M, Hubermann A: Qualitative Data Analysis. 1994, Thousand Oaks, CA: Sage, 2Google Scholar
- Strauss A, Corbin J: Grounded theory methodology: An overview. Handbook of Qualitative Research. Edited by: Denzin N, Lincoln Y. 1994, Thousand Oaks: Sage, 1Google Scholar
- Morse J, Barrett M, Mayan M, Olson K, Spiers J: Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods. 2002, 1 (2): 13-22.Google Scholar
- Streubert H, arpenter D: Qualitative research in nursing: advancing the humanistic imperative. 2007, LWW, 4Google Scholar
- Berer M: Human resources: an impersonal term for the people providing health care. Reprod Health Matters. 2006, 14 (27): 6-11. 10.1016/S0968-8080(06)27241-0.View ArticleGoogle Scholar
- Batljan I, Lagergren M: Inpatient/outpatient health care costs and remaining years of life--effect of decreasing mortality on future acute health care demand. Soc Sci Med. 2004, 59 (12): 2459-66. 10.1016/j.socscimed.2004.04.003.View ArticlePubMedGoogle Scholar
- Eriksson S, Kaati G, Bygren LO: Personal resources, motives and patient education leading to changes in cardiovascular risk factors. Patient Educ Couns. 1998, 34 (2): 159-68. 10.1016/S0738-3991(97)00106-7.View ArticlePubMedGoogle Scholar
- Rayan M: Human resource management and the politics of knowledge: linking the essential knowledge base of the organization to strategic decision making. Leadership & Organization Development Journal. 1995, 16 (5): 3-10. 10.1108/01437739510088464.View ArticleGoogle Scholar
- Birch S: Health human resource planning for the new millennium: inputs in the production of health, illness, and recovery in populations. Can J Nurs Res. 2002, 33 (4): 109-14.PubMedGoogle Scholar
- Carignani V: Management of change in health care organisations and human resource role. Eur J Radiol. 2000, 33 (1): 8-13. 10.1016/S0720-048X(99)00130-8.View ArticlePubMedGoogle Scholar
- Goodwin N: Leadership and the UK health service. Health Policy. 2000, 51 (1): 49-60. 10.1016/S0168-8510(99)00072-X.View ArticlePubMedGoogle Scholar
- Sicotte C, D'Amour D, Moreault MP: Interdisciplinary collaboration within Quebec Community Health Care Centres. Soc Sci Med. 2002, 55 (6): 991-1003. 10.1016/S0277-9536(01)00232-5.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.