Adaptation and validation of the All Aspects of Health Literacy Scale (AAHLS) in healthcare settings of Addis Ababa, Ethiopia

Abstract

Background

Since health literacy is known to be related to health outcomes, it should be measured to explain how it is associated with the health status of the population. Health literacy tools are designed to measure different dimensions of health literacy of individuals based on their objective. The AAHLS tool is comprehensive and can cover all aspects of health literacy. Overall in Ethiopia, there is no standard health literacy tool that has been developed or/and validated. Therefore, the aim of this study was to adapt and validate the All Aspects of Health Literacy Scale (AAHLS) in healthcare facilities in Addis Ababa city, Ethiopia.

Methods

A mixed-method, facility-based, cross-sectional study was conducted in Addis Ababa city from February 1, 2022, to May 30, 2022. The study was conducted in three phases: forward and backward translation and expert review, cognitive interviews and survey administration. For the cognitive interviews, a total of 16 participants and for the survey administration, 199 participants were involved. Coding and analysis of the qualitative data were performed using OpenCode 4.03 computer software. Then, pretesting (survey administration) was conducted to check the validity and reliability of the tool. CFA was conducted using SPSS version 26 and Stata version 14.

Results and discussion

The original three response categories were revised to five response categories based on the cognitive interview findings and expert reviews. The survey was administered to 199 participants, 55.8% of whom were males. The Kaiser‒Meyer‒Olkin measure of sampling adequacy was 0.685, with a significant difference according to Bartlett’s test of sphericity (p < 0.001). After removing the empowerment factor model fit indices, the internal reliability and convergent and divergent validities improved. Confirmatory factor analysis showed that the model fit indices of the tool were satisfactory. The overall internal consistency, Cronbach’s alpha, was 0.71.

Conclusion

The three response categories of the tool were revised to five response categories. The AAHLS tool was revised to include 10 items. The tool has exhibited adequate model fitness. The validated tool can be used for future health literacy assessments and interventions.

Text box 1. Contributions to literature
• There is no validated standard tool on health literacy in health care settings of Addis Ababa, Ethiopia.
• A validated comprehensive health literacy tool is necessary to measure health literacy status of the population which is the global social determinant of public health.
• The tool has exhibited adequate model fitness; so that the tool can be used for future health literacy assessments and interventions in Addis Ababa.

Introduction

Health literacy is defined by the World Health Organization as “cognitive and social skills that determine the motivation and ability of individuals to access, understand and use information in ways which promote and maintain good health”. This definition clearly indicates that health literacy is beyond reading a given piece of health communication material [1, 2]. In general, according to different authors and institutions, there is variety in the definition and dimensions of health literacy [3]. This variety in the definition of health literacy created opportunities for further studies in this newly emerging field.

Health literacy is known to be a global social determinant of health and is highly related to the population's health status [4]. To measure the health literacy of the population, a standardized health literacy tool is necessary. Currently, comprehensive health literacy tools that can measure all levels/scales of health literacy are recommended [5, 6]. Unfortunately, in Ethiopia, there is no such standardized health literacy tool nationally that has been developed, adapted or validated in local languages.

The consequence of not having a validated health literacy tool is associated with the burden of health problems related to the health literacy status of the community. The impact of low health literacy is broad and can be related to social determinants of health and health inequity, which is the major public health challenge of the twenty-first century [7,8,9]. The absence of a validated tool at the national level also indicates that little attention is given to health literacy, which is emphasized by the WHO, particularly for low- and middle-income countries [10]. Additionally, using invalidated tools is not useful for generating valid evidence; in contrast, it may lead to counter effective activity, which can misdirect researchers and policy makers [11].

The AAHLS was selected for this study for adaptation and validation in the Amharic version because of its comprehensiveness. The AAHLS encompasses all dimensions of health literacy as defined by Nutbeam, and it is a theory-based health literacy tool [11,12,13]. These types of health literacy tools are more effective at measuring the health literacy level of an individual or population [14]. Moreover, the AAHLS has 13 items and can be conducted within a short period of time; on average, the AAHLS can be completed within 7 min [11]. This is very important for this research project since it did not receive any funding.

One of the adapted versions of this tool is the Chinese version for Chinese speakers in the U.S., which has limited English proficiency. The authors modified the Likert scale to a five-category response from three after incorporating feedback from cognitive interviews [12]. For pilot testing, the original AAHLS was surveyed through a self-report technique. Self-assessment has potential self-report bias, which has been criticized [2]. To avoid and control such bias in this study, the survey procedures for pilot testing were conducted through face-to-face interviews. Since a comprehensive health literacy tool that has been adapted and validated in the Ethiopian context and language is unavailable, the aim of this study was to adapt and validate the AAHLS in healthcare settings in Addis Ababa, Ethiopia.

Methods and materials

Study setting

This study was conducted in Addis Ababa, the capital city of Ethiopia and home to the African Union. Adult literacy in the city is greater than 93% for males and almost 80% for females [15]. Addis Ababa city has 52 governmental and private hospitals, 98 functioning health centers and 534 private clinics. This study was conducted in a health facility setting with clients aged 18–64 years who came to health facilities.

Study approach

A facility-based cross-sectional mixed-method study was conducted. This study was conducted in three phases. In phase one, forward and backward translation of the AAHLS and expert review of the tool were conducted. In phase two, cognitive interviews and tool revision were conducted on the AAHLS Amharic version. A deductive qualitative content analysis approach was used for the analysis of the cognitive interview data. Phase three of the study, to pretest the validity of the adapted AAHLS survey administration was conducted using a cross-sectional.

Study population

The study population for this study was any person who resided in Addis Ababa city permanently and who came to selected health facilities at the time of the data collection. Participants who were volunteers and could speak the Amharic language were involved in this study.

Tool translation and expert review

The translation-back-translation method was used to adapt the English version of the All Aspects of Health Literacy Scale into Amharic [16]. The translation was performed by language professionals who are fluent in both the English and Amharic languages. Back translation of the tool was carried out by other language professionals who were not exposed to the original English version. Then, the two original and back-translated English versions of the tool were compared and evaluated for consistency and face validity by a group of experts who are fluent in both English and English.

Cognitive interview

For the cognitive interviews, 16 participants were enrolled to identify potential sources of measurement error related to the survey questions. A maximum variation purposive sampling technique was used to select those participants. Voice recording and note taking were used as the methods of data collection. Think-aloud and verbal probing techniques were applied during the cognitive interviews [17]. Regarding the think-aloud approach, the interviewees were asked to describe their cognitive process while answering the survey questions. Before conducting the interviews, preliminary training was provided for the participants. The concurrent probing method was used when specific questions were asked at the end of each cognitive interview. Questions related to comprehensibility, acceptability, relevance, response set, and completeness were asked for each respondent for each item.

After the cognitive interviews, the collected data were transcribed and then translated for coding. The codebook was prepared to include all the possible dimensions that may be assessed by the cognitive interview. The following terms (comprehensibility, acceptability, relevance, equivalence and completeness) were some of the codes documented within the codebook to guide the coding process.

The original three response categories were revised to five response sets to make the items inclusive for possible options, taking into consideration community feedback. On the basis of the cognitive interview reports, some changes were made to the tool by a second expert review of those items, as commented upon by the community. The following (Table 1) summarizes the 13 items of the tool and its response sets.

Table 1 Revision of the adapted version of the AAHLS tool in Addis Ababa, Ethiopia, 2022

Sample size determination for survey administration

The sample size for survey administration was determined by the rule of thumb, which recommends 10:1 (sample size to number of items) [18]. Hence, the sample size was calculated to be 130 according to the rule of thumb. However, as the sample size increases, the validated scale will be psychometrically sound. For this purpose, a 15:1 ratio was used to determine the sample size. In general, 199 participants were selected for pretesting of the adapted scale because of its validity and reliability. For the six health facilities that were selected randomly, the numbers of study participants were allocated equally.

Sampling procedure

A simple random sampling technique was used to select health facilities. From 11 governmental hospitals available in Addis Ababa city, two hospitals, from 98 health centers, two health centers and from private hospitals, two facilities were selected randomly through the lottery method for this study. Within each facility, participants were selected from each service delivery center using a systematic random sampling technique. Considering that 30 clients visit a given health care delivery room/area, from each area to interview 5 participants, we used to interview every sixth client.

Data collection procedures

Pretesting of the tool was conducted through face-to-face interviews with participants. The respondents were asked for answers from options provided after the question had been read. The responses of the respondents were recorded, and the data were collected with ODK (open data kit) technology. After finishing the data collection, the collected data were sent to the central server on a daily basis. After all the data were collected, the data were downloaded and exported to SPSS for analysis.

Data quality management

Short training sessions were provided for the data collectors, and supervision was provided throughout the data collection period. The ODK is empowered to insert constraints on the type and digits of data to be recorded during data collection. Daily checking for any outliers/errors was performed during the time of data collection to recheck the data again or to communicate early with the corresponding data collector. After being exported to SPSS for analysis, the data files were assessed again for any errors or missing values before proceeding to the data analysis.

Statistical analysis

The data were analyzed using SPSS version 26. First, the descriptive characteristics of each item were checked and presented with measurements of summary parameters to check for any anomalies in the entry of data and to assess the sociodemographic characteristics of the participants and response distribution of the items.

The internal consistency of the items was checked per domain using Cronbach’s alpha. Cronbach’s alpha is recommended for checking and assuring the internal consistency of the items with a similar method of presentation in the same domain [14]. Then, after the four domains were computed, interdomain consistency was checked in the same way using Cronbach’s alpha.

CFA was performed to assess whether the items correlated consistently with the hypothesized theoretical structure. Confirmatory factor analysis was carried out after sampling adequacy was checked by Kaiser Meyer Olkin (KMO ≥ 0.5), and data suitability for factor analysis was confirmed with Bartlett’s test of sphericity. In this case, CFA was performed to test whether the data fit the hypothesized 4-factor health literacy measurement model: functional health literacy, communicative health literacy, critical health literacy, and empowerment. To test how well a specific model fit the data, the root mean square error of approximation (0 ≤ RMSEA ≤ 0.1), the comparative fit index (CFI ≥ 0.95), the Tucker‒Lewis index (TLI ≥ 0.95), and the standardized root mean square residual (SRMR < 0.08) were used. The correlation variance among factors and between each variable and factor was also assessed using Stata version 14.

Results

Sociodemographic characteristics of the participants

A total of 199 participants were involved in this study. More than half (55.8%) were males. The mean age of the participants was 32 ± 3.892 S D. Regarding their educational status, 41.7% attended college/university education, and 3.5% could not read and write. Almost half of the participants (49.7%) spoke Amharic as their mother tongue. The details are shown below (Table 2).

Table 2 Frequency distribution of the sociodemographic characteristics of the study participants, Addis Ababa, Ethiopia, 2022

Response to the AAHLS questions

As shown in Table 3, nearly half of the participants (44.2%) responded that they “sometimes” needed someone to help them read information provided by health professionals. Some of the respondents (39.2%) explained that they “sometimes” could get hold of someone to assist them. For FQ3, more than half of the respondents (51.3%) explained that they “never” needed someone to help them fill out official documents.

Table 3 Response distributions of the all aspects of health literacy scale questions, Addis Ababa, Ethiopia, 2022

Fifty-nine percent (58.8%) of participants responded that they “always” provide all the information health professionals need to help them, and 55.3% complained that they “always” ask the questions they need to ask health professionals. Forty-one percent (41.2%) said that they “always” make sure that health professionals explain anything that they do not understand.

Forty-four percent (43.7%) of participants said they were “always” individuals who liked to determine lots of different information about their health. Approximately forty-one percent (40.7%) of the participants said that they “always” think carefully about whether health information makes sense in their particular situation. In addition, 40.2% of the participants complained that they tried to determine whether information about their health could be trusted. More than half (65.3%) of the participants questioned their doctor’s or nurse’s advice based on their own research.

Fifty-six (56.3%) mentioned that they think that there are plenty of ways to have a say in what the government does about health. Fifty-four percent (53.8%) mentioned that within the last 12 months, they have taken action to do something about a health issue that affects their family or community. More than half of the participants (56.8%) said that information and encouragement to lead healthy lifestyles matter most for everyone’s health. The summary of participants’ response distributions to each item of the AAHLS is presented in (Table 3) below.

Confirmatory factor analysis model fitness

The Kaiser‒Meyer‒Olkin (KMO) measure of sampling adequacy was 0.685, which was significant according to Bartlett’s test of specificity (p < 0.001). This suggests that the sample size is adequate to conduct CFA. The analysis was performed using the user-written Stata command “estat gof, stats(all)”. As shown in (Table 4), the Amharic version of the tool exhibited a satisfactory model fit.

Table 4 Confirmatory Factor Analysis Model Fit for the AAHLS Tool in Addis Ababa, Ethiopia, 2022

A further examination of the results of the analysis revealed that the two domains were not supported by their corresponding item data. As (Table 5) shows the Amharic tool models with standardized regression coefficients for all the paths. The functional, communicative and critical factors had correlation coefficient values greater than 0.4 with their corresponding factors.

Table 5 Correlation matrix of the AAHLS items with the scales (factors) in Addis Ababa, Ethiopia, 2022

The items with low correlation with their own dimension were removed, and the analysis was subsequently performed. The empowerment items were removed from the analysis, and then the model fit indices (Table 6) and the standardized regression coefficients of the other items were improved, as shown in the following tables (Table 7).

Table 6 Model fit indices after removal of the empowerment factor in Addis Ababa, Ethiopia, 2022

Table 7 Standardized regression coefficients after removal of the empowerment factor in Addis Ababa, Ethiopia, 2022

Tests of reliability and validity

Internal consistency was first calculated for each item under the same domain to measure their correlation with each other. Accordingly, the items of functional, communicative and critical factors exhibited internal consistency of Cronbach’s alpha of (0.410–0.940) in range. Unfortunately Crobach’s alpha of each item of empowerment factor was found to be less than 0.350. The overall internal consistency of the 13 questions of the tool was measured with Cronbach’s alpha and identified as 0.634. The Cronbach’s alpha values for items in each domain were 0.650 for functional, 0.730 for communicative, 0.790 for critical, and 0.380 for empowerment. The convergent validity for the 13 items was 53.8% (7/13) (item-scale correlation > 0.40). For divergent validity, 9/13 items (69.2%) had a correlation coefficient with the scores of their own dimension greater than those computed with another score.

After removing the empowerment items, the internal consistency for 10 items was 0.710, which is slightly greater than the cutoff value. The overall convergent and divergent validities improved after the removal of the empowerment items. Convergent validity: Seven of the 10 items (70.0%) had a correlation coefficient with the score of their own domain that was greater than 0.400. Divergent validity: Three of the 10 items (30.0%) had a correlation coefficient with the score of their own domain greater than those computed with other scores.

Discussion

In this study, the AAHLS tool was adapted and validated in the Amharic language in Addis Ababa city, Ethiopia. As part of the process, the study followed the recommended process for tool adaptation and validation [19]. Most questions were understood directly the way they were intended to mean. This indicates that the forward and backward translation, as well as the first expert review, has revised the tool as closely as possible to the community’s understanding. On the other hand, during the cognitive interviews, some ambiguous terms were identified and modified by a second expert. This can be subjected to the discrepancy between the community’s understanding and experts’ judgment.

The difficulty of the questions increased from the beginning of FQ1 to Emp3. This indicates an increase in the complexity of health literacy skills as they progress from the functional aspect, which is basic, to the empowerment aspect, which is practical [13]. The respondents mentioned that the questions are different when they range from functional questions to communicative questions, from communicative questions to critical questions and, similarly, when they range from critical questions to empowerment questions. This can be explained by the fact that the items under each factor are more related together than they are related to the items under other components.

The original three-category response scale was adjusted to a five-category response scale based on the findings from cognitive interviews. The response distributions table (Table 3) presented above confirmed that such a change is necessary specifically for our target population. For example, 51.3% of the participants had chosen never to answer the question (FQ3). A total of 58.8% and 55.3% of the participants chose to answer questions ComQ1 and ComQ2, respectively. As a result, it is believed that such a change in the response scale could increase the accuracy of health literacy assessments. This change is consistent with a study conducted on U.S. Chinese immigrants, which made similar changes in response categories [20]. However, it is different from the original English version of the tool, which uses three response categories. They devised the tool with five response categories originally, but later, during cognitive interviews, they revised it to three response categories [12]. This difference may be related to differences in community perception and judgment.

The results of our study showed that our tool is linguistically and culturally appropriate and could elicit a reliable and valid health literacy assessment. The tool fit all the set criteria satisfactorily. After removing factors with low correlation coefficients, i.e., empowerment factors, the model fit indices improved. Model fitness in this study is less than that of studies on the same tool in Chinese [20]. This may be due to the different response sets that are used for different items of the scales. The model-data fit of our study is better than the model-data fit of other studies [21, 22].

Items of empowerment factors were correlated with their dimensions with an insignificant coefficient. As a result, these items were excluded from the subsequent analysis and from the tool. Regarding this factor, another study on the validation of this tool in Chinese found similarly low coefficient values for its items [20]. During the cognitive interviews on the items of the empowerment factor, especially Emp3, most participants complained that those items were not significant at all. This can be subjected to differences in the sociocultural and political context of the study population.

The overall internal consistency of the 10 items was identified to be adequate. For each domain, internal consistency with Cronbach’s alpha was identified as satisfactory. This finding is lower than the reliability of the same tool validation in Chinese individuals [20]. It was identified to be more reliable than other tools in terms of its Cronbach’s alpha value [21, 22]. This difference could be explained by differences in the study context and population. This finding is consistent with the recommended Cronbach’s alpha value [18]. After removing empowerment items, both convergent and divergent validities were adequately improved to their normal threshold level.

Strengths and limitations of the study

Although this is the first study in the country, it is not without limitations. The sample size determination technique used in this study was a rule of thumb, which may have resulted in an inadequate sample size to make the findings of this study generalizable. The study was conducted in an urban setting among Amharic speakers. This study may not be generalizable to non-Amharic speakers or to the rural setting of the country. We recommend that future studies take this into consideration when they use this study as a reference.

Conclusion

The three-response category of the AAHLS was revised to five response categories. The tool has exhibited adequate model fitness. The empowerment factor was removed, and the tool was revised to include 10 items with three dimensions. The tool can be used to measure the health literacy of residents of Addis Ababa city. The empowerment factor items should be developed in this particular context. Hence, future studies can develop the removed items and adapt this scale to other languages at the Addis Ababa and national levels since Ethiopia is home to different ethnic groups, languages and cultures.