|
| |
| ORIGINAL ARTICLE |
|
| Ahead of print publication |
|
|
Factor analysis of oral health literacy-adults questionnaire (OHL-AQ) among patients attending a tertiary institution of Shimla: A validity and reliability study
Deepak Gurung, Vinay Kumar Bhardwaj, Shailee Fotedar
Department of Public Health Dentistry, HP Govt. Dental College and Hospital, Shimla, HP, India
| Date of Submission | 30-Aug-2022 |
| Date of Decision | 02-Nov-2022 |
| Date of Acceptance | 02-Nov-2022 |
| Date of Web Publication | 13-Jan-2023 |
Correspondence Address:
Deepak Gurung,
Department of Public Health Dentistry, HP Govt. Dental College and Hospital, Shimla - 171001, HP
India
 Source of Support: None, Conflict of Interest: None DOI: 10.4103/mjdrdypu.mjdrdypu_769_22
Background and Aim: Oral health literacy (OHL) is an important component of health literacy that is indicated by various constructs of reading, writing, speaking, listening comprehension, and decision-making. Validity and reliability are essential for the accuracy and precision of any questionnaire-based qualitative study. Our aim was to perform a factor analysis of the OHL-AQ scale among patients of Shimla visiting a tertiary institution, for a better understanding of the OHL-AQ scale generalizability in our population. A cross-sectional study was conducted on patients visiting the outpatient department of public health dentistry. Materials and Methods: Data collection procedure included information obtained from the subject that was recorded on a structured proforma using the Oral Health Literacy-Adults Questionnaire (OHL-AQ) developed by Sistani et al. in 2013. Exploratory factor analysis was done in Statistical Package for the Social Sciences (SPSS) 22, and confirmatory factor analysis was done in Jeffreys’s Amazing Statistics Program (JASP) 0.16.3. Convergent validity was determined from the average variance extracted (AVE), and composite reliability (CR) was determined from the rotation component matrix. Discriminant validity was determined by the heterotrait–monotrait matrix ratio of correlation (HTMT). The confirmatory factor analysis results are based on the categories of model fit with various model indices within the recommended accepted levels. Results: The most important to be considered is the root mean square error of approximation (RMSEA = 0.04). The χ2 value was 82.254 and χ2/df was 1.61 with a P value <0.004, which indicates a good fit. Similarly, the goodness of fit index (GFI) was 0.96. Conclusion: Exploratory factor analysis based on correlation matrix reported adequate construct and discriminant validity for this study. Confirmatory factor analysis based on the covariance matrix provided an adequate model fit within the study data. Thus, the OHL-AQ scale has adequate validity and reliability in our study population, though further studies are indicated in other populations and its operationalization is based on predictive validity.
Keywords: Confirmatory factor analysis, factor analysis, oral health literacy, oral health literacy-adults questionnaire (OHL-AQ), validity
How to cite this URL:
Gurung D, Bhardwaj VK, Fotedar S. Factor analysis of oral health literacy-adults questionnaire (OHL-AQ) among patients attending a tertiary institution of Shimla: A validity and reliability study. Med J DY Patil Vidyapeeth [Epub ahead of print] [cited 2023 Mar 24]. Available from: https://www.mjdrdypv.org/preprintarticle.asp?id=367747 |
| Introduction | |  |
Oral health literacy (OHL) is an important component of health literacy that is indicated by various constructs of reading, writing, speaking, listening comprehension, and decision-making.[1] This could be well understood by Sorenson’s model of OHL as an outcome of an oral health promotion program.[2] A higher level of OHL is associated with better oral health status and quality of life. Literature suggests various assessment tools being used for measuring various constructs of multidimensional OHL.[3] Oral health literacy-adults questionnaire (OHL-AQ) is one such important measurement tool, which is used for measuring OHL for the given determinants of oral health and was developed by Sistani et al.[1] in 2013.
The OHL-AQ consists of 17 items with five constructs: reading, numeracy (two constructs), listening, and decision-making. The addition of listening and decision-making construct has improved the performance and quality of the OHL-AQ instrument. This instrument is also unique in understanding the domains of cognitive functioning and development in OHL. The listening part represents lower cognitive functioning based on perception, numeracy and reading represent moderate cognitive functioning, and finally, decision-making represents higher cognitive functioning. The construct adequately covers the various stages of cognitive functioning, considering OHL passes gradually from the lower to higher stages of cognitive understanding of literacy. Sistani et al.[1] have adequately reported the validity of OHL-AQ, considering the face and content validity, which are theoretical validity, but the empirical validity based on construct validity, particularly the discriminant validity, considering one-way analysis of variance (ANOVA) seems to be critically insufficient for a validity study. With this background, we aim to perform factor analysis of the OHL-AQ scale among patients of Shimla visiting a tertiary institution, for a better understanding of the OHL-AQ scale generalizability in our population.
| Materials and Methods | | |
The source of data for this descriptive, qualitative study was patients visiting the outpatient department of public health dentistry. The sampling technique used was nonprobability convenience sampling. As there is no consensus on a specific number of sample size, we used the sample size approach advocated by Comrey and Lee,[4] which is as follows: 100 = fair, 200 = good, 500 = very good, and >1000 = excellent. A conservative approach was considered, and a total sample of 350 subjects was taken. Inclusion criteria included adults aged between 18 and 65 years who visited the outpatient department of the tertiary institute and were willing to participate in the study. Exclusion criteria were 1) subjects not willing to participate in the study and 2) subjects with any physical or medical condition that did not permit participation.
The information obtained from the subject was recorded on a structured proforma, which was self-administered, except for the listening part, which was read out to the patient. Data collection was done for a period of 3 months w.e.f. 04/01/2022 to 06/31/2022. Correct answer was scored as “1” and incorrect answer as “0,” with the OHL total scores ranging from 0 to 17. The OHL-AQ scores were categorized into the three groups – inadequate (0–9), marginal (10–11), and adequate (12–17) OHL.[1] Permission was obtained from the competent higher authority. Written informed consent was obtained from all participants for the present study, which was voluntary and anonymous.
Statistical analysis
The cutoffs for the floor and ceiling effects were set at 5% of the total score. As a consequence, scores below 2 points and scores above 16 points on the OHL-AQ scale were determined as a floor and/or ceiling effect, respectively. The floor and ceiling effects were considered to be significant if more than 20% of the respondents fell outside the lower or upper bound, respectively.[5] An exploratory factor analysis (EFA) was performed using a principal component analysis, and varimax rotated component matrix was obtained using the Statistical Package for the Social Sciences (SPSS version 22 for Windows; SPSS Inc., Chicago, IL, USA). The overall significance of the correlation matrix found through Bartlett’s test of sphericity was significant. The Chi-square statistical value was 938.45 at a significance level of P < 0.001, indicating its suitability for factor analysis. The Kaiser–Meyer–Olkin (KMO) measure of sample adequacy indicated the appropriateness of the data for factor analysis was 0.78, which is close to the criterion of KMO >0.8. The above-said two criteria are essential to run the factor analysis. The communality of the scale, which indicates the amount of variance in each dimension, was also assessed to ensure acceptable levels of explanation. The minimum factor-loading criterion was set to 0.50.
Convergent validity was determined from the average variance extracted (AVE) and composite reliability (CR) from the rotation component matrix calculated in Excel separately, considering λ lambda (factor loading), λ2, ε epsilon, or variance error (1 − λ2). The criteria for convergent validity were set as AVE greater than 0.45 and CR greater than 0.7.[6] The AVE represents the average amount of variance that a construct explains in its latent variables relative to the overall variance of its latent variable. The discriminant validity had been determined in the past by assessing the maximum shared variance (MSV) and the average shared squared variance (ASV), both lower than the AVE for all of the factors, and is the basis for the Fornell and Larcker criteria.[7],[8] Henseler et al.[9] have questioned the sensitivity of the Fornell and Larcker criteria and have proposed an approach based on multitrait–multimethod matrix to assess the discriminant validity, called as heterotrait–monotrait matrix ratio of correlation (HTMT). This is the ratio of between-trait correlation and within-trait correlation of two constructs. These HTMT criteria were used to assess the discriminate validity, based on the interitem correlation matrix, and the HTMT ratio was separately calculated in Excel. An HTMT value below 0.9 shows discriminant validity has been established between the constructs. The internal consistency or homogeneity of reliability with Cronbach’s alpha was 0.71, which was significant at P < 0.001.[10]
The EFA was further validated using confirmatory factor analysis (CFA) using Jeffreys’s Amazing Statistics Program (JASP; version 0.16.3). Standardized estimate was obtained with three factors, and two factors were excluded due to cross-loading. Items excluded due to cross-loading were the items 1, 6, 7, 16, and 17, and exclusion was important for running the CFA and it did not affect the integrity of the OHL-AQ questionnaire. For an easy understanding, the results are shown based on the categories of model fit with various model indices within the recommended accepted levels for the model fit: Bagozzi et al.[11] for P value, Hu et al.[12] for root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR), Hair et al.[6] for goodness of fit index (GFI), Bentler et al.[13] for comparative fit index (CFI), Tucker–Lewis index (TLI), and Bentler–Bonett normed fit index (NFI), and Schumacker et al.[14] for χ2/df <3 criteria.
| Results | | |
The floor and ceiling effect was reported to be 1.42% and was not significant for the given set criteria. The factor solution derived from the analysis showed five factors for the scale, which accounted for 51.92% of the variation in the data. Further, the scree plot in [Graph 1] shows the subjective measure to visually isolate the elbow point at item 5, after which there is a linear descending trend of Eigen value. All communalities in the analysis, though not shown, were above 0.50, except for items 8, 9, and 13, which had values close to 0.40.
[Table 1] shows the rotated component matrix of five factors extracted and the factor loading of each item in the factor. Only item 6 in factor 1 and item 17 in factor 5 did not load to the factor as per the set criteria, but were sufficiently close. The representation of factor loading of items in [Table 1] is completely based on the arrangement of items in the OHL-AQ questionnaire. This was done completely to maintain the integrity of the original OHL-AQ questionnaire. | Table 1: Rotated component matrix extracted from principal component analysis with varimax rotation and Kaiser normalization
Click here to view |
[Table 2] shows the assessment of convergent and discriminant validity. The AVE of numeracy 1 and 2 factors was below the set criterion of AVE >0.45, but the CR of reading and decision-making construct was above the set criterion of CR >0.7 and the rest of the construct was sufficiently close to the set criteria. For the discriminant validity, the ASV and the MSV were lower than the AVE of the entire factors as per the Fornell and Larcker criteria.
[Table 3] shows the assessment of discriminant validity based on the HTMT ratios of all the inter-constructs, which were lower than the set criterion of HTMT <0.9, except for N1–N2 (numeracy 1–numeracy2), N2–L (numeracy 2–listening), and R–DM (reading–decision-making).
[Table 4] shows the model fit considering various model fit indices, and all the models fitted the data adequately within the recommended accepted levels. The P value was significant, which is mainly due to the large sample size. The most important to be considered is RMSEA (=0.04). The χ2 value was 82.254 and χ2/df was 1.61 with a P value <0.004, which indicates a good fit. Similarly, the GFI was 0.96. | Table 4: Confirmatory analysis with model fit summary based on various model fit indices
Click here to view |
| Discussion | | |
This is a comprehensive study on the generalizability of the psychometric measurement of the OHL-AQ in the population of Shimla, considering the construct validity and reliability. The OHL-AQ is an important OHL scale consisting of the life approach concept of literacy. The OHL-AQ comprehensively covers various dimensions based on the life course of cognitive development in OHL. The present study showed no floor and ceiling effect, indicating good content validity. Sistani et al.[1] had reported a content validity index (CVI) of 0.90 and a content validity ratio (CVR) of 0.85. The Cronbach’s alpha for this study was found to be 0.71 (unstandardized), which suggests good internal consistency. This was also reported by Sistani et al.,[1] Vyas et al.,[15] and Pattnaik et al.[16] in their studies.
The factor analysis of OHL-AQ also extracted five factor structures. The factor loading of the item in the given study was adequate as per the set criteria. The convergent and discriminant validity was determined through the correlation matrix. The AVE of the factors reading, listening, and decision-making was adequate, but that of numeracy 1 and 2 was inadequate, though the AVE of combined numeracy was also calculated. This was also true for the CR criteria, which were adequate for other factors, but not for numeracy 2. The probable reason could be due to the overlapping characteristic factors of numeracy 1 and 2. The discriminant validity was adequate based on the set criteria of MSV, average shared variance, and AVE, indicating good discriminant validity among the five factors considered in the OHL-AQ as per the Fornell and Larcker criteria. Henseler et al.’s[9] criteria for discriminant validity was adequate for all the factors. but showed deviation from the set criteria for the factors of numeracy 1 and 2. These are certainly good adjuncts to determine the discriminant validity, though Sistani et al.[1] reported this validity based on one-way ANOVA. Validity studies are important in understanding the internal as well as the external generalizability of a population-based study.
The confirmatory analysis reported adequate model fit within the acceptance levels of various model fit indices, considering three factors. The exclusion of even two factors did not affect the model fit of our study data. The OHL-AQ adequately measured the life course of OHL from lower to higher cognitive development skill in the study.
The strength of the study was firstly, the use of adequate statistical technique that comprehensively covers the important validity of the given factors of OHL, and this is lacking in other studies. Secondly, there was no missing data and there were equal number of males and females, even though our sampling technique in the study was nonprobability convenient sampling. Thirdly, the sample size of our study was adequate for running the factor analysis, and exclusion of some factors due to cross-loadings did not affect confirmatory analysis.
The limitation of our study is the use of nonprobability convenience sampling technique, which limits its generalizability. The criterion validity, in particular, the predictive validity, was not considered and further studies are necessary in this regard.
| Conclusion | | |
Validity and reliability are essential for the accuracy and precision of any questionnaire-based qualitative study. The OHL-AQ scale was found to have good validity and internal consistency with adequate reliability. The factor analysis based on correlation matrix reported adequate convergent and discriminant validity for this study. The CFA based on covariance matrix provided adequate model fit within the study data. Thus, the OHL-AQ scale has adequate validity and reliability in our study population, though further studies are indicated in other populations and its operationalization is based on predictive validity.
Declaration of patient consent
The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.
Acknowledgement
I would like to thank all the participants, Madam Kusum Chopra (statistician) from Chandigarh for doing factor analysis, and Mr. Savitesh Khuswaha, PhD Research Scholar, Department of Community Medicine, PGIMER, Chandigarh.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Sistani MMN, Montazeri A, Yazdani R, Murtomaa H. New oral health literacy instrument for public health: Development and pilot testing. J Investig Clin Dent 2014;5:313-21.  |
| 2. | Sorensen K, den Broucke SV, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health 2012;12:80-92. |
| 3. | Dickson-Swift V, Kenny A, Farmer J, Gussy M, Larkins S. Measuring oral health literacy: A scooping review of existing tools. BMC Oral Health 2014;14:148. |
| 4. | Comrey AL, Lee HB. A First Course in Factor Analysis. Hillsdale, NJ: Erlbaum; 1992. |
| 5. | Khapre M, Dhande N, Mudey A. Validity and reliability of marathi version of edinburgh postnatal depression scale as a screening tool for post natal depression. Ntl J Community Med 2017;8:116-21. |
| 6. | Hair J, Black W, Babin B, Anderson R. Multivariate Data Analysis. 7 th ed. Upper Saddle River, NJ, USA: Prentice-Hall Inc.; 2010. |
| 7. | Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 1981;18:39-50. |
| 8. | Alumran A, Hou XY, Sun J, Yousef AA, Hurst C. Assessing the construct validity and reliability of the parental perception on antibiotics (PAPA) scales. BMC Public Health 2014;14:73. |
| 9. | Henseler J, Ringle CM, Sarstedt M. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 2015;43:115-35. |
| 10. | Nunnally JC, Bernstein IH. Psychometric Theory. 3 rd ed. New York: McGraw-Hill; 1994. |
| 11. | Bagozzi RP, Yi Y. On the evaluation of structural equation models. JAMS 1988;16:74-94. |
| 12. | Hu Li-tze, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Struct Equ Model 1999;6:1-55. |
| 13. | Bentler PM. Comparative fit indexes in structural models. Psychol Bull 1990;107:238-46. |
| 14. | Schumacker RE, Lomax RG. A beginner’s Guide to Structural Equation Modeling. 2 nd ed. Lawrence Erlbaum Associates Publishers; 2004. |
| 15. | Vyas S, Nagarajappa S, Dasar PL, Mishra P. Linguistic adaptation and psychometric evaluation of original Oral Health Literacy-Adult Questionnaire (OHL-AQ). J Adv Med Educ Prof 2016;4:163-9. |
| 16. | Pattanaik S, John MT, Kohli N, Davison ML, Chung S, Self K, et al. Item and scale properties of the Oral Health Literacy Adults Questionnaire assessed by item response theory. J Public Health Dent 2020;81:214-23. |
[Table 1], [Table 2], [Table 3], [Table 4]
|
