Review Article

Psychometric properties of the Flourishing Scale for South African first-year students

Karina Mostert, Leon T. de Beer, Ronalda de Beer

Received: 10 Nov. 2022; Accepted: 29 Jan. 2023; Published: 24 Mar. 2023

Copyright: © 2023. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This study focused on a positive construct of well-being, namely flourishing. In a multicultural and diverse country such as South Africa, it is a legal requirement to provide evidence that measures of psychological constructs, like flourishing, are fair, unbiased, and equivalent for diverse groups in the country. The aim was to test the psychometric properties of the Flourishing Scale, a purpose-made scale that measures positive functioning across various areas of life. This study tested the factorial validity, item bias, measurement invariance and reliability of the Flourishing Scale in a sample of 1088 South African first-year university students. A unidimensional structure was confirmed. Although three items showed statistically significant uniform and total bias for language and campus groups, the magnitude and practical impact were negligible. No evidence of bias across gender groups was found. Configural, metric and partial scalar invariance were established for language and campus groups. Full measurement invariance was established across gender groups. Cronbach’s alpha coefficient was 0.91, indicating high reliability. The study provided promising results for using the Flourishing Scale among South African university students to measure flourishing as an aspect of well-being.

Contribution: This study contributes to the field of student well-being in South Africa. No studies could be found that test for item bias or measurement invariance of the Flourishing Scale, specifically for South African first-year students. This study is the first to test these psychometric properties of a Flourishing Scale in a multicultural setting for students from different languages.

Keywords: flourishing; factorial validity; item bias; differential item functioning; measurement invariance; internal consistency; first-year students; university.

Introduction

It is well established that first-year students face various challenges when transitioning from secondary to tertiary education (Kelly & Finlayson 2016; Nair & Fisher 2000; Van Zyl 2016). As students are often far from their loved ones, they feel alone, isolated, and stressed (Eagan et al. 2015). As a result, transitioning to higher education and adjusting to all the unfamiliar challenges encountered during the first year can negatively affect students’ well-being (Eagan et al. 2015; Vuckovic, Riley & Floyd 2019). However, it is also essential to identify and support students who are doing well and provide resources to help them flourish. The idea of flourishing has emerged as a critical component of subjective well-being (Diener et al. 2010). High levels of positive feelings characterise flourishing – the sense that one has a purpose in life, fosters positive relationships with others, cultivates optimism and strengthens high self-esteem (Diener et al. 2010). Flourishing also refers to a person’s knowledge of their life or how well they believe it to be and is linked to hedonic and eudemonic well-being (Keyes 2002).

Knowledge of students’ levels of flourishing could help Higher Education Institutions (HEIs) to motivate students to make an effort to achieve their academic objectives, enhance their welfare, and help train productive employees (Botha, Mostert & Jacobs 2019). The flourishing of first-year students is essential to HEIs, as this affects the process of graduation and their readiness to work (Jayawickreme & Dahill-Brown 2016; Schneiderman, Ironson & Siegel 2005).

Diener et al. (2010) developed a psychometric scale, the Flourishing Scale, to answer the need for a purpose-made scale to measure psychological flourishing. Although the scale does not give distinct metrics of different aspects of flourishing, it provides an overview of positive functioning across various areas in life generally perceived to be significant. The scale measures universal human psychological needs, meaning and purpose in life, optimism, and feelings of competence (Diener et al. 2010). This scale can be a valuable tool for HEIs to identify students’ flourishing levels to develop effective interventions to enhance levels of understanding and learn from students who are doing well at university who are thriving.

It is crucial to use scales that prove to be psychometrically sound. In a multicultural and diverse country such as South Africa, it is vital to test measures of psychological constructs to ensure they are fair, unbiased and equivalent for all ethnicities, languages, and other diverse groups in South Africa. South African law requires evidence that tests are appropriate, impartial and unbiased. This is stipulated in the Employment Equity Act 55 of 1998, Section 8 (Government Gazette 1998), which states that any form of psychological tests or similar assessments are prohibited unless the test or assessment being used is valid and reliable, can be applied fairly to all employees, and is not biased or discriminating against any employee or group.

The more rigorous testing of measures in diverse contexts are, it is not only applicable to South Africa, but also to other countries with diverse student populations. With the increasing migration and globalisation, many countries have become more diverse and multicultural (Van De Vijver & Rothmann 2004). It is also true for HEIs, where there is an influx of international students who need support (McKay, O’Neill & Petrakieva 2018). Multicultural testing is therefore of interest to other diverse settings, including student populations.

Central to multicultural assessment is bias and equivalence concepts (Van De Vijver & Rothmann 2004). Bias refers to certain nuisance factors that impede the comparability of test scores. Equivalence testing ensures the comparability of test scores across cultures or groups. When test scores are free of bias and demonstrate equivalence (or invariance), the scores can be compared across cultures or different sub-groups. Of particular interest are item bias and measurement invariance. Item bias (also referred to as differential item functioning [DIF]) occurs when respondents from different groups score differently on the item, even though they have the same standing on the underlying construct. Familiar sources of item bias include: differential response styles, poor item translation and ambiguous items, and the connotative meaning and appropriateness of the item content based on cultural specifics. Measurement invariance has: (1) configural invariance (the extent to which a factor structure can be replicated across groups), (2) metric invariance (equal factor loadings for similar items across groups), and (3) scalar invariance (similar meaning or interpretation for different groups) (Laher 2008; Van De Vijver & Rothmann 2004).

In addition, confirmatory factor analysis (CFA) and internal consistency (Cronbach’s coefficient alpha) were used to test the factor structure and reliability of the Flourishing Scale. Concerning factorial validity, the scale has a one-factor structure (Didino et al. 2019; Duan & Xie 2019; Muñoz & Nieto 2019; Singh, Junnarkar & Jaswal 2016), also in student samples (Hone, Jarden & Schofield 2014; Senol-Durak & Durak 2019; Sumi 2014). Many studies have shown that the Flourishing Scale has a high level of internal consistency, with Cronbach’s alpha coefficients ranging from 0.80 to 0.91 (Choudhry et al. 2018; Didino et al. 2019; Muñoz & Nieto 2019; Singh et al. 2016).

No studies could be found that test for item bias or measurement invariance of the Flourishing Scale, specifically for South African first-year students. Therefore, this study aims to provide psychometric evidence for the applicability of the Flourishing Scale in the diverse context of a South African university. More specifically, this study tested the factorial validity, item bias, metric, scalar and configural invariance, and internal consistency of the scale among first-year university students.

Methods

Participants

The study’s target demographic group was first-year university students enrolled at a South African university. A sample of 1088 participants was used, of which 72.4% were between the ages of 17 and 20 years and 16.7% were between 21 and 22 years. South Africa has 11 official languages distributed in different parts of the country. The languages most frequently used by students of the participating university were included in the analyses: Afrikaans (260, 23.9%), Setswana (199, 18.3), Sesotho (152, 14.0%) and English (94, 8.6%). The university has three campuses: Campus 1 is a campus located in a peri-urban area (131, 12%), Campus 2 is located in a medium-sized urban city (478, 43%), and Campus 3 is a smaller campus located in a large industrial city. In total, 689 (63.3%) females and 319 (29.3%) males participated in the study. Most participants were black students (62.3%), followed by white students (22.2%).

Instrument

The Flourishing Scale (Diener et al. 2010) is a concise eight-item measure of respondents’ self-perceived performance in critical life domains such as relationships, self-esteem, intention, and optimism. A 7-point Likert scale was used, ranging from 1 (strongly disagree) to 7 (strongly agree). An example item is: ‘I lead a purposeful and meaningful life’. A high score indicates that the individual possesses psychological resources and strengths. The scale showed good psychometric qualities. The Cronbach’s alpha coefficient is reported as 0.82 (Diener et al. 2010).

Procedure

The participating university accepted and authorised the project, and the study was granted ethics clearance. A secure direct link to the questionnaire was put on the university’s online portal. Throughout the study’s duration, students were informed about the research and encouraged to participate voluntarily. This was accomplished through field workers who presented brief awareness sessions in classrooms. Before completing the questionnaire, participants were required to sign an informed consent form. Furthermore, participants were assured that their reported responses would be anonymous, that the data gathered in the study would adhere to the project’s confidentiality criteria, and that the findings would be carefully stored in a secure database that would be password protected.

Data analysis

MPlus 8.6 (Muthén & Muthén 2021) was used to conduct the statistical analyses. Confirmatory factor analysis was used to test the factorial validity of the Flourishing Scale. Maximum likelihood estimation was used, with the covariance matrix as input. The following fit indices were considered to assess the fit of the measurement model: the χ² statistic, the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardised root mean square residual (SRMR). Proper fit is considered at a value of 0.90 and above for the CFI and TLI (Byrne 2001; Hoyle 1995). For the RMSEA, a value of 0.05 or less indicates a good fit, whereas values between 0.05 and 0.08 are considered an acceptable model fit (Browne & Cudeck 1993; Chen et al. 2008).

Differential item functioning was used to test for the presence of item bias for language (four of the languages most frequently used by students at the participating university: Afrikaans, Setswana, Sesotho and English), campus (the three campuses described here) and also included males and females. Two forms of bias were tested: uniform and non-uniform bias. Uniform bias refers to the systematic difference in ability levels of the underlying construct between compared groups (Swaminathan & Rogers 1990; Teresi & Fleishman 2007). Non-uniform bias is the difference in the likelihood of related answers across different groups fluctuating across all ability levels (Swaminathan & Rogers 1990; Teresi & Fleishman 2007). The lordif package (Choi, Gibbons & Crane 2011) in RStudio Team (2020) was used. The following formulas were used and compared with test for uniform and non-uniform bias, using ordinal logistic regression to generate three likelihood-ratio χ² statistics (Choi et al. 2011):

Biased items are flagged when statistically significant differences are detected, that is when the log-likelihood values of models are compared and p < 0.01; for uniform bias when comparing Models 1 and 2 , for non-uniform bias when comparing Models 2 and 3 ; for a total DIF effect, comparing Models 1 and 3 (Choi et al. 2011). The pseudo-Mcfadden R² statistic is used to quantify the impact or practically significant effect of DIF, classifying the magnitude of DIF as negligible (< 0.13), moderate (between 0.13 and 0.26), or large (> 0.26) (Zumbo 1999). In addition, the impact of uniform DIF can be determined using the β₁ coefficient when Models 1 and 2 are compared (Crane, Van Belle & Larson 2004). Different thresholds, ranging from a 10% difference between Models 1 and 2, indicate a practically meaningful effect (Crane et al. 2004; Maldonado & Greenland 1993).

Measurement invariance was investigated for the same language, campus, and gender groups. This was carried out in a multigroup analysis framework including the: (1) configural invariance model (i.e. the baseline model for the more constrained models and the test if a similar underlying latent factor is evident in the different groups); (2) metric invariance model (assumes the invariance or similarity of the factor loading in the different groups); and (3) scalar invariance model (test if the factor loadings and item intercepts are invariant or similar in the different groups) (Preti et al. 2013). The CFI and RMSEA values were used. For CFI, the fit is considered adequate if values are > 0.90 and better if they are > 0.95. For RMSEA, the cut-off value is < 0.08, but better is < 0.05 (Van De Schoot, Lugtig & Hox 2012). In addition, changes in CFI were used as recommended by Shi et al. (2019). A ΔCFI value higher than 0.01 between two nested models indicates that the added group constraints have led to a poorer fit; in other words, the more constrained model is rejected. By freeing the loading of items, partial metric invariance can be achieved (Cheung & Rensvold 2002; Preti et al. 2013). Cronbach’s alpha coefficient was used to determine the reliability of the scales. A cut-off point of 0.70 is deemed satisfactory (Nunnally & Bernstein 1994).

Ethical considerations

The study was approved by the Ethics Committee, Faculty of Economic and Management Sciences (EC-EMS) (Ethics no.: NWU-HS-2014-0165-A4). Before completing the questionnaire, participants were required to sign an informed consent form. In addition, participants were assured that their reported responses would be anonymous, that the data gathered in the study would adhere to the project’s confidentiality criteria, and that the findings would be stored in a secure database that is password protected.

Results

Factorial validity

With regard to the factorial validity of the Flourishing Scale, a one-factor structure showed a good fit to the data (χ² = 180.11; df = 19; CFI = 0.94; TLI = 0.91; RMSEA = 0.079; SRMR = 0.04). The standardised loadings are shown in Table 1.

All items had high factor loadings (λ) (Shevlin et al. 1998), ranging from 0.65 (Item 8) to 0.80 (Item 1).

Item bias (differential item functioning)

Uniform, non-uniform and total bias were tested (see Table 2).

Items 2, 3 and 7 showed statistically significantly uniform and total bias for the included language and campus groups, while no bias was detected between males and females. To determine if the magnitude of DIF for these three items were of practical significance, pseudo-McFadden R² values and the difference in the β1 coefficient were inspected. In addition, visual graphs are provided for each item to demonstrate the effect between language and campus groups (Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6). Each of these figures present four graphs providing additional diagnostic information, including the item characteristic curve for the different groups (in this case, language and campus groups; upper-left graph); the item response functions for the parameter estimates for each group (lower-left graph); the absolute difference between item characteristic curves for sub-groups (upper-right graph); and the absolute difference between the item characteristic curves of the sub-groups weighted by the score distribution (Choi et al. 2011).

FIGURE 1: Graphical display of Item 2, which shows uniform and non-uniform differential item functioning with respect to language groups. (a) Items True Score Functions - item 2; (b) differences in items True Score Functions; (c) Item Response Functions; (d) Impact (weighed by density).

FIGURE 2: Graphical display of Item 3, which shows uniform and non-uniform differential item functioning with respect to language groups. (a) Items True Score Functions - item 3; (b) differences in items True Score Functions; (c) Item Response Functions; (d) Impact (weighed by density).

FIGURE 3: Graphical display of Item 7, which shows uniform and non-uniform differential item functioning with respect to language groups. (a) Items True Score Functions - item 7; (b) differences in items True Score Functions; (c) Item Response Functions; (d) Impact (weighed by density).

FIGURE 4: Graphical display of Item 2, which shows uniform and non-uniform differential item functioning with respect to campuses. (a) Items True Score Functions - item 2; (b) differences in items True Score Functions; (c) Item Response Functions; (d) Impact (weighed by density).

FIGURE 5: Graphical display of Item 3, which shows uniform and non-uniform differential item functioning with respect to campuses. (a) Items True Score Functions - item 3; (b) differences in items True Score Functions; (c) Item Response Functions; (d) Impact (weighed by density).

FIGURE 6: Graphical display of Item 7, which shows uniform and non-uniform differential item functioning with respect to campuses. (a) Items True Score Functions - item 7; (b) differences in items True Score Functions; (c) Item Response Functions; (d) Impact (weighed by density).

For all three items in language and campus groups, the differences between language and campus groups were slightly different compared with each other; however, these differences were negligible, as can be seen in the density-weighted impact in each figure (bottom right plots). Also, the pseudo-McFadden R² statistic values were all smaller than 0.13 and the difference in β₁ coefficients smaller than 5%. As a result, DIF’s magnitude or practical impact on these three items can be classified as negligible.

Measurement invariance

The results of the configural, metric and scalar invariance testing across the language, campus, and gender groups included in this study are shown in Table 3.

With regard to language and campus, configural and metric invariance were established. The results of scalar invariance showed that ΔCFI for language was –0.024 and for campus –0.018 (higher than 0.01). Consequently, partial scalar invariance was established, releasing the intercept of items 4 and 7 in the Afrikaans and English language groups and items 3 and 7 in all three campus groups. Configural, metric and scalar invariance was confirmed for gender.

Internal consistency

As a measure of internal consistency, Cronbach’s alpha coefficient was calculated to establish the internal consistency of the Flourishing Scale. With α = 0.91, the Flourishing Scale was found to be reliable (Nunnally & Bernstein 1994).

Discussion

This study aimed to test the psychometric properties of the Flourishing Scale to determine if this scale is valid and reliable for assessing flourishing, a positive construct of psychological well-being, in South African first-year university students. The study’s primary objective was to determine the factorial validity, item bias, metric, scalar and structural invariance, and internal consistency.

Concerning the factorial validity, the results showed that a one-factor structure was a good fit for the data. The findings are consistent with previous studies, where a one-factor structure was confirmed in student samples from New Zealand, Turkey, and Japan (Hone et al. 2014; Senol-Durak & Durak 2019; Sumi 2014).

Differential item functioning was used to determine uniform and non-uniform bias. Statistically significant uniform and total bias were found across language and campus groups for items 2, 3 and 7. However, the magnitude or practical impact of this bias was negligible. This means that, on a practical level, the language, campus, and gender sub-groups included in this study understood the items identically across groups, and that no incongruities at the item level exist for participants in these sub-groups (Cleary & Hilton 1968; Van De Vijver & Tanzer 2004).

Regarding measurement invariance, configural invariance was established for all included sub-groups. The results show that the one-factor structure of the Flourishing Scale has the same pattern and fits the data equally well in all groups. Therefore, the factor structure can be replicated similarly for different language, campus and gender groups (Byrne, Shavelson & Muthén 1989; Putnick & Bornstein 2016). Metric invariance was also established for all sub-groups, indicating that the loading of each item contributes equally to the latent construct of flourishing across the different groups. Although scalar invariance was confirmed for gender, only partial scalar invariance was established for language and campus groups because of the ΔCFI values higher than 0.01 (Cheung & Rensvold 2002; Preti et al. 2013). This implies that specific item intercepts were not equivalent between language and campus groups. As a result, the intercepts of items 4 and 7 of two language groups (i.e. Afrikaans and English) and items 3 and 7 in all three campus groups had to be released to establish partial invariance. Even though these parameters can vary across groups, valid inferences can still be made when at least two intercepts and factor loadings are equally constrained, which is in line with the findings of previous studies (Laguna et al. 2017; Van De Schoot et al. 2012).

The Cronbach’s alpha coefficient was calculated to determine the internal consistency of the Flourishing Scale and showed a reliability coefficient of 0.91. Various research studies have found that the Flourishing Scale has a high level of internal consistency, with Cronbach’s alpha coefficients ranging from 0.80 to 0.91 (Choudhry et al. 2018; Didino et al. 2019; Muñoz & Nieto 2019; Singh et al. 2016).

Limitations and recommendations

Even though the findings of this study are promising, several limitations must be mentioned. The study’s primary focus was on first-year university students in South Africa. Therefore, the study should be replicated for senior students, other universities, and other countries with multicultural populations. South Africa has 11 official languages, of which only 4 were included in this study. Other language groups should also be included in future studies. Three items seemed to be somewhat problematic (items 3, 4 and 7) regarding bias and invariance. Even though the practical effect was small and negligible, future studies should investigate how these items function in other samples.

Conclusion

This study provides initial support for using the Flourishing Scale in a South African sample of first-year university students and opens the way for its further use in other student samples. The scale demonstrated high reliability, and the DIF and invariance analyses confirmed that no practically significant incongruities exist between language, campus, and gender groups.

Acknowledgements

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

K.M. conceived of the presented idea and supervised the study. K.M. and L.T.d.B. verified the analytical methods. L.T.d.B. conducted the statistical analyses. R.d.B. wrote the original draft and K.M. supervised the study while reviewing and editing the manuscript. L.T.d.B. assisted with the interpretation of the results. K.M. provided necessary resources and acquired the funding for the project. All authors discussed the results and contributed to the final manuscript.

The material described in this article is based on work supported by: (1) the office of the Deputy Vice-Chancellor: Teaching and Learning at the university.

Derived data supporting the findings of this study are available from the corresponding author, K.M., on request.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors and the publisher.

References