Original Research

The applicability of the UCLA loneliness scale in South Africa: Factor structure and dimensionality

Tyrone B. Pretorius

Received: 18 June 2021; Accepted: 11 Oct. 2021; Published: 10 Jan. 2022

Copyright: © 2022. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This study examines the generalisability of the University of California Los Angeles Loneliness Scale Version 3 (UCLA-LS3) in a South African sample of young adults. In particular, it examined the normative data, reliability, and factor structure of this scale. The participants were young adults (N = 337) who were randomly sampled from a university population and they responded to the UCLA Loneliness Scale. It was found that the sample had higher loneliness scores than those reported in the literature, potentially suggesting that loneliness may be a significant mental health concern amongst this group. Women reported higher levels of loneliness than men. Reliability analysis (Cronbach’s alpha) and analysis of the influence of individual items on the mean, variance, and alpha demonstrated that UCLA-LS3 had highly satisfactory internal consistency in the sample. Confirmatory factor analysis (CFA) was used to test four conceptualisations of the factor structure of UCLA-LS3: a one-factor model, a correlated three-factor model, a bifactor model with two subscales, and a bifactor model with three subscales. Notably, CFA demonstrated that the two bifactor models are a better fit than the one-factor and correlated three-factor models and that the bifactor model with three subscales is marginally a better fit than the bifactor model with two subscales. Ancillary bifactor analysis confirmed the dimensionality of the scale as sufficient variance was accounted for by the three subscales, after the variance attributable to the total scale was partitioned out. Therefore, UCLA-LS3 is best conceptualised as comprising of three subscales (isolation, relational connectedness, collective connectedness), in addition to a total scale.

Keywords: bifactor; COVID-19; loneliness; UCLA-LS3; reliability; South Africa.

Recently, the onset of the novel Corona Virus Disease 2019, referred to as COVID-19, and its rapid outbreak led to a global public health crisis (Luchetti et al., 2020). In the absence of a vaccine or cure, the World Health Organization (WHO) recommended social distancing, self-isolation, and quarantine to slow the spread of the virus and in this way reduce the burdens imposed on healthcare systems (WHO, 2020).

Loneliness has been identified as one of the most salient mental health consequences of the COVID-19 pandemic and the prevention measures resulting from it, like stay-at-home directives, limited in-person contact with family and friends, self-isolation and quarantine, etc. (Killgore, Cloonen, Taylor, & Dailey, 2020; Rosenberg, Luetke, Hensel, Kianersi, & Herbenick, 2020). Loneliness is defined as the discrepancy between one’s actual levels of interaction with others and the desired levels thereof (Perlman & Peplau, 1981). It is a subjective and distressing emotional state characterised by the perception of social isolation and sense of feeling alone. Even prior to COVID-19, loneliness was a risk factor for adverse physical and mental illnesses, including depression, cardiovascular disease, Alzheimer’s disease, and other problems such as suicide, lower life satisfaction, reduced work performance and mortality (Chang et al., 2017; Chiao, Chen, & Yi, 2019; Maguire, Hanly, & Maguire, 2019). Hence, the assessment of loneliness has been identified as a significant public health issue (Killgore et al., 2020). According to evolutionary theory, loneliness serves as a signal to reconnect with significant others (Luchetti et al., 2020). However, in the context of a pandemic characterised by self-quarantine and stay-at-home directives, such reconnection may not be possible. This is even more so in developing contexts where access to digital technologies that assists with social connectedness are not affordable or available to a large part of the population (Padfmanabhanunni & Pretorius, 2021). In such circumstances, feelings of loneliness may become aggravated and evoke further distress (Luchetti et al., 2020).

The most widely used self-report measure for the assessment of loneliness amongst adults and adolescents is the UCLA Loneliness Scale (UCLA-LS: Russell, 1996; Russell, Peplau, & Ferguson, 1978; Russell, Peplau, & Cutrona, 1980). The UCLA-LS operationalises loneliness as an undifferentiated unitary state that varies in intensity and arises from perceived deficits in social relationships (Russell, 1996). Three versions exist for this scale. The original version (Russell et al., 1978) contained 20 negatively worded items that assessed an individual’s perceived experience of loneliness. Because all items were negatively worded, it produced systematic bias in responses. A revised version of the scale was then developed (Russell et al., 1980) that contained 10 negatively worded and 10 positively worded items. However, studies involving exploratory factor analysis (EFA) reported a variety of factor solutions including one-factor (Pretorius, 1993), two-factor with positive and negative items (Mahon et al., 1995), three-factor (Dussault, Fernet, Austin, & Leroux, 2009) as well as four- and five-factor structures (e.g. Neto, 1992). These inconsistencies limited the utility of the scale as an assessment tool and prompted a revision (Russell, 1996).

The third version of the scale (UCLA-LS3: Russell, 1996) contains 11 negative and 9 positive items. The internal consistency, test-retest reliability and discriminant and convergent validity of the UCLA-LS3 has been supported by several studies (e.g. Auné, Abal, & Attorresi, 2020; López-Ramos, Navarro-Pardo, Fernández Muñoz, & Da Silva Pocinho, 2017). Further confirming its utility over the two previous versions, the UCLA-LS3 has been adapted and validated in various cultural contexts and countries (e.g. France, Germany, Greece, Japan, and Russia: Perlman & Peplau, 1998; Argentina: Auné et al., 2020; Saudi Arabia: AlNajjar & Dodeen, 2017; Spain: Sancho, Pinazo-Hernandis, Donio-Bellegarde, & Tomás, 2020; Poland: Kwiatkowska, Rogoza, & Kwiatkowska, 2018; Iran: Zarei, Memari, Moshayedi, & Shayestehfar, 2016; Portugal: López-Ramos et al., 2017; Palestine: Nazzal, Cruz, & Neto, 2018).

With reference to the scale’s dimensionality, Russell (1996) maintained that the UCLA-LS3 has a unidimensional structure. However, one-factor (e.g. Lasgaard, 2007), two-factor (e.g. Dodeen, 2015), three-factor (e.g. Hawkley, Browne, & Cacioppo, 2012; Shevlin, Murphy, & Murphy, 2014), and bifactor solutions (e.g. Auné et al., 2020) have been reported thus suggesting a possible multidimensional factor structure. This multidimensionality implies that loneliness is a complex experience that involves various types of relationships and interactions. In accounting for these findings, Russell (1996) maintained that the scale measures a unitary state which can be reached via deficits in different relationships and social networks. Nevertheless, the model that has received the most support consists of three factors namely, isolation (i.e. the feelings of isolation that underlie loneliness), social connectedness which refers to the need for meaningful group connectedness and relational connectedness which refers to the need for social contact and close friendships (Hawkley et al., 2012; Shevlin et al., 2014).

Given that loneliness has been identified as a priority mental health problem during the COVID-19 pandemic and the UCLA-LS3 continues to be the most widely used instrument for assessing loneliness, it remains important to understand how the scale performs in different cultural contexts. Responses to psychological scales such as the UCLA-LS3 is influenced by the culture, age and socio-economic circumstances of the respondent (Dodeen, 2014; Durak & Senol-Durak, 2010). Younger people and those from disadvantaged backgrounds may be more susceptible to wording effects (Wouters et al, 2012). Hence, it remains relevant to understand how the scale performs in different populations.

This study aims to provide insights into the manifestation of loneliness amongst a sample of young adults during COVID-19 and to investigate the psychometric properties of the UCLA-LS in this population group. Measuring instruments such as the UCLA-LS3 that are developed in the fields of personality or social psychology typically have a well-defined exploratory factor analytic structure (Boffo, Mannarini, & Munari, 2012). However, they are often not adequately supported by Confirmatory Factor Analysis (CFA) and may not reach minimum standards of fit (Boffo et al., 2012). Research on loneliness in developing contexts like South Africa requires suitable instruments with sound psychometric properties that have cross-national applicability. Heppner, Pretorius, Wei, Lee and Wang (2002) argued that such cross-national applicability of instruments would allow building a more comprehensive knowledge base by searching for psychological universals (i.e. etic approach) and identifying culturally specific constructs, which are useful for identifying and explaining cultural differences (i.e. emic approach). Overall, the research findings on factor structure of the UCLA-LS3 have been equivocal. Hence, this study aims to examine the normative data, reliability, and factor structure of the scale in the South African context.

Method

Participants

The study participants were young adults who were doing undergraduate studies at a university in the province of the Western Cape, South Africa. The study design was cross-sectional in nature, and the sample of young adults (N = 337) were randomly sampled from the university student population. In terms of gender, 77.2% of the sample were female and 22.0% were male, whilst 0.8% self-identified as transgender or binary. The mean age of the sample was 21.95 years (SD = 4.7).

Instruments

In addition to a demographic questionnaire, all the participants completed the UCLA Loneliness Scale (UCLA-LS3; Russell, 1996).

The UCLA-LS3 consists of 20-items scored on a 4-point scale that ranged from I never feel this way (1) to I often feel this way (4). The UCLA-LS3 is purportedly a measure of an individual’s subjective feelings of loneliness and feelings of social isolation. Generally, good internal consistency reliability has been reported for the UCLA-LS3 (α 0.94 to 0.96: Doğan, Çötok, & Tekin, 2011). The UCLA-LS3 has also been previously used in one South African study, which reported a Cronbach’s alpha of 0.77 (Pretorius, 1993).

Procedure

Google forms were used to construct an electronic version of the UCLA-LS3 and the link was distributed to a random sample of students via an email by the Registrar’s office. The participants could access and complete the link between March and June 2021.

Data analysis

In this study, CFA was used to examine four operationalisations of the structure of UCLA-LS3. In CFA, the factors are regarded as latent variables which are represented by the items that are the observed measurements (Bentler, 1995). The four models of the structure of the UCLA-LS3 that were assessed, were a model representing only a total loneliness score (one-factor model), a model representing the structure of the UCLA-LS3 as three correlated subscales (correlated three-factor model), and two models in which the structure of the scale is hypothesised to have both a total scale as well as subscales (bifactor model). The first bifactor model postulates that the UCLA-LS3 consists of a total scale (general factor) with two uncorrelated subscales (specific factors) reflecting the variance amongst clusters of items (Mansolf & Reise, 2017), whereas the second bifactor model conceptualised UCLA-LS3 as consisting of a total scale (general factor) with three uncorrelated subscales (specific factors).

In CFA, the chi-square statistic (χ²) is used to determine whether the proposed model fits the observed data. However, it has been reported that χ² test is very sensitive to the violations of distributional assumptions and is affected by sample size (Jöreskog, Olsson, & Wallentin, 2016), It has, therefore, been recommended (Kline, 2005) that the following fit indices should also be reported: the root-mean-square error of approximation (RMSEA; ≤ 0.08 indicates good fit), comparative fit index (CFI; ≤ 0.90 indicates good fit), and standardised root-mean-square residual (SRMR; ≤ 0.08 indicates good fit). Other indices that are commonly reported include the goodness-of-fit index (GFI; ≥ 0.95 indicates good fit) and Tucker–Lewis index (TLI; ≥ 0.90 indicates good fit; Byrne, 1994; Hu & Bentler, 1999). When models are being compared, it is recommended that indices, which is used to compare models such as the Akaike information criterion (AIC), also be included (Arbuckle, 2012). In general, the model with the lowest AIC value is considered to have the best fit. The CFA analyses were conducted using IBM SPSS Amos (version 26; IBM Corp., Armonk, NY, USA).

It has been pointed out that fit indices do not necessarily address the dimensionality of a scale and may lead to incorrect conclusions about the structure of a scale (Pretorius, 2021). We used the Bifactor Indices Calculator (Dueber, 2017) to calculate additional bifactor measures to address the dimensionality of the UCLA-LS3. These measures include (1) explained common variance (ECV), which refers to the proportion of variance accounted for by the specific factor; (2) McDonald’s omega, which is regarded as an alternative to coefficient alpha, as an estimate of reliability (omegaS for subscales); and (3) omega hierarchical (omegaH), which reflects the percentage of variance in total scores that is the result of individual differences on the general factor. An omegaH > 0.80 reflects, that irrespective of good fit indices, the scale is largely unidimensional. With respect to specific factors (subscales), omegaHS reflects the percentage of systematic variance of the specific factors, after the variance accounted for by the general factor is excluded (Rodriguez, Reise, & Haviland, 2016).

Ethical considerations

Ethical approval for this study was obtained from the Humanities and Social Sciences Research Committee of the University of the Western Cape (ethics reference number: HS20/5/1). The participants completed the survey anonymously. The first item in the electronic survey gave participants the opportunity to provide informed consent. At the end of the survey the contact details for the South African Depression and Anxiety Group and the Centre for Student Counselling and participants were urged to reach out to those services if they experienced any distress during completion of the questionnaire.

Results

Descriptive statistics

The mean loneliness score (M = 49.1; SD = 11.6) was much higher than that previously found in a similar sample from South Africa (Pretorius, 1993; M = 38.8, SD = 7.8), as well as those in the literature, M = 34 to 38: e.g. Auné et al., 2020; Hartshorne, 1993; Shevlin et al., 2015). The score was also higher than that found in a COVID-19 study (Killgore et al., 2020; M = 43.8, SD = 13.5).

Reliability

The reliability of the UCLA-LS3 (alpha = 0.923, omega = 0.924) was satisfactory, and compared favourably with the reliabilities documented in previous studies (e.g. Tull et al., 2020). Table 1 shows the impact of each item, if it were to be deleted, on the alpha coefficient, variance and mean, as well as the item-total correlation.

The correlations between individual items and the total score ranged between 0.26 and 0.73, whereas the item-deleted alphas were all 0.92, except for Item 17, which had an alpha of 0.93. Except for one related to items 8 and 15, the interitem correlations ranged between 0.11 and 0.87 and were all significant. The mean interitem coefficient was 0.37. Overall, the evidence for internal consistency was highly satisfactory.

Factor structure

In this study, we compared a one-factor model (UCLA-LS3 total score), a correlated three-factor model, a bifactor model with two subscales, and a bifactor model with three subscales. Notably, the two or three subscales conceptualisations of the bifactor models were based on previous factor analysis studies (Hawkley et al., 2012; Shevlin et al., 2014; Wilson, Cutts, Lees, Mapungwana, & Maunganidze, 1992), which identified a two-factor solution (Social Others = SO; Intimate Others = IO) as well as a three-factor solution (Collective Connectedness = CC; Isolation = I; Relational Connectedness = RC). Table 2 shows the item groupings and factor labelling for the one-factor model, the three-factor model, and the two bifactor models based on these studies.

The four representations of the factor structure of UCLA-LS3 and the results of the CFA are shown in Figures 1–4.

FIGURE 1: One-factor model of UCLA-LS3.

FIGURE 2: Correlated three-factor model.

FIGURE 3: Bifactor model of UCLA-LS3: Two subscales.

FIGURE 4: Bifactor model of UCLA-LS3: Three subscales.

In the one-factor model, it is assumed that a single factor (total loneliness score) best explains the variance amongst the items, whereas the correlated three-factor model presumes that three related factors account for the variance. The bifactor model, in contrast, presumes that a general factor (loneliness) explains a certain proportion of the variance, whereas two/three specific factors (subscales) account for the remaining variance. Table 3 reflects the fit indices for the three models.

As detailed in Table 3, the one-factor model failed to meet any of the criteria indicating a good fit. The correlated three-factor model had much better indices than those of the one-factor model, but they were marginally lower than those of the bifactor models, with only marginal differences in the fit indices of the two bifactor models. In both the two-subscale and the three-subscale bifactor models, TLI, CFI, and RMSEA were identical (TLI = 0.92, CFI = 0.94, and RMSEA = 0.07), indicating a reasonable fit. However, in terms of GFI and SRMR, the three-subscale bifactor model showed a better fit. In addition, the model comparison index (i.e. AIC) was lower for the three-subscale bifactor model than for the two-subscale bifactor model (487.13 in comparison to 506.37), indicating a slightly better fit.

Despite the evidence provided by the CFA in relation to the superiority of the bifactor structure for UCLA-LS3, the CFA did not address the dimensionality of the scale. More specifically, the CFA did not clarify the relative proportion of variance accounted for by the total scale and subscales. In this regard, Rodriguez et al. (2016), for example, urged researchers to use bifactor indices, over and above fit indices to examine the dimensionality of instruments. Table 4 reflects the bifactor indices for the UCLA-LS3.

The ECV is the percentage of all common variance for all items explained by a factor. Table 4 indicates that the general factor (loneliness) explained 59% and 57%, respectively, of the common variance in the two-subscale and three-subscale bifactor models. Therefore, specific factors (IO and SO in the case of the two-subscale bifactor model and I, RC, and CC in the case of the three-subscale bifactor model) explained 41% and 43% of the variance, respectively. This result confirms the multidimensionality of UCLA-LS3, as the specific factors accounted for sufficient variance, after the variance accounted for by the general factor was taken into consideration. In addition, omegaH, which reflects the percentage of variance in total scores accounted for by the general factor, was below the cut-off point suggested in the literature (omegaH = 0.68 and 0.69). Reise, Bonifay and Haviland (2013) proposed that when omegaH is greater than 0.80, the scale can be considered essentially unidimensional.

Lastly, the omegaS coefficient, which is a model-based estimate of reliability, further confirmed that the various subscales in the two-subscale and three-subscale bifactor models (IO: omegaS = 0.92; SO: omegaS = 0.90; I: omegaS = 0.90; RC: omegaS = 0.89; CC = 0.79) demonstrated sufficient reliability.

Both the CFA and bifactor indices provide support for the use of UCLA-LS3, as consisting of a total scale as well as either two or three subscales, although the three-subscale bifactor model fit the data marginally better than the two-subscale bifactor model.

Discussion

The aim of the present study was to extend current research on UCLA-LS3 by investigating its applicability to a sample of young adults in South Africa. The current study, hence, focused on the psychometric properties and normative data of the UCLA-LS3.

There were several important findings. Firstly, we found that the participants in the current study reported higher levels of loneliness than those found in prior studies (e.g. Killgore et al., 2020), thus suggesting a potential mental-health crisis. Such unprecedented levels of loneliness can be attributed to COVID-19 preventive measures, such as prolonged prohibition of in-person social contact and social distancing directives. They could also be the result of disparities in the access to digital technologies that can be used to circumvent these restrictions and maintain social connectedness. Women reported higher loneliness mean scores then men, a finding that confirms previous research on gender differences in loneliness (e.g. Wang & Zhao 2020). Such higher levels of loneliness amongst women may be because of the impact of gender role socialisation, which contributes to women prioritising affiliations with friends and family. Disruptions in the access to these networks as a result of stay-at-home directives may also contribute to increased psychological distress for women (Li & Wang, 2020).

Secondly, reliability analysis (Cronbach’s alpha) and analysis of the influence of individual items on the mean, variance, and alpha demonstrated that UCLA-LS3 exhibited highly satisfactory internal consistency in the sample. The obtained alpha coefficient was comparable to those reported in the literature (e.g. Ausín, Muñoz, Martín, Pérez-Santos, & Castellanos, 2019; Pikea, Parpa, Tsilika, Galanos, & Mystakidou., 2016). The results, therefore, support the use of UCLA-LS3 in culturally diverse contexts, and the findings suggest that meaningful comparisons can be drawn across countries.

Thirdly, the CFA demonstrated that the two bifactor models were a better fit than the one-factor model. The bifactor indices also demonstrated that the two and three subscales of the bifactor models account for sufficient variance after the variance accounted for by the general factor was considered. The bifactor model with three subscales had marginally better fit indices than those of the bifactor model with two specific subscales. These results, therefore, support the findings of Hawkley et al. (2012) and Shevlin et al. (2014). Thus, it can be concluded that UCLA-LS3 is best conceptualised as comprising of three subscales (isolation, relational connectedness, collective connectiveness), in addition to a total scale. The study confirms that loneliness, as measured by the UCLA-LS3, is a multidimensional concept. It also provides support for the future use of the scale in the South African context, especially related to the assessment of loneliness during the COVID-19 pandemic.

Limitations

This study has some limitations. The first limitation is the small sample size. However, the sample size is still within the rule of thumb of 10 cases per variable (Wolf, Harrington, Clark, & Miller, 2013). Moreover, the unequal gender sample, whilst consistent with the demographics in the university, prevented gender comparisons of models.

Conclusion

This study provides support for the generalisability of UCLA-LS3 in a South African sample of young adults and paves the way for its further use in South African samples. The scale demonstrated sound reliability, and the bifactor analysis confirmed the multidimensionality thereof.

Acknowledgements

The author declares that he has no financial or personal relationships that may have inappropriately influenced him in writing this research article.

T.B.P. is the sole author of this article.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

The data will be available at https://uwc.figshare.com/.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the author.

References