Original Research

Usefulness of the English version of the Stress Overload Scale in a sample of employed South Africans

Charles H. van Wijk

Received: 23 Oct. 2020; Accepted: 23 May 2021; Published: 25 June 2021

Copyright: © 2021. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Amidst reports of high levels of stress in South Africa, it remained difficult to quantify psychological stress in the absence of locally validated measures. This study explored the English version of the Stress Overload Scale (SOS) in a South African sample. The first aim was to replicate the basic psychometric analysis of the original English version used in American samples, as well as the Setswana version used in a South African rural community setting. The second aim was to investigate criterion validity to determine its appropriateness for use in South Africa. A total of 2136 employed South Africans with at least 9 years of schooling participated in this study. Participants completed a range of mental health and well-being measures, both clinical and dispositional. Responses were analysed to examine both scale characteristics and validity indices related to the SOS. Little sociodemographic influence (age, gender and first language) was found, with analyses supporting validity across most indices. Furthermore, good predictive ability for mental (ill-) health was observed. This study, for the most part, replicated previous validation findings of the SOS. Validity was further confirmed by correlating the scale with measures of clinical mental health and dispositional well-being. Given the positive support to its validity, when used amongst employed South Africans with at least 9 years of education, the scale holds promise for application in local health-related research, for triage in primary healthcare contexts and for measuring outcomes of mental health interventions in therapeutic settings.

Keywords: anxiety; depression; occupational medicine; stress; stress overload scale; South Africa; workplace health.

Introduction

Headlines such as ‘South Africa second most stressed country in world’ (City Press, 2019) or ‘Work stress cost SA R40 billion’ (Business Report, 2016) certainly have shock value and, whilst seemingly intended to attract readers’ curiosity, are also based on (some) truth. The issue of how stress is understood locally in South Africa (SA) has been brought into sharp focus during the recent coronavirus 2019 (COVID-19) pandemic, where individuals had to face challenges from finding adequate shelter and food to managing social isolation and anxiety about the future. Against this backdrop, psychologists are often asked questions such as ‘what is stress?’ or ‘how do psychologists measure stress?’

There are many models on stress in existence today, ranging from engineering to biology to psychology. Within the field of medical physiology, the term stress was first used by Hans Selye in 1936 to describe the non-specific response of the body to any demand for change. Within the field of psychology, stress is typically described from a transactional perspective (Lazarus & Folkman, 1984), where individuals evaluate both the events or demands they are facing (called primary appraisal) and the resources available to them (called secondary appraisal). In the transactional definition, stress may occur when individuals appraise the demands of their environments as exceeding their personal resources (overload). When overload is experienced, well-being is affected negatively (Lazarus & Folkman, 1984). Putting it differently, stress is the degree to which a person feels overwhelmed or unable to cope as a result of pressures that are perceived as unmanageable (Mental Health Foundation, 2018).

There are many questionnaires and inventories purporting to measure stress, with some focussing on environmental demands and others on personal resources, but a few include a transactional approach that would emphasise overload as a result of a perception of excessive demands and/or depletion of resources. In response to this shortcoming of previous scales, Amirkhan (2012) developed the Stress Overload Scale (SOS). This scale has strong theoretical underpinnings and captures overload through two underlying factors, namely, impinging demands (termed ‘event load [EL]’) and the depletion of resources (termed ‘personal vulnerability [PV]’) to handle those demands (Amirkhan, 2012, 2018). The 30-item SOS has been found to predict illness and cortisol responses, as well as sick days and workdays missed. It also distinguishes between stressed and non-stressed populations (Amirkhan, 2012; Amirkhan, Urizar, & Clark, 2015). Amirkhan (2018) also developed the Stress Overload Scale–short form (SOS-S) for application in contexts where the long form may prove impractical.

The original English version SOS and SOS-S have been extensively validated in different community and college samples in the United States of America (Amirkhan, 2012, 2018; Amirkhan et al., 2015). As context influence both the kind of stressors and the availability of resources in a community, a Setswana version was developed and tested in a rural community sample in SA (Wilson, Wissing, & Schutte, 2018). That study reported good psychometric support for the short form.

Rationale and aims

Although high levels of stress have been reported amongst working South Africans, the multilingual nature of SA society and potentially divergent understandings of stress and mental well-being pose challenges to the use of globally available measures to identify stress and predict possible negative consequences locally. The English version SOS has not been validated in SA, but if sound psychometric support could be found, it would then allow for the use of a single-language version with a larger part of the employed population.

This study set out to explore the English version SOS in a sample of employed South Africans with at least 9 years of schooling (to enable meaningful completion of the English language scale). It aimed to do so in two ways. Firstly, it aimed to replicate aspects of two previous projects, namely, the validation of the original English version used in American validation studies with community- and college-based samples (Amirkhan, 2012, 2018) and the Setswana version used in a South African rural community setting (Wilson et al., 2018). This was performed in order to report on basic psychometric properties. Secondly, it aimed to investigate criterion validity by exploring associations across a range of mental health and well-being measures. This was carried out in order to report on practical value of the scale (for possible use in research or clinical practice). Analysis was performed on both the long and short versions of the SOS.

This study aimed to contribute to the validation of the English version of the SOS (and SOS-S) for use in SA workplace populations. If a single language version can be used on a larger segment of the population, it may enable the identification of stress overload that in turn may pose a risk to poor mental health and emotional distress. Where this can be identified, it could facilitate the allocation of resources to support mental health and well-being in the workplace.

Methods

Participants

Participants were recruited through workplace occupational health programmes and invited to complete the SOS when they were completing their regular health surveillance questionnaires. Potential participants were briefed that completion of the forms would be interpreted as implied consent and that the results would not form part of their occupational health screening outcome. Participants were given time at work to complete the SOS and other measures, in group settings, whilst sitting at individual work stations. All participants had a minimum of 9 years of formal schooling. This was to ensure a level of English proficiency sufficient to complete the indicated range of standard mental health measures. Participants also completed a range of other measures of mental health and well-being, but because of practical considerations (e.g. different protocols used at different sites), not all participants were involved in all aspects of measurement. The numbers of completed measures are shown in Table 1. Furthermore, participants also underwent a semi-structured interview with a clinical psychologist, who was blind to the questionnaire outcome and who allocated a binary category of the presence of ‘any mental health disorder’ at the completion of each interview. For participants who presented with signs or symptoms of poor mental health, referrals were arranged to an appropriate mental health service provider (e.g. psychologist or medical practitioner, depending on the need at the time).

Measures

Stress overload scale

The SOS comprises 30 items and is designed to measure ‘stress overload’ (Amirkhan, 2012). A 5-point Likert scale (1 = not at all and 5 = a lot) is used to indicate subjective feelings and thoughts experienced during the previous week. There are two factors underlying overload, namely, PV and EL, which are measured by two distinct but correlated subscales (12 items each); there are also six filler items included to discourage negative response sets, which are not scored (Amirkhan, 2012). The scales can be summed to obtain a continuous total score, with higher scores indicating higher levels of stress overload. Alternatively, the subscales can be split at their means to form a four-category diagnostic matrix; those scoring in the high EL–high PV category have been shown to be at the greatest risk for subsequent pathology (Amirkhan, 2012). Only the continuous scoring was used in the current study.

The SOS is unique in the sense that it was empirically constructed through a sequenced series of factor analytic and psychometric studies, using community samples matched to US census demographic proportions (Amirkhan, 2012). This provides three advantages. Firstly, it is psychometrically strong, especially in terms of validity; secondly, it is appropriate to community research because of its brevity and fit to a broad demographic spectrum and thirdly, it is unique in its ability to cross-section individuals into risk categories (Amirkhan, 2012).

The SOS has excellent internal consistency (with Cronbach’s α > 0.94 for both subscales and full scale) and good test-retest reliability (with coefficients averaging 0.75 over 1 week; Amirkhan, 2012). Construct validity has been demonstrated through significant correlations with other measures of stress and illness (Amirkhan, 2012; Amirkhan et al., 2015). Criterion validity has been shown in the SOS ability to predict illness following a stressful event. Furthermore, SOS scores have been found to significantly correlate with illness, sick days and workdays missed (Amirkhan, 2012; Amirkhan et al., 2015). Psychometric support for the Setswana version of the SOS was less convincing, with an inconclusive confirmatory factor analysis (CFA) reported. Exploratory factor analysis (EFA) identified four factors, which were not theoretically interpretable (Wilson et al., 2018).

This study used the English version SOS, in its standard administration, with two small modifications. Extensive piloting showed that two 1-word items proved somewhat difficult (14 out of 316 cases), namely, items 3 and 4. Both were modified by including a rider in parentheses after the word. For item 3, ‘like you’re not up to the task’ was added in parentheses and for item 4, ‘like you need to do too many things’ was added in parentheses.

The 10-item SOS-S (Amirkhan, 2018) preserved many of the features of the full measure, including the two-subscale structure that permits both continuous and categorical scoring. It also maintains the full measure’s excellent internal consistency (α > 0.94) and good test-retest reliability (r = 0.75; Amirkhan, 2018). Construct validity was demonstrated by significant convergence with the similar length Perceived Stress Scale (PSS-10). Criterion validity was shown in its associations with both concurrent and future signs of illness, both symptomatic and behavioural (Amirkhan, 2018). Because filler items are missing in the SOS-S, it is more vulnerable to response biases; however, even with social desirability and negative affectivity influences controlled, it maintained significant associations with signs of illness (Amirkhan, 2018). Positive psychometric support for the Setswana version of the SOS-S was demonstrated previously: a CFA reported good model fit and evidence of concurrent validity was found through significant correlations with the PHQ-9 (Wilson et al., 2018).

Markers of mental health and mental well-being

Mood, anxiety and alcohol abuse disorders are the most common mental health conditions in SA (Herman et al., 2009) and four measures were included to examine correlations between the SOS and these clinical constructs. To examine correlations with non-clinical measures, some participants also completed two dispositional measures that are often related to general mental well-being.

The Patient Health Questionnaire for depression (PHQ-9) is a well-established 9-item screening, diagnostic and monitoring tool measuring the severity of depression (Gilbody, Richards, & Barkham, 2007; Kroenke, Spitzer, & Williams, 2001) and previous use in SA reported Cronbach’s α > 0.70 (Bhana, Rathod, Selohilwe, Kathree, & Petersen, 2015; Wilson et al., 2018). Strong correlations were previously found for the SOS-S and PHQ-9 in a Setswana sample (EL: r = 0.42, PV: r = –0.47, p < 0.001 for both; Wilson et al., 2018) and with the related Center for Epidemiological Studies-Depression (CES-D) in a US sample (full scale: r = 0.53, EL: r = 0.46, PV: r = 0.52, p < 0.0001 for all; Amirkhan, 2012).

The Generalized Anxiety Disorder questionnaire (GAD-7) is a well-established 7-item screening, diagnostic and monitoring tool measuring the severity of generalise anxiety (Löwe et al., 2008; Spitzer, Kroenke, Williams, & Lowe, 2006). It also has utility for detecting panic and social anxiety disorder (Kroenke, Spitzer, Williams, Monahan, & Löwe, 2007). Cronbach’s α > 0.90 has been reported (Spitzer et al., 2006). No correlations between the SOS and markers of generalised anxiety have been reported previously.

The primary care post-traumatic stress disorder (PTSD) screen for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (PC-PTSD-5) was developed as a brief 5-item screen for PTSD in PC settings, with Cronbach’s α > 0.90 and high diagnostic accuracy (Bovin et al., 2021; Prins et al., 2016). This measure was included against the background of SA’s high reported prevalence of post-traumatic stress (Williams et al., 2007). No correlations between the SOS and markers of post-traumatic stress have been reported previously.

The 4-item CAGE questionnaire (Ewing, 1984) has been developed for use in identifying problematic alcohol use, with high sensitivity and specificity reported (Dhalla & Kopec, 2007; O’Brien, 2008; Williams, 2014). It has been used extensively across the world and also in SA (Labadarios, 2018; Van Wijk, Cronje, & Meintjes, 2020). No correlations between the SOS and markers of problematic alcohol use have been reported previously.

The State Trait Personality Inventory, trait version (STPI-T; Spielberger, 1996) is a 40-item measure of emotional disposition (including dispositional anxiety, curiosity, anger and depression) in adults (Spielberger & Reheiser, 2009). Acceptable psychometric properties were reported for SA samples (Du Plessis, 2014; Van Wijk, 2017). These dispositional traits have been strongly correlated with measures of both work and non-work stress (Hogan, Carlson, & Dua, 2002) and strong correlations with the PSS-10 (r = 0.7) have been reported (Silver, 2013). Furthermore, significant correlations have been reported for STPI-T anxiety and primary appraisal tasks (r = 0.30, p < 0.001) but not secondary appraisal tasks (r = 0.15, p = 0.064; Abdullatif, 2006). In contrast to the PHQ-9 and GAD-7, the STPI-T is not a clinical (i.e. diagnostic) scale, but rather reflects personal disposition, manifested in general mental well-being (Spielberger, 1996).

The 15-item Dispositional Resilience Scale (DRS-15; Bartone, 2007) measures hardiness, which has been described as a psychological orientation associated with people who remain healthy and continue to perform well in a range of stressful conditions (Kobasa, Maddi, & Kahn, 1982). Hardiness appears to protect against the ill effects of stress on health and performance amongst a wide variety of occupations and contexts (Bartone, 1989; Maddi & Hess, 1992; Maddi & Kobasa, 1984; Topf, 1989). Whilst hardiness and its sub-components could theoretically affect both primary and secondary appraisal, and thus the interpretation of stress, its influence may be most visible on PV (related to secondary appraisal). In this regard, significant correlations with the DRS-45 (a longer version of the same measure) were reported with the full-score SOS (r = –0.31, p < 0.001), PV (r = –0.40, p < 0.001) and EL (r = –0.17, p < 0.01) (Amirkhan, 2012).

Sociodemographic information

Initial validation studies of the SOS and SOS-S observed only small associations (seldom reaching significance) for age and gender (Amirkhan, 2012, 2018). Age and gender information was available for the current sample and was included in the analysis. Home language was retrospectively coded into two categories, namely, English first language and non-English first language, for further analysis. Occupational domains are reported for sample description only.

Diagnostic marker for any mental health disorder

After each interview, the consulting psychologist allocated a binary category to each participant, indicating the presence of ‘any mental health disorder’. This was based on a semi-structured interview and allocated at the discretion of each individual clinical psychologist.

Data analysis

Mean, standard deviation (SD) and range (for the full scale and its factors, as well as the short form and its factors) were calculated and are reported in Table 1. Sociodemographic effects were examined through Pearson’s correlation coefficients (for age) and t-tests for independent samples (gender and language). Cronbach’s alpha coefficient was calculated to describe internal consistency (for full and short forms and factors). Factor analysis was conducted using an EFA, given that the previous SA study did not find meaningful outcomes from a CFA (Wilson et al., 2018). This was performed for both the full scale (24 items) and short form (10 items).

Validity indices were examined through correlations with scores on the four measures of mental health and two measures of mental well-being. Possible predictive associations with mental health diagnoses were calculated using receiver operating characteristics (ROC) curve analysis, with the binary interview outcome as state variable.

All analyses were performed by using the Statistical Package for the Social Science (IBM SPSS for Windows. Version 24).

Ethical considerations

The study has been approved by the Health Research Ethics Committee of Stellenbosch University (reference number: N20-07-078).

Results

Sample characteristics

The sample consisted of 2136 employed South Africans, with a mean age of 34.0 years (± 8.4); of whom 19.8% reported English as their first language. Participants came from all SA language groups and a wide range of occupational domains. Further breakdown of the sample composition can be found in Tables 1 and 2. All participants had a minimum 9 years of schooling and were considered skilled workers.

Sociodemographic effects

Age correlations with SOS scores, whilst significant, were very small (see Table 1) and might not have had much practical impact in the current sample with a limited age range (20–60 years). In the case of both gender and language variables, there were no significant differences between the mean scores of women and men, or of English first language and non-English first language speakers, on any the SOS full-scale or short form totals and subscale totals (see Table 3). Given the lack of significant differences between gender and language groups, and the very small mean differences between them, the combined full sample was used for the remainder of the analyses.

Scale characteristics

High internal reliability was found for the full scale (Cronbach’s α = 0.946) and short form (Cronbach’s α = 0.901). In neither case did any item deletion improve α. The subscales EL and PV (using the original item allocation) were strongly correlated (r = 0.806, p < 0.001). The full scale and short form of the SOS correlated strongly (r = 0.969, p < 0.001) and the full-scale and short-form subscales were also strongly correlated (see Table 1).

The results of the EFA for the full scale showed a Kaiser-Meyer-Olkin (KMO) index of 0.972 and the result of the Bartlett Sphericity test was significant (p < 0.001). Using principal component analysis, two factors with Eigenvalues > 1 could be extracted, explaining 53.1% of variance. There were substantial cross-loading (< 0.40) on eight items, mainly items of the EL subscale loading onto the PV subscale (see Table 4). The two factors were strongly correlated (r = 0.629 and p < 0.001). As the item loading did not follow the clear differentiation of EL and PV in the original studies, it did raise the issue of language influences and a separate EFA was conducted for the English first language and non-English first language groups. The analyses resulted in similar matrices, for both groups, as presented in Table 4 for the combined group.

The results of the EFA for the short form showed a KMO index of 0.921 and significant Bartlett Sphericity test (p < 0.001). Principal component analysis identified a single factor (Eigenvalue = 4.82) that explained 54.0% of variance.

Validity indices

Descriptive statistics of mental health and well-being scales can be found in Table 1, as well as their correlations with the SOS full and short forms and subscales. All correlations between SOS scores and measures of mental health and well-being were significant at p < 0.001. Strong correlations were observed for the clinical scales PHQ-9 and GAD-7, with correlations with PV stronger than with EL in both cases. Correlations with PC-PTSD-5 were moderate and lower for the CAGE measure. Strong correlations were also observed with the STPI-T subscales for dispositional anxiety and depression and in each case, PV displayed stronger correlations than EL. Strong correlation with dispositional anger and moderate correlation with dispositional curiosity and hardiness (DRS-15) were also found.

The SOS demonstrated positive predictive validity for ‘any mental health disorder’. The ROC analysis showed highly significant areas under the curve (full scale = 0.916 [95% CI: 0.896–0.938]; short form = 0.898 [95 % CI: 0.872–0.925]). Optimal sensitivity and specificity appear to be around > 51 (86% sensitivity and 83% specificity) for the full scale and around > 20 (86% sensitivity and 81% specificity) for the short form (see Table 5).

Discussion

This study used data from a large sample of employed adult South Africans to replicate aspects of previous validation studies and extend previous exploration of associations with mental health constructs.

The sociodemographic variables of age and gender appeared to have very little practical effect on SOS scores in this sample. More importantly, first language in this group of relatively educated workers did not meaningfully influence scores on the SOS or SOS-S. It appears that the English version scale may be used across different language groups in SA, providing participants have at least a grade nine level education.

Regarding scale characteristics, high internal consistency was observed, mirroring the original validation studies (Amirkhan, 2012; Amirkhan et al., 2015). The strong correlation between the EL and PV subscales was higher than previously reported (Amirkhan, 2012), suggesting some overlap of the underlying constructs. Furthermore, whilst the two factors identified during the EFA generally adhered to the original subscales (with a few exceptions), the substantial cross-loading and high inter-factor correlation questioned the extent to which the factors could be viewed as distinct constructs. As with previous SA research using the Setswana version (Wilson et al., 2018), the factor structure poses a challenge to the uncritical acceptance of the SOS’s structural validity.

The SOS demonstrated positive predictive validity and scores could predict the risk for mental health disorders with reasonable probability. Current sensitivity and specificity appear adequate for research use. Furthermore, the strong correlations between the full scale and short form suggest that the short form can be used confidently where there are concerns regarding time or respondent fatigue.

Positive criterion validity was demonstrated through significant correlations with all the measures of (clinical) mental health and (dispositional) well-being. Strong correlation with the PHQ-9 and GAD-7 supported earlier studies using comparable clinical measures (Amirkhan, 2012; Wilson et al., 2018). Similarly, strong correlation with the STPI-T closely followed earlier studies using comparable measures (Silver, 2013). One observation was of particular interest, namely, that clinical and dispositional indicators were more strongly associated with PV than EL. However, it could be argued that whilst the pattern of effect sizes (i.e. higher for PV than for EL) was consistent, the actual effect size difference may not have been that meaningful, given the high confidence intervals for significance reported. Furthermore, the problematic factor analyses cautions against confident interpretation of subscale scores. Correlations with hardiness, a construct of personal orientation to life, were moderate and very similar to earlier reports (Amirkhan, 2012), although the difference between EL and PV was not as substantial as previously reported, likely because of the scale overlap observed here.

Local applications

The SOS can be used across a number of local applications: In primary healthcare (in both community and occupational setting), the SOS could be used for screening – on a larger scale – to facilitate the streaming of high-risk individuals to appropriate support services (‘triage’). Within research in the local health context, it could be used to explore associations of stress overload with specific health conditions (in a clinical health framework) and other health outcomes (in an occupational health framework). Within therapeutic settings, the SOS could be used productively to measure outcomes of psychotherapeutic (and other) interventions through longitudinal comparison. In a broader national context, it could also be possibly used for quantifying stress overload in different sectors of the SA economy.

Limitations

This study has a number of limitations. Determination of English language proficiency used years of formal education as proxy. The perils of using this criterion in SA, with its history of disparate educational opportunities, resource allocation and outcome standards, are recognised and future studies may need to include finer calibrated indicators of language competence.

Determination of ‘Any mental health disorder’ was performed by different clinical psychologists as part of their clinical practice. There were a number of psychologists involved over time and it is recognised that the threshold for allocating a YES response might have differed amongst them. Although this might have been mitigated by them being very experienced in this type of work, future studies may need to standardise the criteria for such a category more explicitly to enhance inter-rater reliability.

Future directions

The close association of stress overload with both clinical indicators and dispositional orientation raises the question of direction of influence, which may occur in opposite directions. It has previously been argued that appraisal of stress causally affects mental health because higher perceived stress would lead to a higher incidence of mental health diagnoses (De Lange, Taris, Kompier, Houtman, & Bongers, 2004; Diette, Goldsmith, Hamilton, & Darity, 2012; Shigemi, Mino, & Ohtsu, 2000). It has further been argued that personal disposition would causally affect the appraisal of stress in the sense that a resilient disposition would lead to lower perceived stress (Abdullatif, 2006; Amirkhan & Greaves, 2003). Future research will need to empirically test these hypotheses locally, to add to the understanding of the relationship amongst dispositional or personality constructs, stress appraisal and mental health disorders in the SA context.

Conclusion

This study replicated, for the most part, previous validation of the SOS and extended validity exploration across multiple measures. High internal consistency and positive criterion validation were confirmed. Most of the tested indices provided evidence of validity of the original SOS in the study context, suggesting that it could be usefully employed across different language groups where at least a grade nine-level education can be demonstrated. The subscales might not provide equal confidence, and further research is required to explore the factorial structure of the SOS and the use of subscales as individual markers. There is some support for the use of the full scale and short form in research and clinical practice. For example, this sample was accessed prior to the COVID-19 pandemic and their scores could be viewed as reflective of that period. Using this as baseline, the full-scale SOS (and SOS-S) could be used constructively in studies to explore perceived stress, in local comparable populations, in the post-COVID-19 era.

Acknowledgements

The author would like to thank Nazneen Firfirey for her assistance in managing the collected data.

The author declared that he has no financial or personal relationships that may have inappropriately influenced him in writing this article.

C.H.v.W is the sole author for this article.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

The data that support the findings of this study are available from the author, upon reasonable request. The data are not publicly available because of privacy and ethical considerations.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the author.

References