key: cord-0038923-j9dc5mv1
authors: Abendroth, Adrian; Parry, Douglas A.; le Roux, Daniel B.; Gundlach, Jana
title: An Analysis of Problematic Media Use and Technology Use Addiction Scales – What Are They Actually Assessing?
date: 2020-03-10
journal: Responsible Design, Implementation and Use of Information and Communication Technology
DOI: 10.1007/978-3-030-45002-1_18
sha: 3332d033b8fdccd2a2bc6e8a0c5f639eee248b1d
doc_id: 38923
cord_uid: j9dc5mv1

Increasingly, research attention is being afforded to various forms of problematic media use. Despite ongoing conceptual, theoretical, and empirical debates, a large number of retrospective self-report scales have been produced to ostensibly measure various classes of such behaviour. These scales are typically based on a variety of theoretical and diagnostic frameworks. Given current conceptual ambiguities, building on previous studies, we evaluated the dimensional structure of 50 scales targeting the assessment of supposedly problematic behaviours in relation to four technologies: Internet, smartphones, video games, and social network sites. We find that two dimensions (‘compulsive use’ and ‘negative outcomes’) account for over 50% of all scale-items analysed. With a median of five dimensions, on average, scales have considered fewer dimensions than various proposed diagnostic criteria and models. No relationships were found between the number of items in a scale and the number of dimensions, or the technology category and the dimensional structure. The findings indicate, firstly, that a majority of scales place an inordinate emphasis on some dimensions over others and, secondly, that despite differences in the items presented, at a dimensional level, there exists a high degree of similarity between scales. These findings highlight shortcomings in existing scales and underscore the need to develop more sophisticated conceptions and empirical tools to understand possible problematic interactions with various digital technologies.

Despite recent recognition of gaming disorders in the latest edition of the World Health Organisation's International Classification of Diseases (ICD-11) diagnostic system, there remains much debate about the veracity and theoretical basis for a variety of supposed pathological forms of media use behaviour [9, 12, 13, 17] . Described in relation to a range of technologies (e.g., games, smartphones, the Internet) such behaviours are typically framed as problematic and, in some cases, addictive [8, 21] . Examples include: Internet gaming disorder, problematic media use, Internet addiction, and technology use addiction among many other labels. Despite the widespread adoption of various umbrella terms, some researchers have criticised the use of these labels, arguing that, in many cases, the behaviours and outcomes referred to are often particularly heterogeneous, influenced by a variety of personal characteristics, and potentially driven by distinct etiological mechanisms [13, 15] . For the purposes of this paper, acknowledging the ambiguity, the term problematic media use will be adopted to refer, broadly, to various forms of pathological or addictive engagements with digital technologies.

Nomenclature aside, the notion of problematic media use is further beset by a variety of theoretical and empirical challenges. At a theoretical level, for many proposed technology-related addictions, a conceptual consensus has yet to be reached [9, 13, 17] . Furthermore, many competing theoretical frameworks have been proposed for the same supposed construct. Laconi et al. [15] , for instance, note that Internet addiction is considered by some to be an impulse control disorder and, for others, a behavioural addiction, or even a combination of the two. Additionally, further contributing to the conceptual quagmire, studies have shown a number of comorbid psychopathologies (e.g., depression, attention-deficit hyperactivity disorder, or anxiety disorders) with various forms of problematic media use [14] . Finally, there exist criticisms that many definitions are pathologising everyday behaviour [10, 11] . This is especially the case for gaming-related behaviours where researchers have struggled to differentiate between higher levels of engagement and truly problematic use [13] .

In an early definition for technology-related addictions, Griffiths [7, p. 15 ] noted linkages with substance addictions and defined technological addiction as a class of behavioural addiction that involves some form of human-machine interaction. Subsequently, in considering the clinical diagnostic criteria of substance addiction, Griffiths [8] proposed a 'components' model of addiction consisting of six dimensions: salience, mood modification, tolerance, withdrawal symptoms, conflict, and relapse. He argues that such components are not only key for substance-related addictions but, from the perspective of a biopsychosocial framework of the individual, are also present in behavioural addictions.

Empirically, the assessment of problematic media use is hindered by unresolved theoretical and etiological foundations, as well as general challenges in assessing media use [6] . Adopting a tendency present in much of modern Social Psychology [4] , researchers have relied on retrospective self-report questionnaires (or scales) to assess individuals' propensities for various behaviours, disorders, or other related outcomes. This has seen the proliferation of an inordinate number of scales designed to assess various forms of problematic media use, many of which target the same or substantially overlapping constructs (e.g., the Smartphone Addiction Scale, the Smartphone Application-Based Addiction Scale, and the Problematic Mobile Phone Use Questionnaire). Owing to the scale-development processes typically followed, such measures are normally reliable. Ellis et al. [6, p. 1] , however, argue that, in general, 'less emphasis has been placed on establishing validity' when it comes to the assessment of problematic media use.

While weak correlations have been shown for a number of scales for both smartphone use in general and addiction in particular, and more objective assessments of media use [6] , this does not necessarily imply that scales purporting to assess problematic behaviours are invalid or spurious -use is only one component of the behavioural and mental patterns targeted. Nevertheless, in a review of 45 tools designed to assess Internet addiction, Laconi et al. [15] found that only 26 had been evaluated for their psychometric properties. Additionally, supporting assertions that, for many scales, construct validity is low, outcomes of factor analyses differed substantially between studies for a number of instruments [15] . This lack of validation and conceptual clarity is problematic for both the interpretation of findings produced on the basis of these scales, as well as their usefulness as diagnostic screening tools in clinical settings.

Many scales are based on criteria for substance dependencies described in the Diagnostic and statistical manual of mental disorders (DSM-IV-TR) published by the American Psychiatric Association [2] , or the component model described by Griffiths [8] . Adopting a grounded approach, Lortie and Guitton [16] examined the dimensional structure of 14 Internet addiction scales published between 1993 and 2011. All scale-items were pooled into seven categories on the basis of their conceptual similarity: compulsive use, negative outcomes, salience, withdrawal symptoms, mood regulation, escapism, and social comfort. For scales targeting Internet addiction, this analysis found that 'compulsive use' and 'negative outcomes' were the two most prominent dimensions assessed, followed by 'salience'. To compare their findings to existing norms Lortie and Guitton [16] considered how their seven dimensions related to the diagnostic criteria for substance dependence in the DSM-IV-TR [2] and ICD-10 [20] . With the exception of 'social comfort' the authors mapped each of their dimensions to the relevant diagnostic criteria, providing motivations for each assignment (a description of the dimensions is provided in Table 1 ).

Acknowledging the theoretical and empirical challenges in this domain, and the large number of sometimes overlapping measurement tools, the objective of the present study is to systematically review the dimensional structure of retrospective self-report scales used to assess problematic media use. In particular, the study aims to analyse the dimensional structure of the items presented in these scales in relation to prominent diagnostic criteria for substance dependence, models of behavioural addiction, and previous research in this regard, for four technologies frequently considered in the literature: the Internet, smartphones, video games, and social network sites (SNSs). Such a study will, firstly, aid in identifying shortcomings in existing scales, secondly, contribute a degree of conceptual clarity, thirdly, extend previous research focusing on only a single form of technology and, fourthly, provide a clearer foundation for improved assessment tools. To this end, building on the guidelines provided by Webster and Watson [18] , a systematic review methodology was adopted to identify, collect, extract, and analyse a sample of such assessment tools.

To systematically review scales supposedly assessing various forms of problematic media use we specified inclusion criteria and, subsequently, implemented a bibliographic database search strategy. Next, a set of a priori dimensions were specified and, in a series of iterative coding rounds, applied to each item presented in all relevant scales extracted from the included studies.

Research reports were included in the review if they (i) described the development of a self-report scale for the assessment of problematic use of at least one of the four technologies considered; (ii) were not merely a validation or language translation of an existing scale; (iii) were published in a peer-reviewed outlet; and (iv) were published between January 2007 and August 2018 (the time of data collection), with the exception of scales concerning Internet related addictions. For such scales an earlier start date of 1996 was selected due to the earlier development of relevant scales [21] . For the remainder, 2007 was selected as a cutoff due to the rise of smartphones and popular SNSs from this period [19] .

To locate a sample of eligible scales a systematic search of nine bibliographic databases was conducted over a three-week period in August 2018: GoogleScholar, ScienceDirect, PubMed, EmeraldInsight, Wiley, SpringerLink, ACM, iEEE, and JSTOR. To query these databases a search string was developed and, as required, tailored for each database. The following three clauses, each separated by an OR operator, were designed to target various keyword combinations:

-(consequence * OR impact OR effect OR symptom * ) AND (addiction OR habit * OR obsession OR problem * OR ('problematic use')) AND VARIABLE -(consequence * OR impact OR effect OR symptom * ) AND (addiction OR habit * OR obsession OR problem * OR ('problematic use')) AND (scale OR test OR inventory OR questionnaire) AND VARIABLE -(development OR creating) AND (consequence * OR impact OR effect OR symptom * ) AND (addiction OR habit * OR obsession OR problem * OR ('problematic use')) AND (scale OR test OR inventory OR questionnaire) AND VARIABLE For each of the four technology categories considered, the VARIABLE component in the three primary clauses was replaced with one of the following clauses:

internet OR web OR www OR online smartphone OR cellphone OR 'cell-phone' OR phone OR mobile -(social OR online) AND (network * OR social) AND (site * OR network * OR activity) OR sns OR 'social network sites' OR 'online social networks' -(online OR web OR internet OR www OR video) AND (play * OR gam * )

The bibliographic information and full-text records of each result were downloaded and stored in reference management software for subsequent extraction and analysis. All duplicated results were noted and removed, after which, titles and abstracts were reviewed against the inclusion criteria. For all eligible studies, the relevant scale-items were extracted to be coded. Noting the alignment between the seven dimensions provided by Lortie and Guitton [16] and the diagnostic criteria described in the ICD-10 [20] and DSM-IV-TR [2] , as well as the component model described by Griffiths [8] , these seven dimensions were generalised to refer more broadly to all media and used as deductive, a priori categories for the analysis. Table 1 provides a summary of the dimensions used. The description provided represents a synthesis of Lortie and Guitton [16] 's definitions, as well as their mapping of the dimensions to the diagnostic criteria in the ICD-10 [20] and DSM-IV-TR [2] , the descriptions provided in the component model [8] , and the updated criteria in the DSM-V [3] . Coding took place in three rounds and was performed by three independent coders. When a match was not possible, new dimensions were proposed. The process is described in Sect. 3.2.

We first describe the results of the systematic search procedure before outlining the process of mapping the extracted scale-items to relevant dimensions. Thereafter, we describe the dimensional structure of the included scales.

The systematic search produced 4698 results. After removing duplicates (n = 1676), the remaining records (n = 3022) were examined for inclusion and ineligible records (n = 2908) were removed. After reviewing the full-texts of the remaining reports (n = 114), 58 contained scales which met the inclusion criteria. During extraction eight scales were removed due to either missing or incomplete information, or because the scale described was a language translation of an existing scale. For scales with missing information, if available, supplementary material were reviewed and, if the necessary information was still missing, the original authors were contacted. This left a final sample of 50 scales. The inclusion process is summarised in Fig. 1 , with a list of the included scales available in the online supplementary materials hosted on the Open Science Framework. 1 The 50 scales included in the analysis were assigned to one of four categories -smartphones (n = 20; 40%), Internet (n = 14; 28%), video games (n = 12; 24%), SNSs (n = 4; 8%) -on the basis of their titles and original descriptions. From the 50 scales, 971 items were extracted, with the shortest scale consisting of six items, the largest 52 items, and a mean of 19.42 (SD = 8.91) items per scale. Given the discrepancy in the number of scales included for each technology, as well as the differences in scale length, there was a substantial difference in the number of items considered for each technology: Internet (316; 32.54%), smartphone (408; 42.02%), video games (197; 20.20%), and SNSs (50; 5.15%).

Each of the 971 items were mapped to one of the a priori dimensions. If the mapping from item to dimension was unclear, the descriptions in the DSM-V and Griffiths [8] 'Component' Model were considered as guides. During the initial During three iterative rounds of review new codes were developed to account for items not captured by the seven a priori dimensions. These dimensions and their descriptions are provided in Table 2 . With the exception of cognitive absorption, these dimensions were primarily tangential to problematic media use. Agarwal and Karahanna [1, p. 655] define cognitive absorption to involve: (i) temporal resolution in which passage of time is not registered; (ii) total immersion in the activity where other basic needs are ignored; (iii) intensified pleasure in doing the activity; (iv) the user's perception of being responsible for the interaction; and (v) curiosity that responds to sensory and cognitive sensitivities. Following the development of these dimensions, items were re-categorised and moved to newly specified dimensions if deemed appropriate by all three coders. Table 3 provides a summary of the dimensional structure of the scales for each of the four technologies considered. Columns 2-5 present the number of items for a given dimension for each technology category, with the proportion of items in each category mapped to a given dimension represented in parentheses. The 'total' column represents the total for a given dimension across all technologies, Figure 2 depicts the distribution of the dimensions across the four technologies considered. [5] -all items in this scale were mapped to withdrawal symptoms. Following these two dimensions, 47 and 42 scales included items assessing withdrawal symptoms and salience, respectively. In contrast, the three remaining a priori dimensions -escapism, mood regulation, and social comfortwere only included in 28, 26, and 11 scales respectively. Of the newly created categories, only cognitive absorption appeared with any frequency, being used in nine different scales (through 16 items). A Chi-square test of independence was calculated comparing the proportional frequency of each of the dimensions for each technology category. Owing to the relatively small number of observations for SNSs, this analysis was only conducted for the three remaining technology categories (the Internet, smartphones, and video gaming). The analysis revealed no statistically significant interaction between the technology category and the proportional distribution of dimensions (χ 2 (22) = 29.82, p = 0.123).

Despite a growing amount of research into various forms of media behaviour and the effects thereof, there remains much uncertainty and debate about the existence and possible nature of many proposed pathological technology-interactions. Given the conceptual ambiguity and disputed existence of many frequently cited phenomena-Internet gaming disorder, Internet addiction for instance-there exists a need for more sophisticated and nuanced understandings of the theoretical and etiological nature of these 'disorders'. In particular, it is necessary to understand the factors distinguishing various proposed disorders, or whether they are merely manifestations of other underlying factors, pathologies, or individual characteristics. This study found that, for just four technologies, there exist at least 50 unique assessment scales, many of which substantially overlap. We question the need for such a large number of scales.

To guide our analysis we used Lortie and Guitton [16] 's seven dimensions, augmented with the component model and recent diagnostic criteria for substance dependence, as a priori categories. Five additional categories were produced in three rounds of iterative coding. With the exception of cognitive absorption, these new dimensions were primarily descriptive and labeled items assessing peripheral, and sometimes irrelevant, aspects of media use. Given Agarwal and Karahanna [1] 's definition of cognitive absorption, while conceptual overlaps exist with elements of salience and mood-regulation, this dimension potentially provides a further feature differentiating behavioural addictions involving technology from other more general notions of behavioural addiction.

Across all 50 scales two dimensions -compulsive use and negative outcomeswere found to account for over 50% of all items. Moreover, with the exception of SNSs, where 'withdrawal symptoms' was similarly frequent, these dimensions accounted for a majority of items presented. For all technology categories, while the absolute number of items and the specific questions posed differed, the overall dimensional structure was found to be particularly comparable. The prominence of compulsive use and negative outcomes supports Lortie and Guitton [16] who found a similar outcome when analysing 14 scales produced prior to 2012 only targeting Internet-related addiction. For compulsive use, a possible explanation, in both cases, rests on the inclusion of tolerance, considered by some to be a separate dimension, in its description. Nevertheless, given the imbalance in dimensions considered, future studies need to consider, firstly, how representative these scales are of the supposed dimensions of technology-related behavioural addictions and, secondly, whether under-representation of particular dimensions is a valid concern. Furthermore, researchers intending to use an existing scale should, in addition to considering the scale description, account for the dimensional structure of items when determining the fit between a scale and their research questions.

The present study should be considered as only a single point of reference for further research into the assessment of pathological interactions with digital technology. Despite the systematic approach adopted, there are, nonetheless, a number of limitations which bare acknowledgement. First, while the search string was designed to enable the collection of a comprehensive sample, there remains the possibility that relevant scales could have been inadvertently omitted. Furthermore, while four distinct technology categories were used, it is acknowledged that, in many cases it is not possible to distinguish between them. For instance, many uses of a smartphone involve engagement with the Internet or various SNSs. This is especially the case for scales targeting Internet-related behaviours. While an imperfect solution, scales were allocated to a category on the basis of their titles and original descriptions. Finally, while the a priori dimensions used were based on related previous research [16] , it is acknowledged that, although general characteristics of behavioural addictions are likely to exist [8] , dimensions pertinent to one domain might not be well-suited to other domains (e.g., Internet vs. gaming-related behaviour). Similarly, in using Lortie and Guitton [16] 's dimensions the 'tolerance' dimension from the component model [8] was not included as an explicit dimension itself. Rather, as with Lortie and Guitton [16] , it was considered to be an element of 'compulsive use'.

Building on previous analyses [16] focusing on only a single technology, this study aimed to systematically review the dimensional nature of self-report scales used to assess various forms of problematic media use in relation to prominent models and diagnostic criteria for substance dependence and behavioural addiction [8, 16] for four technologies: Internet, smartphones, video games and SNSs. Lortie and Guitton [16, p. 108] note that the dimensional structure of an assessment questionnaire is inherently linked to the researchers' conception of the phenomenon of interest. Consequently, the findings of the study provide a lens through which to interpret current conceptions of purportedly pathological interactions with technology. Overwhelmingly, a majority of scale-items concern compulsive use, negative outcomes associated with use and, to a lesser extent, withdrawal symptoms. Therefore, whether intentional or not, these dimensions are seen by researchers to be central to various conceptions of problematic use. Building on this study, efforts should be made to establish conceptual clarity about what constitutes problematic or addictive technology interactions and, on this basis, curb the development of novel scales, consolidate existing items, and conduct more extensive validations of current scales. Only with more sophisticated and theoretically-sound measurement tools will we be able to produce the empirical evidence necessary for truly understanding the existence and possible nature of problematic engagements with various digital technologies.

Time flies when you're having fun: cognitive absorption and beliefs about information technology usage

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th edn

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders (DSM-5)

Psychology as the science of selfreports and finger movements: whatever happened to actual behavior?

Development of a mobile phone addiction craving scale and its validation in a Spanish adult population

Do smartphone usage scales predict behavior?

Technological addictions

A 'components' model of addiction within a biopsychosocial framework

Working towards an international consensus on criteria for assessing internet gaming disorder: a critical commentary on Petry et

Meeting the unique challenges of assessing internet gaming disorder

A critical account of DSM-5 criteria for internet gaming disorder

Internet addiction: a systematic review of epidemiological research for the last decade

Chaos and confusion in DSM-5 diagnosis of internet gaming disorder: issues, concerns, and recommendations for clarity in the field

Internet addiction and problematic internet use: a systematic review of clinical research

The measurement of internet addiction: a critical review of existing scales and their psychometric properties

Internet addiction assessment tools: dimensional structure and methodological status

An international consensus for assessing internet gaming disorder using the new DSM-5 approach

Analyzing the past to prepare for the future: writing a literature review

Browsing as the killer app: explaining the rapid success of Apple's iPhone

World Health Organization: The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. World Health Organization

Psychology of computer use: XL. Addictive use of the internet: a case that breaks the stereotype