key: cord-0756448-aau0hnwx
authors: Ge, Fenfen; Zhang, Di; Wu, Lianhai; Mu, Hongwei
title: Predicting Psychological State Among Chinese Undergraduate Students in the COVID-19 Epidemic: A Longitudinal Study Using a Machine Learning
date: 2020-09-17
journal: Neuropsychiatr Dis Treat
DOI: 10.2147/ndt.s262004
sha: 2b799ab6654e83e4144c83453e737e40ce832a2d
doc_id: 756448
cord_uid: aau0hnwx

BACKGROUND: The outbreak of the 2019 novel coronavirus disease (COVID-19) not only caused physical abnormalities, but also caused psychological distress, especially for undergraduate students who are facing the pressure of academic study and work. We aimed to explore the prevalence rate of probable anxiety and probable insomnia and to find the risk factors among a longitudinal study of undergraduate students using the approach of machine learning. METHODS: The baseline data (T1) were collected from freshmen who underwent psychological evaluation at two months after entering the university. At T2 stage (February 10th to 13th, 2020), we used a convenience cluster sampling to assess psychological state (probable anxiety was assessed by general anxiety disorder-7 and probable insomnia was assessed by insomnia severity index-7) based on a web survey. We integrated information attained at T1 stage to predict probable anxiety and probable insomnia at T2 stage using a machine learning algorithm (XGBoost). RESULTS: Finally, we included 2009 students (response rate: 80.36%). The prevalence rate of probable anxiety and probable insomnia was 12.49% and 16.87%, respectively. The XGBoost algorithm predicted 1954 out of 2009 students (translated into 97.3% accuracy) and 1932 out of 2009 students (translated into 96.2% accuracy) who suffered anxiety and insomnia symptoms, respectively. The most relevant variables in predicting probable anxiety included romantic relationship, suicidal ideation, sleep symptoms, and a history of anxiety symptoms. The most relevant variables in predicting probable insomnia included aggression, psychotic experiences, suicidal ideation, and romantic relationship. CONCLUSION: Risks for probable anxiety and probable insomnia among undergraduate students can be identified at an individual level by baseline data. Thus, timely psychological intervention for anxiety and insomnia symptoms among undergraduate students is needed considering the above factors.

The 2019 novel coronavirus disease is caused by a variety of coronavirus (SARS-CoV-2). In March 2020, the World Health Organization (WHO) declared COVID-19 as a global pandemic. 1 The symptoms of COVID-19 are usually nonspecific (eg, fever, cough, and dyspnea). 2 Up to now, COVID-19 is significantly contagious and no effective treatments or vaccines are available. 3 As of 19 June 2020 (10:00 am CET), over 8.45 million cases have been diagnosed globally with more than 453,000 fatalities. 4 Many unprecedented strategies have been taken to cut off the spread of the virus in many countries (eg, China, England, and the United States). For example, the Chinese government released some guidance for the general population to self-isolate. Specifically, lockdown (eg, in Wuhan), temporary closure of schools/factories, and restriction of individuals' activities Isolation and quarantine measures have been effective at preventing the spread of COVID-19. However, consensus has arisen about their potential mental health. 5, 6 In the severe acute respiratory syndrome epidemic, confirmed cases suffered higher stress levels, poor sleep quality, and depressed mood. 7 Recently, Brooks' review suggested that the mental health impact of quarantine is wide-ranging and substantial. 5 In the present epidemic of COVID-19, higher anxiety level (23.04%) was reported among medical staff. 8 And Zhang's study found that more than onethird of medical staff suffered insomnia symptoms. 9 In addition to special groups (eg, medical staff and confirmed cases), Wang's study found that about 33.33% of the general population reported moderate-to-severe anxiety. 10 Isolated people may experience stressful conditions (eg, social activities and face to face communication were restricted). 11 Thus, negative emotions experienced by people may be compounded. 11, 12 Undergraduate education is a special stage that is highly specialized, knowledgeable, and continuous. Theory teaching and practical teaching are two profiles as well as two taches in the same teaching life-cycle. The epidemic prevented undergraduate students from returning to university. Thus, undergraduate students' psychological state may fluctuate. 13, 14 We found that previous studies have some limitations. Firstly, most research focuses on the mental health of medical students, and ignores the mental health of students of other disciplines (eg, humanities, engineering, and agricultural). Although medical students are a special group, they only constitute a very small part of undergraduate students. Secondly, most of the studies have a cross-sectional design, only exploring the psychological state and risk factors of undergraduate students during the COVID-19 epidemic.

In summary, we aimed to investigate the prevalence rate of probable anxiety and probable insomnia and to confirm the risk factors among undergraduate students during the COVID-19 outbreak. Finally, we ranked risk factors in the model based on "feature importance."

We have obtained the Ethics Committee of the Ocean University of China (2,020,001). An online version of an informed consent form is provided to students before starting the survey. In the form, we explained to the students that participation was voluntary, and refusals would have no negative consequences. We also guaranteed data confidentiality and that only the researchers could access the information. Undergraduate students can choose to participate in or reject the survey. If they choose to participate in the survey, they are evaluated via an online platform. If they refuse to take part in the survey, they withdraw from the online platform. We have obtained informed consent from students who accomplished the survey. The target population of the research was undergraduate students at the Ocean University of China. The Ocean University of China is a government university located in Shandong Province. The baseline survey (T1) was collected from freshmen. Specifically, all freshmen underwent psychological evaluation at two months after entering the Ocean University of China. In the presence of the COVID-19 epidemic (T2, February 10th to 13th, 2020), we used convenience cluster sampling and invited undergraduate students (n=2500) from four grades (freshman, sophomore, junior and senior) to participate in the survey. Finally, 2009 participants completed the webbased survey (response rate: 80.36%). We used the students' ID numbers to match the data.

Basic demographic information was collected at T1 stage. We collected basic characteristics using a self-constructed questionnaire. The questionnaire included gender (female=1, male=2), year of education (first=1, second=2, third=3, fourth=4), family economic (low family economic=1, high family economic=2), upbringing place (metropolitan=1, medium and small cities=2, town=3, country=4) and singlechild families (yes=1, no=2).

The Ministry of Education of the People's Republic of China recommended the College Students Mental Health Screening Scale (CSMHSS) as a reliable and valid measurement tool to evaluate the mental health of undergraduate students. The scale of CSMHSS consists of 22 dimensions. Specifically, it includes psychotic experiences (4 items), suicidal ideation (3 items), the history of anxiety symptoms (4 items), the history of depression symptoms (5 items), paranoia (4 items), inferiority (5 items), sensitivity (4 items), social phobia (4 items), somatization (4 items), dependence (4 items), aggression (4 items), impulsive (4 items), obsession and compulsion (4 items), Internet addiction (5 items), self-injury (4 items), eating problems (4 items), sleeping problem (4 items), school adjustment difficulties (4 items), interpersonal distress (4 items), academic pressure (4 items), employment pressure (4 items) and romantic relationship problems (4 items). The scale uses 4-point Likert-scaled items ranging from 1 (not at all like me) to 4 (very like me). The scores in each dimension are added then standardized. The standard score of the total score in each dimension can be categorized at a cutoff of 3 (Fang, Yuan, Hu, Deng, and Lin, 2018).

The scale of General Anxiety Disorder (GAD-7) was used in this research. It was a tool to assess anxiety symptoms. A score ≥7 indicated clinically significant anxiety symptoms. 15,16 GAD-7 was well validated and sensitive to the general population. [17] [18] [19] [20] Studies had shown that GAD-7 has good reliability and validity in China. 21, 22 The Cronbach's alpha value was 0.90 for GAD-7 in this research.

We used the Insomnia Severity Index (ISI-7) which is a 7item instrument that evaluates subjective sleep symptoms. Each item is scored on a 5-point scale, with higher scores representing more severe insomnia symptoms. A score >14 indicates clinically significant insomnia symptoms. 23 The Cronbach's alpha value was 0.86 for ISI-7 in this research.

Descriptive data analysis was implemented in SPSS 21.0 for windows. XGBoost (Extreme Gradient Boosting), which is a machine learning algorithm, was implemented in Python 3.70. XGBoost is a method for regression and classification problems according to the Gradient Boosting Decision Tree. This method has been widely used in all kinds of data fields for regression and classification. 24 The algorithm of XGBoost can utilize a cross-validation approach to divide data into a model "training set" and "testing set." In the current research, we used a 5-fold cross-validation method. Classification performance was scored with the area under the receiver-operation Curve (AUC), sensitivity (Sen), specificity (Spe), and accuracy (ACC).

Finally, there were 2009 undergraduate students who were included in our research. Of the 2009 participants, 50.97% were female, 79.99% were from high family economic background, and 25.35% were from metropolitan areas. In the present epidemic of COVID-19, the prevalence rate of probable anxiety and probable insomnia symptoms was 12.49% (GAD-7≥7) and 16.87% (ISI-7>14), respectively. The detailed basic characteristics are shown in Table 1 .

We integrated the data collected at T1 stage to predict probable anxiety and probable insomnia during the COVID-19 epidemic (at T2 stage). The AUC of probable anxiety and probable insomnia is 99.00% and 98.00%, respectively. Figure 1 shows the AUCs for probable anxiety and probable insomnia.

According to the AUCs and the confusion matrix, we calculated the sensitivity (Sen), specificity (Spe), and accuracy (ACC). The machine learning of XGBoost predicted 1954 out of 2009 as either anxiety or no-anxiety and this translated into 97.3% accuracy (97.3% sensitivity and 96.3% specificity). The machine learning of XGBoost predicted 1932 out of 2009 as either insomnia or no-insomnia and this translated into 96.2% accuracy (95.5% sensitivity and 100.0% specificity). Detailed information is demonstrated in Table 2 . Feature importance assigned positive coefficients via XGBoost, indicating that an increase in probable anxiety included 1) romantic relationship problems, 2) suicidal ideation, 3) the history of anxiety symptoms, and 4) sleep symptoms. It was also indicated that an increase in probable insomnia included 1) aggression, 2) psychotic experiences, 3) suicide ideation, and 4) romantic relationship problems. The confusion matrix and "feature importance" are shown in Figures 2A and B and 3A and B, respectively.

In the current research, the prevalence rate of probable anxiety and probable insomnia among undergraduate students was 12.49% and 16.87%, respectively. The prevalence rate of probable anxiety is higher than the Zhang's study (7.5%) and lower than the Cao's study. Cao's study found that 24.9% of medical students suffered from anxiety symptoms. 25 The variability of prevalence rates could be explained by medical students being a special group who face more academic and employment pressure. And previous studies found that medical students are more likely to have psychological problems. 26, 27 The prevalence rate of probable insomnia is the similarity to Huang's study (18.2%) 28 and lower than the 36.1% reported by Zhang's study. 9 This variability of prevalence rates could be explained by the participants, questionnaires, and regions.

Most relevant variables predicting probable anxiety included romantic relationship problems, suicidal ideation, history of anxiety symptoms, and sleep symptoms. Falling in love is a universal behavior among undergraduate students. Studies indicate that youths experience romantic relationships of joy and happiness. However, a romantic relationship is not entirely a happy period of life. Bajoghli's study found that for youths, falling in love may be also associated with anxiety symptoms. 29 Consistent with Asselmann's study, we found that the history of anxiety symptoms prior to/at baseline predicted a recurrence of probable anxiety at the time of follow-up. 30 Narmandakh's study found that sleep disturbance may precede anxiety symptoms. And anxiety symptoms might be prevented by alleviating sleep disturbance. 31 Previous results suggested that the presence of "any anxiety disorder" increases the risk for suicidal ideation among the general population, even after controlling for confounding factors (Wilcox et al, 2010 ). In the current study, we found that suicidal ideation can be used to predict probable anxiety.

The results may indicate that there is a bidirectional relationship between suicidal ideation and anxiety symptoms among youths.

Most relevant variables predicting probable insomnia included aggression, psychotic experiences, suicidal ideation, and romantic relationship problems. Consistent with previous studies, we also found that insomnia is a consequence of psychotic symptoms. 32 Recent studies demonstrate that insomnia also contributed to the development of psychotic symptoms. 33 Insomnia symptoms may be one of the top warning signs of suicide in a clinical outpatient setting. 34 Suh's study also found that insomnia symptoms were related to concurrent and future ideations of suicide in a population-based longitudinal study. 35 And a meta-analysis showed that sleep disturbances in general, as well as insomnia individually, appear to represent a risk factor for suicidal ideation and behavior. 36 Namely, there may be a bidirectional association of insomnia symptoms with psychotic experiences and insomnia symptoms with suicide ideation. Falling in love is an emotional occurrence at any age, but for undergraduate students, the feelings might be overwhelming. 37 In addition to being a positive feeling (eg, joy and happiness), a romantic relationship may cause stress and negative effect, especially if the feeling is not reciprocal. 38 Kuula's study revealed that romantic relationship is one reason for sleep disturbance in girls and may be associated with symptoms of anxiety in both boys and girls. 39 

We have found relatively reliable and accurate predictive models during the COVID-19 epidemic. And our models provide useful information about the most relevant variables to predict DovePress probable anxiety and insomnia among college students. The stage of university education is an important period of life development, and it is very necessary to carry out psychological assessment of freshmen who have just entered the university. Intervening with students with psychological problems in a timely and effective manner would not only help them recover their mental health, but also help them adjust their state when facing emergencies. Stoessel's study found besides being positive feelings, romantic relationships may cause stress and negative effect, especially if the love is not reciprocal. 38 Thus, in addition to resolving regular psychological problems, it is necessary to help college students to establish healthy romantic relationships, one of the principal developmental tasks of emerging adulthood. How to effectively organize the mental health services for those undergraduates who have present anxiety and/or insomnia symptoms due to the COVID-19 pandemic is also very important.

Community-based and school-based mental health services care be combined into the national health system. 40

There are some strengths in this research, including 1) this is a longitudinal study and we use the data at the time of enrollment to predict college students' anxiety and insomnia during the outbreak; 2) we integrate data from multiple dimensions; 3) we calculate models for individual classification using machine learning. However, the current research has some limitations, including 1) our participants are from a specific university located in Shandong province. And this university does not include medical students. Thus, the results cannot be generalized to all Chinese undergraduates. 2) We used self-reported questionnaires in this research, so response bias and recall bias may exist considering that undergraduate students may have underreported or overreported their anxiety and/or sleep symptoms. We took some steps to reduce this by keeping uniformity of data collection approach. It is worth mentioning that we found that romantic relationship trouble is an important factor in predicting anxiety and insomnia. 3) We used different questionnaires at T1 stage and T2 stage. Thus, it is difficult to directly compare the prevalence rate of anxiety and insomnia at two stages. 4) Temperament is stable across the lifespan and mediate adaptive functioning to some extent. And the attachment system may be activated in stressful situations.

Recently, Moccia's research found that some specific affective temperament (eg, cyclothymic and anxious temperaments) and attachment features (eg, need for approval) can be used to predict the burden of mental health. 41 However, information on temperamental and attachment was not collected in our study. Thus, it is necessary for researchers to consider temperament and attachment in future studies.

This longitudinal research contributes to our understanding of the psychological state of undergraduate students who suffered a sudden public health event. And we found a reliable model to predict anxiety and insomnia during the sudden public health. Thus, timely psychological intervention is necessary, not only to help undergraduate students recover their mental health but also to help them face some emergency events.

Fenfen Ge and Di Zhang are co-first authors. The authors report no conflicts of interest in this work.

Neuropsychiatric Disease and Treatment is an international, peerreviewed journal of clinical therapeutics and pharmacology focusing on concise rapid reporting of clinical or pre-clinical studies on a range of neuropsychiatric and neurological disorders. This journal is indexed on PubMed Central, the 'PsycINFO' database and CAS, and

is the official journal of The International Neuropsychiatric Association (INA). The manuscript management system is completely online and includes a very quick and fair peer-review system, which is all easy to use. Visit http://www.dovepress.com/testimonials.php to read real quotes from published authors. 

China coronavirus: WHO declares international emergency as death toll exceeds 200

A novel coronavirus outbreak of global health concern

We signed up for this!" -student and trainee responses to the Covid-19 pandemic

COVID-19): what you need to do

The psychological impact of quarantine and how to reduce it: rapid review of the evidence

The consequences of the COVID-19 pandemic on mental health and implications for clinical practice

Factors associated with psychosis among patients with severe acute respiratory syndrome: a case-control study

Survey of insomnia and related social psychological factors among medical staff involved in the 2019 novel coronavirus disease outbreak

Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China

Recommended psychological crisis intervention response to the 2019 novel coronavirus pneumonia outbreak in China: a model of West China Hospital

Psychosocial effects of an Ebola outbreak at individual, community and international levels

Prevalence of depression and its associated factors among clinical-year medical students in Eastern Province, Saudi Arabia

Medical student mobilization during a crisis: lessons from a COVID-19 medical student response team

Patient-reported outcome measures in community mental health teams: pragmatic evaluation of PHQ-9

A brief measure for assessing generalized anxiety disorder -The GAD-7

Depressive symptomatology among norwegian adolescent boys and girls: the patient health questionnaire-9 (PHQ-9) psychometric properties and correlates

Psychometric properties of the 7-item generalized anxiety disorder scale (GAD-7) in a large representative sample of Finnish adolescents

Screening for depression in primary care: a Rasch analysis of the PHQ-9

Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection

Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population

Reliability and validity of Chinese version of the generalized anxiety disorder 7-item (GAD-7) scale in screening anxiety disorders in outpatients from traditional Chinese internal department

Empirical validation of the insomnia severity index in cancer patients

XGBoost: a scalable tree boosting system

The psychological impact of the COVID-19 epidemic on college students in China

Stress and depression among medical students: a cross-sectional study

Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students

Mental health burden for the public affected by the COVID-19 outbreak in China: who will be the high-risk group?

I love you more than I can stand!" -romantic love, symptoms of depression and anxiety, and sleep complaints are related among young adults

The bidirectional association between sleep problems and anxiety symptoms in adolescents: a TRAILS report

The role of sleep dysfunction in the occurrence of delusions and hallucinations: a systematic review

Insomnia, negative affect, and psychotic experiences: modelling pathways over time in a clinical observational study

Suicidality and sleep disturbances

Longitudinal course of depression scores with and without insomnia in non-depressed individuals: a 6-year follow-up longitudinal study in a Korean cohort

Meta-analysis of sleep disturbance and suicidal thoughts and behaviors

Adolescent romantic relationships

Differences and similarities on neuronal activities of people being happily and unhappily in love: a functional magnetic resonance imaging study

Emotions relating to romantic love-further disruptors of adolescent sleep

Mental health during and after the COVID-19 emergency in Italy

Affective temperament, attachment style, and the psychological impact of the COVID-19 outbreak: an early report on the Italian general population