key: cord-0755153-5qmukk38 authors: Laranjeira, Carlos; Dixe, Maria Anjos; Valentim, Olga; Charepe, Zaida; Querido, Ana title: Mental Health and Psychological Impact during COVID-19 Pandemic: An Online Survey of Portuguese Higher Education Students date: 2021-12-29 journal: Int J Environ Res Public Health DOI: 10.3390/ijerph19010337 sha: 10d3203b1a0c2b92df5aacb45fdfe15db3f6e77f doc_id: 755153 cord_uid: 5qmukk38 The COVID-19 pandemic has had significant psychological impact on vulnerable groups, particularly students. The present study aims to investigate the mental and psychological impact of the COVID-19 pandemic and associated factors in a sample of Portuguese higher education students. An online cross-sectional study was conducted among 1522 higher education students selected by convenience sampling. The survey assessed mental health symptoms as well as sociodemographic variables, health-related perceptions, and psychological factors. Results were fitted to binary and multivariable logistic regression models. The overall prevalences of stress, anxiety, and depression were 35.7%, 36.2%, and 28.5%, respectively. Poor mental health outcomes were related with being female, having no children, living with someone with chronic disease, facing hopelessness, and lacking resilient coping. Future studies focusing on better ways to promote mental health and wellbeing among students are warranted. It is necessary to gather more evidence on the post-pandemic mental health using robust study designs and standardized assessment tools. On 30 January 2020, the World Health Organization (WHO) declared the Coronavirus Disease 2019 (COVID-19) as a public health emergency of international concern, calling on all countries to prepare for containment of the disease [1] . The virus rapidly spread throughout the globe and, on 11 March 2020, the WHO declared the novel coronavirus outbreak as a global pandemic [2] . With no treatment options yet available, several countries adopted healthcare protocols to mitigate the spread of the virus, including measures restricting free circulation and, in some cases, enforcing quarantine and social isolation [3] . In Portugal, a state of emergency was announced on 18 March. Although isolation, quarantine, and social distancing were vital to prevent the spread of the virus, protect peoples' physical health, and manage medical resources, they may have also led to long-lasting negative consequences, particularly on mental health and wellbeing [4] [5] [6] . In fact, these preventive measures may have increased the levels of depression and/or anxiety by inducing separation of significant relatives and increasing the perception of isolation and loneliness [7] . After a traumatic event, people may suffer from a variety of mental health problems [8] , and some pre-existing psychiatric conditions may worsen, such as the risk of suicidality, An online cross-sectional design was applied. This study was reported following the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist. Our participants were Portuguese Higher Education students registered in four educational institutions during the academic year 2020/2021. The estimated sample size was obtained from the Raosoft sample size calculator [42] . For a total population of 40,000 students (an average of 10,000 students per institution), with a confidence level of 95%, the sample size should be 381 students. The inclusion criteria were (a) adults (over the age of 18); (b) level of education beyond high school, including undergraduate and graduate programs; (c) living in Portugal at the time of data collection; and (d) able to read and understand Portuguese. Students participating in Erasmus or other mobility programs were excluded. Using a convenience sampling method, self-reported data were collected between April 2020 to October 2020 through emails and online social networking platforms. This period corresponded to the first wave of COVID-19 in Portugal. The survey was delivered via Google Forms. Thus, the online distribution of the survey enabled fast access by a large number of students. A network IP address restriction was set to prevent someone from answering multiple times. A pilot study, with a group of 20 students, validated the content and anonymization, and gathered user feedback. Completion of the survey lasted about 10 min. After addressing minor bugs and reformulating questions to avoid possible common method biases caused by wording and distribution of items, the survey was deployed more widely. To increase responses rates, the study was designed to send follow-up reminders every 30 days during the study period. During survey activation, we obtained 390 responses, of which 29 were deemed incomplete and excluded. The invitation to participate was also emailed to 2000 potential participants using institutional mailing lists. After checking, only 1161 responses were considered valid. Thus, overall, 1522 participants completed the survey (a response rate of 64%). The e-questionnaire was divided into five sections. The first section covered the sociodemographic and academic characteristics of the students, including age, gender, marital status, children, living with someone with a chronic condition, present level of study, if they were working students, type of educational institution, study area, and if educational activity continued during the pandemic. The second section asked students about their perception of getting COVID-19, and perception of mental, physical, and global health. Participants answered on a scale from 1 (lowest value) to 10 (highest value), using the last month as a reference. This was followed by a section on how frequently they felt agitated, anxious, down, or sad due to physical distance measures (staying at home, closing schools and universities, closing non-essential goods, etc.). Participants answered on a four-point Likert scale ranging from 1 (never) to 4 (every day). The fourth section concerned participant mental health status, using the Portuguese version of the Depression, Anxiety, and Stress Scale (DASS-21) [43, 44] . The total depression subscale score was divided into normal (0-9), mild (10) (11) (12) , moderate (13) (14) (15) (16) (17) (18) (19) (20) , severe (21) (22) (23) (24) (25) (26) (27) , and extremely severe depression (28) (29) (30) (31) (32) (33) (34) (35) (36) (37) (38) (39) (40) (41) (42) . The total anxiety subscale score was divided into five categories [45] : normal (0-6), mild (7) (8) (9) , moderate (10) (11) (12) (13) (14) , severe (15) (16) (17) (18) (19) , and extremely severe anxiety . Finally, the total stress subscale score was divided into normal (0-10), mild (11) (12) (13) (14) (15) (16) (17) (18) , moderate (19) (20) (21) (22) (23) (24) (25) (26) , severe (27) (28) (29) (30) (31) (32) (33) (34) , and extremely severe stress (35) (36) (37) (38) (39) (40) (41) (42) . This questionnaire is a reliable and valid measure for assessing the mental health of a Portuguese-speaking population [43] . In our study, the Cronbach's alpha coefficient of the DASS-21 was 0.923 for the stress scale, 0.891 for the anxiety scale, and 0.922 for the depression scale. The last section concerned psychological factors, including hope and resilient coping. The hope levels were assessed through the Portuguese version of the Herth Hope Index (HHI) [46, 47] . The HHI measures different dimensions of hope, using a four-point Likert rating scale ranging from 1 (strongly disagree) to 4 (strongly agree). The scale has an overall score ranging from 12 to 48, where a higher score indicates a higher level of hope. Cronbach's alpha for the present study was 0.875. Resilience was defined as an individual's ability to recover from stress despite significant adversity. Participants' tendency to cope adaptively was assessed using the Portuguese version of the Brief Resilient Coping Scale [BRCS] [48, 49] . This is a four-point measure, with the following cut-off points: low resilience from 4 to 13 points; medium resilience from 14 to 16 points; high resilience from 17 to 20 points. The instrument showed high-reliability rates, with a Cronbach's alpha coefficient of 0.815. The protocol of the study was formally approved by the local ethics committee (CE/IPLEIRIA/22/2020). As research data involved human participants, the study was conducted according to the Helsinki Declaration guidelines [50] . Personal data was processed following the EU General Data Protection Regulation (GDPR2016/679). Only the research manager had access to data identifying the participants. The full questionnaire began with information indicating: (1) the purpose of the study; (2) that participation is voluntary; (3) that participation can be terminated at any time, and (4) that all data is handled confidentially. Consent from each student was obtained at the beginning of each online survey. No monetary gifts were given for completing the questionnaire. Administration of the survey considered methods to reduce bias due to social desirability, acquiescence, leniency effects; guaranteeing anonymity; and encouraging respondents to answer honestly. Descriptive data analysis was performed using frequencies, percentages, mean ± standard deviation (SD), and ranges. Pairs of groups were compared using independent samples t-tests for continuous variables. One-way analysis of variance (ANOVA) was performed to evaluate the significance of differences between more than 2 groups. When the effect of ANOVA was significant, the Games-Howell post-hoc tests were used to detect which groups were different. The association between mental health outcomes with each covariate was assessed with bivariate logistic regression to identify potential candidates for multivariate logistic regression [51] . Those variables having a p-value less than 0.05 were entered into the multivariate logistic regression model. Cut-off points were found for each outcome variable: 'depressed' individuals with scores greater than nine; 'anxious' individuals with scores greater than seven; and 'stressed' individuals with scores greater than 14. When a significant independent variable was not a continuous variable, it was converted to a dummy variable for use. Odds ratios (OR) with the corresponding 95% confidence interval (95%CI) were computed to assess the strength of association [51] . Data were analyzed using Statistical Package for the Social Science (SPSS version 27 for Windows, IBM Corp., Chicago, IL, USA). A total of 1522 students (75.1% women and 24.9% men) took part in this investigation. Full demographic and academic characteristics are presented in Table 1 . The ages ranged from 18 to 59, with a mean age of 22.88 ± 6.93 years. Most participants were single (91.2%), had no children (93%), and did not live with someone with a chronic condition (73.7%). Regarding the level of studies, most students attended the Public Polytechnic Higher Education (73.7% at the undergraduate level) and were not working while studying (76.6%). In terms of field of study, about 95% of students were enrolled in programs other than health science. Due to the pandemic, most of the students maintained distance teaching, learning, and training activities (80.9%). Participants reported a moderate risk (51.9%) of getting COVID-19 infection (Table 2) . Regarding their perception of mental, physical, and global health during the last month, students reported a range of 6.16 ± 2.14 regarding mental health, 6.51 ± 1.99 for physical health, and 6.69 ± 1.79 for global health. Around 55% of the students spent a few days feeling agitated, anxious, down, or sad due to physical distance measures. Among the participants, 35.7%, 36.2%, and 28.5% had symptoms of stress, anxiety, and depression above the normal range, respectively. High resilience scores were found in 215 participants (14.1%). The mean hope (HHI) was 35.53 (SD = 5.92). Comparisons between genders and levels of study are displayed in Table 3 . There were gender differences in mental health perception, with females presenting lower levels than males [t(1522) = 2. 49 Perception of mental health during last month 6.16 ± 2.14 (1-10) Perception of physical health during last month 6.51 ± 1.99 (1-10) Perception of global health during last month 6.69 ± 1.79 (1-10) Table 3 . Comparisons between health-related perceptions, mental health symptoms, and psychological factors, according to the student's characteristics (gender and level of study). A binary regression analysis assessed the association between dependent and independent variables, as displayed in Table 4 . Female scores were significantly higher than males on measures of stress and anxiety, but not depression. Students with no children had a significantly lower risk of stress, anxiety, and depression. Students studying health sciences had a higher risk of depression. Participants who did not work had a significantly lower risk of stress, anxiety, and depression. Participants who were living with someone with a chronic condition had a significantly higher risk of stress and anxiety, but not depression. Additionally, participants lacking hope or resilient coping had a significantly higher risk of stress, anxiety, and depression. The multivariable logistic regression revealed that the odds of stress were strongest among female college students (OR = 1. Table 4 . Regression analysis of variables by stress, anxiety, and depression among students (n = 1522). Yes The present study detected levels of depression, anxiety, and stress among higher education students that are in line with other studies [8] : 17.2%, 23.6%, and 6% of participants reported moderate to extremely severe symptoms of depression, anxiety, and stress, respectively. However, these results are somewhat lower than those reported in previous COVID-19 Portuguese studies [36, 52] , especially for depression and stress. In contrast, the severity of mental health symptoms was higher than in Paulino et al. [26] . Similar with Ferreira et al. [36] these differences may be related to when data was collected. While the current study collected the data between April and October 2020, the previous studies had collected their data in late March-after the beginning of the first national lockdown. Overall, most of our sample reported normal levels of depression, anxiety, and stress. These results contradict several published studies performed during the COVID-19 pandemic that assessed its psychological impact and addressed its negative psychological consequences [13, 14, 16, 20, 53, 54] . Differences may be due to the instruments used to measure variables, the sample size, the curricular load, and the existing socio-cultural differences among countries [55, 56] . Considering these contradictory findings, it is important to continue to study this population (university students), namely trying to better understand the possible resilience factors that may buffer the effects of the COVID-19 pandemic. Nevertheless, the researchers suggest that, in some people, the isolation, fear of contracting SARS-CoV-2, and uncertainty during the pandemic may have led to the collection of symptoms that make up health anxiety linked to COVID. This condition is a relatively new concept, manifests as the inability to leave the house because of COVID-19 fears, frequent checking for symptoms despite not being in a high-risk scenario, and avoiding social situations or people [57] . This issue needs further investigation. Participants reported a relatively good average perception of mental, physical, and global health. This result reveals that most participants did not negatively report their mental health. Moreover, most students spent only a few days feeling agitated, anxious, down, or sad due to physical distance measures, suggesting they may have found mechanisms to overcome the negative consequences of physical distance measures. This result is relevant, as greater self-perceived health has been associated with a greater experience of depression, anxiety, and psychological distress in various populations [58] . This reinforces the importance of research on positive outcomes and protective factors [59] . Several group differences were found. For instance, females presented lower levels of perception of mental, physical, and global health than men, in line with previous research in the Portuguese population during the COVID-19 pandemic [19, 53, 54] . Moreover, this tendency is typical in the literature from before the COVID-19 pandemic [60] , suggesting these differences remained even during a pandemic situation. Several studies conducted during the pandemic have confirmed the greater psychological vulnerability in women compared to men [58, 61, 62] . Although females reported reduced perception of mental, physical, and global health, it is noteworthy that females presented higher levels of hope than men. This may seem contradictory, but can be explained given the nature of hope, as described in the literature: describing oneself as being in a hopeless state with poorer self-perceived health, does not mean that hope has entirely vanished [63] . Some interesting results were also found when comparing participants regarding their attending level of study. Undergraduate students presented higher levels of stress than other students. Differences were also found in levels of hope and resilient coping, with undergraduate students presenting lower levels compared to master's students, and master's students presenting higher levels of hope and coping. This is consistent with previous studies reporting that students expressed a need for emotional and psychological support mainly to maintain high performance levels, which can be explained by resilience theory [64] . Academically engaged students tend to be more internally resilient [65] and capable of adapting to changes brought about by external threats and perceived crisis, better known as the need-threat internal resilience [66] . Resilient coping is strongly related with perceived emotional support and close, safe relationships [67] . These networks of emotional and social support serve as buffers to stress and its negative consequences, and provide resources for coping with stress, thus helping to curtail more severe problems. Our results provide useful insights into the predictors of mental health during the COVID-19 pandemic. Poor mental health outcomes seems to be related with being female, having no children, living with someone with chronic disease, studying health sciences, feeling hopelessness and lacking resilient coping. Some associations were reported in previous studies involving similar samples [68] . Women carry a heavier burden to ensure a family's overall wellness, and feel greater pressure with working and/or studying, as this requires a balance between work and family commitments [69] . Therefore, having children and living with relatives with a chronic disease may increase the fear and anxiety related with COVID-19 [65] , in addition to the societal responsibility already imposed on a woman [69] . Typically, students from health sciences experience considerable stress and depressive symptoms during their training, precipitated by many factors, such as adjustment to the novel school environment, information overload, lack of leisure time, financial constraints, family-related stressors, and intense competition for higher grades [70] . Restrictions due to the pandemic delayed numerous clinical teachings and consequently the development/completion of courses, further aggravating the students' signs of distress. The rapid changes imposed upon higher education students, from the suspension of classes to the declaration of a state of emergency, triggered difficulties in adaptation and less positive emotional states [40] . Another interesting finding was that hope and resilient coping had protective effects by reducing the probability of experiencing depression, anxiety, and stress. One recent study [71] showed that hope can lead to positive outcomes, even during a pandemic, as having more positive expectations regarding the future and feeling one leads a meaningful life can mediate how basic health predicts levels of stress, such as those generated by the pandemic. Students have demonstrated the incredible human capacity to reorient under new conditions. Hope seems to be closely linked to wellbeing, as hopeful people approach stressful events in healthier ways. In addition to this innate potential, flexibility and creativity should be emphasized in educational training and institutions can steer education models toward more adaptable practices [72] . The current study has significant implications for practice and policy. First, developing programs to improve mental health should be a priority in a government's response to this pandemic crisis [36] . These plans need to be cross-sectoral, combining measures to protect and promote mental health with actions to treat additional mental ill-health [73] . Second, communication is an important resource in dealing with mental health issues. Higher education institutions play an important role by providing students with educational materials, but also with opportunities to interact with peers and teachers [74] . Universities should also provide guidelines and principles for effective online learning and should ensure that content meets educational requirements, while not overburdening students. Third, healthcare services can communicate with students through social media platforms to help them cope with mental health issues [74] . This can help to maximize the opportunities to reach all individuals in mental distress through psychological counseling services [73] . Restrictions required that psychological and psychiatric support adopt strategies such as remote tools, including videoconferences or other web-based technologies [52, 75] . Finally, as hope can help deal with uncertainties, difficulties, and stressful situations, it is important to research hope and that institutions and societies adopt a language of hope during situations of crisis, and even develop specific hope-focused interventions to mitigate the psychological impact of this and future pandemics [63] . The present study has several limitations that should be noted. First, the cross-sectional design does not provide solid and causal evidence for detected associations. The variables studied can be better understood with longitudinal designs. Second, our final sample comprised mainly female undergraduate students recruited online. A previous study indicated that social media particularly attracts people who are distressed, looking for support [76] . This could bias the data and hinder any representativeness of our sample. This fact, as well as a non-probability sampling technique, decreased the external validity of our study. Additionally, data collection covered a large temporal window (7 months). When our data collection ended (October 2020), fear and anxiety related to COVID-19 was less present in the public consciousness than at the beginning of the pandemic, and thus less likely to have been a major factor influencing the presentation of mental health symptoms. Due to the anonymous nature of responses, the study could not assess whether nonresponders might be students with little interest in the issue of "mental health in the time of COVID-19". The nonresponse bias cannot be investigated online, since the identity of non-respondents is generally unknown [77] . Future studies should integrate a subsequent follow-up system in their procedure, as proposed in Rogelberg and Stanton [78] , to assess possible non-response bias. Despite these limitations, the current study has a number of strengths such as (a) the potential contribution to the mental health field; (b) the use of standard and validated self-report measures that allow the assessment of study variables; and (c) a good sample size. Finally, the study does not focus solely on negative variables, but assesses also positive variables, such as hope and resilient coping. This study found a moderate prevalence of stress, anxiety, and depression in higher education students and highlighted some risk factors significantly associated with these symptoms. These risk factors should be addressed by the respective stakeholders (e.g., university teachers, healthcare practitioners, and public health policymakers) to facilitate a compassionate and mental health-friendly environment for higher education students. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Participation in the study was completely voluntary and anonymous. Implicit consent for the project was assumed when study participants completed the survey. Participants received no compensation. Data Availability Statement: All data generated or analyzed during this study are included in this article. Novel Coronavirus (2019-nCoV): Situation Report-10; WHO: Geneva, Switzerland, 2020. 2. World Health Organization. Novel Coronavirus (2019-nCoV): Situation Report-51 World Health Organization. Critical Preparedness, Readiness and Response Actions for COVID-19-Interim Guidance The psychological impact of quarantine and how to reduce it: Rapid review of the evidence Mental health prevalence and predictors among university students in nine countries during the COVID-19 pandemic: A cross-national study COVID-19: Factors associated with psychological distress, fear, and coping strategies among community members across 17 countries Social isolation in Covid-19: The impact of loneliness Influence of Multiple Traumatic Event Types on Mental Health Outcomes: Does Count Matter? World Health Organization. Mental Health in Emergencies; WHO: Geveva The Risk and Prevention of Novel Coronavirus Pneumonia Infections Among Inpatients in Psychiatric Hospitals Addressing mental health needs: An integral part of COVID-19 response The Impact of Community Disaster Trauma: A Focus on Emerging Research of PTSD and Other Mental Health Outcomes The psychological impact of the COVID-19 epidemic on college students in China Increases in Depression and Anxiety Symptoms in Adolescents and Young Adults during the COVID-19 Pandemic Gender differences of depression and anxiety among social media users during the COVID-19 outbreak in China: A cross-sectional study Exploring lifestyle habits, physical activity, anxiety, and basic psychological needs in a sample of portuguese adults during covid-19 Anxiety and depression in the Republic of Ireland during the COVID-19 pandemic Levels and predictors of anxiety, depression, and health anxiety during COVID-19 pandemic in Turkish society: The importance of gender Anxiety and coping strategies among nursing students during the covid-19 pandemic Prevalence of Depression Symptoms in US Adults Before and During the COVID-19 Pandemic Stress and parenting during the global COVID-19 pandemic Psychiatry in the age of COVID-19 Do psychiatric patients experience more psychiatric symptoms during COVID-19 pandemic and lockdown? A case-control study with service and research implications for immunopsychiatry Psychological Impact of the COVID-19 Pandemic on Health Care Workers in Singapore Nurses' Mental Health During the Covid-19 Outbreak: A Cross-Sectional Study COVID-19 in Portugal: Exploring the immediate psychological impact on the general population Navigating change: A typology of student transition in higher education Burnout in medical students before residency: A systematic review and meta-analysis Prevalence and risk markers of anxiety and depression among health students Depression and anxiety among university students during the COVID-19 pandemic in Bangladesh: A web-based cross-sectional survey Closure of Universities Due to Coronavirus Disease 2019 (COVID-19): Impact on Education and Mental Health of Students and Academic Staff Health Behavior: Theory, Research and Practice Lazarus and Folkman's Psychological Stress and Coping Theory; The Handbook of Stress and Health The Impact of Hope and Resilience on Multiple Factors in Neurosurgical Patients Dealing with the Pandemic of COVID-19 in Portugal: On the Important Role of Positivity, Experiential Avoidance, and Coping Strategies Do coping responses predict better/poorer mental health in Portuguese adults during Portugal's national lockdown associated with the COVID-19? Pers Mental health of young people: A global public-health challenge Anxiety, depression, and stress in university students: The impact of COVID-19 COVID-19 pandemic and its psychological impact among healthy Portuguese and Spanish nursing students Psychiatric symptoms, risk, and protective factors among university students in quarantine during the COVID-19 pandemic in China RaoSoft®Sample Size Calculator Adaptation to Portuguese of the Depression, Anxiety and Stress Scales (DASS) The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories Mental Health Burden in Different Professions During the Final Stage of the COVID-19 Lockdown in China: Cross-sectional Survey Study Abbreviated instrument to measure hope: Development and psychometric evaluation Avaliação da esperança em cuidados paliativos: Tradução e adaptação transcultural do Herth Hope Index Adaptação portuguesa da escala breve de coping resiliente The development and psychometric evaluation of the Brief Resilient Coping Scale World medical association declaration of Helsinki: Ethical principles for medical research involving human subjects Using Multivariate Statistics Protective Elements of Mental Health Status during the COVID-19 Outbreak in the Portuguese Population The relationship between anxiety levels, sleep, and physical activity during COVID-19 lockdown: An exploratory study Higher physical activity levels may help buffer the negative psychological consequences of COVID-19 pandemic Antón-Solanas, I. The Prevalence of Depression, Anxiety and Stress and Their Associated Factors in College Students Prevalence and predictors of depression and anxiety among medical students in Addis Ababa COVID-19 health anxiety Anxiety, Health Self-Perception, and Worry About the Resurgence of COVID-19 Predict Fear Reactions Among Genders in the Cuban Population. Front. Glob Measuring anxiety related to COVID-19: A turkish validation study of the coronavirus anxiety scale Estrutura fatorial e validade de construto da escala Generalized Anxiety Disorder 7-item (GAD-7) entre alunos universitários em Portugal. Cad Immediate Psychological Responses and 347 Associated Factors during the Initial Stage of the 2019 Coronavirus Disease (COVID-19) Epidemic among the 348 General Population in China Hasson-Ohayon, I. Risk factors for psychological distress during the COVID-19 pandemic in Israel: Loneliness, age, gender and health status play an important role Amirav, I. Hope During COVID-19 Lockdown Resilience in individual development: Successful adaptation despite risk and adversity Optimism and resilience among university students Predictors of Mental Health Status among Quarantined COVID-19 Patients in Saudi Arabia Students under lockdown: Comparisons of students' social networks and mental health before and during the COVID-19 crisis in Switzerland Impact of COVID-19 on the mental health of US college students The New Roles of Men and Women and Implications for Families and Societies Health professions students have an alarming prevalence of depressive symptoms: Exploration of the associated factors Loss, and resilience in the time of COVID-19: Meaning making, hope, and transcendence Change, Hope: Student Perspectives During the COVID-19 Pandemic Viewpoint: Investing in strategies to support mental health recovery from the COVID-19 pandemic College students stress and health in the COVID-19 pandemic: The role of academic workload, separation from school, and fears of contagion COVID-19 and psychiatrists' responsibilities: A WPA position paper Mental health problems and social media exposure during COVID-19 outbreak Assessing response rate and nonresponse bias in web and paper surveys Understanding and Dealing with Organizational Survey Nonresponse We acknowledge all the volunteers who participated in the online survey to make this study possible. The authors declare no conflict of interest. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.