key: cord-0936039-y1b0o2eg authors: Navarro-Espinosa, Johanna Andrea; Vaquero-Abellán, Manuel; Perea-Moreno, Alberto-Jesús; Pedrós-Pérez, Gerardo; Aparicio-Martínez, Pilar; Martínez-Jiménez, Maria Pilar title: The Influence of Technology on Mental Well-Being of STEM Teachers at University Level: COVID-19 as a Stressor date: 2021-09-12 journal: Int J Environ Res Public Health DOI: 10.3390/ijerph18189605 sha: 2b50faefad9d04dc8e15e7da284b6b46ce6e7706 doc_id: 936039 cord_uid: y1b0o2eg Stress can result in psychopathologies, such as anxiety or depression, when this risk factor continues in time. One major stressor was the COVID-19 pandemic, which triggered considerable emotional distress and mental health issues among different workers, including teachers, with another stressor: technology and online education. A mixed-method approach is presented in this research, combining a cross-sectional study of university teachers from Ecuador and Spain with a medium of twenty years of working experience (N = 55) and a bibliometric analysis carried out in three databases (161 documents). The levels of anxiety and depression, and therefore the risk of developing them as mental disorders, were high. The lack of training (p < 0.01), time (p < 0.05), or research regarding the use of technology in education (p < 0.01) and stress caused by COVID-19 (p < 0.001) were linked to frequency. The most relevant observational study obtained through the bibliometric analysis (138 citations and over 65% of methodological quality) indicated that previous training and behavioral factors are key in the stress related to technology. The combination of the results indicated that mental health in STEM teachers at university is related to diverse factors, from training to the family and working balance. The occupational safety and health (OSH) standards indicated the obligation of any employer to maintain both the physical and psychological well-being of the employees in the working environment [1, 2] . Despite the guidelines and the legal framework, the adaptation of the working environment, especially to maintain mental health, continues to be limited or inadequate [3, 4] . The problem resides in the fact that psychological and psychiatric disorders of workers impact their physiological and biological health as well as the productivity of the corporate, company, or employer [5, 6] . In this sense, different authors have indicated how the prevention or treatment of mental disorders and their psychological component [7] , including the adequation of the working place, equally contributes to workers' well-being [8, 9] . This definition in combination with other authors [44, 45] reflect how STEM education incorporates not only the teaching of scientific or mathematical concepts but also the engineering use and design of technology as well as the skills needed to adequately use such tools [42] . Despite this, Bybee [46] indicates that perhaps this definition depends on the individual's context and needs, being modifiable, subject to the current moment. Besides, a STEM teacher, which can be defined as an educator with a curriculum based on the four disciplines (science, technology, engineering, and mathematics) and understanding of the STEM education, should use technology and know the engineering design that provide the base for the ICTs [42] . Therefore, a STEM teacher is not only someone that instructs different scientific knowledge and skills, but rather has previous training and reasonable comprehension of the dynamics and bases of ICTs in education [47] . Usually, STEM teachers are more commonly framed in elementary or high school education, although the definition of this type of teacher is also applicable to the higher education institutions [48] . Nevertheless, there is no precise description for STEM teachers, much less a conceptualization of STEM teachers at university level [49] . Despite the lack of conceptualization of the figure, several studies have analyzed the perception and aptitudes of pre-service STEM teachers, mainly in the primary and secondary educational level [42, 44, 46, 50] that indicated a need for training. Only one article [51] focused on in-service teachers, both STEM and non-STEM, which showed a concern about the engineering course within STEM disciplines since many STEM and non-STEM teachers had low levels of engineering bases, which contradicts the definition of STEM education [43] . Berisha and Vula [50] indicated in their study how pre-service teachers that received a course in this methodology perceived a lack of collaboration or unwillingness as challenges to implement this education. In this sense, teachers with a STEM background usually avoid including new educational methodologies, such as flip classrooms, gamification, or blended learning, since they tend to carry out the same educational methodologies that they were taught [51] . One reason for the reticence of teachers is that STEM teaching has, as a purpose, deep understanding of science and/or mathematics content, engagement of students through the educational pedagogics and ICTs, and improvement of communication skills and teamwork among students [52] . These teachers' expectations could provoke high levels of stress and other psychological issues [51] . Despite this risk, most studies have indicated that STEM teachers tend to have fewer mental issues, possibly for the training received during the early stages of their career [53] . The previous data showed that STEM teachers in mainly primary and secondary educational levels had lower levels of anxiety and other emotional issues related to their ICTs' knowledge, skills, and training [36, 38, 39] . These studies also focused on mixed approaches, including surveys and interviews, to further understand the relationship between these workers' perception and their mental health, who seemed to be more prepared against the stress related to ICTs [37] [38] [39] . Nevertheless, these teachers' skills, knowledge, or ability might not have had to be at the highest level when the pandemic struck the world [29, 30, 41] . The pandemic of COVID-19 obliged to switch the educational process from face-to-face to online, putting further pressure and stressor on the teachers at any educational level [17, 38] . Several studies have analyzed psychological impact, such as stress or anxiety, among teachers from high schools, with relevance to Latin America or Spain [17, 54, 55] . Little studies have focused on the in-service teachers at the university level and their psychological status [18, 56] . Other researchers have analyzed the university teachers' skills or resources available at the beginning of the pandemic [57, 58] , including two intercontinental researches focusing on Ecuador and Spain whose primordial population were students [59, 60] . In these two articles [59, 60] carried out by the same authors, the results indicated how students and university teachers indicated that they had faced difficulties, being extremely significant to the lack of training or the lack of teachers' digital skills. This comparison between the two countries is achievable since the educational systems, laws, and structure are similar [61] , showing that the deficiency of resources or training for the switch to online education seemed to be present at the same percentage across countries [57, 58] . Another aspect that has been highlighted in previous studies is the importance of reviewing the scientific content, especially in education during the pandemic [62] . This fact was already pointed out by Milner-Bolotin [63] , who emphasized the relevance of review and accessible evidence regarding STEM teaching and the education or training that these employees had. Nonetheless, no study has reviewed neither analyzed any intercontinental differences, mainly Ecuador and Spain, regarding the emotional impact of in-service university teachers with high experience in STEM education (e.g., university teachers in engineering courses), regarding their perception about skills or absence of resources and being framed in the STEM education. In accordance with the absence of information and the significance of reviewing scientific content [63] , the current research has implemented a mixed-method combining the study and recruiting of new data and bibliographic analysis of current research focusing on this topic in order to obtain a better comprehension of the results. Therefore, and based on the lack of evidence regarding the teachers' psychological impact of COVID-19 at the university level, the role of STEM and their training and no current analysis of previous studies, the current research had as objectives: O1. To analyze the level of anxiety and depression among teachers with high expertise in STEM education, and therefore, high level of ICTs' skills and engineering understanding. O2. To determine the associations between the level of anxiety and depression, and their risk related to the difficulties that this population may perceive, such as the deficiency of training, resources or the stress caused by COVID-19. O3. To review the current knowledge available about the ICTs, university, and their mental health and comprehend the results obtained in the observational study. O4. To examine the importance of STEM education and its training as a protective factor to mental problems among university teachers. A transverse descriptive investigation was implemented through a target population of teachers with a median of twenty years of working experience in different institutions from Spain and Ecuador (Table 1 ). The selection of the sample and institutions was founded on the fact that teachers with a STEM background seemed to have more ICTs' skills, and therefore, have lower risk of mental health issues related to the technostress [64, 65] . The survey was distributed from September to October 2020 after the first lockdown in both countries. This non-probabilistic sampling had, as selection criteria, the accessibility and the relevance according to international ranking since the positions of institutions in this ranking would imply higher levels of ICTs' skills and accessibility of resources. These criteria delimited to universities from Spain and Ecuador with an interest for Engineering courses, being selected two universities with higher rankings in STEM education (Madrid and Valencia), two in the middle and two in the lower section of the ranking (Cordoba and Ecuador) [66] . The invitation to participate was sent to different teachers from STEM faculties, mainly Engineering, since the level of ICTs' skills and engineering knowledge should be the highest, and after receiving confirmation and willingness to participate, the survey link was sent. Additionally, these participants were asked to distribute the survey among other colleges working in the same school, providing a randomness to the sample. The sample of teachers that accessed the survey was 173, however, only 90 teachers started the survey ( Table 1) . Out of these 90 teachers, two indicated "No" to give their consent to participate and 20 only filled two questions. The remaining 70 teachers completed at least a minimum of 5 questions, with the questionnaire completely filled by 55 teachers, most of them from Ecuador. The teachers who accepted to contribute were from the top, medium, and bottom of the ranking; three were from Spain and one from Ecuador. This study was completed through an original and specific online survey, which included informed consent and the study's objective. The survey included 11 items focused on ICTs and education (ICTs' skills, frequency of using ICTs, opinion regarding lack of ICTs), the impact of COVID-19 on their level of stress ("Do you consider that COVID-19 has increased your stress level?"), and the anxiety and risk of depression scales. The online survey was distributed via QuestionPro (Survey Analytics LLC, San Francisco, CA, USA). The first page of the survey was formed by the information about the research, its purpose, ethical code, confidentiality, and anonymization. After the agreement to partake in the research, the participants were redirected to the items of the questionnaire; it was possible to unmark or indicate 'unknown' for each question. The survey included teachers' opinions about ICTs (frequency of using ICTs, relevance of the ICTs and role in education, ICTs' skills, and obstacles of using ICTs), whose validity, reliability, and consistency was acceptable [69] ; the next section included the perception of COVID-19 in stress levels (with a response of Yes, No or Maybe), a Linkert scale (from 1 = No stress to 5 = Maximum stress) of stress caused by the imposition of ICTs, the technical issues suffered and balance between family and work, and finally, the Rosenberg subscales of risk to have anxiety and depression, with nine items each and dichotomic responses [70] . This research followed the Declaration of Helsinki and the Data Protection Law 3/2018 following these codes and receiving approval For this study, the qualitative variables, such as country or teachers' perspectives regarding ICTs, were calculated via frequencies (absolute and relative) and the median. For the quantitative variables, which were the exact level obtained in Rosenberg subscales of risk to have anxiety and depression [70] , the mean, standard deviation, and 95% confidence intervals (CI) were used. Additionally, the breakpoint established for the subscales were used as a positive diagnostic of risk for anxiety and depression (four or more for the anxiety and two or more for anxiety) were analyzed to determine associations [70] . The Kolmogorov-Smirnov test was applied and indicated that the data did not follow the normality (p < 0.001). The chi-square test, Mann-Whitney U test, Kruskal-Wallis, and Spearman's correlation tests were used. Three simultaneous searches using the Scopus, Web of Science (WOS), and Medline via PubMed databases were performed using the PICO (Population, Intervention, Comparison, and Outcomes) structure. The bibliographic search was carried out using the Medical Subject Heading (MeSH) terms, which were previously selected in accordance with the research's purposes (O3 and O4) ( Table 2 ). Additionally, the keyword "teachers" or "academic" were incorporated in the search to gather more publications. Mental Health "Emotional, psychological, and social well-being of an individual or group" Mental Disorders "Psychiatric illness or diseases manifested by breakdowns in the adaptational process expressed primarily as abnormalities of thought, feeling, and behavior producing either distress or impairment of function" Anxiety Disorders "Persistent and disabling anxiety" Mood Disorders "Those disorders that have a disturbance in mood as their predominant feature" Depressive Disorder "An affective disorder manifested by either a dysphoric mood or loss of interest or pleasure in usual activities. The mood disturbance is prominent and relatively persistent" Anxiety "Feelings or emotions of dread, apprehension, and impending disaster but not disabling as with anxiety disorders" Stress Disorders, Traumatic "Anxiety disorders manifested by the development of characteristic symptoms following a psychologically traumatic event that is outside the normal range of usual human experience. Symptoms include re-experiencing the traumatic event, increased arousal, and numbing of responsiveness to or reduced involvement with the external world. Traumatic stress disorders can be further classified by the time of onset and the duration of these symptoms" Universities "Educational institutions providing facilities for teaching and research and authorized to grant academic degrees" Schools "Educational institutions" Teaching "A formal and organized process of transmitting knowledge to a person or group" Faculty "Teaching and administrative staff having academic rank in a post-secondary educational institution" Technology "The application of scientific knowledge to practical purposes in any field. It includes methods, techniques, and instrumentation" Educational Technology "Systematic identification, development, organization, or utilization of educational resources and the management of these processes. It is occasionally used also in a more limited sense to describe the use of equipment-oriented techniques or audiovisual aids in educational settings" Computer User Training "Process of teaching a person to interact and communicate with a computer" "Theoretical models which propose methods of learning or teaching as a basis or adjunct to changes in attitude or behavior. These educational interventions are usually applied in the fields of health and patient education but are not restricted to patient care" The bibliometric analysis was implemented in July 2021 using the research accordingly to each database. The Boolean operators chosen were "OR" and "AND", and the fields used to identify the relevance were "title", "abstract", and "keywords". For Scopus, the research included the use of abstract, title, and keywords ((TITLE-ABS-KEY ("mental health") OR TITLE-ABS-KEY ("mental disorders") OR TITLE-ABS-KEY ("Anxiety Disorders") OR TITLE-ABS-KEY ("Mood Disorders") OR TITLE-ABS-KEY ("Depressive Disorder") OR TITLE-ABS-KEY ("Anxiety") OR TITLE-ABS-KEY ("stress disorders, traumatic")) AND (TITLE-ABS-KEY ("universities") OR TITLE-ABS-KEY ("schools") OR TITLE-ABS-KEY ("teaching")) AND (TITLE-ABS-KEY ("faculty") OR TITLE-ABS-KEY ("teachers") OR 7 of 23 TITLE-ABS-KEY ("academics")) AND (TITLE-ABS-KEY ("technology") OR TITLE-ABS-KEY ("Educational Technology") OR TITLE-ABS-KEY ("Computer User Training") OR TITLE-ABS-KEY ("Models, Educational") OR TITLE-ABS-KEY ("ICTs")). In the case of WOS, the research was implemented focusing on the topic as the descriptor obtaining the following research strategy: TS = (("mental health" OR "mental disorders" OR "Anxiety Disorders" OR "Mood Disorders" OR "Depressive Disorder" OR "Anxiety" OR "stress disorders, traumatic") AND ("universities" OR "schools" OR "teaching") AND ("faculty" OR "teachers" OR "academics") AND ("technology" OR "Educational Technology" OR "Computer User Training" OR "Models, Educational" OR "ICTs")). Finally, the Medline research was based on the use of the MeSH terms and other terms obtaining the following strategy: ( The exclusion criteria were publications whose population were students, teachers from secondary or primary level, studies over 20 years, studies that did not include the technology, or whose topic concentrates on patients. Selected studies were those related to teachers from the university level's use of ICTs and the influence of mental health. Before 1 July 2021, there were identified 2776 documents published, that addressed the topic of teachers, mental health, and technology. During the screening, 2541 studies were eliminated from the study since in their titles, abstracts, and/or keywords referred to different populations (such as teachers from high schools), intervention (that did not include any type of technology) or outcomes (related to the mental health or emotional distress). The following stage, further articles were excluded according to content of the text and timeframe, articles whose population were teachers of secondary level or the lack of inclusion of ICTs. In this phase, 59 documents were eliminated, also excluding 15 papers that were duplicated ( Figure 1 ). The statistic package called SPSS version 24 (IBM Corporation, Armonk, NY, USA), VOSviewer version 1.6.15 (Ness Jan van Eck, The Netherlands), and Excel version 17 (Microsoft Corporation, Redmond, Washington, USA) were implemented to study the data obtained after the screening of the search ( Figure 1 ). Additionally, with the information provided by the databases, the Clarivate Journal Citation Report was used to define which journal was indexed, the Journal Impact Factor of the year of publication, the quartile of the journal, and the JIF percentile. Relative frequencies and medium were implemented for qualitative variables, i.e., country, journal, or year of publication. Based on the normalization test (p < 0.001), Mann-Whitney, chi-square U, Kruskal-Wallis tests, and Spearmen's correlation were applied for the quantitative variables of the 161 documents (Appendix A Table A1 ), Additionally, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Checklist [71] was applied for the analysis of the quality of the methodology of the top five observational studies in the area (Appendix A Table A2 ). Int. J. Environ. Res. Public Health 2021, 18, x 8 of 23 Figure 1 . Flow diagram followed for collecting data using the stages of identification, screening, eligibility, and inclusion. The statistic package called SPSS version 24 (IBM Corporation, Armonk, NY, USA), VOSviewer version 1.6.15 (Ness Jan van Eck, The Netherlands), and Excel version 17 (Microsoft Corporation, Redmond, Washington, USA) were implemented to study the data obtained after the screening of the search ( Figure 1 ). Additionally, with the information provided by the databases, the Clarivate Journal Citation Report was used to define which journal was indexed, the Journal Impact Factor of the year of publication, the quartile of the journal, and the JIF percentile. Relative frequencies and medium were implemented for qualitative variables, i.e., country, journal, or year of publication. Based on the normalization test (p < 0.001), Mann-Whitney, chi-square U, Kruskal-Wallis tests, and Spearmen's correlation were applied for the quantitative variables of the 161 documents (Appendix Table A1 ), Additionally, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Checklist [71] was applied for the analysis of the quality of the methodology of the top five observational studies in the area (Appendix Table A2 ). The observational study's initial analysis showed that 32.7% of the participants came from Spanish high education institutions, while 67.3% worked in Ecuador; 76.4% of the university teachers taught undergraduate students, with a median of 10 years of experience (34.5% had less than ten years, and 27.3% had between 10 to 20 years of experience). The mean level of anxiety was set at 6.84 ± 2.54 (95% CI 6.15-7.24); meanwhile, the mean depression level was 4.91 ± 2.89 with a 95% IC 4.13-5.69. The analysis of the anxiety and depression levels indicated that for the country, working experience, and teaching at different levels (Table 3) showed no significant differences except for the case of the depression level, which varied per country (p = 0.009). The mean of the depression level was 5.5 ± 3.24 (95% CI 3.88-5.53) among Spanish teachers in contrast with the 4.6 ± 2.71 (95% CI 3.71-7.11) of Ecuadorian university teachers. Nevertheless, the correlations for the level of anxiety and depression were not linked to any of the previous variables (p > 0.05). The anxiety and depression levels of the teachers were linked to the perception regarding the ICTs, the availability of resources, lacking resources as an obstacle, and COVID-19 (p < 0.05) ( Table 2) . Additionally, COVID-19 as a stressor (presented in 78.18%) showed significance regarding the level of anxiety (p = 0.022) and depression (p = 0.006). Another factor that showed relevance for the levels of anxiety and depression was the balance between family and work (p = 0.01). The correlations ( Table 2) Based on the levels of anxiety and depression, the risk of developing each was analyzed. The risk of developing anxiety among the teachers was 85.5%, being presented as similar frequency among Spanish (83.3%) and Ecuadorian (86.5%) (p > 0.05). The risk of developing anxiety showed significant differences in the role of ICTs in education and lack of ICTs (resources, software, training, models, time, and research about the benefits of ICTs) (p < 0.05) ( Table 4 ). Additionally, COVID-19 as a stressor showed a significant difference between the teachers with a risk of anxiety (82.98%) (X 2 = 8.03; p = 0.005). Mean- Based on the levels of anxiety and depression, the risk of developing each was analyzed. The risk of developing anxiety among the teachers was 85.5%, being presented as similar frequency among Spanish (83.3%) and Ecuadorian (86.5%) (p > 0.05). The risk of developing anxiety showed significant differences in the role of ICTs in education and lack of ICTs (resources, software, training, models, time, and research about the benefits of ICTs) (p < 0.05) (Table 4) . Additionally, COVID-19 as a stressor showed a significant difference between the teachers with a risk of anxiety (82.98%) (X 2 = 8.03; p = 0.005). Meanwhile, the risk of depression was related to COVID-19 as a stressor (X 2 = 4.39; p = 0.037), the balance between family and work (X 2 = 16.3; p = 0.006), availability of computers and internet, the role of the ICTs in education, lack of models, and time (p < 0.05) ( Table 3) . Furthermore, the correlations were analyzed for each variable and the risk of anxiety and depression ( Table 4 ). The anxiety risk was linked to the frequency of using virtual environments (ρ = 0.30; p = 0.026), the lack of resources (p = 0.032), training (p < 0.001), models (p < 0.001), time (p = 0.001), and research about the benefits of using ICTs in the education (p < 0.001) ( Table 4 ). Additionally, COVID-19 (p = 0.037), need to use ICTs (p = 0.005), technical issues (p = 0.005), and the balance between family and work (p = 0.006) were linked to having a higher risk of anxiety (Table 5) . Besides, risk of having depression (Table 4 ) was linked to the unavailability of internet (ρ = −0.67; p = 0.049), less ICTs' skills (ρ = −0.34; p = 0.011), the lack of resources (p = 0.038), training (p = 0.025), and models (p = 0.006). This risk was associated with COVID-19 as a stressor (p = 0.004), need to use ICTs (p = 0.001), technical issues (p = 0.007), and the balance between family and work (p = 0.001) were linked to having a higher risk of anxiety. Finally, the risk of anxiety and depression was connected (ρ = 0.59; p < 0.001). Finally, a multivariant analysis based on linear regression for positive risk of anxiety among STEM teachers at university level indicated (R 2 = 0.62; p = 0.016) that this risk is dependent on lack of time (p < 0.001), software (p = 0.016), and research that clarified the benefits of using ICTs (p = 0.005); this risk is codependent of the chance to developing depression (p < 0.001). The analysis of the bibliometric method reflected that most investigations were carried out in the United States (USA, with 39 documents), followed by China (with 15 papers), Turkey (with 14 documents), the United Kingdom (10 articles), and other countries (Spain and Ecuador with less than five documents each) with few publications on the topic studied ( Figure 3 ). This figure presents the nations from whom the authors have published the documents (N = 161), being more often the case of countries, such as Japan or Chile, that provided only one paper. The number of publications per year and citations showed significant differences between countries (p = 0.01), with the USA being the most influential producer of publications per year (p = 0.003), indexing most of the publications (p = 0.011) in higher quartiles (p = 0.043), and higher JCR (p = 0.048). Despite this, the frequency of publications in the latest years indicated that countries with fewer investigations have increased during the last three years (ρ = −0.87; p < 0.001). Nevertheless, when analyzed per continent and the year, the number of citations, journal citation report (JCR), and percentile of the journal, there were no significant differences (p = 0.71). There were substantial differences between continents regarding whether the journal was indexed (X 2 = 12.54; p = 0.014) and the quartile (X 2 = 32.76; p = 0.036). The correlations indicated how other continents such as Oceania had increased the rate of publications mainly in the last years (p = 0.033) but had more minor citations (p < 0.001), indexed journals (p = 0.005), and published articles in lower quartiles (p = 0.002) than compared to other continents, such as Europe. Another aspect analyzed was the frequency of publications per year (Figure 4) , whose highest rate was achieved in 2020, the year of the COVID-19 pandemic (29.6% of the publications made in the last 20 years). The analysis indicated that there was a significant difference between the year of publication and quartile of the article (p < 0.001) and the number of citations (p < 0.001). The correlations indicated that the latest publications had more minor citations (ρ = −0.47; p < 0.001), although the rate of publications has no significant differences (p = 0. journal citation report (JCR), and percentile of the journal, there were no significant differences (p = 0.71). There were substantial differences between continents regarding whether the journal was indexed (X 2 = 12.54; p = 0.014) and the quartile (X 2 = 32.76; p = 0.036). The correlations indicated how other continents such as Oceania had increased the rate of publications mainly in the last years (p = 0.033) but had more minor citations (p < 0.001), indexed journals (p = 0.005), and published articles in lower quartiles (p = 0.002) than compared to other continents, such as Europe. Another aspect analyzed was the frequency of publications per year (Figure 4) , whose highest rate was achieved in 2020, the year of the COVID-19 pandemic (29.6% of The number of citations of each article diverged according to the theme (p < 0.001), with a mean of 8.2 citations (SD = 15.47; IC 95% = 10.63-5.77). The citations per document were linked to journal indexed, JCR of the journal of the year of publication, quartile, and percentile (p < 0.001). According to the citations, the top ten articles were reviews, two observational studies, two guidelines or theoretical studies, and two qualitative studies. The results indicated that this area is more common than the reviews. Based on the citations and the type of studies, the top five observational or crosssectional studies with a higher number of citations were analyzed ( Table 6 ). The most cited article, with 138 citations and from Turkey, focused on determining the relationship between attitude to computers and anxiety for pre-teachers at the university level [72] . This article highlights the importance of computer self-efficacy and anxiety related to using such technology as predictors in the education that this teacher will provide. For teachers and education, this article clarifies how the attitude, including the technostress and ICTs' According to the citations, the top ten articles were reviews, two observational studies, two guidelines or theoretical studies, and two qualitative studies. The results indicated that this area is more common than the reviews. Based on the citations and the type of studies, the top five observational or crosssectional studies with a higher number of citations were analyzed ( Table 6 ). The most cited article, with 138 citations and from Turkey, focused on determining the relationship between attitude to computers and anxiety for pre-teachers at the university level [72] . This article highlights the importance of computer self-efficacy and anxiety related to using such technology as predictors in the education that this teacher will provide. For teachers and education, this article clarifies how the attitude, including the technostress and ICTs' skills, are intrinsic factors related to the mental distress that university teachers may experience. The second article focused on nursing teachers and how ICTs are a risk factor for technostress [73] . The third focused on university teachers, in which it was linked the use of ICTs for the different functions and ICTs' skills, and the anxiety related to ICTs [74] . The fourth and fifth focused on how the use of ICTs in the future seems to be determined by anxiety caused by self-efficacy, behavior, or willingness [75, 76] . This table displayed how more relevant observational research were published in Turkey (3/5), which is also the third country with a higher number of articles which focused on the topic, analyzed through the bibliometric analysis. Additionally, it is reflected in the top articles the existing link between an individual's skills and perception regarding ICTs and the risk of technostress; furthermore, the fourth most cited article showed how teachers whose work is related to STEM education have lower levels of anxiety or stress related to ICTs. Besides, the quality of the methodology was analyzed ( Table 5 ), indicating that despite having more minor citations, the second most cited article [73] had over 70% in the quality of the methodology, followed by the first article with a higher number of citations [72] . The third article with the most methodological quality was published in 2016 and had the third-ranking number of citations [74] . The remaining papers with fewer citations, and more recently, had less than 50% of methodological quality. These results indicate that the number of citations and methodological quality are critical factors in the relevance of the research published. The analysis of index keywords of each document, based on a minimum of two nodes, showed seven clusters as the most concurrency topics ( Figure 5 ). The issues identified seven groups (Table 7 ) (formed by 224 keywords indicated with 4819 links and a total of 7350 ties). The first and foremost cluster (representing 30.49% and being in red) focused on educational technology, STEM education with great emphasis on students, and teachers' self-efficacy (presenting 17.56% of co-occurrence among the keywords). This red cluster represented one of the main sub-topics based on the technology, teaching, and students focusing on STEM. The next sub-topic (green), conformed by 44 keywords (19.73%), has a central theme regarding the educational model in nursing education and the relevance of the management and organization for the psychological impact. In higher educational institutions, the third cluster (in blue), which represented 16.59% of keywords, focused on psychological issues, especially anxiety. The fourth cluster, in yellow, with the exact representation as to the third, represented the sub-topic of mental health in the health faculty, specifically the medical education, focusing on the distance education caused by the coronavirus and the psychological adaptation. The following cluster (in purple), with 20 keywords and an occurrence of 8.97%, concentrated on mental health in the health faculty and its relationship with practical education in hospitals. The sixth cluster, formed by 8.52% of the keywords and represented in pink, focused on mental stress and technology. The last cluster (orange) was created by seven keywords and presented 3.14% of weight from the seven clusters and was centered on sciences, self-concept, and the psychology in the educational sector. Influential factors on pre-service teachers' intentions to use ICT in future lessons [75] 2016 Turkey Pre-teachers at different educational levels and university teachers (N = 2904) Preservice Teachers ICT Acceptance Scale was used and included: perceived usefulness, ease-of-use, and efficacy, social influence, facilitating conditions, and computer anxiety There was an inverse correlation between anxiety and ICT integration. There was also a negative relationship between anxiety and the teachers from scientific departments or STEM backgrounds. with the exact representation as to the third, represented the sub-topic of mental health in the health faculty, specifically the medical education, focusing on the distance education caused by the coronavirus and the psychological adaptation. The following cluster (in purple), with 20 keywords and an occurrence of 8.97%, concentrated on mental health in the health faculty and its relationship with practical education in hospitals. The sixth cluster, formed by 8.52% of the keywords and represented in pink, focused on mental stress and technology. The last cluster (orange) was created by seven keywords and presented 3.14% of weight from the seven clusters and was centered on sciences, self-concept, and the psychology in the educational sector. Psychology, education-science-self-concept-job satisfaction Sciences, self-concept, and psychology in the educational sector The mixed-method analysis has identified several important aspects. First, from the descriptive analysis, it can be resumed in the link between anxiety levels, depression, and various impediments for online education, such as the lack of training, and the relevance of other stressors like COVID-19 or the balance between family and working, these being results independent from the country. Meanwhile, the bibliometric analysis had highlighted the lack of studies that focused on measuring the STEM teachers at the university level of mental discomfort or problems, and instead identified the STEM training that these workers have as a protective factor. Few studies have been carried out in Spain or Ecuador focusing on this relevant topic, and much less were intercontinental studies identified. The data from the observational analysis has highlighted how STEM teachers at the university level had a high level of anxiety and depression, with the risk of developing them as a disorder elevated. The mental well-being of Spanish and Ecuadorian teachers seemed to be related to working conditions and workers' attitudes. These workers indicated that the impact of COVID-19, the imposition of using ICTs for online teaching, the technical problems, and the balance between the familiar and working environments were critical features for their stress. There were no significance differences according to country or working experience for the anxiety and depression level; rather, these levels were associated with the perception of the STEM teachers at higher education institutions concerning diverse difficulties, such as lack of resources, training, or time. Moreover, the ICTs' skills, described as moderatehigh, were not linked to anxiety or depression. This result indicated that previous skills to the pandemic seemed to have not been a protective factor against mental issues. The associations previously described may reflect how the transfer from face-to-face to online teaching was a stressor independently of the previous training in STEM education or previous ICTs' skills. These outcomes are in accordance with other researchers [77, 78] , whose findings showed the need for training as a vital element to prevent technostress among teachers. Furthermore, another study carried out in Ecuador specified that previous training and experience with online teaching before the pandemic was a protective factor, reducing the stress level and possibly other psychological symptoms [55] . Nevertheless, the previous studies in higher education institutions did not differentiate between STEM teachers or where they teach [55, 77, 78] . Another interesting finding was the impact balancing family and the working environment in the mental well-being of the participants. These results were consistent with previous studies that indicated how the inclusion of the ICTs in the living environment increases the hostility and stress between the workers and the rest of the family members [77] . However, because of the COVID-19 restrictions, the anxiety levels of these workers continue for a long time because of the obligation to balance workload combined with the work of carrying out family care duties [79] . This imposition to balance the work demands and family duties could be one of the reasons why teachers working in the home environment indicated having higher levels of emotional issues and psychological symptoms [80] . Several studies indicated diverse psychological symptoms, mainly anxiety and stress, among teachers at different educational levels and at the university [81] . In this sense, a Chinese study indicated that posttraumatic stress disorder was around 25% among college teachers, reflecting that the incidence of mental health issues is higher in this group [23] . The current observational analysis indicated that 85% of the university teachers were at risk of developing anxiety as chronic mental health, which was also connected to depression [82, 83] . The relationship between anxiety and depression, established long ago [84] , was also associated with lack of time, specific software, and research that clarified the benefits of using technology. This relationship between anxiety, depression, and these obstacles could be different and be more conditional depending on the previous background or new stressors such as the impact of the pandemic. However, other studies indicated that STEM teachers had less risk of anxiety related to ICTs [72] , being contradictory to the current studies since the STEM teachers also had high levels of anxiety. Other articles that analyzed the relationship between technostress and COVID-19 suggested that age or gender play a role in the anxiety linked to the technology [32, 67] , although these studies used only technostress and the possible effect of the family/work-the stress caused by COVID-19 or the lack of training was missing. This lack of association from previous studies has been magnified by the observational studies found in the bibliometric analysis [72] [73] [74] [75] [76] . The top five articles (Table 5 ) indicated that pre-service and in-service STEM teachers in higher education institutions had fewer mental issues and less risk. These articles [72] [73] [74] [75] [76] were in contract with the findings from the current observational analysis, whose findings indicated that STEM education or experience in STEM courses was not a definitive shielding factor against mental issues. In fact, the STEM teachers also suffered from psychological problems during the pandemic, which could be associated with the lack of previous experience with online learning [53] . Another aspect identified in the bibliographic analysis was the lack of studies carried out in the population selected (Spain or Ecuador), and the papers did not consider the lack of technical support as a stressor [72] [73] [74] [75] [76] . Additionally, another result was that the current investigations are more commonly reviews [85, 86] , being also more cited. The top publications focused on recommendations [87] to prevent mental issues, qualitative approaches, and little provided current data; being less common, the perspective of STEM teachers at university level. Other prior works indicated that the highest prevalence of work-related technostress among teachers (around 10%) was related to a specific age group (women from 30 to 39 years old with children), which could be related to training, working workload, and family chores, and the field in which the education is provided [73, 87] . Thus, most works focused on analyzing prevalence and its relationship with the teachers' attitudes but did not include the teachers' perspective regarding obstacles such as the lack of ICTs [72] [73] [74] [75] [76] . Additionally, these articles (Table 5) focused on Turkey and pre-service faculty members, not including the STEM teachers as an independent group but studying this population in combination with other educators [72] [73] [74] [75] [76] . Additionally, one study focused on healthcare teachers [73] showed that lectures from health sciences despite being close to STEM education, presents high levels of stress and anxiety linked to ICTs. These differences among teachers from higher education institutions [72] [73] [74] [75] [76] also amplifies the fact, highlighted in the observational analysis, that training with ICTs is a major factor to protect these workers [88] . In this sense, UNESCO described the relevant preventative factors and how to decrease such prevalence, and highlighted that workers need training, updating the ICTs available, and supporting the continuity of adequate education [88] . Moreover, the bibliometric analysis showed that the quality of the observational studies is under 70% despite being indexed, underlining the need to further descriptive studies with higher methodological quality. Another aspect identified in the bibliometric analysis was the fact that most of observational studies focused on STEM teachers at higher institutions previous to the pandemic, and therefore, further analysis during the current pandemic are needed. In this sense, the current observational analysis might provide more relevant data, although such information and its application in the education field (such as increasing the training with ICTs and its later analysis via experimental study) should be made with precautions based on the reduced number of participants. Despite the findings and application to the field, the current research also has limitations. The major limitation of this study is its small sample of teachers, although no previous research has focused only on STEM teachers during COVID-19 at the university level. This limitation is associated with the population and the rate of response from participants, being under 10%. Moreover, another limitation from the methodology is the use of a survey based on perception and the cross-sectional design that limits the results to a timeframe. Besides, the data from university teachers has limited the results to the countries that participated, so the extrapolation of the data needs to be carefully carried out. Additionally, some personal data of the sample were not included, such as gender, number of children or their age, which could provide further information. Additionally, since the research method was based on a mixed approach, another weakness of the methodology could be the choice of MeSH or other keywords not being included, which might have enclosed the number of publications and therefore, the possibility of framing the survey results into the current knowledge. Nonetheless, the current research based on a mixed-method has raised interesting questions about mental health and its relationship with the ICTs, taken in educational higher institutions environment during the pandemic, and e-learning education. Despite these limitations and based on the topic, timeframe, population, and sample size, including a few publications, the findings presented with the mixed-method could provide further knowledge in this area. The results, mainly through the bibliographic analysis, highlight that previous studies were not focusing on STEM teachers and there was a lack of data in this specific population, as well as stressors that also affect these workers. A vital practical repercussion is the essential nature of training and adequate technical support as well as giving mental support to the workers in the educational sector. Among university teachers, the anxiety and depression levels and their risk of developing them as a mental disorder from a sample of STEM Spanish and Ecuadorian teachers at university were highly elevated and linked to their perspective. One of the most relevant findings is that anxiety and depression can be linked to various obstacles, such as lack of training, resources, time, or research. The results showed that STEM teachers at university level perceived COVID-19, the imposition of using ICTs, the technological issues, the balance between family and work, and their perception of the lack of means as obstacles towards the mental issues among these workers. These results are fascinating since the teachers, independently of the country, indicated that their mental health and stress were high, caused by different factors related to technology. Other factors were also noteworthy, such as the impact of COVID-19, since these data indicated their relevance on mental health, although these factors were not conclusive in the risk of anxiety and depression. The second significant finding was that the described variables (e.g., training, or behavioral attitudes regarding ICTs) in most significant previous works were connected to a higher risk of stress and anxiety associated with technology. Another important finding that emerged from the bibliometric analysis was that most results focused on countries such as Turkey or the USA, with a lack of research carried out in Spain or Ecuador. Moreover, there was also a lack of observational analysis whose focus was determining factors linked to mental distress and risk factors. Most of the documents were reviews that analyzed the impact of ICTs and stress, and highlighted how STEM teachers have lower risk of mental issues, which could be not applicable in the current pandemic. These results of the previous observational studies, independently of the year or number of citations, in contrast with the observational data from the current research, did not analyze the effect of STEM education as a preventive factor related to the training of teachers. Finally, all these findings suggest that to decrease the prevalence of mental issues, greater compliance with the relevant measures, mainly training with ICTs, is needed, along with further research that focuses on whether such actions are implemented. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Data Availability Statement: The data are available; please contact the authors. We would like to thank the participants and institutions that helped us to gather the data. The authors declare no conflict of interest. Appendix A Table A1 . Correlation and p-values between the variables of the bibliometric analysis. Year Foresight on New and Emerging Occupational Safety and Health Risks Associated with Digitalisation by 2025; European Risk Observatory Report; European Agency for Safety and Health at Work: Luxembourg Priorities for Occupational Safety and Health Research in Europe European Agency for Safety and Health at Work; Publications Office of the European Union: Luxembourg European Agency for Safety and Health at Work. Psychosocial Risks and Stress at Work Protecting and promoting mental health in the workplace: Developing a systems approach to job stress. Health Promot Effects of a work-related stress model based mental health promotion program on job stress, stress reactions and coping profiles of women workers: A control groups study Workplace risk management practices to prevent musculoskeletal and mental health disorders: What are the gaps? The Future of research on work, safety, health and wellbeing: A guiding conceptual framework Canadian Centre for Occupational Health and Safety Mental Health-How to Address and Support Workplace mental health: Developing an integrated intervention approach Stress and the HPA Axis: Role of glucocorticoids in alcohol dependence Psychological stress reactivity and future health and disease outcomes: A systematic review of prospective evidence Intergenerational transmission of stress in humans Mechanisms by which stress affects the experimental and clinical inflammatory bowel disease (IBD): Role of brain-gut axis Stress and the HPA Axis: Balancing homeostasis and fertility Benefits and stressors-Perceived effects of ICT use on employee health and work stress: An exploratory study from Austria and Hong Kong The effect of yoga on stress, anxiety, and depression in women Ozamiz-Etxebarria, N. Teacher stress, anxiety and depression at the beginning of the academic year during the COVID-19 pandemic Gender differences in stress-and burnoutrelated factors of university professors The social impact of COVID-19 as perceived by the employees of a UK mental health service COVID-19-related mental health effects in the workplace: A narrative review Public perceptions and experiences of social distancing and social isolation during the COVID-19 pandemic: A UK-based focus group study Mental health of health-care workers in the COVID-19 era Trauma exposure and the PTSD symptoms of college teachers during the peak of the COVID-19 outbreak Mental health problems faced by healthcare workers due to the COVID-19 pandemic-A review Respiratory protection considerations for healthcare workers during the COVID-19 pandemic Beyond the assistance: Additional exposure situations to COVID-19 for healthcare workers COVID-19: Implications for Employment and Working Life European Foundation for the Improvement of Living and Working Conditions; Publications Office of the European Union: Luxembourg Impact of COVID-19 pandemic on information management research and practice: Transforming education, work and life School's Out, But Class's On", the largest online education in the world today: Taking China's practical exploration during the COVID-19 epidemic prevention and control as an example European Higher Education in the COVID-19 Crisis Sense of coherence, occupational stressors, and mental health among Japanese high school teachers in Nagasaki prefecture: A multiple regression analysis Impact of the COVID-19 pandemic on teacher quality of life: A longitudinal study from before and during the health crisis Systematic review on the prevalence of illness and stress and their associated risk factors among educators in Malaysia Tecnoestrés en la sociedad de la tecnología y la comunicación: Revisión bibliográfica a partir de la web of science Social media-induced technostress: Its impact on the job performance of it professionals and the moderating role of job characteristics Do individual characteristics influence the types of technostress reported by workers? Motivation and continuance intention towards online instruction among teachers during the COVID-19 pandemic: The mediating effect of burnout and technostress The outbreak of the COVID-19 pandemic and its social impact on education: Were engineering teachers ready to teach online? Relationships between teachers' technostress, technological pedagogical content knowledge (TPACK), school support and demographic variables: A structural equation modeling La educación secundaria en España en medio de la crisis del COVID-19 The right to education and ICT during COVID-19: An international perspective Investigating preservice STEM teacher conceptions of STEM education The need for a STEM road map. In STEM Road Map: A Framework for Integrated STEM Education Conceptualizing integrated STEM education What is STEM? A discussion about conceptions of STEM in education and partnerships The Case of STEM Education: Challenges and Opportunities Revisiting the conceptualisation of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals Conceptualizing teacher professional learning Meta-analysis of inquiry-based learning: Effects of guidance Developing pre-service teachers conceptualization of STEM and STEM pedagogical practices. Front. Educ. 2021, 6, 142 Perceptions of in-service teachers toward teaching STEM in Thailand Advancing the state of the art of STEM integration Measuring science teachers' stress level triggered by multiple stressful conditions Prevalence of anxiety, depression, and stress among teachers during the COVID-19 pandemic: Systematic review Teachers' mental health and self-reported coping strategies during the COVID-19 pandemic in Ecuador: A mixed-methods study Subjective discomfort and personality traits of university teachers during the COVID-19 pandemic Learning and technology: Enablers for university education resilience. An experience case under COVID-19 in Spain Ibero-american higher education institutions facing COVID-19 Educación en tiempos de pandemia: Reflexiones de alumnos y profesores sobre La enseñanza virtual universitaria en España, Italia y Ecuador Digital literacy and higher education during COVID-19 lockdown Shall COVID-19 accelerate the transfer of passive learning to active education? Maskana 2020 Professional development and use of digital technologies by science teachers: A review of theoretical frameworks Evidence-based research in STEM teacher education: From theory to practice Virtual laboratories for education in science, technology, and engineering: A review The development of skills in the ICT sector: Analysis of engineering students' perceptions about transversal skills Technostress of chilean teachers in the context of the COVID-19 pandemic and teleworking The higher education sustainability before and during the COVID-19 pandemic: A spanish and ecuadorian case Psicometría de la ansiedad, la depresión y el alcoholismo en Atención Primaria for the STROBE Initiative. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies Attitudes to technology, perceived computer self-efficacy and computer anxiety as predictors of computer supported education The incidence of technological stress among baccalaureate nurse educators using technology during course preparation and delivery A study on academic staff personality and technology acceptance: The case of communication and collaboration applications Influential factors on preservice teachers' intentions to use ICT in future lessons A model for preservice teachers' intentions to use ICT in future lessons Impact of educational technology on teacher stress and anxiety: A literature review The moderating roles of technological self-efficacy and time management in the technostress and employee performance relationship through burnout Remote-learning, time-use, and mental health of ecuadorian high-school students during the COVID-19 quarantine How bad is it? Suicidality in the middle of the COVID-19 pandemic The psychological state of teachers during the COVID-19 crisis: The challenge of returning to face-to-face teaching Teacher self-efficacy, instructional quality, and student motivational beliefs: An analysis using multilevel structural equation modeling. Learn Psychological stress and vocal symptoms among university professors in Israel: Implications of the shift to online synchronous teaching during the COVID-19 pandemic The relationship between anxiety and depression Vocational psychology: Agency, equity, and well-Being Resilience, apps and reluctant individualism: Technologies of self in the neoliberal academy. In Women's Studies International Forum Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic UNESCO Global Education Monitoring (GEM) Report. 2020. Available online [72] No