key: cord-0894046-x6rck5qj
authors: Ćosić, Krešimir; Popović, Siniša; Šarlija, Marko; Kesedžić, Ivan; Gambiraža, Mate; Dropuljić, Branimir; Mijić, Igor; Henigsberg, Neven; Jovanovic, Tanja
title: AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients
date: 2021-12-28
journal: Front Psychol
DOI: 10.3389/fpsyg.2021.782866
sha: ea00f1872fb123e32f39362b3c2d2dfd934277a9
doc_id: 894046
cord_uid: x6rck5qj

The COVID-19 pandemic has adverse consequences on human psychology and behavior long after initial recovery from the virus. These COVID-19 health sequelae, if undetected and left untreated, may lead to more enduring mental health problems, and put vulnerable individuals at risk of developing more serious psychopathologies. Therefore, an early distinction of such vulnerable individuals from those who are more resilient is important to undertake timely preventive interventions. The main aim of this article is to present a comprehensive multimodal conceptual approach for addressing these potential psychological and behavioral mental health changes using state-of-the-art tools and means of artificial intelligence (AI). Mental health COVID-19 recovery programs at post-COVID clinics based on AI prediction and prevention strategies may significantly improve the global mental health of ex-COVID-19 patients. Most COVID-19 recovery programs currently involve specialists such as pulmonologists, cardiologists, and neurologists, but there is a lack of psychiatrist care. The focus of this article is on new tools which can enhance the current limited psychiatrist resources and capabilities in coping with the upcoming challenges related to widespread mental health disorders. Patients affected by COVID-19 are more vulnerable to psychological and behavioral changes than non-COVID populations and therefore they deserve careful clinical psychological screening in post-COVID clinics. However, despite significant advances in research, the pace of progress in prevention of psychiatric disorders in these patients is still insufficient. Current approaches for the diagnosis of psychiatric disorders largely rely on clinical rating scales, as well as self-rating questionnaires that are inadequate for comprehensive assessment of ex-COVID-19 patients’ susceptibility to mental health deterioration. These limitations can presumably be overcome by applying state-of-the-art AI-based tools in diagnosis, prevention, and treatment of psychiatric disorders in acute phase of disease to prevent more chronic psychiatric consequences.

The COVID-19 pandemic has created enormous stressors across the globe by elevating rates of anxiety, depression, posttraumatic stress disorder (PTSD), or even suicidal behaviors (Bäuerle et al., 2020; Sherman et al., 2020; Wu T. et al., 2020; Cheung et al., 2021; Czeisler et al., 2021) . The inadequate number of psychiatrists, lack of predictive biomarkers (Walker et al., 2017) , and human subjectivity in assessments can further aggravate these problems, leading to a surge in the incidence and prevalence of mental health disorders across the globe. Therefore, urgent actions on extensive monitoring of mental health are needed, particularly concerning prediction and early treatment of individuals who were exposed to a high level of COVID-19-related distress, especially if they exhibit low overall stress resilience (Riehm et al., 2021) and other specific vulnerabilities Tuman, 2021) . Recognition and identification of such high-risk individuals in the early stages of acute stress is extremely important to prevent the development of more serious long-term mental health disorders. The proposed predictive methodology presented in this article is based on appropriate multimodal stimuli, corresponding multimodal neuro-psycho-physiological features and their AIbased integration and analysis. A new generation of AIbased tools emerging in psychiatry (Ćosić et al., 2020a,b) might facilitate prediction of risk of chronic psychopathology (Karstoft et al., 2015; Schultebraucks et al., 2020a) and detection of suicidal ideation (Just et al., 2017) , enable automated delivery of treatment (Fitzpatrick et al., 2017) , help understand crucial ingredients of psychotherapy (Ewbank et al., 2020) , as well as destigmatize reporting of mental health symptoms as virtual human interviewers (Lucas et al., 2017) . Therefore, whether in prediction, prevention, diagnosis or treatment, AIbased technologies exhibit transformational potential to shift a self-report, conversation-, and observation-based qualitative system in psychiatry toward a more data/information-rich and quantitative one, even if several challenges still exist along this path (Graham et al., 2019; Rainey and Erden, 2020) . A major strength of statistical and AI methods is their ability to identify specific non-obvious patterns beyond human computational capabilities within highly heterogeneous multimodal sets of data relevant for mental health assessment (Eyre et al., 2016; Garcia-Ceja et al., 2018) , which is essential for early detection of ex-COVID-19 patients at high risk of mental health deterioration. Diverse forms of data for inclusion into such heterogeneous data sets have been encountered in several publications on AI in mental health (Eyre et al., 2016; Garcia-Ceja et al., 2018; Graham et al., 2019) : various rating scales used by patients and clinicians, electronic health records (EHRs), brain imaging data, genomics, blood biomarkers, data collected while using smartphones, textual posts from social media platforms, speech and language audio data, facial video data, multiple peripheral physiological signals etc. Assuming that mentally vulnerable individuals affected by COVID-19 pandemic vs. resilient individuals have different profile of neuro-psycho-physiological indicators together with differences in personality traits and psycho-behavioral status, a comprehensive AI-based predictive approach should lead to better detection of high-risk ex-COVID-19 patients (Ramos-Lima et al., 2020) and more effective prevention strategies. The following sections present the concept of prediction of potential chronic mental health diseases among ex-COVID-19 patients, which includes comprehensive psychological, neurophysiological, semantic, acoustic, and facial/oculometric measurements and features, as well as the concept of mental health disorder prevention using AI methods.

The main objective of the proposed methodology for prediction of potential mental health disorders among ex-COVID-19 patients is to extend clinical interview and self-report information with more objective metrics. Therefore, this section provides an overview of a variety of psychological, speech-based, neurophysiological, as well as facial/oculometric features as potential predictors of mental health deterioration in the setting of post-COVID clinics. Such comprehensive data obtained through extended screening procedures of ex-COVID-19 patients in post-COVID clinics (Rubin, 2020; Walter, 2021) , analyzed with AI-based tools of unsupervised machine learning methods (such as multivariate correlation analyses, clustering, and principal component analysis), as well as a range of traditional supervised machine learning methods (such as artificial neural networks, support vector machines and contemporary deep learning supervised approaches) can significantly improve prediction accuracy. A variety of AI-based tools which are already being used in research on prediction of adverse mental health outcomes from diverse data sets, are covered in the subsection on "Multimodal data fusion: statistical and ML analysis."

Understanding that self-report psychological screening is also important, we will briefly discuss these instruments for prediction of chronic psychopathology. A wide variety of self-report questionnaires, many of which have already been employed in COVID-19-related mental health research, is potentially applicable within the proposed prediction methodology:

• Questionnaires regarding overall mental health and wellbeing, like the General Health Questionnaire (GHQ-12) (Gao et al., 2004; Pierce et al., 2020; Veer et al., 2021) and the Short Form Health Survey questionnaires (SF-36 and SF-12) (Ware and Sherbourne, 1992; Ware et al., 1996; Chen et al., 2020; Zhang S. X. et al., 2020) . • Questionnaires regarding stress resilience as an important protective factor against chronic psychopathology, e.g., the Connor-Davidson Resilience Scale (Connor and Davidson, 2003; Cai et al., 2020) and the Brief Resilience Scale (Smith et al., 2008; Riehm et al., 2021) , as well as questionnaires targeting other related protective or risk factors, like hardiness (Eschleman et al., 2010; Rezaei and Vakilian, 2021) , distress tolerance (Simons and Gaher, 2005; Liu N. et al., 2020) , appraisal style (Kalisch et al., 2015; Veer et al., 2021) , self-control (Maloney et al., 2012; Li et al., 2020) , trait anxiety (Speilberger and Vagg, 1984; Lin et al., 2020) , anxiety sensitivity (Deacon et al., 2003; Warren et al., 2021) , and type D personality (Denollet, 2005; Tuman, 2021) . • Questionnaires regarding preexisting vulnerability to psychopathology that stems from adverse experiences encountered in life prior to pandemic, such as Life Events Checklist (Gray et al., 2004; Marco et al., 2020) , Adverse Childhood Experiences Questionnaire (ACE-Q) (Felitti et al., 1998; Zarse et al., 2019; McCarthy et al., 2021) , and the Trauma History Questionnaire (Hooper et al., 2011; Kolacz et al., 2020) . • Questionnaires closely aligned with diagnostic criteria for specific mental health disorders, like PTSD Checklist for DSM-5 (PCL-5) (Blevins et al., 2015; Liu N. et al., 2020) , Impact of Event Scale-Revised (Weiss, 2004; Wang et al., 2020) , Beck Depression Inventory (Beck et al., 1961; Di Tella et al., 2020) , Generalized Anxiety Disorder scale (GAD-7) (Spitzer et al., 2006; Bäuerle et al., 2020) . These questionnaires are an essential part of any future predictive algorithm, since a very high score on any of them suggests an immediate risk that the disorder is already emerging or present. • Questionnaires regarding general distress experienced by ex-COVID-19 patients during the pandemic, like K10 Kessler Psychological Distress Scale (Andrews and Slade, 2001; Janiri et al., 2020) , Perceived Stress Scale (Roberti et al., 2006; Eubank et al., 2021) , and Peritraumatic Dissociative Experiences Questionnaire (Marmar et al., 1997; Ranieri et al., 2021) . • Questionnaires on COVID-19-specific distress, which are either adapted from existing illness-related questionnaires, like Brief Illness Perception Questionnaire (Broadbent et al., 2006; Hubbard et al., 2021) or newly developed, e.g., Fear of COVID-19 Scale (Ahorsu et al., 2020; Bitan et al., 2020) , Post-COVID-19 Functional Status Scale (Klok et al., 2020) , and COVID-19 Peritraumatic Distress Index (CPDI) (Qiu et al., 2020) .

Finally, self-reports regarding the presence of a previously diagnosed psychopathology (Asmundson et al., 2020) would be important in cases where mental health history is not otherwise available, since it is known that depression and anxiety occur more commonly in patients with previous psychiatric diagnosis (Hao et al., 2020) . In selection of questionnaires in line with this brief review, it is important to minimize their overlap, while favoring all of those with strong scientific evidence as risk assessment instruments, to acquire efficiently the most valuable psychological features for prediction of future psychopathology.

The potential of human speech patterns in detection and prediction of psychopathology has already been demonstrated in several research works (Cummins et al., 2015; Marmar et al., 2019; Schultebraucks et al., 2020b) . Therefore, this section discusses speech features that are the most promising for application in risk assessment of ex-COVID-19 patients. A wide range of methods to elicit speech features for assessment and prediction of mental health disorders include: vocal exercises; reading neutral or emotionally charged words, sentences or passages, with or without mood induction by pictures or videos; as well as free-form speaking while describing emotionally arousing events or during interviews conducted by humans or virtual agents (Cummins et al., 2015) . Due to prior experiences that free-form speech might offer better prediction performance compared to a read passage (Cummins et al., 2015) , free-form speech elicitation methods might be preferable for ex-COVID-19 patients. While free-form speech collection during timeconsuming disorder-specific diagnostic interviews can provide valuable speech information, e.g., in PTSD assessment (Marmar et al., 2019) , it would require excessive time to conduct multiple diagnostic interviews for a broader range of COVID-19-relevant mental health problems, like generalized anxiety, depression, PTSD and suicidal ideation/behavior. Accordingly, for elicitation of speech in ex-COVID-19 patients, it might be more appropriate to use: (a) relevant modules of shorter structured clinical interviews, like M.I.N.I. (Sheehan et al., 1998; Winkler et al., 2020) ; (b) semi-structured interviews on COVID-19 experiences, which have been applied in qualitative research during the pandemic (Gillard et al., 2021; McKinlay et al., 2021) ; (c) COVID-19-related adaptations of very brief time-limited interviews, like the 3-min interview from Schultebraucks et al. (2020b) ; or (d) COVID-19-related concretizations of broad interview formats that are generally structured around 5W and H questions, e.g., (Dilts, 1990) .

More accurate prediction of potential psychiatric disorders among ex-COVID-19 patients would benefit from AI-based natural language processing (NLP) of free-form speech to make distinctions between stress vulnerable vs. stress resilient ex-COVID-19 patients. Such linguistic information and related unstructured data might be of great importance as valuable clinical information to identify relevant keywords for early detection of various mental health problems. NLP can capture patterns of irrational or distorted language associated with patterns of specific psychopathologies, like patient's mental distortions, biases, core beliefs, negative wording, and compensatory strategies. Using NLP techniques, we can uncover patterns of how specific psychopathologies are reflected in language, and related semantic and acoustic features across time, what might be helpful in post-COVID clinical settings. From a technical perspective, all answers, comments and observations during the interview are continuously monitored and serve as a basis for real-time, and subsequently off-line, NLP analysis. To support this, an automated conversion of microphone-recorded speech to text is performed in the background.

Many studies analyze speech content in terms of linguistic measures such as word frequencies, lexical diversity, narrative coherence, sentiment of speech content and others, using them as features to classify major depressive disorder (MDD) (Cummins et al., 2011 (Cummins et al., , 2015 Alghowinem et al., 2013; Calvo et al., 2017; Schultebraucks et al., 2020b) and PTSD He et al., 2017; Schultebraucks et al., 2020b) . Schultebraucks et al. (2020b) showed that the most important features for predicting PTSD and MDD status were the NLP features, while the features from other groups (face, acoustic and movement) also showed high potential. Features were calculated using the Linguistic Inquiry and Word Count (LIWC) 2015 dictionary (Pennebaker et al., 2015) and DeepSpeech (Hannun et al., 2014) from freeform speech responses to a brief interview 1 month after the trauma. Predictive importance ranking was done using SHAP (SHapley Additive exPlanation). The most important predictor for PTSD was NLP LIWC "self-assured, " while for MDD, it was age, followed by NLP LIWC "workhorse."

LIWC is a widely used software tool in mental health projects (Calvo et al., 2017) . It is composed of more than 6,000 words, word stems and selected emoticons and calculates approximately 90 output variables for each text input, i.e., word count, summary language variables (e.g., authenticity, emotional tone), general descriptor categories (e.g., words per sentence), word categories tapping psychological constructs (e.g., affect, cognition, biological processes), informal language markers (e.g., assents, fillers, swear words, netspeak) and others (Pennebaker et al., 2015) . In addition to LIWC, authors often use other linguistic tools such as N-gram language models, emotional thesauri like WordNet-affect (Strapparava and Valitutti, 2004) and normative databases such as the Affective Norm for English Words (ANEW) (Bradley and Lang, 1999) .

A study was done on the topic of wording trend analysis and other NLP analyzes in 15 of the world's largest mental health support groups found on Reddit (e.g., r/schizophrenia, r/SuicideWatch, r/Depression) during the initial stage of the COVID-19 pandemic (Low et al., 2020) . Beside the LIWC features, the authors also included sentiment metrics, basic word and syllable counts, punctuation, readability metrics, term frequency-inverse document frequency (TF-IDF) ngrams and manually built lexicons about suicidality, economic stress, isolation, substance use, domestic stress and guns. Although the paper is not directly related to the prediction or diagnosis of mental health disorders in ex-COVID-19 patients, the presented textual features and conclusions of the analyses can be used as valuable information for tuning and improving such prediction and diagnosis models.

Most recent studies in the field of mental health-related NLP, as shown above, are still based on the analysis of keywords and statistical representations of words, although NLP has been elevated to the context-based semantic level of word representations in the last few years. The reason behind this is mainly due to the ease of interpretation of keywordbased features, while on the other hand, the context-based representation of words is more abstract.

In general, this semantic NLP progress can be shown through the following most important achievements: bag-of-words (wordcount metrics that ignore grammatical structure) (Harris, 1954) , word2vec (word embeddings-static representation of words in a semantic multidimensional space based on the context in which these words occur in a specific language) (Mikolov et al., 2013) , and BERT (transformer-based language model that represents words and subwords through contextual embeddings in a way that they dynamically change their position in the semantic space depending on the specific environment, e.g., sentence, in which they are located) (Devlin et al., 2019) .

Acoustic features of speech have widely been used as indirect markers of underlying physiological phenomena. These features are useful within the clinical interview paradigm, described at the beginning of this section, as speech production involves both simultaneous cognitive planning and complex motoric muscular actions (Cummins et al., 2015) . Slight physiological and cognitive changes can produce noticeable acoustic changes (Scherer, 1986) in the perceived speech during such events. Examples of such changes were documented as far as 1921 (Kraepelin, 1921) , where voices of depressed patients were described as low, slow, hesitant and monotonous (amongst other things). While some of those descriptions can easily be explained with singular features, e.g., lower speech fundamental frequency (sometimes called pitch) in the case of low voice, or lower number of voiced frames in an utterance (speech rate) and pauses between words in the case of slow voice, we often find more complex groups of features describing such phenomena as they can produce more information for such a varied process like speech. Such features can also help to identify other breathing abnormalities (e.g., Mukherjee et al., 2021a,b) .

In our previous research we have focused on developing optimized stimulation paradigms for elicitation of multimodal responses related to stress resilience (Dropuljić et al., 2017; Ćosić et al., 2019b) and cognitive functioning estimation (Mijić et al., , 2019 . Acoustic feature sets were related to: (a) fundamental frequency, an estimate of the base harmonic that the vibrating vocal cords are producing during vocalization; RMS energy, a signal-processing-based estimate of the energy of the sound recorded by the microphone; (b) formant frequencies (f1-f4) and mel-frequency-cepstral-coefficients which describe the spectral and cepstral behavior of the recorded utterances; (c) jitter and shimmer, representing voice perturbations; and (d) number of voiced segments per second and mean voiced/unvoiced segment lengths in seconds, which are related to speech rate.

Predefined experimental paradigms (e.g., Dropuljić et al., 2017; Ćosić et al., 2019b ) may be applicable for collecting speech features that would be helpful in identifying ex-COVID-19 patients suffering from post-COVID-19-related psychopathology. However, we believe that using utterances from clinical interviews would be more beneficial as it enables a more "close-nit" integration with semantic features and freeform speaking has evident advantages when collecting speech data in the context of psychopathologies (Cummins et al., 2015) . Afshan et al. (2018) describe a particularly similar usecase, where depressed vs. normal speech was successfully classified using speech recordings of 1688 clinical interviews (735 depressive and anxiety disorders, 953 healthy). The used feature sets greatly overlapped with the ones we used in our stress resilience and cognitive load estimation research, described in the previous paragraph. A comprehensive framework for analyzing depression in the context of voice features is presented in Cummins et al. (2015) , alongside an exhaustive overview of papers that deal with particular voice features and how they relate to either the physiological or cognitive effects of depressive disorders. Such body of work provides the necessary interpretability needed in usecases where we deploy the abovedescribed features as diagnostic tools for usecases presented in this paper. An important property of speech as a modality is that it can easily and reliably be collected within the paradigm proposed in this paper, and can serve within the multimodal framework, complementing other modalities.

In summary, speech as a modality within the clinical interview paradigm, and as a component of a larger set of paradigms would prove a suitable augmentation alongside semantic, neurophysiological, and facial/oculometric features. Acoustic speech analysis would be particularly valuable alongside semantic analysis of the clinical interview, for prediction of potential mental health disorders like depression and early identification of various mental health risks in ex-COVID-19 patients. Furthermore, acoustic speech analysis may potentially be used as a real-time analysis tool that provides valuable information about the dynamics of the patients' emotional change during AI-based therapy described in section on Prevention, or a tool that helps show differences between resilient vs. vulnerable patients.

This section provides an overview of multimodal neurophysiological features as potential predictors of mental health deterioration in ex-COVID-19 patients. The main idea is a comprehensive set of carefully selected, elicited, and quantified neurophysiological features which can help identify those ex-COVID-19 patients who are at risk of developing post-COVID-19-related psychopathology, such as depression, anxiety, or PTSD (Liu C. H. et al., 2020; Mazza et al., 2020) . These features are assessed during specific test paradigms and tasks which elicit corresponding multimodal responses and can be associated with patients' neuro-psycho-physiological risk states, like acute stress, anxiety (Shaffer and Ginsberg, 2017; Arza et al., 2019) , and cognitive decline (Lynch and Lachman, 2020) ; as well as protective traits, like stress resilience (Walker et al., 2017) . Translation of earlier findings, particularly in the area of physiology-based resilience/vulnerability assessment (Ćosić et al., 2019a; Šarlija et al., 2021) , as well as in the area of objective neurophysiological assessment of cognitive functioning , could facilitate the design of efficient and reliable methods for multimodal prediction of potentially vulnerable individuals among ex-COVID-19 patients.

Stress resilience can be defined as an ability or a process of maintaining normal psychological, physiological, and physical functioning when exposed to high levels of stress and trauma (Russo et al., 2012) . Moreover, a review of various objective biomarkers of stress resilience by Walker et al. (2017) , described resilience as the absence of trauma-related psychiatric disorder symptoms, i.e., PTSD, as well as of other disorders like depression (Edward, 2005; Schultebraucks et al., 2021) . A primary vision highlighted in the review by Walker et al. (2017) , as well by our research on multimodal physiology-based stress resilience assessment (Ćosić et al., 2019a; Šarlija et al., 2021) , is the "early identification of individuals who are at risk of developing PTSD or depression" (Walker et al., 2017) , which becomes particularly valuable in the face of the COVID-19 pandemic (Ćosić et al., 2020a; Liu C. H. et al., 2020; Mazza et al., 2020) .

The deficits in a wide range of cognitive processes and/or dysfunction of the prefrontal cortex (PFC) and its connected circuitry are already associated with many mental disorders and neurological conditions (Friedman and Robbins, 2021) . The cognitive decline in ex-COVID-19 patients, related to cognitive deficits (Hampshire et al., 2021) , as well as global cognitive impairment, impairment in memory, attention, and executive function (Daroische et al., 2021) , is one of the specific characteristics of mental health vulnerabilities among the ex-COVID-19 patients. Cognitive assessment of patients with recent COVID-19 infection through specialized clinics dedicated to further diagnostic assessment and tailored rehabilitation is recognized as a preventive measure of mental health deterioration (Liotta et al., 2020; Daroische et al., 2021) .

Peripheral physiological features of interest are mainly related to resting autonomic nervous system (ANS) activity, like resting heart rate variability (HRV) and respiratory sinus arrhythmia (RSA); features of psychophysiological allostasis, like cardiac allostasis; as well as electromyogram-and electrodermal activity-based (EMG and EDA) acoustic startle response (ASR) features, like startle reactivity, startle habituation, and startlemodulation-related features, e.g., fear/anxiety-potentiated or prepulse inhibited startle response (Ćosić et al., 2019a; Šarlija et al., 2021) . In the following paragraphs, we aim to expand on our prior research related to identification and assessment of various multimodal physiological features of stress resilience (Ćosić et al., 2019a; Šarlija et al., 2021) in the direction of potential discriminators of ex-COVID-19 patients which are either resilient or at-risk for the development of COVID-19related psychopathology.

In our research on objectivization of stress resilience assessment (Ćosić et al., 2019a) , several physiological features confirmed discriminative power between resilient and nonresilient mentally healthy individuals. These were related to: (a) resting RSA, which measures HRV in phase with inhalation and exhalation (Grossman and Taylor, 2007) ; (b) EMG-based startle reactivity, which measures the strength of reflexive defensive responding to an aversive unconditioned stimulus, i.e., abrupt, loud noise (Bradford et al., 2014) ; and (c) cardiac allostasis, which measures adaptive reaction to a stressful event, involving a vigorous cardiac response to stress coupled with a significant cardiac recovery in the aftermath (Souza et al., 2013) . These three simple physiological features were also recognized as potential biomarkers of stress resilience according to an extensive review of relevant prior research on the psychophysiology of resilience (Walker et al., 2017) .

This paragraph provides a brief overview of physiological features supported by evidence from prospective studies, which are in the context of the current paper considered the most relevant category of literature. A simple EDA-based measure of sympathetic activity, collected during a standard trauma interview in the immediate aftermath of a traumatic event, was shown to predict risk of developing PTSD (Hinrichs et al., 2019) . Elevated heart rate following a traumatic event is a simple physiological feature that was associated with later development of warzone-related PTSD (Shalev et al., 1998; Dean et al., 2019) . Both increased heart rate and reduced high frequency HRV (HF-HRV) predicted PTSD development following a trauma exposure in women (Seligowski et al., 2021) . Another simple measure of resting vagally-mediated HRV, i.e., RMSSD, was negatively associated with development of depressive symptoms in healthy young subjects, over a period of 3 years (Carnevali et al., 2018) . HF-HRV and RMSSD are, besides RSA, also features which are linked to resting autonomic nervous system activity. Startle reactivity, introduced above as a marker of resilience, as well as other ASR-related features, e.g., startle habituation, were also found to be predictive of PTSD development (Shalev et al., 2000; Pole et al., 2009; Risbrough et al., 2012 Risbrough et al., , 2017 . In the context of COVID-19, EMG-based startle reactivity, interpreted as a physiological marker of anxiety was increased in COVID-19 physicians compared to non-COVID-19 physicians, and was also predictive of later anxiety scores measured by GAD-7 (Dolev et al., 2021) . A systematic literature review on the association between HRV reactivity to laboratory stress (which is similar to the concept of cardiac allostasis, mentioned in the paragraph above) and depression, states that none of reviewed studies have evaluated the prospective prediction of depressive symptoms (Hamilton and Alloy, 2016) . The authors claim that this "will be important in differentiating whether HRV reactivity is a consequence or correlate of depression or a marker of vulnerability to future depression, " which is in line with our discussion on the importance of prospective research. Besides physiological features, other markers such as polygenic scores (Schultebraucks et al., 2021) , cortisol (Galatzer-Levy et al., 2017) , or measures of hippocampal activity (van Rooij et al., 2018 (van Rooij et al., , 2021 were also identified as predictive of mental health deterioration after major life stressors, but a review of such an expanded set of potential predictors exceeds the scope of this section. A vast amount of research was also done on various other physiological features as correlates or diagnostic markers of PTSD, depression, anxiety, etc. but are here omitted due to their already mentioned decreased relevance in comparison with prospective research.

A brief literature review presented above demonstrates the potential of simple yet informative physiological features like resting RSA or EMG-based startle reactivity, which can be elicited and assessed using a relatively short and generic stimulation and evaluation protocol (Ćosić et al., 2019a,b; Šarlija et al., 2021) . Therefore, the proposed protocol for elicitation and assessment of physiological features in the prediction of mental health deterioration in ex-COVID-19 patients should include a resting period, a stress-inducing period with acoustic startle stimuli, and a post-stress recovery period, while the patients' peripheral physiological responses are being continuously recorded.

Cognitive decline, which is observable in the PFC activation on a variety of cognitive tasks, is one of the characteristics of mental health vulnerabilities in ex-COVID-19 patients (Daroische et al., 2021; Hampshire et al., 2021) . The prediction of vulnerable individuals among the ex-COVID-19 patients can be improved using brain monitoring techniques, and particularly features extracted from the PFC activation. The neuroanatomical connectivity of the PFC to most parts of the cortical and subcortical brain makes it well suited for participating in a number of neural networks and carrying out cognitive control operations in different functional domains (e.g., spatial, visual, and verbal) (Friedman and Robbins, 2021) . The PFC is linked through its extensive association connections with distant and broadly dispersed parts of the association and limbic cortices and these interconnections with the amygdala, hypothalamus, midbrain, and pons represent important subcortical linkages of the extended prefrontal neural system.

Research on mental disorders using brain imaging techniques, including functional near-infrared spectroscopy (fNIRS), a simple, wearable, and low-cost brain imaging technique which enables PFC activation estimation, suggests that these disorders change the way the brain functions during different cognitive challenges. fNIRS was used to estimate the PFC activation on a variety of cognitive tasks in patients with PTSD (Matsuo et al., 2003; Tian et al., 2014; Yennu et al., 2016; Gramlich et al., 2017) , panic disorder (Nishimura et al., 2007) , social anxiety disorder (Yokoyama et al., 2015) , and MDD (Nishizawa et al., 2019; Zhu et al., 2020) . Differences between these groups of patients and healthy individuals were found, suggesting a hypoactivation of the PFC in individuals with PTSD compared to healthy controls in verbal fluency task (Matsuo et al., 2003) , as well as hypoactivation in the left lateral PFC during the incongruent Stroop task (Yennu et al., 2016) . A general conclusion from these studies is that PTSD patients experience PFC hypoactivation during cognitive tasks, but their PFC activation increases dramatically when the person encounters an object associated with tragic and traumatic events (Kim et al., 2017) . A systematic review by Ho et al. (2020) on applications of fNIRS for MDD indicated that fNIRS consistently demonstrated attenuated cerebral hemodynamic changes in depressed compared to healthy individuals. Patients with MDD or panic disorder show hypoactivation in the PFC during cognitive tasks (Kim et al., 2017) , while high-trait-anxiety individuals showed a reduced PFC activation during the cognitive task compared to those with low anxiety (Brugnera et al., 2017) . In our research on cognitive functioning, we classified different cognitive states using neurophysiological signals , and showed differences in brain activation among individuals during cognitive tasks and discussed the applicability of such objective monitoring systems in a real-working environment . In addition to the research in patients with mental disorders and healthy controls, the fNIRS was used on ex-COVID-19 patients in the research of hypoxemia (Ferrari and Quaresima, 2020) as well as during presentation of olfactory stimuli (Ho et al., 2021) .

In summary, the tasks used in research include verbal fluency tasks, Stroop tasks, mathematical tasks, neutral and fearful faces presentation, and pleasant, unpleasant, and neutral sounds. The proposed paradigm for ex-COVID-19 patients should include cognitive tasks, such as mathematical tasks, n-back task, or Stroop task. In addition to the standard tasks for the study of memory processing and decision making, such paradigms should consist of images, sounds, and videos related to COVID-19 traumatic experiences. These paradigms would likely show differences in PFC activation on different tasks and might predict possible future mental disorders in vulnerable ex-COVID-19 patients.

Distortions in facial and oculometric system have been reported in patients with various psychiatric disorders when compared to controls. These detected oculometric distortions can be categorized either as behavioral or as physiological depending on the extracted features. This section presents an overview of a limited number of studies that discuss prediction of psychiatric disorders from facial and/or oculometric features. This method is still in the discovery phase, but it has great potential, particularly due to non-invasiveness and inexpensiveness.

Behavioral changes include dysfunctions in oculomotor performance related to attention and fixation. Several studies have shown that attention deficit is a feature of psychiatric disorders such as depression, anxiety disorder, schizophrenia, and bipolar disorder (Bittencourt et al., 2013; Kleineidam et al., 2019) . One often reported cognitive parameter for attention evaluation is saccadic eye movement. Patients with depression experienced shorter scanpath length in the free-viewing test, shorter duration of saccades and lower peak saccade velocity in the smooth pursuit test (Takahashi et al., 2021) , as well as showed a greater latency, a reduction in movement precision, and a reduced peak velocity in the antisaccade task (Bittencourt et al., 2013; Li et al., 2016) . Furthermore, individuals affected by PTSD and/or depression maintained their attention longer on the fearful, disgusted, and depressed expressions relative to the happy expression in comparison to controls (Sears et al., 2011; Armstrong et al., 2013; Lee and Lee, 2014; Powers et al., 2019) . Additionally, PTSD patients demonstrated a greater number of initial eye fixations on the threat word and more orienting responses on all threatening and neutral words than controls (Felmingham et al., 2011; Powers et al., 2019) .

Further, attention bias toward positive emotional faces studied in the laboratory has been shown to predict self-reported stress resilience (Thoern et al., 2016) . We have developed and optimized general stimulation paradigms for elicitation of facial and eye gaze features for stress resilience assessment/prediction (Ćosić et al., 2016, 2019b) .

Stress-induced physiological changes include dysfunctions in oculomotor performance exhibited in physiological parameters such as pupil diameter and spontaneous eye blink, which are modulated by ANS. Pupil diameter, modulated by the noradrenergic locus coeruleus (Rajkowski et al., 1993.) , is reported to be larger in patients with PTSD exposed to threatrelevant stimuli and negatively valanced pictures (Kimble et al., 2010; Cascardi et al., 2015; Mckinnon et al., 2020) . Previous studies have also indicated a relationship between spontaneous blink rate, a proxy of dopaminergic activity (Eckstein et al., 2017) , and anxiety and/or depression (Mackintosh et al., 1983; Kojima et al., 2002; Al-gawwam and Benaissa, 2018) .

Facial features that have been reported to be informative in classification of psychiatric disorders, especially depression, are facial expressions and face movement measures (Schultebraucks et al., 2020b; Stolicyn et al., 2020) . Facial expressions used for prediction are in the literature described and supported by individual components of muscle movement, called Action Units (AUs) (Yao et al., 2021) . There are 23 AUs all together, but some have stood out more than others in predicting psychopathological states (Gavrilescu and Vizireanu, 2019; Kumar et al., 2020) . Beside facial expressions, face movement measures are also indicators of various disorders. A study by Jaiswal et al. (2019) included head pose in prediction of depression and anxiety, along with AUs and eye gaze but the discriminative power of the feature itself was not discussed.

The abovementioned studies did not include ex-COVID-19 patients, but the presented findings are useful in the context of prediction of psychopathological states in ex-COVID-19 patients, who reported increased symptoms of anxiety and depression and high risk of developing PTSD, sleep abnormalities, and cognitive impairments (Mazza et al., 2020; Wu C. et al., 2020) .

The proposed protocol, based on the overview of the literature, should include clinical interview which would be video recorded and from which facial features then would be extracted. In addition to the recorded interview, the proposed protocol should also consist of oculometric paradigms such as antisaccade, free-viewing and smooth pursuit task, as well as images related to COVID-19 traumatic experiences, while the patients' oculometric responses are being continuously recorded.

Based on the literature review and discussion of potential multimodal predictors of psychopathology in ex-COVID-19 patients, we first summarize the proposal of an experimental protocol for elicitation, acquisition, and assessment of the described multimodal data (Figure 1) . The patients should first fill out a series of above listed self-report instruments. A relatively short structured clinical interview is suggested right after the self-report assessment. These two parts of the protocol can quickly and simply indicate the need of immediate psychiatric intervention in patients which are in the phase of emerging and evident psychopathology at the time of the screening. Furthermore, the clinical interview should be both audio and video recorded, similarly to a protocol recently used in deeplearning-based diagnosis of PTSD and depression following trauma (Schultebraucks et al., 2020b) . The audio recording would therefore provide data for the extraction of various semantic and acoustic speech features, and the video recording would provide data for extraction of various video-based, i.e., facial/oculometric features. Patients which are classified as mentally healthy based on the self-report instruments and the clinical interview are administered the remainder of the experimental protocol, which should include: (a) a short and generic stimulation protocol for elicitation and assessment of physiological features of stress resilience (Ćosić et al., 2019a; Šarlija et al., 2021) ; and (b) a set of appropriate tasks for assessment of various fNIRS-based and attention-related features.

The potential of AI in alleviating the consequences of the COVID-19 pandemic on mental health has already been FIGURE 1 | Illustration of time-synchronized multimodal input stimulation and multimodal output response of ex-COVID-19 patient during prediction protocol.

recognized (Ćosić et al., 2020a,b; Thenral and Annamalai, 2020) . The comprehensive data obtained through the proposed screening protocol of ex-COVID-19 patients can be analyzed by AI-based tools of unsupervised ML methods (e.g., multivariate correlation analyses, clustering, and principal component analysis), as well as a range of supervised ML methods (e.g., artificial neural networks, support vector machines, deeplearning supervised approaches). Such technologies are already being used in forecasting COVID-19 cases (e.g., Lalmuanawma et al., 2020; Santosh, 2020) , as well as in COVID-19-specific (e.g., Prout et al., 2020; Herbert et al., 2021) and more general mental health context related to prediction of stress-or trauma-related adverse mental health outcomes from different types of data. For example, traditional supervised learning methods have been applied on: (a) self-report psychological measures and variables collected in emergency room to predict PTSD and depression (Galatzer-Levy et al., 2014; Karstoft et al., 2015; Ziobrowski et al., 2021) ; (b) bio-physio-clinical-demographic features from EHRs augmented by acute stress self-report measures to predict PTSD course (Galatzer-Levy et al., 2017; Schultebraucks et al., 2020a) ; (c) clinical and demographic features from EHRs to identify veterans at high suicide risk (Kessler et al., 2017) ; and (d) polygenic scores to predict the trajectory of depression development (Schultebraucks et al., 2021) . Unsupervised clustering methods and physiological startle-based measures have been used in the long-term prediction of adverse psychological outcomes among physicians treating COVID-19 or non-COVID-19 patients (Dolev et al., 2021) . Deep learning methods applied on facial and speech data have been shown to successfully classify PTSD and depression following a trauma (Schultebraucks et al., 2020b) . The mentioned examples, as well as a recent review of 33 prognostic and 16 diagnostic studies on application of ML in trauma-related mental health disorders (Ramos-Lima et al., 2020) , highlight the ability of ML techniques to improve not only diagnosis, but also to enable prediction of negative trauma-related mental health outcomes, which is important in the context of the current work.

Besides establishing a comprehensive set of potentially predictive features (illustrated in Figure 2) , how we label the data deserves particular attention. The hierarchical data acquisition protocol described in the previous section would also be accompanied by a complex hierarchical fuzzy labelling system, which includes, e.g., PTSD risk ("high, " "medium, " "low"), depression risk ("high, " "medium, " "low"), state anxiety ("high, " "medium, " "low"). This labelling process requires longitudinal prospective research years after the initial measurement, which is feasible within the framework of post-COVID-19 clinics. The hierarchical data acquisition protocol should additionally result with a hierarchical prediction and decision support system, where preventive actions can truly be taken in a timely manner.

Methods based on supervised ML, while maximizing classification/prediction accuracy, often sacrifice model explainability and rigorous statistical validation. Accordingly, the need to focus on explainability, transparency, as well as generalizability of the ML models applied in psychiatry, by means of external validation of the developed models, has been recognized (Shatte et al., 2019; Chandler et al., 2020; Ramos-Lima et al., 2020) . For example, only 6 out of 28 studies reviewed in 2019 by Graham et al. included external validation. One generalizability-related challenge for AI in mental health is dealing with prediction models for rare disorders or events, e.g., suicides in particular, which can generate unacceptably high number of false positive predictions once deployed on a large scale in practice, even when exhibiting high accuracies, sensitivities and specificities (Belsher et al., 2019; Jiang et al., 2020; Machado et al., 2021) . Explainability, as a key prerequisite for the clinical use of the developed ML methods (Tjoa and Guan, 2020) , is important for reliable prediction, but also for personalized treatment and prevention of mental health deterioration in ex-COVID-19 patients. When mentioning explainability, people often refer to global explainability (Jha et al., 2021) , which is useful for understanding the underlying mechanisms of mental health deterioration, e.g., what variables contribute most significantly to a given prediction. Besides such global explanations, in the context of health applications, the local explainability of the developed models is crucial, i.e., providing the information to the end user, in this case the mental health practitioners, on why a particular decision was made. Examples of local explainability methods are LIME (Ribeiro et al., 2016) and SHAP (Lundberg and Lee, 2017) . Such explanations for risk assessments and predictions are needed for mental health practitioners to justify their reliability, as well as to better tailor the preventive action for a given patient (discussed in the next section), according to the main determinants of the predicted negative mental health outcome. A more extensive discussion on the potential of explainable AI (XAI) in addressing the complexity of mental health research, particularly related to human-computer interaction, is available in Roessner et al. (2021) . Furthermore, these new AI methods in prediction and prevention of mental health disorders should take into account human-computer interaction design principles on ethical, fair, trustworthy, and human-centered AI, which is important in the context of mental healthcare/suicide prevention and beyond (Xu, 2019; Robert et al., 2020) . Additionally, other challenges like ethics due to sensitive data assessment, social factors, suitable education and training of medical professionals, as well as lack of evidence for clinical and economic impact of using AI in clinical psychiatric practice, are discussed in the literature (Graham et al., 2019; Aung et al., 2021; Lee et al., 2021) .

Timely and appropriate preventive interventions among highrisk ex-COVID-19 patients, before the onset of serious psychopathology, should be of paramount common interest (Billings et al., 2020; Duan and Zhu, 2020) . A variety of digital technologies which may assist in this globally increasing problem, including AI-based tools, are being discussed in the literature (Bickman, 2020; Ćosić et al., 2020b; Maulik et al., 2020; Gargot et al., 2021) . Along these lines, the proposal in this section is based on our prior research work on tools and methods for prevention and treatment of stress-related disorders (Popovic et al., 2006; Ćosić et al., 2007 , 2010a , 2013a Popović et al., 2009) . The proposed preventive approach facilitated by AI tools is based on our research on an adaptive system for multimodal stimulation and multimodal real-time neuropsycho-physiological feedback loop in the context of cognitivebehavioral exposure therapy (Rothbaum and Schwartz, 2002) and stress inoculation training (Meichenbaum, 2017) , and is closely related to the emotion regulation framework (Gross, 1998) , emotion regulation therapy (Mennin, 2006; Renna et al., 2017 Renna et al., , 2020 , as well as cognitive restructuring (Clark, 2013) . Realtime processing of an individual's multimodal neuro-psychophysiological features, e.g., those presented in the previous section, is a prerequisite for a closed-loop emotion regulation strategy in the context of exposure therapy and stress inoculation training (Hodges and Rothbaum, 1998; Ćosić et al., 2010a) ; a similar closed-loop approach for a specific phobia has been demonstrated by Bãlan et al. (2020) . Closing the loop over a single feature is commonly found in bio/neurofeedback (Choi et al., 2011; Tan et al., 2011) , but more complex multimodal feedback has been more recently investigated in the context of fatigue and anxiety (Chang et al., 2016; Apostolidis et al., 2021) . A study by Chang et al. (2016) demonstrated multimodal biofeedback in AIbased driver fatigue detection and alerting system implemented on an embedded real-time digital signal processing hardware, using six HRV parameters extracted from ECG signal and classification of yawning and eye closure events from face recordings. A study by Apostolidis et al. (2021) presented an AIbased biofeedback system for anxiety awareness, using anxiety classification from galvanic skin response, heart rate, and skin temperature signals on a low-cost real-time processing board and displaying relaxing pictures in cases of high anxiety. Both studies introduced AI methods into the multimodal feedback loop, but still used only one or two modalities and a limited FIGURE 2 | Multimodal data acquisition as input for real-time and offline feature computation. Illustrated subset of selected features includes: General Health Questionnaire (GHQ-12), Connor-Davidson Resilience Scale (CD-RISC), COVID-19 Peritraumatic Distress Index (CPDI); NLP feature LIWC "sad," voice fundamental frequency (F 0 ), voice root mean square (RMS); respiratory sinus arrhythmia (RSA), EMG-based startle reactivity (SR EMG ), prefrontal cortex activity (PFC act ); saccadic peak velocity (SPV), pupil dilation (PD), a feature related to facial action coding system (FACS).

number of features. In the context of mental health disorder prevention, more modalities and features are required, which would then require an optimal feature selection, faster AI models, as well as a higher computational power with an emphasis on real-time performance. As modalities and input complexity increase, so do real-time processing device requirements (Adams and Marketing, 2002) , including sufficient number of cores for parallel processing and intensive computing, adequate amount of memory, high-speed interfaces capable of moving large amounts of data, and cost efficiency.

In this context, multisensory and multimodal interaction between the AI-assisted therapist and the treated ex-COVID-19 patient is based on a variety of customized multisensory inputs, such as non-interactive visual and auditory stimuli, interactive virtual reality stimuli and digital serious games (Li et al., 2014; Lau et al., 2017) , as well as comprehensive multimodal neuro-psycho-physiological feedback features. Closed-loop control strategies, which match selected multisensory stimuli to multimodal feedback in real-time using a broad range of AI-based algorithms, from fuzzy logic (Popovic et al., 2006; Cosić et al., 2007) to machine learning (Ćosić et al., 2020b) , are related to different emotion regulation, exposure, stress inoculation and cognitive restructuring methods (Popovic et al., 2006; Ćosić et al., 2007 , 2010a , 2013a Popović et al., 2009) . Conceptual illustration of the integration of AI-based tools with interventions relying on various psychotherapeutic preventive strategies is presented in Figure 3 .

Patient-personalized real-time multisensory stimuli generation uses semantically and emotionally annotated pictures, natural and artificial sounds, recorded speech, written text messages, video-clips and films or virtual reality synthetic environments, which reflects the underlying complexity of potential chronic psychopathology. Usage of existing and dedicated large databases of images and/or sounds, which are semantically (Deng et al., 2009 ) and emotionally annotated (Horvat, 2017) via AI-tools related to ontologies (Horvat et al., , 2012 (Horvat et al., , 2014 and automated semantic segmentation (Chen et al., 2018) , should optimize the delivery of the most relevant semantic and emotional stimuli content to the ex-COVID-19 patient during the intervention course, which is adapted to the patient's reactions in real time (Ćosić et al., 2010a) . Real-time computation of multimodal neurophysiological, speech and facial/oculometric features, as well as real-time computation of optimal control multimodal closed-loop feedback stimuli is a particular software and hardware challenge. Efficient search of comprehensive stimuli databases and selection of semantically and emotionally optimal stimuli within therapeutic context and strategy also deserve particular attention.

Comprehensive patient monitoring during emotion regulation in a multisensory and multimodal closedloop feedback paradigm, based on real-time tracking of multidimensional neuro-psycho-physiological features, better reflects the complexity of the problem of stress-induced psychopathologies than focusing exclusively on conscious thinking process or a dysregulation of a single physiological variable. It is expected that augmentation of the number and variety of features to be computed in real time for closed-loop regulation definitely exceeds the capacities of any mental health professional. The presented AI-based approach, if available in post-COVID clinics after its scientific validation, may significantly relieve the overburdened mental health system. Future research on personalized AI-based multisensory stimulation and multimodal feedback regulation may lead to novel digital clinical tools for more comprehensive patient screening, monitoring and reducing therapist burden, which is in line with calls for development and integration of new technology-enabled models of care to gain substantial improvements in patients' mental health (Iorfino et al., 2021 ). An AI-assisted closed-loop psychotherapeutic approach within COVID-19 pandemic may also assist in lower cost, better and faster stratification of ex-COVID-19 patients according to high-medium-low risk of chronic psychopathology. Stratification of risk using the AI-based prediction methods, supplemented with personalized face-to-face risk assessments as necessary, can facilitate matching of ex-COVID-19 individuals with the most appropriate preventive psychotherapeutic intervention in severely resource-limited psychotherapeutic practices. For example, high-risk individuals might require the most effective first-line interventions with extensive involvement of the psychotherapist, while persons with moderately elevated risk might undergo validated online preventive intervention programs with minimal or no psychotherapist involvement. Prediction of risk will benefit from AI tools, as proposed in this paper, keeping in mind the necessity of future work on trustworthiness and generalizability of the developed methods, as discussed in subsection 2.6. On the other hand, the preventive intervention prescribed to a particular ex-COVID-19 patient based on his/her stratified risk might or might not use AI, depending on the selected intervention from the pool of validated preventive approaches that are available in a particular region. The proposed AI-based predictive and preventive approach should be viewed as a part of the whole package of measures focusing on protection of mental health during COVID-19 pandemic, which also include wide-ranging efforts emphasizing the protective potential of, e.g., physical exercise, sleep, and finding the sense of meaning in life (De Jong et al., 2020; Zhang Y. et al., 2020; Rajkumar, 2021) , in times of physical distancing, isolations and other negative psycho-socioeconomic consequences of attempts to limit the spread and mortality of the virus.

The paper conceptually contributes to the accelerating stateof-the-art research in AI-based prediction of mental health disorders, particularly in ex-COVID-19 patients, calling for further coordinated large scale research efforts in prediction and prevention of mental health disorders, while integrating different categories of multidisciplinary data that have been shown promising in a growing number of separate research studies. The proposed AI-based methodology in prediction and prevention of negative psychological and behavioral consequences in ex-COVID-19 patients, potentially applied in newly established post-COVID clinics, deserves more attention. It expands traditionally used self-rating scales and psychiatric assessment questionnaires and interviews in diagnosis of mental health disorders with more objective multimodal neuro-psychophysiological metrics based on state-of-the-art AI tools. The proposed prediction protocol may contribute to selection of post-COVID-stress most vulnerable individuals in immediate need for psychiatric treatment and psychological interventions. This approach might also be applied generally for all individuals who were exposed to higher levels of mental health risks during the COVID-19 pandemic and might assist in diagnostic process and in selection of the most appropriate preventive treatment strategies. Early prediction and effective treatment of ex-COVID-19 patients with the proposed methodology, particularly for most vulnerable individuals, is crucially important in prevention of widespread serious mental health diseases. The complexity of such interdisciplinary and multidisciplinary research deserves more global attention, particularly within the World Health Organization. Establishing a multinational interdisciplinary task force for policy design, planning and development of more advanced AI-based innovation in digital psychiatry may result in more successful and efficient strategies in coping with post-COVID overall mental health deterioration, leading to better, faster and more affordable global mental health services for all.

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

KĆ conceptualized the idea and design of the research study and wrote the first draft of the manuscript. SP, MŠ, IK, MG, BD, and IM wrote sections of the manuscript. TJ and NH provided critical revision of the work. All authors contributed to manuscript revision, read, and approved the submitted version.

Choosing the Right Architecture for Real-Time Signal Processing Designs. Texas Instruments, Document Number SPRA879

Effectiveness of voice quality features in detecting depression

The fear of COVID-19 scale: development and initial validation

Depression detection from eye blink features

Detecting depression: a comparison between spontaneous and read speech

Interpreting scores on the Kessler psychological distress scale (K10)

Deployment and dynamics of a biofeedback system for anxiety awareness during online examination activities

Dwelling on potential threat cues: an eye movement marker for combat-related PTSD

Measuring acute stress response through physiological signals: towards a quantitative assessment of stress

Do pre-existing anxiety-related and mood disorders differentially impact COVID-19 stress responses and coping?

The promise of artificial intelligence: a review of the opportunities and challenges of artificial intelligence in healthcare

An investigation of various machine and deep learning techniques applied in automatic fear level detection and acrophobia virtual therapy

Increased generalized anxiety, depression and distress during the COVID-19 pandemic: a cross-sectional study in Germany

An inventory for measuring depression

Prediction models for suicide attempts and deaths: a systematic review and simulation

Improving mental health services: a 50-year journey from randomized experiments to artificial intelligence and precision mental health

Supporting hospital staff during COVID-19: early interventions

Fear of COVID-19 scale: psychometric characteristics, reliability and validity in the Israeli population

Saccadic eye movement applications for psychiatric disorders

The posttraumatic stress disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation

Not just noise: individual differences in general startle reactivity predict startle response to uncertain and certain threat

Affective Norms for English Words (Anew): Instruction Manual and Affective Ratings

The brief illness perception questionnaire

Cortical and autonomic stress responses in adults with high versus low levels of trait anxiety: a pilot study

A cross-sectional study on mental health among health care workers during the outbreak of corona virus disease 2019

Natural language processing in mental health applications using non-clinical texts

Heart rate variability mediates the link between rumination and depressive symptoms: a longitudinal study

Pupil response to threat in trauma-exposed individuals with or without PTSD

Using machine learning in psychiatry: the need to establish a framework that nurtures trustworthiness

Real-time physiological and facial monitoring for safe driving

Predictors of health-related quality of life and influencing factors for COVID-19 patients, a follow-up at one month. Front

Masklab: instance segmentation by refining object detection with semantic and direction features

Global imperative of suicidal ideation in 10 countries amid the CoViD-19 pandemic. Front

Is alpha wave neurofeedback effective with randomized clinical trials in depression? A pilot study

Cognitive restructuring

Development of a new resilience scale: the Connor-Davidson resilience scale (CD-RISC)

Virtual reality adaptive stimulation in stress resistance training

Computer-aided psychotherapy based on multimodal elicitation, estimation and regulation of emotion

Multimodal paradigm for mental readiness training and PTSD prevention

Physiology-driven adaptive VR system: technology and rationale for PTSD treatment

Virtual reality adaptive stimulation of limbic networks in the mental readiness training

Multimodal analysis of startle type responses

Physiology-driven adaptive virtual reality stimulation for prevention and treatment of stress related disorders

Impact of human disasters and COVID-19 pandemic on mental health: potential of digital psychiatry

Artificial intelligence in prediction of mental health disorders induced by the COVID-19 pandemic among health care workers

New tools and methods in selection of air traffic controllers based on multimodal psychophysiological measurements

Stress resilience assessment based on physiological features in selection of air traffic controllers

An investigation of depressed speech detection: features and normalization

A review of depression and suicide risk assessment using speech analysis

Follow-up survey of US adult reports of mental health, substance use, and suicidal ideation during the COVID-19 pandemic

Cognitive impairment after COVID-19-a review on objective test data

From shattered goals to meaning in life: life crafting in times of the COVID-19 pandemic

The anxiety sensitivity index-revised: psychometric properties and factor structure in two nonclinical samples

Multi-omic biomarker identification and validation for diagnosing warzone-related post-traumatic stress disorder

Imagenet: a large-scale hierarchical image database

DS14: standard assessment of negative affectivity, social inhibition, and type D personality

BERT: pre-training of deep bidirectional transformers for language understanding

Mental health of healthcare workers during the COVID-19 pandemic in Italy

Changing Belief Systems With NLP

Physiological parameters of mental health predict the emergence of posttraumatic stress symptoms in physicians treating COVID-19 patients

Vocal analysis of acoustic startle responses

Psychological interventions for people affected by the COVID-19 epidemic

Beyond eye gaze: what else can eyetracking reveal about cognition and cognitive development?

Resilience: a protector from depression

A meta-analytic examination of hardiness

Examining the psychometric properties of a refined perceived stress scale during the COVID-19 pandemic

Quantifying the association between psychotherapy content and clinical outcomes using deep learning

Tech giants enter mental health

Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: the adverse childhood experiences (ACE) Study

Eye tracking and physiological reactivity to threatening stimuli in posttraumatic stress disorder

Hypoxemia in COVID-19: cerebral oximetry should be explored as a warning indicator for mechanically ventilated adults with COVID-19

Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial

The role of prefrontal cortex in cognitive control and executive function

Quantitative forecasting of PTSD from early trauma responses: a machine learning application

Utilization of machine learning for prediction of post-traumatic stress: a reexamination of cortisol in the prediction and pathways to non-remitting PTSD

Does the 12-item general health questionnaire contain multiple factors and do we need them? Health Qual

Mental health monitoring with multimodal sensing and machine learning: a survey

Preventing post traumatic stress disorder in the general population induced by trauma during the COVID pandemic: a simple brief intervention based on cognitive science that could be delivered digitally

Predicting depression, anxiety, and stress levels from videos using the facial action coding system

Experiences of living with mental health problems during the COVID-19 pandemic in the UK: a coproduced, participatory qualitative interview study

Artificial intelligence for mental health and mental illnesses: an overview

A functional near-infrared spectroscopy study of trauma-related auditory and olfactory cues: posttraumatic stress disorder or combat experience?

Psychometric properties of the life events checklist

The emerging field of emotion regulation: an integrative review

Toward understanding respiratory sinus arrhythmia: relations to cardiac vagal tone, evolution and biobehavioral functions

Atypical reactivity of heart rate variability to stress and depression across development: systematic review of the literature and directions for future research

Cognitive deficits in people who have recovered from COVID-19

Do psychiatric patients experience more psychiatric symptoms during COVID-19 pandemic and lockdown? A case-control study with service and research implications for immunopsychiatry

Distributional structure

Automated assessment of patients' self-narratives for posttraumatic stress disorder screening using natural language processing and text mining

How do you feel during the COVID-19 pandemic? A survey using psychological and linguistic selfreport measures, and machine learning to investigate mental health, subjective experience, personality, and behaviour during the COVID-19 pandemic among university students

Increased skin conductance response in the immediate aftermath of trauma predicts PTSD risk

Diagnostic and predictive applications of functional near-infrared spectroscopy for major depressive disorder: a systematic review

Comparison of brain activation patterns during olfactory stimuli between recovered COVID-19 patients and healthy controls: a functional near-infrared spectroscopy (fNIRS) study

The development, use, and psychometric properties of the Trauma history questionnaire

A brief overview of affective multimedia databases

STIMONT: a core ontology for multimedia stimuli description

WNtags: a web-based tool for image labeling and retrieval with lexical ontologies

Tagging multimedia stimuli with ontologies

Sociodemographic and psychological risk factors for anxiety and depression: findings from the Covid-19 health and adherence research in Scotland on mental health (CHARIS-MH) cross-sectional survey

The impact of technology-enabled care coordination in a complex mental health system: a local system dynamics model

Automatic prediction of depression and anxiety from behaviour and personality attributes

Psychological distress after Covid-19 recovery: reciprocal effects with temperament and emotional dysregulation. An exploratory study of patients over 60 years of age assessed in a post-acute care service. Front

Learning the mental health impact of COVID-19 in the United States with explainable artificial intelligence: observational study

Supervised machine learning: a brief primer

Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth

A conceptual framework for the neurobiological study of resilience

Bridging a translational gap: using machine learning to improve the prediction of PTSD

Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans health Administration

Air traffic control competencies measurements based on functional near-infrared spectroscopy

Classification of cognitive load based on neurophysiological features from functional nearinfrared spectroscopy and electrocardiography signals on n-back task

Application of functional near-infrared spectroscopy to the study of brain function in humans and animal models

Eye tracking and visual attention to threating stimuli in veterans of the Iraq war

Antisaccade and prosaccade eye movements in individuals clinically at risk for psychosis: comparison with first-episode schizophrenia and prediction of conversion

The Post-COVID-19 functional status scale: a tool to measure functional status over time after COVID-19

Blink rate variability in patients with panic disorder: new trial using audiovisual stimulation

Adversity history predicts self-reported autonomic reactivity and mental health in US residents during the COVID-19 pandemic. Front

Manic depressive insanity and paranoia

Analysing the effective psychological state of students using facial features

Applications of machine learning and artificial intelligence for Covid-19 (SARS-CoV-2) pandemic: a review

Serious games for mental health: are they accessible, feasible, and effective? A systematic review and meta-analysis. Front

Artificial intelligence for mental healthcare: clinical applications, barriers, facilitators, and artificial wisdom

Attentional bias towards emotional facial expressions in survivors of dating violence

Game-based digital interventions for depression therapy: a systematic review and meta-analysis

Self-control moderates the association between perceived severity of coronavirus disease 2019 (COVID-19) and mental health problems among the Chinese public

Eye movement indices in the study of depressive disorder. Shanghai Arch

Knowledge, attitudes, impact, and anxiety regarding COVID-19 infection among the public in China

Frequent neurologic manifestations and encephalopathy-associated morbidity in Covid-19 patients

Factors associated with depression, anxiety, and PTSD symptomatology during the COVID-19 pandemic: clinical implications for US young adult mental health

Prevalence and predictors of PTSS during COVID-19 outbreak in China hardest-hit areas: gender differences matter

Natural language processing reveals vulnerable mental health support groups and heightened health anxiety on reddit during covid-19: observational study

Reporting mental health symptoms: breaking down barriers to care with virtual human interviewers

A unified approach to interpreting model predictions

The effects of lifetime trauma exposure on cognitive functioning in midlife

Prediction of suicide attempts in a prospective cohort study with a nationally representative sample of the US population

Blink rate in psychiatric illness

The multi-factor structure of the brief self-control scale: discriminant validity of restraint and impulsivity

Post-traumatic stress and stress disorders during the COVID-19 pandemic: survey of emergency physicians

Speech-based markers for posttraumatic stress disorder in US veterans

The Peritraumatic dissociative experiences questionnaire, " in Assessing Psychological Trauma and Posttraumatic Stress Disorder

Hypoactivation of the prefrontal cortex during verbal fluency test in PTSD: a near-infrared spectroscopy study

Roadmap to strengthen global mental health systems to tackle the impact of the COVID-19 pandemic

Anxiety and depression in COVID-19 survivors: role of inflammatory and clinical predictors

New-onset and exacerbated insomnia symptoms during the COVID-19 pandemic in US military veterans: a nationally representative, prospective cohort study

A qualitative study about the mental health and wellbeing of older adults in the UK during the COVID-19 pandemic

Enhanced emotional response to both negative and positive images in post-traumatic stress disorder: evidence from pupillometry

Stress inoculation training: a preventative and treatment approach

Emotion regulation therapy: an integrative approach to treatment-resistant anxiety disorders

Classification of cognitive load using voice features: a preliminary investigation

MMOD-COG: a database for multimodal cognitive load classification

Distributed representations of words and phrases and their compositionality

Lung health analysis: adventitious respiratory sound classification using filterbank energies

Automatic lung health screening using respiratory sounds

Frontal dysfunction during a cognitive task in drug-naive patients with panic disorder as investigated by multi-channel near-infrared spectroscopy imaging

fNIRS assessment during an emotional stroop task among patients with depression: replication and extension

The Development and Psychometric Properties of LIWC2015

Mental health before and during the COVID-19 pandemic: a longitudinal probability sample survey of the UK population

Prospective prediction of posttraumatic stress disorder symptoms using fear potentiated auditory startle responses

Stress inoculation training supported by physiology-driven adaptive virtual reality stimulation

Scenario self-adaptation in virtual reality exposure therapy for posttraumatic stress disorder

Attention bias toward threatening faces in women with PTSD: eye tracking correlates by symptom cluster

Identifying predictors of psychological distress during COVID-19: a machine learning approach

A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: implications and policy recommendations

Correcting the brain? The convergence of neuroscience, neurotechnology, psychiatry, and artificial intelligence

Correlations between locus coeruleus (LC) neural activity, pupil diameter and behavior in monkey support a role of LC in attention

Sleep, physical activity and mental health during the COVID-19 pandemic: complexities and opportunities for intervention

The use of machine learning techniques in trauma-related disorders: a systematic review

Predictive risk factors for posttraumatic stress symptoms among nurses during the Italian acute COVID-19 outbreak

Emotion regulation therapy and its potential role in the treatment of chronic stress-related pathology across disorders

Emotion regulation therapy: a mechanism-targeted treatment for disorders of distress

The mediating role of illness perception in the relationship between psychological hardiness and the health status of covid-19 (coronavirus disease) survived patients

Explaining the predictions of any classifier

Association between psychological resilience and changes in mental distress during the COVID-19 pandemic

Prospective assessment of psychophysiological risk factors for PTSD

88. Identification of psychophysiological markers of PTSD risk and potential use as intermediate phenotypes

ICIS 2019 SIGHCI workshop panel report: human-computer interaction challenges and opportunities for fair, trustworthy and ethical artificial intelligence

Further psychometric support for the 10-item version of the perceived stress scale

Taming the chaos? Using eXplainable artificial intelligence (XAI) to tackle the complexity in mental health research

Exposure therapy for posttraumatic stress disorder

As their numbers grow, COVID-19 "long haulers" stump experts

Neurobiology of resilience

AI-driven tools for coronavirus outbreak: need of active learning and cross-population train/test models on multitudinal/multimodal data

Prediction of task performance from physiological features of stress resilience

Vocal affect expression: a review and a model for future research

Selfreported symptoms of depression and PTSD are associated with reduced vowel space in screening interviews

Discriminating heterogeneous trajectories of resilience and depression after major life stressors using polygenic scores

A validated predictive algorithm of post-traumatic stress course following emergency department admission after a traumatic stressor

Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood

Attention to emotional images in previously depressed individuals: an eye-tracking study

A prospective examination of sex differences in posttraumatic autonomic functioning

An overview of heart rate variability metrics and norms

Auditory startle response in trauma survivors with posttraumatic stress disorder: a prospective study

A prospective study of heart rate response following trauma and the subsequent development of posttraumatic stress disorder

Machine learning in mental health: a scoping review of methods and applications

The mini-international neuropsychiatric interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10

Mental health outcomes associated with the COVID-19 pandemic: prevalence and risk factors in a southern US state

The distress tolerance scale: development and validation of a self-report measure

The brief resilience scale: assessing the ability to bounce back

Resting vagal control and resilience as predictors of cardiovascular allostasis in peacekeepers

Psychometric properties of the STAI: a reply to Ramanaiah, Franzen, and Schill

A brief measure for assessing generalized anxiety disorder: the GAD-7

Prediction of depression symptoms in individual subjects with face and eye movement tracking

WordNet-affect: an affective extension of WordNet

Eye movement abnormalities in major depressive disorder

Heart rate variability (HRV) and posttraumatic stress disorder (PTSD): a pilot study

Telepsychiatry and the role of artificial intelligence in mental health in post-COVID-19 India: a scoping review on opportunities

Attentional bias towards positive emotion predicts stress resilience

Prefrontal responses to digit span memory phases in patients with post-traumatic stress disorder (PTSD): a functional near infrared spectroscopy study

A survey on explainable artificial intelligence (XAI): toward medical XAI

The effect of type D personality on anxiety, depression and fear of COVID-19 disease in healthcare workers

Hippocampal activation during contextual fear inhibition related to resilience in the early aftermath of trauma

The role of the hippocampus in predicting future posttraumatic stress disorder symptoms in recently traumatized civilians

Psycho-social factors associated with mental resilience in the Corona lockdown

In the search for integrative biomarker of resilience to psychological stress

An inside look at a post-COVID-19 clinic

A longitudinal study on the mental health of general population during the COVID-19 epidemic in China

A 12-Item short-form health survey: construction of scales and preliminary tests of reliability and validity

The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection

Anxiety sensitivity, COVID-19 fear, and mental health: results from a United States population sample

The impact of event scale-revised

Increase in prevalence of current mental disorders in the context of COVID-19: analysis of repeated nationwide cross-sectional surveys

Mental health status and related influencing factors of COVID-19 survivors in Wuhan

Prevalence of mental health problems during the COVID-19 pandemic: a systematic review and meta-analysis

Toward human-centered AI: a perspective from human-computer interaction

Action unit classification for facial expression recognition using active learning and SVM

Prefrontal responses to Stroop tasks in subjects with post-traumatic stress disorder assessed by functional near infrared spectroscopy

Dysfunction of ventrolateral prefrontal cortex underlying social anxiety disorder: a multi-channel NIRS study

The adverse childhood experiences questionnaire: two decades of research on childhood trauma as a primary cause of adult mental illness, addiction, and medical diseases

At the height of the storm: healthcare staff 's health conditions and job satisfaction and their associated predictors during the epidemic peak of COVID-19

Mental health problems during the COVID-19 pandemics and the mitigation effects of exercise: a longitudinal study of college students in China

Classifying major depressive disorder using fNIRS during motor rehabilitation

Development and validation of a model to predict posttraumatic stress disorder and major depression after a motor vehicle collision

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.Publisher's Note: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.Copyright © 2021Ćosić, Popović, Šarlija, Kesedžić, Gambiraža, Dropuljić, Mijić, Henigsberg and Jovanovic. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.