key: cord-0185214-6luiv6gq
authors: Mukherjee, Sayanti; Rintamaki, Lance; Shucard, Janet L.; Wei, Zhiyuan; Carlasare, Lindsey E.; Sinsky, Christine A.
title: The Invisible COVID-19 Crisis: Post-Traumatic Stress Disorder Risk Among Frontline Physicians Treating COVID-19 Patients
date: 2021-10-25
journal: nan
DOI: nan
sha: c02e311b920755fa9ba4f695fcac766bb2f1b03c
doc_id: 185214
cord_uid: 6luiv6gq

This study evaluated post traumatic stress disorder (PTSD) among frontline US physicians (treating COVID-19 patients) in comparison with second-line physicians (not treating COVID-19 patients), and identified the significance and patterns of factors associated with higher PTSD risk. A cross-sectional, web-based survey was deployed during August and September, 2020, to practicing physicians in the 18 states with the largest COVID-19 cases. Among 1,478 responding physicians, 1,017 completed the PTSD Checklist (PCL-5). First, the PCL-5 was used to compare symptom endorsement between the two physician groups. A greater percentage of frontline than second-line physicians had clinically significant endorsement of PCL-5 symptoms and higher PCL-5 scores. Second, logistic regression and seven nonlinear machine learning (ML) algorithms were leveraged to identify potential predictors of PTSD risk by analyzing variable importance and partial dependence plots. Predictors of PTSD risk included cognitive/psychological measures, occupational characteristics, work experiences, social support, demographics, and workplace characteristics. Importantly, the final ML model random forest, identified patterns of both damaging and protective predictors of PTSD risk among frontline physicians. Key damaging factors included depression, burnout, negative coping, fears of contracting/transmitting COVID-19, perceived stigma, and insufficient resources to treat COVID-19 patients. Protective factors included resilience and support from employers/friends/family/significant others. This study underscores the value of ML algorithms to uncover nonlinear relationships among protective/damaging risk factors for PTSD in frontline physicians, which may better inform interventions to prepare healthcare systems for future epidemics/pandemics.

• Physicians working at the frontlines of COVID-19 are at high risk for PTSD • Depression, burnout and negative coping strategies are top ranked PTSD risk factors

• Fears of contracting/transmitting COVID-19 and perceived stigma increase PTSD risk • Higher resilience and support from employer, friends, and family decrease PTSD risk • Nonlinear relationships exist between PTSD risk and specific risk/protective factors

During previous epidemics, including SARS-2003 , H1N1 Influenza-2009 , and MERS-2012 frontline physicians (i.e., those treating infected patients) endured formidable social, psychological, and emotional stressors (De Brier et al., 2020; Goulia et al., 2010; Imai et al., 2010; Khalid et al., 2016 ); yet, COVID-19 far exceeds in scope and scale the devastation wrought by these earlier outbreaks. Physicians have been overrun by caseloads of acutely ill patients (Restauri and Sheridan, 2020; Wu et al., 2009) , insufficient resources (Marco et al., 2020; Restauri and Sheridan, 2020) , and risks inherent to working with a new and highly infectious disease, culminating in the deaths of over 596,572 Americans (Jun 11, 2021) (Centers for Disease Control and Prevention (CDC), 2021). A recent survey found that 30% of physicians experienced high stress, anxiety, or depression due to COVID-19's impact, and 62% had considerable fear of contracting or transmitting COVID-19 (Linzer et al., 2021) . The traumatic nature of COVID-19 associated stressors makes post-traumatic stress disorder (PTSD) of special consequence for COVID-19 frontline physicians (Carmassi et al., 2020) .

As defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), PTSD is a psychiatric disorder that follows exposure to a traumatic event (Criterion A), and is characterized by four behavioral, cognitive, and emotional symptom criteria: (A) intrusive, distressing thoughts; (B) persistent avoidance of trauma-related stimuli; (C) alterations in cognitions and mood; and (D) heightened arousal and reactivity (American Psychiatric Association, 2013) . These symptoms can persist for decades (Rintamaki et al., 2009) . Physicians with PTSD are susceptible to negative coping strategies (e.g., substance abuse) in attempt to manage their symptoms, other mental health conditions (e.g., depression), and suicidality (Gradus et al., 2010; Gregory et al., 2019; Liebschutz et al., 2007) . Thus, although physicians on the frontlines of emergent infectious diseases are at high risk of PTSD Maunder et al., 2003; Nickell et al., 2004; Wu et al., 2009) ; assessment of these phenomena in the context of COVID-19 is just emerging in the United States (Blekas et al., 2020; Kang et al., 2020; Lai et al., 2020; Li et al., 2020) , which leads the world in COVID-19 cases.

The aims of the current research were to: 1) evaluate the symptoms of PTSD among frontline physicians compared to second-line physicians; 2) predict PTSD risk among frontline physicians using nonparametric machine learning algorithms; 3) identify and rank key predictors associated with PTSD risk of frontline physicians; and 4) determine the linear/nonlinear patterns of these predictors. Novel COVID-19 related social, emotional, and cognitive factors were assessed while accounting for other known factors influencing PTSD, which included psychological resilience (Colville et al., 2017; Hamid and Musa, 2017; Winkel et al., 2019) , exposure rate (Jung et al., 2020; Kang et al., 2020; Lai et al., 2020) , occupational role (Lai et al., 2020; Maunder et al., 2004) , age (Sim et al., 2004; Wu et al., 2009) , sex , marital status (Chan and Huak, 2004) , isolation (Wu et al., 2009) , coping strategies (Chan and Huak, 2004; Maunder et al., 2006; Wu et al., 2009) , and social support from family (Chan and Huak, 2004; Su et al., 2007) , colleagues and organization (Chan and Huak, 2004; Lancee et al., 2008; Maunder et al., 2006) .

Following IRB approval, a cross-sectional, web-based survey developed by our interdisciplinary team was deployed to physicians from the American Medical Association's (AMA) Physician

Masterfile database between August 7 and September 26, 2020. Sampling immediately followed the second COVID-19 US contagion peak (Johns Hopkins University & Medicine, 2021) .

Participation was voluntary and targeted physicians from states reporting the greatest COVID-19 caseloads (more than 40,000 COVID-19 cases as of June 2020) (CDC, 2020), including New York, California, New Jersey, Illinois, Texas, Massachusetts, Florida, Pennsylvania, Michigan, Georgia, Maryland, Virginia, North Carolina, Arizona, Louisiana, Connecticut, Ohio, and Indiana.

Physicians from all specialties were recruited using the AMA Physician Masterfile, a nearcomplete record of all US physicians, independent of AMA membership. Canvassing e-mails (including study description and survey link) were sent on August 7 and 26, 2020. From 36,372 physicians (opening invitation), 1,478 responses were recorded, of which 1,017 responses (completing PTSD Checklist: PCL-5) were analyzed (eFigure 1).

We employed the PTSD Checklist, a commonly used research and clinical screening questionnaire based on the DSM-5 (PCL-5) (Weathers et al., 2013) to assess PTSD symptoms (see Table 2 ). Using this 20-item, 5-point scale (0=not at all to 4=extremely), respondents rated how bothered they were by PTSD symptoms in the past month. The total score range is 0-80, with 33 or greater indicating probable PTSD. A diagnosis of PTSD can only be made by a trained clinician using an in-person interview; thus, we use the term "probable PTSD" to indicate physicians with the highest symptom ratings who are likely to have PTSD.

To optimize the categorization of physicians into PTSD groups (low PTSD risk to high PTSD risk) for the predictive analyses, we combined the DSM-5 and PCL-5 scoring criteria, similar to methods suggested by the National Center for PTSD (Lu et al., 2006) . This procedure included using only PCL-5 items rated as 2 (moderately) or higher, which constitutes clinically significant symptom endorsement, and then applying this level of endorsement to the number of DSM-5 items required for each of the four criterion: at least one item in Criterion B (reexperiencing), one in Criterion C (avoidance), two in Criterion D (negative beliefs), and two in Criterion E (hyperarousal) (American Psychiatric Association, 2013) . Table 3 presents the frequency of physicians who endorsed PCL-5 symptoms as 2 or higher. Physicians working directly with COVID-19 patients were designated as frontline physicians and those who were not, as second-line physicians. For modeling analyses, the frontline physicians then were categorized into two groups: a "high risk group" (probable/subclinical PTSD symptoms), and a "low risk group" (none/pre-subclinical PTSD symptoms) (eMethods 1 details PTSD classification).

The Patient Health Questionnaire (PHQ-9) [9 items, 4-level Likert scale: 0-3 score range per item, total score: 0-27] (Kroenke et al., 2001) and the single-item, 5-point burnout scale [1 item, 5-level Likert scale: 1-5 score range] (Dolan et al., 2015) were used to assess the severity of symptoms of depression and burnout, respectively. We categorized depression into five levels based on total PHQ-9 scores (Kroenke et al., 2001) (Davidson, 2020) and, the Brief-COPE Scale [28-item, 4level Likert scale: 1-4 score range per item] (Carver, 1997) . The Brief-COPE scores indicate individuals' negative/positive dominant coping strategies among 14 categories, each scored separately with a range of 2-8. The 14 categories include self-distraction, active coping, denial, substance abuse, use of emotional support, use of instrumental support, behavioral disengagement, venting, positive reframing, planning, humor, acceptance, religion, and selfblame (Carver, 1997) .

Occupational characteristics and COVID-19 specific experiences included living arrangement changes, workload, non-routine work, resource availability, decision-making, exposure rates (e.g., time with COVID-19 patients, intubation/aerosol-generating procedures of suspected/confirmed COVID-19 patients), perceived stigma from treating COVID-19 patients, and turnover intent (switching units/teams, leaving current employer, or leaving healthcare entirely). Perceived organizational and social support was assessed using the 8-item 7-point Survey of Perceived Organizational Support scale [8 items, 7-level Likert scale: 1-7 score range per item] (SPOS; items 1, 3, 7, 9, 17, 21, 23, 27) (Eisenberger et al., 1986) , which was classified into positive support (item numbers: 1, 9, 21, 27) (Eisenberger et al., 1986) and negative support (item numbers: 3, 7, 17, 23) (Eisenberger et al., 1986) for our analysis. Perceived available support from family, friends, and significant others was measured employing the 3-item Multidimensional Scale of Perceived Social Support (MSPSS) (Zimet et al., 1988) .

Demographics included age, sex, ethnicity, race, immigration status, and marital status.

Workplace characteristics included training/years of experience, primary work setting, hospital type, and work setting within hospital.

Independent samples t-tests and Chi-square ( " test) analyses were used to test the PCL-5 scores ( Table 2 ) and frequency of endorsed symptoms (Table 3) between frontline and second-line physicians, respectively. Chi-square analyses ( " test) and -tests were also implemented after obtaining the key variables from the model implementation to determine if the findings were considered significant at 2-tailed ≤ .05.

If complex, nonlinear interplay exists for PTSD risk factors, then conventional, linear models (e.g., logistic regression) would likely fail to capture this phenomenon. Therefore, we trained and tested seven machine learning models alongside logistic regression, including random forest, bootstrap-aggregating (bagging), Naïve Bayes, gradient boosting method, Bayesian additive regression trees, support vector machines, and neural networks. Selection of an optimal predictive model was based on the generalization performance of the models (details in eMethods 2).

Next, to identify the key factors of PTSD risk, we conducted variable importance (VI) analysis, where importance of each PTSD predictor variable was measured using the Gini index . We implemented partial dependency analysis to evaluate the marginal effect of a particular predictor on PTSD risk, while keeping other variables constant Greenwell, 2017) . Quantified marginal effects could indicate an increase/decrease in PTSD risk with increase/decrease in magnitude of the predictor, thus helping to categorize predictors as damaging/protective factors, respectively. All analyses were performed in R (version-3.1) and

RStudio (version-1.1.463). Table 1 presents demographic data and statistical comparisons between frontline and second-line physicians. The groups differed across age, years in practice, sex, primary work setting, current work status, and underlying conditions. Frontline physicians were younger and less experienced than second-line physicians. Sex composition of frontline physicians was similar, whereas second-line skewed female. Most frontline physicians worked in hospitals, group-practice, or academic medical centers, whereas second-line physicians worked in group-practice, academic medical centers, or solo-practice. Lastly, more frontline than second-line physicians were employed full-time, whereas more second-line than frontline physicians worked part-time. 

Overall, 717 frontline physicians and 300 second-line physicians completed the PCL-5. Table 2 presents the PCL-5 data using the full-scale scores of 0 to 4. Section 1 lists the means (SDs) of frontline vs. second-line physicians for each of the 20 PCL-5 items. More frontline than secondline physicians had significantly higher scores for all of the items in Criterion B (reexperiencing) and C (avoidance), four of seven items in Criterion D (negative cognition and mood), and four of six items in Criterion E (heightened arousal). Section 2 of Table 2 lists PCL-5 composite criterion scores and the PCL-5 total score (calculated by summing all PCL-5 item scores) for both groups. Frontline compared to second-line physicians had significantly higher criterion scores and PCL-5 total score. Table 3 lists the number (percentage) of frontline vs. second-line physicians who endorsed each of the PCL-5 items with scores of 2 or higher (considered clinically significant). Chi-square ( " test) analyses indicated that a greater number of frontline than second-line physicians endorsed four items in Criterion B, one item in Criterion C, three items in Criterion D, and two items in Criterion E. Table 4 presents categorization of physicians into four PTSD symptom groups, as described in Section 2.3. The frontline physicians were then divided into two groups for subsequent machine learning analyses (see Table 4 , low risk group and high risk group). For additional confirmation of the difference between the low risk group and the high-risk group, PCL-5 total scores were calculated for each group. Low risk group *** 496 (69.18) High risk group 221 (30.82) * The four groups were determined by combining scoring methods of the DSM-5 and PCL-5 (see eMethods 1) ** Only data for Frontline physicians were used in the machine learning analyses (Low risk group and High risk group). *** Low risk group = No PTSD group and pre-subclinical PTSD group; High risk group = Sub-clinical PTSD group and probable PTSD group

As discussed previously, selection of an optimal predictive model was based on generalization performance of the models . This included assessment of goodness-of-fit (eTable 2) and predictive accuracy (eTable 3). We conducted significance tests of F1 scores (eTable 4) and Area Under the Curve (AUC) measures (eTable 5) across every pairing of predictive models. Ultimately, we selected random forest, given it offered: 1) best goodness-of-fit performance among all the models (eTable 2); 2) highest overall predictive accuracy (accuracy=82.52%, 95% CI, 81.16-83.89; recall=93.11%, 95% CI, 91.92-94.31) (eTable 3, eFigure 2); 3) best prediction performance determined by F1 score (eTable 4); and, 4) more interpretability than its competitors such as the support vector machine and other black-box algorithms (Rustam et al., 2019 ) (eTable 5).

The central aim of this study was to determine which variables best predict PTSD risk among frontline physicians and how these variables relate to higher risk of PTSD, or possible protection from PTSD. Fig. 1 depicts the variable rankings (rank no.1-most important predictor) (see eMethods 3 for details). Table 5 illustrates statistical significance of the top 20 variables indicated in Fig. 1 . For the continuous variables, we used the -test whereas for the ordered variables we used the " test for comparing if the factors were significantly different between the low risk group and high risk group of frontline physicians. Our findings are organized into two categories: 1) damaging variable/risk factors (higher scores associated with higher PTSD risk); and, 2) protective variables (higher scores associated with lower PTSD risk). The relationships were obtained using the partial dependence plots (PDPs) (see eMethods 3 for details). The PDPs of the top 20 variables are illustrated in Fig. 2 . . 1) . PTSD risk increased dramatically even when mild-moderate depressive symptoms were present ( Fig. 2-1) . PTSD risk became significantly prevalent with moderate-high burnout levels (burnout score≥3) ( Fig. 2-2) . Also, two types of fear-fear of contracting COVID-19 ( Fig. 2-4) , fear of transmitting it to loved ones ( Fig. 2-5 )-coincided with higher PTSD risk, presenting a "V-shaped," nonlinear relationship.

Additionally, three coping strategies that were associated with increased PTSD risk include selfblame, venting, and behavioral disengagement. Self-blame ( Fig. 2-3 ) and behavioral disengagement ( Fig. 2-12 ) demonstrated a strong linear correlation with PTSD risk, whereas, venting presented a "V-shaped" curvilinear relationship with PTSD risk (Fig. 2-14) . Three occupational characteristics-increases in job difficulty ( Fig. 2-7) , lack of resources ( Fig. 2-16 ), perceived stigma for working with COVID-19 patients (Fig. 2-10) -were also associated with increases in PTSD risk. Physicians demonstrated resilience to all the challenges until they reached their highest levels. Among demographics, only age influenced PTSD risk, where service beyond 30 years demonstrated a positive association with higher risk of PTSD ( Fig. 2-13 ). Lastly, attrition variables, represented by physician's intention to switch medical units ( Fig. 2-9) , leave their employer (Fig. 2-20) , or leave healthcare entirely ( Fig. 2-6) , were positively associated with increased risk of PTSD. Our data show that frontline physicians with higher risk of PTSD have higher intentions to switch medical units (1 in 2.5 2 ), leave current employer (1 in 2.2), or even leave the healthcare industry altogether (1 in 3.2) compared to their peers with lower risk of PTSD (switch teams=1 in 6.5; leave current employer=1 in 4.75; leave healthcare industry=1 in 10).

Numerous variables appeared to be protective in nature. The degree to which people felt supported by loved ones, such as friends/family (Fig. 2-15 ), significant others ( Fig. 2-19) and

their organization (Fig. 2-8) coincided with lower risk of PTSD (Blekas et al., 2020) . PTSD risk spiked slightly as participants reported higher degrees of organizational support ( Fig. 2-8) .

Participants who received training from employers in psychosocial care reported lower risk of PTSD ( Fig. 2-11 ) compared to their peers who didn't receive any such care or were unsure of receiving such care. Increases in resiliency were associated with drops in PTSD risk ( Fig. 2-17) ;

however, PTSD risk begins to climb at the highest resilience levels. Lastly, Fig. 2-18 suggests that as participants relied more on denial, PTSD risk was higher. red lines indicate the 95% confidence intervals)

Physicians on the frontlines of COVID-19 are in crisis. In our study, among physicians across multiple specialties and states working directly with COVID-19 patients, 10.74% were classified as having probable PTSD and 20.08% as having sub-clinical PTSD, for a combined total of 30.82%. By comparison, in the US general population the lifetime risk of developing PTSD by age 75 is 8.7% and the twelve-month prevalence is 3.5% (American Psychiatric Association, 2013). Table 2 shows frontline physicians had significantly higher PCL-5 item scores and total scores when the full range of scoring was used. These results revealed the symptoms associated with an increased risk of PTSD for frontline physicians, as well as the need to identify the at-risk physicians for interventions. Our results also support the assumption that working with COVID-19 patients meets PTSD Criterion A of the DSM-5 (having experienced a traumatic event).

The findings in Table 2 underscore not only how overall rates of PTSD differ between frontline and second-line physicians, but also how symptoms vary between these groups. The

International Classification of Diseases, 11th Edition (World Health Organization, 2018) in aiming to create an abbreviated symptom assessment for PTSD, selected six items (among the 20 symptoms of PTSD included in DSM-5 and PCL-5) that are the most specific to PTSD (Stein et al., 2014) . These six items include flashbacks and nightmares (Criterion B, Re-experiencing), avoiding memories and avoiding external reminders (Criterion C, Avoidance), and hypervigilance and exaggerated startle response (Criterion D, Hyperarousal). Our analysis indicated that all six of these items differed significantly between frontline and second-line physicians ( Table 2) . Interestingly, three items on which both the physician groups endorsed similar and considerable rates (negative beliefs, difficulty concentrating, and trouble sleeping), are highly correlated with other dysphoric conditions, such as depression and anxiety (Silverstein et al., 2020) . Rather than being specific to PTSD, these three items may instead indicate concurrent stress-related symptoms among physicians, which warrant further examination. Of major consequence, the modeling analysis indicated that the cognitive outcomes such as depression and burnout were the greatest predictors of increased PTSD risk in the physicians working with COVID-19 patients. Previous studies have also shown that PTSD risk among physicians is positively associated with depression and burnout (Jackson et al., 2019; Restauri and Sheridan, 2020) . Although burnout often coincides with depression among physicians (Jackson et al., 2019) , it has been shown to have a different construct (Menon et al., 2020) .

Consistent with this, burnout and depression were independent predictors of PTSD risk in our analyses. In fact, depression, which has long been found to co-occur with PTSD in the aftermath of traumatic events (Gainer et al., 2021; Liebschutz et al., 2007; Pajonk et al., 2012; Shih et al., 2010) , was the greatest predictor of higher risk of PTSD in physicians treating COVID-19 patients in the aftermath of the pandemic. Its significance underscores the need for physicians and healthcare administration to remain vigilant for indicators of depression, potentially engaging in active monitoring of its presence. Unlike with depression, PTSD risk was positively associated with only high burnout levels. In addition, PTSD risk levels were observed to be high when fears of contracting or transmitting COVID-19 were minimal, dipped when these fears were moderate, then dramatically spiked when the fears were great. Although moderate fear appears protective, low and high levels of fear may follow patterns similar to those discussed earlier, with high fear triggering trauma and low fear serving as a proxy for negative coping strategies such as denial.

Although the cross-sectional nature of the data prevents inferring causality, the higher intention of the frontline US physicians with higher PTSD risk to switch units, leave their employer, or leave healthcare completely compared to their peers with lower PTSD risk or even all the physicians in general (Sinsky et al., 2017) not only revealed interplay between damaging and protective factors for PTSD risk, but also invites speculation on the nature of curvilinear relationships between the key factors and PTSD risk. For instance, physicians who vented (complained or processed trauma experiences with others) the least are the ones who suffered from higher PTSD risk, while those who vented a moderate amount presented lower PTSD risk. We speculate that minimal venting may forego the benefits of externally processing the traumatizing events and provoke trauma-induced symptoms (Bodie and Burleson, 2008; Mallinckrodt et al., 2012) , whereas high levels of venting are a proxy for turmoil, a phenomenon found in other high stress-contexts (Coyne and DeLongis, 1986; Mallinckrodt et al., 2012) . Similarly, increased institutional support largely decreased the risk of PTSD; however, our modeling reveals a spike in the PTSD risk at the highest degrees of organizational support. As with venting, the highest levels of organizational support may be provided for those, who are in the greatest need. Lastly, the spike in PTSD risk at the highest levels of resilience may come from physicians coping via a form of denial-another factor within the top 20 PTSD risk predictors. Understanding and exploring such nuances will further inform both theory and practice, helping support the frontline physicians through such crises.

Among demographics, only age was found to be a key predictor of PTSD risk. Previous research suggests younger physicians are less resilient to COVID-related trauma Lai et al., 2020; Lancee et al., 2008; Sim et al., 2004; Su et al., 2007; Wu et al., 2009 ); however, the extant literature argues trauma is additive, suggesting older physicians would be more vulnerable following COVID-related trauma exposure (Ogle et al., 2014) . Our findings offer support for both bodies of work, with the younger suffering the most, but PTSD risk markedly increasing among the older physicians.

However, there are certain limitations of our study. It is to be noted that the crosssectional nature of these analyses prevents us from drawing conclusions about the causal relationship between predictor variables and the PTSD risk. In the future, longitudinal studies that bolster participant uptake are needed to confirm and expound upon these findings.

Additionally, the study provoked a relatively low response rate, which is common for unincentivized, voluntary surveys.

In summary, our study identified some of the key factors associated with higher risk of PTSD among practicing physicians in the US due to the prolonged impact of the COVID-19

pandemic. Identifying key intervention variables can help stakeholders develop the means to proactively support individuals at higher PTSD risk. Thus, either through mitigating damaging variables and/or bolstering protective variables, this research provides both physicians and their institutions useful information to defend against the trauma-related threats to physicians resulting from the current and possible future epidemics/pandemics. The stability and future of the US healthcare system may depend on it.

The authors report no conflict of interest.

The study was supported by the 2019-20 SUNY Research Seed Grant Program RFP #20-03-COVID, Proposal ID # COVID202060.

The 

The different groups of PTSD risk were created by using a combination of the PCL-5 score and DSM criteria. This produced four levels of PTSD symptom severity: (1) "Probable PTSD,"

scoring 2 or higher in three or more DSM-5 categories and a total PCL-5 score >=33; (2) "Subclinical PTSD," scoring 2 or higher in two categories and a PCL-5 score >=12 and <=33

(three respondents had a score of 34 or 35, but in less than 4 categories); (3) "Pre-subclinical,"

scoring 2 or higher in none or one category and a PCL-5 score >=12 and <33; and (4) "No PTSD," a PCL-5 total score <12, irrespective of DSM-5 item scores.

In the study, we leveraged eight models to predict the likelihood of developing PTSD symptoms, including the traditionally-used parametric Logistic regression (Glonek and McCullagh, 1995; Hosmer Jr et al., 2013) , nonparametric tree-based models (i.e., random forest , bagging (James et al., 2014) , gradient boosting method (Friedman, 2002) , Bayesian additive regression trees (Chipman et al., 2008) ), and black-box methods (i.e., Naïve Bayes (Rish, 2001) , support vector machines (Gunn, 1998; Hearst et al., 1998; Steinwart and Christmann, 2008) , neural networks (Nielsen, 2015) ). Machine learning algorithms have been gaining more attention in the field of public health recently Mooney and Pejaver, 2018) . Compared with the conventional linear statistical models, the major advantages of applying machine learning models are: 1) the ability to capture the underlying interdependent and nonlinear relationships of the data (Beam and Kohane, 2018; Mooney and Pejaver, 2018) ; 2) capable of discover specific patterns and trends that could be unknown to humans, and producing the strong predictive ability (Char et al., 2018) .

Logistic regression (Glonek and McCullagh, 1995; Hosmer Jr et al., 2013 ) is a parametric model that assumes a linear relationship between the feature vector 0 1 = ( 05 , 0" , … , 08 ) and the logodds of the response variable being a success or a failure. In this study, the response variable is representative of the likelihood of developing the PTSD symptoms among physicians and can be treated as a binary categorical variable, i.e., ∈ {+1, −1}. Let Pr( 0 = 1| 0 , ) = 0 . And, the mathematical formula is given by

where the sigmoid function is the inverse of the logit function.

Random forest ) is a non-parametric ensemble tree-based model that aggregating bootstrapped decision trees denoted as V . By constructing a set of single decision trees, random forest takes the average of every output generated by each tree, which usually yields the better predictive accuracy than the single decision tree model. The formulation is denoted as

Random forest randomly selects a subset of the predictor variables that can potentially avoid the issue of overfitting the data, but this model is sensitive to outliers.

Bootstrap aggregation or Bagging (James et al., 2014) for short is another ensemble method that used decision tree-based classifiers to reduce the variance. different bootstrapped training data are generated by sampling from the training data with replacement. Each of the training set is used to develop the single classification tree, and in total different classifiers are constructed. Each constructed tree is considered weak learner, and is independent of each other. Then, the test data is used to validate each single tree model among trees, and the final output is determined by the majority vote across all trees for the classification problem. The major advantage of Bagging is to reduce the variance and it can provide an unbiased estimate of the test error (a.k.a. out-of-bag error).

Boosting is also the ensemble technique that the predictors are made sequentially, and are dependent upon previous trees, whereas a random forest is a collection of decision trees that are built in parallel. Gradient boosting method (Friedman, 2002) is another popular machine learning technique for both regression and classification problems. In GBM, a loss function is typically defined as squared error in the regression problem and logarithmic loss in the classification problem. Each decision tree is considered as weak learner that fitted to a smaller number of terminal nodes. The first tree model is fitted using the small data, and the second tree is developed using the residuals from the first tree that attempts to reduce the error. This process is being implemented through a sequence of trees, and during each iteration, gradient descent is used to minimize the loss function when sequentially adding more trees.

Bayesian additive regression tree (BART) (Chipman et al., 2008 ) is a non-parametric, treeensemble method, where a set of single tree models are aggregated to approximate the outcome variable denoted by . Generally speaking, the BART model has two main parts: a sum-of-tree model and a regularization prior. The single regression tree is constructed by partitioning the space of a set of predictors into different sub-regions that are representative of nodes, where the decision rules (in this study, the classification rules) are determined by the sequence of splits from root node to leaf node. Mathematically, the BART can be given by the sum of regression trees, that is,

where ] ; L , L` assigns the parameters 0L ∈ L of the regression tree L to the predictor . We applied probit function to map a set of predictor variables into a binary probability denoting the likelihood of developing PTSD symptoms among healthcare physicians. The regularization prior term is leveraged to control the complexity of tree's structure, balancing between computationally cost and the accuracy of the function approximation.

Naïve Bayes (Rish, 2001) is grounded in the Bayes' theorem with an assumption of independence among a set of predictors. Bayesian classification provides prior knowledge in probabilistic model to capture uncertainty about the model in predicting the outcome response. In the classification problem, we aim to estimate the function that maps from to , or equivalently ( | ). By applying the Bayes rule, ( = 0 | ) can be given by

Where a indicates the m-th possible class for outcome variable , and c is the k-th predictor variable. The final response label is denoted as

.

Support vector machine (SVM) (Gunn, 1998; Hearst et al., 1998; Steinwart and Christmann, 2008) is a generalization the maximal margin classifier which essentially maximizes the soft margin. It essentially extends the support vector classifier by using a larger feature space using higher order polynomials of the predictors. To introduce non-linearity a smarter way is to use kernels. SVM is known to be robust to the outliners and computationally efficient when the number of feature dimensions is greater than the number of total data points.

Neural networks (Nielsen, 2015) are the widely-used machine learning models that basically simulate the nature of the biological neurons. Typically, it has three common layers for data processing: input layer, hidden layer and output layer. Data are first passed through the input layer.

Hidden layer is used for intermediate processing, and translates the weights from the input layer to the next layer (it could be either another hidden layer or output layer). Using an activation function to map the information from the hidden layer to the final output layer, the predictive values could be obtained. The activation function is used to incorporate the non-linearity of the data. We used feedforward neural network to develop models for the training set.

To achieve optimal generalization performance and select the most robust predictive model (i.e., the model which accurately predicts without overfitting the data), the complexity of a statistical model should be controlled through bias-variance trade-off using the most widely used cross validation technique . More specifically, predictive accuracy of each model was calculated by implementing 30-fold 3 random hold-out validation tests where in each iteration, 20% of the data was randomly held out and the model was trained

with the remaining data and tested using the held-out sample . To evaluate the performance of the classification models, we leveraged the widely-used statistical metrics based on the Confusion matrix, such as F1 score, Area Under the Curve (AUC, or C-statistic), Accuracy (%), Recall (%) and Precision (%), for both goodness-of-fit and predictive accuracy (Forman and Scholz, 2010) . F1 score computed by the harmonic mean of Precision and Recall was applied to balance between positive predictive value and the true positive rate. AUC was calculated by the area under the receiver operating characteristic (ROC)

curve. F1 score and AUC, which are mainly used in unbalanced datasets, were leveraged in this study to evaluate models' performance (Forman and Scholz, 2010) . Finally, the model that outperformed all the other models in terms of goodness-of-fit and predictive accuracy was 3 Given that cross-validation is based on randomized sub-sectioning of the data into 80-20% portions, 30 times repetition is naturally a conservative measure to ensure all the values have been used at least once. selected as the final predictive model for statistical inferencing. This is a well-known statistical process that ensures that the machine learning models do not overfit the data and provide accurate predictive performance, while providing interpretability benefits .

Although the non-parametric models outperform parametric models in terms of predictive performance, the improved predictability comes at the cost of reduced interpretability. However, statistical inferencing can be conducted for the non-parametric models using the variable importance ranking and partial dependence plots (PDPs) . 

Diagnostic and Statistical Manual of Mental Disorders

The psychological well-being of physicians during COVID-19 outbreak in Oman

COVID-19: PTSD symptoms in Greek health care professionals

Explaining variations in the effects of supportive messages a dual-process framework

Random Forests

PTSD symptoms in healthcare workers facing the three coronavirus outbreaks: What can we expect after the COVID-19 pandemic

You want to measure coping but your protocol'too long: Consider the brief cope

Cases of Coronavirus Disease (COVID-19) in the U.S

Psychological impact of the 2003 severe acute respiratory syndrome outbreak on health care workers in a medium size regional general hospital in Singapore

Psychological impact of severe acute respiratory syndrome on health workers in a tertiary hospital

Coping with staff burnout and work-related posttraumatic stress in intensive care

Going beyond social support: the role of social relationships in adaptation

Connor-Davidson Resilience Scale (CD-RISC) Manual

Factors affecting mental health of health care workers during coronavirus disease outbreaks (SARS, MERS & COVID-19): A rapid systematic review

Using a single item to measure burnout in primary care staff: a psychometric evaluation

Perceived organizational support

Depression, anxiety, stress levels of physicians and associated factors in Covid-19 pandemics

Association Between Proportion of Workday Treating COVID-19 and Depression, Anxiety, and PTSD Outcomes in US Physicians

General hospital staff worries, perceived sufficiency of information and associated psychological distress during the A/H1N1 influenza pandemic

Posttraumatic stress disorder and completed suicide

pdp: An R Package for Constructing Partial Dependence

The impact of the Paris terrorist attacks on the mental health of resident physicians

The mediating effects of coping strategies on the relationship between secondary traumatic stress and burnout in professional caregivers in the UAE

The Elements of Statistical Learning, Second

Factors associated with motivation and hesitation to work among health professionals during a public crisis: a cross sectional study of hospital workers in Japan during the pandemic (H1N1)

The crossroads of posttraumatic stress disorder and physician burnout: a national review of United States trauma and nontrauma surgeons

An Introduction to Statistical Learningwith Applications in R

Outbreak evolution for the current most affected countries

Maps Trends New COVID-19 Cases Worldw

Assessing the presence of post-traumatic stress and turnover intention among nurses post-Middle East respiratory syndrome outbreak: the importance of supervisor support. Workplace Health Saf

Impact on mental health and perceptions of psychological care among medical

Preliminary Report: US Physician Stress During the Early Days of the COVID-19 Pandemic

Machine learning for predicting outcomes in trauma

Evaluation of machine-learning algorithms for predicting opioid overdose risk among medicare beneficiaries with opioid prescriptions

The mental health of hospital workers dealing with severe acute respiratory syndrome

A threshold model of social support, adjustment, and distress after breast cancer treatment

Posttraumatic stress and stress disorders during the COVID-19 pandemic: Survey of emergency physicians

The immediate psychological and occupational impact of the 2003 SARS outbreak in a teaching hospital

comparative assessment of socio-environmental factors using a data-driven predictive approach. arXiv Prepr

Psychosocial effects of SARS on hospital staff: survey of a large tertiary care institution

Cumulative exposure to traumatic events in older adults

Trauma and stressrelated disorders in German emergency physicians: the predictive role of personality factors

Machine learning approaches to predict 6-month mortality among patients with cancer

Burnout and posttraumatic stress disorder in the coronavirus disease 2019 (COVID-19) pandemic: intersection, impact, and interventions

Persistence of Traumatic Memories in World War II Prisoners of War

Random-Forest (RF) and Support Vector Machine (SVM) Implementation for Analysis of Gene Expression Data in Chronic Kidney Disease (CKD)

Prevalence of PTSD and major depression following trauma-center hospitalization

An item response theory analysis of the PTSD checklist for DSM-5: Implications for DSM-5 and ICD-11

Severe acute respiratory syndromerelated psychiatric and posttraumatic morbidities and coping responses in medical staff within a primary health care setting in Singapore

Professional satisfaction and the career plans of US physicians

DSM-5 and ICD-11 definitions of posttraumatic stress disorder: Investigating "narrow" and "broad" approaches

Prevalence of psychiatric morbidity and psychological adaptation of the nurses in a structured SARS caring unit during outbreak: a prospective and periodic assessment study in Taiwan

The ptsd checklist for dsm-5 (pcl-5). Scale available from Natl

Health-behaviors associated with the growing risk of adolescent suicide attempts: A data-driven cross-sectional study

Physician burnout: contributors, consequences and solutions

Physician resilience: a grounded theory study of obstetrics and gynaecology residents

International classification of diseases for mortality and morbidity statistics

Predicting posttraumatic stress disorder risk: a machine learning approach

The psychological impact of the SARS epidemic on hospital employees in China: exposure, risk perception, and altruistic acceptance of risk

Physician stress and burnout

The multidimensional scale of perceived social support

Big data and machine learning in health care

Random Forests

Implementing machine learning in health careaddressing ethical challenges

BART : Bayesian Additive Regression Trees

Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement

Stochastic gradient boosting

Multivariate logistic models

Support vector machines for classification and regression

The Elements of Statistical Learning, Second

Support vector machines

Applied logistic regression

An Introduction to Statistical Learning: With Applications in R

An Introduction to Statistical Learningwith Applications in R

Evaluation of machine-learning algorithms for predicting opioid overdose risk among medicare beneficiaries with opioid prescriptions

Big data in public health: terminology, machine learning, and privacy

Neural networks and deep learning

An empirical study of the naive Bayes classifier

Support vector machines