key: cord-0852866-hthgkoew
authors: Huang, Rui-Wen; Shen, Tao; Ge, Lei-Ming; Cao, Lu; Luo, Jian-Feng; Wu, Shi-Yu
title: Psychometric Properties of the Chinese Version of the Primary Care Post-Traumatic Stress Disorder Screen-5 for Medical Staff Exposed to the COVID-19 Pandemic
date: 2021-09-02
journal: Psychol Res Behav Manag
DOI: 10.2147/prbm.s329380
sha: 70df93b87d5ccbdf03511a4700513076a79a4b07
doc_id: 852866
cord_uid: hthgkoew

PURPOSE: The COVID-19 pandemic may increase the development of psychiatric disorders, such as posttraumatic stress disorder (PTSD) among medical staff. A brief validated screening tool is essential for the early diagnosis of PTSD. The purpose of the present study was to evaluate the validation of a Chinese version of the Primary Care-PTSD-5 (C-PC-PTSD-5) and determine an appropriate cutoff score with optimal sensitivity and specificity for medical staff in China during the COVID-19 pandemic. PARTICIPANTS AND METHODS: An online cross-sectional survey was conducted on medical staff (n = 1104) from 17 medical institutions in Shanghai. Questionnaires comprising general information, medical-related traumatic event experiences, the PTSD Checklist (PCL-5), and C-PC-PTSD-5 were distributed to participants using the online Questionnaire Star electronic system. Internal consistency, convergent validity, and test–retest reliability were calculated. Receiver operating characteristic (ROC) analysis was performed to determine diagnostic accuracy and the optimal cutoff score of the C-PC-PTSD-5 for medical staff. RESULTS: We included 1062 valid questionnaires for the analysis. Data of 838 traumatic experiences were analyzed. Internal consistency of the C-PC-PTSD-5 was satisfied (Cronbach’s α = 0.756). The total score of the C-PC-PTSD-5 showed good test–retest reliability (r = 0.746). We found a strong correlation between the C-PC-PTSD-5 score and PCL-5 total score (r = 0.669, p < 0.001), which indicated good convergent validity. The ROC analysis showed an area under the curve of 0.81 ± 0.016. A cutoff score of 2 provided optimal sensitivity and specificity for the C-PC-PTSD-5 (sensitivity = 0.632, specificity = 0.871, Youden index = 0.503, and overall efficiency = 0.768). CONCLUSION: Our results indicated that the C-PC-PTSD-5 can be employed as a brief and efficient screening instrument for medical staff exposed to the COVID-19 pandemic. A score of 2 was identified as the optimal threshold for probable clinical PTSD symptoms.

The outbreak of COVID-19 pandemic has resulted in various mental health problems among the public in China since December 2019. In the fight against the COVID-19 pandemic, medical healthcare workers in China have faced enormous pressure, which has included a high risk of infection, overworking, and social isolation. 1 These stressors may trigger various mental disorders, such as anxiety and depressive disorders and posttraumatic stress disorder (PTSD), 2 which in turn, could result in hazards that exceed the consequences of the COVID-19 epidemic itself. 3 Because PTSD is a highly prevalent condition with a significant economic burden, early detection of symptoms in medical staff is essential. 4 Efforts have been made to improve early detection and diagnosis accuracy of PTSD through the development of self-reported screening instruments. The PTSD Checklist (PCL) is one of the most widely applied tools for assessing PTSD in clinical and research settings. 5 Recently, the PCL was updated to PCL-5, which contains 20 items that are rated on a five-point Likert-type scale. The items are based on the Diagnostic and Statistical Manual of Mental Disorder version 5 (DSM-5) criteria for PTSD. 6 In addition to the original English version of the PCL-5, Swedish, 7 Chinese, 8 and German versions 9 of the PCL-5 have also been developed.

The Primary Care Posttraumatic Stress Disorder Screen (PC-PTSD) is a four-item measure that is short and easy to administer, 10 and despite its brevity, it has shown to be equivalent to other longer screening tools. 11, 12 In 2013, the PC-PTSD was revised to the PC-PTSD-5, according to the DSM-5 criteria for PTSD. 13 The PC-PTSD-5 comprises five "Yes" or "No" items for re-experiencing, avoidance, hyperarousal, numbing, and alteration in mood and cognition, which correspond to the symptom clusters of the DSM-5 criteria for PTSD. One of the purposes of the PC-PTSD-5 is to screen individuals for PTSD and make a provisional PTSD diagnosis. The diagnostic accuracy of the PC-PTSD-5 was initially demonstrated in a veteran sample. 14 The prevalence of probable PTSD in medical staff is approximately three times that of the general population, which is attributed to the long hours and work-related stressors. 15, 16 In particular, emergency department staff were found to exhibit more severe PTSD symptoms than general staff in psychiatric wards during previous pandemics. 17 It is essential to explore the psychometric properties of the Chinese version of the PC-PTSD-5 (C-PC-PTSD -5) before it is implemented for the identification of probable medical-related PTSD among medical staff.

The objective of present study was to evaluate the validation of the C-PC-PTSD-5 and determine an appropriate cutoff score with optimal sensitivity and specificity for medical staff in China during the COVID-19 pandemic.

The survey was carried out from December 6, 2020, to December 24, 2020, in Shanghai approximately 1 year following the outbreak of COVID-19 in China. The study was conducted in accordance with the Declaration of Helsinki and was approved by the ethics committee of Ruijin Hospital/Lu Wan Branch, School of Medicine, Shanghai Jiaotong University. Fifteen hospitals and two centers for disease control (CDCs) across five districts of Shanghai were selected as research sites using the multistage stratified random cluster sampling method. With the help of the labor union of selected hospitals and CDCs, questionnaires were distributed to participants using the online Questionnaire Star electronic system (Changsha Ranxing IT Ltd. Corporation, https://www.wjx.cn/). All participants have signed an informed consent form on the first page of the questionnaire, and used WeChat or the webpage to complete the questionnaires. The inclusion criteria for subjects were those working in a hospital or a CDC, which included doctors, nurses, allied healthcare workers (pharmacists, physiotherapists, and occupational therapists), technicians, administrators, clerical staff, and maintenance workers. To ensure data quality, the same IP address could only be used once.

The study questionnaire was written in Chinese and comprised four main components: General information, medical-related traumatic experiences, the PCL-5, and the C-PC-PTSD-5.

General subject data collected included sex, age, marital status, type of medical institution (hospital or CDC), years working, average hours worked per week, and monthly salary.

The checklist for traumatic events in the questionnaire comprised nine typical medical-related traumatic experiences. A free-text "other" option was provided for participants to describe traumatic events experienced that were not included in the checklist. Participants indicated whether they had experienced any of the nine listed traumatic events and when they had occurred. Participants who had experienced at least one traumatic experience were asked to complete both the PCL-5 and PC-PTSD-5. If they had not experienced a traumatic experience, the survey was terminated and participants were excluded from the validation test of the C-PC-PTSD-5.

The PCL-5 is a 20-item self-report measure to assess the 20 DSM-5 symptoms of PTSD. The PCL-5 is applied as a Likert scale, and the severity of each symptom is divided into five levels from "Not at all" = 0, to "A little bit" = 1, "Moderately = 2", "Quite a bit" = 3, and "Extremely" = 4. The maximum score is 80.

The PCL-5 is scored using two methods. First, a total score of between 31-33 indicates probable PTSD. Second, a provisional PTSD diagnosis can be made by treating each item rated as 2 or higher as a symptom endorsed and following the DSM-5 diagnostic rule, which requires at least: one B item (questions 1-5), one C item (questions 6-7), two D items (questions [8] [9] [10] [11] [12] [13] [14] , or two E items (questions [15] [16] [17] [18] [19] [20] . In the present study, a cutoff score of 33 points was used as a diagnosis of PTSD. 9 Chinese Version of the PC-PTSD-5

The C-PC-PTSD-5 was translated by a group of psychiatrists and psychologists. It was adapted using a two-stage process of translation and reverse translation. First, we invited a bilingual psychiatrist to translate the PC-PTSD -5 into Chinese, which was then translated back into English by a bilingual psychologist. Discrepancies were resolved and corrected until the translated version was considered adequate.

To analyze for test-retest reliability of the C-PC-PTSD-5, 30 medical health workers were invited to complete the online questionnaires before the cross-sectional survey, and those who reported medical-related traumatic experiences were required to be re-assessed 3 weeks after the initial assessment.

The results of the survey administered using the online Questionnaires Star were exported as an Microsoft Office Excel 2007 data table A total of 1104 questionnaires from 15 hospitals and two CDCs in Shanghai were collected, and 42 questionnaires were excluded because of incomplete data (18 questionnaires) or response times of less than 90 s (24 questionnaires). The valid response rate was 96.19% as 1062 valid questionnaires were included. In addition, 224 participants who answered "No" to all nine medical-related traumatic experiences were excluded for the validation test of the C-PC-PTSD-5, which resulted in a final sample size of 838 ( Figure 1 ).

All statistical analyses were performed using IBM SPSS Statistics for Windows version 25.0 (Armonk, NY). Categorical variables are expressed as absolute values (percentages) and continuous variables are expressed as means ± standard deviations (SDs). Test-retest reliability of the C-PC-PTSD-5 was assessed using Spearman correlation and a paired t-test. Internal consistency of the C-PC-PTSD-5 total score was evaluated using Cronbach's alpha. Sensitivity, specificity, Youden index, and overall efficiency were used to quantify the diagnostic utility of the C-PC-PTSD-5. A receiver operating characteristic (ROC) analysis was performed to determine the optimum cutoff value by comparing the patterns of sensitivity and specificity with those of the PCL-5 as the standard.

Data from 838 valid questionnaires were used for the analysis. Baseline characteristics, PCL-5 scores, and PC-PTSD-5 scores of the 838 participants are summarized in Table 1 . Participants had an average total score of 34.464 for the PCL-5, and 359 (33.8%) participants met or exceeded the recommended cutoff score of 33 for a standard PTSD diagnosis.

Internal Consistency, Convergent Validity, and Test-Retest Reliability of the C-PC-PTSD-5 We found a strong correlation between the C-PC-PTSD -5 score and PCL-5 total score (r = 0.669, p < 0.001), which indicated good convergent validity.

For test-retest reliability, 24 participants were reassessed using the C-PC-PTSD-5 2 weeks after the initial assessment. The total score of the C-PC-PTSD-5 showed good test-retest reliability (r = 0.746). The paired t-test revealed no significant difference in scores between the assessment times (Time 1: Total score [mean ± SD] = 1.333 ± 2.060; Time 2: Total score [mean ± SD] = 0.916 ± 1.782, t = −0.959, p = 0.358), which indicated good consistency across both assessment times.

Receiver operating characteristic (ROC) analyses were performed to calculate the sensitivity and specificity of the C-PC-PTSD-5 total score when compared with the PCL-5 score's PTSD diagnosis cutoff value of 33 (Figure 2 ). The analysis was performed to establish the optimal cutoff score for the C-PC-PTSD-5 for medical staff who had experienced traumatic events and was determined by calculating the maximum area under the curve (AUC). As shown in Figure 2 , the AUC of the C-PC-PTSD-5 was 0.81 ± 0.016, which indicated that the C-PC-PTSD-5 has a high degree of discriminatory power (95% CI: 0.779, 0.841).

When examining the cutoff of a score of 2 for the C-PC-PTSD-5, diagnostic utility was considered more acceptable (sensitivity = 0.632, specificity = 0.871, Youden index = 0.503, and overall efficiency = 0.768) than that of a cutoff of 1 (sensitivity = 0.799, specificity = 0.689, Youden index = 0.488, and overall efficiency = 0.736) or 3 (sensitivity = 0.454, specificity = 0.958, Youden index = 0.412, and overall efficiency = 0.742; Table 2 ).

This cross-sectional survey in a large sample of participants who had experienced traumatic events indicated that the C-PC-PTSD-5 (a Chinese version of the PC-PTSD-5 that is designed to reflect the new DSM-5 PTSD criteria) is a valid and reliable screening tool for probable PTSD for medical staff exposed to the COVID-19 pandemic. The diagnostic utility analysis of the C-PC-PTSD-5 identified a recommended cutoff score of 2, based on an optimal balance of sensitivity and specificity.

The current results need to be interpreted with consideration of several limitations. First, because clinical interviews in a large sample were not approved in light of the COVID-19 pandemic in China, an online electronic PCL-5 was employed as the standard in the present study, which likely decreased model accuracy. The PCL-5 is currently used globally and has been found to have highly reliable psychometric properties across various settings. 9, [18] [19] [20] In addition, the Chinese version of the PCL-5 has shown good validity and has been used to screen for probable PTSD in earthquake survivors, 8, 21 and has also been previously implemented as a control to test the psychometric properties of the PC-PTSD-5 in Chinese children. 22 Further research comparing the PCL-5 with structured clinical interviews, such as the Clinician-Administered PTSD Scale for DSM-5, is needed. Second, most study participants were physicians, nurses, or technicians (N = 1062, 99.06%), and only 10 labor workers completed the questionnaires. This was due to unfamiliarity or reluctance to complete the electronic questionnaire on a mobile phone. Moreover, socioeconomic status and education level may have influenced our results. Third, 1 year following the COVID-19 outbreak, the spread of virus and infections had been controlled in the Shanghai region, and most participants had not been in direct contact with COVID-19-infected patients. Therefore, our findings may not generalize to medical staff in severely affected areas. 

Various studies have reported an increased risk of psychopathologies and stress-related disorders among medical staff, as well as a high rate of PTSD symptomatology. [23] [24] [25] [26] In the present study, 359 (33.8%) healthcare workers met the PCL-5 standard for a PTSD diagnosis, which is higher than that observed in recent research. 27 Specific sample characteristics may play a role in this discrepancy. Healthcare workers have a relatively high level of education, which has been shown to be a significant predictor of PTSD. 28 In fact, the mental health of healthcare workers during the Severe Acute Respiratory Syndromes (SARS) and Middle East Respiratory Syndrome (MERS) epidemic was found to be poorer in the phase following the acute outbreak than in the initial phase. 29 PTSD symptoms usually appear months or years after the traumatic experience, and it may increase along with the pandemic. The detection of probable PTSD symptoms in medical staff is essential, even several years after experiencing a traumatic event.

Although we did not compare the time taken by participants to complete the PCL-5 and C-PC-PTSD-5, it is clear that the C-PC-PTSD-5 can be completed more quickly than the PCL-5. Participants are required to respond with a "yes" or "no" to only five items compared with the 20 items of the PCL-5, which has five options for each item. We randomly interviewed several healthcare workers who were not involved in the survey and found that a brief questionnaire was more popular and suitable for large samples in medical institutions in light of the COVID-19 pandemic.

Our findings indicated that the C-PC-PTSD-5 has good psychometric properties and the contextually validated cutoff score provides an effective screening instrument for identifying medical staff with PTSD symptoms. A cutoff score of 2 showed an optimal balance of sensitivity and specificity, although a cutoff of 1 revealed the highest sensitivity (0.799). Sensitivity is more important than other diagnostic accuracy measures in primary care settings, especially in non-treatment-seeking medical staff exposed to the COVID-19 pandemic. Participants who score over 1 should be recommended for further assessment for PTSD using a clinical interview.

The present findings demonstrate that the C-PC-PTSD-5 possesses good psychometric properties and can be used as a reliable, valid, and time-saving tool to diagnose and assess PTSD. A score of 2 was identified as a threshold for probable medical-related PTSD symptoms. The Chinese C-PC-PTSD-5 will help identify medical staff with PTSD who may benefit from early intervention, which will facilitate the delivery of mental health care. 

The mental health of medical workers in Wuhan, China dealing with the 2019 novel coronavirus

The 2014 Ebola outbreak and mental health: current status and recommended response

2019-nCoV epidemic: address mental health care to empower society

Screening for posttraumatic stress disorder in civilian substance use disorder patients: cross-validation of the Jellinek-PTSD screening questionnaire

Posttraumatic Stress Disorder Checklist. PTSD Checkl Reliab validity, diagnostic Util. The 9th Annual Conference of the ISTSS

The PTSD Checklist for DSM-5 (PCL-5)

Psychometric properties of the PTSD Checklist for DSM-5: a pilot study

The underlying dimensions of DSM-5 posttraumatic stress disorder symptoms in an epidemiological sample of Chinese earthquake survivors

The German version of the Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): psychometric properties and diagnostic utility

The primary care PTSD screen (PC-PTSD): deveJopment and operating characteristics

Efficiency of a four-item posttraumatic stress disorder screen in trauma patients

Validation of the Primary Care Posttraumatic Stress Disorder screening questionnaire (PC-PTSD) in civilian substance use disorder patients

The American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)". Guidelines

The Primary Care PTSD Screen for DSM-5 (PC-PTSD-5): development and Evaluation Within a Veteran Primary Care Sample

PTSD and surgical residents: everybody hurts sometimes

The Crossroads of Posttraumatic Stress Disorder and Physician Burnout: a National Review of United States Trauma and Nontrauma Surgeons

Psychological impact of the 2015 MERS outbreak on hospital workers and quarantined hemodialysis patients

Psychometric properties of the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (PCL-5) in veterans

Validation of the posttraumatic stress disorder checklist -5 (PCL-5) in a primary care population with high HIV prevalence in Zimbabwe

The validity of Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5) as screening instrument with Kurdish and Arab displaced populations living in the Kurdistan region of Iraq

Assessing the underlying dimensionality of DSM-5 PTSD symptoms in Chinese adolescents surviving the 2008 Wenchuan earthquake

Applications of the Chinese version of the primary care PTSD screen for DSM-5 (PC-PTSD-5) for children

Psychological Distress and Post-Traumatic Symptomatology Among Dental Healthcare Workers in Russia: results of a Pilot Study

COVID-19 Pandemic in the Italian Population: validation of a Post-Traumatic Stress Disorder Questionnaire and Prevalence of PTSD Symptomatology

A multinational, multicentre study on the psychological outcomes and associated physical symptoms amongst healthcare workers during COVID-19 outbreak

Occupational burnout syndrome and post-traumatic stress among healthcare professionals during the novel coronavirus disease

PTSD symptoms in healthcare workers facing the three coronavirus outbreaks: what can we expect after the COVID-19 pandemic

The presence of psychological trauma symptoms in resuscitation providers and an exploration of debriefing practices

The Prediction of Levels of Posttraumatic Stress Levels by Depression Among Veterans with Disabilities

We would like to thank all participants for their considerable support and assistance.

This work was supported by grants from Shanghai hospital association management research fund(Q2020027) and Shanghai "Rising Stars of Medical Talent" Youth Development Program (Shanghai Municipal Health Commission 2019-72).

The authors report no conflicts of interest in this work.

Psychology Research and Behavior Management is an international, peer-reviewed, open access journal focusing on the science of psychology and its application in behavior management to develop improved outcomes in the clinical, educational, sports and business arenas. Specific topics covered in the journal include: Neuroscience, memory and decision making; Behavior modification and management; Clinical applications; Business and sports performance management; Social and developmental studies; Animal studies. The manuscript management system is completely online and includes a very quick and fair peer-review system, which is all easy to use. Visit http://www. dovepress.com/testimonials.php to read real quotes from published authors.