key: cord-0801407-0e5olivo
authors: Hermans, Joep J. R.; Groen, Joost; Zwets, Egon; Boxma-De Klerk, Bianca M.; Van Werkhoven, Jacob M.; Ong, David S. Y.; Hanselaar, Wessel E. J. J.; Waals-Prinzen, Lenneke; Brown, Vanessa
title: Chest CT for triage during COVID-19 on the emergency department: myth or truth?
date: 2020-07-20
journal: Emerg Radiol
DOI: 10.1007/s10140-020-01821-1
sha: cd36779d7d28af9481aea5c8c5b4089bed74e190
doc_id: 801407
cord_uid: 0e5olivo

ABSTRACT: PURPOSE: We aimed to investigate the diagnostic performance of chest CT compared with first RT-PCR results in adult patients suspected of COVID-19 infection in an ED setting. We also constructed a predictive machine learning model based on chest CT and additional data to improve the diagnostic accuracy of chest CT. METHODS: This study’s cohort consisted of 319 patients who underwent chest CT and RT-PCR testing at the ED. Patient characteristics, demographics, symptoms, vital signs, laboratory tests, and chest CT results (CO-RADS) were collected. With first RT-PCR as reference standard, the diagnostic performance of chest CT using the CO-RADS score was assessed. Additionally, a predictive machine learning model was constructed using logistic regression. RESULTS: Chest CT, with first RT-PCR as a reference, had a sensitivity, specificity, PPV, and NPV of 90.2%, 88.2%, 84.5%, and 92.7%, respectively. The prediction model with CO-RADS, ferritin, leucocyte count, CK, days of complaints, and diarrhea as predictors had a sensitivity, specificity, PPV, and NPV of 89.3%, 93.4%, 90.8%, and 92.3%, respectively. CONCLUSION: Chest CT, using the CO-RADS scoring system, is a sensitive and specific method that can aid in the diagnosis of COVID-19, especially if RT-PCR tests are scarce during an outbreak. Combining a predictive machine learning model could further improve the accuracy of diagnostic chest CT for COVID-19. Further candidate predictors should be analyzed to improve our model. However, RT-PCR should remain the primary standard of testing as up to 9% of RT-PCR positive patients are not diagnosed by chest CT or our machine learning model. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s10140-020-01821-1) contains supplementary material, which is available to authorized users.

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), the virus which causes coronavirus disease 2019 (COVID- 19) , was first reported in Wuhan, China, in December 2019 [1] . Since then, the virus spread around the world and on February 27, the first patient with COVID-19 was identified in the Netherlands. As of the 1 May, 39,791 Dutch patients tested positive for COVID-19, 10,854 patients were admitted to hospitals, and 4893 patients died due to SARS-CoV-2 [2] . Although the Dutch government enforced measurements to enter a mitigation phase, to prevent a peak of demand which could exceed the capacity of hospital treatment, increasing numbers of patients visited the emergency department (ED).

Patients suspected of COVID-19 are recognized and triaged mainly based on presenting clinical characteristics and symptoms [1, 3] . The definitive diagnosis is performed by real-time reverse transcription polymerase chain reaction (RT-PCR) on samples obtained from oropharyngeal and nasopharyngeal swabs [4] . Recent studies reported that the gold standard, RT-PCR has a sensitivity of 59-71% when compared with repeated testing with the same RT-PCR method [5, 6] . This means that if the clinical suspicion is high, repeated testing is necessary. RT-PCR is thereby a time-consuming procedure and resources are or can become scarce during an outbreak.

Large-scale RT-PCR testing can be challenging, and during an outbreak peak, it can be unfeasible to wait hours for RT-PCR results to identify, isolate, and treat patients at an ED. Recent studies suggest that chest computed tomography (CT) could be leading in triage and aid in the early diagnosis of patients suspected of COVID-19 infection [7] . Recent studies from China reported high sensitivities (97-98%) for chest CT in the early detection of COVID-19 in patients who later had a positive RT-PCR test result [5, 6] . The National Health Commission of the People's Republic of China even stated that diagnosis of COVID-19 could solely be based on chest CT findings [8] . Typical abnormalities seen on chest CT in COVID-19 patients [9] [10] [11] are also seen in patients with an initial false negative RT-PCR result [12] .

Earlier attempts were also made to create a machine learning prediction model for COVID-19, based on chest X-ray and additional data in order to improve diagnostic accuracy of conventional chest X-ray alone [13] . For chest CT, this has not been done yet.

This study was initiated at the start of an outbreak peak in the Netherlands when RT-PCR testing capacity was considered to be a potential bottleneck in the flow of patients at an ED. The aim of this study was to investigate the diagnostic performance of chest CT compared with the first RT-PCR test in adult patients suspected of COVID-19 infection in the setting of the ED. In addition, we also constructed and internally validated a predictive machine learning model based on chest CT and additional data sources such as laboratory data for early prediction of COVID-19 infection. The ultimate goal was to assess whether chest CT could be used to substitute RT-PCR testing in triage during COVID-19 outbreaks where scarcity of RT-PCR tests would hinder efficient and rapid diagnosis, isolation, and treatment of patients.

This single-center prospective cohort study was performed at both locations of the Franciscus Gasthuis & Vlietland hospital in Rotterdam and Schiedam, the Netherlands, which has a level 2 trauma center with 48,000 visits annually at the ED's. Waiver of informed consent was obtained by the medical ethical commission (MEC-U, W20.076).

Consecutive patients who visited the ED between March 27-April 20, 2020, and who met the following inclusion criteria for this study were included (a) age ≥ 18 years; (b) suspected infection with COVID-19 in combination with at least one of the following: (1) new respiratory symptoms persisting for ≤ 2 weeks and present during the last 24 h, (2) saturation ≤ 94% and/or respiration rate ≥ 20/min and/or abdominal complaints, and (3) a high clinical suspicion even in the absence of symptoms; and (c) RT-PCR and chest CT performed within 24 h after each other.

Exclusion criteria for this study were (a) previously confirmed COVID-19 infection; (b) instability defined as a peripheral oxygen saturation < 92% despite 5 l of oxygen and/or a systolic blood pressure < 90 mmHg; (c) principle presentation due to high energetic trauma, thrombolysis, or acute coronary syndrome; (d) pregnancy; and (e) non-interpretable first RT-PCR result.

COVID-19 suspected patients were triaged by a nurse in a triage tent especially set-up for the crisis as part of routine care. The Manchester Triage System [14] was used to triage patients; additional symptoms related to possible COVID-19 were assessed and vital signs were registered. Thereafter, arterial blood gas, blood samples, and nasopharyngeal swabs were obtained from all patients. Nasopharyngeal swabs were sampled from the oral cavity and subsequently from the nasal cavity using the same swab. After swabs were obtained, patients immediately underwent a chest CT, after which the physician took the medical history and performed a physical examination.

Chest CT and blood results followed within 60 min. Based on the clinical performance of the patient and the test results, a decision was made to admit the patient or not. RT-PCR results would mostly follow after 5-12 h. Due to scarce resources, repeated RT-PCR testing, preferably from sputum samples, was only performed in admitted patients in case of a persisting high suspicion for COVID-19 despite earlier negative RT-PCR results.

Data of additional variables was extracted from electronic medical records of all patients included in the study. These variables encompassed demographic information, information about ED triage, COVID-19 presenting symptoms and vital signs, laboratory, microbiology, and CT results, and treatment-related variables (Online Resource 1).

All chest CT images were obtained with patients in supine position in one of five CT scanners (Canon Aquilion One Genesis 320 slices, Canon Aquilion Prime 80 slices, Philips Brilliance 64 64 slices, Philips Big Bore 16 slices, Siemens Symbia T16 16 slices). Twenty board-certified radiologists were trained to read the CT images and classify them according to the CO-RADS classification, which was recently created by the Dutch Association of Radiologists (NVVR) [15] . Examples of CT images of all 5 CO-RADS categories are shown in Fig. 1 . A CO-RADS score of 1-3 was classified as non-COVID-19,whereas a CO-RADS score of 4 or 5 was classified as COVID-19 positive (Online Resource 2). A standardized reporting format was developed. Two independent radiologists were consulted in case of any doubt about the classification.

RT-PCR was performed according to the national reference method that was established after international collaboration [16] , or by the CE-IVD kit GeneFinder TM COVID-19 Plus RealAmp Kit using the sample-to-result platform ELITe InGenius®.

Hematology tests were performed on Beckman Coulter DxH-800 and Sysmex XN-10 analyzers. Clinical chemistry tests were performed on Abbott Architect c-8000 and i-2000 platforms. Blood gas analyses were performed using Werfen GEM-4000 and Siemens RAPIDlab 1265 analyzers. Siemens CS-2100i and CS-5100 analyzers were used to determine fibrinogen concentrations.

With RT-PCR as reference, four groups were identified based on combining chest CT and RT-PCR outcomes, namely, true positives (TP) with a positive RT-PCR and a positive chest CT, false positives (FP), true negatives (TN), and false negatives (FN).

Statistical analysis was performed using SPSS version 26 and the scikit-learn machine learning library for Python version 3.7.

Characteristics of patients were summarized using the mean (± SD) or median (IQR) for continuous variables and counts and percentages for categorical variables.

Normality of variables was tested using Shapiro-Wilk tests. Normally distributed variables were compared using unpaired t-tests, while non-normally distributed variables were compared using Mann-Whitney U-test. Nominal variables were tested using Pearson's chi-square tests.

Diagnostic performance of chest CT was assessed as diagnostic accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), and positive likelihood ratio (LR+) and negative likelihood ratio (LR-), taking the first RT-PCR result as reference. These measures are summarized as proportions with (exact) 95% confidence intervals (CI). A ROC was constructed using logistic regression.

A prediction model for COVID-19 diagnosis based on radiological, laboratory, and clinical data was constructed using logistic regression while using scikit in order to improve the diagnostic accuracy of CT. Again, the first RT-PCR was taken as a reference. Regression analysis was performed on complete cases (no missing variables). Due to missing variables in some patients, fewer patients could be used to train the model than could be used to evaluate chest CT performance. Variables were checked for multicollinearity by calculation of variance inflation factors (VIF). A VIF higher than 4 was determined to be unsatisfactory due to high collinearity [17] . The model was validated using 10-fold cross validation, and a ROC curve was obtained from logistic regression probabilities.

Approximately 2100 patients were presented to the ED during our inclusion period. Both a nasopharyngeal swab and a chest CT were obtained from 404 of these patients. Of these, 85 patients retrospectively did not meet the case definition and were subsequently excluded leaving 319 patients eligible for analyses (Fig. 2) . Of these patients, 186 had a negative RT-PCR result and 133 had a positive RT-PCR result. There were no significant differences in the presence of comorbidities (Table 1) . Fever, coughing, dyspnea, myalgia, malaise, and diarrhea were the symptoms experienced most often in RT-PCR positive patients, whereas RT-PCR negative patients more often experienced a sore throat, were more often current smokers, and had fewer moments of contact with confirmed COVID-19 patients. No differences were found for risk factors like obesity, recent travel to high-risk areas, or employment in the healthcare sector. At admission, average temperature and respiratory rates were higher and median saturation levels were lower in patients with a positive RT-PCR result. Finally, patients with a positive RT-PCR result were more likely to be admitted, before RT-PCR results were known, compared with patients with a negative RT-PCR result.

Performance of CO-RADS score in the diagnosis of COVID-19 compared with first RT-PCR Table 2 shows the performance characteristics of chest CT using the CO-RADS score, compared with the first RT-PCR. Figure 3 shows the ROC curve with an AUC of 0.914 (0.879-0.949), and Fig. 4 illustrates the percentage of RT-PCR positive and negative patients per CO-RADS subgroup: 95% of patients with a CO-RADS score of 1 had a first RT-PCR which was negative, whereas 90% of patients with a CO-RADS score of 5 had positive RT-PCR results. Of all patients, 4.1% tested false negative on CT and 6.9% tested false positive. In this cohort with a COVID-19 prevalence of 41.7%, CT scan using the CO-RADS scoring yielded a PPV of 84.5% and an NPV of 92.7%.

Differences between groups as classified using chest CT (CO-RADS) scoring FN patients experienced fever, coughing, dyspnea, chest pain, and malaise more often compared with TN ( Table 1 ). The FN group showed the highest number of obese patients. Most current smokers were TP. Patients in the FP group had a median longer duration of complaints compared with patients from the other three groups (11 vs 7 days). Interestingly, false negative patients presented more often with CO-RADS 1 than CO-RADS 2 or CO-RADS 3.

Vital signs and laboratory results on admission also differed between groups. Both true positive and false negative patients needed oxygen more often, and also in higher levels. Elevated levels of ferritin, procalcitonin, lactate dehydrogenase (LDH), C-reactive protein (CRP), and creatine kinase (CK), as well as leucopenia, lymphopenia, and neutropenia were most often seen in TP patients and least seen in TN patients. Of true positives, 80.8% were admitted, followed by FN (76.9%), FP (63.6%), and TN (59.1%).

A predictive machine learning model based on chest CT and additional data sources Our prediction model is based on the CO-RADS score from chest CT complemented with the following additional Figure 5 shows the ROC curves of CO-RADS alone and the prediction model. For performance characteristics see Online Resource 3, and for VIF scores of the variables see Online Resource 4. The prediction model has a significantly higher specificity than the CORADS score alone in categorizing COVID-19 patients (0.934 vs 0.886), with comparable sensitivity (0.910 vs 0.893). K-fold cross validation (k = 10) was performed on this prediction model to check for overfitting. This resulted in an accuracy of 0.91 ± 0.10 and an adjusted R 2 of 0.652.

In this prospective observational study during the peak of the COVID-19 outbreak in the Netherlands, 319 patients with suspected COVID-19 presenting to the ED were assessed by chest CT and nasopharyngeal swabs. With a sensitivity of 90.2% and specificity of 88.2%, the chest CT performed relatively well as a diagnostic modality compared with the first RT-PCR performed in COVID-19 suspected ED patients. Additionally, we constructed a machine learning prediction model by using variables which were readily available during ED visit. Ferritin, leucocyte count, creatine kinase, presence of diarrhea, and the number of days after start of complaints were predictors of COVID-19 infection in addition to the CO-RADS classification. With an AUC-ROC of 0.953, our model showed excellent performance implying that this model could possibly further aid clinicians in early recognition and diagnosis of COVID-19.

We found a sensitivity of chest CT using CO-RADS scores which was only slightly lower (90.2%) as compared with the first European study from Caruso et al. who reported a sensitivity, specificity, and diagnostic accuracy of 97%, 56%, and 72%, respectively [18] for chest CT for COVID-19 and used repeat RT-PCR as a reference. Other studies performed in China, who classified chest CT as positive based on consensus among radiologists, reported higher sensitivities for chest CT of 97% [6] and 98% [5] . Our slightly lower sensitivity could be an underestimation as 4/22 false positive patients later tested positive for COVID-19 (repeated RT-PCR or SARS-CoV-2 serology) during admission. It is plausible that these 4 patients were initially misdiagnosed due to the fact that RT-PCR tests are prone to sampling errors related to the quality and method of sampling [16] .Repeated testing is recommended to decrease the number of FN results as previous studies showed that it is possible for patients with initial negative RT-PCR results to turn positive over time [12, 19, 20] . Due to limited RT-PCR testing capacity during the peak of the outbreak, repeated testing was only available and deemed meaningful for admitted patients with a high clinical suspicion of COVID-19 infection meaning that the remaining 18 FP patients were never retested.

Our FN group consisted of 13 patients. In the setting of this study, we decided not to perform repeat chest CTs unless [21] studied possible causes of misinterpretations of chest CTs in COVID-19 positive (RT-PCR) patients and found that 56% of patients who received a scan during early onset of symptoms (0-2 days) and 9% of patients within the intermediate group (3-5 days symptoms) had a normal chest CT at that moment without any abnormalities. Another study found that chest CT abnormalities are moreover found after symptoms persisting for about 10 days [22] . The fact that 4/13 FN patients had complaints shorter than 6 days could therefore account for a part of the FN results. The added value of our prediction model, compared with chest CT alone, is mainly the reduction of false positives, thus increasing specificity, PPV, and LR+, probably due to the fact that certain features of COVID-19 (e.g., high ferritin levels, leukopenia, and high CK) are not present in other pathologies that result in high CO-RADS scores. In this study, the accuracy of the predictive model was higher compared with chest CT alone (93.1% vs 90.4%). In addition, 30% of RT-PCR negative patients and 21% of RT-PCR positive patients are correctly classified without generating FP or FN results using cutoff thresholds with 100% sensitivity and 100% specificity, respectively.

A strength of our study is that we constructed a machine learning predictive model by using readily available ED variables and CT characteristics. Something previously done with chest X-rays instead of chest CT [13] . Conventional chest X-ray could be limited since pulmonary lesions can be ambiguous or absent while they are already there on chest CT the same day [23] . Our prediction model showed that in addition to the CO-RADS score, ferritin, leucocyte count, creatine kinase, diarrhea as a symptom, and the number of days since onset of disease enhanced the prediction capacity for COVID -19 infection. An association between the laboratory parameters we included in our model and COVID-19 has also been reported in other studies [1, 3] and our findings confirm these previous observations. Our model's parameters were found to have acceptable VIF's (see Table, Supplemental Digital Content 5); the model has an adjusted R 2 of 0.652 and has a k-fold cross validation accuracy of 0.91 ± 0.10.

Finally, our study has several limitations. Firstly, this was a single-center study. Secondly, only the first RT-PCR result was used as a reference as repeated RT-PCR was only performed on admitted patients with persistent high suspicion of COVID-19. Thirdly, we did not perform repeated chest CT's in FN patients unless necessary for other medical reasons.

Chest CT, using the CO-RADS scoring system, is a sensitive and specific method that can aid in the diagnosis of COVID- Large prospective studies should further analyze additional candidate predictors that could be used to improve the performance of this machine learning prediction model. Still, 8-9% of patients in our cohort with RT-PCR positive results were classified as negative by chest CT and our prediction model. Therefore, RT-PCR is still indispensable in the diagnosis of COVID-19 and should remain the primary standard of testing. 

Conflict of interest The authors declare that they have no conflict of interest.

Ethical approval Waiver of informed consent was obtained by the medical ethical commission (MEC-U, W20.076). 

The result of the first nose/throat swab PCR test was used to categorize patients. Confidence intervals are given for the significance of difference between RT-PCR+ and RT-PCR-patients

China Medical Treatment Expert Group for C (2020) Clinical characteristics of coronavirus disease 2019 in China

RIVM Actuele informatie over het coronavirus (COVID 19

Clinical features of patients infected with 2019 novel coronavirus in

Potential rapid diagnostics, vaccine and therapeutics for 2019 novel coronavirus (2019-nCoV): a systematic review

Sensitivity of chest CT for COVID-19: comparison to RT-PCR

Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases

Diagnosis of the coronavirus disease (COVID-19): rRT-PCR or CT?

China's Hubei reports jump in new cases of COVID-19 after diagnosis criteria revision. National Health Commission of the People's Republic of China website

CT features of coronavirus disease 2019 (COVID-19) pneumonia in 62 patients in Wuhan, China

Essentials for radiologists on COVID-19: an update-radiology scientific expert panel

CT imaging features of 2019 novel coronavirus (2019-nCoV)

Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing

BMJ publishing group

CO-RADS -A categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation

Multivariate data analysis, 7th edn

Chest CT features of COVID-19 in

Computed tomographic imaging of 3 patients with coronavirus disease 2019 pneumonia with negative virus real-time reverse-transcription polymerase chain reaction test

A case report of COVID-19 with false negative RT-PCR test: necessity of chest CT

Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection

Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia

Chest radiographic and CT findings of the 2019 novel coronavirus disease (COVID-19): analysis of nine patients treated in Korea

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations