key: cord-0781035-ttf9zxsi
authors: Ciccarese, Federica; Coppola, Francesca; Spinelli, Daniele; Galletta, Giovanni Luca; Lucidi, Vincenzo; Paccapelo, Alexandro; De Benedittis, Caterina; Balacchi, Caterina; Golfieri, Rita
title: Diagnostic Accuracy of North America Expert Consensus Statement on Reporting CT Findings in Patients with Suspected COVID-19 Infection: An Italian Single Center Experience
date: 2020-07-23
journal: Radiol Cardiothorac Imaging
DOI: 10.1148/ryct.2020200312
sha: 58b63015acc03e2df0043ec06140cc8918410ff4
doc_id: 781035
cord_uid: ttf9zxsi

PURPOSE: In suspected COVID-19, Real-Time Polymerase Chain Reaction (RT-PCR) is the reference standard but affected by long reporting time (6-48 hours); the aim of the study was to evaluate the diagnostic accuracy of the four standardized categories for CT reporting proposed by Radiological Society of North America (RSNA) to support a faster triage. MATERIALS AND METHODS: Retrospective analysis of 569 HRCT performed for suspected COVID-19 from February 27 to March 27, 2020 (peak of infection in Italy). Imaging pattern was classified according to RSNA statement in “typical”, “indeterminate”, “atypical” and “negative” and compared to RT-PCR, available in 460 patients. Inter-observer variability in reporting between a senior and a junior radiologist was evaluated. Utility of vascular enlargement sign in indeterminate cases, was also assessed. RESULTS: Diagnosis of COVID-19 was made in 211/460 patients (45.9%). “Typical” pattern (n=172) showed a sensitivity of 71.6%, a specificity of 91.6% and a Positive Predictive Value (PPV) of 87.8% for COVID-19. “Atypical” (n=67) and “negative” (n=123) pattern demonstrated PPV of 89.6% and 86.2% for non COVID-19. “Indeterminate” (n=98) was non-specific, but vascular enlargement was most frequently found in patients with COVID-19 (86.1%- p<0.001). Inter-observer agreement was good for “typical” and “negative” pattern, fair for “indeterminate” and “atypical” (k=0.5, p=0.002). CONCLUSION: In an epidemic setting, application of the four categories proposed by RSNA provides a standardized diagnostic hypothesis, strongly linked to the RT-PCR results for “typical”, “atypical” and “negative” pattern. In “indeterminate” pattern, the analysis of vascular enlargement sign could facilitate the interpretation of imaging features.

In January 2020, a novel coronavirus named Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) was identified as responsible of several cases of pneumonia referred as coronavirus disease in Wuhan. Italy was the first Western Country were the epidemic spread; first case was detected on February 20, 2020, followed by a rapid increase in the number of cases, especially in the Northern Italy, reaching 236.989 cases by June 15 (1) . In these setting, early detection and containment become crucial.

The gold standard for diagnosis is Real-Time Polymerase Chain Reaction (RT-PCR). However, this test has shown several limitations: 1) limited testing capacity due to insufficient kits or laboratory supplies 2) long reporting time (from 6 to 48 hours) hardly compatible with urgent decision-making 3) great variability in sensitivity, ranging from 37-71% (2) (3) (4) . In the clinical practice, one negative RT-PCR does not exclude COVID-19 and multiple repeat tests may be required to make the final diagnosis. Thus, imaging has emerged as an important tool to guide diagnosis in case of clinical-laboratory discordance (2) .

Imaging protocols greatly vary, depending on local public health directives: chest X-Ray is widely used, though not enough accurate in mild or early COVID-19 (5) ; there is a great interest in bed-side lung ultrasonography, but limited experiences are reported (6, 7) ; among imaging modalities, Computed Tomography (CT) is the most sensitive (sensitivity of 60-98%), but on the other side affected by low specificity and possible false negatives in the first stage of the disease (8) . For these reasons, most radiological societies do not recommend performing screening CT (9, 10) . Nevertheless, the number of CT performed for suspected COVID-19 has increased. Indeed, CT better demonstrates early pulmonary manifestation of COVID-19, with reported findings consistent with COVID-19 even in false negative RT-PCR tests and in asymptomatic patients (11, 12) .

Moreover, it could highlight some ancillary findings, as the recently reported "vascular enlargement", frequently associated with COVID-19 (13) .

In order to provide guidance to radiologists, a standardized language was proposed by the Radiological Society of North America (RSNA) to reduce variability in reporting (8) , but data on its application in the clinical practice are lacking. The aim of the present study was to evaluate the diagnostic accuracy of each category proposed ("typical", "indeterminate", "atypical", "negative") versus RT-PCR and to assess the inter-observer variability between a senior and a junior radiologist. Utility of vascular enlargement sign in indeterminate cases, was also assessed.

This study is an observational, retrospective, single center study and was approved by our local institution review board (IRB). The informed consent was waived by the IRB due to the retrospective nature of the study.

We consecutively selected all patients, both hospitalized or accessed to the Emergency Department, who underwent CT for suspected pneumonia COVID-19-related, from February 27 to March 27, 2020, which represented the peak period of the infection in Italy.

In case of suspected COVID-19, imaging workflow in our hospital consists in the execution of chest X-Ray, eventually followed by CT scan if radiography provides negative or uncertain findings. Lung-ultrasonography could be eventually performed depending on the skill of operator, but it is not routinely recommended.

Inclusion criteria were: 1) symptomatic patients presenting fever (of unknown origin) or respiratory symptoms such as cough or dyspnea; 2) patients undergoing CT at our Institute.

The exclusion criteria were: 1) patients with suspected COVID-19-related interstitial pneumonia, but investigated with only other imaging techniques (X-ray or ultrasound); 2) patients with suspected COVID-19related interstitial pneumonia with CT performed outside our hospital; 3) severe motion artifact on chest CT (n=46).

Overall, a total of 569 CT scans were collected.

RT-PCR results were considered as reference standard and were extracted from the patients' electronic medical records in our hospital information system. As some patients had more than one test, only RT-PCR performed within 24 hours from CT scan were considered.

RT-PCR was a 2-site test, performed through oro and nasopharyngeal swab; patients with RT-PCR not available or not carried out were finally excluded from the analysis (n=109)- Figure 1 .

Chest CT acquisitions were obtained with the patients in supine position during end-inspiration without intra venous contrast medium injection. Expiration scan was not performed.

CT scans were performed on two CT scanners dedicated only to patients with suspected COVID-19:

1) 64-slice CT (GE Medical System, Light speed VCT 64 slice), with the following technical parameters: tube voltage: 120 kV; tube current modulation 100-250 mAs; spiral pitch factor: 0.98; collimation width: 64 x 0.625.

Reconstructions were made with convolution kernel BONEPLUS at a slice thickness of 1.25 mm (n=391).

2) 128 slice CT (PHILIPS Ingenuity CT 128 slice, with the following technical parameters: tube voltage: 120 kV; tube current modulation 100-250 mAs; spiral pitch factor: 1.224; collimation width: 64 x 0.625.

Reconstructions were obtained with convolution kernel Y-SHARP at a slice thickness of 1 mm (n=178).

After each patient chest CT examination, passive air exchange and a decontamination of CT room was performed with surface disinfection at 62-71% ethanol or 01% sodium hypochlorite.

All CT examinations were reviewed by accessing to the PACS (Picture Archiving and Communication System) of our hospital by 2 radiologists involved in the study, blind to each other and to RT-PCR results, with different level of experience: a senior reader, with more than 10 years of experience in thoracic imaging, and a junior reader 1 year experienced in thoracic imaging.

Radiological findings were classified according to the four categories proposed by RSNA (Table 1) 

Inter-observer variability in assessing radiological pattern was finally evaluated.

All data were anonymized and collected in a shared database.

Data were expressed as means, ranges and frequencies. The diagnostic performance of CT was evaluated with sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) considering RT-PCR as the reference standard. We considered "typical" and "indeterminate" pattern to predict COVID-19 disease and "atypical" and "negative" pattern to predict non COVID-19 disease.

Cohen's Kappa were used for evaluate inter-rater reliability, analysis of variance was performed to assess age differences between the groups and Fisher's exact test was used when appropriate (comparison between CT I n p r e s s categories among COVID-19 and non COVID-19; presence of vascular enlargement; differences in age and gender).

Scatter plot was used to assess the relation between PPV and prevalence for "typical" pattern.

Two tailed p-values<0.05 were considered statistically significant. Statistical analysis was performed using SPSS version 13.0 (SPSS Inc. Chicago, IL).

The demographics of patients and CT results are summarized in Table 2 The analysis of distribution of pattern documented that the "typical" (n=151) showed a sensitivity of 71.6%, a specificity of 91.6%, a PPV of 87.8% for COVID-19 ( Figure 2 ). "Indeterminate" pattern (n=36) was particularly non-specific, with low sensitivity (17.1%) and PPV (36.7%).

This pattern was most frequently detected in elderly patients (mean age of 68 years versus 63 of patients with other patterns, P=0.016). In this group of patients, we conducted a further analysis by evaluating sub-segmental vascular enlargement (defined as more than 3 mm of diameter). This sign was found in 31/36 (86.1%) with COVID-19 versus 23/62 (37.1%) of patient without COVID-19, involving both arteries and veins, with a difference that was statistically significant (p<0.001) (Figure 3 and 4) .

"Atypical" (n=7) and "negative" (n=17) pattern counted only for 11.4% of total COVID-19 patients.

2. Non-COVID-19 patients were 249/460 (54.1%). In these patients, the most prevalent patterns were "atypical" (n=60) and "negative" (n=106), representing the 66.7% of patterns observed and demonstrating a really strong PPV for non COVID-19, respectively of 89.6% and 86.2% (Figure 5 and 6) . None of the "atypical" pattern was related to a viral infection. Regarding the distribution of patterns, "typical" was most frequently detected in COVID-19, while "atypical" and "negative" in non COVID-19, with a difference that was statistically significant (p<0.001). Only for "indeterminate" pattern there was no statistically significant difference among COVID-19 and non COVID-19 groups, though it was most often observed in non COVID-19 (p=0.052).

The comparison between senior and junior radiologist documented a very high agreement for "typical" and "negative" pattern (respectively 84.7% and 93.1%), while a fair agreement for "indeterminate" (13.1%) and "atypical" (30.9%) was observed. Overall, we observed a moderate agreement: K=0.500 (p=0.002).

For a better interpretation of the PPV value for the "typical" pattern in relation with the prevalence of COVID-19 disease in our population, we also performed an analysis of the PPV among time and we found that at 3 march 2020 we had a prevalence of 16.7% and a PPV of 40.0%, an increasing prevalence among time up to the peak at 27 march 2020 with prevalence of 46.4% and a PPV of 87.8% (Figure 7 ).

The exact role of CT in diagnosis of COVID-19 remains debatable. Sensitivity of CT is influenced by temporal changes of radiological appearance: CT can be negative in the first stage of the disease; early CT features consist in ground-glass opacities, with predominantly bilateral and peripheral distribution, followed by progressive transformation into multifocal consolidations, crazy paving, bandlike pattern, perilobular opacities and reversed halo sign (14-17); interpretation of imaging findings is further complicated as some patients could 

However in our series, "typical" pattern showed a specificity of 91.6%. This is probably due to the specific setting of high community disease burden. In this setting, application of the categories proposed by RSNA demonstrated a really strong link to RT-PCR results: "typical" pattern had a high PPV (87.8%) for COVID-19, while "atypical" and "negative" demonstrated high PPV for non COVID-19 (89.6% and 86.2%). Recognition of "atypical" pattern is particularly important because it could provide a differential diagnosis.

However, a small percentage of false negatives and positives were detected: specifically, 11.4% of COVID-19 patients had or a "negative" or an "atypical" pattern. Thus these patterns could not exclude COVID-19. On the other side, 8.4% of patients with negative RT-PCR showed a "typical" pattern on CT.

In addition, recognition of pattern should be provided by expert radiologists, as the inter-observer agreement analysis demonstrated: while "typical" pattern is easy to identify, interpretation of "indeterminate" and "atypical" pattern requires greater skills.

The application of a radiologic algorithm to triage the massive load of acute respiratory referral has been recently proposed (26) . This proposal is in agreement with the position of Fleischner Society that identified a specific clinical scenario where imaging is advised to support a more rapid triage, that concerns patients with moderate-severe features of COVID-19 in a resource constrained environment (2) . Also Dangis demonstrated that CT could play a complementary role to RT-PCR for the early triage of patients, as CT results were rapidly available, with an accuracy that reached 94.4% (25) .

The main diagnostic challenge remains the "indeterminate" pattern. Indeterminate pattern consisted in ground glass opacities without the typical distribution (multifocal, diffuse, perihilar or unilateral), non-rounded and non-peripheral. Patients belonging to this category were older than the general population and this could be associated to a more complex clinical-radiological scenario. A further analysis demonstrated that subsegmental vascular enlargement was observed in most of these patients with COVID-19 (86.1%), as also other authors observed (13, (27) (28) (29) . To our knowledge, the exact meaning of this sign is still unclear, but could be related to the mechanism of action of the virus; SARS-CoV-2 infects the host using the angiotensin converting enzyme 2 receptor, which is expressed in several organs, including the lung and endothelial cells (30) . Ye et al hypothesized that vascular enlargement could be attributed to the damage of the endothelial wall caused by pro-inflammatory factors (27) , while Albarello found that this sign could precede the development of new lung infiltrates (29) . Pathological post-mortem studies demonstrated both capillary congestion and platelet-fibrin I n p r e s s thrombi in small arterial vessels, suggesting pulmonary thrombotic microangiopathy (31) . However, further pathological studies are needed to clarify the exact pathogenesis of this sign.

The development of faster laboratory tests could limit the role of CT in the diagnostic assessment of COVID-19. However, baseline CT scan has other applications; Colombi et al demonstrated that the extent of CT lung abnormalities at admission were predictors of intensive care unit recovery (32) . Moreover, CT could be involved in several research scenarios through the application of structured reported and radiomics (33, 34) .

The various roles of CT justify the increase number of requests, even to rule out COVID-19 infection.

Our study has several limitations: 1) it was a retrospective analysis influenced by the high prevalence of COVID-19 in our population; 2) RT-PCR was considered as the reference standard, though several limitations of this test have been reported (2) (3) (4) .

In conclusion, our experience demonstrated that the application of the diagnostic categories proposed by RSNA could provide a correct diagnosis in most patients, in a setting of epidemic spread, with high pre-test probability of COVID-19.

RT-PCR remains the hallmark for diagnosis and could not be replaced by CT; however, we strongly recommend the application of standardized report to all patients with suspected COVID-19 as a first useful method to support a more rapid triage. .

Imaging findings scan demonstrated an "atypical" pattern characterized by small peri-bronchial consolidations with tree-in-bud appearance, located in the apical segment of the superior right lobe (Fig 5a-arrow) , and multiple right hilar and mediastinal lymphadenopathy (Fig 5b-circle) . Final diagnosis was of lung and nodal tuberculosis. CT did not show findings suggestive for pneumonia-"negative" pattern (Fig 6a) , but RT-PCR was positive for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection. After 5 days, due to a worsening of the clinical conditions, CT was repeated, revealing a "typical" pattern characterized by bilateral ground-glass opacities (Fig 6b-circle) . After 12 days (17 days from the onset of symptoms), further radiological progression with development of diffuse alveolar damage and acute respiratory distress syndrome (Fig 6c) which lead to the death of patient. 

WHO Coronavirus disease (COVID-2019) situation reports-147. World Health Organization website

The Role of Chest

Multinational Consensus Statement from the Fleischner Society. Chest

Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases

Stability issues of RT-PCR testing of SARS-CoV-2 for hospitalized patients clinically diagnosed with COVID-19

Frequency and Distribution of Chest Radiographic Findings in COVID-19 Positive Patients

Is There a Role for Lung Ultrasound During the COVID-19 Pandemic?

Proposal for International Standardization of the Use of Lung Ultrasound for Patients With COVID-19: A Simple, Quantitative, Reproducible Method

Radiological Society of North America Expert Consensus Statement on Reporting Chest CT Findings Related to COVID-19. Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. Radiology: Cardiothoracic Imaging

ACR Recommendations for the use of Chest Radiography and Computed Tomography (CT) for Suspected COVID-19 Infection. American College of Radiology website

Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected

COVID19-Infection

COVID-19 patients and the radiology department -advice from the European Society of Radiology (ESR) and the European Society of Thoracic Imaging (ESTI)

Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR

Chest CT Findings in Cases from the Cruise Ship "Diamond Princess

Radiology: Cardiothoracic Imaging2020

Chest CT Features of COVID-19 in

Coronavirus Disease 2019 (COVID-19): A Systematic Review of Imaging Findings in 919 Patients

Emerging 2019 Novel Coronavirus (2019-nCoV) Pneumonia

Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection

Time Course Changes On Chest CT During Recovery From 2019 Novel Coronavirus (COVID-19) Pneumonia

Early CT features and temporal lung changes in COVID-19 pneumonia in Wuhan, China

CT Features of Coronavirus Disease 2019 (COVID-19) Pneumonia in 62

Coronavirus Disease 2019: Initial Detection on Chest CT in a Retrospective Multicenter Study of 103 Chinese Subjects

AI Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Etiology on Chest CT

Radiographic and CT Features of Viral Pneumonia

Middle East respiratory syndrome coronavirus (MERS-CoV) infection: chest CT findings

Pneumonia in novel swine-origin influenza A (H1N1) virus infection: high-resolution CT findings

Accuracy and reproducibility of low-dose submillisievert chest CT for the diagnosis of COVID-19. Radiology: Cardiothoracic Imaging

Integrated Radiologic Algorithm for COVID-19 Pandemic

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): a pictorial review

Chest CT for Typical 2019-nCoV Pneumonia: Relationship to Negative RT-PCR Testing

COVID 19 INMI Study Group. 2019-novel Coronavirus severe adult respiratory distress syndrome in two cases in Italy: An uncommon radiological presentation

Endothelial cell infection and endotheliitis in COVID-19

Pulmonary post-mortem findings in a series of COVID-19 cases from northern Italy: a two-centre descriptive study

Well-aerated Lung on Admitting Chest CT to Predict Adverse Outcome in COVID-19 Pneumonia

COVID-19 Standardized Reporting" Working Group of the Dutch Radiological Society1. CO-RADS -A categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation

Use of CT and artificial intelligence in suspected or COVID-19 positive patients: statement of the Italian Society of Medical and Interventional Radiology

The authors thank Massimo Sammons, MD, for the technical assistance in the supervision of the manuscript.This copy is for personal use only. To order printed copies, contact reprints@rsna.org