key: cord-0928898-p1r89ogz authors: Ohana, Mickaël; Muller, Joris; Severac, François; Bilbault, Pascal; Behr, Martin; Oberlin, Mathieu; Leyendecker, Pierre; Roy, Catherine title: Temporal variations in the diagnostic performance of chest CT for Covid-19 depending on disease prevalence: Experience from North-Eastern France date: 2020-11-20 journal: Eur J Radiol DOI: 10.1016/j.ejrad.2020.109425 sha: ba5459ed5fba460e49adc8443b47ccdc5cee49c1 doc_id: 928898 cord_uid: p1r89ogz RATIONALE AND OBJECTIVE: The purpose of this work was to analyze temporal variations in the diagnostic performance of chest CT for Covid-19 throughout the first wave, depending on disease prevalence variations between the ascending, peak and descending phases of the epidemic in North-Eastern France. MATERIALS AND METHODS: From March 6th to April 22nd 2020, all consecutive adult patients referred to the “Covid-19 clinic” of our Emergency Department with the availability of chest CT and of at least one RT-PCR result were retrospectively included in the present study. Chest CT was considered positive when typical Covid-19 lesions were observed (bilateral and predominantly peripheral and sub-pleural ground glass opacities and/or alveolar consolidations). RT-PCR results were considered as the reference standard. Ascending, peak and descending phases were determined based on the number of CT scans performed daily. CT diagnostic performance were calculated and variations between phases were tested for equivalence or difference using Bayesian methods. RESULTS: 2194 consecutive chest CT were analyzed. Overall CT diagnostic performance was Se = 84.2 [82.0 ; 86.3], Sp = 86.6 [84.5 ; 88.5], PPV = 86.1 [84.0 ; 88.1], NPV = 84.7 [82.6 ; 86.7] and accuracy = 85.4 [83.9 ; 86.8], with no significant differences between chest and non-chest radiologists. Variations between the ascending (11 days, 281 chest CT, disease prevalence 37.0 %), the peak (18 days, 1167 chest CT, disease prevalence 64 %) and the descending phases (19 days, 746 chest CT, disease prevalence 32.2 %) were highest for PPV and NPV with a probability of difference >99.9 %, and smallest for accuracy and specificity with a probability of equivalence >98.8 %. CONCLUSION: In a homogenous cohort of 2194 consecutive chest CT performed over a 7-week epidemic wave, we observed significant variations of CT predictive values whereas CT specificity appeared marginally affected. Chest CT is the imaging modality of choice for diagnosis, severity evaluation and follow-up of Covid-19 patients [1, 2] . Characteristic chest CT lesions have been broadly described in the recent literature [3] , with significant differences from other viral pneumonia [4] . An initial Chinese study on 1014 patients reported a higher sensitivity as compared to RT-PCR [5] . However, the exact diagnostic performance of chest CT for Covid-19 remains debated, with a meta-analysis [6] reporting high sensitivity (94 %, 95 % CI 91 %-96 %) and low specificity (37 %, 95 % CI 26 %-50 %). Actually, changes in disease prevalence could also lead to a significant degradation of both negative and predictive positive values (NPV and PPV) [7] , with concerns for a particularly low PPV that could hamper the use of CT as a screening tool in patients with moderate to severe symptoms [8] . Indeed, studies focusing on CT diagnostic performance were mainly retrospective [9] [10] [11] , with no indication about the timing of inclusions with regards to the epidemic wave. To the best of our knowledge, no study has yet reported the potential temporal variation of CT diagnostic performance in a homogeneous cohort over the course of the epidemic, when disease prevalence varies significantly between the ascending, the peak and the descending phases. The purpose of this work was therefore to analyze the temporal variations in diagnostic performance of chest CT for Covid-19 throughout the first epidemic wave, depending on disease prevalence variations between the ascending, peak and descending phases at a single tertiary clinical centre located in a severe SARS-CoV-2 cluster in North-Eastern France. The local ethics committee of Strasbourg University Hospital approved this retrospective study and waived the need of informed consent. From March 6th to April 22nd, 2020, all the consecutive adult patients with a clinical suspicion of Covid-19 admitted to the Emergency Department (ED) of a single tertiary care centre (Nouvel Hôpital Civil, Strasbourg University Hospital) were retrospectively analyzed and considered for inclusion. March 6th was the opening day of this dedicated Covid-19 ED pathway [12] . All patients referred for CT had moderate to severe symptoms, and chest CT was systematically used at ED arrival to expedite triage [8] , due to the major delays (6-24 hours) in RT-PCR results at that time. The inclusion criteria were: -clinical suspicion of Covid-19, -availability of chest CT at the time of ED admission, -availability of at least one RT-PCR result. The only exclusion criterion was a delay of more than 6 days between chest CT and the first positive RT-PCR. Gender, age and direct ICU transfer for acute respiratory insufficiency after ED passage were documented. Non enhanced chest CT were acquired on an 80-row scanner (Aquilion Prime SP, Canon Medical Systems), with parameters optimized for the patient's morphotype (tube voltage 100− 135 kV and maximum mAs of 2-50). Images were reconstructed with a slice thickness of 1 mm in mediastinal and parenchymal windows using an iterative reconstruction algorithm (AIDR-3D, Canon Medical Systems) and read on dedicated workstations with multiplanar and maximum intensity projection reconstructions. Ten consultant radiologists (4 specialized in chest imaging) with 5-30 years of experience were involved in the reading during the inclusion period. Rotations between readers changed significantly during the epidemic, thus the number of cases read by each radiologist could not be equally distributed between phases. Standardized structured reporting was systematically used and examinations were adjudicated as either positive or negative for Covid-19: -Chest CT with typical Covid-19 appearance, i.e. bilateral and predominantly peripheral and sub-pleural ground glass opacities and/or alveolar consolidations, were classified as positive [1] . -Alternative infectious findings (bronchiolitis with centrilobular nodules, lobar consolidation), non-infectious abnormalities (lung nodule, pneumothorax, effusion…) and normal examinations were classified as negative [4] . A nasopharyngeal swab (Puritan Medical Products, Guilford, ME, USA) concomitant to the chest CT was systematically performed at ED admission. Some patients underwent multiple sampling with either nasopharyngeal swab, sputum or bronchoalveolar lavage during their stay. RT-PCR for SARS-CoV-2 was chosen as the reference standard and determined the definite diagnosis: any positive result was adjudicated as a confirmed Covid-19 infection. The epidemic wave was arbitrarily divided into three phases: ascending, peak and descending, based on the number of CT scans performed daily and later confirmed by differences in prevalence of the disease, as assessed by the positivity rate of RT-PCR. Bayesian methods were used to estimate Sensitivity (Se), Specificity (Sp), Positive Predictive value (PPV), Negative Predictive Value (NPV) and accuracy with their 95 % credible interval. Posterior distributions were calculated using the Dirichlet distribution with Jeffrey's prior. Differences between the phases ascending VS peak and descending VS peak were computed. Overall differences in diagnostic performances between the 4 radiologists specialized in chest imaging versus the 6 other readers were also computed. A hypothesis of equivalence of the indicators was first tested. The equivalence was defined as an estimated difference (with its 95 % credible interval) included between +/-10 %. A probability of equivalence (PrE) was therefore calculated, and equivalence was considered achieved when PrE ≥ 95 % (bilateral hypothesis). When equivalence was not achieved, a hypothesis of difference was secondly assessed. Difference was confirmed when the 95 % credible interval of the difference didn't include the value 0, corresponding to a probability of difference PrD ≥ 97.5 % (unilateral hypothesis). The flowchart of the study is presented in Fig. 1 . 2278 chest CT were performed in 2164 patients over the inclusion period, with 2194 chest CT in 2080 patients having a RT-PCR result (of which 1091 were positive, 49.7 %). In the 1103 chest CT with a negative RT-PCR, 198 (18 %) had more than one assaypersistently negativeperformed over the course of their hospital stay. Chest CT was positive in 1067 cases, with 919 true positive and 148 adjudicated as false positive cases. Out of these 148 patients with positive chest CT and negative RT-PCR, 49 (33.1 %) underwent more than 1 RT-PCR assay, all remaining negative. Chest 281 chest CT were performed during the ascending phase (March 6th to March 16th, 11 days), with a RT-PCR prevalence of 37.0 %, a male/ female sex ratio of 0.52, a mean patient age of 59 ± 17.6yo (range 18-97) and 19 direct ICU transfers (6.8 %). 1167 were performed during the peak (March 17th to April 3rd, 18 Temporal variations of diagnostic performances are summarized in Fig. 2 ; their differences between ascending VS peak and descending VS peak with probability of equivalence +/-probability of difference are given in Fig. 3 . Variations of both positive and negative predictive values were significant and related with disease prevalence: the probabilities of equivalence for PPV and NPV was never met and their probability of difference was always >99.9 %. Conversely, variations in accuracy and in specificity were considerably lower and their probability of equivalence between phases was >98.8 %. In a homogenous population of 2194 consecutive chest CT with RT-PCR performed over a 7-week epidemic wave, we observed an equivalence in specificity and accuracy and a difference in predictive values, between the ascending, peak and descending phases. Variations in predictive values are connected to the variations in prevalence of Covid-19. The equivalence in accuracy support the use of CT as an efficient triage tool throughout the epidemic [8] . This study was carried out in a practical setting, pooling the readings of radiologists with various levels of expertise and in ED patients with a broad range of clinical manifestations. When compared to the existing literature [9] , while we reported a sensitivity similar to most publications, we noted a higher specificity than Ai et al. [15] . We report a higher specificity than in these papers, probably because exclusion criteria suggesting alternative infectious disease were considered, such as bronchiolitis with centrilobular nodules or lobar consolidation [1] . Our findings on a consecutive and homogenous ED cohort might be more representative of the "real-world" diagnostic performance of chest CT [16] . A potential "learning curve" effect must be raised. Indeed, new data on Covid-19 were published daily, and our knowledge of the disease grew along the epidemic wave [2] . Therefore, one might suggest that the variations in sensitivity and specificity were also secondary to a massive exposure to the disease. Yet, variations in sensitivity were lower between the ascending and peak than between the descending and peak phases, which does not support an improvement over time. In addition, we did not find a significant difference in performance between the radiologists specialized in chest imaging and the other readers. These observations could disfavor a potential learning curve effect in this setting. Variations of PPV and NPV with regards to variations of prevalence are mathematical and expected [7] , while the moderate temporal variations of sensitivity are more difficult to explain. We hypothesize that the decreased sensitivity in the descending phase might be related to a/ an older population, where the presentation might not be as typical as in younger patients [17, 18] and b/ to an overall decrease in the severity of the disease as assessed by the decreased ICU transfer rate and an overall higher number of CT read as normal (72 % of all false negative examinations in the descending phase compared to 57-58 % in the peak and ascending phases). First, we chose RT-PCR as the gold standard for both positive and negative definite diagnosis, which is a debatable position for cases with positive chest CT and negative RT-PCR (148 in our cohort), considering that the risk of false negative of RT-PCR is non-negligible is these cases [5, 19, 20] . However, 33.1 % of these 148 patients with a positive chest CT and an initially negative RT-PCR had a repeat RT-PCR assay (up to 6 times) that remained negative. Second, our study is retrospective, even though we included all consecutive cases from our ED Covid-19 clinic, and therefore can be subject to inclusion bias. Third, only basic demographics (age and gender) were recorded, and useful clinical information such as Oxygen saturation, onset to symptoms and outcome could not be documented in the present study for the whole population. Therefore, a strict comparison of the disease severity between phases could not be achieved, and one can imagine that potential differences such as a less "aggressive" or respiratory-focused Fig. 3 . Differences in sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy between ascending VS peak (a) and descending VS peak phases (b) with probability of equivalence (PrE) +/probability of difference (PrD) calculated. * indicates differences that met the hypothesis of either an equivalence (PrE≥95 %) or either a difference (PrD≥97.5 %). presentation in the decreasing phase could lead to potential variations in performance of chest CT. Yet, the clinical criteria for inclusion in the ED Covid-19 pathway were similar through the epidemic, and patients had a relatively consistent presentation, with clinical suspicion of Covid-19 and moderate to severe symptoms. Fourth, we took into account only the initial reading of the chest CT at the time of ED admission, which involved 10 consultant radiologists with diverse experience, sub-specialty and training backgrounds. Even though a standardized report was used to mitigate inter-reader variations, a dedicated reading or second look by radiologists specialized in chest imaging might have further improved the performance of CT. Finally, our results come from a single centre in an intense SARS-CoV-2 cluster and, as is, cannot be generalizable to other places with lesser prevalence of the disease or other medical systems. To conclude, our study showed a significant difference of predictive values, a moderate variation of sensitivity and an equivalence of specificity and accuracy between the ascending, peak and descending phases in a homogenous cohort of 2194 consecutive chest CT performed over a 7-week epidemic wave. This could consolidate the use of chest CT throughout the epidemic, regardless of significant variations in disease prevalence. The author declares no sources of support or conflict of interest. European Society of, I. the European Society of Thoracic, COVID-19 Patients and the Radiology Department -Advice From the European Society of Radiology (ESR) and the European Society of Thoracic Imaging (ESTI) Chest CT in COVID-19 pneumonia: a review of current knowledge COVID-19 pneumonia: a review of typical CT findings and differential diagnosis Characteristic CT findings distinguishing 2019 novel coronavirus disease (COVID-19) from influenza pneumonia Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases Diagnostic performance of CT and reverse transcriptase-polymerase chain reaction for coronavirus disease 2019: a metaanalysis Imaging publications in the COVID-19 pandemic: applying new research results to clinical practice The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the fleischner society Chest CT and coronavirus disease (COVID-19): a critical review of the literature to date Sensitivity of chest CT for COVID-19: comparison to RT-PCR COVID-19): a systematic review of chest CT findings in 4410 adult patients The organisation of a French emergency department in a coronavirus hotspot Clinical Features and Chest CT Manifestations of Coronavirus Disease 2019 (COVID-19) in a Single-Center Study in Initial clinical features of suspected coronavirus disease 2019 in two emergency departments outside of Hubei Chest CT features of COVID-19 in Lessons learned from chest CT in COVID-19 Clinical features of COVID-19 in elderly patients: a comparison with young and middle-aged patients COVID-19): a systematic review of chest CT findings in 4410 adult patients Negative Nasopharyngeal and Oropharyngeal Swabs Do Not Rule Out COVID-19 Patients with initial negative RT-PCR and typical imaging of COVID-19: clinical implications The authors would like to thank Ms Sybille EICHERT for her precious help in the data collection, Mr Cedric HINTZPETER for his precious help in the data analysis and Ms Amandine ELCHINGER for her help in proofreading the manuscript.