key: cord-1042004-9qcmdkne
authors: Pang, Caishuang; Hou, Qingtao; Yang, Zhaowei; Ren, Liwei
title: Chest computed tomography as a primary tool in COVID-19 detection: an update meta-analysis
date: 2021-05-26
journal: Clin Transl Imaging
DOI: 10.1007/s40336-021-00434-z
sha: 4343bf32f00e2ad98a4baf6d7baba92f52c2af4f
doc_id: 1042004
cord_uid: 9qcmdkne

PURPOSE: A growing number of publications have paid close attention to the chest computed tomography (CT) detection of COVID-19 with inconsistent diagnostic accuracy, the present meta-analysis assessed the available evidence regarding the overall performance of chest CT for COVID-19. METHODS: 2 × 2 diagnostic table was extracted from each of the included studies. Data on specificity (SPE), sensitivity (SEN), negative likelihood ratio (LR−), positive likelihood ratio (LR+), and diagnostic odds ratio (DOR) were calculated purposefully. RESULTS: Fifteen COVID-19 related publications met our inclusion criteria and were judged qualified for the meta-analysis. The following were summary estimates for diagnostic parameters of chest CT for COVID-19: SPE, 0.49 (95% CI 46–52%); SEN, 0.94 (95% CI 93–95%); LR−, 0.15 (95% CI 11–20%); LR+, 1.93 (95% CI 145–256%); DOR, 17.14 (95% CI 918–3199%); and the area under the receiver operating characteristic curve (AUC), 0.93. CONCLUSION: Chest CT has high SEN, but the SPE is not ideal. It is highly recommended to use a combination of different diagnostic tools to achieve sufficient SEN and SPE. It should be taken into account as a diagnostic tool for current COVID-19 detection, especially for patients with symptoms. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40336-021-00434-z.

In December 2019, cases of pneumonia called COVID-19 caused by Severe Acute Respiratory Syndrome coronavirus type 2 (SARS-CoV-2) occurred in Wuhan, China, and, since then, has become a global epidemic disease [1] [2] [3] . The symptoms of COVID-19 are nonspecific, ranging from asymptomatic to respiratory failure and even death. The dominant clinical manifestations are fever and dry cough. Pathological diagnosis is established by nucleic acid assay testing, gene sequencing, and serological examination (IgM and IgG) of throat swabs or blood samples. Based on what we know about COVID-19 epidemic, updated clinical protocols are regularly provided to guide screening strategies for COVID-19 in affected areas by the Chinese government [4] . According to reports, in all patients, the crude case-fatality rate of COVID-19 is 2.3% [5] , and it is even higher in critically ill patients, 61.5% [6] . Therefore, early diagnosis of COVID-19 remains a significant challenge. In this outbreak, rapid and accurate diagnosis is essential for isolating and treating patients.

To date, real-time reverse transcriptase-polymerase chain reaction (RT-PCR) testing has been considered as the reference standard for the diagnosis of COVID-19 [4, [7] [8] [9] [10] .

Nonetheless, considering the latent period of the infection (estimated as 2-14 days), an initial negative RT-PCR result does not exclude the possibility of SARS-CoV-2 infection [11] . RT-PCR can also give potential false negatives if the amount of viral loads is insufficient or if the correct timewindow of viral replication is missed [12] . Besides, falsenegative results may due to improper sample collection or laboratory mistake [13] . Hence the test should be repeated in patients who are persistently suspected by clinical findings but with negative RT-PCR results [14, 15] . Besides, RT-PCR comes with disadvantages, relatively long waiting time, invasive sampling procedure to process and generate results, and needs for specialized operators and certified laboratories.

Radiology plays an essential role in the early detection as well as in the management of COVID-19 patients. Among various imaging methods, chest computed tomography (CT) occupies a large share due to its higher sensitivity (SEN) and specificity (SPE) rates [16] [17] [18] . 67.4-88.0% of COVID-19 infection is detected by chest CT, showing that pneumonia is the most common presentation [13] [14] [15] . Typical diagnostic methods include chest CT, RT-PCR test, or both. The number of RT-PCR testing capacity is insufficient, and CT can be conveniently performed for COVID-19 screening according to the situation [19, 20] . Hence CT image is widely used for early diagnosis of this disease. Several recent studies have suggested that chest CT is conductive to detect COVID-19. [13, 17, 18, [21] [22] [23] The main CT finding of COVID-19 was bilateral distribution of ground-glass opacities (GGOs) with or without consolidation. With further analysis of increasing cases, multiple CT imaging features were found, which may elucidate the possible mechanism of lung injury in COVID-19 [13] . However, individual studies may be subjected to small sample sizes, applicability to demographics, wrong methodology, or a combination of all the above-mentioned disadvantages. The potential danger is to make critically clinical decisions based on flawed information. When chest CT is used as a diagnostic tool in clinical practice, a critical evaluation of the literature is necessary. A previous metaanalysis drew a conclusion that chest CT had a high SEN but poor SPE in COVID-19 detection [24] . However, more and more studies are dedicated to expand the application of chest CT in COVID-19 detection. Now the present meta-analysis was undertaken to comprehensively assess the diagnostic value of chest CT for COVID-19.

This meta-analysis was done according to the guidelines about diagnostic research before data collection commenced, as well as the preferred reporting items for systematic reviews and meta-analyses were followed for reporting this meta-analysis [25] [26] [27] . This retrospective meta-analysis waived the approval of the institutional review board.

We did the literature retrieval systematically in PubMed, Wanfang, and CNKI to find potentially eligible studies published up to September 6, 2020, using the following keywords as search terms: "coronavirus disease 2019" OR "novel coronavirus" OR "SARS-CoV-2" OR "COVID-19" AND "computed tomography" OR "CT" AND "RT-PCR".

Additionally, we also searched the aforementioned terms using different combinations.

Two reviewers independently examined the titles and abstracts first, then downloaded the full text of all the possibly eligible studies for further review. Any conflicts between the two reviewers (CSP and QTH) were resolved by a third author (ZWY). We brought into studies which met the following eligibility criteria: (1) original research papers; (2) either in English or Chinese language; (3) investigated the diagnostic value of chest CT for COVID-19; (4) the reference standard is RT-PCR; (5) enough raw data to construct the 2 × 2 table.

Data were extracted using Microsoft Excel from eligible studies by two reviewers independently, including name of first author, year of publication, country, number of patients, inclusion criteria, age, sex, reference standard, specimen type or location, type of CT, diagnostic CT criteria, true positive (TP), false positive (FP), false negative (FN), and true negative (TN). Disagreements between two reviewers were resolved through negotiation. When different diagnostic CT criteria were used, TP, FP, FN, and TN were reconstructed separately.

The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) checklist was used to assess the methodological quality in each study [28] . QUADAS-2 assesses the methodological quality in two parameters: the risk of bias and applicability concerns. Two reviewers independently assessed these in each eligible study based on predefined key factors. Conflicts were resolved by discussion.

Standard methods recommended for diagnostic accuracy meta-analysis were followed. We extracted the accuracy data (TP, FP, FN, and TN) from each eligible study and calculated the following diagnostic estimates: pooled SEN, specificity SPE, positive likelihood ratio (LR+), negative likelihood ratio (LR−), diagnostic odds ratio (DOR), and their corresponding 95% confidence intervals (CIs). We also plotted summary receiver operating characteristic (SROC) curve to obtain the value of area under the curve (AUC). Based on the estimates of the heterogeneity tests, a random effects model or a fixed effects model was used to calculate correlation index across studies.

Heterogeneity across the studies was evaluated using chi-squared and Fisher's exact tests. Meta-regression analysis was conducted to determine potential covariates of the heterogeneity. Deeks funnel plot was conducted to evaluate the potential publication bias [29] . Forest plot was performed using Meta-DiSc 1.4 (XI. Cochrane Colloquium, Barcelona, Spain). Quality assessment was performed using RevMan 5.2 (Cochrane Collaboration, Oxford, UK), and publication bias using STATA 12.0 (Stata Corp., College Station, TX) software. A two-tailed P value smaller than 0.05 was judged statistically significant.

We initially screened on title and abstract levels, and then reviewed on full-text level. Finally, 15 publications were deemed eligible for inclusion. [17, 18, [21] [22] [23] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] . Studies were eliminated mainly because of unpaired chest CT and RT-PCR, as well as incomplete data. Figure 1 shows the selection process of potentially qualified studies. Considering that there were more than one criteria used for CT diagnosis (high-risk vs moderate and low-risk cases, moderate-and high-risk vs low-risk cases) [39] . The publication was perceived as two independent research studies, which offered up 16 studies in our meta-analysis in the aggregate.

Twelve studies were conducted in China, two in Italy, one in Japan, and one in Netherland. The average sample size was 187 (varied from 21 to 1014), with a total of 2992 patients. Most studies enrolled patients consecutively and retrospectively, except two studies were prospectively [33, 37] . The inclusion criteria in these studies were patients clinically suspected COVID-19 infection, and who underwent both chest CT and RT-PCR. The CT images were read by at least two radiologists independently and reached a consensus in 13 included studies [17, 21-23, 30, 32-35, 37-39] , and others unknown. In these 13 studies, radiologists were blind to the RT-PCR results, except two unknown [33, 38] . A standardized method for CT report was reported by Stanzione et al. [40] , however, none of the included studies mentioned about the standardized method. Nasopharyngeal and oropharyngeal swab was the most common specimen type for RT-PCR in eligible studies [21-23, 33, 36, 37] . Three studies had sampled lower respiratory tract specimen for RT-PCR [18, 23, 32] . Four studies did not mention about the specimen type or location [34, 38, 39] . Details of pivotal characteristics of included studies are indicated in Table 1 .

QUADAS-2 appraised methodology of selected studies in the following four fields: patient selection, index test, reference standard, and flow and timing with the utility of RevMan [25, 41] . The quality of studies included in the present meta-analysis was not that good. Most studies that were included did not mention whether the reference standard results and the CT results were blinded to each other, and whether an appropriate interval was applied between CT and RT-PCR [31, 33, [36] [37] [38] [39] . Thus, there were unclear risk and/or unclear concern with respect to the index text, reference standard, and flow and timing. However, most studies included in the present meta-analysis did well in patient selection. Except for one of which was concluded unclear risk of bias [36] . Because it did not Table 2 shows the details. A high level of significant heterogeneity was noted by chi-squared values for SEN (P < 0.001), SPE (P < 0.001), LR+ (P < 0.001), DOR (P < 0.001), and AUC (P = 0.02). To distinguish possible sources of heterogeneity, meta-regression was conducted. Meta-regression analyses revealed that country and experience of radiologists (P = 0.02) accounted for part of the heterogeneity. While there was no evidence to show that the heterogeneity can be explained by study design and proportion of females. The results of the RDOR analysis are shown in Table 3 .

Deeks funnel plot was used to test publication bias. As shown in Fig. 5 , there was no evidence suggesting any potential publication bias (P = 0.79).

According to the 7th edition of the "2019 New Coronavirus Pneumonia Diagnosis and Treatment Program" issued by the National Health Commission of China, it combines the information of contact history, clinical manifestations shown by chest CT, presence of viral genes and virus-specific antibodies to diagnose COVID-19. Patients suspected of having COVID-19 symptoms and/or have a confirmed exposure history will be assessed by CT scan and RT-PCR test first. Patients with positive PCR test results are confirmed infected cases. Patients with a definite history of contact and showing typical CT imaging manifestations of viral pneumonia but negative PCR are reported to support COVID-19 clinical diagnosing [42, 43] . Chest CT has been recommended as a time-saving and reliable auxiliary examination for the detection of COVID-19 [17] . However, its diagnostic value has come under debate. In fact, a previous meta-analysis which included four original papers has been launched to analyze the role of chest CT for COVID-19 diagnosis [24] . Since then, several new studies investigating the diagnostic ability of chest CT scanning in COVID-19 identification have been published. We carried out this updated meta-analysis to systematically review the articles with regard to the diagnostic ability of chest CT for COVID-19. We analyzed the existing evidence and found that chest CT had a combined SEN of 0.94 and SPE of 0.49 when using RT-PCR as the reference standard. The moderate SPE indicates a high rate of misdiagnosis (51%). The relatively high SEN indicates a low missed diagnoses (6%), which is more important than misdiagnosis rate of a primary tool for COVID-19 detection. A normal chest CT scan negates the diagnosis in a majority of patients. A previous meta-analysis which included four eligible studies finds a rather high SEN (0.95) but poor SPE (0.09) of chest CT, using RT-PCR as the reference method [24] . With more original studies included, CT shows a good SEN and a concordantly higher SPE for diagnosing COVID-19 than previous studies. Meanwhile, we obtained an AUC of 0.93 from the SROC curve. On account of an AUC of 1.0 showing us perfect discrimination, our meta-analysis shows that the level of overall diagnostic accuracy is relatively high. The optimal AUC indicates that chest CT might be a suitable diagnostic tool for patients who are clinical suspected cases of COVID-19, especially those with negative initial RT-PCR results. However, in our study, the combined LR+ and LR− were moderate. The larger the LR+ is the higher the diagnostic accuracy will be, while the smaller the LR− is the higher the diagnostic accuracy will be. The present meta-analysis revealed a combined LR+ of 1.93. This is somewhat frustrating for clinical applications. Similarly, the combined LR− was 0.15, which is not low enough to make an excluded diagnosis in the clinic.

Patients from four countries were enrolled in our included studies. We compared the pooled diagnostic accuracy of studies carried out in China and Italy, and found CT for COVID-19 detecting performed better in Italian epidemic [22, 33] . We found this was in accordance with the experience of radiologists evaluated the images. The veteran chest radiologists would comprehensibly attain better SPE than those inexperienced. Studies with more experienced radiologists have more optimal diagnostic performance [18, 22, 33, 38] . There is good reason to suspect that two Italian studies performed better maybe because they adopted more experienced radiologists (15 and 25 years, > 10 years of experience) to evaluate the chest CT. The experience and skill of radiologist have affected the evaluation of diagnostic performance. The criteria of chest CT for COVID-19 detection are important for diagnostic accuracy. A high level of heterogeneity was found, we attempt to discriminate sources with meta-regression. Countries and experience of radiologists have an effect on diagnostic accuracy (P = 0.02).

Typical chest CT findings include peripherally distributed multifocal GGOs with patchy consolidations. CT findings can change as the disease progresses [1, 44] . SARS-CoV-2 first causing pulmonary interstitial damages and subsequent with parenchymal changes. GGO was believed to be the earliest radiographically visible CT findings in some patients and consolidation was considered as an indication of disease progression. As duration of the disease gets longer, the prevalence of reticulation could increase [44] [45] [46] . Varied findings and manifestations of chest CT were evaluated in included studies, for instance, GGO, consolidation, reticulation, thickened interlobular, septa, nodular lesions, traction bronchiectasis, bronchial wall thickening, subpleural bands, vascular enlargement. However, there is a paucity of details permitting corresponding sub-group analysis to determine their effects on diagnostic performance. Testing interval between CT and RT-PCR in seven studies were no more than a week, especially two within 24 h. [33, 35] Symptomatic patients may have lung abnormalities till the disease develops to a certain extent [47, 48] . And many symptomatic upper respiratory tract infections do not end up with pneumonia [49] . Those all made CT findings of COVID-19 a certain degree of empirical dependence.

Specimen type or location used for RT-PCR was different in included studies, including nasopharyngeal and/or oropharyngeal swab, sputum, urine stool. Four studies did not mention about the specimen type or location. A previous study by Abbas Mohammadi et al. suggests sputum sampling as a primary diagnosis and monitoring method of COVID-19 [50] . According to the 7th edition of the Diagnosis and Treatment Program of the 2019 New Coronavirus Pneumonia issued by the National Health Commission of China, multiple samples can be used for RT-PCR, such as nasopharyngeal swab, sputum, blood and urine and stool. Lower respiratory tract specimen is most recommended, sputum and endotracheal aspirate, for example. However, collection of lower respiratory tract samples often involves the production of aerosols and poses a high risk of virus transmission to staff, upper respiratory tract samples are more frequently used in practice. Only three studies [17, 18, 23] have sampled lower respiratory tract specimen for RT-PCR. In addition, it was indistinct whether all COVID-19 suspects with a negative initial RT-PCR test result were tested for repeated RT-PCR in two studies [34, 39] . These potential defects in the reference standard may have led to improper diagnosis of COVID-19 in a part of patients. It is the thing that urgently needs to be improved in the following research.

It is unavoidable to mention the limitations of this metaanalysis. First, after the rigorous literature search and study selection, only 15 publications were finally included. It is difficult to draw a definitive conclusion about the diagnostic capacity of chest CT due to the scant statistical power. The truth is that most studies currently available are impossible to extract SEN and SPE estimates due to insufficient data. Second, based on QUADAS-2 evaluation, the methodological quality of original studies was not so good. It was mainly because of insufficient data. Lacking of large sample size studies and high-quality studies which compare the chest CT in paired RT-PCR urgently calls for further studies. Third, although publication bias was not detected by statistical methods, it must be pointed out that due to language restrictions, we only take into English and Chinese articles.

Chest CT alone is not accurate enough for the diagnosis of COVID-19 infection. It should be taken into account as a diagnostic tool for the current COVID-19 detection, particularly for patients who show symptoms. The high SEN indicates that CT can be used as a quick tool to divide patients into "probably positive" and "probably negative" cohorts. The poor SPE indicates that it may be necessary to combine epidemiological characteristics, laboratory examinations and chest CT findings to confirm the presence of infection.

A novel coronavirus from patients with pneumonia in China

Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia

A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster

China National Health Commission (2020) 7th edition of the Diagnosis and Treatment Program of the 2019 New Coronavirus Pneumonia issued by the National Health Commission of China

Covid-19 and community mitigation strategies in a pandemic

Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med

World Health Organization (2020) World Health Organization Coronavirus disease (COVID-19) technical guidance: laboratory testing for 2019-nCoV in humans

European Centre for Diseases Prevention and Control (ECDC) (2020) An overview of the rapid test situation for COVID-19 diagnosis in the EU/EEA

The laboratory diagnosis of COVID-19 infection: current issues and challenges

In vitro diagnostic assays for COVID-19: recent advances and emerging trends

Diagnostic testing for the novel coronavirus

SARS-CoV-2 viral load in upper respiratory specimens of infected patients

Radiological findings from 81 patients with COVID-19 pneumonia in, China: a descriptive study

Emerging 2019 novel coronavirus (2019-nCoV) pneumonia. Radiology

World Health Organization website (2020) Laboratory testing for coronavirus disease 2019 (COVID-19) in suspected human cases: interim guidance

Chest CT findings in cases from the cruise ship "Diamond princess

Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases

Sensitivity of chest CT for COVID-19: comparison to RT-PCR

Clinical features and chest CT manifestations of coronavirus disease 2019 (COVID-19) in a single-center study in Shanghai, China

The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner Society

Can chest CT features distinguish patients with negative from those with positive initial RT-PCR results for coronavirus disease (COVID-19)?

Chest CT accuracy in diagnosing COVID-19 during the peak of the Italian epidemic: a retrospective correlation with RT-PCR testing and analysis of discordant cases

Diagnostic performance between CT and initial real-time RT-PCR for clinically suspected 2019 coronavirus disease (COVID-19) patients outside Wuhan, China

Can computed tomography be a primary tool for COVID-19 detection? Evidence appraisal through meta-analysis

Preferred reporting items for systematic reviews and meta analyses: the PRISMA statement

Cochrane diagnostic test accuracy reviews

Cochrane diagnostic test accuracy working group, systematic reviews of diagnostic test accuracy

QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies

Gefitinib or erlotinib vs chemotherapy for EGFR mutation-positive lung cancer: individual patient data meta-analysis of overall survival

Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing

Comparison of the clinical characteristics between RNA positive and negative patients clinically diagnosed with 2019 novel coronavirus pneumonia. Zhonghua Jie He He Hu Xi Za Zhi

Coronavirus disease 2019: initial detection on chest CT in a retrospective multicenter study of 103 chinese subjects

Chest CT features of COVID-19 in

Diagnostic performance of chest CT to differentiate COVID-19 pneumonia in non-high-epidemic area in Japan

Initial clinical features of suspected coronavirus disease 2019 in two emergency departments outside of Hubei China

Comparison of different samples for 2019 novel coronavirus detection by nucleic acid amplifcation tests

CT in relation to RT-PCR in diagnosing COVID-19 in the Netherlands: a prospective study

Diagnosis of the coronavirus disease (COVID-19): rRT-PCR or CT?

Differentiating pneumonia with and without COVID-19 using chest CT images: from qualitative to quantitative

Chest CT in COVID-19 patients: structured vs conventional reporting

Computer program] Version 5.2. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration

Novel Coronavirus Pneumonia Emergency Response Epidemiology Team (2020) The epidemiological characteristics of an outbreak of 2019 novel coronavirus diseases (COVID-19) in China

COVID-19 diagnostic process in mainland China: the math beyond pneumonia

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): a pictorial review

Computed tomography of the chest in patients with COVID-19: what does radiologists want to know?

Interstitial lung fibrosis imaging reporting and data system: what radiologist wants to know?

Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection

Systematic review and metaanalysis on the value of chest CT in the diagnosis of coronavirus disease (COVID-19): Sol Scientiae

China medical treatment expert group for COVID-19 clinical characteristics of coronavirus disease 2019 in China

SARS-CoV-2 detection in different respiratory sites: a systematic review and meta-analysis