key: cord-1012535-u1o2u5ku authors: Kao, Yung-Shuo; Lin, Kun-Te title: A Meta-Analysis of Computerized Tomography-Based Radiomics for the Diagnosis of COVID-19 and Viral Pneumonia date: 2021-05-29 journal: Diagnostics (Basel) DOI: 10.3390/diagnostics11060991 sha: be91df27f1dbd41064adcb7177a9ef25f38b4a54 doc_id: 1012535 cord_uid: u1o2u5ku Introduction: Coronavirus disease 2019 (COVID-19) led to a global pandemic. Although reverse transcription polymerase chain reaction (RT-PCR) of viral nucleic acid is the gold standard for COVID-19 diagnosis, its sensitivity was found to not be high enough in many reports. As radiomics-based diagnosis research has recently emerged, we aimed to use computerized tomography (CT)-based radiomics models to differentiate COVID-19 pneumonia from other viral pneumonia infections. Materials and methods: This study was performed according to the preferred reporting items for systematic review and meta-analysis diagnostic test accuracy studies (PRISMA-DTA) guidelines. The Pubmed, Cochrane, and Embase databases were searched. The pooled sensitivity and pooled specificity were calculated. A summary receiver operating characteristic (sROC) curve was constructed. The study quality was evaluated based on the radiomics quality score. Results: A total of 10,300 patients were involved in this meta-analysis. The radiomics quality score ranged from 13 to 16 (maximum score: 36). The pooled sensitivity was 0.885 (95% CI: 0.818–0.929), and the pooled specificity was 0.811 (95% CI: 0.667–0.902). The pooled AUC was 906. Conclusion: Our meta-analysis showed that CT-based radiomics feature models can successfully differentiate COVID-19 from other viral pneumonias. Coronavirus disease 2019 (COVID- 19) led to a global pandemic featuring a highly contagious disease which has led to millions of deaths worldwide. Although reverse transcription polymerase chain reaction (RT-PCR) of viral nucleic acid is the gold standard for COVID-19 diagnosis [1] , its sensitivity was found to not be high enough in many reports [2] [3] [4] . On the other hand, chest x-ray (CXR) and computerized tomography (CT) are helpful in the diagnosis of COVID-19 pneumonia [5] [6] [7] . According to recent experience, lung image findings are observed earlier than clinical manifestations, highlighting the importance of image exams for screening pneumonias [8] . Early diagnosis can also reduce disease transmission risk and prevent the endemic of COVID-19 [9] . CXR is convenient, easily accessible, and can avoid cross contamination between users. Although the sensitivity of CXR is lower than CT in diagnosing COVID-19 pneumonia. With artificial intelligence machine learning model assistance, CXR can achieve an improved COVID-19 diagnostic rate [10] [11] [12] . However, the power of generalization is low when the machine learning model was trained on sets of heterogeneous images. The absence of an adequate evaluation protocol also limited the artificial intelligence assistance in CXR and its utility in clinical settings [13] . Nonetheless, CXR is useful to monitor disease progression in unstable patients. In previous studies, computerized tomography (CT) had noninferior sensitivity to the RT-PCR test in diagnosing COVID-19 pneumonia [2, 5] . However, CT images have an 1. Articles written in English; 2. Full text available. The exclusion criteria were as follows: 1. Studies using only deep learning features; 2. Conference papers or studies with only the abstract available. In the selected studies, the radiomics-based machine learning process was similar. First, the CT images acquisition and region-of-interest segmentation were performed by radiologists. Next, the radiomics feature extraction, model training and cross validation were performed by artificial intelligence. Finally, the data analysis and clinical application was evaluated by clinicians. The main outcome was set as the highest area under the curve (AUC) in the validation dataset. In the absence of an external validation set, we chose the value from the cross-validation process or hold-out dataset. We also extracted the true positive (TP), false positive (FP), true negative (TN), and false negative (FN) values from the literature. Furthermore, we gathered the name of the first author, the nation of the first author, the publication year of the study, the region of interest (ROI), the patient number, and other characteristics from the selected studies. Pooled analysis was carried out using a random effects model. The pooled sensitivity and pooled specificity were calculated. A forest plot and summary receiver operating characteristic (ROC) plot were created. The heterogeneity was assessed using the chi-square test. The statistical analysis was performed with the R language using R studio. The publication bias was assessed using a funnel plot. Egger's test was only conducted if the number of included articles was more than 10. The statistical analysis was performed with the R language using R studio. The RQS (radiomics quality score) was used to evaluate the chosen studies [29] . Two authors independently scored the table. Any inconsistencies between the authors were resolved by consensus. First, we collected a total of 348 articles. After duplicate removal, 283 publications were selected for meticulous evaluation. After assessment of the title and abstracts, ten publications were selected, and their full texts were retrieved. One observational study [30] without radiomics application in the research and two observational studies [31, 32] with a repetitive patient population were excluded. After the article selection process, seven articles were used in the qualitative analysis [33] [34] [35] [36] [37] [38] [39] , and six articles were further used in the meta-analysis. The PRISMA flowchart is provided in Figure 1 . Details of the selected studies are listed in Table 1 . Only the six studies were used in the meta-analysis [33] [34] [35] [36] [37] [38] . Wang's study [39] was excluded due to the predictions being made on the basis of the CT slice number. A total of 10,300 patients were involved in this meta-analysis. Among them, 3587 patients had COVID-19 pneumonia. The RQS table is provided in Table 2 . The score range of included studies was 13 to 16 (maximum score: 36). None of the evaluated articles performed a phantom study, discussed biological correlates, conducted the study in a prospective design, or performed a cost-effectiveness analysis. The radiomics-based machine learning process workflow was shown in Figure 2 . The RQS table is provided in Table 2 . The score range of included studies was 13 to 16 (maximum score: 36). None of the evaluated articles performed a phantom study, discussed biological correlates, conducted the study in a prospective design, or performed a cost-effectiveness analysis. The radiomics-based machine learning process workflow was shown in Figure 2 . Only six studies were included in the meta-analysis. The pooled sensitivity was 0.885 (95% CI: 0.818-0.929), and the pooled specificity was 0.811 (95% CI: 0.667-0.902). The heterogeneity of sensitivity was low (p = 0.434), whereas the heterogeneity of specificity was high (p = 0.000661). The pooled AUC was 0.906. The forest plots for sensitivity and specificity are provided in Figure 3 . The sROC curve is shown in Figure 4 . Only six studies were included in the meta-analysis. The pooled sensitivity was 0.885 (95% CI: 0.818-0.929), and the pooled specificity was 0.811 (95% CI: 0.667-0.902). The heterogeneity of sensitivity was low (p = 0.434), whereas the heterogeneity of specificity was high (p = 0.000661). The pooled AUC was 0.906. The forest plots for sensitivity and specificity are provided in Figure 3 . The sROC curve is shown in Figure 4 . As shown in Figure 5 , the publication bias was assessed using a funnel plot. The funnel plot result was symmetrical, indicating no obvious publication bias in this meta-analysis. According to the International Symposium on Biomedical Imaging (ISBI), radiomics features can be classified as shape-based features, first-order features, gray-level run-length matrix (GLRM) features, gray-level co-occurrence matrix (GLCM) features, gray-level distance-zone matrix (GLDZM) features, gray-level size-zone matrix (GLSZM) features, neighborhood gray tone difference matrix (NGTDM) features, or neighboring gray-level dependence matrix (NGLDM) features [40] . Three studies used shape-based As shown in Figure 5 , the publication bias was assessed using a funnel plot. The funnel plot result was symmetrical, indicating no obvious publication bias in this meta-analysis. As shown in Figure 5 , the publication bias was assessed using a funnel plot. The funnel plot result was symmetrical, indicating no obvious publication bias in this meta-analysis. According to the International Symposium on Biomedical Imaging (ISBI), radiomics features can be classified as shape-based features, first-order features, gray-level run-length matrix (GLRM) features, gray-level co-occurrence matrix (GLCM) features, gray-level distance-zone matrix (GLDZM) features, gray-level size-zone matrix (GLSZM) features, neighborhood gray tone difference matrix (NGTDM) features, or neighboring gray-level dependence matrix (NGLDM) features [40] . Three studies used shape-based According to the International Symposium on Biomedical Imaging (ISBI), radiomics features can be classified as shape-based features, first-order features, gray-level run-length matrix (GLRM) features, gray-level co-occurrence matrix (GLCM) features, gray-level distance-zone matrix (GLDZM) features, gray-level size-zone matrix (GLSZM) features, neighborhood gray tone difference matrix (NGTDM) features, or neighboring gray-level dependence matrix (NGLDM) features [40] . Three studies used shape-based features, while all studies used first-order and second-order features. The details of used features are listed in Table 3 . The number of the studies in which the radiomics type was used was shown in Figure 6 . features, while all studies used first-order and second-order features. The details of used features are listed in Table 3 . The number of the studies in which the radiomics type was used was shown in Figure 6 . First-order, GLCM Lesion distribution, pleural effusion, maximum lesion range, mediastinal and hilar lymph node enlargement, Huang [36] China, 2020 Shape-based, first-order, GLCM, GLDM *, GLSZM, GLRM Halo sign, ground glass opacity (GGO), intralobular interstitial thickening (IIT) Chen [37] China, 2020 Shape-based, first-order, GLSZM Number of lesions with pleural thickening, white blood cell count, platelet count, number of lesions with crazy paving appearance Liu [38] China 2021 first order, GLCM, GLDM*, GLRM age, lesion distribution, neutrophil ratio, CT score, lymphocyte count Note: GLRM, gray-level run-length matrix; GLCM, gray-level co-occurrence matrix; GLDZM, gray-level distance-zone matrix; GLSZM, gray-level size-zone matrix; NGTDM, neighborhood gray tone difference matrix; NGLDM, neighboring gray-level dependence matrix. * The gray-level dependence matrix (GLDM) is not listed by the International Symposium on Biomedical Imaging (ISBI). Four studies used least absolute shrinkage and selection operator (LASSO) regression, one study used logistic regression, and one study used support vector machine (SVM) models with a radial basis function kernel. The details of the used models are listed in Table 4 . Four studies used least absolute shrinkage and selection operator (LASSO) regression, one study used logistic regression, and one study used support vector machine (SVM) models with a radial basis function kernel. The details of the used models are listed in Table 4 . This meta-analysis is the first to explore CT-based radiomics features for the differentiation of COVID-19 from other viral pneumonias. The advantage of conducting this meta-analysis is that a large number of patients were included. A total of 10,300 patients were included in this meta-analysis, increasing the robustness of our results. The prediction performance was fascinating with a pooled AUC of 0.906. Among the studies that included viral pneumonia comparisons, two studies included only influenza infections [33, 34] . Other studies included influenza and other viral pneumonia infections [35] [36] [37] [38] [39] . Considering that influenza also represents a highly contagious disease with a high prevalence of adult viral pneumonias [41, 42] . We included the two studies which compared COVID-19 with influenza pneumonia in this meta-analysis [33, 34] . The sensitivity of RT-PCR for COVID-19 diagnosis varied from 59 to 71% depending on viral load and test sample quality [2, 5] . That is to say, an RT-PCR negative result was still shown in some COVID-19 infected patients [5, 43] . Therefore, chest CT played a crucial role in the early diagnosis of COVID-19 pneumonia for the RT-PCR negative patients [44] . The identification of COVID-19 pneumonia on chest CT depended on radiologists' interpretation. However, radiologists qualified only moderate sensitivity in differentiation COVID-19 from other viral pneumonia on chest CT [14] . With artificial intelligence assistance, radiologists achieved higher sensitivity in diagnosis of COVID-19 pneumonia [3, 45] . In this meta-analysis comparing COVID-19 with other viral pneumonia under CT-based radiomics assistance, the pooled sensitivity was 0.885. The radiomics quality scores in the included studies ranged from 13 to 16 points. The maximum RQS is 36 points. However, none of the collected studies were conducted prospectively, which led to a loss of 7 points. Thus, future studies should be conducted prospectively to achieve better-quality results. Among the six studies included in the meta-analysis, four of them used LASSO regression. LASSO regression is a commonly used feature selection algorithm in the data science discipline. It is a logistic regression method with L1 regularization, which renders the prediction model more prone to noise, thus increasing its robustness [46] . One study used traditional logistic regression, whereas another used SVM. The SVM algorithm works well in the high-dimensional space, making it popular in machine learning. First-order features, shape-based features, and second-order features were used in the prediction models. The power of radiomics features was displayed in many cancers [47] [48] [49] . However, this meta-analysis showed that radiomics features are useful not only in cancer, but also in other diseases, such as COVID-19 pneumonia. The limitation of this meta-analysis is that all the studies were retrospective and conducted in China. However, there was no other suitable article that met the inclusion criteria in the three databases (Pubmed, Cochrane Library, and Embase) search process by two authors. One selected article was conducted in China but some of the involved patients were collected in the USA [39] . In the future, prospective and multinational studies should be conducted to validate the effectiveness of radiomics in COVID-19 detection using CT scans. Our meta-analysis showed that CT-based radiomics feature models can successfully be used to differentiate COVID-19 from other viral pneumonias. Informed Consent Statement: Patient consent was waived due to the chosen studies that did not involve patient consent. Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR Artificial Intelligence Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Origin at Chest CT Positive rate of RT-PCR detection of SARS-CoV-2 infection in 4880 cases from one hospital in wuhan, china Correlation of Chest CT and RT-PCR Testing for Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV) Determinants of chest x-ray sensitivity for covid-19: A multi-institutional study in the united states Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): A study of 63 patients in Wuhan, China Effect of delay in diagnosis on transmission of COVID-19 Machine-learning classification of texture features of portable chest X-ray accurately classifies COVID-19 lung infection An automated machine learning model to assist in the diagnosis of COVID-19 infection in chest X-ray images Deep-learning convolutional neural networks with transfer learning accurately classify COVID-19 lung infection on portable chest radiographs Current limitations to identify COVID-19 using artificial intelligence with chest X-ray imaging Diagnostics 2021 Performance of Radiologists in Differentiating COVID-19 from Non-COVID-19 Viral Pneumonia at Chest CT Applications and limitations of radiomics The Applications of Radiomics in Precision Diagnosis and Treatment of Oncology: Opportunities and Challenges. Theranostics The study of automatic machine learning base on radiomics of non-focus area in the first chest CT of different clinical types of COVID-19 pneumonia A CT-based radiomics nomogram for predicting prognosis of coronavirus disease 2019 (COVID-19) radiomics nomogram predicting COVID-19 Machine learning-based CT radiomics method for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study Differentiating novel coronavirus pneumonia from general pneumonia based on machine learning Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy Incidence of medically attended influenza during pandemic and post-pandemic seasons through the Influenza Incidence Surveillance Project Seasonality of Respiratory Viral Infections Global patterns in monthly activity of influenza virus, respiratory syncytial virus, parainfluenza virus, and metapneumovirus: A systematic analysis Relation Between Chest CT Findings and Clinical Conditions of Coronavirus Disease (COVID-19) Pneumonia: A Multicenter Study Radiomics: Images Are More than Pictures, They Are Data Discrimination of pulmonary groundglass opacity changes in COVID-19 and non-COVID-19 patients using CT radiomics analysis Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping End-to-end automatic differentiation of the coronavirus disease 2019 (COVID-19) from viral pneumonia based on chest CT CT-based radiomics combined with signs: A valuable tool to help radiologist discriminate COVID-19 and influenza pneumonia Multi-classifier-based identification of COVID-19 from chest computed tomography using generalizable and interpretable radiomics features Radiomics-based model for accurately distinguishing between severe acute respiratory syndrome associated coronavirus 2 (SARS-CoV-2) and influenza A infected pneumonia Development and evaluation of an artificial intelligence system for COVID-19 diagnosis Radiomics nomogram for the prediction of 2019 novel coronavirus pneumonia caused by SARS-CoV-2 CT-based radiomics combined with signs: A valuable tool to help physician discriminate COVID-19 and other viral pneumonia Machine learning-based CT radiomics model distinguishes COVID-19 from other viral pneumonia CT radiomics facilitates more accurate diagnosis of COVID-19 pneumonia: Compared with CO-RADS Decoding COVID-19 pneumonia: Comparison of deep learning and radiomics CT image signatures Regression Shrinkage and Selection Via the Lasso Epidemiology of Viral Pneumonia. Clin Viral infection in community-acquired pneumonia: A systematic review and meta-analysis Chest CT in COVID-19 pneumonia: Correlations of imaging findings in clinically suspected but repeatedly RT-PCR test-negative patients Early chest CT features of patients with 2019 novel coronavirus (COVID-19) pneumonia: Relationship to diagnosis and prognosis corona: Radiologist-assistant deep learning framework for COVID-19 diagnosis in chest CT scans Radiomics: The bridge between medical imaging and personalized medicine Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma-a systematic review and meta-analysis A systematic review and meta-analysis of the prognostic value of radiomics based models in non-small cell lung cancer treated with curative radiotherapy Analysis for Using Radiomics to Predict Complete Pathological Response in Esophageal Cancer Patients Receiving Neoadjuvant Chemoradiation Data Availability Statement: All the data used were presented in the article. The authors declare no conflict of interest.