key: cord-0124494-qmjcpc8m
authors: Liu, Mengqiu; Liu, Ying; Yang, Yidong; Liu, Aiping; Li, Shana; Qu, Changbing; Qiu, Xiaohui; Li, Yang; Lv, Weifu; Zhang, Peng; Wen, Jie
title: Automated assessment of disease severity of COVID-19 using artificial intelligence with synthetic chest CT
date: 2021-12-11
journal: nan
DOI: nan
sha: 16ee6fba96db84a561cc6c7fc81e1a93520fb16b
doc_id: 124494
cord_uid: qmjcpc8m

Background: Triage of patients is important to control the pandemic of coronavirus disease 2019 (COVID-19), especially during the peak of the pandemic when clinical resources become extremely limited. Purpose: To develop a method that automatically segments and quantifies lung and pneumonia lesions with synthetic chest CT and assess disease severity in COVID-19 patients. Materials and Methods: In this study, we incorporated data augmentation to generate synthetic chest CT images using public available datasets (285 datasets from"Lung Nodule Analysis 2016"). The synthetic images and masks were used to train a 2D U-net neural network and tested on 203 COVID-19 datasets to generate lung and lesion segmentations. Disease severity scores (DL: damage load; DS: damage score) were calculated based on the segmentations. Correlations between DL/DS and clinical lab tests were evaluated using Pearson's method. A p-value<0.05 was considered as statistical significant. Results: Automatic lung and lesion segmentations were compared with manual annotations. For lung segmentation, the median values of dice similarity coefficient, Jaccard index and average surface distance, were 98.56%, 97.15% and 0.49 mm, respectively. The same metrics for lesion segmentation were 76.95%, 62.54% and 2.36 mm, respectively. Significant (p<<0.05) correlations were found between DL/DS and percentage lymphocytes tests, with r-values of -0.561 and -0.501, respectively. Conclusion: An AI system that based on thoracic radiographic and data augmentation was proposed to segment lung and lesions in COVID-19 patients. Correlations between imaging findings and clinical lab tests suggested the value of this system as a potential tool to assess disease severity of COVID-19.

Up to September 25, 2020, 32,110 ,656 coronavirus disease 2019 (COVID- 19) cases have been reported worldwide, and 980,031 cases were confirmed death, resulting a fatal rate up to 3.05% (1) . Although over 169 COVID-19 vaccine candidates are under development, with 26 of them in the human trial phase, efficient diagnosis and assessment are crucial to tracking the disease and mitigating its spread.

Currently, Reverse Transcription-Polymerase Chain Reaction (RT-PCR) test is considered as the gold standard of COVID-19 diagnosis. (2) However, the sensitivity of RT-PCR assay was reported to be as low as 60-89%, (3) (4) (5) which might lead to false negative diagnosis, especially in the early phase of infection. On the other hand, imaging techniques, such as chest X-ray and thoracic CT, have been demonstrated to have improved sensitivity compared to RT-PCR, and can provide huge assistance to clinicians. A recently study (6) conducted in France showed that, chest CT sensitivity and specificity for COVID-19 diagnosis can reach up to 90%, and, as high as 90% of CT-positive and RT-PCR-negative COVID-19 patients had later been confirmed by positive repeat RT-PCR results.

Severity assessment is crucial for patient triage and treatment planning, and can help to optimize the allocation of limited clinical resources, particularly during the peak of the pandemic. (7) (8) (9) (10) (11) (12) (13) Lymphopenia is a potential biomarker for disease severity, which is known to be prevalent among patients with COVID-19. Previous studies have reported that peripheral blood lymphocyte cell count decline is predictive of disease severity and death. (7) (8) (9) (10) (11) (12) 14) It was also shown that COVID-19 patients with lymphopenia could benefit from treatment with recombinant human granulocyte colony-stimulating factor (rhG-CSF). (15) Since the outbreak of the pandemic, artificial intelligence (AI) techniques have been used actively in fighting COVID-19, such as disease diagnosis, severity assessment and so on. Lung and lesion segmentation is usually an essential step in disease severity assessment of COVID-19. Currently, many popular networks -such as U-Net (16) , UNet++ (17) and VB-Net (18) -have been used for lung and lesion segmentation. However, considerable large datasets with manual annotation are required to train a robust segmentation network, which is labor-intensive and timeconsuming. Previously, several studies (18, 19) incorporated human knowledge into the training process. This human-in-the-loop strategy, together with other weaklysupervised machine learning methods (20, 21) , are highly demanded when annotated medical images are lacking in COVID-19 studies.

Here we propose an AI system that can efficiently segment lung regions and COVID-19 pneumonia lesions, namely ground glass opacities and consolidations. By using data augmentation techniques, we were able to create "pseudo" lesions as labels in the lung regions on public available CT datasets. This minimizes the need for large number of COVID-19 data and labor intensive manual annotations. It also allows us to quantify disease severities in COVID-19 patients. The disease severity scores calculated from CT images showed significant correlations with percentage lymphocytes (LYM%), indicating the reliability of these scores as useful measures for disease severity assessment.

As a retrospective study, it was approved by the local ethics committees to waive the informed consent of the patients. A total number of 203 lung CT datasets were collected from multiple hospitals in Anhui province, China. All images were collected from COVID-19 patients with positive RT-PCR test. CT scans were collected on scanners from different manufacturers. Lab tests, including white blood cell count (WBC) and percentage lymphocytes (LYM%), were also collected for each patient.

Detailed information of demographic, CT scan parameters and lab test results are shown in Table 1 . Public available lung CT datasets (LUNA16 -Lung Nodule Analysis 2016) were also used in this study. To minimize the influence of pulmonary nodules on our analysis, we removed all the datasets that had pulmonary nodules > 3mm, which retained a total number of 285 datasets collected on scanners with different manufacturers and slice thicknesses. These images were then used to generate synthetic chest CT with pseudo lesions inside the lung regions and mimicked the abnormalities in COVID-19 cases. Figure 1 Overview of the AI system used for lung and lesion segmentation. A. "Normal"

CT images from LUNA16 dataset were used to generate synthetic CT with "pseudo"

lesions. Synthetic CT images and original LUNA16 labels were used to train a standard 2D U-net. Output model was then applied on COVID-19 images to generate "lung" masks -M. B. "Healthy" lung tissue masks were also generated when creating the synthetic CT images, and used as ground truth to train a standard 2D U-net. Output model was then applied on COVID-19 images to produce "healthy" lung tissue masks -Mht. C. "Pulmonary air" masks -Ma -were also generated based on magnitude thresholding, as described in the main text. Original LUNA16 images and generated Ma were used to train a standard 2D U-net, which was then tested on COVID-19 images.

All three masks were combined together to create fine segmentations of lungs and lesions, which were later used for disease severity assessment.

An Overview of the AI system is shown in Figure 1 . "Normal" (with nodule diameters < 3mm) CT images (n=285) from LUNA16 dataset were used to generate synthetic CT images with "pseudo" pneumonia lesions. First of all, a "seed" was placed on the boundary between lung and tissue, and used as a starting point. Secondly, a "random walk" strategy was used to spread the "lesion" area. "Lesion" voxels were randomly assigned CT values between -650 HU and -180 HU to mimic COVID-19 pneumonia lesions. The number of walking steps were limited not to exceed the size of the lung masks. A mask of "healthy" lung tissue -Mht -was also generated for each dataset. The generated images and masks (as shown in Figure 2 ) were then used for Unet training. "Normal" LUNA16 images were also used to generate "pulmonary air" masks -

For each 3D CT dataset, a histogram of CT values in the whole lung regions was plotted ( Figure 3) . The upper half of the highest peak of the distribution was fitted to a normal distribution. The fitting parameters μ (mean) and σ (standard deviation) were used to calculate a threshold = μ+2.58σ. A binary "pulmonary air" mask was created by setting all lung regions that had CT values smaller than the threshold to 1. 

where M, Mht and Ma represent masks of the whole lung, "healthy" lung tissue and "pulmonary air", respectively. The segmentations were then used in disease severity assessment.

Algorithm-generated segmentations were compared with manual annotations. To reduce the heavy annotation burden, we employed a human-in-the-loop strategy, similar to what other researchers did in (18, 22) . Three individuals performed refinements to algorithm-generated results to produce manual annotations. Dice similarity coefficients (DSC), Jaccard index (JI) and average surface distance (ASD) were calculated to evaluate the performance of the automated segmentations. The evaluation metrics were defined as follows: (4) where A and B represent regions of automated and manual segmentations, respectively.

Two abnormality measures were developed: (5) where DL and DS represent damage load and damage score, respectively. VL is the volume of the lesions and V represents volume of the entire lung. MCTL represents median CT HU values calculated from the lesion regions.

All statistical analyses were performed using MATLAB (The MathWorks, Inc).

calculated. In all cases, a p-value < 0.05 was considered as statistically significant.

Results of lung segmentation are shown in Figure 4 . Red contours indicate segmentations obtained using a U-net trained on the original LUNA16 images, which introduced significant segmentation errors. On the contrary, blue contours illustrated segmentations obtained using a U-net trained on the synthetic images with generated "pseudo" lesions. Adding this data augmentation step greatly reduced errors and improved the performance compared with manual annotations. Achieved median values of DSC, JI and ASD were 98.56%, 97.15% and 0.49 mm, respectively. Figure 4 Illustration of the performance of lung segmentation method that used a U-net trained on original LUNA16 images without (red contours) and with "pseudo" lesions (blue contours). Images were collected from 8 different COVID-19 patients.

Lesion segmentations are illustrated in Figure 5 , as marked by the red contours. The same images were used here as in Figure 4 .

A WBC count can detect hidden infections within the body and Lymphopenia is known to be prevalent among patients with COVID-19. Previous studies have reported that peripheral blood lymphocyte cell count decline is predictive of disease severity and death. (7) (8) (9) (10) (11) (12) 14) Our results showed that DL and DS scores that were evaluated from thoracic CT scans correlated with LYM% measures, indicating that DL and DS can be used as potential measures for disease severity assessment in COVID-19 patients. 

In this study, we proposed an AI system that can provide accurate segmentations of lung and COVID-19 related pneumonia lesions which appear as ground glass opacities and consolidations. By using data augmentation techniques, we were able to create "pseudo" lesions in the lung regions on public available CT datasets. This strategy minimizes the need for large number of COVID-19 training samples and labor intensive manual annotation. The auto-segmentations can be directly used to quantify disease severities in COVID-19 patients.

AI techniques provide an efficient and accurate way for patient triage and disease diagnosis, and can help to optimize the allocation of medical resources. Fast and precise assessment of disease severity and progression allows physicians to quickly determine treatment strategies and is the most important step in ensuring the quality of follow-up treatment. Automatic segmentation of lung and lesion regions is an important initial step for rapid assessment of pulmonary infections and disease progression in COVID-19 patients. Although AI has been used to generate lung and lesion segmentations in patients with COVID-19, most of the previous applications heavily rely on training data that require labor-intensive manual annotations. The manual contouring process is very time consuming and there exists considerable inter-physician variations heavily depending on personal experience. Therefore, it is almost impossible to obtain a unified and objective evaluation.

The contribution of this study is that we introduced data augmentation techniques to generate "pseudo" lesions to mimic COVID-19 abnormalities on the CT images using public available dataset -LUNA16. This greatly reduced the need of manual annotations and allowed the neural network to be trained on existing public datasets.

The results showed that the proposed automatic system used in this study can achieve satisfying segmentations of lung and lesion, which is comparable with existing methods that requrie manual annotations in their training data. The correlations between disease severity scores and clinical lab tests further indicate that this system can provide measures for disease severity assessment.

There is still room for improvement in the proposed system in the future. In current study, we used a "random walk" strategy to generate "pseudo" lesions. Although comparison with manual annotations has validated this strategy, more advance technique, such as semi-supervised learning, (25, 26) can be used to further minimize the distribution discrepancy between synthetic and real datasets.

In conclusion, we proposed an AI system that can efficiently segment lung regions and COVID-19 related pneumonia lesions. By using data augmentation in the training procedure, we were able to create synthetic chest CT images with "pseudo" lesions to mimic COVID-19 abnormalities and avoid the requirements of labor-intensive manual annotations. The disease severity scores calculated based on auto-segmentations were used to quantify disease severity and showed significant correlations with percentage lymphocytes (LYM%) tests, indicating that this system can provide potential measures for disease severity assessment in COVID-19.

Weekly Operational Update on COVID-19

Correlation of Chest CT and RT-PCR Testing for Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases

Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR

Essentials for Radiologists on COVID-19: An Update-Radiology Scientific Expert Panel

Diagnostic Performance of CT and Reverse Transcriptase Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis

Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China

China Medical Treatment Expert Group for C. Clinical Characteristics of Coronavirus Disease

Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study

Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China

Model-based Prediction of Critical Illness in Hospitalized Patients with COVID-19

Lymphopenia predicts disease severity of COVID-19: a descriptive and predictive study

Effect of Recombinant Human Granulocyte Colony-Stimulating Factor for Patients With Coronavirus Disease 2019 (COVID-19) and Lymphopenia: A Randomized Clinical Trial

Convolutional Networks for Biomedical Image Segmentation

UNet++: A Nested U-Net Architecture for Medical Image Segmentation

Lung Infection Quantification of COVID-19 in CT

Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study

Deep Learningbased Detection for COVID-19 from Chest CT using Weak Label

A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT

A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images

Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images

Serial Quantitative Chest CT Assessment of COVID-19: Deep-Learning Approach

Semi-supervised Transfer Learning for Image Rain Removal

Syn2Real Transfer Learning for Image Deraining using Gaussian Processes. arXiv 2020