key: cord-0102429-h8k1jur8
authors: Wang, Yixin; Zhang, Yao; Liu, Yang; Tian, Jiang; Zhong, Cheng; Shi, Zhongchao; Zhang, Yang; He, Zhiqiang
title: Does Non-COVID19 Lung Lesion Help? Investigating Transferability in COVID-19 CT Image Segmentation
date: 2020-06-23
journal: nan
DOI: nan
sha: ce871c293416eb182101cf0af068eab1008f1765
doc_id: 102429
cord_uid: h8k1jur8

Coronavirus disease 2019 (COVID-19) is a highly contagious virus spreading all around the world. Deep learning has been adopted as an effective technique to aid COVID-19 detection and segmentation from computed tomography (CT) images. The major challenge lies in the inadequate public COVID-19 datasets. Recently, transfer learning has become a widely used technique that leverages the knowledge gained while solving one problem and applying it to a different but related problem. However, it remains unclear whether various non-COVID19 lung lesions could contribute to segmenting COVID-19 infection areas and how to better conduct this transfer procedure. This paper provides a way to understand the transferability of non-COVID19 lung lesions. Based on a publicly available COVID-19 CT dataset and three public non-COVID19 datasets, we evaluate four transfer learning methods using 3D U-Net as a standard encoder-decoder method. The results reveal the benefits of transferring knowledge from non-COVID19 lung lesions, and learning from multiple lung lesion datasets can extract more general features, leading to accurate and robust pre-trained models. We further show the capability of the encoder to learn feature representations of lung lesions, which improves segmentation accuracy and facilitates training convergence. In addition, our proposed Multi-encoder learning method incorporates transferred lung lesion features from non-COVID19 datasets effectively and achieves significant improvement. These findings promote new insights into transfer learning for COVID-19 CT image segmentation, which can also be further generalized to other medical tasks.

I N December 2019, the coronavirus disease 2019 (COVID- 19) broke out and has become a global challenge since then. This new virus was spreading rapidly, affecting countries worldwide. Up to 13 June 2020, 7,533,177 identified cases of COVID-19 have been reported in over 216 countries and territories, resulting in 422,770 deaths. This severe disease has been declared as a Public Health Emergency of International Concern by the World Health Organization (WHO).

A gold standard method to screen the COVID-19 patients is the real-time reverse transcription-polymerase chain reaction(rRT-PCR) [1] , which requires inconvenient repeated tests. Researchers found that chest computed tomography (CT) images show strong ability to capture ground glass and bilateral patchy shadows which are typical CT features in affected patients [2] . Thus, chest CT can help identify distinguishing patterns and features in patients. Given that traditional CT imaging analysis methods are time-consuming and laborious, it is of great importance to develop artificial intelligence (AI) systems to aid COVID-19 diagnosis [3] .

Segmentation in CT slices is a key component of the diagnostic work-up for patients with COVID-19 infection in clinical practice [4] . Recently, deep learning with CNNs has showed significant performance improvements on the automatic detection and automatic extraction of essential features from CT images, related to the diagnosis of the Covid-19. Though deep learning has made great progress in medical image segmentation, it remains a challenging task in the field of COVID-19 lung infection, as existing public datasets on COVID-19 are relatively small and weakly labelled. Thus, training deep networks from scratch with inadequate data and task-specific nature such as COVID-19 infection may lead to over-fitting and poor generalization.

Transfer learning is an effective method to solve this problem, which helps to leverage knowledge and latent features from other datasets and avoids over-parameterization. In transfer learning, successful deep learning models such as ResNet, DenseNet and GoogLeNet have been trained on large datasets such as ImageNet. These pre-trained models have proven impressive performance on natural image downstream tasks. However, there exists no large-scale annotated medical image datasets as data acquisition is difficult, and high-quality annotations are expensive. Recent research [5] shows that transfer learning from natural image datasets to medical tasks produces very limited performance gain. In particular, though with large medical datasets for pre-training, transfer learning in the task of COVID-19 infection segmentation is still much more difficult: 1) The shape, texture and position of COVID-19 infections are in high variation. 2) Existing large medical CT datasets differ in domains with COVID-19 datasets. Thus, similar in domain, a pre-trained model from lung lesions may share more knowledge with COVID-19 infection and learn some general-purpose visual representations for lung lesions.

Evidence shows that larger datasets are necessarily better for pre-training and the diversity of datasets is extremely important [6] . In medical domain, pre-training from medical datasets, especially chest CT datasets tends to be more homogeneous compared to non-medical and other medical areas' data. Thus, non-COVID19 lung lesion CT imaging manifestations may serve as potential profit for COVID-19 segmentation. Existing works have proven that multi-task training through simply fusing different lesions with COVID-19 affects model's representation ability for COVID-19 infection segmentation [7] . Therefore, with limited COVID-19 datasets, 1) whether these non-COVID19 lesions help and to what extent they can contribute to COVID-19 infection segmentation? 2) How to train a better pre-trained model using these non-COVID19 datasets for transfer learning? 3) In what manners can the pre-trained models fitted on non-COVID19 lesions be effectively transferred to In this paper, we aim to answer the above questions which are significant for COVID-19 segmentation. To our best knowledge, this is the first study to explore the transferability of non-COVID19 datasets for COVID-19 CT images segmentation.

Our contributions are as follows.

• We experimentally assess the extent of contributions from non-COVID19 to COVID-19 infection segmentation. We found that despite the disparity between non-COVID19 lung lesion images and COVID-19 infection images, pretraining on the large scale well-annotated lung lesions may still be transferred to benefit COVID-19 recognition and segmentation.

• We conduct extensive experiments using various non-COVID19 lung lesion datasets. Although pre-training on a single non-COVID19 dataset is unstable among different transfer strategies, learning from different non-COVID19 lesions demonstrates promising performance since such a multi-task learning process can share knowledge from different related tasks and discover common and general representations of lung lesions.

• We design four different transfer learning strategies with various performance gains and training time costs: Continual Learning, Body Fine-tuning, Pre-trained Lesion Representations and Multi-encoder Learning. The Multi-encoder strategy effectively combines both non-COVID19 and COVID-19 features and shows the best performance. We further conclude that it is possible to freeze the encoder and train only the decoder on COVID-19 tasks during fine-tuning, which provides significant performance gains and fast convergence speed.

In this section, we review the recent related research on COVID-19 CT images, COVID-19 segmentation and transfer learning for COVID-19.

A. Research on COVID-19 CT Images.

CT is widely used in the screening and diagnosis of COVID-19 since ground-glass opacities and bilateral patchy shadows are the most relative imaging features in pneumonia associated with infections. Recent research on CT-based diagnosis for COVID-19 has indicated great performance. Compared with traditional CT images processing, artificial intelligence (AI) serves as a core technology, enabling a more accurate and efficient solution. The machine learning-based CT radiomics models such as random forest, logistic regression showed feasibility and accuracy for predicting hospital stay in patients affected in the work of [8] . Machine learning method was also adopted by Tang et al. [9] to realize automatic severity assessment (non-severe or severe) of COVID-19 based on chest CT images, and to explore the severity-related features from the resulting assessment model. Gozes et al. [10] presented a system that utilized robust 2D and 3D deep learning models, modifying and adapting existing AI models and combining them with clinical understanding. Huang et al. [11] monitored the disease progression and understood the temporal evolution of COVID-19 quantitatively using serial CT scan by an automated deep learning method.

Segmentation is an essential step in AI-based image processing and analysis [4] . In particular, segmenting the regions of interests (ROIs) of COVID-19 infections is crucial and helpful for doctors to make further assessment and quantification. However, manual contouring of these infections is timeconsuming and tedious. Although plenty of methods have been explored on COVID-19 diagnosis and classification, there are very few works on the segmentation of COVID-19 infection due to its great annotation challenges.

Shan et al. [12] developed a DL-based system for automatic segmentation and quantification of infection regions and adopted a human-in-the-loop (HITL) strategy to accelerate the manual delineation of CT training. Zheng et al. [13] designed a weakly-supervised deep learning algorithm to investigate the potential of a deep learning-based model for automatic COVID-19 detection on chest CT volumes using the weak patient-level label. Based on semi-supervised learning, Fan et al. [14] presented a COVID-19 Lung Infection Segmentation Deep Network (Inf-Net) for CT slices based on randomly selected propagation. Yan et al. [15] introduced a feature variation block which adaptively adjusted the global properties of the features for segmenting COVID-19 infection.

In transfer learning, deep models are first trained on large datasets such as ImageNet, then these pre-trained models are fine-tuned on different downstream tasks. Several studies [6] , [16] , [17] have investigated transfer learning methodologies on deep neural networks applied to medical image analysis tasks. Plenty of works used networks pre-trained on natural images to extract features and followed by another classifier [18] , [19] . Carneiro et al. [19] replaced the fully connected layers of a pre-trained model with a new logistic layer and only trained the appended layer, yielding promising results for classification of unregistered multi-view mammograms. Other studies performed layer fine-tuning on the pre-trained networks for adapting the learned features to the target domain. In [20] , CNNs were pre-trained as a feature generator for chest pathology identification. Gao et al. [21] fine-tuned all the layers of a pre-trained CNN to classify interstitial lung diseases. Ghafoorian et al. [22] trained a CNN on legacy MR images of brain and evaluated the performance of the domainadapted network on the same task with images from a different domain.

Due to the limited labeled COVID-19 data, several transfer learning methods have been applied to address this problem. Chouhan et al. [23] proposed an ensemble approach of transfer learning using pre-trained models trained on ImageNet. Researchers used five different pre-trained models and analyzed their performance in chest X-ray images. In the work of [24] , several deep CNNs were employed for automatic COVID-19 infection detection from X-ray images through tuning parameters. Majeed et al. [25] presented a critical analysis for 12 off-the-shelf CNN models and proposed a simple CNN architecture with a small number of parameters that performed well on distinguishing COVID-19 from normal X-rays. By combining three different models which were fine-tuned on 3 datasets, Misra et al. [26] designed a multi-channel ensemble TL method based on ResNet-18 in such a way that the model could extract more relevant features for each class and identify COVID-19 features more accurately from the X-ray images.

Even though the above studies on transfer learning present inspiring achievement for COVID-19 research, there are several limitations: 1) They only focus on the ensemble of existing CNNs, but ignore the contribution of various datasets for pretraining. 2) Their studies are limited to X-ray dataset and only dedicate to COVID-19 detection and classification. 3) They lack an in-depth study on the transferability of different transfer manners related to COVID-19 infections. Our work contributes to a much more difficult task of semantic segmentation in COVID-19 CT images. We explore the transferability from the perspective of transferring knowledge from various non-COVID19 lung lesions. Moreover, we investigate better transfer methods to assist COVID-19 segmentation.

In this section, we briefly describe our backbone network, then introduce the multi-task learning method to get a multilesion pre-trained model. We further give detailed illustration on four transfer strategies employed in our work.

The U-Net is a commonly used network for medical semantic segmentation. As an advanced architecture, it has a Ushape like structure with an encoding and a decoding signal path. The encoder serves as a contraction to capture semantically image contextual features. The decoder is a symmetric expanding path recovering spatial information. The two paths are connected using skip connections on each same level, which recombine with essential high-resolution features from the encoding path. In this work, we make some minor changes following nnU-Net [27] . Original batch normalization and ReLU are replaced by instance normalization and leaky ReLU. What's more, deep supervision loss [28] is aggregated to obtain multi-level deep supervision and facilitate the training process.

Multi-task learning is an effective method to share knowledge among different related tasks. As for segmentation tasks, the performance of each task highly depends on the similarity among these tasks. Due to the large domain distance among those existing non-COVID19 lung lesion datasets, fusing these lesions to train a multi-task segmentation model tends to underperform on each single task. However, this multi-task training can exploit the shared knowledge which is essential for learning some general-purpose visual representations about lung lesions. Therefore, in our work, besides separately training segmentation models on each non-COVID19 dataset as pre-trained models for transfer learning, we provide a multilesion model learning from multiple lung lesions. Compared with learning from separate tasks, this multi-task strategy leads to a more robust pre-trained model across all lesion tasks and empowers downstream COVID-19 task.

Continual Learning aims to learn from an endless stream of tasks [29] . This paradigm is capable of learning consecutive tasks without forgetting how to perform previously trained tasks. This is challenging that the training process tends to lose knowledge from the previous tasks due to the information relevant to current tasks. To avoid this, we adopt a training schedule to pre-train the model. The value of the initial learning rate is set as 0.01 and decays throughout the training process following the poly learning rate policy (1 − epoch / epoch max ) 0.9 . In this way, the learning rate is decreasing continuously so that the weight parameters after training non-COVID19 lesion tasks tend to follow its training process while slightly being updated by the current COVID-19 infection task.

Fine-tuning is the most standard strategy to transfer knowledge from domains where data are abundant. In general, it is conducted by copying the weights from a pre-trained network and tuning them on the downstream task.

Recent work [30] shows that fine-tuning can enjoy better performance on small datasets. It is of additional interest to assess the contribution of the encoder and decoder relative to learning COVID-19 knowledge. While the progressive reinitializatons demonstrate the incremental effect of each layers, it is unnecessary to prob the extent of localized reinitialization cause our encoder-decoder network is not very deep. Therefore, we adopt two strategies for tuning a non-COVID19 lesion model on the COVID-19 downstream task.

1) Body Fine-tuning: Due to the large domain difference between pre-trained tasks and downstream tasks, the most secured fine-tuning method is Body Fine-tuning, which means all parameters of the pre-trained models are used as the initial values to complete the training process of the model. When we train COVID-19 infection networks, all of the parameters are assigned initially from the pre-trained models on non-COVID19 lesions. In this fine-tuning strategy, the update of parameters largely depends on COVID-19 infection training process itself. Thus, it is a conservative fine-tuning approach that the training task of COVID-19 infection is not affected by the upstream pre-trained non-COVID19 models too much.

2) Pre-trained Lesion Representations: Our segmentation model is an encoder-decoder architecture and the encoder serves as a series of convolution operations to encode image features into context representations. These representations are trained on large relative datasets from upstream tasks, and fed as features to downstream ones. In natural language processing tasks, features extracted from internal representations of sequence language models are encoded as pre-trained text representations [29] , [30] . In this fine-tuning strategy, we aim to learn general lung lesion features, which we call Pretrained Lesion Representations. We first train models on non-COVID19 lung lesion datasets. The encoders of these models are capable of encoding lesion features. In other words, the encoders' parameters can exhibit some transferability. Thus, in the following fine-tuning process on COVID-19 dataset, we preserve and freeze them while only fine-tuning and retraining the decoding parts.

When performing the above fine-tuning strategies, there exist two problems: 1) Body Fine-tuning easily falls into overfitting cause the downstream COVID-19 infection dataset is much smaller. 2) Utilizing the encoder of pre-trained models to capture feature representation is unstable, because the label spaces and losses for the upstream and downstream tasks differ (Though they are all lung lesions but differ in appearance and shape). Therefore, we present a new method for transfer learning. Our method has three key properties:

• It leverages transferred knowledge from other datasets and combines knowledge from COVID-19 in a better manner.

• It gains performance improvement in terms of training from scratch and other transfer learning methods.

• It does not cost more training time. To achieve these properties, we propose a Multi-encoder architecture. As shown in Fig.1 , this standard U-Net network has two encoders: self-encoder and adapted-encoder. These two encoders have the same architecture and share the same decoder. Self-encoder is a task-specific feature extractor, focusing on segmenting COVID-19 infection through reinitializing all the parameters. These parameters are COVID19-specific which enable update to the network. It is better to assign initial values from pre-trained models to these parameters for convergence, which is adopted in our work. The adaptedencoder is totally frozen while totally using the pre-trained encoder to capture general lung lesion features. Features from these two paralleled encoders are concatenated as fusion of COVID-19 feature representations and then fed into the decoder part. 

In this section, we will describe in detail the datasets, experimental setup and results of our investigation.

A. Dataset Introduction 1) COVID-19 Dataset: This dataset is released by Coronacases Initiative and Radiopaedia 1 . It is a publicly available COVID-19 volume dataset which contains 20 COVID-19 CT scans. In the work of [7] , [31] , left lung, right lung, and infections are well-labelled by two radiologists and verified by an experienced radiologist. Thus, with over 1800 annotated slices, this dataset serves as a downstream COVID-19 infection segmentation dataset for transfer learning in our work.

2) Non-COVID19 Lung Lesion Datasets: In order to better explore the transferability from various non-COVID19 lung lesions to COVID-19 infections, the following relationships need to be satisfied among these datasets: 1) The size of different lesion datasets are similar. 2) The shape, size and location of different lesion areas are relatively distinguishing. Therefore, in this paper, we introduce three public datasets.

MSD Lung Tumor This dataset was used in a crowdsourced challenge of generalized semantic segmentation algorithms called the Medical Segmentation Decathlon (MSD) held in MICCAI 2018 2 . This dataset includes patients with non-small cell lung cancer from Stanford University (Palo Alto, CA, USA) publicly available through TCIA and previously utilized to create a radiogenomic signature [32] , [33] , [34] . The tumor regions are denoted by an expert thoracic radiologist on a representative CT cross section using OsiriX [35] . 63 3D CT scans with corresponding tumor segmentation masks are utilized in our paper.

StructSeg Lung Cancer StructSeg organ dataset is a collection of 3D organ CT scans along with their segmentation ground-truth from 2019 MICCAI challenge 3 . This dataset contains two types of cancers, nasopharyngeal cancer and lung cancer. We adopt the gross target volume segmentation of lung cancer from 50 patients. Each CT scan is annotated by an experienced oncologist and verified by another one.

NSCLC Pleural Effusion This dataset is developed from a Non-Small Cell Lung Cancer (NSCLC) cohort of 211 subjects 4 . 78 cases with pleural effusion are selected with their segmentation masks.

All the experiments are implemented in Pytorch and trained on NVIDIA Tesla V100 32GB GPU. For fair comparison, we follow the settings on COVID-19 dataset benchmarks in [7] .

For the COVID-19 dataset, we use 5-fold cross validation based on a pre-defined dataset split file. Each fold contains 4 scans (20%) for training and 16 (80%) for testing.

For non-COVID19 lung lesion datasets, we all randomly select 80% of the data for training and the rest of 20% for validation. Pre-trained models based on these non-COVID19 lesions are all trained from scratch with random initial parameters using 3D U-Net network. Due to the limited number of different lesion cases, we do data pre-processing following nnU-Net [27] . The input patch size is set as 50×160×192 and batch size as 2. Stochastic gradient descent optimizer with an initial learning rate of 0.01 and a nesterov momentum of 0.99 are used for non-COVID19 pre-training. We adopt a summation of Dice loss and Cross entropy loss as loss function.

Diagnostic evaluation is often used in clinical practice for disease diagnosis, patient follow-up or efficacy monitoring. Whether the results of a certain diagnostic evaluation are true, reliable and practical, will largely determine a reasonable medical decision. In this work, we introduce five evaluation metrics for the exploration of transferability.

Dice similarity coefficient (DSC) measures volumetric overlap between segmentation results and annotations. It is computed by:

where A is the sets of foreground voxels in the annotation and B is the corresponding sets of foreground voxels in the segmentation result, respectively. Normalized surface distance (NSD) [36] serves as a distance-based measurement to assess performance. It is computed by:

where R We also consider three other evaluation metrics. Accuracy denotes the correct rate for both positive and negative predictions. Sensitivity shows the percentage of positive instances correctly identified positive. F1-score is the weighted harmonic average of Precision and Sensitivity which is an effective and comprehensive evaluation.

All of the four pre-trained models (MSD, StructSeg, NSCLC, Multi-lesion) are utilized to investigate the ability of transfer learning to COVID-19 datasets. With extensive experiments in Table I , we observe that different pre-trained models show different transferability. The best scores of models corresponding to each transfer learning strategies are highlighted along with the detailed comparative analysis of 5 validation folds.

1) Single-lesion Pre-trained Model: Among pre-trained models on each single lesion (MSD, StructSeg, NSCLC), compared with training COVID-19 datasets from scratch, MSD tumor pre-trained model improves the segmentation by 3.0% DSC and 3.2% NSD at most. Meanwhile, StructSeg and NSCLC lung lesion pre-trained models show instability under different transfer strategies. As shown in Table I , with Continual Learning and Body Fine-tuning strategies, StructSeg and NSCLC pre-trained models can achieve promising improvement in most folds, but instead, they obtain lower performance with the strategy of Pre-trained Lesion Representations. The rationale is that there exists a large domain distance between these two lesions and COVID-19 infection. The strategy of Pre-trained Lesion Representations totally uses encoded non-COVID19 lesion features. Thus, the transferability of a single-lesion dataset model largely depends on their domain difference.

2) Multi-lesion Pre-trained Model: As shown in Table I , we further notice that among all transfer learning strategies, multilesion pre-trained model not only achieves a high percentage for DSC and NSD values, but also performs the most stably among all pre-trained models, with the average DSC value of 0.696, 0.696, 0.693, 0.703, respectively. This shows the robustness of a multi-lesion pre-trained model used for transfer learning to COVID-19 infection. Table II further verifies this demonstration using Sensitivity, F1-score and Accuracy concerned with different transfer strategies. It is observed that transferring from multi-lesion pre-trained model outperforms training from scratch on all these evaluation metrics among all strategies. Significantly, when it comes to Pre-trained Lesion Representations, where the encoder of pre-trained models is totally frozen and serves as a non-COVID19 lesion features extractor, the multi-lesion pre-trained model can still perform well. This confirms the effectiveness of multi-task training for multiple lung lesions, which generates more robust and general-purpose representations to help COVID-19 infection tasks. Fig.2(a)-(e) show some examples of segmentation results of the above pre-trained models. It is clear that compared with training from scratch, pre-training from single non-COVID19 models can obtain more accurate massive structures of COVID-19 but shows dissatisfying instability. However, pre-training from multi-lesion model shows high precision and smooth boundary like manually annotated.

Based on the above conclusion that multi-lesion pre-trained model brings consistently higher accuracy and robustness, we further analyze the performance of different transfer strategies adopted in this paper. Table III shows the results of COVID-19 infection segmentation using the same multi-lesion pre-trained model under different transfer strategies. These results suggest that all the transfer strategies improve the performance of training from scratch on average DSC, NSD and Sensitivity. In particular, in fold 3, they improve the segmentation by more than 6.0% DSC and 6.4% NSD on maximum. The strategies of Continual Learning and Body Fine-tuning get similar promising results, which both improve the segmentation by 2.3% DSC, 1.8% NSD and 3.4% Sensitivity on average.

An interesting observation is that, compared with Continual Learning and Body Fine-tuning, where all the parameters are updated on COVID-19 dataset, the strategy of Pre-trained Lesion Representations still achieves a competitive performance with an entirely frozen encoder and inherited weights of pretrained non-COVID19 models. It is observed to improve the DSC from 0.673 to 0.693, NSD from 0.700 to 0.716 and It is also observed in Table III that our proposed Multiencoder transfer learning strategy exhibits significantly better segmentation performance than other strategies using multilesion pre-trained model. It improves the average DSC from 0.673 to 0.703 and NSD from 0.700 to 0.732, which also performs best among all the pre-trained models in Table I . In terms of F1-score and Accuracy in Table II, proposed Multi-encoder learning outperforms other strategies and enhances the values to 0.7035 and 0.9942, respectively. The transfer ability of Multi-encoder is also confirmed by Fig.3 . Compared with training from scratch, Multi-encoder learning yields segmentation results with more accurate boundaries in Fig.3 (b)(c)(e) and identifies some minor COVID-19 infection areas in Fig.3 (a)(d)(f). The success of proposed Multi-encoder learning strategy is owed to the designed parallel encoders, where the COVID-19 and non-COVID19 lesions are both employed for encoding feature representations, leading to better generalization and lower over-fitting risks.

In this paper, we investigate the transferability in COVID-19 CT segmentation. We present a set of experiments to better understand how different non-COVID19 lung lesions influence the performance of COVID-19 infection segmentation and their different transfer ability under different transfer learning strategies. Our results reveal clear benefits of pre-training on non-COVID19 datasets.

A valuable demonstration is that a multi-lesion pre-trained model can make the best of multiple lesion representations, and advance the generalization and robustness of pre-trained models. There is value in recognizing that the CT appearance of these different lung lesions share some similarity. Thus, with more kinds of lung lesion datasets incorporated to pre-train a model, we could achieve better performance. This exploration is an important contribution, enabling more research on transfer learning to COVID-19 infection from the perspective of utilizing non-COVID19 lung lesions. Moreover, this paper examines a series of different transfer learning strategies, including Continual Learning, Body Finetuning, Pre-trained Lesion Representations and the proposed Multi-encoder Learning. We observe segmentation improvement in all performance metrics. It is also noted that the strategy of Pre-trained Lesion Representations with a frozen encoder enhances performance as well. This gains more insight into the significant transferability exhibited by the encoding parameters. Though the encoding layers do not contain any explicit knowledge of the COVID-19 infection, their parameters still enable the optimizer to reach a higher performance while fine-tuning. The rationale is that the encoded multi-lesion representations contain more high-level and abundant encoding information of the medically relevant lung lesion observed in CT images. Through combining the multi-lesion representations and COVID-19 infection features, the proposed Multi-encoder achieves significant improvement. These observation and exploration are important not only in COVID-19 transfer learning but also in the general medical domain, because feature reuse from pre-training out-of-domain datasets shows significant improvement for task performance and training convergence.

Future research directions include utilizing more various non-COVID19 lung lesion datasets and investigating better transfer learning methods, so that non-COVID19 lung lesions can be effectively used to improve the quality of COVID-19 infection segmentation in the absence of sufficient high-quality COVID-19 datasets.

Improved molecular diagnosis of covid-19 by the novel, highly sensitive and specific covid-19-rdrp/hel real-time reverse transcription-pcr assay validated in vitro and with clinical specimens

Ct imaging features of 2019 novel coronavirus (2019-ncov)

Ai-assisted ct imaging analysis for covid-19 screening: Building and deploying a medical ai system in four weeks

Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for covid-19

Transfusion: Understanding transfer learning for medical imaging

Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis

Towards efficient covid-19 ct annotation: A benchmark for lung and infection segmentation

Machine learning-based ct radiomics model for predicting hospital stay in patients with pneumonia associated with sars-cov-2 infection: A multicenter study

Severity assessment of coronavirus disease 2019 (covid-19) using quantitative features from chest ct images

Rapid ai development cycle for the coronavirus (covid-19) pandemic: Initial results for automated detection & patient monitoring using deep learning ct image analysis

Serial quantitative chest ct assessment of covid-19: Deep-learning approach

Lung infection quantification of covid-19 in ct images with deep learning

Deep learning-based detection for covid-19 from chest ct using weak label

Inf-net: Automatic covid-19 lung infection segmentation from ct images

COVID-19 Chest CT Image Segmentation -A Deep Convolutional Neural Network Solution

Convolutional neural networks for medical image analysis: Full training or fine tuning?

Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning

Transfer learning for multicenter classification of chronic obstructive pulmonary disease

Unregistered multiview mammogram analysis with pre-trained deep learning models

Chest pathology identification using deep feature selection with nonmedical training

Holistic classification of ct attenuation patterns for interstitial lung diseases via deep convolutional neural networks

Transfer learning for domain adaptation in mri: Application in brain lesion segmentation

A novel transfer learning based approach for pneumonia detection in chest x-ray images

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

Covid-19 detection using cnn transfer learning from x-ray images

Multi-Channel Transfer Learning of Chest X-ray Images for Screening of COVID-19

nnu-net: Breaking the spell on successful medical image segmentation

3d deeply supervised network for automatic liver segmentation from ct volumes

Parameter-efficient transfer learning for nlp

How to fine-tune bert for text classification

COVID-19 CT Lung and Infection Segmentation Dataset

Nsclc radiogenomics: Initial stanford study of 26 cases

A radiogenomic dataset of non-small cell lung cancer

Non-small cell lung cancer: identifying prognostic imaging biomarkers by leveraging public gene expression microarray data-methods and preliminary results

Osirix: an open-source software for navigating in multidimensional dicom images

Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy