key: cord-0847238-bvkpn0kx
authors: Amini, Nasrin; Shalbaf, Ahmad
title: Automatic classification of severity of COVID‐19 patients using texture feature and random forest based on computed tomography images
date: 2021-11-26
journal: Int J Imaging Syst Technol
DOI: 10.1002/ima.22679
sha: 7a8575847424e966af2ee977531f9f9aab546af7
doc_id: 847238
cord_uid: bvkpn0kx

Severity assessment of the novel Coronavirus (COVID‐19) using chest computed tomography (CT) scan is crucial for the effective administration of the right therapeutic drugs and also for monitoring the progression of the disease. However, determining the severity of COVID‐19 needs a highly expert radiologist by visual assessment, which is time‐consuming, boring, and subjective. This article introduces an advanced machine learning tool to determine the severity of COVID‐19 to mild, moderate, and severe from the lung CT images. We have used a set of quantitative first‐ and second‐order statistical texture features from each image. The first‐order texture features extracted from the image histogram are variance, skewness, and kurtosis. The second‐order texture features extraction methods are gray‐level co‐occurrence matrix, gray‐level run length matrix, and gray‐level size zone matrix. Finally, using the extracted features, CT images of each person are classified using random forest (RF) as an ensemble method based on majority voting of the decision trees outputs to four classes. We have used a dataset of CT scans labeled as being normal (231), mild (563), moderate (120), and severe (42) determined by expert radiologists. The experimental results indicate the combination of all feature extraction methods, and RF achieves the highest result compared with the other strategies in detecting the four classes of severity of COVID‐19 from CT images with an accuracy of 90.95%. This proposed system can work well and can be used as an assistant diagnostic tool for quantification of lung involvement of COVID‐19 to monitor the progression of the disease.

In December 2019, the novel Coronavirus (COVID- 19) was reported in Wuhan, China. A month later, it was identified in many other countries around the world, and the World Health Organization (WHO) announced that the COVID-19 pandemic as a public health emergency. COVID-19 is an infection that is highly contagious and has spread rapidly worldwide. This is not only due to its fast person-to-person spread but also because most infected people are not immune to it. 1, 2 The diagnostic tests for COVID-19 are mostly based on reverse transcription-polymerase chain reaction (RT-PCR). 3 However, PCR results often require more than 5-6 h and also have a false negative. Moreover, the severity of the disease is not detectable by this method. For these reasons, a rapid and accurate diagnostic method is required to isolate and treat the patients as soon as possible. 4 Chest computed tomography (CT) scan is another way to diagnose COVID-19, which is more efficient and faster test. 5 Furthermore, lung CT images contain important information as the severity of disease with COVID-19 based on lung involvement. According to the guidelines, lung CT findings of COVID-19 are ground glass opacity (GGO), crazy-paving pattern, and subsequent consolidation. 6, 7 Severity assessment is crucial for the effective administration of the right therapeutic drugs and choosing the best cure for them and also useful for following the changes of the patient's condition. [8] [9] [10] Follow-up scans every 3-5 days for patients with COVID-19 are often recommended for disease progression. 11 So, from the beginning of this epidemic until now, processing of lung CT images has played an important role in clinical discovery of suspected cases, a suitable tool for treatment review 12, 13 and monitoring the disease. [14] [15] [16] [17] The severity of COVID-19 can be done based on the assessment of CT images by a highly expert radiologist, visually. But visual evolution is tedious and subjective. So, quantitative analysis of CT images using an advanced machine learning tool as an automatic, reliable, and objective estimation of the disease is needed for solving this problem during the COVID-19 pandemic.

Recently, there have been some research studies for automatic detection of COVID-19 disease based on lung CT images using artificial intelligence. Chen et al. used UNet+ +, a novel architecture for medical image segmentation, to extract infected areas in CT images for identification of COVID-19 pneumonia. 18 The other researchers have used convolutional neural network (CNN) and deep transfer learning algorithms for detecting COVID-19 cases. [19] [20] [21] [22] Finally, a feature extraction method from CT images using the second-order statistical texture features such as graylevel co-occurrence matrix (GLCM) and local directional pattern (LDP), as well as discrete wavelet transform (DWT), is proposed. 23 Then, researchers used support vector machines (SVM) method to classify CT images to detect the COVID-19. Zhang et al. used AlexNet (a pretrained deep learning network) to diagnose COVID-19 from chest CT. Then, it was enhanced by adding batch normalization to reduce internal covariance shift and accelerate the training and also replacing the fully connected layer in AlexNet with three different classifiers. 24 But the above studies are only about automatic detection of COVID-19 on CT images and do not consider the severity of it.

Few studies worked on the CT images to determine the severity of patients based on lung involvement. In one recent study, changes in chest CT findings (GGO, crazy-paving pattern, and consolidation) of 21 patients associated with COVID-19 from initial diagnosis until patient recovery are determined. 25 Based on this study, most patients who recovered from COVID-19 pneumonia showed the greatest severity of lung disease on CT at approximately 10 days after the initial onset of symptoms. In another study, U-Net deep learning method was used for segmentation of CT lung opacification of the whole lung and then classified the CT images of 126 patients with COVID-19 into mild, moderate, severe, and critical cases. 26 Finally, image segmentation system based on deep learning methods has also been developed to automatically quantify infection regions of chest CT scans of COVID-19 patients and identify lesion percentage cover and mean lesion density to evaluate 3-class severity in 44 patients. 27 However, these few methods need image segmentation of lesion percentage of lung involvement in COVID-19 patients, which is a problemist task.

The aim of this article is to propose a new automatic system for diagnosing the severity of COVID-19 from the lung CT images to four classes as normal, mild, moderate, and severe without any need to image segmentation based on a combination of first and second statistical features and random forest (RF) classifier as advanced machine learning method. This proposed severity assessment system can help physicians to observe the changes in the patients' condition and take better decisions to cure and finally control infections for COVID-19 patients.

Lung CT images in the sagittal views from people suspected of having COVID-19 were used in this study from free open access datasets provided by municipal hospitals in Moscow, Russia. 28 These images were obtained between March 1, 2020, and April 25, 2020. Noncontrast standard dose CT scan data were acquired using Toshiba Aquilion 64 machine with number of slices 64, slice thickness 1 mm. 29 Table height and alignment are adjusted, and gantry of CT machine is centered at the thorax of patients. A total of 956 patients (males-42%, females-56%, other/unknown-2%) were included with age from 18 to 97 years, median-47 years, max: 97 years old. Clinical information of patients is shown in Table 1 .

CT images are categorized into five classes including normal lung tissue without symptoms of COVID-19 (231 cases) and COVID-19-related findings in four different levels depending on the percent of pulmonary parenchymal involvement from mild (563 cases), moderate (120 cases), severe (40 cases), and critical (2 cases) by highly experienced radiologist. In mild and moderate stages, several GGO are obtained, and involvement of lung parenchyma in mild is less than 25% and in moderate is between 25% and 50%. In the severe stage, GGO and regions of consolidation are obtained, and involvement of lung parenchyma is between 50% and 75%. Finally, in the critical stage, diffuse GGO and consolidation as well as reticular changes in lungs are obtained. Involvement of lung parenchyma exceeds 75%. Due to the limited amount of data in the severe and critical stages, we considered the images of these two stages to be one stage as named severe stage. So, in this study, CT images are categorized into four classes including normal lung tissue (231 cases) and COVID-19-related findings in three levels of severity from mild (563 cases), moderate (120 cases), and severe (42 cases). Figure 1 shows some CT slices of four normal, mild, moderate, and severe COVID-19 cases in our dataset. 

In lung CT imaging, a set of images approximately between 20 and 30 slices is taken for each patient from the upper part of the chest to the bottom part. In order to find the best images that contain lung organ completely from the collection of images, we removed the upper and bottom slices of the chest CT images during the preprocessing stage. So, the images that clearly contain lung volume are created. Then, from each image, 20% on both sides of the vertical part and 5% of the horizontal part are cropped, automatically. It is because of the anatomical structure of the human body and the slices of CT scan images that are oval shaped. Finally, histogram equalization is applied in each image for equalizing the gray level of image lightness.

In order to classify the patients into four classes, a set of quantitative features must be extracted from the CT images. We have used the first-and second-order statistical texture features from each image. These statistical features provide information about the gray level of CT images and also about the spatial arrangement of intensities. Variance, skewness, and kurtosis as the first-order statistical features are extracted based on the image histogram of CT images and we named them global features in the results. Variance shows the variations in the intensity of the image histogram. Skewness indicates the relative level of the slope of the curve of image histogram, and kurtosis indicates the level of sharpness of the curve of the image histogram. 30 The second-order statistical texture features extraction methods from each image used in this study are GLCM, 29 gray level run length matrix (GLRLM), 31 and gray-level size zone matrix (GLSZM). 30 31 GLSZM is a developed version of the GLRLM method, which is composed of large areas of the same intensity, and not of small groups of pixels or segments in any given direction. 32 Consequently, 28 features (3 first-order statistical features and 25 second-order features) have been measured in this study for each image in the MATLAB Image Processing Toolbox. Then, for each person, these mentioned features are extracted from all CT images. Finally, the sum of these features from all CT images is considered as the feature sets of that person.

RF classifier as a supervised machine learning algorithm 33 is applied to classify CT images of each person into four classes as normal, mild, moderate, and severe COVID-19. RF algorithm is one of the most successful classifiers for multiclass problems. [33] [34] [35] It is an ensemble classifier that consists of a set of decision trees, and the final decision of RF is the majority voting among all of the trees to make it more accurate and stable. A combination of different learning models leads to a better model to increase the overall results. In the training phase, each decision tree is constructed using a subset of the training sample, which is drawn from the total training sample using the bootstrap sampling method. So, each decision tree is fed by only a small number of randomly selected features and also dataset, randomly. This algorithm looks for the best selected features for different decision trees. Then, decision trees constructed according to each subset are typically grown and the classification results are obtained. Finally, the decision of RF as the optimal results is determined using majority voting among the result of different trees, which reacts independently. For comparison, linear discriminant analysis (LDA) and knearest neighbor (KNN) methods are also used to classify CT images into four classes. 36 LDA is a supervised algorithm that classifies dataset into a lower-dimensional space with good class separability and maximizes the separation between two classes. In the KNN classification, which is very simple, highly efficient, and effective, test data are classified by a majority vote of k neighboring classes.

In this study, a new automatic methodology is performed to classify chest CT images of 956 people into four classes as normal, mild, moderate, and severe COVID-19 depending on the percent of pulmonary parenchymal involvement. Figure 2 shows the block diagram of our method. All processing was performed using MATLAB R2019b on a 4.5 GHz quad-core computer with NVIDIA GeForce RTX 1660Ti graphics processing unit (GPU). In the preprocessing step, CT slices that contain lung organs from all chest CT slices are taken. Then, from each image, 20% on both sides of the vertical part and 5% of the horizontal part of the image are cropped. Finally, histogram equalization is applied in each image. The next step is to extract the features from the CT images. We have used the first-and second-order statistical texture features from each image. The first-order statistical texture features from the image histogram named variance, skewness, and kurtosis are extracted. The second-order statistical texture features extraction methods are GLCM (23 features), GLRLM, and GLSZM. Consequently, for each person, 28 features (3 first-order statistical features and 25 second-order features) are extracted from all CT images (approximately between 20 and 30). The sum of these features from all CT images is considered as the feature sets of that person. Finally, using the extracted features, CT images of each person are classified into four classes as normal, mild, moderate, and severe COVID-19 using RF classifier. Table 2 shows the values of classification accuracy by different feature extraction methods. We compared RF classifier with two other common classifiers: LDA and KNN, to classify CT images into four classes. The numbers of neighbors for KNN classifier are optimized by applying the trial and error method and set to 8 to obtain the best results. For assessment of the classifiers, the different k-fold cross-validation methods were used and the best results for eightfold were obtained and reported in Table 2 . In the appendix section (Table A1) , results from fivefold and 10-fold are compared with our proposed method (eightfold). It is shown RF with GLRLMS feature extraction method yields a classification accuracy of 88.74% in detecting the four classes of severity of COVID-19. Clearly, it is demonstrated that RF with GLRLMS achieves the highest result in recognition of CT images compared with the other mentioned methods. To improve the classification results, we combined all feature extraction methods. The best result for combination of all features was also obtained with the RF classifier with an accuracy of 90.95%. The p value of Wilcoxon rank-sum test for the best results and RF with GLRLMS is 0.008, which demonstrates the statistical significance of our proposed approach (p < 0.05). So, the RF classifiers performed more powerfully than the others. It should be noted that the number of trees in RF is a structural parameter and needs to be tuned in an optimization process. Therefore, the classification error for different number of trees (10-70 trees) is calculated. As the number of trees increases, more decision units participate in voting. The optimum number is where adding more trees does not significantly improve the result. The best value for the tree number is set to 40 in order to achieve the best performance. The results of these computations for selecting the best number of trees are demonstrated in Figure 3 . Finally, a comparison of estimation of labels provided by the proposed method (RF classifier with all combination of features) against those assigned by the reference one (highly experienced radiologist) is achieved and presented in Table 3 . It shows our classifier had acceptable accuracy with 96.6% in normal class, 98.6% in mild class, and 100% in severe class, but our bad result was accurate up to 60% in moderate COVID-19.

F I G U R E 2 Classification process for computed tomography (CT) images to normal, mild, moderate, and severe COVID-19

In this study, we have proposed an automatic method for diagnosis of severity of COVID-19 to normal, mild, moderate, and severe based on lung involvement percentage from all lung CT images of a person. We have demonstrated that the proposed method based on a combination of first and second statistical texture features along with an RF classifier has higher results with an accuracy of 90.95%. So, it can help physicians to take better decisions for monitoring the patient and choose the best cure for them based on diagnosed level of a patient.

Our problem in the proposed method is to separate CT images of mild and moderate classes. Our classifier separated class normal and severe very well, but moderate and mild classes are mistaken for each other. It is because of the similarity of finding CT patterns in these two classes. CT finding for patients with mild and moderate levels of COVID-19 is GGO with different involvement of lung parenchyma (less than 25% in mild and between 25 and 50 in moderate). So, the patterns of these images are really similar, and it was too difficult for a classifier to separate them exactly. Another reason could be a smaller number of datasets in moderate level in comparison with mild level, and consequently moderate classes are detected to mild class wrongly (Table 2) . So, because of the similarity of pattern and the low number of datasets in the moderate level, our classifier is unable to separate these two classes very well. Table 4 compares the results of our work with other studies in the classification of severity of COVID-19 patients from CT images. As it is observed, the performance achieved in this study is promising. Compared with other research studies, this work has some advantages and shortcomings. Our method has used a combination of the first-and second-order statistical texture features of all CT slices of a patient, which means feature extraction was done in 3D. All CT slices of a patient with COVID-19 must be considered as one data and the infection may appear in some CT image slices and not be shown in other slices. So, one slice alone cannot indicate the stage of COVID-19 lung involvement of a person, and the sum of features from all CT images must be considered for each person to classify the patients into four classes depending on the percent of pulmonary parenchymal involvement. Also, we have improved the results by finding the best classifier with RF. The advantages of the proposed method with RF, such as great classification rate and high-speed calculation, make it applicable for medical classification problems. Aggregation result of many decision trees can lead to a more generalized outcome. In other words, when different predictors try to participate in learning independently, the average result can overcome each individual predictor in terms of accuracy. By utilizing several decision trees built by selected subset of features, the robustness of the classifier significantly improves. Therefore, RF classifier has several superiorities compared with other classifiers. Finally, our method compared with deep neural learning has some advantages and disadvantages. Deep learning methods are computationally very expensive and time-consuming to train with traditional CPUs. These methods depend a lot on training data. This leads to the problem of over-fitting and generalization in a small database. Moreover, these networks are black boxes, meaning we cannot know how much each independent variable is influencing the dependent variables. However, with big database, deep learning algorithms can automatically extract significant features in hierarchical manner and classify them directly on raw pixels. Our proposed method has some limitations. Other current factors that effect on severity of COVID-19 patients such as demographic, vital, blood test, laboratory, background disease (like diabetes, etc.), and clinical features are not considered in this study. In addition to lung CT images, these features can also contain useful information to train classifiers and can be added to enhance the accuracy of this automatic system. It is true that we were able to separate the data well with CT scan images alone, but having these features from the patient can also make the result more accurate and reliable. Moreover, the number of patients must be increased. So, in future works, we want to work on a new larger dataset with more information than lung CT images.

We have proposed a new automatic practical system based on lung CT images for diagnosis of severity of COVID-19 to normal, mild, moderate, and severe based on lung involvement percentage and provide insightful findings successfully. The experimental results on 956 patients indicate the combination of first and second statistical texture features along with an RF classifier has the higher results with an accuracy of 90.95%. So, based on the excellent performance, it can be used as an assistant diagnostic tool for physicians to monitor the progression or response of the disease to treatment.

The article is supported by Iran National Science Foundation: INSF (99009215) and Research Department of Shahid Beheshti University of Medical Sciences.

T A B L E 3 Comparison of estimation of labels provided by the proposed method (RF classifier with all combination of features) against those assigned by the reference one (highly experienced radiologist)

Estimated labels by the proposed method 

The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health-the latest 2019 novel coronavirus outbreak in Wuhan, China

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT?

Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection

A rapid advice guideline for the diagnosis and treatment of 2019 novel coronavirus (2019-nCoV) infected pneumonia (standard version)

Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner society 2017

Early prediction of the risk of severe coronavirus disease 2019: a key step in therapeutic decision making

Coronaviruses-drug discovery and therapeutic options

CT features of Coronavirus disease 2019 (COVID-19) in the original district of this disease (Wuhan): a pictorial review

CT imaging features of 2019 novel coronavirus (2019-nCoV)

The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner society

The role of repeat chest CT scan in the COVID-19 pandemic

The role of CT for Covid-19 patient's management remains poorly defined

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): a pictorial review

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Deep learning-based model for detecting 2019 novel coronavirus pneumonia on highresolution computed tomography

Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms

A fully automated deep learning-based network for detecting COVID-19 from a new and large lung CT scan dataset

Deep learning system to screen coronavirus disease 2019 pneumonia

Automated detection of COVID-19 using ensemble of transfer learning with deep convolutional neural network based on CT scans

Coronavirus (covid-19) classification using ct images by machine learning methods

Diagnosis of COVID-19 pneumonia via a novel deep learning architecture

Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia

Serial quantitative chest ct assessment of covid-19: deep-learning approach

Lung infection quantification of COVID-19 in CT images with deep learning

chest CT scans with COVID-19 related findings dataset

Mosmeddata: data set of 1110 chest ct scans performed during the covid-19 epidemic

Application of feature extraction for breast cancer using one order statistic

GPUaccelerated GLRLM algorithm for feature extraction of MRI

A hybrid method for brain tumor detection using advanced textural feature extraction

Random forests

Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning

Comparison of random forest, support vector machine and back propagation neural network for electronic tongue data classification: application to the recognition of orange beverage and Chinese vinegar

LDA: a comparative analysis on EMG data from people with trans-radial amputation

Severity assessment of coronavirus disease 2019 (covid-19) using quantitative features from chest ct images

Quantitative computed tomography analysis for stratifying the severity of coronavirus disease 2019

How to cite this article: Amini N, Shalbaf