key: cord-0210064-026u6wwl authors: Luz, Eduardo; Silva, Pedro Lopes; Silva, Rodrigo; Silva, Ludmila; Moreira, Gladston; Menotti, David title: Towards an Effective and Efficient Deep Learning Model for COVID-19 Patterns Detection in X-ray Images date: 2020-04-12 journal: nan DOI: nan sha: 854b71ac18a4a8c4a55229e33e1106f23bdbd2db doc_id: 210064 cord_uid: 026u6wwl Confronting the pandemic of COVID-19 caused by the new coronavirus, the SARS-CoV-2, is nowadays one of the most prominent challenges of the human species. A key factor in slowing down the virus propagation is the rapid diagnosis and isolation of infected patients. Nevertheless, the standard method for COVID-19 identification, the Reverse transcription polymerase chain reaction (RT-PCR) method, is time-consuming and in short supply due to the pandemic. Researchers around the world have been looking for alternative screening methods. In this context, deep learning applied to chest X-rays of patients has been showing promising results in the identification of COVID-19. Despite their success, the computational cost of these methods remains high, which imposes difficulties in their accessibility and availability. Thus, in this work, we propose to explore and extend the EfficientNet family of models using chest X-rays images to perform COVID-19 detection. As a result, we can produce a high-quality model with an overall accuracy of 93.9%, COVID-19, sensitivity of 96.8% and positive prediction of 100% while having about 30 times fewer parameters than the baseline literature model, 28 and 5 times fewer parameters than the popular VGG16 and ResNet50 architectures, respectively. We believe the reported figures represent state-of-the-art results, both in terms of efficiency and effectiveness, for the COVIDx database, a database comprised of 13,800 X-ray images, 183 of which are from patients affected by COVID-19. In December of 2019, many Chinese citizens in the province of Wuhan were affected by a severe pneumonia. In January of 2020, the cause was found to be a new virus of the coronavirus family, named, SARS-CoV-2 [1] . The virus quickly spread to other countries, and in a short time, it became a pandemic, This article has supplementary downloadable material available at http://www.decom.ufop.br/csilab, provided by the authors. ©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. changing the lives of many people around the globe. In February of 2020, The World Health Organization (WHO) named the disease caused by SARS-CoV-2 as COVID-19 and researchers from different fields have turned their efforts to fight it. The COVID-19 infection may manifest itself as a flulike illness potentially progressing to an acute respiratory distress syndrome. Disease severity resulted in global public health efforts to contain person-to-person viral spread by early detection [2] . The Reverse-Transcriptase Polymerase Chain Reaction (RT-PCR) is currently the gold standard for a definitive diagnosis of COVID-19. However, false negatives have been reported (due to insufficient cellular content in the sample or inadequate detection and extraction techniques) in the presence of positive radiological findings [3] . Therefore, effective exclusion of COVID-19 infection requires multiple negative tests, possibly exacerbating test kit shortage [4] . As COVID-19 spreads in the world, there is growing interest in the role and suitability of chest radiographs (CXR) for screening, diagnosis, and management of patients with suspected or known COVID-19 infection [5] - [7] . Besides, there have been a growing number of publications describing the CXR appearance in patients with COVID-19 [4] . The accuracy of CXR diagnosis of COVID-19 infection strongly relies on radiological expertise due to the complex morphological patterns of lung involvement which can change in extent and appearance over time. The limited number of sub-specialty trained thoracic radiologists hampers reliable interpretation of complex chest examinations, specially in developing countries, where general radiologists and occasionally clinicians interpret chest imaging [2] . Deep Learning is a subset of machine learning in artificial intelligence (AI) concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. Since deep learning techniques, in particular convolutional neural networks (CNNs), have been beating humans in various tasks of computer vision [9] - [11] , it becomes a natural candidate for the analysis of chest radiography images. Deep learning has already been explored for the detection and classification of pneumonia and other diseases on radiography. In [12] , a 10-layer CNN was proposed to identify seven patterns observed in different interstitial lung diseases. In their study, 120 CT scans were divided into 14696 image patches, and the goal was to classify each patch into each pattern. The Fig. 1 : Original images and their activation maps according to the proposed approach. The first column represents the original images, and the second, the activation maps. First row presents a healthy chest x-ray sample, the second, from a patient with pneumonia, and the third one, from a patient with COVID-19. All samples correctly classified by the proposed approach. Images of pneumonia and COVID-19 are from COVIDx [8] and the normal one (healthy) are from the Internet. 1 authors reported an accuracy of 85% for the proposed method. In [11] , the authors proposed a 121-layer convolutional neural network trained on the ChestX-ray14 dataset [13] which contains over 100, 000 frontal view X-ray images for 14 diseases. The authors reported a superior performance of the method when compared with four practicing academic radiologists. Recently, in [14] , a convolutional neural network (CNN) with a branch for predicting a segmentation mask was used to classify CXRs from the RSNA dataset [15] into pneumonia-negative and pneumonia-positive. In addition, in the pneumonia positive cases, the method produces a bounding box around the lung opacities. Addressing the COVID-19, in [16] , a comparison among seven different well-known deep learning neural networks architectures was presented. In the experiments, they use a small data set with only 50 images in which 25 samples are from healthy patients and 25 from COVID-19 positive patients. The models were pre-trained with the ImageNet dataset [17] , which is a generic image dataset with over 14 million images of all sorts, and only the classifier is trained with the radiography. In their experiments, the VGG19 [18] and the DenseNET201 [19] were the best performing architectures. In [8] , a new architecture of CNN, called COVID-net, is created to classify CXR images into normal, pneumonia, and COVID-19. Differently from the previous work, they use a much larger dataset consisting of 13, 800 CXR images across 13, 645 patient cases from which 182 images belong to COVID-19 pacients. The authors report an accuracy of 92.4% overall and sensitivity of 80% for COVID-19. In [20] , the ResNet50 [21] is fine tuned for the problem of classifying CXRs into normal, COVID-19, bacterialpneumonia and viral pneumonia. The authors report better results when compared with the COVID-net, 96.23% accuracy overall, and 100% sensitivity for COVID-19. Nevertheless, it is important to highlight that the problem in [20] has an extra class and that its dataset is a subset of the dataset used in [8] . In [20] the dataset consists of 68 COVID-19 radiographs from 45 COVID-19 patients, 1203 healthy patients, 931 patients with a bacterial pneumonia and 660 patients with nonCOVID-19 viral pneumonia. Simultaneously to this work, we highlight the one from [22] , in which the authors also performed a hierarchical analysis for the task of detecting COVID-19 patterns on CXR images. A dataset was built, from other public datasets, containing 1,144 X-ray images, of which only 90 were related to COVID-19 and the remaining belonging to six other classes: five types of pneumonia and one normal (healthy) type. Several techniques were used to extract features from the images, including one based on deep convolutional networks (Inception-V3 [23] ). For classification, the authors explored classifiers such as SVM, Random Forest, KNNs, MLPs, and Decision Trees. A F1-Score of 0.89 for the COVID-19 class is reported. In spite of having a strong relation to the present work, we emphasize that a direct comparison is not possible, due to the different nature of the datasets employed on both works. It is important to highlight that, at this point, there is no other peer-reviewed work that deals with COVID-19 screening through radiography images. Unfortunately, all the related work found so far, that is [8] , [16] , [20] , [22] , are preprints available at https://arxiv.org/. From these, the only work which is fully reproducible, and because of that, the only one reliable at this point is [8] from which the authors have made all the code for model and protocols for their dataset construction available at https://github.com/lindawangg/COVID-Net. Thus, this work aims to investigate deep learning models that are capable of finding patterns in CT / X-ray images of the chest, even if the patterns are imperceptible to the human eye, and to advance on a fundamental issue: computational cost. In our opinion, the most promising method to date is the COVID-net [8] and it has over 110 million parameters (over 2GB in memory for the compact model, the large model requires over 3.5GB). We believe that a mobile application that integrates deep learning models for the task of recognizing patterns in x-rays or CT must be easily accessible and readily available to the medical staff. For such aim, the models must have a low footprint and low latency, that is, the models must require little memory and perform inference quickly to allow use on embedded devices and large scale, enabling integration with smartphones and medical equipment. Following the experimentation protocol proposed in [8] , our results suggest that it may be feasible to embed our proposed neural network model in a mobile device and make fast inferences. Despite its low computational cost, the proposed model achieves high accuracy (93.9%) and detect infection caused by COVID-19 on chest X-rays with a Sensitivity of 96.8% and Positivity Prediction of 100% (without false positives). The development of this work may allow the future construction of an application for use by the medical team, through a camera on a regular cell phone. The source code as the pre-trained models are available in https://github.com/ufopcsilab/EfficientNet-C19. Our models are capable of producing activation maps according to Figure 1 , capturing the location in which the issues caused by COVID-19 is even more pronounced. We also exploit the natural taxonomy of the problem and investigate the use of hierarchical classification for the task of COVID-19 detection on X-ray images. In addition to consuming more computational resources than the flat classification, the hierarchical one showed to be less effective for minority classes, which is the case for the COVID-19 class in this work. The remainder of this work consists of five sections. Section II defines the problem tackled in this paper. The methodology and the dataset are described in Section III. In Section IV, the results of a comprehensive set of computational experiments are presented. In Section V, propositions for future research in the area are addressed. Finally, conclusions are pointed out in Section VI. The problem addressed by the proposed approach can be defined as: given a chest X-ray, determine if it belongs to a healthy patient, a patient with COVID-19, or a patient with other forms of pneumonia. FIGURE 2 shows typical chest Xray samples in COVIDx dataset [8] .As can be seen, the model should not make assumptions regarding the view in which the X-ray was taken. Thus, given an image similar to these ones, a model must output one of the following three possible labels: • normal -for healthy patients. • COVID-19 -for patients with COVID-19. • pneumonia -for patients with non-COVID-19 pneumonias. Following the rationale in [8] , choosing these three possible predictions can help clinicians in deciding who should be prioritized for PCR testing for COVID-19 case conïňĄrmation. Moreover, it might also help in treatment selection since COVID-19, and non-COVID-19 infections require different treatment plans. We analyze the problem in two manners: 1) the traditional flat classification, in which we disregard the relationship between the classes; 2) the hierarchical classification approach, in which we assume the classes to be predicted are naturally organized into a taxonomy. In this section, we present the methodology for COVID-19 detection by means of a chest X-ray image. We detail the main datasets and briefly describe the COVID-Net [8] , our baseline method. Also, we describe the employed deep learning techniques as well as the learning methodology and evaluation. A. Datasets 1) RSNA Pneumonia Detection Challenge dataset: The RSNA Pneumonia Detection Challenge [15] is a competition that aims to locate lung opacities on chest radiographs. Pneumonia is associated with opacity in the lung, and some conditions such as pulmonary edema, bleeding, volume loss, lung cancer can also lead to opacity in lung radiography. Finding patterns associated with pneumonia is a hard task. In that sense, the Radiological Society of North America (RSNA) has promoted the challenge, providing a rich dataset. Although The RSNA challenge is a segmentation challenge, here we are using the dataset for a classification problem. The dataset offers images for two classes: Normal and Pneumonia (nonnormal). We are using a total of 16, 680 images of this dataset, of which 8, 066 are from Normal class and 8, 614 from the Pneumonia class. 2) COVID-19 image data collection: The "COVID-19 Image Data Collection" [24] is a collection of anonymized COVID-19 images, acquired from websites of medical and scientific associations [25] , [26] and research papers. The dataset was created by researchers from the University of Montreal with the help of the international research community to assure that it will be continuously updated. Nowadays, the dataset includes more than 183 X-ray images of patients who were affected by COVID-19 and other diseases, such as MERS, SARS, and ARDS. The dataset is public and also includes CT scans images. According to the authors, the dataset can be used to assess the advancement of COVID-19 in infected individuals, and also allow the identification of patterns related to COVID-19 helping in differentiating it from other types of pneumonia. Besides, CXR images can be used as an initial screening for the COVID-19 diagnostic processes. So far, most of the images are from male individuals (approx. 60/40% of males and females, respectively), and the age group that concentrates most cases is from 50 to 80 years old. 3) COVIDx dataset: In [8] , a new dataset is proposed by merging two other public datasets: "RSNA Pneumonia Detection Challenge dataset" and "COVID-19 Image Data Collection". The new dataset, called COVIDx, is designed for a classification problem and contemplates three classes: Normal, Pneumonia, and COVID-19. Most instances of the Normal and Pneumonia classes come from the "RSNA Pneumonia Detection Challenge dataset", and all instances of the COVID-19 class come from the "COVID-19 Image Data Collection". The dataset has a total of 13, 800 images from 13, 645 individuals and is split into two partitions, one for training purposes and one for testing (model evaluation). The distribution of images between the partitions is shown in Fig. 2 : Radiograph example of images from COVID-19 image data collection [24] . (a) X-ray of a 54-year-old male, infected with COVID-19 [24] . (b) X-ray a 70-year-old female, infected with COVID-19 [24] . The COVID-Net architecture is based on the generative synthesis technique [27] . The generative synthesis consists of a generative-inquisitor pair, formulated to learn how to generate neural network architectures under constraints. For the particular case of the COVID-Net, a human specialist defined two restrictions: (i) test accuracy ≥ 80% and (ii) network computational complexity ≤ 2.5 billion multiply-accumulate (MAC) operations. The human also has to determine the basic network block. For the COVID-Net, the main building block is the residual projection-expansion-projection-extension (PEPX), formed by the operations illustrated by FIGURE 3 inspired on residual blocks of [28] . The COVID-Net was pre-trained on the ImageNet and then fine-tuned with the COVIDx dataset, using transfer learning. Weights are updated by the Adam Optimizer, with a schedule rule decreasing the learning rate by a factor of 10 in the event of stagnation during training ('patience policy'). For fine-tuning, learning rate started with 2 −5 , number of epochs = 10, batch size = 8, factor = 0.7 and patience = 5 were used. Translation, rotation, horizontal flip, and intensity shifts were also used to augment training data. Finally, a batch rebalancing strategy was used to promote a better distribution of data during the creation of mini-batches at training time. The EfficientNet [29] is in fact a family of models defined on the baseline network described in The rationale behind the EfficientNet family is to start from high quality yet compact baseline model and uniformly scale each of its dimensions systematically with a fixed set of scaling coefficients. Formally, an EfficientNet is defined by three dimensions: (i) depth; (ii) width; and (iii) resolutions as illustrated in FIGURE 5 . Starting from the baseline model in TABLE V each dimension is scaled by the parameter φ according to where α, β, γ are constants obtained by a grid search experiment. As stated in [29] , Eq. 1 provides a nice balance between performance and computational cost. The coefficient φ controls the available resources. Eq. 1 determines the increase or decrease of model FLOPS when depth, width or resolution are modified. Notably, in [29] , a model from EfficientNet family was able to beat the powerful GPipe Network [19] on the ImageNet dataset [31] running with 8.4x fewer parameters and being 6.1x faster. In classification problems, it is common to have some sort of relationship among classes. Very often, on real problems, the classes (the category of an instance) are organized hierarchically, like a tree structure. According to Silla Jr. and Freitas [32] , one can have three types of classification: flat classification, which ignores the hierarchy of the tree; local classification, in which there is a set of classifiers for each level of the tree (one classifier per node or level); and finally, global classification, in which one single classifier is built with the ability to classify any node in the tree, besides the leaves. The most popular type of classification in the literature is the flat one. However, here we propose the use of local classification, which we call hierarchical classification. Thus, the target classes are located in the leaves of the tree, and in the intermediate nodes, we have classifiers. In this work, we need two classifiers, one at the root node, dedicated to discriminate between the Normal and Pneumonia classes, and another one in the next level dedicated to discriminate between pneumonia types. The problem addressed here can be mapped as the topology depicted in FIGURE 6 in which there are two levels of classification. To make the class inference for a new instance, first, the instance is presented to the first classifier (in the root node). If it is predicted as "Normal", the inference ends there. If the instance is considered "Pneumonia", it is then presented to the second classifier, which will discern whether it is a Pneumonia caused by "COVID-19" or "Not". Deep learning models are complex, and therefore require a large number of instances to avoid overfitting, i.e., when the learned network performs well on the training set but underperform on the test set. Unfortunately, for most problems in real-world situations, data is not abundant. In fact, there are few scenarios in which there is an abundance of training data, such as the ImageNet [31] , in which there are more than 14 million images of 21, 841 classes/categories. To overcome this issue, researchers rely on two techniques: data augmentation and transfer learning. We also detail here the proposed models, based on EfficientNet. 1) Image Pre-processing and Data Augmentation: Several pre-processing techniques may be used for image cleaning, noise removal, outlier removal, etc. The only pre-processing applied in this work is a simple intensity normalization of the image pixels to the range [0, 1]. In this manner, we rely on the filters of the convolutional network itself to perform possible data cleaning. Data augmentation consists of expanding the training set with transformations of the images in the dataset [33] provided that the semantic information is not lost. In this work, we applied three transformations to the images: rotation, horizontal flip, and scaling, as such transformations would not hinder, for example, a physician to interpret the radiography. Figure 7 presents an example of the applied data augmentation. 2) Proposed models: The EfficientNet family has models of high performance and low computational cost. Since this research aims to find efficient models capable of being embedded in conventional smartphones, the EfficietNet family is a natural choice. We explore the EfficientNets by adding more operator blocks atop of it. More specifically, we add four new blocks, as detailed in TABLE III. Since the original EfficientNets were built to work on a different classification problem we add new fully connected layers (FC) responsible for the last steps of the classification process. We also use batch normalization (BN), dropout, and swish activation functions for the following reasons. The batch normalization constrains the output of the last layer in a range, forcing zero mean and standard deviation one. That acts as regularization, increasing the stability of the neural network, and accelerating the training [34] . The Dropout [35] is perhaps the most powerful method of regularization. The practical effect of dropout operation is to emulate a bagged ensemble of multiple neural networks by inhibiting a few neurons, at random, for each mini-batch during training. The number of inhibited neuronal units is defined by the dropout parameter, which ranges between 0 to 100 percent. The most popular activation function is the Rectified Linear Unit (ReLU), which can be formally defined as f (x) = max(0, x). However, in the added block we have opted for the swich activation function [36] defined as: (2) Differently from the ReLU the swish activation produces a smooth curve during the minimization loss process when a gradient descent algorithm is used. Another advantage of the swish activation regarding the ReLU, it does not zero out small negative values which may still be relevant for capturing patterns underlying the data [36] . 3) Transfer learning: Instead of training a model from scratch, one can take advantage of using the weights from a pre-trained network and accelerate or enhance the learning process. As discussed in [37] , the initial layers of a model can be seen as feature descriptors for image representation, and the latter ones are related to instance categories. Thus, in many applications, several layers can be re-used. The task of transfer learning is then to define how and what layers of a pre-trained model should be used. This technique has proved to be effective in several computer vision tasks, even when transferring weights from completely different domains [33] , [38] . The steps for transfer of learning are: 1) Copying the weights from a pre-trained model to a new model; 2) Modifying the architecture of the new model to adapt it to the new problem, possibly including new layers; 3) Initialize the new layers; 4) Define which layers will pass through a new the learning process; and 5) training (updating the weights according to the loss function) with a suitable optimization algorithm. We apply transfer learning to EfficientNets pre-trained on the ImageNet dataset [31] . It is clear that the ImageNet domain is much broader than the chest X-rays that will be presented to the models in this work. Thus, the imported network weights are taken just as an initial solution and are all fine-tuned (i.e., the weights from all layers) by the optimizer over the new training phase. The rationale is that the imported models already have a lot of knowledge about all sorts of objects. By permitting all the weights to get fine-tuned we allow the model to specialize to the problem in hands. In the training phase, the weights are updated with the Adam Optimizer and a schedule rule decreasing the learning rate by a factor of 10 in the event of stagnation ('patience=2'). The learning rate started with 10 −4 , and the number of epochs fixed at 10. The final evaluation is carried out with the COVIDx dataset, and since the COVIDx comprises a combination of two other public datasets, we follow the script 3 provided in [8] to load the training and test sets. The data is then distributed according to the TABLE I. In this work, three metrics are used to evaluate models: accuracy (Acc), COVID-19 sensitivity (Se C ), and COVID-19 positive prediction (+P C ), i.e., Acc = T P N + T P P + T P C #samples wherein T P N , T P P , T P C , F N C , and F P C stand for the normal samples correctly classified, non-COVID-19 samples correctly classified, the COVID-19 samples correctly classified, the COVID-19 samples classified as normal or non-COVID-19, the non-COVID-19 and normal samples classified as COVID-19. The number of multiply-accumulate (MAC) operations are used to measure the computational cost. In this section, we present the experimental setup and results for both flat and hierarchical approaches. The execution environment of the computational experiments was conducted on an Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz, 64Gb Ram, two Titan X with 12Gb, and the TensorFlow/Keras framework for Python. Three different training set configurations were analyzed with the COVIDx dataset: i) (Raw Dataset) -the raw dataset without any pre-processing; ii) (Raw Dataset + Data Augmentation) -the raw dataset with a data augmentation of 1,000 new images on COVID-19 samples and a limitation of 4,000 images for the two remaining classes; and iii) (Balanced Dataset) -the dataset with a 1,000 images per class achieved by data augmentation on COVID-19 samples and undersampling the other two classes to 1, 000 samples each one. Learning with an unbalanced dataset could bias the prediction model towards the classes with more samples, leading to inferior classification models. In this work, we evaluate two scenarios: flat and hierarchical. Regardless of the scenarios, the three training sets remain the same (Raw, Raw + Data Augmentation, and Balanced). However, for the hierarchical case, there is an extra process to split the sets into two parts: the first part, the instances of Pneumonia and COVID-19 classes are joined and receive the same label (Pneumonia). In the second part, the instances related to the Normal class are removed, leaving in the set 3 https://github.com/lindawangg/COVID-Net only instances related to Pneumonia and COVID-19. Thus, two classifiers are built for the hierarchical case, and each one works with a different set of data (see Section III-D for more details). We evaluate four families of convolutional neural networks: EfficientNet, MobileNet, VGG and ResNet. Their features are summarized in TABLE IV. Among the presented models, we highlight the low footprint of MobileNet and EfficientNet. Regarding the base models (B0-B5 models of EfficientNet family), the simplest one is the EfficientNet-B0. Thus, we assess the impact of the different training sets and the two forms of classification (flat and hierarchical) from this one. The results are shown in TABLE V. Since there are more pneumonia, and normal x-ray samples than COVID-19, the neural network learning process tends to improve the classification of the majoritarian classes, since they have more weight for the loss calculation. This may justify the results obtained by balancing the data. As described in Section III-D, the hierarchical approach is also evaluated here. First, classes of COVID-19 and common Pneumonia are combined and presented to the first level of classification (Normal vs Pneumonia). At the second level, another model classifies between pneumonia caused by COVID-19 and other causes. It is possible to see on TABLE V that better results are achieved with the flat approach on balanced data. This scenario is used to evaluate the remaining network as base architectures. The training loss for this scenario is presented in FIGURE 8 . The results of all evaluated architectures are summarized in TABLE VI. We stress that we adapted all architectures by placing the same four blocks on top. It can be seen that all the networks have comparable performances in terms of accuracy. However, the more complex the model is, the worse is the performance for the minority class, the COVID-19 class. The cost of a model is related to the number of parameters. The higher the number of parameters, the higher the amount of data the model needs to adjust them. Thus, we hypothesized that the lack of a bigger dataset may explain the difficulties faced by the more complex models. TABLE VII presents a comparison of the proposed approach and the one proposed by Wang et al. [8] (COVID-net) under the same evaluation protocol. Even though the accuracy is comparable, the proposed approach presents an improvement on positive prediction without losing sensitivity. Besides, a significant reduction both in terms of memory (our model is >15 times smaller) and latency is observed. It is worth highlighting that Wang et al. [8] apply data augmentation to the dataset but it is not clear in their manuscript how many new images are created. The COVID-Net [8] is a very complex network, which demands a memory of 2.1GB (for the smaller model) and performs over 3.5 billion MAC operations implying three main drawbacks: computation-cost, time-consumption, and infrastructure costs. A 3.59 billion MAC operations model takes much more time and computations than a 11.5 million MAC model -in the order of almost 300 times -, and the same GPU necessary to run one COVID-Net model can run more than 15 models of the proposed approach (based on the EfficienteNet B3 flat approach) keeping a comparable (or even better) figures. The improvements, in terms of efficiency, are even greater using the EfficientNet B0 -with a small trade-off in terms of the sensitivity metric. The complexity can hinder the use of the model in the future, for instance, on mobile phones or common desktop computers (without GPU). The presence of the COVID-19 infection can be observed through some opacity (white spots) on chest radiography imaging. In FIGURE 9, we present an X-ray image of a nonhealthy person and its activation map. The main activation spots of the proposed approach have a considerable overlay with opacity points, which could indicate the presence of Pneumonia or COVID-19. In FIGURE 11 , the confusion matrices of flat and hierarchical approaches are presented. It is possible to observe that the hierarchical model classifies the normal class better, though it also shown a noticeable reduction in terms of sensitivity and positive prediction for the COVID-19 class. One hypothesis is that both Pneumonia and COVID-19 classes are similar (both kinds of pneumonia) and share key features. Thus, the lack of normal images on second classification level reduces the diversity of the training set, interfering with model training. Besides, the computational cost is twice higher than flat classification since two models are required. We summarize our findings as follows. • An efficient and low computational approach was proposed to detect COVID-19 patients from chest X-ray images. Even with only a few images of the COVID-19 class, insightful results with a sensitivity of 90% and a positive prediction of 100% were obtained, with the evaluation protocol proposed in [8] . • Regarding the hierarchical analysis, we conclude that there are no significant gains that justify the use on the present task. Although it classifies better the majority classes, we believe that the main purpose of this work is focused on detecting the minority class (COVID-19) . Also, the computational cost is significantly higher for the hierarchical approach. • The proposed network blocks, put on top of the base models, showed to be very effective for the CRX detection problem, in particular, CRX related to COVID-19. • The evaluation protocol proposed in [8] is based on the public dataset "COVID-19 Image Data Collection" [24] , which is being expanded by the scientific community. With more images from the COVID-19 class, it will be possible to improve the training. However, the test partition tends to become more challenging. For sake of reproducibility and future comparisons of results, our code is available at https://github.com/ufopcsilab/EfficientNet-C19. • The Internet of Medical Things (IOMT) [39] is now a hot topic on industry. However, the internet can be a major limitation for medical equipment, especially in poor countries. Our proposal is to move towards a model that can be fully embedded in conventional smartphones (edge computing), eliminating the use of the internet or cloud services. In that sense, the model achieved in this work requires only 55Mb of memory and has a viable inference time for a conventional cell phone processor. In this paper, we exploit an efficient convolutional network architecture for detecting any abnormality caused by COVID-19 through chest radiography images. Experiments were conducted to evaluate the neural network performance on the COVIDx dataset, using two approaches: flat classification and hierarchical classification. Although the datasets are still incipient and, therefore, limited in the number of COVID-19 related images, effective training off the deep neural networks has been made possible with the application of transfer learning and data augmentation techniques. Concerning evaluation, the proposed approach brought improvements compared to baseline work, with an accuracy of 93.9%, COVID-19 Sensitivity of 96.8% and Positivity Prediction of 100% with a computational efficiency more than 30 times higher. We believe that the current proposal is a promising candidate for embedding in medical equipment or even physicians' mobile phones and for helping in the diagnosis screening of COVID-19, as more mature COVID-19 image datasets are made available. WHO. coronavirus disease (covid-19) outbreak situation Novel screening and triage strategy in iran during deadly coronavirus disease 2019 (covid-19) epidemic: Value of humanitarian teleconsultation service COVID-19 pneumonia: what is the role of imaging in diagnosis? Acr recommendations for the use of chest radiography and computed tomography (ct) for suspected covid-19 infection Use of chest ct in combination with negative rt-pcr assay for the 2019 novel coronavirus but high clinical suspicion Correlation of chest ct and rt-pcr testing in coronavirus disease 2019 (covid-19) in china: a report of 1014 cases Imaging profile of the covid-19 infection: radiologic findings and literature review Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest radiography images Deep learning Fixing the train-test resolution discrepancy: Fixefficientnet Chexnet: Radiologistlevel pneumonia detection on chest x-rays with deep learning Lung pattern classification for interstitial lung diseases using a deep convolutional neural network Hospitalscale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases Identifying pneumonia in chest x-rays: A deep learning approach RSNA pneumonia detection challenge Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images ImageNet: A Large-Scale Hierarchical Image Database Very deep convolutional networks for large-scale image recognition Gpipe: Efficient training of giant neural networks using pipeline parallelism Covid-resnet: A deep learning framework for screening of covid19 from radiographs Inception-v4, inception-resnet and the impact of residual connections on learning Covid-19 identification in chest x-ray images on flat and hierarchical classification scenarios Rethinking the inception architecture for computer vision Covid-19 image data collection Facing the covid-19 emergency: we can and we do COVID-19. database Ferminets: Learning generative machines to generate efficient neural networks via generative synthesis Deep residual learning for image recognition Efficientnet: Rethinking model scaling for convolutional neural networks Proceedings of the IEEE conference on computer vision and pattern recognition Imagenet large scale visual recognition challenge A survey of hierarchical classification across different application domains Deep learning Batch normalization: Accelerating deep network training by reducing internal covariate shift Dropout: a simple way to prevent neural networks from overfitting Searching for activation functions Learning and transferring mid-level image representations using convolutional neural networks Deep periocular representation aiming video surveillance Internet of medical things (iomt): applications, benefits and future challenges in healthcare domain The authors thank UFOP and funding Brazilian agencies CAPES, FAPEMIG and CNPq. We gratefully acknowledge the support of NVIDIA Corporation with the donation of two Titan X Pascal GPU used for this research.