key: cord-0547645-sltyp00j
authors: Maghdid, Halgurd S.; Ghafoor, Kayhan Zrar; Sadiq, Ali Safaa; Curran, Kevin; Rabie, Khaled
title: A Novel AI-enabled Framework to Diagnose Coronavirus COVID 19 using Smartphone Embedded Sensors: Design Study
date: 2020-03-16
journal: nan
DOI: nan
sha: d511f3a98d460f7967dd490e34d664caac1529d1
doc_id: 547645
cord_uid: sltyp00j

Coronaviruses are a famous family of viruses that causes illness in human or animals. The new type of corona virus COVID-19 disease was firstly discovered in Wuhan-China. However, recently, the virus has been widely spread in most of the world countries and is reported as a pandemic. Further, nowadays, all the world countries are striving to control the coronavirus disease COVID-19. There are many mechanisms to detect the coronavirus disease COVID-19 including clinical analysis of chest CT scan images and blood test results. The confirmed COVID-19 patient manifests as fever, tiredness, and dry cough. Particularly, several techniques can be used to detect the initial results of the virus such as medical detection Kits. However, such devices are incurring huge cost and it takes time to install them and use. Therefore, in this paper, a new framework is proposed to detect coronavirus disease COVID-19 using onboard smartphone sensors. The proposal provides a low-cost solution, since most of the radiologists have already held smartphones for different daily-purposes. People can use the framework on their smartphones for the virus detection purpose. Nowadays, smartphones are powerful with existing computation-rich processors, memory space, and large number of sensors including cameras, microphone, temperature sensor, inertial sensors, proximity, colour-sensor, humidity-sensor, and wireless chipsets/sensors. The designed Artificial Intelligence (AI) enabled framework reads the smartphone sensors signal measurements to predict the grade of severity of the pneumonia as well as predicting the result of the disease.

many countries internationally [1] , [2] . The Emergency committee of WHO on January 30, 2020 declared as a pandemic disease due to its fast person-to-person spread and most of infected people do not have immunity.

The COVID-19 is common in people and many different species of animals, including camels, cattle, cats, and bats. Firstly, infected people with novel COVID-19 at the epicenter in Wuhan had links with seafood and live animal markets, indicating animal-to-person spread. Thereafter, rising number of patients who did not have contact with the live animals, resulted in person-to-person spread [3] . Subsequently, the WHO on March 11,2020 announced the novel COVID-19 outbreak a pandemic as at that date the number of cases reached 118,000 and more than 4000 deaths, and people got infected in every continent [4] .

The COVID-19 and human coronaviruses are categorised under the family of Coronaviridae. These viruses infect people with moderate cold Middle East Respiratory Syndrome (MERS) or Severe Acute Respiratory Syndrome [5] . SARS is also a viral respiratory disease caused by SARS-associated coronavirus (SARS-CoV), first in 2003 reported in Southern China and was spread in many countries worldwide. Moreover, MERS virus cases was first announced in Saudi Arabia and caused the death of 858 out of 2494 person. Based on the analysis of virus genomes, this virus was originated in bats [6] . The clinical presentation of the COVID-19 is complicated and could be manifested as fever, cough and severe headache. There are several techniques for COVID-19 detection, such as A Nucleic Acid Test (NAT) and Computed Tomography (CT) scan. The NAT is utilized to detect specific nucleic acid sequence and species of organism, predominately a virus or bacteria that can cause disease in blood, tissue or urine. Despite NAT technique and detection kits becoming significant for COVID-19 virus detection, CT scan is most effective and functional for detecting the severity and degree of the lung inflammation [7] . The National Health Commission of China confirmed the inclusion of radiographic presentation of pneumonia for clinical diagnostic standard in Hubei Province [8] . This assures the significant of the CT scan images for the diagnosis of COVID-19 pneumonia severity.

WHO reported COVID-19 as a pandemic and thousands of patients spending hours waiting at the hospital for CT scan image examination. This is not only an overload of the medical system, gest the patients frustrated, but also leads to a serious risk of cross-infection with other patients. Particularly, at the province of Hubei, suspected cases, confirmed COVID-19 patients and cases under medical observation need to go through CT imaging of lung. Even the opacity of infected lung are small in the early stage of a patient suspected of contracting COVID-19. Besides, the number of radiologists compares much less than the increased number of patients. As a result, this makes medical systems and physicians overloaded. This leads to late detection and quarantine of infected people and less efficient treatment of patients [8] . More precisely, recently in Italy, hospitals are only receiving high priority people, who has huge fever and shortness of breathing [9] .

The pandemic of COVID-19 and the resultant huge demand for diagnosis has driven companies, academician and researchers to provide high responsive, intelligent and more efficient detection methods. Ping a Smart Healthcare company, revealed an intelligent technique for COVID-19 smart CT image reading system that can analyse results in about 15 seconds with an accuracy rate above 90% [10] . However, both Reverse Transcription Polymerase Chain Reaction (RT-PCR) and CT scan for COVID-19 diagnosis are not perfect [10] . Thus, most reliable technique is the combination of several methods. Furthermore, medical detection kits are used to detect the COVID-19. But, this device is costly and requires installation for diagnosis.

Modern smartphones are embedded with large number of sensors and have powerful computation capabilities. Using smartphones, it is possible to sense information about daily activities and even capturing visual data [11] . One of the important features of smartphones is the capability of capturing, collecting and storing large volume data from either suspected and confirmed COVID-19 cases. In particular, a smartphone has a capability to scan CT images of a COVID-19 patient for analysis purpose. Further, multiple CT images of the same COVID-19 patient can be feed to the smartphone for comparative analysis on how lesions have been developed. The analysis is very useful to the suspected COVID-19 cases to diagnosis and monitor the grade of lung inflammation.

In this paper, we present a new framework to diagnose the coronavirus disease COVID-19 using onboard smartphone sensors. The framework provides a low cost solution, since most of the radiologists have already have smartphones for different daily-purposes. People can use the framework on their smartphones for COVID-19 diagnosis. Moreover, the people can use the proposed smartphone-based framework to monitor their grade of lung inflammation. Today's smartphones are embedded with computation-rich processors, memory space, and a large number of sensors including cameras, microphone, temperature sensor, inertial sensors, proximity, color-sensor, humidity-sensor, and wireless chipsets/sensors. The designed framework reads the smartphone sensors' signal measurements and scans CT images to identify viral pneumonia. The developed framework takes much less time to identify COVID-19 as compared to the expert radiologist. Even radiologists can use the framework to track the development of the disease grades and evaluate for treatment.

The rest of this paper is arranged as follows: Section II explains the background of the literature on developed AI systems for COVID-19 detection. This is followed by discussing an overview of the proposed approach and also presenting the detail of designed algorithm. Finally, section IV concludes the paper.

This section presents the review of very recent existing literature on techniques that are used for novel COVID-19 diagnosis. There are several ways to identify the viral pneumonia on suspected cases.

Although very scarce literature exists on the diagnosis COVID-19, because of its new emergence, there are a few state-of-the-art reviews in this field. In an attempt, the authors in [1] developed an AI engine based on deep learning to detect COVID-19 disease using high resolution CT images. However, their proposed model only relies on CT images. Based on the latest research [10] , the COVID-19 detection results are more reliable when a combination of several methods are used. In another attempt, Ping An Insurance Company of China Ltd [10] , developed a smart CT image reading system. The system can read and diagnosis in very short period of time.

This section presents the motivation and the detailed design of the proposed approach. The proposed approach elaborates the process of creating a smartphone-based framework, which includes smartphone, algorithms and embedded sensors.

The proposed framework and its associated algorithms utilise smartphone sensors to diagnose the preliminary results of the coronavirus disease COVID-19. Although, there are several methods to get the result of the disease test, the proposal provides a low cost and friendly solution. The solution could be used by radiologists or people who have held the smartphones at anytime, anywhere. Therefore, such framework is needed in the emergency situations.

To understand how the workflow of the framework, first, the symptoms of the confirmed COVID-19 patient should be realized. The well-known symptoms of the disease are: fever, fatigue, headache, nausea, dry-cough, lung CT imaging features, and shortness of breath. Each of these symptoms has its own level which could be differentiate from other disease including flue-symptoms, cold-symptoms and hay fever symptoms. For this reason, the framework is trying to discover the level of each symptoms based on the onboard sensors measurements. In addition, a set of sensors technologies are embedded on the smartphones including cameras, inertial sensors, microphone, and temperature sensor. In addition, the readings of these sensors technologies have been used for those symptoms which are exist in the coronavirus diseases. As state of the art, algorithms are separately applied on each of these sensors' readings to detect the level of the symptoms for the human-health purposes. For example, temperature-fingerprint sensor as it is located under the smartphone's touch-screen has been used to predict the fever level in [12] . Camera captured images and videos are used to detect human fatigue in different environment via human-gait analysis in [13] . Further, onboard inertial sensors (such as accelerometer sensors) have been used in [12] and [14] to detect the fatigue level, as well. In [15] , the nausea prediction has been analysis based on smartphones-enabled video observation and directly observed treatments. In another vain, both camera sensor and inertial sensors' measurements have been utilized in [16] to monitor neck posture and consequently to predict the level of humanheadache. A comprehensive work has been studied to indicate the type of cough via using smartphone-microphone chipset in [17] and [18] .

In this article, the proposed framework is aimed to use all these aforementioned sensors and algorithms (could be improved) in a single solution. This is followed by getting the predicted level results of symptoms from the applied algorithms and stored them in a dataset as a single record. Thus, such records from different patients could be collected and used as input to a machine learning techniques. There are several machine learning techniques for the human-health purposes such as: decision tree, support vector machine, knearest neighbors, and neural networks. The most recent and accurate machine learning technique is deep learning techniques, as they are referred to neural network family. Many deep learning algorithms have been utilized for the classification or recognition purposes such as convolution neural network (CNN) and recurrent neural network (RNN) algorithms. CNN is feed-forward neural network which is generally used for spatial data such as image recognition [19] . While with RNN, the output of each layer will be saved and then it will be used as an input for the next layers as well as the RNN is good for temporal data such text [20] and signal measurements [21] .

The proposed framework is constructed as a set of layers, as shown in Fig. 1 The first layer functionalities are responsible to read the data from the sensors. For example, reading the captured CT scan images of lung and captured videos through using smartphone camera, getting the inertial sensors (accelerometer sensor) measurements during 30-second sit-tostand, recording microphone voice measurements during a series of cough, and finally scanning temperature sensor measurements during fingerprint touching on the smartphone screen. The second layer is structured to configure the onboard smartphone sensors including reading intervals, image size, buffers' size, timer resolution, and etc. Further, the readings and configurations are then used as the input of the symptoms algorithms which are running on the smartphone application. The third layer of the framework provides a the calculated symptoms level, separately, and then stored as a record input to the next layer. The last layer is to applying the machine learning techniques to predict the coronavirus disease COVID-19. The machine learning techniques could be used according to the nature of the recorded data. For example, for the abnormal sub-image of the CT scan images, the CNN could be used. This is followed by using RNN for the rest of the recorded data. Therefore, a new combined CNN and RNN machine learning technique can be proposed for the Fig. 2 . CT images of the patient: opacities can be seen in the right lung of images A and B. 2 days later, an increase of opacities can be observed in images C and D. In sixth day, lesions can be observed. In images F, G, H, in ninth day, progressive lesions is shown in left and right lungs. Image F shows the appearance of small bilateral pleural effusions. [22] . framework.

In the proposed framework, one significant input to the smartphone is the CT scan images of the pneumonia-probable cases. Certainly, CT scan images is a key method to detect COVID-19. An algorithm can be developed to diagnose the lesions of a patient caused by COVID-19 and analysing its size and density. The algorithm can compare multiple CT images of the lung lesions. The most important output of the algorithm is the volume and density of lesions. The most compelling evidence of the confirmed COVID-19 case is an increase of density and volume of lesions in CT images. Certainly, comparing multiple CT images takes long time and its interpretation, by the radiologists, cannot accurately be concluded manually. Thus, the proposed framework assists radiologists and enables people to make efficient and confident decision on the suspected cases.

In Fig. 2 , the progressive effect of COVID-19 of a 61-yearold man admitted to the hospital in Lanzhou, China on January 25, 2020 due to close contact with confirmed COVID-19 patient 10 days prior. The authors in [22] , based on multiple CT images, described features of lung inflammation in a laboratory-confirmed COVID-19 patient. As can be seen, the volume quantity, grade and density of lesions and opacities are growing progressively from CT images A to G. Analysis of Fig. 2 by radiologists is very time consuming. In the proposed framework, the CT images of Fig. 2 is scanned and an algorithm can compute the volume and density of lesions and opacities in order to identify the stage of pneumonia in the course of the disease.

Further, to improve the proposed framework or to get a reliable prediction result, the recorded data and the result of the prediction could be exchanged in the cloud, as shown in Fig. 3 . Thus, by using the framework from different users or patients, the framework dataset will grow and will construct a large data set. Further, such process will provide transfer learning from multiple smartphone and various onboard sensors to the new smartphones. 

In this study, the novel coronavirus disease COVID-19 and the techniques for the COVID-19 detection are discussed in detail. Recent studies are investigated and explained regarding to their solutions from the range of radiology to the available of modern technologies including medical detection Kit, CT image reading system and AI technologies engines. These solutions suffer from either lack of the accuracy, long time to get the result, or take huge cost. However, this study proposed a novel framework to avoid such issues. Further, the proposed framework is based on using only smartphone sensors measurements and it can be used by doctor or radiologists who held the smartphone at anytime, anywhere. Further, the recommended or proposed framework could be run as an application on different smartphone-platforms, since it does not require any external or additional sensors and provides higher accuracy.

The proposed framework includes four separate layers which are: input/reading sensors' measurements layer, sensors configuration layer, computing symptoms disease layer and predict the disease layer via using combine machine learning approach. Also, the machine learning model in the final stage could be further improved by using transfer learning method when the model has worked on the cloud.

The framework is more reliable in comparison with the state-of-the-art; this is because, the framework relying on multi-readings from multiple sensors referring to the related symptoms of the disease. Hopefully, in near future, we will try to implement the designed framework on the smartphones if the related-companies (including Google, Huawei, and Apple) are supporting us to release the application, very quickly, in the play store, AppGallery and apple store.

Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography: a prospective study

Novel coronavirus in the united states

Novel coronavirus -china. world health organization

Cnn health

Novel coronavirus (2019-ncov). world health organization

Summary of probable sars cases with onset of illness from

Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in wuhan, china

Big data from small devices: The future of smartphones in oncology

Use of a smartphone thermometer to monitor thermal conductivity changes in diabetic foot ulcers: a pilot study

Smartphone-based human fatigue detection in an industrial environment using gait analysis

Fatigue detection during sit-to-stand test based on surface electromyography and acceleration: A case study

Smartphoneenabled video-observed versus directly observed treatment for tuberculosis: a multicentre, analyst-blinded, randomised, controlled superiority trial

Neck posture monitoring system based on image detection and smartphone sensors using the prolonged usage classification concept

A comprehensive approach for cough type detection

Nocturnal cough and snore detection in noisy environments using smartphone-microphones

Deep learning for medical image processing: Overview, challenges and the future

Web news mining using new features: A comparative study

Gait-based gender classification using smartphone accelerometer sensor

Asymptomatic novel coronavirus pneumonia patient outside wuhan: The value of ct images in the course of the disease