key: cord-0056377-0g6bself
authors: Ben Ayed, Mossaad; Massaoudi, Ayman; Alshaya, Shaya A.
title: Smart Recognition COVID-19 System to Predict Suspicious Persons Based on Face Features
date: 2021-02-15
journal: J
DOI: 10.1007/s42835-021-00671-2
sha: 9aab81a2229fb445630614c2b4c2712b809a8a6a
doc_id: 56377
cord_uid: 0g6bself

The coronavirus (COVID-19) is identified at first in Wuhan in December 2019. The apparition of the COVID-19 virus is widely spread to concern all countries worldwide. The World Health Organization (WHO) on March 11 declare COVID-19 a pandemic. This Virus causes a serious infection of the respiratory system. Its high transmission constitutes great problems and challenges. The WHO proposes many actions to limit the spread of the virus such as quarantine and decrease or halt flights between states. The actions taken by states in airports are to detect suspicious persons with COVID-19. We aimed to provide a Computer-Aided Diagnosis (CAD) framework to predict suspicious COVID-19 person. This prediction identifies suspicious persons who suffer from shortness breath which is the main symptom of this disease. Extract shortness breath anomaly through the estimated heart rate from face based-video is the main contribution of the present paper. We developed a Smart Recognition COVID-19 (SRC) system to estimate the breath score. In conclusion, our study achieves an accurate breath score. The error is about 1 breath per minute. The proposed solution is of great importance because it helps managers in the airport to predict suspicious COVID-19 passengers.

The coronavirus 2 (SARS-CoV-2) called COVID-19 appeared in Wuhan China since December 2019. This new virus damages severely the respiratory system and led to a shortness breath [1] . The COVID-19 is quickly spread through other countries. At the end of September 2020, there are more than 30 million coronavirus cases in which more than 900 000 deaths. An average of one hundred thousand cases per day and an average of three thousand deaths per day. These statistics prove the seriousness of the situation. It is affecting 215 countries and territories around the world [2] . The main problem of this virus compared with previous viruses is the increased spread rate [3] [4] [5] .

Many states attempt to reduce the spread rate by applying confinement and quarantine strategies. Airports and hospitals perform preventive systems to identify COVID-19 persons and separate them from health persons [4, 6] . Despite all these preventions, many countries that have advanced health-care systems as China, Italy, USA, Spain, and France are unable to support the increasing cases of COVID-19 [2, 6] .

The COVID-19 is characterized by three essential symptoms: fever, cough, and shortness of breath [7] . This paper tries to identify an infected passenger with COVID-19 based on the Breath Score (BS) as an indication of the shortness of breath. Most airports used a thermal camera to read the passenger's temperature to identify suspicious ones. Unfortunately, some passengers who did not want to be confined take paracetamol to avoid the fever. Coughs have partially occurred. For these reasons, we choose to identify suspicious COVID-19 passengers according to the breath score computed through face features. The BS is the most reliable indication of a suspicious COVID-19 passenger.

The respiratory system ensures airflow. The breathing phase is delimited into two steps. The first one enters oxygen to the cell through inhalation. The exhalation step goes out of the carbon dioxide from the intracellular.

The estimation of the Respiratory Rate (RR) is the purpose of many devices. This rate is an important key to evaluate the health condition of a patient and it is a primary factor to predict persons infected by the COVID-19 virus [8] .

The COVID-19 is characterized by three essential symptoms: fever, cough, and shortness of breath [7, 9] . The need for continuous surveillance of shortness breath is increasingly requested in the healthcare system [10] . Many attempts are proposed in the literature to provide a portable or wearable devices to estimate/compute the RR.

Pulse oximeters devices characterized by their noninvasiveness, availability, and cost-effectiveness are widely used to evaluate the respiratory status of patients [11] . The shortcoming of these devices is the late detection of respiratory depression [12] . Recently, some devices are developed according to advanced signal processing from photoplethysmography [13] . Another accurate non-invasive method is performed to monitor the respiration based on changes measuring variations of thoracic electrical impedance. These attempts not only provide the RR but also compute the ventilation factor [14] . Unfortunately, methods based on impedance suffer from the accuracy of the RR when the rate is small. Therefore, this method is not suitable in the case of shortness of breath [11] .

Devices using piezoelectric sensors are developed to estimate the RR through the motion of the chest and abdomen [15] .

Acoustic transducers sensors tied to the throat are used to compute the RR. These devices achieve high accuracy [7, 9] . The RR accuracy decreases when noise is added to the acoustic signal such as coughs, snores, cries or sneezes [16] .

Revolutionary devices based on contactless methods are proposed. Optical cameras and radars are used to supervise the activities of the respiratory system. The authors in [17, 18] use a thermal camera to estimate the RR according to variations of colors of the skin of patients. RR can be determined via camera either radar that supervises the movement of the chest through tracking pixels' coordinates [19] . The motion of bodies constitutes a great challenge for the accuracy of the RR.

A recent attempt described in [20] presents a non-invasive method to measure the RR. The rate is computed using the thermal transfer between the breathing airflow under the nose and the mouth.

This paper tries to identify an infected passenger with COVID-19 based on the Breath Score (BS) as an indication of the shortness of breath. The proposed SRC system is implemented on Raspberry PI4 board.

Most airports used a thermal camera to read the passenger's temperature to identify suspicious ones. Unfortunately, some passengers who did not want to be confined take paracetamol to avoid the fever. Coughs have partially occurred. For these reasons, we choose to identify suspicious COVID-19 passengers according to the breath score computed through face features. The BS is the most reliable indication of a suspicious COVID-19 passenger.

Section 2 describes the proposed method to estimate the BS based on the video streaming of the face. Section 3 highlights the achieved results performed by the proposed system. Finally, Sect. 4 concludes the paper and proposes some perspectives.

Provide an accurate tool to predict suspicious COVID-19 persons helps managers at airports specially to separate between healthy passengers and infected ones.

The proposed method aims to recognize the shortness of breath as an indication of a suspect COVID-19 person. We attempt to find the breath score according to a face basedvideo. The method is composed of two phases as mentioned in Fig. 1 .

The first phase estimates the heart rate by following these steps: (a) find the forehead as the best Region Of Interest (ROI), (b) apply the Fast Fourier Transformer (FFT) to move from spatial description to temporal one, (c) apply in cascade the Band Pass filter, Eulerian transformer, and Lagrangian transformer. The heart rate is the result of these steps. This phase is described in-depth in our previous work [21] .

The second phase provides the Respiration Rate (RR) using the heart rate provided by the first phase [22] . The RR is computed using two parameters as shown in Eq. 1: the HR and the Pulse-Respiration Quotient (PQR).

As described in [22] , PRQ depends on different factors: sex, age, behavior, psychology, body posture, environmental influences, and health status. The PRQ normalization is

defined as the optimal ratio concerning the functioning of the cardiovascular. Figure 2 defines the PRQ normalized based on the HR (Fig. 2 .a) and the RR (Fig. 2.b) .

The relationship between the PRQ and HR can be defined by a linear equation. The second curve (b) can be defined by an exponential equation, see Eq. 2. Equation 3 is computed through the Eq. 2 to provide an expression of RR using only the HR. Figure 3 presents the curve of the RR based on HR. The breath score is computed for 1 min. The curve proves that when the heart rate is increasing the BS is decreased. A suspicious COVID-19 person is identified when the BR is inferior to 10.

The SRC system is implemented as a CAD tool to estimate the BS using the face features extracted from video streaming. Figure 4 shows the implementation results of the SRC system developed using Python language associated with the OpenCV library for video processing.

To evaluate the accuracy of the achieved results of the CAD, 10 participants use it. Each one provides ten measurements. These participants are chosen with the following criterion: equitable sex five, adults ([30, 60] range), healthy person, and multi-status.

The computed results through the SRC system are compared with Breath Counter Application found in the Play Store developed by Vadion company [23] . The Mean Absolute Error (MAE) and the Mean Squared Error (MSA) are computed for evaluation. The Mean Absolute Error (MAE) [24] computes the average of all absolute errors which is defined by Eq. 4.

The Mean Squared Error (MSE) [24] computes the average of all errors related to the distance between the regression line and the value, see Eq. 5. When the MSE is smaller, the accuracy is higher. where x is the true value provided by the Breath Counter App and xi is the measured value. Figure 5 describes all attempts done based on the SRC measures and Breath Counter App measures. Curves present a higher rate of overlap. According to these results, Table 1 shows the MAE and the MSE metrics to evaluate the accuracy of the proposed SRC system. Most MAE is inferior to 1 and the MSE belongs to [0.5,2] range. The computed averages of the MAE and the MSE prove that the error is about 1 breath per minute. This is an accurate result which managers and deciders could rely on to identify respiratory problems and especially the case of COVID-19 disease. Figure 6 shows curves associated with the MAE and the MSE metrics related to each participant. Some cases present an increased error (P1, P5, and P10). This is could be justified by the psychological factors which are not considered in the present paper.

In light of this discussion, we announce that the achieved results are with high accuracy. The SRC system estimates the breath score accurately. Therefore, this CAD tool could help airport managers to recognize suspicious COVID-19 persons and decrease the rate of infections.

The SRC system is implemented on the Raspberry PI4 board to discuss the reliability. We focus on this section on hardware performances which are: (1) the memory usage, (2) the CPU usage, (3) the CPU load, (4) the CPU temperature, and (5) the GPU temperature. Table 2 shows the hardware performances.

The SRC requests a memory size of around 56 MB which is provided by the board. The CPU usage is 92%. A monoprocessor architecture is sufficient in this case. The CPU load corresponds to the number of processes running at the same time per second. There is no need for a bigger buffer because of the CPU load in inferior to 2 processes/s. The temperature of components is a critical hardware factor to avoid the fall-down of the system. The CPU temperature is 88 °C which is acceptable, but the GPU temperature (92 °C) needs a cooling mechanism to be mounted.

Since the epidemic COVID-19 virus that swept the whole world, many studies are involved in different disciplines. All studies prove that this virus damages the respiratory system. Therefore, during this paper, we attempt to estimate the breath score via face based-video related to passengers. Based on the BS a passenger could be suspicious COVID-19. To achieve our objective, we computed at first the heart rate. Then according to the normalized PQR, the SRC system computes the BS. Achieved results show an accurate estimation in which the error is about 1 breath per minute. The hardware implementation on a raspberry pi 4 board proves the high reliability of the proposed SRC system. This work would be enhanced by supporting all factors related to PQR and implement the SRC system into an embedded system. Acknowledgments The authors would like to thank Deanship of Scientific Research at Majmaah University for funding this project under the number R-2021-2.

The authors certify that the data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to their containing information that could compromise the privacy of research participants.

The code is not publicly available. It can be requested from the corresponding author.

Conflict of interests The authors declare that there is no conflict of interest regarding the publication of this paper.

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Dr. Shaya Alshaya Coming from the invaluable educational background in both under and postgraduate degrees from Saudi Arabia, USA, and Italy, Dr. Alshaya has put his knowledge into practicality as an international entrepreneur, his endeavors include education and research, ICT sector. He is a member of the National Association of Industrial Technology, the Institute of Electrical Engineers and the Sloan-C Online Association amongst others. Dr. Alshaya has a wealthy experience as Head of IT in Majmaah University, Saudi Arabia. His experience as a researcher, consultant, project manager and information technologist has allowed him to bring his extensive experience of technology planning, training and implementation to all of the projects he endeavors. The numerous national and international companies that utilize his expertise as a consultant evidence Dr. Alshaya's success. Dr. Alshaya's wealthy experience, vast knowledge is integral to his vision and future projects.

Presumed asymptomatic carrier transmission of COVID-19

Corona virus cases

Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia

Interventions to mitigate early spread of SARS-CoV-2 in Singapore: a modelling study

A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating personto-person transmission: a study of a family cluster

Coronavirus disease 2019 ( COVID-19): situation report

Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China

APACHE-acute physiology and chronic health evaluation: a physiologically based classification system

Assessment of noninvasive acoustic respiration rate monitoring in patients admitted to an emergency department for drug or alcoholic poisoning

Why don't nurses monitor the respiratory rates of patients?

Critical review of non-invasive respiratory monitoring in medical care

A brief review of non-invasive monitoring of respiratory condition for extubated patients with or at risk for obstructive sleep apnea after surgery

Pulse oximetry-derived respiratory rate in general care floor patients

Low minute ventilation episodes during anesthesia recovery following intraperitoneal surgery as detected by a non-invasive respiratory volume monitor

Clinical evaluation of a novel respiratory rate monitor

Paper-based electrical respiration sensor

Estimating respiratory rate in post-anesthesia care unit patients using infrared thermography: an observational study

Robust respiration detection from remote photoplethysmography

Noncontact vision-based cardiopulmonary monitoring in different sleeping positions

Non-invasive continuous respiratory monitoring using temperature-based sensors

Suspicious behavior recognition based on face features

The pulse-respiration quotient: a powerful but untapped parameter for modern studies about human physiology and pathophysiology

Statistical methods for the qualitative assessment of dynamic models with time delay (R Package qualV)

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations