key: cord-0062064-erdbu292 authors: Al-Humairi, Safaa N. Saud; Kamal, Ahmad Aiman A. title: Design a smart infrastructure monitoring system: a response in the age of COVID-19 pandemic date: 2021-04-17 journal: Innov DOI: 10.1007/s41062-021-00515-y sha: d954fe02a50d28797b747ec34a6a81d654714f18 doc_id: 62064 cord_uid: erdbu292 Since the end of 2019, COVID-19 has been a challenge for the world, and it is expected that the world must take precautionary steps to tackle the virus spreading prior produces an efficient vaccine. Currently, most government efforts seek to avoid disseminating the coronavirus and forecast probable hot areas. The most susceptible to coronaviral infection are the healthcare staff due to their daily contact with potential patients. This article proposes a COVID-19 real-time system for tracking and identifying the suspected cases using an Internet of Things platform for capturing user symptoms and notify the authority. The proposed framework addressed four main components: (1) real-time symptom data collection via thermal scanning algorithm, (2) facial recognition algorithm, (3) a data analysis that uses artificial intelligence (AI) algorithm, and (4) a cloud infrastructure. A monitoring experiment was conducted to test three different ages, kid, middle, and older, considering the scanning distance influence compared with contact wearable sensors. The results show that 99.9% accuracy was achieved within a (500 ± 5) cm distance, and this accuracy tends to decrease as the distance the camera scanning and objects increased. The results also revealed that the scanning system's accuracy had been slightly changed as the environmental temperature dropped lower than 27 °C. Based on the high-temperature presence's simulated environment, the system demonstrated an effective and instant response via sending email and MQTT message to the person in charge of providing accurate identification of potential cases of COVID-19. Coronavirus disease (also referred to as COVID-19) is a new infectious disease caused by a newly identified coronavirus. It is close to influenza viruses and poses concerns because of its spread and troubling effects. As of January 10, 2021, more than 95 million confirmed cases of COVID-19 had been recorded worldwide [1] , with approximately a 2% rise in activity per day with a record of more than 95,000 deaths were in these cases and about 4.2% of deaths. This novel coronavirus was first isolated on March 11, 2020 , and was subsequently identified as a pandemic on July 1, 2020, by World Health Organization (WHO) [2] . In the modern age of artificial intelligence (AI) [3] [4] [5] , emerging technologies are playing an imperative role in helping the world to fight the current pandemic. Coronavirus is a disease that can be transmitted with skilled nursing facilities. If 90% of the patients exposed in the facilities have symptoms, this is important in suggesting highly likely transmission. From this example, it is evident that if the symptom-based screening approach is not performed correctly, this drug will fail to identify more than 50% of patients with COVID-19 infections. Temperature measurement is essential for detecting the existence of COVID-19 in a suspect [6] . Still, it is often used as an instant test to determine whether a traveller or a resident has been infected with the disease [7] . A medication to maintain a normal body temperature can help prevent its spread. Still, there are monetary risks to the procedure used and other people's protection in the future. Continuous monitoring of the skin can be a successful method [8] since tracking this kind of way can yield better results. At present, the only way for the world to cope with this coronavirus is by distancing socially, hand-washing and face masks to flatten the curve of COVID-19. However, technologies could also slow down its spread through early detection and tracking of new cases [9, 10] . These innovations include comprehensive data and cloud and fog capabilities [11] , the use of data obtained by remote surveillance, such as mHealth, Tele-health, and patient status monitoring in real time [12] . China uses open, web-based and cloud-based tools to screen and inform comparative benefits decisions. High-performance infrared thermal cameras set-up at Taiwanese airports have been used to capture people's thermal images in real time, easily identifying people with fever [13] . In Singapore, people have their temperature measured at the entrances to offices, schools and public transport. The thermometer data are monitored and used to identify emerging hot spots and infection clusters where research could be initiated [14, 15] . The IoT-based body temperature surveillance, particularly for infants, was developed by Zakaria et al. [16] . The system is small-sized, lightweight and controls the body temperature continuously and is used efficiently for the infant. Kulkarni et al. [17] suggested another IoT-based system called Health Companion, which controls the temperature and pulse. This system monitors and collects various human body parameters, lets users monitor their physical conditions and allows doctors to analyse patients' diseases very carefully. This system is ideal for disease fever monitoring. The instrument also warns users and the clinical personnel of abrupt temperature or fever changes. The positive relation between COVID-19 mortality and health care burden was substantial, as Ji et al. [18] pointed out. Clinicians and radiologists had to minimize workload to enable patients to receive early diagnoses and early therapies. It was important. It is almost impossible to train so many experienced physicians in a big country like China to track this novel disease on time, especially in areas without an outbreak. Din and Paul [19] proposed an intelligent health monitoring and management architecture based on IoT. The architecture consists of three layers: (1) data generation from and processing of medical sensors battery-powered source; (2) processing of Hadoop; and (3) applications. The work used an energy-harvesting method using piezoelectrical devices connecting to the human body because of the battery's limited ability to power the sensors [20] . The different techniques used to track the COVID-19 pandemics are presented in Table 1 . To our knowledge, no one has developed a complete framework for using IoT technology to identify and monitor COVID-19. This paper proposes a COVID-19 artificial intelligence (AI) detection and monitoring system that would collect real-time symptom data from the thermal scanning and facial recognition system. To quickly identify potential coronaviruses cases from this real-time data, this paper proposes a framework that consists of four main components: (1) real-time symptom data collection via thermal scanning algorithm, (2) facial recognition algorithm (3) a data analysis centre that uses artificial intelligence (AI) algorithm, and (4) a cloud infrastructure. Based on the framework in Fig. 1 , there are four stages taken into consideration in this project; (1) data capture and collection, (2) data evaluation based on the pre-strain temperature algorithm, (3) data transmission to firebase cloud and (4) communication module with the building management via email and MQTT message (Message Queuing Telemetry Transport). The AI-based monitoring system's captured raw data can be turned into actionable and valuable. Through this module, multiple data communications would benefit data collaboration, sharing, and processing to hinder the spreading of the COVID-19 pandemic. These COVID-19 operation statistics can be efficiently computed if data is collected systemically. Thus, an accurate communication and tracing process will be in place. This is to identify the project requirement's knowledge in terms of the hardware and software implementations within an adequate design process according to the systems development life cycle (SDLC) diagram, as shown in Fig. 2 . During this process, objectives, data on scope, and different variables have been acknowledged and researched thoroughly. Everybody emits electromagnetic radiation from its surface, proportional to its intrinsic temperature, with a temperature above absolute zero (−273.15 °C). Infrared radiation was used to calculate the body's temperature, which is a component of this so-called intrinsic radiation. The atmosphere penetrates with this radiation. The beams are centred on a detector element with the assistance of a lens (input optics) that produces an electrical signal proportional to the radiation. The signal is amplified and converted into an output signal proportional to the object's temperature, using subsequent digital signal processing. The measurement value likely displayed on the LCD (Fig. 3) as an analogue output signal that supports easy connection to process management control systems. The proposed AI-based monitoring system aims to capture the person's information using two types of cameras, Panasonic AMG8834 with a 24 × 32 array of IR thermal sensor and a Raspberry Pi Camera. Then, transmit the data to the management remotely if the temperature triggered to be more than the threshold level. However, data processing, security, and privacy concerns need to be addressed for successful considerations. The proposed project's flow chart is shown in Fig. 4 , whereas the integration includes facial recognition, thermal scanning, and Al data analysis for the captured data. The AI's implementation aims to provide an efficient design and development for the internal process. Thereafter, the cloud data approach is applied to provide a proficient data storage from the enquired locations. It shows the proposed system's functioning procedure where it will have continued display on the Raspberry Pi screen where it searches for the face. When it detects a face, it will take a picture, including recording the body temperature. It will then send the image, including the body temperature, into the data cloud. If there is a specific body temperature above 37.5 °C, it will send an SMS of the person's body temperature and an e-mail with its picture. Every file taken on every image is written with the following format, year, month, date, time and temperature for the naming on the picture. Figure 5 shows the block diagram of the proposed system's data capturing and processing. The firebase cloud's stored data can be communicated and browsed through website, SMS, and email for mobile phone. The monitoring system will also be mirrored to a second device, either using laptops or mobile phones. The data are obtained employing manual data sharing or automated data sharing with the government-designated medical board. The images can be collected and used in social distancing steps and density-based thermal imagery to alternative sensor deployment studies. The Panasonic AMG8834 senses the thermal image object and signal, the facial image and the data obtained by the Raspberry Pi Camera with this device. The components listed here are used to build a working prototype for the project. All of the components and hardware have been picked in order to do the work efficiently and quickly at the same time, choosing the most cost-effective hardware. Raspberry Pi 4 uses a Broadcom BCM2711, Quad-core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5 GHz for the processor, which is equivalent to modern desktops now. It can run typical features same as desktops and laptops now. It has a different variance from 2 GB up to 8 GB. It is a desktop, but it also has a built-in 2.4 and 5.0 GHz IEEE 802.11ac wireless while Bluetooth 5.0. Moreover, it has 4 USB ports to be remotely checked manually using a wireless keyboard and mouse. It also has a complete 40-pin GPIO socket. It is cheaper than other models, and you can only get it for as low as RM174. It is the most used boards for programmers, and it is easy to use. This touchscreen LCD provides the ability to create a standalone device that can be utilized as a custom table or all-inone interactive interface for a future project. Its resolution is 800 × 480 with a capacitive touch display. It only uses two ports, the USB port of the Raspberry Pi, to power up the screen and then use an HDMI port to feed the Raspberry Pi's live display. The AMG8834EK is a high-performance low-gain infrared array sensor Grid-EYE test kit. The high-precision sensor for the infrared array is based on advanced MEMS technology. For thermal presence, path and temperature values, it offers digital output (I2C). A 60-degree viewing angle is included in the built-in lens. Grid-EYE uses MEMS thermopile technology to feature a lightweight SMD interface. It has a total number of 64 pixels and only operates at 3.3 V. This component will calculate temperatures between 0 and 80 °C (32 °F-176 °F) with a precision of ± 2.5 °C (4.5 °F). It can detect a human being from up to 7 m. The Raspberry Pi Camera Module V2 is the newly updated Raspberry Pi Foundation camera board. It features an 8MP (megapixel) SONY IMX219 image sensor of ultra-high resolution up to 5MP on the V1 camera board and a fixed focus lens. Capable of 3280 × 2464-pixel static images, this V2 camera module also supports 1080p @30fps, 720p @ 60fps and 640 × 480 p90 footage. Interface to the Raspberry Pi camera module via a 15-pin Ribbon Cable (FFC-Flexible Flat Cable) to a dedicated 15-pin MIPI Camera Serial Interface (CSI) specifically designed for cameras. The CSI bus can exceptionally high data speeds, bringing pixel data exclusively to the Raspberry Pi processor. With every camera, a 150 mm CSI cable is included. Raspbian has been developed for the Raspberry Pi hardware as a free Debian operating system. Raspberry Pi is an operating system to collect entire programs and utilities. However, Raspbian provides more than just a single OS: over 35,000 pre-compiled software packages are available in a friendly format for rapid Raspberry Pi installation. More than 35,000 Raspbian packages were initially installed in June 2012 for the Raspberry Pi's highest results. Raspbian remains actively developed with as many Debian packages as possible based on improving stability and performance. This SD card is used to store data and the project's coding. It only uses less than half of the total storage so that the extras can store data of the person coming through for the project. It can store many pictures as Raspbian OS does not use a lot of storage, which is an advantage as it can be upgraded to higher storage in the future. Concerning thermal imaging cameras [31] , the relevant technical and performance elements for reliable measurement are provided. Firstly, it offers a scanning range between 34 and 39 °C with increments of no more than 0.1 °C with an adjustable threshold temperature. Secondly, the face temperature between 30 and 40 °C with a minimum interval range. Thirdly, more significant than 0.1 °C temperature resolution. Fourth, it provides an alarm system when a temperature over 37.5 °C and last but not least, the software is also used to detect the febrile of subjects during the image processing. In the case of people-group testing systems for screening, the software must automatically identify various subjects or targets. Therefore, temperature estimation accuracy depends on device components such as the camera, the software for data post-processing, detection methods, and environmental conditions. In a controlled environment for individual control, the best accuracy is obtained, while accuracy worsens to 2 °C for unordered groups of individuals in uncontrolled temperature environments. Finally, in order to ensure proper use of these systems and accurate interpretation of the results, it is essential to carefully follow the indications given in the technical annotation [32] . The drift of IR thermal cameras depends strongly on instrument efficiency and operating conditions and usually amounts to around 0.1 °C annually. This source of uncertainty can be minimized by periodic calibrations that are regular. An infrared blackbody is typically used for calibration due to the surface heated at various known temperatures with an emissivity range between 0.95 and 0.98. The resolution of the measurement devices' temperature depends on the number of bits of the analogue to digital converter and the LCD monitor option. The temperature resolution is usually equal to 0.1 °C in infrared thermometers. The pretesting results are divided into two parts: facial and thermal detection. The primary point in this is the accuracy of this project with respect to the detection distance. The difference in range carried out the first variable for facial detection and how far can it be detected. Figure 6 shows the distance measuring face to the device detected; besides, the results in Fig. 6b indicated also that the accuracy of the detected temperature was varied according to the measured distance. The results also revealed that the accuracy tends to decrease as the object reaches a far distance (with a regression ratio R 2 of 0.9497). However, no record was obtained at a 2500 cm distance. Figure 7a -c shows the deployment in-practice for object identification and thermal scanning over different ages, kid, middle-and older-age compared with contact wearable sensors. It was found that the proposed system and algorithm as practical help to fight against COVID-19 with real-time implementation. The recorded accuracy was about 99.9% within the optimized best distance of (500 ± 5) cm. In essence, if the skin's emissivity is below the reference value of 0.98, the ambient atmosphere's radiation will evaluate dramatically. Hence, the reflected radiation can no longer be neglected than the emission. An example is an error of around 0.2 °C when the emissivity is equal to 0.98, and the average radiant temperature is 10 °C higher or lower than the outdoor temperature. Simultaneously, simple calibration cannot be corrected, depending on room temperature or even if the room temperature remains stable [33, 34] . The application of a reference objective can reduce and offset influence accordingly. As a second trial of the prototype, the test was carried out to find the thermal sensor's accuracy under different room temperatures (Fig. 8a, b) . Firstly, it will be tested at a normal room temperature of 27 °C. The detected readings' results were around 36.75 °C-35.5 °C, as shown in Fig. 8a . The following variable for the temperature is taken in an air-conditioned room with a 23 °C temperature (Fig. 8b) . The room is left to get cold for 15 min for a stabilized temperature measurement. The results revealed that the detected temperature was around 35.75 °C ± 0.2. Based on the comparison results, it can be confirmed that the monitoring system has a firm accuracy under different circumstances. Based on the theory principles [35] , the body's temperature regulation can be subject to two separate screening types: (1) is deterministic and usually set at 37.5 °C to prevent a large number of false positives; (2) the second is based on a statistical threshold value calculated on the sample measured temperature at actual measurement conditions and the pre-approved threshold value. For the selection of suspicious cases to comply with the decision laws, the unintended and structural ambiguity inputs need to be considered in the first instance [36] . In this second case, only specific causes of uncertainty, only type A tensions do not determine a substantial difference between measuring the particular topic and the population's median value during the survey. It can also be considered if the measurement is always performed with the same instruments, process, environment and operator. This uncertainty is not to be taken into account in the checking process as the environment decides the measurement uncertainties of 0.3 °C but is stable and causes systematic measuring changes [37] . The pre-checking process unveiled that negative subjects are in absolute terms consistent with the absolute threshold value but can be considerably simplified, which do not surpass those values. Therefore, only subjects with greater precision than the absolute limit value can be involved in a second screening projection stage. In all events and regarding the corresponding core temperature value, an efficient screening cannot disregard the clearly defined site on which measurement has to be carried out. The measuring protocols provided by the competent authorities provide that employees should be measured at a temperature, and this does not necessarily indicate that a measuring operator is present. The worker's self-assessment may, in theory, also be performed under the guidance of or subordinate supervisor to ensure consistency of the calculation. The legislator makes the most appropriate measuring instrument option: both innovative, remote and conventional thermometers [38] . In some instances, however, this would not guarantee a sufficient measurement duration, even because the thermometer has to be disinfected each time. Infrared thermometry allows a more considerable distance between the operator and the worker to be preserved compared with conventional contacted thermometers. In particular, infrared thermometers must be measured within 5-15 cm away because the limited SSR and thermometers can measure at distances of a few metres from the subject. These considerations often direct the employer in selecting a remote measurement method such as thermal images or thermoscanners. Figure 9 shows the trial test during the temperature rise simulation using an external heating source. The results were indicated based on the stated algorithm temperature and facial recognition. Once the external source temperature rises to be higher than 38 °C, an immediate warning notification was received via mobile phone and email. The warning email includes an image of the scanned pre-strain temperature person and his/her details. This paper has proposed an IoT-based framework to reduce infectious diseases. The proposed framework was used to employ potential COVID-19 case information and health records of confirmed COVID-19 cases to develop an artificial intelligence model to monitor, detect, and communicate with authorizing if a pre-strain person was found. The framework also communicates these results to building management, who can then respond swiftly to suspected cases identified by the predictive high-temperature model by following up with any further medical investigation needed to confirm the case. This allows the confirmed cases to be isolated and given appropriate health care. Employing the proposed real-time framework could reduce infectious diseases and mortality rates through early detection cases. The performance could be quickly improved by updating the model with continuous data to be considered. To further enhance the detection accuracy, we need to focus on adding training samples with complicated facial shape and angle to view the wearing mask or not mechanism, which may affect facial recognition accuracy. Emergence of novel coronavirus and COVID-19: whether to stay or die out? World Health Organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19) AI techniques for COVID-19 AI-driven quantification, staging and outcome prediction of COVID-19 pneumonia Conceptual design: a novel covid-19 smart ai helmet Opportunities and challenges for the building monitoring systems in the age-pandemic of COVID-19: Review and prospects COVID-19 pandemic: effects and evidence-based recommendations for otolaryngology and head and neck surgery practice Wearable sensors for COVID-19: a call to action to harness our digital infrastructure for remote patient monitoring and virtual assessments Position paper on COVID-19 imaging and AI: from the clinical needs and technological challenges to initial AI solutions at the lab and national level towards a new era for AI in healthcare Review of big data analytics, artificial intelligence and natureinspired computing models towards accurate detection of COVID-19 pandemic cases and contact tracing COVID-X: novel health-fog framework based on neutrosophic classifier for confrontation covid-19 COVID-19 and applications of smartphone technology in the current pandemic Applications of digital technology in COVID-19 pandemic planning and response Public transport planning adaption under the COVID-19 pandemic crisis: literature review of research needs and directions Singapore COVID-19 pandemic response as a successful model framework for low-resource health care settings in Africa IoT (internet of things) based infant body temperature monitoring Health companion device using IoT and wearable computing Potential association between COVID-19 mortality and health-care resource availability Smart health monitoring and management system: toward autonomous wearable sensing for internet of things using big data analytics Energy harvesting for the implantable biomedical devices: issues and challenges An IoT-based framework for early identification and monitoring of COVID-19 cases Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning A novel facial thermal feature extraction method for non-contact healthcare system Systems and methods for monitoring driver state ed: Google Patents Explainable AI and mass surveillance system-based healthcare framework to combat COVID-I9 like pandemics Design and Modeling of IoT IR Thermal Temperature Screening and UV Disinfection Sterilization System for Commercial Application using Blockchain Technology Low cost thermal sensor array for wide area monitoring COVID-robot: Monitoring social distancing constraints in crowded scenarios Adversarial examples-security threats to COVID-19 deep learning systems in medical IoT devices Mobile application for monitoring body temperature from facial images using convolutional neural network and support vector machine Evaluation of the size-of-source effect in thermal imaging cameras Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography Stable mid-infrared polarization imaging based on quasi-2D tellurium at room temperature A calibration procedure for field and UAV-based uncooled thermal infrared instruments Infrared thermography and thermal nondestructive testing Reverse logistics network design for medical waste management in the epidemic outbreak of the novel coronavirus (COVID-19) Uncertainty analysis of thermocouple measurements used in normal and abnormal thermal environment experiments at Sandia's Radiant Heat Facility and Lurance Canyon Burn Site Noncontact body temperature measurement: uncertainty evaluation and screening decision rule to prevent the spread of COVID-19 Acknowledgements The authors gratefully acknowledge the supported facilities provided by Management and Science University (MSU). Fig. 9 a Lighter simulation on prototype and feedback on b SMS and c email The authors have no conflict of interest.