key: cord-0819822-pov34ilo authors: Lim, Wei Jie; Abdul Ghani, N. M. title: COVID-19 Mandatory self-quarantine wearable device for authority monitoring with edge AI reporting & flagging system date: 2022-01-10 journal: Health Technol (Berl) DOI: 10.1007/s12553-021-00631-w sha: f2ce902e6f39124d503eb711fac6dadd584044cb doc_id: 819822 cord_uid: pov34ilo A mandatory self-quarantine is necessary for those who return from overseas or any red zone areas. It is important that the self-quarantine is conducted without the non-adherence issue occurring and causes the self-quarantine individual to be the carrier of the COVID-19 in the community. To navigate and resolve this issue, most countries have implemented a series of COVID-19 monitoring and tracing systems. However, there are some restrictions and limitation which can lead to intentional non-adherence. The quarantined individuals can still travel within the community by removing the wristband or simply providing an incorrect contact status in the tracing application. In this paper, a novel configuration for mandatory self-quarantine system is proposed. It will enable interaction between the wearable and contact tracing technologies to ensure that the authorities have total control of the system. The hardware of the proposed system in the wearable device is low in cost, lightweight and safe to use for the next user after the quarantine is completed. The software (software and database) that linked between the quarantine user and normal user utilizes edge artificial intelligence (AI) for reporting and flagging mechanisms. The recent Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) also known as the COVID 19 pandemic, has a major impact on to our society due to the threats to our health. In many ways, it has changed our lifestyle. The World Health Organization (WHO) had declared that the COVID-19 is a Public Health Emergency of International Concern (PHEIC) in January 2020 when the virus began to spread and transmit dramatically around the world [1] . Various governments are forced to mandate and carry out a few security measures to withstand and overcome the virus or disease outbreak. Lockdown entails restricted community traveling, the practice of working from home, and the recommended self-quarantine for those traveling back from overseas or red zones. In comparison with other types of coronavirus such as SARS and MERS, COVID-19 is considerably contagious and brings about a higher mortality rate. Without experiencing any signs or symptoms, someone may be a COVID-19 carrier. The carrier who tests positive may take between 2-14 days to show symptoms because it takes a long time for the incubation of virus. Thus, by that time, the carrier may have close contact with others and spread the virus around the community [2] . In line with the efforts of the Malaysian Government to contain the spread of COVID-19, the Malaysia Government has decided on a new regulation on 10 June 2020, which stated that "all Malaysians and non-citizens having permission to enter Malaysia will need to undergo a mandatory self-quarantine period of 14 days upon arrival after they have been cleared from COVID-19 by the Ministry of Health and will be monitored through the "MySejahtera" phone application". The person who has to undergo quarantine for the period of 14 days will be provided with a quarantine wristband to restrict and prohibit the person's movement during quarantine. However, this measure has led to an issue caused by the non-adherence of the quarantined individual who continues to travel around the community by removing the wristband [3] [4] [5] . Contact tracing in Malaysia has developed and integrated a contact monitoring application called "MySejahtera" [3] . The tracing approach employed by this application uses the QR Code check-in method; a user is compelled to perform a self-registration by scanning the QR code using the application before entering public places, including restaurants, shops, public transport, companies, etc. However, the application has its challenges and constraints, especially when it involves people who refuse to follow the standard operating procedure. This non-adherence includes not willing to check themselves in when entering any premises. Apart from that, an infected individual may travel outside of the quarantined premises and establish contact with members of the community they cannot recognize and identify [6] . The tracking of infectious diseases can be made easier with the presence of widespread technology and availability of interaction between mobile phones, data, and empowering computing technology. Defining the smartphone contact tracing consists of different framework layers and functionality, depending on the country adopting the technology. The region-based contact tracing applications are integrated using diverse technologies and techniques to collect and analyze the data. For instance, location tracking or contact tracing can be accomplished using GPS, contacting or distance tracing using Bluetooth, as well as by self-reporting [8] . Most countries have integrated a series of COVID-19 monitoring and tracing systems after the declaration and proclamation of the COVID-19 epidemic as a public health emergency international concern (PHEIC) in January 2020 by the World Health Organization (WHO). Monitoring and controlling the spreading of diseases is crucial to protect community health. Hence, a famous coined term 'digital epidemiology' was introduced [7] , comprising related approaches and technologies used for dealing with diseases. China has proposed the Health Code application [9] to detect community movement. The application is based on QR codes technology. It will verify the user's travel history, any possibility of any level of contact with a COVID-19 patient, and other relevant individual information such as the address, telephone, passport number, and further personal details. Based on the report, the red code indicates those who are under the government-imposed quarantine or selfquarantine status of 14 days. The amber color code refers to those ready to be quarantined for 7 days while a user is free from the risk of getting the disease if the color code is green. Singapore's Ministry of Health has also released and proposed a contact tracing application called 'TracedTogether' in March 2020 [10] . The application is based on the BLE (Bluetooth technology) which is related to the transferring and sharing of the pseudo-random ID, which keeps the broadcasting background. The technology used is considerably similar to the Google/Apple Exposure Notification [11] implemented by Google & Apple. This service has already been adopted in various parts of the United States and the United Kingdom since 25 May 2020. The approaches generate the Rolling Proximity Identifiers (RPIs) as a temporary exposure key to track the person with the device. Once the exposure notification is turned on, the user's device starts to create the (RPIs) with random pseudocode and scans the nearby devices. Once a user shares the id and gets confirmation about a positive infection, the users near the id or those who have been in contact with the id earlier will receive a push notification for the next diagnosis or self-quarantine. A similar application was proposed by Singapore [10] . The 'TracedTogether' application will generate a unique sharing token for every registered user. The Bluetooth will be responsible for exchanging the unique token from the application to other user applications and stored. One research proposed that an Artificial Intelligence mobile health platform for early detection for COVID-19. The system involves the interaction between the wearable and mobile phones to retrieve the health parameter from the wearable sensor and transfers it to the smartphones. The parameters include the temperature of the asymptomatic COVID-19 close contact subject, respiratory rate, blood In this paper, a mandatory self-quarantine system with the interaction between the wearable, contact tracing technologies, and AI face recognition using a server-to-edge device is developed to ensure the quarantine is well-executed under different back-end and front-end control structures. The proposed system in hardware (wearable) is low in cost and made with the biodegradable materials. At the same time, the software (software and database) centralizes the data between the quarantine user and normal user. A complete and comprehensive reporting and flagging system is proposed to force and provide strict quarantine measures to curb the pandemic. The proposed wearable is also able to detect a post-vaccination symptom to retrieve the health parameters such as the heart rate and blood oxygen saturation from the Table Table Primary In this study, the proposed new design framework will have three distinct systems whereby the monitoring application system and working principle, front-end system & back-end system development and the wearable hardware design. The entire system consists of 3 parties involved, including the quarantine user, normal citizens/users, and the medical authorities. The system is proposed with a centralized approach, giving a full data interaction between these 3 parties. The system involves Bluetooth and Live Signal (Wi-Fi/ Telco Data) to retrieve and transmit the data. The system and the working principle of the interaction between the normal user and quarantine user are shown in Fig. 1 . The initial step or the initial requirement of the application is the registration of the quarantine user. A normalized database entity-relationship diagram is proposed in Fig. 2 . The quarantine user will register the user id with the authorities. The requirements include the user's identity card number, email, gender, address, provided wearable id, and face data (photo will be uploaded to the database). The quarantine user's application is built mainly for health data, location data monitoring, and reporting and flagging system. For a non-risk user, the application only involves the always discoverable and reporting system. The registration table will be separated into two types, one is specific for quarantine users, and another database table is for non-risk users. The wearable is built with two BLE Modules, one mainly as always discoverable for user tracing. The tracing method is implemented using the Received Signal Strength Indicator (RSSI) as an approach to obtain and detect the quarantine user in an area with a setting range (e.g., > 2 m). Another BLE module mainly functions as the data transmitter to transmit the health parameter retrieved from the oximeter sensor at the wearable. The proposed two BLE Module is to ensure the tracing activity can perform without any restriction of data transmitting. BLE modules are implemented in this project due to their low cost and power consumption benefits compared to the latest Ultra-Wide Band Module proposed by Apple Inc [13] . The function of the wearable is to retrieve the health parameter to monitor the vital sign of the quarantine user (heart rate/bpm & blood oxygen saturation/SpO2). The quarantine user's application will periodically check the health parameter of the user, and the sensor will send the data to the application. Next, the application will post the data into the database. The health parameter will be written into the table of 'hid', which consists of the heart rate, blood oxygen saturation, and id. To ensure that the data posting has the correct quarantine user id, the database has been developed by setting the main 'users' table as a primary key to connect it to the rest of the five tables id. The relationship is built as the primary key from the 'users ' table and the foreign key as the table ' hinfo', 'lid', 'binfo', 'finfo' and 'uinfo' (refer Fig. 2.) . The system working principle of the non-risk user's application only involves their own smartphones as the main peripheral to detect the presence of non-adherence quarantine users. The process starts after completing all the registration verification, and the users have turned on the Bluetooth. The relationship of the Entity Relation Diagram which the primary key, foreign key and composite key for each SQL database table are shown in Table 1 . After the Bluetooth is turned on, the application will always be triggered and search for the wearable devices for the registered quarantine users. Once it is detected that the quarantine user has revoked the regulation and travels within the community, the application will start promoting the warning interface and a report button. If the report button is pressed, the scanned wearable id will send it to the local database called 'binfo' and notify the authorities that someone who is a quarantine user is moving around within the community (refer to Fig. 3. ). The proposed system is able to double the inspection and restriction to control the movement of the mandatory quarantine user. This is accomplished with the cooperation of the community to track or trace the presence of the quarantine user and the location tracker from the quarantine user's application. HEISM (Healthcare, Isolation & Monitoring) and CTRACE (Isolation Wearable Tracing App) is developed for quarantine users and non-risk users, respectively. These applications are built using the Android Studio as the main development platform. The design and interaction of data retrieval of the applications have been implemented with comprehensive techniques, including creating an updated recycler view section, notification prompting to specific conditions, and data posting to the localhost server (phpMyAdmin) to write the PHP scripting for retrieving and posting purposes. Table 2 shows the overall system or application for the users (including front-end and back-end systems) with its development platform. The HEISM Monitoring Dashboard is developed to integrate a server-to-edge device face recognition by the OpenVINO face recognition engine. Compared to the market or other university-proposed quarantine monitoring apps, the Android Smartphone Biometrics API saves the encrypted fingerprint or face data locally in the smartphone. The disadvantages will lead to the quarantine users replacing it with another by changing the fingerprint or face data in the smartphone. With the edge device OpenVINO face recognition, the quarantine users are forced to upload their selfie photos periodically. The OpenVINO will run the recognition and give the output of the photo similarity compared to the photo taken on the first day of quarantine registration. The HEISM Monitoring Dashboard is developed complete with the reporting and flagging system with 5 non-adherence possibilities that could be performed by the quarantine users. All the action for the sorting of the result is performed by the PHP and MySQL scripting to interact the data from the quarantine users, non-risk users, and the action of the dashboard to the database. Figure 4 shows the integrated or the architecture of the OpenVINO in the HTML dashboard. The wearable is designed and implemented with the main microcontroller (Arduino Lilypad) with the ATmega328P, 2 JDY-08 BLE Bluetooth Module and a GY-MAX30100 pulse oximeter sensor. Figure 5 shows the wearable components. The 3D design of the wearable exterior to fit the entire components is designed via Solidworks, consisting of two layouts, a design for the upper part and bottom part. Finally, the real prototype of the wearable is printed out by fitting all the components, and it is able to function. Figure 6 shows the prototype from different views. Table 3 shows the overview of the HEISM (Healthcare· Iso-lation· Monitoring) and the CTRACE (Isolation Wearable Tracing App) application. The Fig. 7 shows the overview of the HEISM Monitoring Dashboard developed by the HTML scripting. The HEISM complete with quarantine user's health parameters and location checking periodically. The Table 4 shows the steps for performing the periodically reporting. The face recognition is implemented with the HEISM Monitoring Dashboard to ensure the application use is under the same user who initially registered. Table 5 illustrates the steps of the Integrated OpenVINO Face Recognition Engine. A complete reporting and flagging system have been developed, and it is integrated with the possibility will perform by the quarantine users. Therefore, any non-adherence can be avoided by proposed this system for quarantine or isolation use. Figure 8 shows the reporting & flagging system. The reporting and flagging system will trigger once the users refuse to isolate and go outside of the designated quarantine area. The summary of the reporting and flagging system is listed in Table 6 and the summary of the Edge AI Face Recognition reporting system is listed in Table 7 . Testing is an essential parameter to measure the performance and load of an application. The overall average CPU load and performance testing have been conducted and inspected for the HEISM application by using the Android Studio Profilers. The Fig. 9 shows the average CPU load testing for the Main Page Activity and Health Check Activity for users. The CPU usage for the Main Page Activity take less than 25% while the memory usage less than 128 Megabytes. When the testing enters to the Health Check Activity, the CPU usage started increase due to the photo, Bluetooth data and location data uploading process during this Activity. The peak CPU usage for the Health Check Activity around 31% in the first step photo uploading process which require Steps D escription 1 . Quarantine users will be received a notification every 20 minutes. Once the notification is received, the users need to click it, and the HEISM app will open, and they are required to take a selfie photo. 2. The wearable is paired with the Bluetooth once the application appears the 'Checking BPM & SPO2' means the wearable has been connected successfully. The application will collect the health parameters and current location then post to the server. 3. The users can view all the historical checking data at the 'Health' tab and the 'Location' tab. All these data will be submitted to the server for further analyzation. Repeat steps 1 to 3 until the countdown end and release from quarantine. more CPU load for taking photo and transfer the photo data from the application to server. The memory usage for the Health Check Activity take average below 150 Megabytes until the end of the activity. The CPU and memory usage is much stable for the entire process of the HEISM application from app start until the app destroy. Only have some CPU usage spiked on the image uploading activity due to the heavier process compared to the other activities. Network load testing has been conducted for the Login Process and the Image Uploading Process. The network load testing for Login Process recorded very light usage for the network receiving. The testing was conducted under local server and 10 user logins simultaneously. The network load testing for the Image Uploading Process showed heavier load as compared to Login Process at around 100 KB/s for sent and 3 KB/s for received. The Fig. 10 shows the network usage for the Login Process and the Fig. 11 shows the Image Uploading Process. The exact Received Signal Strength Indicator (RSSI) value for the entire system is approximately 2 m which provided by the Android beacon library reference value [14] as tabulated in Table 8 . To study the error of the Received Signal Strength Indicator (RSSI) affected by different BLE module from difference smartphone, an experiment has been Steps D escription 1. The HEISM Monitoring will show the OPENVINO at the column of face recognition. The page will show the image of the quarantine user during the registration. Run the detector, and the engine will start analyzing the registration image. 2. Once the detector run, the python OpenCV window will be prompted, and the user is required to key in the name of the quarantine user during the registration stage. 3. The quarantine users will upload the image every 20 minutes, as mentioned in the procedure part. The uploaded images will be analyzing, and the output result will show beside the uploaded image. In addition, the name and similarity percentage will show in the output image. conducted for five different brands of smartphone which are Huawei P40, Honor 20, Xiaomi 9, Redmi k20 and Samsung Galaxy S3 as shown in Fig. 12 . The corresponding distances for different types of smartphones are tabulated in Table 9 . Firstly, the smartphone will be allocated at the distance of 2.5 m. The software is implemented with three versions to compare with RSSI value of -40dbm(3 m), -45dbm(2 m) and -50dbm (1 m) respectively. Figure 13 shows the graph actual distance versus RSSI estimated distance. The study has been concluded that different type of smartphones will not directly affect and fluctuate the measured value from Bluetooth device to the wearable. Market research has been done to compare all the available and proposed quarantine monitoring applications in the world. Malaysia currently has only the contact tracing application for COVID-19 but not specific for quarantine monitoring use. Figure 14 shows the matrix comparison for available quarantine or isolation monitoring applications. As can be seen in Fig. 14 , the wearable device used in Hong Kong and Poland does not provide periodically health status checking. Also, other countries do not have the function for individual identification, wearable tracing integration and live report for flagging system as compared to the proposed system. The works present a complete and comprehensive novel approaches for the entire quarantine monitoring system. This new system design is unique as it is integrated with health status periodically reporting, location tracking, artificial intelligent face recognition over the server to the edge device and non-risk user discovering wearable. The wearable is successfully implemented using Dual Bluetooth technologies for normal user to trace the movement of the quarantine user and transmit the quarantine user data from the wearable to smartphone and the centralized database. Furthermore, the convolutional neural network face recognition engine powered by the Intel OpenVINO is applied to the system to provide a capability and functional flagging system to solve the issues or cases of non-adherence. Moreover, the wearable is still beneficial after the COVID-19 pandemic has completely ended. It can be repurposed as a home healthcare monitoring device, especially for the elderly and mentally handicapped people. For future works, the proposed system can be further added with established dataset/model to check the vital sign or symptoms for the post-vaccinated persons to detect side effects such as blood clots. The statistics are beneficial to provide reliable and confident vaccine data to the community. The authors declare that they have no conflict of interest. A flood of coronavirus apps are tracking us. now it's time to keep track of them Coronavirus contact-tracing: World split between two types of app Available: https:// mysej ahtera. malay sia. gov. my/ intro/. Accessed 2021 4. Ministry of Tourism, Arts and Culture. Requirements To All Travelers Entering Malaysia From Overseas Guidelines for entry point screening of travellers (screeningfor Covid-19 Available: https:// world ofbuzz. com/ here-are-the-pros-and-cons-betwe en-check ingin-with-mysej ahtera-and-manual-form-filli ng Digital epidemiology: What is it, and where is it going? COVID-19 Digital Contact Tracing Applications and Techniques: A Review Post Initial Deployments, Transportation Engineering China launches coronavirus' close contact detector' app Exposure Notification reference Monitoring of vital signs with flexible and wearable medical devices