key: cord-0781266-b2p2q1yv authors: Sharma, Abhinav; Oulousian, Emily; Ni, Jiayi; Lopes, Renato; Cheng, Matthew Pellan; Label, Julie; Henriques, Filipe; Lighter, Claudia; Giannetti, Nadia; Avram, Robert title: Voice-Based Screening For SARS-CoV-2 Exposure In Cardiovascular Clinics date: 2021-06-16 journal: Eur Heart J Digit Health DOI: 10.1093/ehjdh/ztab055 sha: 4732b1c7b8870dcfbd436ac0919747aae3022d6b doc_id: 781266 cord_uid: b2p2q1yv AIMS: Artificial intelligence (A.I) driven voice-based assistants may facilitate data capture in clinical care and trials; however, the feasibility and accuracy of using such devices in a healthcare environment are unknown. We explored the feasibility of using the Amazon Alexa (‘Alexa’) A.I. voice-assistant to screen for risk-factors or symptoms relating to SARS-CoV-2 exposure in quaternary care cardiovascular clinics. METHODS: We enrolled participants to be screened for signs and symptoms of SARS-CoV-2 exposure by a healthcare provider and then subsequently by the Alexa. Our primary outcome was interrater reliability of Alexa to healthcare provider screening using Cohen’s Kappa statistic. Participants rated the Alexa in a post-study survey (scale of 1 to 5 with 5 reflecting strongly agree). This study was approved by the McGill University Health Centre ethics board. RESULTS: We prospectively enrolled 215 participants. The mean age was 46 years (17.7 years standard deviation [SD]), 55% were female, and 31% were French speakers (others were English). In total, 645 screening questions were delivered by Alexa. The Alexa mis-identified one response. The simple and weighted Cohen’s kappa statistic between Alexa and healthcare provider screening was 0.989 (95% CI: 0.982, 0.997) and 0.992 (955 CI 0.985, 0.999) respectively. The participants gave an overall mean rating of 4.4 (out of 5, 0.9 SD). CONCLUSION: Our study demonstrates the feasibility of an A.I. driven multilingual voice-based assistant to collect data in the context of SARS-CoV-2 exposure screening. Future studies integrating such devices in cardiovascular healthcare delivery and clinical trials are warranted. REGISTRATION: https://clinicaltrials.gov/ct2/show/NCT04508972 Artificial intelligence (A.I.) driven voice-based technologies have significant potential to streamline and optimize delivery of healthcare and conduct of clinical trials. 1-3 Numerous commercial voice-based are now on the market and are poised to potentially play a larger role in healthcare environments. 2 Consumer data suggested that the use of voice assistant grew by 103% from 21.5 million in 2018 to 43.7 million in 2020. 4 However, the current major use case of these technologies is for search engine queries. 5 There is a lack of data surround the evaluation of such technologies to within a healthcare environment and furthermore, the reliability, accuracy, and concordance with manual or healthcare provider data collection has not been extensively explored. The SARS-CoV2 pandemic has provided an opportunity to test the feasibility of using A.I. driven voice-based devices to collect patient level information. Furthermore, such devices may facilitate optimal healthcare environment. For instance, within a healthcare setting, screening for risk factors or symptoms relating to SARS-CoV-2 exposure is often done directly by healthcare personnel, using standardized institution-specific questionnaires. Such methods are inconsistent and inefficient in screening multiple people. Amazon Alexa® ('Alexa') is a voice-based A.I. enabled personal assistant that can activate cloud based 'Skills' through verbal triggers. 6 The ability to create skill that comply with privacy regulation (for example the Health Insurance Portability and Accountability Act [HIPAA] in the U.S) may enable devices to be integrated into clinical environments. There is limited evidence in the evaluation of voicebased technologies in healthcare settings or in the context of clinical trials. 1,2,6-8 By testing the utility and accuracy of using the Alexa in screening for symptoms or risk factors relating to SARS-CoV2, we will obtain greater evidence into the ability to integrate such devices into clinical care and future clinical trials. We prospectively evaluated the ability of Alexa to screen for risk-factors or symptoms relating to SARS-CoV-2 exposure (VOICE-COVID-19 Study; NCT04508972). 9 The device utilized was the Amazon Echo Show 8. With Amazon Web Services, an Alexa Skill was developed using the Amazon Alexa Developer Kit. 7 Questions relating to risk factors for SARS-CoV-2 exposure were based on Health Canada recommendations (Table 1) . 10 The content of the skill was initially developed in English and then translated into French. As Alexa can only have one default language we set this in French but entered skill questions in both English and French. This resulted in a skill where the participant could select either French or English. The resulting English voice was spoken with a slight French accent. An Alexa skill uses a vocal interaction model and application logic to determine the participant's request ( Figure 1 ). The voice algorithm is based on Natural Language Understanding (NLU) and Automatic Speech Recognition (ASR). 11 The ASR involves the recognition and translation of speech into text. However, as vocal intonations and inflections vary widely, the device needs NLU to first rearrange spoken data into a machine-readable format. The skill built for this study is a predetermined sequence of screening questions driven by a vocal request: answering yes/no, numbers for age, etc. The questionnaire is activated through an utterance (a specific phrase): "Alexa, ___" The patient's words are streamed to the Alexa service in the cloud, ASR and NLU will respond through voice recognition and structure the information into a request, and the screening process will begin. Each response is stored as a slot value and will act as a request for the next question. The participants were recruited from October 1 st -December 31 st 2020. Potential participants were recruited by verbally discussing with the individual upon entry into the cardiovascular clinic. No compensation was provided to the participant. Among participants who verbally consented, the screening questions were initially asked by the healthcare professional. Subsequently, the participant would then move towards the Amazon Echo device and initiate the screening process with the device. The first question asked by the Alexa was whether the participant wanted to proceed in French or English. Subsequent questions were asked about the purpose of visit to the clinic ( Table 1) . Then the Alexa then proceeded to initiate the screening questions using the language the participant selected. The screening questions asked by the healthcare provider and the Alexa were the same: Any international travel in past 14 days out of Canada; in close contact with a person with Covid-19 without personal protective equipment; or presence of any symptoms (in the prior 14 days) including cough, fever, runny nose, sore throat, diarrhea ( screened by healthcare personnel. The entry location had a small enclave that enable a more private discussing with the healthcare personnel but still had ambient noise from the surrounding areas. If participants responded 'yes' to the screening questions either with the healthcare provider or the Alexa, following completion of the post-survey questions, the healthcare provider would then move the participant to an isolation area and conduct a more detailed history. After completion of the screening process, the Alexa Skill was rated by participants on a five-point scale. The scale was based on a previously validated scale of user preference and app engagement. 12 Participants were subsequently asked about any privacy concerns related to data storage or use of the Alexa application by asking the following question with a binary yes/no response: "Do you have any privacy concerns regarding the use of the Amazon Alexa to screen for symptoms". The high-level data flow and study architecture for the VOICE-COVID-19 study is described in Figure 1 . When the researchers are defining the specific questions and triggered, it gets uploaded onto the AWS cloud platform through the AWS S3 bucket. This will integrate into the AWS Lambda server which will transfer information into the Aws DynamoDB. With DynamoDB researchers can create database tables that can store and retrieve any amount of data and serve any level of request data flow. This study was approved by the ethics board at the McGill University Health Centre. Verbal consent (written consent waiver was granted) was obtained from all adults participants 18 years and older (or from their legal representatives) prior to screening by the healthcare provider. Categorical variables were presented as counts (percentages) and continuous variables were presented as median, 25 th and 75 th percentiles. The primary endpoint of the current study was the interrater reliability of Alexa versus manual screening using simple and weighted Cohen's Kappa statistic. To facilitated pragmatic data collection, data collection on the specific comorbidities of patients were not collected. Data were analyzed using SAS version 9.4 software (SAS, Cary, North Carolina). Statistical significance was based on a p value of ≤0.05. Coding for the Amazon Alexa skill will be made available by requests to the corresponding author. In total, 215 participants were screened (mean age of 46.1 ± 17.7 years; 118 [55%] females; Table 2 ). The range of age was 16 to 84 years. There were no participants that were offered the opportunity to participate who refused. There were 66 (31%) French speakers and the remainder were English. There were 114 (53%) patients or family members and the rest were hospital workers. In total, 645 screening questions were delivered by Alexa. There were 13 responses missing due to participant withdrawal (n=2) and loss of wireless internet connection (due to our hospital network) after starting. In (Figure 2 ). In the oldest tertile of age (range 57-84 years), the concordance was perfect with Cohen's kappa correlation of 1.0. There was perfect concordance in French with a Cohen's Kappa 1.00 (95% 1.00-1.00); for English the concordance was 0.96 (95% CI 0.886-1.000). The median time for completion of the Alexa screening questions was 40 seconds (interquartile range 33 to 44 seconds). The mean (with standard deviation) participant rating (out of five) was as follows: 4.7 (0.7) for appropriately asking the questions; 4.7 (0.7) for ease of language comprehension; 4.5 (1.0) for ease of device use; and 4.4 (0.9) for overall rating ( Table 3) . No participant expressed any privacy concern. There has been limited evaluation of the use of voice-based devices in healthcare settings. In the present study, we demonstrated that the Amazon Alexa demonstrated near perfect concordance with screening questions compared to healthcare personnel. Furthermore, it appeared that there was no decrement in the ability to be screened by Alexa among older participants. The ability to use commercial voice-based devices to facilitate SARS-CoV-2 screening may represent a strategy to reduce burden on healthcare personnel and minimize their exposure to potentially infected individuals. 2 Furthermore, these results suggest that future studies evaluating the utility of integrating the such devices into routine clinical care and clinical trials in non-SARS-CoV-2 related settings are warranted. There are several aspects to the functionality of voice-based technologies. The hands-free SARS-CoV-2 screening is built on the Alexa Skills Kit's interactive voice-driven interface. 13 The ability to activate the skill without requiring additional step, and just through voice, makes the use of such technologies ideal for populations where data capture via electronic devices is challenging, such as amongst older participants. As healthcare delivery and future clinical trials become more pragmatic, there will be increasing need to leverage digital technologies to facilitate rapid data acquisition. Our result demonstrate the potential for such devices to collect medical grade data with high degree of accuracy and reliability. Such technologies may have the potential to reduce the burden of healthcare delivery and trial conduct by enabling accurate remote data collection. In addition, leveraging voice-based A.I. to supplement such routine screening can possibly enable more effective workforce utilization. Our results provides important feasibility information to identify that data can be accurately collected through the Alexa. Extending the use of such voice-based technologies in clinical and non-clinical environments (such as schools, workplaces, or home settings) to evaluate clinical workflow and burden warrants further evaluation. Our study also highlights the practical elements faced with such devices (Table 4) . Voicebased technologies generally need continuous internet access. As we experienced, loss of wireless internet access may impair device functionality. Additional testing into functioning in 'offline' modes will be needed to ensure continuous function in clinical settings. Furthermore, while privacy concerns are significant issues, especially around devices and technologies related to identifying those at risk of SARS-CoV2 exposure, 14 in our study, no participant expressed any privacy concern. Such privacy concerns will need to be explored with larger deployment of such voicebased technologies. Expanding on the results of our study, there is great need in cardiovascular clinical studies and care delivery for accurate remote patient monitoring. 15, 16 Currently, most remote monitoring strategies focus on either telephone or video-based follow-up along with additional wearable devices or patches. 17, 18 Voice-based assistants may provide one potential tool to facilitate remote patient follow-up, especially in settings where directed synchronous interaction with a patient (through a phone call or via video-platform) is limited. While our study does not directly demonstrate the use of Alexa in these use cases, the high degree of concordance of data collected by Alexa to standardized questions, compared to a healthcare provider, suggests that further exploration of such technologies are warranted within cardiovascular studies and care delivery. There are several limitations. Our results cannot be extrapolated to other voice-based devices. In the one case where the Alexa mistook the response, an English response was recognized as French and miscoded Future studies will have to evaluate the accuracy of data capture in other languages. Challenges and solutions in using voice-based device are shown in Table 4 . Our study was a non- Currently have any of the following symptoms: cough, fever, runny nose, sore throat, diarrhea 6 (2.8%) SD standard deviation Note, score of 5 reflects 'strongly agree' while the score of '1' represents 'strongly disagree'. SD standard deviation Using Digital Health Technology to Better Generate Evidence and Deliver Evidence-Based Care Readiness for voice assistants to support healthcare delivery during a health crisis and pandemic Voice Assistant Use Through Smart Hearables Consumer Report -Voicebot Prepare for the voice revolution Alexa Skills Kit Official Site Applications of digital technology in COVID-19 pandemic planning and response The VOICE-COVID-19 -ClinicalTrials.gov What Is Automatic Speech Recognition? -Alexa Skills Kit Official Site Design and testing of a mobile health application rating tool About Voice Interaction Models | Alexa Skills Kit The need for privacy with public digital contact tracing during the COVID-19 pandemic Wearable Devices for Ambulatory Cardiac Monitoring: JACC State-of-the-Art Review Remote Patient Monitoring in Heart Failure: Factors for Clinical Efficacy Smart wearable devices in cardiovascular care: where we are and how to move forward Patient Use and Clinical Practice Patterns of Remote Cardiology Clinic Visits in the Era of COVID-19 Funding for VOICE-COVID was supplied by the MUHC-Foundation and in-kind development support was provided by Amazon Web Services (AWS). Amazon Web Services (AWS) was the vendor that assisted in making the Alexa Skill. AWS did not influence study design or have access to the primary data nor decision to publish the manuscript. Authors have no disclosures pertaining to this manuscript. A.S. reports receiving support from the Fonds de Recherche Santé Quebec (FRSQ) Junior 1 clinician scholars program, Alberta Innovates Health Solution, European Society of Cardiology young investigator grant, Roche Diagnostics, Boeringer-Ingelheim, Novartis, and Takeda. There are no other relevant disclosures. Abhinav Sharma MD, PhD: study design, analysis, manuscript writing Emily Oulousian BSc: study conduct, critical revision for manuscript Jiayi Ni MSc: study design, analysis Renato Lopes MD, PhD: critical revision for manuscript Matthew Pellan Cheng MD: manuscript writing, critical revision for manuscript Julie Label BSc: study design Filipe Henriques BSc: study design and analysis Claudia Lighter: study conduct, critical revision for manuscript Nadia Giannetti MD: study conduct, critical revision for manuscript Robert Avram MD, MSc: study conduct, critical revision for manuscript