key: cord-0906009-h9kvzlzk authors: Lee, Jungju; Kong, So Yeon; Kim, Byung Woo; Kim, Seung-young; Park, Gwan-Jin; Chai, Hyun-Seok; Kim, Young Min; Lee, Suk Woo; Kim, Sang-Chul title: Assessment of novel ICT-EMS systems to improve emergency patient transportation during the COVID-19 pandemic date: 2022-03-22 journal: Am J Emerg Med DOI: 10.1016/j.ajem.2022.03.035 sha: de8864c1036421f48d438c173bed672f4198faae doc_id: 906009 cord_uid: h9kvzlzk OBJECTIVES: This study aimed to examine the usability, feasibility, acceptability, and appropriateness of the information and communication technology for emergency medical services (ICT-EMS) systems to improve the transportation of emergency patients during the COVID-19 pandemic. METHODS: Emergency medical technicians (EMTs) (n = 229) employed at 7 fire stations operated by the North Chungcheong Fire Service Headquarters, South Korea were trained to use ICT-EMS devices prior to a 1-month implementation period. System Usability Scale (SUS), Feasibility of Intervention Measure (FIM), Acceptability of Intervention Measure (AIM), and Intervention Appropriateness Measure (IAM) questionnaires were conducted in the 4th week of the 1-month implementation period to assess the perceived usability, feasibility, acceptability, and appropriateness of the ICT-EMS systems. RESULTS: Among a total of 229 EMTs, 187 EMTs (81.7%) completed the survey. The overall SUS score was significantly low (score of 35.6) indicating an overall negative perception of the ICT-EMS systems. With regard to the feasibility, acceptability, and intervention appropriateness of ICT-EMS, roughly 50 (26.7%) participants agreed that ICT-EMS implementation was possible, appealing, and suitable. CONCLUSION: Many potential areas of improvement were identified within the ICT-EMS systems. System alterations regarding usability, feasibility, acceptability, and appropriateness may be necessary to successfully implement the ICT-EMS systems. The emergency medical services (EMS) system is an expansion of emergency medical care into the prehospital phase Many countries, including Korea, trace basic concepts in EMS systems development back to the United States EMS Systems Act of 1973, which addressed fifteen key elements: 1) access to care, 2) patient transfers, 3) communications, 4) mutual aid, 5) training, 6) facilities, 7) critical care units, 8) disaster plans, 9) manpower, 10) transportation, 11) public safety agencies, 12) consumer participation, 13) coordinated patient record keeping, 14) public information and education, and 15) reviews and evaluations (1) . EMS systems serve the communities by mobilizing ambulances in response to emergency situations, transporting patients to appropriate medical facilities, and providing emergency medical care (2) . The COVID-19 pandemic, a worldwide historical event, has affected the healthcare system as well as the society at large, including social, economic, political, and cultural aspects. The COVID-19 pandemic has had a negative impact on EMS systems worldwide. During the early pandemic phase, transportation of critically ill patients through EMS to emergency departments (EDs) decreased, while prehospital deaths increased because of termination of resuscitation before transportation to the ED and death at home without resuscitation (3) (4) (5) . During the pandemic, the prehospital time required to transport suspected COVID-19 patients as well as those who were critically ill to appropriate hospitals has increased due to the lack of isolation rooms (6). Accepting suspected COVID-19 patients can be a burden for local hospitals with limited resources because of the risk of nosocomial infection of medical personnel or non-infected patients. In order to overcome these issues during the COVID-19 pandemic, solutions for improving the EMS systems have been proposed, such as regionalization of EMS systems, improvement of emergency room infrastructure, sharing of emergency medical resources using information and communication technology, and use of rapid COVID-19 diagnostic devices (2, (7) (8) (9) . A coordinated and integrated response across the community is necessary to maintain essential EMS services on the frontlines of the COVID-19 pandemic to minimize the spread of the SARS-CoV-2 and support care for critically ill patients. Information and Communications Technology (ICT) systems have been applied to the healthcare industry, and the demand for related ICT services such as mobile health care is increasing (10) . Cheongju, the capital of North Chungcheong Province in South Korea, is preparing to establish an ICT system for EMS (ICT-EMS) to coordinate EMS activities with the regional fire agency and emergency medical centers to determine hospital destinations during the prehospital phase. Before implementation of a technology-based intervention and for its suitability, it is important to assess the perceived utility and ease of use, feasibility and acceptability of the new ICT-EMS systems by the end users. The purpose of this study was to examine usability, feasibility, acceptability, and appropriateness of the ICT-EMS systems by emergency medical technicians (EMTs) acting in Cheongju. We conducted a questionnaire survey to determine the usability, feasibility and acceptability of the use of ICT-EMS systems to improve emergency patient transport during the COVID-19 pandemic prior establishment of ICT-EMS systems in the central area of North Chungcheong Province. This prospective study was conducted between February and August 2021 at North Chungcheong Fire Service Headquarters in South Korea. North Chungcheong Fire Service Headquarters operates 81 ambulances with 461 EMTs in 12 fire stations that cover 1.6 million residents in North Chungcheong Province, South Korea. North Chungcheong Province has 3 cities and 8 counties and consists of north, south, and central areas. The central area has 0.8 million residents serviced by 229 EMTs of 7 fire stations and has 1 regional emergency medical center, 3 local emergency medical centers, and 2 local emergency medical agencies. The regional emergency medical center is in charge of treating critical patients and coordinating between community stakeholders during crises such as pandemics and disasters. Study participants were EMTs who consented to the study and completed the study survey. EMTs (n= 229) employed at 7 fire stations operated by the North Chungcheong Fire Service Headquarter were recruited. We used a stepped wedge study design for this community-based pilot study of ICT-EMS establishment. We developed ICT-EMS systems to improve the EMS transport process for emergency patients (cardiac arrest, severe trauma, cardio-cerebrovascular disease) and patients with suspected COVID-19. We (IAM) were used for evaluating whether the ICT-EMS systems were implementable (11, 12) . The SUS contains 10 questions to be answered using the 5-point Likert scale (1; Strongly disagree; 2, Disagree; 3, Neither agree nor disagree; 4, Agree; 5, Strongly agree). Of these ten questions, five questions are positively formulated (statements with odd numbers) and the other five ones are negatively formulated (statements with even numbers). We explained the alternating formulation of these questions to all participants and asked them to answer each question with caution. The FIM questionnaire contains 4 statements assessing the extent to which a new system can be successfully used within a given setting, while AIM comprises 4 statements measuring the perception among implementation stakeholders of whether a delivered service is agreeable or satisfactory. IAM is used to measure the perceived fitness or compatibility of the new service with a given practice setting. Table 1 shows the statements and response categories for each SUS, FIM, AIM, and IAM item. Usability and acceptability of the ICT-EMS systems to facilitate community-based patient transportation during the COVID-19 pandemic and feasibility and participant's satisfaction regarding the use of the ICT-EMS systems were assessed to determine how well the ICT-EMS systems were received by participating EMTs. We used Excel The general characteristics of the study participants are presented in Table 2 . Among a total of 229 EMTs employed in the central region of North Chungcheong Province, 187 EMTs (81.7%) participated in the survey, 44% of which were male. The median age was 31 years (IQR, 28-36). A total of 179 participants (95.7%) were level 1 EMTs, and the median duration of EMS employment was 4 years. The mean SUS score was significantly low at 35.6 out of a possible 100. Participants' perception of the usability of the ICT-EMS systems was as follows: the system was very cumbersome (mean ± SD, 3.8 ± 1.0), unnecessarily complex (3.7 ± 1.1), had too many inconsistencies (3.7 ± 0.9), required technical support (3.6 ± 1.1), and required extensive learning before use (3.4 ± 0.9), which are all negative assessments of the system's usability. In contrast, mean scores for responses to positive statements were approximately neutral (i.e., ranged from 2.4 to 2.7) ( Table 3 ). In the FIM survey used to assess the feasibility of the ICT-EMS systems, the responses "agree" or "strongly agree" were selected by 53 participants (28.3%) in response to the statement "ICT-EMS seems implementable," and by 25 participants (13.4%) in response to the statement "ICT-EMS seems easy to use," the latter of which had the lowest agreement rate. In response to the statement "ICT-EMS seems doable," the responses "agree" or "strongly agree" and "disagree" or "strongly disagree" were equally selected by 52 participants (27.8%) (Fig. 2A) . In the AIM survey regarding the acceptability of the ICT-EMS systems, the responses "disagree" or "strongly disagree" in all statements were higher in number compared with "agree" or "strongly agree." The responses "agree" or "strongly agree" were selected by 19 participants (10.2%) in response to the statement "ICT-EMS meets my approval", which had the lowest agreement rate (Fig. 2B ). In the IAM survey used to assess the appropriateness of the ICT-EMS systems as an intervention, "agree" or "strongly agree" were more selected than "disagree" or "strongly disagree," by 54 (28.9%) and 51 participants (27.3%), respectively, in response to the statement "ICT-EMS seems fitting." The responses "agree" or "strongly agree" in other statements were lower in number compared with "disagree" or "strongly disagree. The agreement rate (14.4%) by 27 participants was the lowest in response to the statement "ICT-EMS seems suitable" (Fig. 2C) . This study was aimed to assess the usability, feasibility, acceptability, and appropriateness of ICT-EMS systems developed for coordinating the transportation of patients with suspected COVID-19, as well as critically ill patients during the COIVD-19 pandemic. The overall SUS score in our study was 35.6, which was significantly lower compared with the standard average score of 68 or higher, which typically indicates a sufficiently usable ICT-EMS systems (12) . In the FIM, AIM, and IAM surveys, the agreement rate of the approvability, ease of use, and suitability of the questionnaires were the lowest. The results of this study indicate that there is resistance among EMTs due to usability, feasibility, acceptability, and appropriateness flaws within the ICT-EMS systems. Through the construction of digital platforms for sharing patient data and information regarding emergency resources at community hospitals, EMTs and medical personnel in participant hospitals can communicate to determine where to transport patients for optimal patient care within an appropriate amount of time. ICTs have the potential to improve the treatment of critical patients diagnosed with acute myocardial infarction, acute stroke, and COVID-19 (13, 14) . The use of applications and ICTs in EMS systems is expected to have a positive effect on treatment outcome and administrative and supporting processes. User experience could be a vital factor affecting the SUS score and the agreement rate of the FIM, AIM, and IAM surveys obtained in this study. The SUS, originally created by John Brooke in 1986, was designed to provide a reliable and expedient scale for assessing the perceived usability of a wide variety of products, services, or systems, including applications, mobile devices, and websites (15) . Items were presented alternatively, switching between positively-and negatively-phrased statements. This alternating structure increases the cognitive burden on the respondents answering the questionnaire (16) . In the analysis of responses to the SUS questionnaire, the mean of negatively-phrased items was lower than that of positively-phrased items (2.5 vs. 3.6). After negatively-phrased items were inverted, a mean SUS score of 35.6 was obtained, indicating that the overall SUS response of EMTs regarding the ICT-EMS systems was negative. We believe that the low SUS scores and the low agreement rate of the FIM, AIM, and IAM surveys regarding the ICT-EMS systems are due to the EMTs' dissatisfaction with the system configuration, such as required multitasking, complexity, and inconsistency. EMTs were required to manage the patient and enter patient information into the ICT-EMS systems during transportation. Previously, EMTs used an existing tablet PC to enter basic patient information during transport, so it may have felt cumbersome to use an additional device to enter data into the ICT-EMS systems. ICT-EMS systems modification is necessary to be effectively integrated into the workflow of the community EMS (17). Given that the system's usability, feasibility, acceptability, and appropriateness scores were related to user satisfaction, it may be improved upon by collecting the opinions of users, including EMTs and medical personnel, to fully identify issues (18) . Then, training EMTs and medical personnel on these upgraded ICT-EMS will be needed. Modifying EMT behaviors related to the ICT-EMS J o u r n a l P r e -p r o o f may take time, but it can be done by generating positive motivation to reach better outcomes for emergency patients (19) . This study has some limitations which should be acknowledged. First, study surveys were conducted at the early stages of a project after 3 weeks of ICT-EMS system implementation, and participating EMTs' perceptions of the ICT-EMS systems were not positive overall. The one-month implementation period may have been too short for sufficient exposure to the ICT-EMS systems prior to perception assessment. In addition, EMTs employed at fire stations that are located at some distance from the central area of North Chungcheong Province occasionally transport patients to neighboring communities where ICT-EMS systems are not used; therefore, they may not have had ample opportunities to use the ICT-EMS systems during patients' transportation. Third, half of the participants (47.8%) were employed at two fire stations in the study community. The average time of transportation was about 10 minutes, which is insufficient time to use ICT-EMS application during transportation. This was the primary point of dissatisfaction reported by EMTs regarding the ICT-EMS systems. Improvements to ease of use and EMT familiarity with the ICT-EMS systems are necessary to promote the use of ICT-EMS systems during patients' transportation across short distances. The perceived usability, feasibility, acceptability, and appropriateness of ICT-EMS systems implementation reported by EMTs was poor, indicating that there are many areas of improvement that can be addressed to improve the ICT-EMS systems. Before the widespread establishment of ICT-EMS systems for facilitating emergency transport of patients during the COVID-19 pandemic, additional surveys and interviews of proposed ICT-EMS systems can be helpful to understand the issues associated with the ICT-EMS systems. After addressing areas of improvement, such as reducing multitasking and simplifying the complexity of the ICT-EMS systems, this survey may be repeated to assess the perception of the modified ICT-EMS systems. Tintinalli's emergency J o u r n a l P r e -p r o o f Journal Pre-proof medicine: a comprehensive study guide COVID-19 Pandemic: The Role of EMS Physicians in a Community Response Effort Effect of the Coronavirus Disease 2019 (COVID-19) pandemic on the US Emergency medical services system: a preliminary report Outcomes for out-of-hospital cardiac arrest in the United States during the coronavirus disease 2019 pandemic Comparison of out-ofhospital cardiac arrests and fatalities in the metro Detroit area during the COVID-19 pandemic with previous-year events Coronavirus Disease (COVID-19): A primer for emergency physicians Challenge of COVID-19 crisis managed by emergency department of a big tertiary centre in Saudi Arabia The use of ict-based applications to support the implementation of smart cities during the covid-19 pandemic in Indonesia A conceptual framework for Emergency department design in a pandemic Information and communication technologies in emergency care services for patients with COVID-19: a multi-national study Psychometric assessment of three newly developed implementation outcome measures Information Technology-Based Management of Clinically Healthy COVID-19 Patients: Lessons From a Living and Treatment Support Center Operated by Seoul National University Hospital The usefulness of information and communication technologies in crisis response Is It Time to Go Positive? Assessing the Positively Worded System Usability Scale (SUS) Memory and decision processes: The impact of cognitive loads on decision regret Emerging digital technologies in emergency medical services: Considerations and strategies to strengthen the continuum of care, National Highway Traffic Safety Administration DOT HS 811 999c Dimensionality of the system usability scale among professionals using internet-based interventions for depression: A confirmatory factor analysis Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Front public Heal I would like to use the ICT application for EMS improvement frequently 2. I found the ICT application for EMS improvement unnecessarily complex 3. I thought the ICT application for EMS improvement was easy to use I think that I would require technical support to be able to use the ICT application for EMS improvemen 5. I thought the various functions in the ICT application for EMS improvement were well integrated I thought there was too much inconsistency in the ICT application for EMS improvement I think that most people would learn to use the ICT application for EMS improvement very quickly 8. I found the ICT application for EMS improvement very cumbersome to use I felt very confident using the ICT application for EMS improvement I needed to learn a lot of things before I could begin using the ICT application for EMS improvement 1. The EMS improvement project using ICT seems implementable. 2. The application of the EMS improvement project using ICT seems possible. 3. The EMS improvement project using ICT seems doable. 4. The application of the EMS improvement project using ICT seems easy to use. 1. The application of the EMS improvement project using ICT meets my approval.2. The application of the EMS improvement project using ICT is appealing to me. 3. I like the application of the EMS improvement project using ICT 4. I welcome the application of the EMS improvement project using ICT Intervention Appropriateness Measure 1. The application of the EMS improvement project using ICT seems fitting. 2. The application of the EMS improvement project using ICT seems suitable. 3. The application of the EMS improvement project using ICT seems applicable. 4. The application of the EMS improvement project using ICT seems like a good match.