key: cord-1017733-q8f8tf1g
authors: BARKAI, Galia; GADOT, Moran; AMIR, Hadar; MENASHE, Michal; SHVIMER-ROTHSCHILD, Lilach; ZIMLICHMAN, Eyal
title: Patient and clinician experience with a rapidly implemented large-scale video consultation program during COVID-19
date: 2020-12-14
journal: Int J Qual Health Care
DOI: 10.1093/intqhc/mzaa165
sha: 69bd0365e7ce44faf61ca6338716d5bdc97cb27f
doc_id: 1017733
cord_uid: q8f8tf1g

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic has forced health-care providers to find creative ways to allow continuity of care in times of lockdown. Telemedicine enables provision of care when in-person visits are not possible. Sheba Medical Center made a rapid transition of outpatient clinics to video consultations (VC) during the first wave of COVID-19 in Israel. OBJECTIVE: Results of a survey of patient and clinician user experience with VC are reported. METHODS: Satisfaction surveys were sent by text messages to patients, clinicians who practice VC (users) and clinicians who do not practice VC (non-users). Questions referred to general satisfaction, ease of use, technical issues and medical and communication quality. Questions and scales were based on surveys used regularly in outpatient clinics of Sheba Medical Center. RESULTS: More than 1200 clinicians (physicians, psychologists, nurses, social workers, dietitians, speech therapists, genetic consultants and others) provided VC during the study period. Five hundred and forty patients, 162 clinicians who were users and 50 clinicians who were non-users completed the survey. High level of satisfaction was reported by 89.8% of patients and 37.7% of clinician users. Technical problems were experienced by 21% of patients and 80% of clinician users. Almost 70% of patients but only 23.5% of clinicians found the platform very simple to use. Over 90% of patients were very satisfied with clinician’s courtesy, expressed a high sense of trust, thought that clinician’s explanations and recommendations were clear and estimated that the clinician understood their problems and 86.5% of them would recommend VC to family and friends. Eighty-seven percent of clinician users recognize the benefit of VC for patients during the COVID-19 pandemic but only 68% supported continuation of the service after the pandemic. CONCLUSION: Our study reports high levels of patient satisfaction from outpatient clinics VC during the COVID-19 pandemic. Lower levels of clinician satisfaction can mostly be attributed to technical and administrative challenges related to the newly implemented telemedicine platform. Our findings support the continued future use of VC as a means of providing patient-centered care. Future steps need to be taken to continuously improve the clinical and administrative application of telemedicine services.

Telemedicine, defined by the World Health Organization as 'the delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies' [1] , has been increasingly recognized as a means to improve access to health care. Despite recognition of telemedicine's potential to augment in-person visits, several barriers have prevented its widespread implementation [2] . Those include clinician and patient acceptance, technological connectivity problems, reimbursement as well as ethical and regulatory complexities regarding data security and privacy [3] .

Small randomized trials in selected samples of hospital outpatients with chronic conditions have shown that video consultations (VC) are associated with high satisfaction among patients and staff with no difference in clinical outcome, and with lower transaction costs compared with traditional clinic-based care [4] [5] [6] [7] [8] . However, in a systematic review of studies published recently, VC were not deemed appropriate for all situations and face-to-face consultations were preferable when possible. VC were potentially more convenient for patients but were not considered superior to face-to-face consultations [9] .

The coronavirus disease 2019 (COVID-19) pandemic has caused a new world order and has forced all health-care providers to find creative ways to allow continuity of care in times of social distancing measures. In view of this new reality, when it was much more challenging for chronic patients to physically attend the clinics, it was only natural that telemedicine would thrive as an effective means to provide care.

On 27 February 2020, the first COVID-19 patient was diagnosed in Israel and hospitalized at Sheba Medical Center. Soon after, social distancing measures began to be gradually implemented, and on 19 March 2020, an almost complete lockdown was declared. Sheba Medical Center had already integrated a telemedicine platform (Datos © , Ramat Gan, Israel) to its services and was deploying an array of specialized telemedicine services such as home hospitalization for psychiatric patients, remote cardiac rehabilitation, symptom management programs and home monitoring of chronic patients. However, there was no widespread adoption of VC as a state-of-theart modality in the outpatient clinics. Sheba Medical Center made a rapid transition of outpatient clinics from in-person visits to VC in order to provide its patients continuity of care during the lockdown. High-speed internet and mass spread of smartphones in Israel made it possible to deploy VC services to be performed from patients' homes.

We aimed to get a wide perspective on both patient and clinician experience and perceptions of VC and learn about barriers in implementation, in order to be able to quickly respond and improve the programs' adoption. Therefore, we distributed patient and clinician satisfaction surveys.

This study reports on patient and clinician experience and may yield insights for further implementation of VC services.

Sheba Medical Center is the largest hospital in Israel with 1900 beds and a large ambulatory service of 1 600 000 patient visits per year. During March 2020, within 10 days, more than 1200 clinicians (physicians, psychologists, nurses, social workers, dietitians, speech therapists, genetic consultants and others) were able to provide VC either from the hospital or from home using the Datos © telemedicine platform. Sheba Medical Center's daily VC sessions increased from less than 30 to more than 700, within 2 weeks.

Datos © is a web-based platform that allows clinicians to easily contact patients through a link sent as a text message or an email. By clicking on the link, the patient connects to the VC without having to download an application or a software.

A patient satisfaction questionnaire was sent as a link through a text message to the cellular phones of all consecutive outpatients who attended VC during the study period not more than 3 days following the visit. Those who did not respond received two reminders 3 days apart. Questions referred to general satisfaction, ease of use, technical issues and medical and communication quality of the VC. The survey's questions and scales were based on surveys used regularly in outpatient clinics of Sheba Medical Center, which are partially based on the Israeli Ministry of Health (MOH) patient experience survey for outpatient clinics. To these, we added questions about technical issues and ease of use. Since satisfaction questionnaires are part of our quality program and due to the fact that the survey is anonymous, and that the only information that was retrieved was the patients' phone number and clinic affiliation, an informed consent was not obtained.

In order to understand the barriers to telemedicine adoption among clinicians, an anonymous clinician experience survey was sent in a similar way to staff members who were both users of telemedicine (performed at least five VC) and non-users (those who performed less than five VC). Those who did not respond received two reminders 3 days apart. The survey questions referred to their overall experience with VC, technical aspects, subjective perception of the service and thoughts of future directions. The non-user clinicians received a shorter survey regarding their perception and barriers to performing VC.

Answers to strategic questions as general satisfaction and willingness to recommend the service were ranked on a 1-10 Likert scale. Ranks of 8-10 were considered high level of satisfaction, 4-7 were considered intermediate level of satisfaction and 1-3 were considered low level of satisfaction. Answers to tactical questions as ease of use and adherence to schedule were ranked on a 1-5 Likert scale where 5 was considered high, 3-4 were considered intermediate and 1-2 were considered low. For 'user support for continuation of VC after the pandemic', ranks of 4-5 were considered as support and 1-3 as do not support. The questionnaires (translated from Hebrew) are included in the supplementary material.

Statistics: Descriptive statistics were calculated with the Excel software (Microsoft, Redmond, WA). A chi-square test of indepen- 

Between 12 April and 3 May 2020, a questionnaire was sent to 2455 patients. Five hundred and forty patients (22%) filled the survey. Patients' distribution by clinics attendance is presented in Table 1 .

Four hundred and eighty-five patients (89.8%) reported high level of satisfaction, 34 (6.3%) reported intermediate and 21 (3.9%) reported low satisfaction with VC. Figure 1a outlines overall satisfaction rates in different clinics. No significant difference was noted in satisfaction level between different clinics (P = 0.211).

In 383 (71%) cases VC were performed using the Datos © platform. Eighty-one patients (21%) experienced technical problems (connection, sound and/or video issues). In 108 (20%) cases a telephone conversation was eventually performed due to technical difficulties and 49 (9.1%) used other VC platforms. For 376 six patients (69.6%) the platform was very simple to use, for 116 (21.4%) it was quite simple and only 48 (8.9%) found it complicated to operate. No significant difference regarding ease of use was noted between patients in different clinics (Figure 1b ; P = 0.269).

In 378 (70%) of consultations the appointment began within 30 minutes of scheduled time. Sixty-five patients (12%) waited more than 1 h for their appointment. High satisfaction rates with waiting times (4-5 on a 1-5 scale) were noted in 363 (96%) patients who waited less than 30 minutes for their VC. This rate dropped dramatically to 57% for patients who waited 30-60 minutes, to 52% for those who waited 1-2 h and to 27% for patients waiting more than 2 h.

Measures of satisfaction with the quality of interaction with clinicians yielded high results. High satisfaction was noted when patients were asked about clinician's courtesy (502, 93%), feeling of trust (491, 91%), clinician's understanding of the patients' complaint (486, 90%) and clarity of the clinician's explanations and recommendations (486, 90%).

When patients were asked if they would recommend VC to family and friends, 515 patients replied. In 445 (86.5%) cases, patients would recommend the service with high probability, in 49 (9.5%) with intermediate and in 21 (4.1%) cases with low probability. No significant difference was noted between patients in different clinics (Figure 1c ; P = 0.354).

Ongoing patient satisfaction surveys in our outpatient face-toface clinics have consistently shown high levels of satisfaction. In 2019, 33 322 patients (20% of survey recipients) replied. This rate is similar to the VC survey response rate. Satisfaction rates with VC were as high as historical face-to-face visits: high rates were noted in general satisfaction (85% in 2019 vs. 89.8% in VC) as well as willingness to recommend the service to family or friends, an important measure of satisfaction (87% in 2019 vs. 86.5% in VC).

Between 4 May and 20 May 2020, a clinician experience survey was sent to 469 users, 162 (35%) replied. Among users, 86 (53%) users were physicians, 38 (23%) psychologists, 29 (18%) dietitians and speech therapists, 6 (4%) social workers and 3 (2%) nurses. Fifty (22.7% of 220) clinicians replied to a survey that was sent to VC nonusers. Among non-users 31 (62%) were physicians, 8 psychologists (16%), 8 dietitians and speech therapists (16%) and 3 (6%) were social workers. Clinics of origin are presented in Table 1 .

Sixty-one clinicians (37.7%) reported high levels of satisfaction with the service, 83 (51.3%) reported intermediate and 18 (11.1%) reported low satisfaction with VC. Figure 2a outlines general satisfaction rates in different clinics. Although no significant differences were noted in levels of general satisfaction between different services (P = 0.056), when we compared the service with the lowest satisfaction rate (oncology) to that of the service with the highest satisfaction rate (psychiatry) we found a significant difference (P = 0.006).

Thirty-eight (23.5%) users found the platform very simple to use, for 99 (61%) it was quite simple and 25 (16%) found it complicated to operate. Significant differences regarding ease of use were noted between clinicians in different clinics (Figure 2b ; P < 0.01 between all groups).

All the clinicians mentioned encountering at least one problem while practicing VC compared to face-to-face visits. Problems causing increased workload were: technical problems at patient side (138, 80%); technical issues at clinician side (42, 26%); delays in VC initiation (83, 51%); pre-visit preparation (34, 21%); difficulties to access out-of-hospital records (29, 18%); need for a face-to-face visit (39, 24%) and difficulty in transmitting documents (32, 20%). Twenty-six (16%) clinicians expressed difficulty understanding the patients' complaints and performing a medical evaluation and 23 (14%) found it hard to evaluate the patients' mental state.

The clinicians estimated that on average 31% of scheduled VC were eventually performed by phone.

Eighty (49%) clinicians assumed that more than half of their VC were on time. The rest of the clinicians (82, 51%) who reported delays in most of VC stated that the causes were: delays in clinicians' schedules (41, 50%), technical problems (28, 34%) and patient unavailability (22, 27%) .

Most of the users (141, 87%) recognize the high benefit of VC for patients during the COVID-19 pandemic but only 110 (68%) users supported continuation of the service after the pandemic is over. It should be noted that a lower proportion of oncologist users (25, 58%) supported continuation of VC after the pandemic (Figure 2c ). 

Reasons for not practicing VC among non-users were: estimation that VC are not appropriate for their patients (21, 42%); concern that VC will prolong patients' waiting time (15, 30%), 8 (16%) were not aware of the service and 13 (26%) other reasons.

Twenty-two (44%) non-users supported continuation of the service after the pandemic is over. Of note, 2/3 supported further promotion of VC services in the future.

During the COVID-19 pandemic, telemedicine, particularly VC, took an important place in addressing the crisis's challenges by limiting patient and clinician exposure, reducing protective equipment waste and maintaining high-quality continuous medical care. Sheba Medical Center, as other medical centers, adopted the use of VC and expanded it rapidly.

Almost 90% of patients expressed high levels of satisfaction with VC during the COVID-19 lockdown, regardless of the clinic in which they were treated. The high satisfaction rate was maintained despite the technical and administrative difficulties some of the patients experienced. Overall, most of the patients were able to have high-quality interactions with the clinicians. We were encouraged by the fact that VC yielded very high satisfaction levels in terms of the communication experience with clinicians, since this is the core of medical practice.

Clinician satisfaction, however, was significantly lower (less than 40% expressed high levels of satisfaction), mainly because of technical problems leading to increased workload. In addition, there was a noticeable difference in satisfaction rates between clinicians. We believe that the major cause of the low satisfaction rates of the oncologists was related to the cumbersome administrative process that affects them more than other clinicians since they often need to retrieve information from external sources (e.g. laboratory results and imaging performed outside) and need to make further appointments. In contrast, we assume that the high satisfaction rates among the psychiatrists has to do with the fact that they do not perform a physical examination, the time frame for their appointments is much longer than that of oncologists (50 minutes vs. [15] [16] [17] [18] [19] [20] and that the administrative tasks burden is relatively low in their practice.

This study is unique by its focus on the overall human experienceboth clinician and patient experience with the rapid and widespread adoption of VC service in a wide range of outpatient clinics in a large medical center during a unique period-the COVID-19 pandemic.

Our study has some limitations. Since the satisfaction questionnaires were adopted from routinely used questionnaires, partly created by the MOH, not all scales were ranked similarly. This created difficulty in comparing and presenting the results. In addition, we did not have access to demographic information of patients and clinicians. This information could have shed more light on personal differences in adoption of VC and satisfaction with this platform. Since only 22% of patients who received the survey completed it, a potential selection bias could have occurred. It is possible that those who answered the survey were the outliers (this is a generic limitation of satisfaction surveys). Still, this rate is considered common for patient satisfaction surveys and is similar to compliance rate at Sheba Medical Center in previous years.

Studies performed before COVID-19 have shown that VC were associated with high satisfaction among patients and staff with no difference in clinical outcome [4] [5] [6] [7] [8] . However, face-to-face consultations were considered preferable when possible.

Researchers have addressed the various aspects of rapid telemedicine implementation and use at this critical time: technical, regulatory and economic challenges [10] [11] [12] [13] . A few recently published studies investigated patient satisfaction of newly deployed telemedicine programs during COVID-19 [14] [15] [16] [17] [18] . Like our study, all these studies show very high levels of patient satisfaction with VC, stating that from the patient's point of view, there is no barrier for paradigm shift away from traditional in-personal clinic visits to VC.

As for clinicians, studies have shown that VC introduction raises their concern about technical and clinical quality, privacy, safety and accountability [19] [20] [21] [22] . Similar concerns were previously raised by clinicians in our institution, hindering the implementation of telemedicine initiatives. The COVID-19 lockdown forced some of our clinicians to experience VC for the first time. Very few studies have reported on clinician satisfaction with VC during COVID-19. In one study [23] physicians' satisfaction levels were acceptable (around 70%).

We found only one study reporting on both patient and clinician experience with a newly expanding VC service during COVID-19 in a sports clinic [24] . Very high satisfaction levels were noted by patients, but unlike our study, physicians were highly satisfied with the services as well. This difference may be explained by the fact that this study was performed in a narrow-field clinic. Therefore, its results may not be applicable to a wide range of specialties. Differences in VC platform and administrative tasks may also explain the higher level of clinician satisfaction. This study also identifies technical issues as being the key barrier for VC acceptance.

Telemedicine dramatically changes the mode of delivery, but a strong patient-clinician relationship must be maintained. Technical and administrative issues can be relatively easily addressed. Indeed, we have already made improvements in the platform such as addition of a virtual waiting room, a voice message notice, a patient training service prior to the first VC and a technical support center. However, full integration with the hospital's administrative and booking systems could not be completed in such a short period; therefore, the burden of performing VC is still significant. In addition, technical problems have become less frequent but still occur.

A simple-to-use, integrated platform as well as a seamless administrative process are key to the success of such a project that requires clinicians' adaptive changes. This means not merely installing or using new technology but introducing and sustaining major changes to a complex system. The COVID-19 pandemic forced us to rapidly deploy a minimum viable product, without having the time to develop a properly integrated service. This created a burden of clinicians' increased workload, and could explain, at least in part, their relatively low satisfaction. The upside of this rapid deployment was that it enabled us to quickly respond and improve the platform and process. The high patient satisfaction rates reinforced that such a service has a good potential to persist after the pandemic.

Our findings support the continued future use of VC as a means of providing patient-centered care. Future steps need to be taken to continuously improve the clinical, technical and administrative application of telemedicine services.

Future prospective controlled studies that include in-depth interviews and demographic information of patients and clinicians as well as characteristics of non-responders should provide more insight to specific barriers, allowing wide-scale implementation of telemedicine services.

All authors contributed significantly and revised the final manuscript.

The study received Sheba Medical Center's IRB approval for retrospective analysis of prospectively collected data. No identifying demographic data of patients and clinicians were available.

'Supplementary material' includes the questionnaires (translated to English) and selected detailed results and statistics. Any additional data that is not included within supplementary material will be shared on reasonable request to the corresponding author.

Telemedicine: Opportunities and Developments in Member States: Report on the Second Global Survey on eHealth. World Health Organization

Telemedicine: why the delay?

Evaluating barriers to adopting telemedicine worldwide: a systematic review

Global telemedicine implementation and integration within health systems to fight the COVID-19 pandemic: a call to action

The clinical use of Skype-for which patients, with which problems and in which settings? A snapshot review of the literature

Videoconferencing psychotherapy: a systematic review

Internet videoconferencing for patient-clinician consultations in long-term conditions: a review of reviews and applications in line with guidelines and recommendations

The medium, the message and the measure: a theory-driven review on the value of telehealth as a patientfacing digital health innovation

Exploring patients' and clinicians' experiences of video consultations in primary care: a systematic scoping review

The COVID-19 pandemic and rapid implementation of adolescent and young adult telemedicine: challenges and opportunities for innovation

Keep calm and log on: telemedicine for COVID-19 pandemic response

Global telemedicine implementation and integration within health systems to fight the COVID-19 pandemic: a call to action

Telemedicine in the era of COVID-19

Patient satisfaction with telemedicine during the COVID-19 pandemic: retrospective cohort study

Telemedicine for head and neck ambulatory visits during COVID-19: evaluating usability and patient satisfaction

Telemedicine during the COVID-19 pandemic: impact on care for rare cancers

Pilot study using telemedicine video consultation for vascular patients' care during the COVID-19 period

Evaluation of patient satisfaction with an ophthalmology video consultation during the COVID-19 pandemic

Adoption of routine telemedicine in Norwegian hospitals: progress over 5 years

Exploring factors associated with the uneven utilization of telemedicine in Norway: a mixed methods study

Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods study

Acceptability, benefits, and challenges of video consulting: a qualitative study in primary care

The use and acceptance of telemedicine in orthopedic surgery during the COVID-19 pandemic

Telemedicine during COVID-19 for outpatient sports andmusculoskeletal medicine physicians