key: cord-0799648-f6nio6m7
authors: Bilimoria, Karl Y; Zhan, Tiannan; Durst, Dalya; Merkow, Ryan; Sama, Danny; Behavelos, Stratis; Chrisman, Howard
title: Comparison of Patient Experience with Telehealth vs. In-Person Visits Before and During the COVID-19 Pandemic
date: 2021-01-29
journal: Jt Comm J Qual Patient Saf
DOI: 10.1016/j.jcjq.2021.01.009
sha: 7c2cbb4d5cbd7a3d3d9f3a5ff5ff946d626036ad
doc_id: 799648
cord_uid: f6nio6m7

nan

Healthcare providers, who often minimally utilized telehealth before the pandemic, rapidly transitioned to meet the health needs of their patients (i.e., <10% prior to COVID to >70% at the peak). 1, 2 However, with the swift conversion to telehealth, the impact on patient experience is unknown. 3, 4 Our two-fold objectives were to compare patient experience for telehealth vs. in-person visits both before and during the COVID-19 pandemic.

Methods: Patient experience was assessed using a novel electronic, web-based survey at 405 ambulatory clinics (1,920 clinicians) covering all specialties in a nine-hospital healthcare system (1 academic medical center, 7 community hospitals, 1 critical access hospital). Two questions, each scored on an 11-point scale from 0 (not likely at all) to 10 (extremely likely), were sent to patients, by text message and/or email, after each clinic visit to evaluate the likelihood to recommend (LTR) (1) "the clinic for care" and (2) "the provider for care." LTR question used in our survey is very similar to the LTR measure used widely across the country in ambulatory patient experience surveys. 5 To develop the survey questions, cognitive interviews were conducted with a diverse group of patients to assess overall survey coherence and clarity. The survey was iteratively revised and re-tested in a larger sample of patients from multiple institutions.

We compared patient experience metrics for those who had telehealth visits during the early COVID-19 era (defined as 3/17/2020-4/28/2020) versus two comparison groups: (1) patients contemporaneously having in-person visits (3/17/2020-4/28/2020) and (2) patients having in-person visits prior to COVID-19 (11/1/2019-3/16/2020). The visit type was determined by a combination of a required billing code modifier and visit type recorded in Electronic Health Record (EHR). Our definition of telehealth visits include synchronous visits, conducted by telephone or video. Patient and provider characteristics were matched to each patient encounter.

Multivariable logistic regression models were developed to predict patient dissatisfaction (LTR scores ≤8), adjusting for patient-reported gender, age, patient type (new patient versus returning visit), race, operating units, provider gender, provider specialty, and provider years in practice.

Robust standard errors clustered by provider specialties were calculated. Stratified models were run for each of the three groups, adjusting for the same factors. Sensitivity analyses were carried out for new patients. The Northwestern University Institutional Review Board deemed this study Exempt.

Of 844,483 eligible encounters, 200,987 surveys were completed between November 1, 2019, and April 28, 2020, with an overall response rate of 23.8%. Comparing telehealth to inperson visits during the COVID era, LTR scores for the provider were similar for in-person visits and telehealth visits (mean LTR: 9.72 vs. 9.74, p=1.00) ( Table 1) . Both in-person and telehealth visit scores during the COVID era were significantly higher than pre-COVID in-person visits (mean LTR: 9.64 p<0.001 for both comparisons). When comparing telehealth vs in-person visits, there were no differences when separately examining new patient visits; no differences were detected in scores between pre-COVID and COVID era for new patients. Results for clinic LTR were comparable to those for provider LTR.

Similarly, there was not a significant difference between COVID-era telehealth visits and COVID in-person visits when adjusting for patient and clinician factors (5.5% vs. % 5.8% dissatisfaction; OR 1.06, 95% CI 0.94 -1.20; Table 2 ). Patients were more likely to report dissatisfaction if they were female, younger, non-White, new visits, or seeking care during the pre-COVID era.

Clinicians who were in practice less than 10 years or more than 30 years received lower ratings than those with 11-30 years of experience.

Use of telehealth during the COVID era resulted in increased LTR scores (i.e., decreased dissatisfaction) among Black patients (Pre-COVID in person: 8.8%, COVID in person 9.4%, COVID telehealth 6.6%; Table 2 ). Discussion: Patient experience scores during the rapid initial transition to telehealth because of the COVID-19 pandemic resulted in comparable patient experience scores between in-person and telehealth visits. It is unclear why COVID-era scores were higher than pre-COVID scores, but in general, there may have been a tendency for patients to be more benevolent to healthcare workers during the early part of the COVID-19 pandemic 6 , though less so for newvisit patients who showed no difference in patient satisfaction comparing telehealth vs in-person visits during pre-COVID vs COVID era. Limitations of this study should be acknowledged. First, there is an inability to assess the telehealth visit modality (telephone versus video) during the early stage of the pandemic. Second, the clinics examined are part of a single healthcare system within a single state, so results may not be generalizable. Third, while the response rate may appear low for a survey, this rate is higher than that of typical patient experience survey response rates nationally. 7 Fourth, the effects of the broad utilization of telehealth on diagnostic errors, care delays, and quality of care was not examined. Nonetheless, these results suggest that the rapid shift to telehealth generally resulted in favorable patient experience, and possibly some improvement in racial disparities; however, attention should be paid to new patient visits as these may not be as ideal via telehealth. ‡ Net promoter score is calculated by multiplying the difference between promoter (a score of >8) % and detractor (≤6)% by 100; § Kruskal-Wallis test for mean rank was significant (P<0.001); ¶ Comment rate refers to the percent of responders who wrote a comment in the optional free-text comment box. Post hoc pairwise Dunn test with bonferroni adjustment was significant (P<0.001) for: * COVID in-person visit vs pre-COVID in-person visit, and ** COVID telehealth visit vs pre-COVID in-person visit. 8 

Using Telehealth to Expand Access to Essential Health Services during the COVID-19 Pandemic

Telemedicine and e-Health Solutions for COVID-19: Patients' Perspective

The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence

Agency for Healthcare Research and Quality, The CAHPS Ambulatory Care Improvement Guide: Practical Strategies for Improving Patient Experience

Insights from National Patterns in Patient Experience During the COVID-19 Crisis

Evidence of non-response bias in the Press-Ganey patient satisfaction survey

Additional Contributions: We thank the Department of Patient Engagement, specifically

for their administrative study support. We also thank Kirsty