key: cord-284188-ujdla954
authors: Smith, Silas W.; Tiu, Janelle; Caspers, Christopher G.; Lakdawala, Viraj S.; Koziatek, Christian A.; Swartz, Jordan L.; Lee, David C.; Jamin, Catherine T.; Femia, Robert J.; Haines, Elizabeth J.
title: Virtual Urgent Care Quality and Safety in the Time of Coronavirus
date: 2020-10-16
journal: Jt Comm J Qual Patient Saf
DOI: 10.1016/j.jcjq.2020.10.001
sha: 
doc_id: 284188
cord_uid: ujdla954

INTRODUCTION: Telemedicine use rapidly increased during the coronavirus disease 2019 pandemic. We assessed quality aspects of rapid expansion of a virtual urgent care (VUC) telehealth system and the effects of a secondary telephonic screening initiative during the pandemic. METHODS: We performed a retrospective cohort analysis in a single hospital network of VUC patients from March 1, 2020 through April 20, 2020. We abstracted demographic data, comorbidities, VUC return visits, emergency department (ED) referrals, and ED visits, dispositions, intubations, and deaths. We also reviewed incomplete visits. For comparison, we evaluated outcomes of non-admission dispositions from the ED: return visits with and without admission and deaths. We separately analyzed the effects of enhanced callback system targeting higher risk patients with COVID-like illness during the last two weeks of the study period. RESULTS: A total of 18,278 unique adult patients completed 22,413 VUC visits. Separately, 718 patient-scheduled visits were incomplete; the majority were no-shows. 50.9% of all patients and 74.0% of patients aged 60 years or older had comorbidities. Of VUC visits, 6.8% had a subsequent VUC encounter within 72 hours; 1.8% had a subsequent ED visit. Of patients with enhanced follow-up, 4.3% were referred for ED evaluation. Mortality was 0.20% overall; 0.21% initially and 0.16% with enhanced follow-up (p = 0.59). Males and black patients were significantly overrepresented in decedents. CONCLUSION: Appropriately deployed VUC services can provide a pragmatic strategy to care for large numbers of patients. Ongoing surveillance of operational, technical, and clinical factors is critical for patient quality and safety with this modality.

Telemedicine use rapidly increased during the coronavirus disease 2019 pandemic. We assessed quality aspects of rapid expansion of a virtual urgent care (VUC) telehealth system and the effects of a secondary telephonic screening initiative during the pandemic.

We performed a retrospective cohort analysis in a single hospital network of VUC patients from March 1, 2020 through April 20, 2020. We abstracted demographic data, comorbidities, VUC return visits, emergency department (ED) referrals, and ED visits, dispositions, intubations, and deaths. We also reviewed incomplete visits. For comparison, we evaluated outcomes of nonadmission dispositions from the ED: return visits with and without admission and deaths. We separately analyzed the effects of enhanced callback system targeting higher risk patients with COVID-like illness during the last two weeks of the study period.

A total of 18,278 unique adult patients completed 22,413 VUC visits. Separately, 718 patientscheduled visits were incomplete; the majority were no-shows. 50.9% of all patients and 74.0% of patients aged 60 years or older had comorbidities. Of VUC visits, 6.8% had a subsequent VUC encounter within 72 hours; 1.8% had a subsequent ED visit. Of patients with enhanced follow-up, 4.3% were referred for ED evaluation. Mortality was 0.20% overall; 0.21% initially and 0.16% with enhanced follow-up (p = 0.59). Males and black patients were significantly overrepresented in decedents.

Appropriately deployed VUC services can provide a pragmatic strategy to care for large numbers of patients. Ongoing surveillance of operational, technical, and clinical factors is critical for patient quality and safety with this modality.

As healthcare delivery systems confronted coronavirus disease 2019 (COVID- 19) , telemedicine was encouraged as a "Virtually Perfect" 1 solution. The US Surgeon General explicitly advocated for telehealth "revolution." 2 Telemedicine would permit structured assessments while maintaining social distance, mitigating supply-demand mismatch, and preserving the health care workforce. [3] [4] [5] We previously suggested the need for flexible, adaptive, scalable care systems to assess, stabilize, and treat patients in future disasters. 6 As New York City's COVID-19 epicenter precipitously expanded in March 2020 and presentations to our institution rose abruptly, a nascent virtual urgent care (VUC) service rapidly expanded to address patient care.

Telemedicine has existed for decades, with a successful history of being leveraged to treat both acute and chronic infectious disease, including respiratory illness. 7, 8 In many cases, telehealth has provided equivalent outcomes to in-person care. 9 Telemedicine has also demonstrated an important role within urban health care, where a variety of barriers may impair access. 10 However, several limitations have been recognized, particularly constraints on the physical examination. 11 Technological, administrative, and legislative infrastructure present additional barriers. 1, 12, 13 With widespread staff redistribution, telehealth training modalities 1 in the crisis application of technology might insufficiently provide clinicians with technologic expertise and diagnostic and disposition comfort. Absent robust antecedent telemedicine experience, providers might struggle to translate in-person skills to a virtual medium, and precise referral or escalation guidelines 4 might be nonexistent in the face of rapidly changing recommendations from public health entities. Minimal or nonexistent outpatient laboratory and radiographic diagnostics (due to patient preference, provider attempts to preserve social distancing, and site closures), which would be typically available during normal operations, could further impede management decisions. A lack of dedicated follow-up 5 and consultation services might similarly complicate management. Last, telemedicine might not maintain its generally low-risk demographics, might fail to address distinct patient population needs, and might overlook social determinants of health and health disparities. 13, 14 Technology may follow a "hype cycle" or curve. Following an "innovation trigger," a period of "inflated expectations" gives rise to a subsequent "trough of disillusionment" and a "slope of enlightenment," until achieving a "plateau of productivity." 15 However, inflated expectations of telemedicine could compromise patient quality and safety if applied to undeveloped or immature systems. In this investigation, we review quality and safety aspects of rapid deployment of a VUC telehealth system during a pandemic. These data are critical to appraise telehealth quality and safety issues in a disaster, and to design patient care processes for potential subsequent COVID-19 outbreaks. Furthermore, they may inform rapid telemedicine implementation for healthcare delivery in general.

This was a single-hospital network, retrospective cohort analysis of all patients who presented to VUC services from March 1, 2020 to April 20, 2020. Operational deployment, technology, geographic distribution, and patient satisfaction in the early pandemic phase are described in a separate manuscript. 16 The study was approved by the Institutional Review Board of the New York University Grossman School of Medicine, Office of Science and Research. A waiver of consent and authorization was granted for this study. No funding was received.

NYU Langone Health is an urban, multisite tertiary academic network of four acute care hospitals and more than 350 outpatient locations. More than 300,000 patients are treated annually across its five emergency department and immediate care spaces. These sites care for a cross-section of patients both new to the system and associated through prior ED visits or other care elements (e.g., outpatient providers). VUC services, administered by the academic department of emergency medicine, were extant since 2017. VUC was staffed entirely by emergency physicians prior to the study period. During the pandemic, allied health providers were also utilized. Visits were conducted via an online video telehealth platform, which was integrated into the electronic health record (EHR) (Epic Systems Corporation, Verona, WI). A custom augmented EHR application on mobile devices and computers allowed patients to schedule visits up to 48 hours in advance (and within 30 minutes of any open time slot).

Patients' location was limited only by access to a mobile device or a computer with internet access. Patients completed a questionnaire of their medical history, medications, allergies, pharmacy, and reason for visit. Providers, working stand-alone shifts or portions of shifts, accessed the EHR through computers for health record review and documentation, and conducted the video visit via EHR mobile applications. Twenty-four-hour coverage was provided throughout the study period. In March, patients were screened prior to network ambulatory visits for symptoms consistent with viral upper respiratory tract infection, prior to the closure of all network ambulatory sites on March 23. If they screened positive for these symptoms, they were directed to use VUC instead of an in-person visit. Additionally, direct emails regarding VUC availability to health system patients occurred. A formal VUC quality and safety framework was not in existence prior to the study period.

Of note, a severe deficit and delay in testing capacity and explicit public health authority recommendations discouraged COVID-19 testing for both outpatients and ED discharges (e.g., "DOHMH [NYC Department of Health and Mental Hygiene] strongly recommends against testing persons with mild illness who can be safely managed at home" [emphasis in original]). 20 This meant that for the vast majority of VUC patients, COVID-19 testing was never performed, even if warranted. For local context during the study period, given these significant constraints (i.e., testing reserved for hospitalized patients), the rates of positive COVID tests in Bronx, Kings, Nassau, New York, Queens, Richmond, Suffolk, and Westchester counties were 48%, 48%, 43%, 39%, 50%, 40%, 41%, and 33%, respectively. 21 Hospitals faced widespread shortages of personal protective equipment (PPE) throughout the study period. 22 Mandatory public face coverings were not required until April 17, 2020. 23 All patients presenting to VUC from March 1, 2020 to April 20, 2020 were included. As outcomes can trail index visits, patients were followed at least 21 days after the index VUC presentation. For study purposes, we evaluated only adult patients aged 18 or greater at the time of the visit. To provide context for patients managed as outpatients, we evaluated ED patients presenting during the same time period who were not admitted. We undertook several post-hoc analyses for comparison purposes, evaluating cohorts of patients using VUC in the prior year.

Emergency services, inpatient, and outpatient practices throughout the network utilize a uniform, integrated EHR. Regional data in the Epic platform is available, with extant patient consent, in sections of the EHR (Care Everywhere) and the public health information exchange (Healthix, New York). Software, Inc., Seattle, WA) was then used on the extracted data to input all of the necessary data fields, join the multiple queries, and remove duplicate rows.

To assess patient and provider data, systems effectiveness, and ability to provide care with and without admission to the ED, 17, 18 we identified the following variables from the cohort records: provider type, patient age, self-assigned gender, visit date, self-assigned race/ethnicity, 19 comorbidities, previous primary care visits (as an indirect marker of regular access to care), VUC return visits, ED referrals and spontaneous ED visits, ED and dispositions, intubations, and deaths following VUC visits. All decedents' charts for clinical presentation and hospital course, including medications, interventions, consults, and procedures were manually reviewed by emergency medicine (EM) attendings with additional subspecialty training in critical care, informatics, pediatric EM, and toxicology.

Partway through the study period, based on departmental and institutional assessments of COVID-19 risk factors, 24 VUC services introduced a workflow on April 7 targeting patients with telehealth visits for COVID-19 like complaints. Because this operational change might affect results, we further evaluated data within these two periods: March 1 to April 6, 2020 (period one) and April 7 to 20, 2020 (period two). VUC visits in patients aged 60 years or older who had fever, cough, or gastrointestinal complaints or a VUC discharge diagnosis of advice for or suspected COVID-19 were identified. Patients meeting criteria were called within 36 to 48 hours by a licensed independent provider who performed a standardized screening. Patients were asked about illness duration > 6 days, worsening symptoms from index VUC visit, presence of symptoms on ambulation or new neurologic symptoms including dizziness or falls, and presence of BMI > 25. Providers were instructed to consider immediate referral for in-person ED examination in patients having affirmative answers to any question. We evaluated and manually reviewed all patients identified for follow-up to assess outcomes including ED referral during follow-up screening call, hospital admission and course, presence or absence of ED referral on index VUC visit, mortality, and interval metrics.

Proposed telehealth performance measures include equivalent outcomes compared to face-toface care. 17 While no direct comparators existed during the pandemic, when decision thresholds to present to the ED were multifactorial, to provide context for departments managing patients as outpatients, we evaluated similar outcomes of patients presenting for ED care who were discharged (not admitted) during the same time period. These included patients who were discharged, were discharged to a nursing facility, departed against medical advice, eloped, left without being seen before triage, and left without being seen after triage. We evaluated return visits, return visits with admission (admitted, expired, sent to operating room, sent to labor and delivery, transferred to a procedure area, and transferred to another facility), and deaths.

Poison Control Centers have provided telephonic triage, medical decision-making, and follow-up of millions of emergent and non-urgent cases for decades, including disasters and events of public health significance. 25 Therefore, previous telephonic quality and safety measures such as risks posed by dropped calls were applied to VUC. 26 Separately, telehealth frameworks also suggest that these "dropped sessions" 17 represent a continuous performance improvement benchmark, and that technical issues should be assessed. 18 Visit failures (recorded as terminations in the EHR) are mutually exclusive categories-"no show," "criteria unqualified" (e.g., an out-of-state patient prior to revised CMS guidelines), and "connection issue" (technological failure). We further evaluated these categories as to whether patients completed check-in to VUC (i.e., were apparently actively intending to be seen) after scheduling an appointment. Terminations after check-in could approximate potentially concerning dropped visits. We also assessed whether patients were subsequently seen in VUC after visit termination (recovered) as an additional assessment of the access to care domain. 18

Descriptive analysis was performed for age, self-assigned gender, and self-assigned race/ethnicity, VUC returns, ED presentations, subsequent intubations, and death.

Comorbidities were abstracted and stratified by age. Data visualization, manipulation, and analysis were performed in Tableau.

We stratified revisits to VUC by 24-hour periods to coincide with return visit measures ("bouncebacks") previously published in emergency medicine literature. [27] [28] [29] These were evaluated in two ways. First, we determined outcomes on a per-visit basis. Any repeat visit by a patient must be staffed; therefore, this metric better reflects operational load requirements. We additionally evaluated revisits on a per-patient basis, as multiple visits by a small group of patients might skew results. We did not differentiate whether revisits were planned versus unplanned. We similarly stratified presentations to ED, as informed by quality frameworks, 17 by 24-hour periods on a per-visit and per-patient basis. We did not differentiate whether these represented progression of disease (after a reasonable trial of outpatient management) versus an errant decision.

We conducted several post hoc analyses. We compared age and comorbidities with VUC patients presenting from an identical time frame one year earlier (March 1, 2019 to April 20, 2019). We also compared age and comorbidities in VUC patients to those presenting to the ED who were discharged. Additionally, we evaluated a common telemedicine quality measure of antibiotic prescription rates for diagnoses of sinusitis and upper respiratory infection. 30 Mortality rates for the periods of March 1, 2020 to April 6, 2020 and April 7 to 20, 2020 were compared using Chi-square test with Yates correction for any 2 × 2 contingency tables. 31 A p value of less than 0.05 was considered significant. Decedent self-assigned gender and race/ethnicity were similarly evaluated. The Wilcoxon rank-sum test was used to compare median ages and number of comorbidities between the cohorts. 

Comorbidities present in patients are presented in Table 1 . We found 50.9% of patients had at least one pre-established comorbidity. This prevalence was greater than 74% for patients aged 60 years and older. We found a substantial portion of patients at risk due to elevated BMI.

Comorbidities on a per-visit basis are provided in Supplementary Table 1 . VUC patients had a median of one comorbidity (IQR, 0-2 comorbidities), which was the same as the VUC cohort one year prior (1; IQR, 0-2 comorbidities; p = 0.09). VUC patients had a higher median number of comorbidities than the ED discharged population (0; IQR, 0-2 comorbidities; p < 0.001).

Similar results were obtained when assessed on a per-visit basis (data not shown).

1,521 unique visits (6.8%) had at least one subsequent VUC encounter within 72 hours following their index visit (see Figure 1a for trend). While ED revisit benchmarks typically utilize 72-hour intervals, we were interested in trending VUC revisits for up to one week. The cumulative revisits up to 1 week, stratified by age, are presented in Figure 2 . We were unable to distinguish planned repeat VUC encounters from those spontaneously chosen by patients. The To provide context, we evaluated ED revisit rates for patients who had originally presented to the ED but were then discharged (21,618 visits in 19,321 total patients; 19,245 visits in 16, 888 adult patients) over the same period. The ED cumulative revisit rate for discharged ED patients on a per-visit basis was 4.4% at 72 hours (7.0% in adult patients), with an ED revisit with admission rate of 1.5% (1.7% of adults). Values for ED re-presentation on a per-patient basis were 4.6% and 1.6% (5.9% and 1.9% in adult patients). ED revisit and admission percentages through one week are provided on a per-visit basis in Figure 4 and per-patient basis in Supplementary Figure 6 . Following their VUC visit, a total of 37 individuals (0.17%) required intubation during a subsequent ED presentation and hospitalization (12 of which were in the ED). There were no significant differences in intubation rates between the periods (p = 0.49).

Overall mortality was 0.20% using visits as a denominator, and 0.25% per unique patient. All deaths after VUC visits appeared related to COVID-19 illness. All VUC decedents in the entire cohort had comorbidities, with the exception of two patients whose only risk factor was age (66 years and 76 years, respectively). Despite comprising only 38% of the visit population, male decedents (30, 0.44%) exceeded female decedents (15, 0.13%) (p < 0.001). Self-assigned decedents' race was as follows: American Indian or Alaska Native, 0; Asian, 1 (0.08%); Black or African American (0.55%), 12; Native Hawaiian or Other Pacific Islander, 0; White, 23 (0.25%); other race, 7 (0.29%); unknown, 2 (0.13%); patient refused, 0; not recorded, 0. These differences were significant (p = 0.03). The EHR insufficiently reflected ethnicity (null values in 84.3%, and unknown in an additional 2.4%) and statistical analysis was not undertaken.

During the period from March 1 to April 6, there were 37 deaths (0.21%). Of these deceased patients, 14 had been referred directly to the ED at their VUC encounter ( Figure 5a ); 17 of the remaining 23 would have met age callback criteria in period two. The ED referral rate from initial VUC visit was 5.7% during this period. The median time from VUC to ED presentation in these 37 deceased patients was 4 days (IQR, 1,7) . Twelve patients presented to the ED within 48 hours of their initial VUC visit; nine were referred. Of three who were not referred, two had no pulmonary complaints at all, and one had no discernable pulmonary symptoms at the time of visit. No patient deaths occurred within 48 hours of the initial VUC visit (Figure 5b ). The median time from VUC visit to death in period one was 13 days (IQR, 9, 20) .

In period two there were 4,991 visits and 8 deaths (0.16%). Mortality rate did not differ between the periods (p = 0.59). Four decedents had been referred on their initial VUC visit. Three patients presented to the ED within 48 hours of their initial VUC visit (all referred). Two deaths occurred within 48 hours; both patients had been referred to the ED. The overall ED referral rate from initial VUC visits was 7.6% during this period, which was statistically different from the prior period (5.7%) (p < 0.001). Time to ED presentation and death in this period are provided in Figures 5a and 5b.

By comparison, there were 103 deaths in patients who were seen in network EDs and had been discharged (overall mortality 0.5% per visit and 0.6% per patient). ED dispositions were discharged in 86%, discharged to a nursing or group facility in 8%, departure against medical advice in 4%, eloped in 1%, and left without being seen after triage in 1%. The cause of death in these patients was confirmed COVID-19 illness in 87 (85%); probable COVID-19 illness in 5 (5%); other causes in 7 (7%), including bowel perforation, complications of malignancy (2), malignancy and COPD, pulmonary embolism, drug overdose, and sepsis; and unknown/undetermined in 4 (4%).

A total of 517 patients met criteria for COVID-19 telephonic screening follow-up at 36-48 hours. 366 patients (71%) were successfully reached for telephonic follow-up. At the follow-up call, 24 patients (4.6%) were hospitalized; 13 (54%) of these were referred during their index VUC visit.

Among these hospitalized patients, there was one mortality, who was among those initially referred. Among the 11 remaining hospitalized patients not referred at the index VUC visit, there were no mortalities. At follow-up, 22 patients (4.3%) had a positive screening and were referred to the ED during the call. Of those referred during follow-up VUC screening, seven patients (33%) were captured as having a return ED visit. 71% of those who returned as instructed were admitted to the hospital. There were no mortalities among those with ED referral at 48-hour follow-up screening. Among 320 patients reached at 48-hour follow-up call who were not referred, one death occurred 13 days after VUC visit. The decedent was referred at the index visit, but in conjunction with her family, persistently declined hospital care. Finally, among 151 patients (29%) who were not reached at VUC telephonic screening, there were four deaths. Of these four decedents, one had been immediately referred during their index VUC visit and presented approximately 24 hours after the index encounter. One patient, co-managed with the oncology service, was referred by her oncologist at day 6 of illness. One patient presented to VUC for COPD, requesting a medication refill for chronic pulmonary disease, but denied active symptoms. The patient was not reached for a 48-hour follow-up screening and was noted to have expired at an outside health system 4 days after their index VUC visit. The final deceased patient requested only a work note at the index VUC visit and presented to the ED with COVID-19 complaints 4 days later. This patient expired 32 days after the VUC visit. Figure 6 summarizes these results.

Last, in evaluating antibiotic use in a respiratory pandemic, VUC antibiotic prescription rates for patients given a diagnosis of sinusitis or upper respiratory infection were 12.1% (13.7% in period 1 and 6.5% in period 2).

Contingency and crisis standards of care (CSC) are invoked when conventional care is no longer possible due to pervasive disaster and when demand surges overwhelm available resources. 32 The National Academies implored health care systems to do everything possible to avoid CSC in the COVID-19 pandemic. 33 As shifts from conventional to contingency and crisis standards occur along a continuum, recognition and mitigation of surge must occur as early as possible. Telehealth carries significant value in the pandemic setting, including decreased exposures to patients, providers, and those in transit; ongoing monitoring; education and treatment recommendations; and patient reassurance. Patients who were fearful of or outright refused to seek in-person care no longer required a physical visit. VUC removed barriers for providers, who could work virtually themselves from homes and offices and flexibly navigate between ED and VUC settings according to demand.

The VUC platform sustained access to care for thousands of patients. This immense workload was highlighted by the VUC's excess of the network's ED visits for 25 days. Our single institution VUC exceeded the pre-COVID volume seen by a North Carolina statewide VUC center by 30 times. 34 Continuous evaluation of "failures" (e.g., dropped visits) during the crisis improved situational awareness to devise solutions in a flexible platform (e.g., bringing on additional providers with nonconventional and unfixed shifts to meet demand or convert to alternative means of reaching patients in the face of technological disruption). Brief, minor technological disruptions were apparent. There did not appear to be a coincident increase in noshows after check-in during the periods of highest volume, which would have suggested patients becoming frustrated with provider delays. We have separately reported that patients welcomed telehealth as a health care delivery modality. They were satisfied with their VUC care (average score 4.4/5), would recommend it (4.6/5), and would utilize VUC services again (4.4/5). 16 In the midst of any crisis, operational quality and safety cannot be ignored. "Virtual perfection" 1 pronouncements must be pragmatically assessed. Along with the operational strain of a surge in patient volume, the VUC cohort included many patients who had multiple comorbidities. This presented a risk of demographic shift (i.e., from "typical" urgent care cohorts to urgent and chronic care, compounded by COVID-19). COVID-19 thus presented a catch-22 to vulnerable groups. Telemedicine services were provided to avoid hospital referrals 5 -with the intent to minimize exposures-and many patients refused referral. However, patients with risk factors for disease complications were precisely the patients most likely to need a physical and diagnostic assessment in complex decision-making scenarios to determine suitability for ongoing outpatient management. More than half of the overall cohort and 74% of patients aged older than 60 years had comorbidities. Interestingly, younger patients were more likely to have a VUC revisit, while older patients, as expected, had more frequent ED presentations.

These data extend to the outpatient setting previous findings of higher adverse outcomes in COVID-19, specifically mortality, in males 35, 36 and those of self-assigned black or African American race. 37, 38 VUC return visit and mortality rates compared favorably with those patients managed as outpatients after presenting to the ED. While the comparator group is imprecise, many of these ED patients similarly did not receive laboratory testing or radiographs, as these modalities were actively discouraged for patients not anticipated to require admission. Return visit causality may be difficult to determine, including anticipated and non-preventable progression of disease, an unrelated second diagnosis, a missed diagnosis, inadequate or inappropriate therapy of a known diagnosis, unanticipated progression of disease, inappropriate discharge instructions, and inappropriate follow-up. However they represent an operational load that must be re-triaged, evaluated, treated, and dispositioned.

The balance of responsibility of patients to seek follow-up versus the requirements of providers or health systems to actively follow them has been interminably debated, codified, regulated, and litigated. [39] [40] [41] [42] [43] We found the statement "it is the patient's responsibility … to return as advised for ongoing assessments of health, illness, and treatment outcomes" 44 to be problematic in the face of pandemic. Many patients outright refused in-person assessment, follow-up, and ED referrals, despite explicit instructions. Follow-up calls have been part of ED safe discharge strategies for more than 30 years. 45, 46 A 71% successful contact rate compares favorably with previous reports, [45] [46] [47] which is notable in crisis. While historically it has been difficult to demonstrate a definitive decrease in revisits and mortality, multiple studies report referral and aftercare interventions as a direct result of follow-up calls. 45, 46, 48 A 48-hour follow-up process targeting those aged 60 years and older to assess for length of illness, symptom progression, and associated risk factors correlated with a decreased mortality rate, although this did not achieve statistical significance. While a number of epidemiological confounders affect mortality, we did find a 48-hour follow-up process improved the safety of patients undergoing telehealth visits for suspected COVID-19 like illness, in that 4.3% of patients were recommended for inperson evaluation, of whom the majority were admitted. Opportunities for further refinement would include broadening enhanced callbacks to other high-risk groups. Planners, operational leadership, and payors should anticipate and support the full-time equivalents required to accomplish critical callback and follow-up functions performed by various provider levels.

This was a descriptive study of a single-network health care entity expanding VUC services during pandemic response to a public health emergency. Volume diverged from normal operations at peak by more than 25 times baseline and was uncontrolled. Definitive COVID-19 testing was largely unavailable and actively discouraged by public health authorities. Call type and provider level (previously board-certified emergency physicians and then expanded to emergency medicine physician assistants and nurse practitioners) did not mirror normal operations. VUC patients are not directly comparable with ED patients; various complex factors contribute to decision thresholds to access these various care modalities. However, our data provide a pragmatic spectrum of patients presenting during the COVID-19 pandemic, which can help hospitals plan for future surges. We provide comorbidity data, so that our patient cohorts could be placed in context with other telehealth programs. Based on our manual reviews of decedents, comorbidities were incompletely recorded in EHR problem lists and past medical history fields. This would suggest that that our percentages provide only a lower bound for extant comorbidities. There is no Emergency Severity Index (ESI), 49 such as that used for triage in EDs, to apply to VUC care. Measures of severity for use in virtual settings might be considered for future development, particularly in disaster. While our results may not be generalizable to all institutions, health care delivery structures, or disasters, they are reflective of a large-scale surge response in an academic medical network. We used surrogate markers to assess quality and safety, against a backdrop with no commonly agreed upon benchmarks in pandemic.

Return VUC visits and return ED visits were not differentiated as to whether or not they were planned or unplanned, whether they were for the same reason as the VUC visit, or whether they represented simple progression of disease after a reasonable trial of outpatient management.

Follow-up of patients was limited by the nature of multiple, independent health care systems in New York and the lack of a readily assessable national health care system database, which precluded us from completely evaluating visits to other EDs. However, both direct callbacks to patients and regional care functions within the EHR were utilized to mitigate this. Last, antibiotic prescription rates must be interpreted with caution, as outpatient treatment was largely empiric, weighted toward overtreatment to avoid social contact required of confirmatory diagnostics, and evolving.

Appropriately deployed and rapidly scaled VUC services appear to provide a pragmatic strategy to care for large numbers of patients during a pandemic or other surge scenarios. With this BMI, body mass index; CAD, coronary artery disease; CHF, congestive heart failure; CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; Dialysis, dialysis dependent; DM, diabetes mellitus; HLD, hyperlipidemia; HTN, hypertension. "no shows" (green), patient criteria unqualified (blue), technology/connection issue (red). ED, emergency department; VUC, virtual urgent care.

Cumulative revisits to virtual urgent care are shown on a per-visit basis. The outcomes captured during period two, where telephonic follow-up occurred in the virtual urgent care workflow for selected patient visits, are shown.

RJF provided administrative, technical, or material support. SWS and EJH drafted the manuscript, and all authors contributed substantially to its revision for important intellectual content

Virtually Perfect? Telemedicine for Covid-19

Telemedicine and the COVID-19 Pandemic, Lessons for the Future

COVID-19 and telemedicine: Immediate action required for maintaining healthcare providers well-being

The Role of Telemedicine to Control CoVID-19. Arch Clin Infect Dis

Exploring the Translation of the Results of Hurricane Sandy Research Grants into Policy and Operations: Proceedings of a Workshop-in Brief

The Current and Future Use of Telemedicine in Infectious Diseases Practice. Curr Infect Dis Rep

Use of telemedicine technologies in the management of infectious diseases: a review

The Current State Of Telehealth Evidence: A Rapid Review. Health affairs (Project Hope)

The role of telemedicine in fostering health-care innovations to address problems of access, specialty shortages and changing patient care needs

The effective use of telemedicine to save lives and maintain structure in a healthcare system: Current response to COVID-19

Role of Telehealth in the Management of COVID-19: Lessons Learned from Previous SARS, MERS, and Ebola Outbreaks

Removing regulatory barriers to telehealth before and after COVID-19

Telehealth use rises, but new trends highlight demographic divides. Healthcare IT News

Mastering the hype cycle. How to choose the right innovation at the right time

Assessing the Impact of a Rapidly Scaled Virtual Urgent Care in New York City During the COVID-19 Pandemic

VHA Office of Health Informatics: Connected Care/Telehealth. Connected Care/Telehealth, Telehealth Manual

Creating a Framework to Support Measure Development for Telehealth

NOT-OD-15-089. Racial and Ethnic Categories and Definitions for NIH Diversity Programs and for Other Reporting Purposes: National Institutes of Health

COVID-19 Updates for

Hospital Experiences Responding to the COVID-19 Pandemic: Results of a National Pulse Survey

Interim Guidance on Executive Orders 202.17 and 202.18 Requiring Face Coverings in Public During the COVID-19 Outbreak

Factors associated with hospitalization and critical illness among 4,103 patients with COVID-19 disease in New York City

Incidents of potential public health significance identified using national surveillance of US

High call volume at poison control centers: identification and implications for communication

Factors associated with short-term bounce-back admissions after emergency department discharge

Unscheduled return visits with and without admission post emergency department discharge

The prevalence of quality issues and adverse outcomes among 72-hour return admissions in the emergency department

Association Between Antibiotic Prescribing for Respiratory Tract Infections and Patient Satisfaction in Direct-to-Consumer Telemedicine

Contingency Tables Involving Small Numbers and the χ2 Test

Crisis standards of care: A toolkit for indicators and triggers

Rapid Expert Consultation on Crisis Standards of Care for the COVID-19 Pandemic

Interpreting COVID-19 and Virtual Care Trends: Cohort Study. JMIR Public Health Surveill

Gender Differences in Patients With COVID-19: Focus on Severity and Mortality. Frontiers in Public Health

Risk factors for disease severity, unimprovement, and mortality in COVID-19 patients in Wuhan, China

The color of coronavirus: COVID-19 deaths by race and ethnicity in the

Hospitalization Rates and Characteristics of Patients Hospitalized with Laboratory-Confirmed Coronavirus Disease 2019 -COVID-NET, 14 States

Limits on patient responsibility

Responsibility and the limits of patient choice

Patient Responsibilities. Code of Medical Ethics Opinion 1.1.42020

The Joint Commission. Specifications Manual for Joint Commission National Quality Measures (v2020A2). Discharges

Advisory Commission on Consumer Protection and Quality in the Health Care Industry

Consumer bill of rights and responsibilities: report to the President of the United States

Advisory Commission on Consumer Protection and Quality in the Health Care Industry

Patient Responsibility for Follow-Up of Diagnosis and Treatment

The effectiveness of an organized emergency department follow-up system

Efficacy of a telephone follow-up system in the emergency department

Development and implementation of an emergency department telephone follow-up system

Telephone Follow-Up for Older Adults Discharged to Home from the Emergency Department: A Pragmatic Randomized Controlled Trial

Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, Version 4. Implementation Handbook