key: cord-1013873-oxg2phlx authors: Holmgren, A Jay; Downing, N Lance; Tang, Mitchell; Sharp, Christopher; Longhurst, Christopher; Huckman, Robert S title: Assessing the impact of the COVID-19 pandemic on clinician ambulatory electronic health record use date: 2021-12-09 journal: J Am Med Inform Assoc DOI: 10.1093/jamia/ocab268 sha: 27be53071ca52b6d942ce8f0e3199bf8e309d0fe doc_id: 1013873 cord_uid: oxg2phlx OBJECTIVE: The COVID-19 pandemic changed clinician electronic health record (EHR) work in a multitude of ways. To evaluate how, we measure ambulatory clinician EHR use in the United States throughout the COVID-19 pandemic. MATERIALS AND METHODS: We use EHR meta-data from ambulatory care clinicians in 366 health systems using the Epic EHR system in the United States from December 2019 to December 2020. We used descriptive statistics for clinician EHR use including active-use time across clinical activities, time after-hours, and messages received. Multivariable regression to evaluate total and after-hours EHR work adjusting for daily volume and organizational characteristics, and to evaluate the association between messages and EHR time. RESULTS: Clinician time spent in the EHR per day dropped at the onset of the pandemic but had recovered to higher than prepandemic levels by July 2020. Time spent actively working in the EHR after-hours showed similar trends. These differences persisted in multivariable models. In-Basket messages received increased compared with prepandemic levels, with the largest increase coming from messages from patients, which increased to 157% of the prepandemic average. Each additional patient message was associated with a 2.32-min increase in EHR time per day (P < .001). DISCUSSION: Clinicians spent more total and after-hours time in the EHR in the latter half of 2020 compared with the prepandemic period. This was partially driven by increased time in Clinical Review and In-Basket messaging. CONCLUSIONS: Reimbursement models and workflows for the post-COVID era should account for these demands on clinician time that occur outside the traditional visit. The COVID-19 pandemic triggered sudden and dramatic changes in many aspects of the health care delivery system, including clinician workflows and use of electronic health record (EHR) systems. In a matter of weeks, health care organizations shifted a substantial portion of their ambulatory care delivery to phone-and video-based telemedicine encounters and saw large reductions in emergency and hospital care. [1] [2] [3] [4] [5] Clinicians still delivering face-to-face care were forced to restructure many aspects of their practice to ensure the safety of patients and staff. The pandemic-driven shifts toward virtual treatment, and the corresponding change in patient expectations and awareness of communication tools such as secure messaging via the EHR, have substantially altered the nature of ambulatory care. Further, as telehealth and the pandemic incentivized patients to become familiar with the use of online portals to access their health information and connect with clinicians, the time required for those clinicians to manage the care of their patients through the EHR increased. The long-term shift to a mix of face-to-face and virtual care, as ambulatory patient volume returned in the second half of 2020, 2 may have exacerbated these issues as clinicians delivered care across multiple modalities in a single day. Given that many of these changes, such as increased patient familiarity with asynchronous messaging, may persist beyond the COVID-19 pandemic, it is critical to evaluate how they have impacted clinician work. Although near-universal adoption of EHRs was an important enabler of the shift to telemedicine, [6] [7] [8] increased reliance on EHRs may exacerbate existing issues of low job satisfaction and poor wellbeing amongst clinicians. Many clinicians spend a significant amount of time working in the EHR, 9 and clinicians in the United States already face a greater EHR burden than their international peers. 10, 11 Total time spent working in the EHR, "after-hours" time working outside of scheduled clinic hours, and responding to In-Basket messages without protected time or reimbursement for messages have been associated with a variety of negative impacts, including burnout, 12-15 which can translate into higher costs and lower-quality care. [16] [17] [18] Further, if the increased EHR burden is driven by activities such as messaging with patients, these new demands on clinician time may be concentrated on tasks that are outside of the bounds of the traditional "visit" and are currently nonreimbursable for most clinicians. 19 To address this topic, we used national, longitudinal EHR metadata to examine three research questions. First, how did ambulatory clinician EHR use change in response to the COVID-19 pandemic? Second, what changes were observed across different EHR functions? Finally, what specific EHR interactions are responsible for changes in EHR active-use time? Our findings have important implications for understanding ongoing changes in clinical work. The sample for our longitudinal, observational study consisted of deidentified EHR metadata from 366 ambulatory care health systems in the United States that use Epic as their EHR software vendor. Epic has the largest ambulatory care market share in the United States with 33.4% of the market, and the sample represents nearly all of Epic's ambulatory customer base. 20 Health systems that adopted Epic during the study period, or who switched from Epic to another EHR vendor during the study period, were excluded from the analysis. The sample included physicians and advanced practice practitioners (eg, physician assistants and nurse practitioners). In the data, each organization represents an EHR installation, which may include more than one facility, such as a hospital with multiple ambulatory clinics, multiple hospital systems using a single EHR instance, or a chain of ambulatory clinics. Data were aggregated at the organization level, and all measures were calculated weekly. Not all organizations reported data in each week, and our final analytic sample is an unbalanced panel from December 29, 2019 through January 2, 2021 with 19 398 observations at the organization-week level. We conducted robustness tests for all analyses using a balanced panel of the 245 organizations that reported data in every week. This study using deidentified data aggregated at the health system level was deemed exempt from human subjects research by the Institutional Review Board at Stanford University. Clinician EHR activity was monitored by the software, which collects metadata extracted via Epic System's Signal data aggregation tool. These metadata track activity including mouse movement, keystrokes, clicking, scrolling, and any other interaction with the EHR. EHR time is defined in Signal as the time the user was performing active tasks in the software, and the system stops counting if no interactions are conducted after 5 seconds. Although this is a conservative measurement of EHR time, it allows our study to focus on active EHR work while excluding time where the EHR may be open that the clinician is performing non-EHR tasks. However, our data is unable to capture clinician work that is generated by the EHR but not conducted through it, such as a phone call with a patient that is initiated by receiving an In-Basket message. Our measures should be interpreted with this context in mind. We measured EHR active-use time across four different functions. 9,21 These functions were Clinical Review (eg, time spent reviewing patient history or test results), Notes (eg, time spent documenting), Orders (eg, time spent inputting orders, performing diagnostic association for orders), and In-Basket messaging (eg, time spent reading or writing messages or otherwise managing the In-Basket). We defined after-hours time as any time between 5:30 PM and 7:00 AM local time on weekdays and any time on weekends, unless the clinician was scheduled during those times. After-hours activeuse time in the EHR per day was measured as time spent in the EHR on clinical work, excluding tasks related to research, data analysis, customization, or performance measurement. This measure was developed by Epic to measure after-hours time and is broadly consistent with the literature on after-hours EHR work. 9, 12 We measured mean In-Basket messages received per clinician per day a user logged in to the system, in total and by source. In-Basket messages in the Epic EHR can be generated from seven possible sources: team, results, prescription, patient, system, custom, and other. These categories were defined to be consistent with previous literature on clinician messaging. 15 We measured both counts of messages received, as well as message volumes normalized to each organization's prepandemic baseline-an 11-week period from December 29, 2019, to March 14, 2020. We measured mean ambulatory encounter volume per clinician per day for each health system in each week of the study using EHR metadata from the Signal tool to capture our measure of daily volume. Finally, we included a measure of national confirmed COVID-19 case counts from the Centers for Disease Control COVID Data Tracker 22 to provide context regarding the state of the pandemic in each week. We calculated descriptive statistics for our two primary measures of EHR active-use time-total EHR time per day and after-hours time per day per clinician-for each week in the study period, comparing those figures to national COVID-19 case counts and daily encounter volume. To estimate the change in EHR active-use time and after-hours time after the onset of the pandemic, while controlling for daily encounter volume per clinician in each week as well as unobserved organization-level variation, we used ordinary least squares regression models in an event study design. We used the date of implementation of the nation's first shelter-in-place order-in the state of California on March 19-as our proxy for the onset of the pandemic, with the week beginning (March 1, 2020) as our baseline prepandemic week. Our models included controls for encounter volume per clinician per day during the current week, organization-level fixed effects to control for any timeinvariant organizational characteristics, and robust standard errors clustered at the organization level to account for serial autocorrelation over time. We calculated descriptive statistics for EHR active-use time per day in each of our four functions: In-Basket, Clinical Review, Orders, and Notes. We calculated descriptive statistics for mean In-Basket messages received per day, in total as well as by source, for each week. To facilitate interpretation, we normalized message volume by each organization's prepandemic message volume during the 11-week period from December 29, 2019, to March 14, 2020. To estimate the impact of messages by type on clinician EHR time, we ran an ordinary least squares regression model, with clinician time spent in In-Basket per day as the dependent variable, and number of messages of each type (system, team, results, prescription, patient, custom, and other) as the independent variables of interest. The model also included organization fixed effects and robust standard errors clustered at the organization level. We conducted a robustness test including week fixed effects to control for any secular changes during the pandemic. Finally, for all analyses, we conducted robustness tests with a balanced panel including only health systems that reported data in every week. Our study has limitations. First, our dataset only includes scheduled ambulatory appointments with physicians and advanced practice practitioners. Due to data limitations, we were unable to evaluate inpatient or emergency settings, nor were we able to consider the impact of the pandemic on other clinicians, such as registered nurse (RN) or medical assistants, who are critical members of care teams and may face significant pandemic-driven EHR burdens. Second, our measure of EHR active-use time is conservative and narrowly defined, counting only the time clinicians spent actively using the EHR. Our study may therefore underestimate the impact of COVID on clinician work, as time spent helping patients connect to telemedicine calls and reading patient histories without clicking in the EHR for 5 s was not captured in our measures of active-use time. We also lack data on clinician scheduling, making it impossible to normalize to 8 h of scheduled clinician time as some previous studies have been able to do. [23] [24] [25] However, our data are internally consistent over time, and the main insights of our study focus on relative changes over time rather than absolute measures of EHR time. To provide clinical context, we have also expressed changes over time as percent changes relative to the prepandemic baseline where possible. Third, our data cannot differen-tiate between EHR time for telemedicine and virtual care compared with EHR time during face-to-face, in-person visits. Our results therefore estimate the impact of the COVID-19 pandemic in its entirety, rather than simply through the shift to telemedicine. Fourth, we use data with proprietary metrics from a single EHR vendor, whereas Epic Systems is the largest ambulatory care vendor in the United States, clinicians and health systems using other EHR vendors may have different experiences. Finally, we do not have data on patient phone calls and are unable to assess changes in this modality of patient-clinician communication relative to secure messaging or scheduled encounters. Mean clinician EHR time per day was relatively stable from December 29, 2019, to March 14, 2020, with weekly means ranging between 80.4 and 85.9 min/day and a mean of 84.3 min across weeks ( Figure 1 ). After-hours time per day was similarly stable during this time period, with weekly means between 19.2 and 20.6 min/day and a mean of 19.4 min across weeks. Both EHR time per day and afterhours time per day dropped significantly during the onset of the pandemic, from March 15, 2020 through July 4, 2020. In this period, EHR time per day dropped to a low of 64.9 min (23.01% decrease) in the week beginning March 29, 2020, before recovering to 85.7 min (1.67% increase compared with prepandemic mean) during the week of June 28, 2020. Similarly, after-hours time per day reached a minimum of 12.1 min during the week of April 5, 2020, and increased to 20.5 min during the week of June 28, 2020. From July 5, 2020, onward, total EHR time per day was slightly higher than in the pre-COVID period, with a mean of 86.4 min (2.49% increase relative to prepandemic mean) across those weeks, while after-hours time saw a smaller increase to a mean of 19.8 min (2.06% increase relative to prepandemic mean) in this period. We found similar results in our event study models adjusting for daily encounter volume and time-invariant organizational characteristics ( Figure 2 ). When compared with our baseline week (March 1, 2020), by the week of July 5, 2020, mean EHR time per day increased by 3.9 min (95% confidence interval [CI]: 2.98-4.85 min), and remained at a similar level (excepting holidays, annotated in Supplementary Appendix Exhibit SA1) through the remainder of the year. After-hours time per day followed a similar trajectory, with an increase of 1.55 min over the prepandemic baseline by the week of June 28, 2020 (95% CI: 1.19-1.92 min). Clinician EHR time per day across functions varied significantly during the onset of the COVID-19 pandemic (Figure 3 ). Time in In-Basket per day increased from 11.21 min during the week of March 1, 2020, to 12.68 min (13.11% increase) in the week of March 15, 2020, decreased back to the prepandemic baseline, and then increased again and stayed higher than prepandemic levels throughout 2020. Time in Clinical Review per day decreased significantly at the onset of the pandemic, then increased to higher levels, with average time from December 29, 2019, through March 14, 2020, of 17.17 min, compared with 18.37 min (7.00% increase) from July 5, 2020, onward. Time in Orders and Notes per day both saw similar decreases in the early days of the pandemic and recovered to their prepandemic levels by July 5, 2020. . Caption: total electronic health record time and after-hours time, controlling for volume and organization. Notes: Graphs are event study plots controlling for organization fixed effects and daily volume. All point estimates are relative to the week, 2-weeks prior to the first state-wide shelter in place (SIP) order in California, our proxy for the onset of the pandemic. Gray regions represents 95% confidence intervals with standard errors clustered at the organization level. Overall In-Basket messages received per day increased slightly compared with the prepandemic baseline, with clinicians receiving 4% more messages the week of July 5, 2020, compared with the 11week prepandemic average (Figure 4) . The greatest increase was in messages from patients, with clinicians receiving 157% of their prepandemic average per day; that increased level of messages remained consistent through the end of the year. Team and prescription messages also remained at higher levels (relative to baseline) during the postpandemic period after July 5, 2020, whereas results messages remained below the prepandemic average after July 5, 2020. Other message types saw a similar pattern to the results messages, and raw message counts were also similar (Supplementary Appendix Exhibit SA2). Our multivariate fixed effects regressions suggest that each patient message increased In-Basket time per day by 2.32 min (P < .001, 95% CI: 2.16-2.48). Each Results message also significantly increased In-Basket time per day by 0.24 min (P < .001, 95% CI: 0.20-0.28). No other message type was associated with a statistically significant increase in In-Basket time per day (Table 1) . We found similar results in our robustness tests including week fixed effects to control for secular time trends and seasonality (Supplementary Appendix Exhibit SA3). The COVID-19 pandemic resulted in a large and immediate shock to clinical work, as reflected in our EHR metadata. Our national, longitudinal study of ambulatory clinician EHR work throughout 2020 illustrates that time spent in the EHR and after-hours EHR time per day dropped dramatically at the onset of the pandemic, but recovered by July 2020 to a level consistently higher than the prepandemic baseline. These results held after adjusting for daily encounter volume and organizational-level characteristics, suggesting that delivering care in the COVID era required more EHR work by clinicians compared with the pre-COVID baseline. This increased EHR time was driven by two functions: Clinical Review, where clinicians view test results and patient history, and In-Basket, where clinicians send, receive, and manage messages from a variety of sources. In-Basket messages received by clinicians increased dramatically during the pandemic, including receiving 157% of their baseline prepandemic messages from patients per day. Those patient messages were a significant driver of clinician work, with each message per day resulting in an additional 2.32 min of EHR active-use time. These results provide insight into the short-and medium-term impact of the COVID-19 pandemic on clinician work. Furthermore, they have important implications for designing policy and payment systems for a future that includes greater use of telemedicine and patients who may be more willing to communicate with their clinicians through secure messages. The fact that clinicians spent more time working in the EHR during the COVID-19 pandemic is unsurprising but still concerning. An obvious potential driver of this finding is that many clinicians delivered care virtually via telemedicine, including both phone and video visits. The rapid proliferation of virtual visits may have had implications not only for how clinicians deliver care during the encounter but also how patients expect to interact with their clinician outside of the boundaries of the scheduled appointment. Telemedicine may demand more EHR time, especially in EHR work outside of the tra- ditional "visit" as clinicians rely more on EHR-based Clinical Review and receive more In-Basket messages. Team-based care, a promising mechanism to improve quality in the prepandemic era, 26 suddenly required more EHR work as distributed teams had to communicate through In-Basket messaging. Clinicians may have even served as impromptu technical support for their patients trying to navigate video conferencing applications, which required troubleshooting audio and video issues before appointments could begin. These problems may have been compounded as clinicians delivered both virtual and face-to-face care, which are visit modalities with very different structures and demands on clinician time. 27 One surprising aspect of our results was that, after the immediate post-COVID acclimation period, COVID case counts appeared to have little impact on either encounter volume or any of our EHR use measures. This speaks to the resilience and adaptability of the health care system, which managed to continue delivering outpatient care both virtually and in-person after a period of adjustment. We found Clinical Review and In-Basket to be the main drivers of increased EHR work. For Clinical Review, it may be that telemedicine encounters led to an increased reliance on previous documentation and viewing of test results in the EHR, or the increase in messages required clinicians to go back and review their own documentation before responding to patients. 27 The increase in messages received across a variety of sources, resulting in a significant increase in In-Basket time, offers an important clue as to how the pandemic has changed clinical work. The aforementioned increase in messages from other care team members may reflect a new reliance on asynchronous communication as teams had to work in a distributed manner-a phenomenon observed across multiple industries in the COVID era. 28 The increase in prescription messages-likely refill requests-was sharpest at the onset of the pandemic as patients may have rushed to ensure they had sufficient medication supply in the face of an uncertain future, and decreased closer to prepandemic levels as the year progressed. The most striking and durable increase, however, was in patient messages, with clinicians receiving up to 157% of their prepandemic volume of patient messages. Increased patient messaging may have been due to a variety of factors. Initially, patients may have substituted messaging for in-person visits due to fear of COVID exposure. The proliferation of telemedicine may have encouraged more patient portal sign-up and use 29 -both of which had historically been low. 30 Once patients became familiar with portals, they likely became more willing to message with clinicians. Given that the increase in patient messages was relatively stable through the end of 2020-even after telemedicine volume declined and in-person visits increased nationally 2,31 -we suspect that the latter explanation is a more-prominent driver of increased messaging and that the higher message volume may persist in the post-COVID era. Our results have important implications for policymakers as the US health care system prepares for the postpandemic period. Though calls for continued use of telemedicine have grown, 32 finding the right reimbursement structure may be challenging, with many open questions such as whether phone visits should be reimbursed at the same level as video visits 33 and how virtual care should be integrated into clinician workflows. 27 Beyond the first-order impact of virtual care, the expansion of telemedicine may have resulted in an increased awareness among patients of the ability to communicate with clinicians through secure messaging. The distribution of that EHR work across types of activity, however, suggests that the additional demands on clinician time may prove challenging to integrate with existing fee-for-service payment systems while changing patient expectations regarding how they communicate with their clinicians. Time reviewing patient records and messaging with patients often takes place outside of a traditional visit and is rarely a service reimbursed by payers or health systems. 34 Furthermore, many ongoing efforts to address EHR burden are focused on documentation work, such as the recent CMS changes to ambulatory E/ M documentation codes, and are unlikely to address this new form of EHR work driven by patient messaging even if they are successful in reducing EHR documentation time. To the extent that the increase in patient messages is durable, clinicians will need to allocate more of their time to answering those messages, even after the pandemic has subsided. This is especially important given recent evidence identifying In-Basket messages in particular as a predictor of physician burnout, as well as other studies finding a significant increase in patient messages received by clinicians during the pandemic. 12, 15, 35 Policymakers and health system leaders should keep these new demands on clinician time in mind as they develop future reimbursement models and workflows, taking care not to exacerbate EHR-driven clinician burnout. 18, 36 CONCLUSION In our national study, we found that EHR active use time fell in the immediate onset of the COVID-19 pandemic, but by July 2020 clinicians spent more time in the EHR and more EHR time after-hours per day. This increase was driven by time spent in Clinical Review and In-Basket messaging, with clinicians receiving 157% of their prepandemic baseline messages from patients. These patient messages represented a significant demand on clinician time, with each requiring an additional 2.3 min of daily EHR work on average. Policymakers and health system leaders looking to create sustainable workflows incorporating telemedicine in the post-pandemic period should be aware of these new demands on clinician time not only to avoid clinician burnout but also to accommodate rethinking the model of ambulatory medicine as patient expectations for care expand beyond the scope of the traditional face-to-face visit. AJH and NLD conceived the study, acquired the data, designed the analysis plan, drafted the manuscript, and participated in the critical revisions process. MT performed statistical analysis and helped draft the manuscript. CS, CL, and RH designed the analysis plan, drafted the manuscript, and participated in the critical revisions process. Supplementary material is available at Journal of the American Medical Informatics Association online. Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US Telehealth: Fad or the Future. Epic Health Research Network Telehealth transformation: COVID-19 and the rise of virtual care Fueled By Coronavirus Fears And Shift In Payment Rules. Kaiser Health News Fewer hospitalizations for acute cardiovascular conditions during the COVID-19 pandemic Electronic health record adoption in US hospitals: the emergence of a digital "advanced use" divide Virtually Perfect? Telemedicine for Covid-19 COVID-19 transforms health care through telemedicine: evidence from the field Physician time spent using the electronic health record during outpatient encounters: a descriptive study Assessment of electronic health record use between US and non-US health systems Physician burnout in the electronic health record era: are we ignoring the real cause? Electronic health records and burnout: Time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians Physician stress and burnout: the impact of health information technology Are specific elements of electronic health record use associated with clinician burnout more than others? Physicians' well-being linked to In-Basket messages generated by algorithms in electronic health records Estimating the attributable cost of physician burnout in the United States Physician burnout: contributors, consequences and solutions Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between Growth of secure messaging through a patient portal as a form of outpatient interaction across clinical specialties Top 10 Ambulatory EHR Vendors by 2019 Market Share. Definitive Healthcare Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine CDC COVID Data Tracker Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis Implementing measurement science for electronic health record use Metrics for assessing physician activity using electronic health record log data Provider Teams Outperform Solo Providers In Managing Chronic Diseases And Could Improve The Value Of Care: Study examines care management and biomarker outcomes after the onset of three chronic diseases differed both by team-based versus solo care and by physician versus non-physician A qualitative study of primary care physicians' experiences with telemedicine during COVID-19 Collaborating during coronavirus: the impact of COVID-19 on the nature of work (No. w27612) Rapid development of telehealth capabilities within pediatric patient portal infrastructure for COVID-19 care: barriers, solutions, results Are patients electronically accessing their medical records? evidence from national hospital data Trends in the use of telehealth during the emergence of the COVID-19 pandemic -United States Telemedicine and the COVID-19 pandemic, lessons for the future The Doctor Will Call Me Maybe: The Uncertain Future Of Audio-Only Visits And Why We Need Them To Address Disparities j Health Affairs Blog Patient-to-physician messaging: volume nearly tripled as more patients joined system, but per capita rate plateaued Trends in electronic health record inbox messaging during the COVID-19 pandemic in an ambulatory practice network in New England Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction Data are can be requested in de-identified format from Epic Systems.