key: cord-0694257-mn9p1ezi
authors: Kawamoto, Kensaku; Kukhareva, Polina V; Weir, Charlene; Flynn, Michael C; Nanjo, Claude J; Martin, Douglas K; Warner, Phillip B; Shields, David E; Rodriguez-Loya, Salvador; Bradshaw, Richard L; Cornia, Ryan C; Reese, Thomas J; Kramer, Heidi S; Taft, Teresa; Curran, Rebecca L; Morgan, Keaton L; Borbolla, Damian; Hightower, Maia; Turnbull, William J; Strong, Michael B; Chapman, Wendy W; Gregory, Travis; Stipelman, Carole H; Shakib, Julie H; Hess, Rachel; Boltax, Jonathan P; Habboushe, Joseph P; Sakaguchi, Farrant; Turner, Kyle M; Narus, Scott P; Tarumi, Shinji; Takeuchi, Wataru; Ban, Hideyuki; Wetter, David W; Lam, Cho; Caverly, Tanner J; Fagerlin, Angela; Norlin, Chuck; Malone, Daniel C; Kaphingst, Kimberly A; Kohlmann, Wendy K; Brooke, Benjamin S; Del Fiol, Guilherme
title: Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center
date: 2021-07-31
journal: JAMIA Open
DOI: 10.1093/jamiaopen/ooab041
sha: 9c2e55e6233e79579decec1f5bf1aee48256f209
doc_id: 694257
cord_uid: mn9p1ezi

OBJECTIVE: To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations. MATERIALS AND METHODS: We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations. RESULTS: University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape. DISCUSSION: Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation. CONCLUSION: EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Given the information-intensive nature of clinical medicine, it was long hoped that electronic health record (EHR) systems could help optimize health and health care. 1 Indeed, it was anticipated that EHRs could help address key challenges such as patient deaths due to preventable medical errors, 2 patients receiving as little as half of evidence-based recommended care, 3, 4 and medical costs increasing at unsustainable rates. 5 To realize the promise of the EHR, many countries have invested heavily in EHR adoption. 6 Following over $30 billion of government investment, 7 the United States has achieved near-universal EHR adoption. 8 Despite such heavy investments, EHRs have often failed to achieve their promise. 9 While customization and training can improve the EHR user experience, 10, 11 physicians spend as much as 2-5 h on the EHR for every hour spent in direct patient care, 12, 13 and they can go through 4000 clicks per shift in the emergency department. 14 In a recent study, 80% of physicians experienced physiological fatigue within the first 22 min of EHR use. 15 Physicians often rate EHR usability as "unacceptable," 16, 17 with low EHR usability contributing to stress and burnout. 9,18 EHR adoption has not been associated with improved outpatient care quality in the United States. 19, 20 Clearly, the promise of EHRs remains unfulfilled.

In seeking to realize the promise of EHRs, an exciting development is the evolution of EHRs from single-vendor products to vi-brant platforms enhanced through third-party digital innovations ( Figure 1 ). Just as smartphone users can augment their phones via apps, healthcare systems can increasingly augment their EHRs through digital innovations downloaded through EHR app stores. 21, 22 A key enabler for this innovation ecosystem is an interoperability standard known as SMART on FHIR (pronounced "smart on fire" and an acronym for Substitutable Medical Applications, Reusable Technologies on Fast Healthcare Interoperability Resources). 23 SMART on FHIR allows digital innovations to be seamlessly integrated with the EHR's native user interface through a single sign-on mechanism while reading and writing EHR data through standard application programming interfaces (APIs).

SMART on FHIR innovations can help meet clinician needs inadequately supported by the native EHR. Such needs can include support for high-level clinical reasoning, communicating and coordinating care, and complying with rules and regulations. 24 Importantly, these innovations allow for rapid continuous improvement often not possible in the EHR itself. Because they are shareable via EHR app stores, duplication of effort can be reduced; and as they are substitutable with similar innovations, healthy competition can be promoted.

SMART on FHIR adoption is increasing, with use in over 500 hospitals 25 and inclusion in federal regulations. 26 Several healthcare systems have incorporated SMART on FHIR into their digital inno-

Just as smartphones can be improved through downloaded apps, the electronic health record (EHR) can now be enhanced through apps developed by third-party innovators in digital health. Recognizing the promise of this approach early on, clinicians and scientists at the University of Utah launched an initiative called ReImagine EHR in 2016 to develop healthcare apps that enhance the EHR and help clinicians take better care of their patients. Since then, the ReImagine EHR team has developed more than 10 apps that follow a consistent vision: to create a seamless experience of use within the EHR, to provide clinicians with all the relevant information in one place, and to ensure recommendations are individualized and data-driven. We have developed apps supporting conditions ranging from diabetes to cancer to COVID-19. Some apps support shared decision-making between the patient and the clinician, including by providing personalized predictions on what treatment is likely to work best for a given patient. These apps can improve the care patients receive while saving clinicians time. In this article, the authors describe their journey and the lessons learned, with the hopes of helping others join them in the quest to improve patient lives through digital health innovation.

vation strategy. [27] [28] [29] [30] [31] Evidence is emerging that such digital innovations can improve clinical care. 27, 32, 33 For example, a SMART on FHIR app for neonatal bilirubin management implemented at University of Utah Health (UUH) reduced the provider time required by two-thirds and was associated with significant improvements in guideline-compliant phototherapy. 27 Knowledge sharing and coordination could enable more rapid progress in the adoption and impact of FHIR-based digital health innovations. However, beyond descriptions of individual innovations, there is a lack of literature on the larger initiatives producing these innovations and the lessons learned. To address this gap in the literature, we describe here the UUH ReImagine EHR initiative. UUH was an early innovator in the development, implementation, and evaluation of digital innovations leveraging SMART on FHIR and related interoperability standards. By describing the initiative and lessons learned, we hope to catalyze a more rapid transformation of health care through this promising approach to digital medicine.

UUH is an academic health system with 5 hospitals and 12 community clinic centers. UUH has used the Epic V R EHR system-wide since 2014.

Presented as a case study, we describe an enterprise initiative for EHR optimization at an academic health system leveraging SMART on FHIR and related interoperability standards. Key lessons learned are highlighted in the discussion. All human subjects research was approved by the Institutional Review Board (IRB).

In the methods, we describe our motivation, facilitators, mission, vision, governance, structure, strategy, projects phases, and funding. In the results, we describe the developed infrastructure, individual innovations, and return on investment. In the discussion, we describe lessons learned.

The primary motivation for launching ReImagine EHR was growing recognition that traditional EHR optimization methods were insufficient for fully meeting patient and provider needs. We also sought to better coordinate emerging efforts spanning both research and operations to maximize synergies. For example, research innovations must be translated into clinical practice to impact care. Similarly, difficult operational challenges can be addressed through research. We also aimed to increase our technical capacity to adapt to a rapidly changing healthcare landscape. Facilitators included the local availability of relevant experts as well as the increasing adoption of underlying standards by EHR systems including Epic V R . 34 A key catalyst for launching the initiative was the visionary leadership and resourcing support provided by health system leaders including the Chief Medical Information Officer (CMIO) and Chief Information Officer (CIO). This initial investment sparked substantial multidisciplinary research led by ReImagine EHR faculty in collaboration with investigators across the university.

The mission of ReImagine EHR is to improve patient care and the provider experience. The focus is EHR-integrated digital innovations capable of widespread dissemination across diverse health systems and EHR platforms. Table 1 outlines our vision. ReImagine EHR is a multi-stakeholder collaborative effort. The steering committee is co-chaired by the CMIO and CIO, and includes the Associate CMIO, the chair of the Department of Biomedical Informatics, the Director of Clinical Information Systems, the Director of Nursing Informatics, and the Chief Technology Officer. Key stakeholders are regularly consulted. ReImagine EHR is directed by the Associate CMIO.

The ReImagine EHR team includes 10 core members reporting to the director; other members contributing to many or most projects; and project-specific collaborators. The team includes clinical informaticists, physician leaders, cognitive psychologists, evaluation experts, and operational information technology leaders.

Core elements of the ReImagine EHR strategy include (1) establishing a world-class team; (2) creating supportive infrastructure; (3) focusing on the most impactful, sustainable, desirable, and feasible projects; (4) following best practices; and (5) maximizing synergies. Table 2 describes our strategy and associated tactics, rationale, and considerations. The core elements of our strategy were proposed by the director of the initiative (KK) and adopted by the steering committee at its first meeting. Since then, the strategy has been iteratively refined, with increased emphasis on evaluation and the pursuit of grant-funded research.

Projects typically consist of 5 iterative and overlapping phases: Exploration; Design; Development; Implementation; and Sustainment and Dissemination. These phases are aligned with the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation science framework, 35 with the EPIS Preparation phase encompassing our Design and Development phases.

Exploration consists of inception, prioritization, and resourcing. The ReImagine EHR Steering Committee prioritizes projects and allocates operational resources based on the principles outlined in Table 2 . This exploration phase includes a "build vs. buy" analysis, in which we evaluate whether we have already purchased the needed capabilities, such as EHR modules that can be configured to meet user needs. When existing capabilities are insufficient, we evaluate third-party products and proceed to solution design and development only when the best course of action is to build the required functionality ourselves.

Design involves proposing a technology-facilitated solution to address user needs. ReImagine EHR uses an iterative, user-centered design process. Functional requirements are defined early based on an understanding of the nature of the clinical problem to be addressed. Both formal methods (eg, critical incident interviews, user workflow analyses, eye-tracking, card sorting, observation, heuristic evaluation, simulation studies with think-aloud, and/or focus groups) and informal methods (eg, stakeholder meetings, feedback from experts) may be used to assess user needs. [36] [37] [38] [39] Based on these needs, technical requirements are specified. User interface design is iteratively refined based on feedback from end users and socio-technical experts. The design is updated well into the implementation phase based on emerging user needs.

Development is conducted using the agile software development model. A typical "sprint" to develop additional functionality lasts 2 weeks, and all software updates undergo comprehensive testing prior to clinical deployment. Interoperability standards used include the Health Level Seven International (HL7) SMART standard for integrating the user interface, the HL7 US Core FHIR standard for data integration, 40 and the HL7 Clinical Decision Support (CDS) Hooks standard for integrating point-of-care alerts and reminders. 41 This phase is completed when testing is complete, governance approvals have been obtained, and an implementation plan has been developed.

Implementation involves operational use, refinement, and evaluation at initial clinical implementation sites. Typically, the applica-tion is first rolled out to pilot users to identify and address any major issues. A wider rollout then follows, with the system monitored and maintained to ensure proper functioning and updates being implemented as needed. The use, usability, efficiency, effectiveness, and impact of the system are formally or informally evaluated.

The Sustainment and Dissemination phase involves providing long-term support for innovations and extending reach and impact beyond the initial healthcare system. Most ReImagine EHR innovations are designed and developed with a goal of such dissemination. Dissemination may be conducted through EHR app stores or via institutional partnerships.

ReImagine EHR is supported by both our health system and external funding. There are significant synergies between operations and research: innovations can be initially motivated by scientific questions and funded through research grants, then later evolve to support clinical operations. Likewise, innovations can be initially developed for clinical operations then lead to research projects, for example to further enhance a tool or to evaluate it through a multisite trial. Regardless of the initial funding source, individual innovations must demonstrate sufficient clinical value to justify long-term operational support and maintenance. For projects supported by external funding, it is important to demonstrate clinical value during the project period if ongoing operational support is desired.

ReImagine EHR was launched at UUH in January 2016. Below we describe enabling infrastructure developed through the initiative, 5 representative innovations, and return on investment to date.

We have encountered and resolved several challenges, many of which are technical in nature. Table 3 describes these challenges and the infrastructure developed to address these challenges. Additional approaches used and relevant considerations are also described. We are currently exploring potential approaches for making these infrastructure tools available outside the ReImagine EHR initiative.

This supportive infrastructure has been iteratively developed to address challenges encountered in the pursuit of individual innovations, including those described in the section "Representative Innovations." Some of these challenges impact most projects, including the need for value set specification, the need for EHR data mapping, and the need for rigorous testing and evaluation. Other challenges affect only a subset of projects, such as native EHR FHIR APIs lacking support for desired interfaces or transmitting unnecessary data to third-party applications hosted outside the enterprise firewall. 

ReImagine EHR has engaged in the design, development, and implementation of over 10 applications to date. To illustrate how patient care and the provider experience across the care continuum can be improved through such innovations, 5 representative innovations are described below in terms of the problem addressed, the solution developed, the outcomes to date, and collaborating partners.

Problem: Elevated bilirubin levels in newborns can cause brain damage. While a clinical guideline is available for neonatal bilirubin management, 44 this management requires complex data synthesis and interpretation, and EHRs provide limited support for these tasks. Solution: The SMART on FHIR Bilirubin App ( Figure 2 ) retrieves relevant information from across the chart, graphically displays these data, and provides patient-specific care recommendations according to the clinical guideline as well as risk predictions for rebound hyperbilirubinemia following phototherapy.

Outcomes: The Bilirubin App is used in the care of over 90% of applicable newborns at UUH. 27 It was rated as having "best imaginable" usability by attending providers on the System Usability Scale, reduced the time needed for bilirubin management by 66%, and was associated with a significant increase in guideline-compliant phototherapy. 27 The app has been recognized by several awards, including awards from the Department of Health and Human Services 45 and the American Medical Informatics Association. 46 Partners: UUH pediatricians, Intermountain Healthcare.

Problem: Clinical calculators are an essential tool for evidencebased care. While many clinical calculators are accessible through the Web, providers must manually search for the relevant input data (eg, age, gender, co-morbidities, vital signs, lab results) and enter the data into the calculator in a time-consuming and potentially errorprone process. Solution: MDCalc Connect, an EHR-integrated version of MDCalc, a leading clinical calculator platform historically available as Web and mobile apps (Figure 3 ). Calculator inputs are auto-filled with relevant EHR data in this SMART on FHIR app.

Outcomes: An initial evaluation of MDCalc Connect for the CHA 2 DS 2 -VASc calculator showed that EHR integration enabled the automatic identification of potential patient risk factors that were not otherwise noted by clinicians in more than half of patients. 47 Partners: UUH providers and MD Aware, the makers of MDCalc. MDCalc covers over 200 conditions and is used by approximately two-thirds of US physicians. 48 Diabetes pharmacotherapy outcome prediction app Problem: Diabetes mellitus is a major source of mortality and morbidity, and pharmacotherapy is a core aspect of treatment. However, for most patients with diabetes, there is little evidence-based guidance on pharmacotherapy following initial treatment with metformin. 49 Solution: Artificial intelligence (AI) was used to predict the medications most likely to be effective for individual patients to reach a patient-specific hemoglobin A1c goal. The application provides these predictions as well as information on relevant side effects and associated costs (Figure 4 ). 50 Outcomes: A clinical trial is underway. The predictive models were found to have high predictive value in a validation data set, with an accuracy of 0.81 and an area under the curve of 0.88. 50 Partners: Hitachi, Ltd. and the UUH Community Physicians Group.

Lung cancer screening shared decision-making app Problem: Lung cancer is the leading cause of cancer deaths in the United States. 51 Lung cancer screening with low-dose computed tomography scans could save more lives than breast cancer screening, and the US Preventive Services Task Force recommends screening to be considered for eligible patients with a heavy smoking history. [52] [53] [54] However, screening rates among eligible patients in 10 US states was only 12.5% in 2017, 55 due in part to the need for shared decision-making that considers patient-specific risks and benefits from screening.

Solution: To address this issue, ReImagine EHR developed an EHR-integrated version of Decision Precision, a lung cancer screen-ing shared decision-making tool developed at the University of Michigan and the Department of Veterans Affairs (VA) ( Figure 5) .

Outcomes: An AHRQ-funded clinical trial at UUH is underway. Partners: University of Michigan, Ann Arbor VA Center for Clinical Management Research, Salt Lake City VA Center for Informatics Decision Enhancement and Surveillance (IDEAS), UUH Community Physicians Group, UUH Department of Radiology.

Problem: Chronic diseases affect most adults, with many adults having multiple chronic conditions such as hypertension, diabetes, and chronic lung disease. 56, 57 Chronic diseases account for most morbidity and mortality in the United States, and significant clinical resources are required for managing these diseases. 58 Despite their importance, EHRs provide limited cognitive support for chronic disease management, requiring clinicians to search for and organize disparate data throughout the EHR in order to obtain an adequate sense of the patient and the required clinical actions. As a result, patients oftentimes do not receive recommended care. 3 Solution: A Disease Manager was developed to serve as a onestop-shop in ambulatory care for chronic disease management and health maintenance. Initial modules supported include hypertension, diabetes, chronic obstructive pulmonary disease (COPD), and health maintenance ( Figure 6 ). More than just an individual application, the Disease Manager is increasingly a foundational platform for other applications. For example, the Disease Manager incorporates both the Diabetes Pharmacotherapy Outcome Predictor App and the Lung Cancer Screening Shared Decision-Making App.

Outcomes: An experimental simulation study of this tool for COPD management found that compared to the EHR alone, the app was associated with improved completion of recommended care (81% vs. 48%, P < 0.001), reduced time spent per task, and reduced user frustration. 59 A clinical trial is underway.

Partners: UUH primary care and specialist providers. 

Annual institutional investment in ReImagine EHR has been in the range of approximately $500 000 to $1 million, with almost all of the funds used to support the salaries of informaticists, software engineers, sociotechnical experts, and trainees. The return on this investment has included the positive impact of individual projects on patient care and the provider experience, grant funding, scientific outputs, and commercialization. To date, the initiative has helped to bring in over $35 million in external grants and contracts. Table 4 provides a summary of exemplar grants. ReImagine EHR also increases our health system's capacity to adjust to change, whether it be a shift to value-based payment 60 or a need to adapt to a pandemic. For example, in response to the COVID-19 pandemic, Re-Imagine EHR collaborated with its partners at MDCalc to develop a free SMART on FHIR application for COVID-19 management (Figure 3) . 61

Digital innovations that extend native EHR capabilities have immense potential for improving health and health care. ReImagine EHR was among the first large-scale initiatives at an academic health system to embrace this approach to digital innovation. Key aspects of ReImagine EHR include a commitment to collaboration, a focus on delivering value, investment in human capital and infrastructure, the embrace of interoperability standards such as SMART on FHIR, and a high degree of synergy achieved across research and operations. As the healthcare community increasingly embraces digital innovation enabled by interoperability standards, we believe that our experiences can help guide future progress. Key lessons learned are described below.

By many metrics, ReImagine EHR has been a successful initiative: many innovative EHR add-on apps have been developed and implemented; long-term institutional funding has been sustained; substantial grant funding and corporate partnerships have been secured; and formal evaluations have demonstrated high user satisfaction and positive clinical impact. 27 An initiative such as ReImagine EHR can serve as an important strategic asset for reducing cognitive load and addressing provider burnout. It can also help healthcare systems adapt to a rapidly changing landscape. We hope that the return on investment described in this manuscript can help fellow innovators at other organizations make the case for institutional investment in similar programs.

In some cases, valuable digital health innovations can be implemented across health systems using only the US Core FHIR APIs 40 that are widely supported across EHR systems and required by federal regulations. 26 For example, MDCalc Connect was designed to rely only on natively supported FHIR APIs to enable widespread dissemination.

Non-native APIs can expand functionality but require more expertise and effort

There may be cases where APIs natively supported by the EHR are inadequate for meeting user needs. For example, for the bilirubin app, custom APIs were needed to support critical data such as the baby's gestational age and phototherapy orders. 42 Because substantial expertise and effort are required to develop these APIs, and because custom APIs make dissemination more difficult, the tradeoff between functionality and dissemination potential must be carefully considered. 42 Infrastructure tools can accelerate the pace of innovation As described in Table 3 , even when native EHR APIs meet all data needs, other technical challenges often arise. Infrastructure tools addressing such needs support the efficient design, development, and implementation of interoperable digital health innovations.

Critical to achieving wide uptake and impact is a systematic and iterative approach to user-centered design. With the help of experts in cognitive psychology, we elicit user needs, design our solutions to meet those needs and associated workflows, and make iterative enhancements based on user feedback. Key user needs that we strive to meet include (1) a clear value proposition such as time savings;

(2) a coherent user and team experience spanning both the digital innovation and the native EHR; (3) a self-explanatory user interface that requires minimal training; and (4) easy access to the innovation within usual workflows. In-depth user needs analyses sometimes uncover unanticipated ways in which the tool can provide additional value, such as by supporting resident education, patient education, or shared decision-making with patients. 

In addition to user-centered design, adoption of innovations can be increased through implementation support. Approaches we employ include (1) user education through laminated quick reference guides, brief presentations at staff or faculty meetings, and short videos by clinical champions; (2) monitoring and encouraging adoption; (3) tailoring implementation strategies based on the results of user needs assessments; (4) soliciting user feedback through an in-app widget that captures user comments along with a snapshot of the application state; and (5) making every effort to address user feedback in a timely fashion. Another potential approach to increasing adoption is through alerts or non-interruptive reminders recommending use; while potentially effective, this approach should be used with caution due to the potential for alert fatigue. 62

Evaluation is critical across the project lifecycle. In our experience, however, evaluation is often under-resourced outside of grant-funded initiatives due to never-ending demands for develop-ing new functionality. This problem is compounded by the expertise needed to conduct rigorous evaluations, including for biostatistics, economics, informatics, clinical medicine, and cognitive psychology. We are continuing to refine our roadmap and supportive tooling for conducting such evaluations. In many cases, a rigorous evaluation may require effort comparable to the development of the technology itself, such as when complex clinical decision support logic needs to be replicated for impact assessment. ReImagine EHR has taken some initial steps through the assembly of highly trained evaluation experts and the conduct of several rigorous evaluation studies. However, much more rigorous evaluations are needed to comprehensively assess impact, unintended consequences, and areas for improvement. Ideally, digital innovations should be evaluated through randomized controlled trials conducted across multiple sites and guided by implementation science frameworks. 63, 64 App fatigue may be coming

Just as an increasing use of alerts in the EHR led to alert fatigue, 62 as apps proliferate, it has the potential to lead to "app fatigue." Po- tential solutions to such app fatigue include (1) vetting potential additions through formal governance, 65 (2) delivering a consistent user interface experience aligned with that of the EHR, (3) developing a small number of broad solutions rather than a large number of narrow solutions (eg, one app for managing multiple chronic conditions versus multiple apps for managing individual conditions), (4) embedding links to narrower applications within broader applications, and (5) enhancing or retiring applications with limited usage, user satisfaction, and/or impact.

Effective execution of a digital innovation initiative such as ReImagine EHR requires interdisciplinary, cross-institutional, and crosssector partnerships. As demonstrated by the representative innovations described above, successful projects often require contributions from multiple stakeholders. ReImagine EHR relies on interdisciplinary partnerships across diverse domains including clinical medicine, patient engagement, user-centered design, cognitive psychology, standards and interoperability, data science, AI, business, and software engineering. Moreover, since practice patterns and the technology infrastructure may differ significantly across health systems, it is often necessary to engage external institutional partners. Furthermore, industry partners can bring unique resources and expertise. Thus, we believe both internal and external partnerships are essential to maximize impact.

The healthcare community is still in the early stages of applying FHIR-based digital innovations to improve care. There are many exciting, untapped areas for future innovation. Exciting areas for future research and development include direct patient engagement through patient-facing SMART on FHIR applications such as Apple Health 66 ; the further application of AI in medicine, such as to predict and proactively provide the information that a provider will likely seek for a given patient; and the integration and use of nontraditional data such as free text narrative in the EHR, sensor data, and payer claims data.

EHR-integrated digital innovation initiatives can be key enterprise assets for improving users' experience with the EHR, enhancing patient care, and helping address provider burnout. ReImagine EHR provides a case study of how health systems can begin to seize this opportunity. Co-author TJC is supported by a Career Development Award from VA HSR&D (CDA 16-151) and a grant from the VA QUERI National Program (PrOVE QUERI). TR and KLM were supported by the National Library of Medicine of the National Institutes of Health (T15LM007124). Some of the research reported in this publication utilized the Genetic Counseling Shared Resource at Huntsman Cancer Institute at the University of Utah and was supported by the National Cancer Institute of the National Institutes of Health (P30CA042014).

The funding organizations had no role in the conceptualization, design, data collection, analysis, decision to publish, or manuscript preparation for this case study. The content is solely the responsibility of the authors and does not necessarily represent the official views of the organizations involved.

Each author made substantial contributions to (1) the conception or design of the ReImagine EHR initiative or one of the described projects (all authors); (2) the acquisition, analysis, or interpretation of data (KK, PVK, CW, TJR, HSK, TT, RLC, KM, DB, CHS, JHS, JPH, ST, WT, HB, and GDF); (3) the creation of new software used in the work (KK, PVK, CW, MCF, CN, DKM, PBW, DS, SRL, RLB, and RCC); or (4) the drafting or substantial revision of the manuscript (KK, PVK, CW, TJR, HSK, TT, and GDF). All authors also approved the manuscript for submission and agreed both to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work are appropriately investigated, resolved, and documented in the literature.

Protocol-based computer reminders, the quality of care and the non-perfectability of man

To Err Is Human: Building A Safer Health System

The quality of health care delivered to adults in the United States

Few Americans receive all high-priority, appropriate clinical preventive services

Health information technology: Laying the infrastructure for national health reform

A glimpse at EHR implementation around the world: the lessons the US can learn

Stimulus bill implementation: expanding meaningful use of health IT

Office of the National Coordinator for Health Information Technology. Percent of hospitals, by type, that possess certified health IT

Death by a thousand clicks: where electronic health records went wrong. Kaiser Health News

Advanced proficiency EHR training: effect on physicians' EHR efficiency, EHR satisfaction and job satisfaction

Physician opinions about EHR use by EHR experience and by whether the practice had optimized its EHR use

Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties

Assessment of inpatient time allocation among first-year internal medicine residents using time-motion observations

4000 clicks: a productivity analysis of electronic medical records in a community hospital ED

Association of electronic health record use with physician fatigue and efficiency

The association between perceived electronic health record usability and professional burnout among US physicians

Determining what individual SUS scores mean: adding an adjective rating scale

Association of electronic health record design and use factors with clinician stress and burnout

The quality of outpatient care delivered to adults in the United States

Comprehensive methodology to monitor longitudinal change patterns during EHR implementations: a case study at a large health care delivery network

Epic App Orchard-Homepage

Cerner App Gallery-Homepage

SMART on FHIR: A standards-based, interoperable apps platform for electronic health records

Why aren't they happy? An analysis of end-user satisfaction with Electronic health records

SMART on FHIR: big gains, big challenges j Executive Education at HMS. Harvard Medical School

ONC's Cures Act Final Rule

Association of an electronic health record add-on app for neonatal bilirubin management with physician efficiency and care quality

Children's Hospital. FHIR-Innovation & Digital Health Accelerator

The digital drag and drop pillbox: design and feasibility of a skill-based education model to improve medication management

Preliminary evaluation of the chest pain dashboard, a FHIR-based approach for integrating health information exchange information directly into the clinical workflow

SMART App Gallery

The effect of an electronic health record-based tool on abnormal pediatric blood pressure recognition. Congenit

Beyond one-off integrations: a commercial, substitutable, reusable, standards-based, electronic health recordconnected app

Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework

Participatory design, user involvement and health IT evaluation

Cognitive and usability engineering methods for the evaluation of clinical information systems

Innovating from within: a process model for user-centered digital development in academic medical centers

Patient information organization in the intensive care setting: Expert knowledge elicitation with card sorting methods

Health Level Seven International (HL7)

Balancing functionality versus portability for SMART on FHIR applications: case study for a neonatal bilirubin management application

SMART App Launch: Scopes and Launch Context

American Academy of Pediatrics Subcommittee on Hyperbilirubinemia. Management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation

Provider User Experience App Challenge Awards from the Department of Health and Human Services

Implementation of a fast healthcare interoperability resources-based clinical decision support tool for calculating CHA2DS2-VASc scores

Understanding and creating calculators for medical diagnoses

EHR-integrated, machine learning-driven SMART on FHIR pharmacotherapy decision support system for type-2 diabetes mellitus

Reduced lung-cancer mortality with low-dose computed tomographic screening

Screening for lung cancer: U.S. Preventive Services Task Force Recommendation Statement

US Preventive Services Task Force. Final recommendation statement: lung cancer screening

Screening for lung cancer-10 states

Adults 55 and over with chronic conditions. centers for disease control and prevention

Multiple Chronic Conditions in the United States

Integrated displays to improve chronic disease management in ambulatory care: A SMART on FHIR application informed by mixed-methods user testing

Emerging lessons from regional and state innovation in value-based payment reform: balancing collaboration and disruptive innovation

Clinical decision support alert appropriateness: a review and proposal for improvement

Context and approach in reporting evaluations of electronic health record-based implementation projects

Designing, conducting, and reporting clinical decision support studies: recommendations and call to action

A pragmatic guide to establishing clinical decision support governance and addressing decision support fatigue: a case study

We would like to thank Bryce Covey, Robert Richens, Carrie King, Dean Taylor, Saundra S. Buys, MD, Molly B. Conroy, MD, MPH, FASCM, FAHA, and Maribel Cedillo, MS, RD for their contributions to the described work.

KK reports honoraria, consulting, sponsored research, licensing, or codevelopment with McKesson InterQual, Hitachi, Pfizer, Premier, Klesis Healthcare, RTI International, Mayo Clinic, Vanderbilt University, the University of Washington, the University of California at San Francisco, MD Aware, and the US Office of the National Coordinator for Health IT (via ESAC, JBS International, Aþ Government Solutions, Hausam Consulting, and Security Risk Solutions) in the area of health information technology. KK was also an unpaid board member of the non-profit Health Level Seven International health IT standard development organization, and he is an unpaid member of the US Health Information Technology Advisory Committee. JPH is cofounder, CEO, and part owner of MD Aware, LLC, which owns and operates MDCalc. ST, WT, and HB are employees of Hitachi, Ltd. RH serves on a Data and Safety Monitoring Board for Astellas Pharmaceuticals. WWC is a scientific advisor for Health Fidelity. While the only digital health innovation described in this manuscript that is currently a commercial product is MDCalc Connect, other health IT tools described in this manuscript may be commercialized in the future to enable wider impact.

No new data were generated or analyzed in support of this research.