key: cord-0894034-7biitcue
authors: Greenberg, Jacob K.; Otun, Ayodamola; Nasraddin, Azzah; Brownson, Ross C.; Kuppermann, Nathan; Limbrick, David D.; Yen, Po-Yin; Foraker, Randi E.
title: Electronic clinical decision support for children with minor head trauma and intracranial injuries: a sociotechnical analysis
date: 2021-05-19
journal: BMC Med Inform Decis Mak
DOI: 10.1186/s12911-021-01522-w
sha: 64b2d29e85cc26b1aebed9da7fbf556467a95b43
doc_id: 894034
cord_uid: 7biitcue

BACKGROUND: Current management of children with minor head trauma (MHT) and intracranial injuries is not evidence-based and may place some children at risk of harm. Evidence-based electronic clinical decision support (CDS) for management of these children may improve patient safety and decrease resource use. To guide these efforts, we evaluated the sociotechnical environment impacting the implementation of electronic CDS, including workflow and communication, institutional culture, and hardware and software infrastructure, among other factors. METHODS: Between March and May, 2020 semi-structured qualitative focus group interviews were conducted to identify sociotechnical influences on CDS implementation. Physicians from neurosurgery, emergency medicine, critical care, and pediatric general surgery were included, along with information technology specialists. Participants were recruited from nine health centers in the United States. Focus group transcripts were coded and analyzed using thematic analysis. The final themes were then cross-referenced with previously defined sociotechnical dimensions. RESULTS: We included 28 physicians and four information technology specialists in seven focus groups (median five participants per group). Five physicians were trainees and 10 had administrative leadership positions. Through inductive thematic analysis, we identified five primary themes: (1) clinical impact; (2) stakeholders and users; (3) tool content; (4) clinical practice integration; and (5) post-implementation evaluation measures. Participants generally supported using CDS to determine an appropriate level-of-care for these children. However, some had mixed feelings regarding how the tool could best be used by different specialties (e.g. use by neurosurgeons versus non-neurosurgeons). Feedback from the interviews helped refine the tool content and also highlighted potential technical and workflow barriers to address prior to implementation. CONCLUSIONS: We identified key factors impacting the implementation of electronic CDS for children with MHT and intracranial injuries. These results have informed our implementation strategy and may also serve as a template for future efforts to implement health information technology in a multidisciplinary, emergency setting. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12911-021-01522-w.

Background Minor head trauma (MHT) is among the most frequent and most damaging health problems affecting children [1, 2] . While longer-term health effects are one important concern following pediatric MHT, the acute management is largely focused on identifying and managing groups at risk of acute neurological decline [3] . Consequently, there have been several large-scale efforts to develop and validate clinical decision support (CDS) tools guiding the need for computed tomography (CT) imaging in children with MHT [4, 5] . One of these tools developed by the Pediatric Emergency Care Applied Research Network (PECARN) network has subsequently been implemented in several large-scale trials [6, 7] .

Head CT decision tools, such as the PECARN rules, offer substantial value to the more than 500,000 children with MHT seen in United States emergency departments each year [1] . However, far less evidence-based guidance is available for managing the 4-14% of children with MHT who are found to have intracranial injuries (ICI) on neuroimaging [5, 8] . While most of these children remain neurologically stable, some will require neurosurgical or other advanced interventions, emphasizing the importance of risk-stratification and close monitoring for highrisk subgroups. At the same time, universal intensive care unit (ICU) admission strains limited resources and may subject patients and families to unnecessary emotional distress [9] [10] [11] .

Addressing this evidence gap, a provider-facing risk score was recently developed and internally validated using a large-scale multicenter dataset, which is currently undergoing external validation in a large, multicenter cohort [8] . The goal of this risk score is to support evidence-based decision-making (e.g. regarding disposition to an appropriate level-of-care) among pediatric neurotrauma providers at the point-of-care. However, successfully integrating CDS into routine practice remains a challenge, with major barriers including clinician difficulty remembering tool details and absence of a userfriendly interface [13, 14] . Electronic CDS may mitigate many of these barriers by presenting clinician users with relevant evidence at the point-of-care.

Implementing electronic CDS also requires navigating complex sociotechnical dimensions, including areas such as workflow and communication, institutional culture, and hardware and software infrastructure [15] . Failure to investigate these dynamics prior to introducing new health information technology can lead to ineffective efforts and potential unintended consequences [16] [17] [18] . Recognizing the importance of such foundational work, we conducted a sociotechnical analysis of electronic CDS intended to promote evidence-based decision-making among clinicians managing children with MHT and ICI. Given our focus on matching patient risk to an appropriate level-of-care, our primary objective was to investigate the sociotechnical influences on implementing electronic CDS to guide clinicians' decisions regarding the need for ICU admission. Our secondary objective was to investigate alternative uses of electronic CDS among children with MHT and ICI.

Leveraging qualitative methods, we conducted a sociotechnical analysis to guide the development and implementation of electronic CDS for children with MHT and ICI. Participants in this study were either physicians who care for children with MHT and ICI, physicians with relevant administrative roles, or information technology specialists experienced in the development and implementation of electronic CDS. All participants were recruited via email solicitation. For the physician participants, we contacted all attending and fellow physicians from neurosurgery, pediatric general surgery, pediatric emergency medicine, and pediatric critical care at a single large academic medical center. Due to the small number of post-residency fellows, we also contacted senior (> post-graduate-year four) residents from neurosurgery and general surgery. To broaden the generalizability of the results, we also solicited participation from attending physicians from these specialties at six other academic hospitals, one community medical center, and one military-affiliated institution (nine total centers). Participants were offered $50 compensation. Interviews were conducted from March through May, 2020.

Individuals who agreed to participate were invited to partake in focus group interviews. Due to the COVID-19 pandemic, all focus groups were held online via Zoom (Zoom Video Communications, San Jose, CA). To avoid imbalances of experience and seniority, physician participants were separated into groups with trainees (residents and fellows) or attendings. We grouped physicians into heterogenous focus groups with representation from different clinical specialties to promote interactive discussions [19] . We also included a dedicated focus group Keywords: Sociotechnical analysis, Traumatic brain injury, Head trauma, Electronic clinical decision support, Implementation science, Health information technology with physicians that also held relevant administrative leadership roles. Non-clinician information technology specialists were organized in a separate group, which included software engineers with focused expertise in CDS graphic design as well as the CDS implementation into the EHR.

Each focus group was organized in a semi-structured fashion and followed an interview guide developed using input from qualitative methods experts (PYY and RF). The guide (available in the Additional file 1) was intended to investigate key sociotechnical dimensions, including: hardware and computing infrastructure; clinical content; human-computer interface; people; workflow and communication; organizational policies, procedures, and culture; and system measurement and monitoring [15, 20] . These domains are displayed in Fig. 1 . Participants were shown a wireframe prototype of the preliminary CDS tool, which displayed predicted risk based on imaging findings and a mental status assessment provided by the Glasgow Coma Scale (GCS) score. While the wireframe was updated across interviews, the final version shown is depicted in Fig. 2 . The wireframe depicted an input screen with four questions and an output screen with the following four domains: overall (composite) predicted risk; predicted risk of individual outcomes (e.g. neurosurgical intervention); hypothetical institutional level-of-care recommendations; and costs associated with different level-of-care decisions. A single moderator (JKG, a senior neurosurgery resident with masters-level clinical research training) led all of the interviews, with the aid of a note-taker experienced in qualitative methods (AN and RS). The focus groups were video recorded for subsequent review, and audio recordings were professionally transcribed (Landmark Associated Inc., Phoenix, AZ). Focus groups continued until the research team felt thematic saturation had been reached and no substantially new ideas emerged [19] .

We initially analyzed all focus group transcripts using an inductive process of thematic analysis. [21] First, all focus group transcripts were independently analyzed and coded by two authors (JKG, AO) using Dedoose software version 8.3.35 (Dedoose, Hermosa Beach, CA). A codebook was developed by comparing and reconciling codes and code applications after each transcript in an iterative fashion. These codes were then further modified based on input from qualitative methods experts (PYY and RF). The final coding scheme was applied to each transcript based on consensus agreement between both reviewers, with additional input from other team members (PYY and RF) in ambiguous cases. Next, overarching themes and sub-themes were inductively assigned based on the most important and frequent ideas represented in the interview codes. Based on consensus of both reviewers and qualitative methods experts, final themes were chosen to reflect the most impactful interview results, without seeking to cover every minor comment that was recorded. We then used a deductive process to crossreference the newly created themes with the defined categories of sociotechnical analysis. Finally, we created a schematic diagram based on the interview results indicating the potential points in the care pathway where electronic CDS could facilitate or impede the management of children with MHT and ICI (Fig. 3 ).

We conducted seven focus groups with a total of 32 participants from nine institutions. The duration of the interviews ranged from 39 to 62 min (median 51 min) and included two to six participants per focus group (median five participants). Most participants were either attending (72%) or resident/fellow (16%) physicians, while 13% were non-clinician information technology specialists. Ten (31%) physician participants held relevant administrative positions at their respective institutions (e.g. trauma director, division chief ). Most (61%) participants were between 30-39 years and male (59%), and the largest percentage of participants were from either neurosurgery (34%) or emergency medicine (28%). A complete list of participant demographics is shown in Table 1 .

Across the seven focus groups, 46 unique codes were identified (available in the Additional file 1). These were then inductively grouped into a total of five themes with associated sub-themes: clinical impact; stakeholders and users; tool content; clinical practice integration; and Table 2 .

A major topic of discussion related to how the proposed electronic CDS tool could be used in clinical practice. Participants generally supported our primary proposed application of using the tool to help guide the need for ICU care. One participant explained, Beyond the primary use of guiding the appropriate level-of-care, some participants thought that the tool could also support family counseling. While no physicians endorsed using the tool itself as a shared decisionmaking aid, several suggested that the risk data provided could inform their discussions with families. One physician stated, I think it would be good to get the information to us so that we can share it with them appropriately. Some physicians also endorsed additional "exploratory" uses of the CDS intervention they felt could be appropriate with further validation and acceptance of the underlying evidence. For example, some participants thought the tool could guide the need for hospital transfer or provide reassurance for discharging home some low-risk patients.

Along with these potential uses, participants also identified several unintended consequences that could emerge from implementing the CDS tool. One participant expressed concern that the tool could be misinterpreted or misapplied by non-neurosurgeons.

Other participants expressed concern with the notion of using the tool to guide hospital transfers, explaining that community hospitals may be unprepared for a rare deterioration, and may also lack comprehensive trauma services, including care for non-accidental trauma. One physician stated, if you're going to use this in outside facilities…it could be used incorrectly if you're not really careful about it and very explicit in your instructions on how it's supposed to be used. Finally, some participants noted that inter-departmental conflicts could also arise if clinicians had differing interpretations regarding the clinical significance of a particular risk score. These potential uses and unintended clinical consequences are summarized in Fig. 3 .

An important area of discussion for physician participants was the tool's potential use by neurosurgeons and non-neurosurgeons. Several neurosurgeons and nonneurosurgeons supported the idea that the CDS tool would help expand risk knowledge across specialties and training levels, empowering broader groups of providers. For example, one neurosurgeon stated, Despite those differing opinions, multiple participants noted the importance of multidisciplinary buy-in for successful CDS implementation. For example, one physician stated,

Outside of the clinical specialties targeted for the focus groups, multiple participants also highlighted the importance of nurses as key stakeholders impacting implementation. For example, some participants noted the need for buy-in from nursing leadership for possible changes in clinical workflow. Additionally, participants felt that nursing capabilities would have an important impact on how the CDS tool is used. For example, one physician stated, 

Participants provided feedback about the appropriateness of tool components presented in the wireframe, which was then used to refine the initial prototype. There was generally broad support for the predicted risk estimates provided in the tool. One physician explained,

Similarly, most physicians were supportive of including general institutional recommendations based on different risk-thresholds, though there was some resistance. For example, one physician said,

Another area of concern related to cost estimates associated with different levels-of-care. While some participants saw value in the cost estimates, many others expressed concern with cost information being considered at the point-of-care and also noted that accurate cost estimates (versus charges) could be hard to obtain. One physician explained,

In addition, some participants noted aspects of the tool content that may make it challenging to implement in clinical practice. For example, some participants noted that imaging diagnoses may vary, even among expert radiologists. Additionally, two participants commented that the appropriate timing for assessing the GCS score is unclear (e.g. initial assessment versus worst value), which could impact the values recorded. Other participants noted that some target end-users may lack appropriate expertise to assign all input variables. For example, one ED physician stated, with regard to the…width of the skull…if I don't know the answer to that, that would be a very limiting step for me.

Multiple participants also noted that the tool does not evaluate social or non-cranial concerns. For example one physician stated,

Finally, several participants emphasized that the tool does not capture all of the nuances of clinical decisionmaking. For example, one physician stated.

I'm sure we've all felt that pain of decision support that is evidence-based, and people immediately get hung up on, 'well the one time this patient had X. Therefore, I need to do this for the rest of the time. '

All participants generally agreed that to be maximally effective, the electronic CDS tool needed to be integrated within the EHR environment. Participants also emphasized in particular that satellite apps or websites would be a barrier to use, particularly for physicians who may not care for children with MHT on a frequent basis. One resident physician explained, I think that the optimal place to put it would be in the EHR…If it's an app that I need to have, or even worse, a website that I need to go to, I'm gonna completely forget about it… At the same time, several physician participants with informatics experience explained that although building the CDS directly within the EHR is technically feasible, having a website linked to the EHR may be easier to implement. For example, one physician stated, It's such an easy thing to do; Epic has the ability and the power to do it, but the logistics of actually pushing it through are so overwhelming that the solu-tion…of doing a web app is actually easier.

While participants generally agreed that the tool needed to be integrated with the EHR, there was less agreement regarding how end-users could best be targeted. Although the IT specialists generally agreed that either a pop-up or a link in the EHR hyperspace were options, participants lacked a clear approach for triggering the tool. Participants discussed linking an alert to an order for a head CT scan or neurosurgery consult, but felt that approach would be non-specific and ineffective. Other participants explained that relying on diagnoses in the EHR would also be challenging, as ED triage diagnoses may not be accurate and ED discharge diagnoses likely would not be recorded sufficiently early in the encounter.

The last major theme addressed how participants thought the impact of the CDS should be evaluated after implementation. The complete list of suggested post-implementation evaluation measures is shown in Table 3 and grouped into three categories. First, patient safety/clinical outcome measures included metrics such as unexpected floor-to-ICU transfers that reflected the extent to which the CDS tool improved patient safety. Second, resource use and process measures described changes in the costs following CDS implementation, along with changes in practice patterns (e.g. timing of neurosurgery consults). Finally, participants noted the importance of implementation outcomes, such as how often clinicians actually used the tool when caring for children with MHT and ICI.

This report presents the results of a sociotechnical analysis of electronic CDS to aid the management of children with MHT and ICI. Our findings reflect the relatively early stage of this development and implementation effort and also help guide the next steps in this process. The major interview themes covered the anticipated primary and secondary uses of the tool, important stakeholders in implementation, suggestions for improving the tool's content, along with perspectives on how to integrate the CDS into clinical workflow. Finally, participants highlighted key measures that could evaluate the CDS tool's impact after implementation.

Despite some concerns and reservations, the focus groups highlighted substantial interest in using the proposed CDS intervention to help guide level-of-care decisions, standardize care, build consensus decisions, and expand risk knowledge across specialties. Although less frequently discussed, there was also support for using the tool to inform family counseling. Participants also raised concerns related to potential unintended consequences to be avoided, such as misuse of the tool to avoid appropriate consults. Another challenge identified related to how risk predictions could be integrated into institution-specific level-of-care recommendations. At present, decision-making varies widely by both physician and institution [8, 22] , explaining the need for physician leaders to identify recommendations that reflect institutional culture and capabilities. However, concerns related to blanket recommendations also emphasized the importance of balancing consensus-derived institutional recommendations with physician autonomy in final decision-making, as other similar tools have done [23] .

Outside of the primary intended goal of guiding levelof-care recommendations, there was mixed and measured support for other possible uses, such as stratifying the need for hospital transfer. While head injury regionalization is known to vary across the United States, [12] MHT is a major cause of transfer to academic ED's [24] [25] [26] , and avoiding some routine transfers may have important implications for distributed trauma systems [9, 27, 28] . Nonetheless, participants explained that changing such practices will require broader acceptance of the underlying evidence. In particular, further prospective validation is needed to evaluate the safety of this application. Likewise, although non-accidental trauma and non-cranial injuries were not identified as important predictors during risk model development [8] , these factors must be considered during transfer decisions.

Feedback from the focus groups also impacted the CDS content. For example, the focus groups identified the need to explicitly define the GCS score based on the patient's first assessment in the treating hospital. Likewise, participant feedback highlighted potential difficulty Frequency of the tool being presented and how often it is used Frequency with which the recommendations are being followed some physicians may have measuring fracture depression, and a visual aid to assist this task was built into a future prototype of the CDS tool. Finally, the widespread concerns related to including cost information led us to remove that component from future CDS prototypes.

While many of the challenges the interviews identified could be readily addressed, likely the greatest implementation barrier that emerged related to effective approaches of incorporating electronics CDS into clinical workflow. Universally, participants felt that integrating the CDS tool within the EHR was key, and in an ideal setting, CDS elements would auto-populate from the EHR [34, 35] , likely improving both clinician satisfaction and CDS use [36] . Likewise, automating CDS alerts within clinical workflow is key to reducing click fatigue and expanding utilization [37] [38] [39] , which was noted by several participants. While still exploratory, perhaps the most encouraging route to automating CDS use involves the application of machine learning algorithms to detect acute intracranial hemorrhage on CT [44, 45] . Although still requiring physician review, such tools offer a promising avenue for both provider targeting and CDS data capture and should be explored in future efforts.

Beyond identifying new themes, the structure of the focus group interviews was intended to address key sociotechnical elements [15, 20] . We found that the themes and sub-themes identified corresponded to almost all key tenets of sociotechnical theory, suggesting that this model captured most considerations relevant to implementation planning. Given that interface design was not considered in this analysis, the human computer interface dimension was expectedly lacking. In addition, we did not identify any major themes or sub-themes corresponding to external rules or regulations. Likely, this absence reflected the early stage of CDS development, and further evaluation later in the implementation process may yield additional findings. In addition, we found that one sub-theme-"challenges to using the tool in clinical practice"-could not be effectively mapped to a sociotechnical dimension. We believe this difficulty also reflected the early stage of development, where potential problems, such as ambiguous input variables, were still being identified and remediated.

There are limitations regarding our study. First, while we included multidisciplinary physicians, most participants came from academic hospitals, which may limit the generalizability of the results. However, based on the participant feedback, academic clinicians are likely to be the primary targets of future CDS implementation, supporting their higher focus group representation. Second, we did not include nurse practitioners or nurses in the focus groups, who may be potential end-users or stakeholders. Likewise, we did not include lawyers or regulatory experts, whose input could be needed during later stages of implementation to address potential medicolegal concerns raised. Third, given the early stage of the implementation planning, participants reviewed wireframes of the CDS prototype, rather a fully developed prototype. An interactive prototype that integrates the focus group feedback will undergo dedicated user testing in future work. Fourth, more than 80% of our participants were younger than 50 years, and previous efforts have shown older provider age is associated with lower rates of electronic CDS use [46, 47] . Therefore, future larger-scale implementation efforts will need to evaluate potential agerelated disparities in CDS use. Finally, we only included physician participants practicing in the United States. Given known differences in EHR use and regulatory structures in other countries, these results might differ if examined in other healthcare systems [48, 49] .

This sociotechnical analysis identified the primary anticipated uses of electronic CDS for children with MHT and ICI, along with other exploratory uses warranting further consideration. By identifying key factors impacting CDS adoption, these results provide a strong foundation for a future implementation trial. This analysis may also inform the development of other electronic CDS tools used in an interdisciplinary emergency treatment setting. 

The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12911-021-01522-w.

A word document containing the focus group interview guides and a full list of the 46 codes applied to the interview transcripts.

The epidemiology of outpatient visits for minor head injury

Report to Congress on Mild Traumatic Brain Injury in the United States: Steps to Prevent a Serious Public Health Problem

Centers for disease control and prevention guideline on the diagnosis and management of mild traumatic brain injury among children

Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study

Accuracy of PECARN, CATCH, and CHALICE head injury decision rules in children: a prospective cohort study

Use of traumatic brain injury prediction rules with clinical decision support

Effect of the head computed tomography choice decision aid in parents of children with minor head trauma: a cluster randomized trial

Development and internal validation of a clinical risk score for treating children with mild head trauma and intracranial injury

Distribution of specialized care centers in the United States

Posttraumatic stress disorder in children and their parents following admission to the pediatric intensive care unit: a review

The impact on parents of a child's admission to intensive care: integration of qualitative findings from a cross-sectional study

Pediatric traumatic brain injury is inconsistently regionalized in the United States

Implementation of clinical decision rules in the emergency department

When do clinical decision rules improve patient care?

A new sociotechnical model for studying health information technology in complex adaptive healthcare systems

Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis

The extent and importance of unintended consequences related to computerized provider order entry

Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality

Qualitative and mixed methods provide unique contributions to outcomes research

Health information technology (HIT) adaptation: refocusing on the journey to successful HIT implementation

Thematic analysis: Striving to meet the trustworthiness criteria

North American survey on the post-neuroimaging management of children with mild head injuries

Development of a clinical decision support system for pediatric abdominal pain in emergency department settings across two health systems within the HCSRN

Preventable transfers in pediatric trauma: a 10-year experience at a level I pediatric trauma center

Interfacility transfers of noncritically ill children to academic pediatric emergency departments

Characteristics of pediatric trauma transfers to a level I trauma center: implications for developing a regionalized pediatric trauma system in California

Geographic proximity to specialized pediatric neurosurgical care in the contiguous United States

Paediatric mild head injury: is routine admission to a tertiary trauma hospital necessary?

Back to the bedside: developing a bedside aid for concussion and brain injury decisions in the emergency department

convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year submit your research ?

Scope and influence of electronic health record-integrated clinical decision support in the emergency department: a systematic review

The value of participatory development to support antimicrobial stewardship with a clinical decision support system

The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support

What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation

Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review

A framework and model for evaluating clinical decision support architectures

Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings

Avoiding alert fatigue in pulmonary embolism decision support: a new method to examine 'trigger rates

Evaluating the alert appropriateness of clinical decision support systems in supporting clinical workflow

On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop

Smart Forms" in an electronic medical record: documentation-based clinical decision support to improve disease management

Clinical decision support: the experience at Brigham and women's hospital/partners healthcare

Deep learning for natural language processing in radiology-fundamentals and a systematic review

Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning

An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets

Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning

eHealth in modern patient-caregiver communication: high rate of acceptance among physicians for additional support of breast cancer patients during long-term therapy

Optimizing clinical decision support in the electronic health record: clinical characteristics associated with the use of a decision tool for disposition of ED patients with pulmonary embolism

Assessment of Electronic Health Record Use Between US and Non-US Health Systems

The HIPAA privacy rule: flawed privacy exposed when compared with the European Union's general data protection regulation

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

We thank Mr. Romario Smith for his assistance with the early focus group interviews. We also thank our participants for their time and thoughtful feedback while participating in this study.Dr. Randi Foraker is a member of the editorial board of BMC Medical Informatics and Decision Making.