key: cord-0859087-ykcl1kqv authors: Moulik, Supratik K.; Kotter, Nina; Fishman, Elliot K. title: Applications of artificial intelligence in the emergency department date: 2020-07-08 journal: Emerg Radiol DOI: 10.1007/s10140-020-01794-1 sha: dc8cb2f0414f67bbb74b48de3bf1ccf7e0b44504 doc_id: 859087 cord_uid: ykcl1kqv nan Over the past 5 years, there has been an increasing number of applications of artificial intelligence and machine learning (AI/ML) throughout various sectors of the economy. This wide-reaching impact has fueled global venture capital investments of around $83 billion since 2014 and nearly $26.6 billion in 2019, of which the single largest sector was healthcare with $4 billion [1] . These investments have yielded many impressive advances in varied fields ranging from autonomous driving through novel drug discovery. As the technologies continue to mature, the capabilities as well as the limitation of AI/ML are coming to the forefront and helping guide future development efforts. Despite enthusiasm in the press and among researchers, the adoption of new technologies in the medical space has historically been simultaneously enthusiastic and cautious. The introduction of new technologies into a complex system like medicine often yields unpredictable results. Despite this uncertainty, it is instructive to view new technologies through the lens of existing treatment paradigms to see how patient outcomes may be impacted and potentially improved. Personalized or precision medicine is a treatment paradigm most of us are familiar with at some level. Precision medicine focuses on patient-specific information in order to help guide treatment. With the advent of electronic medical record systems, physicians are confronted with a deluge of information during each patient encounter ranging from historical laboratory values, past clinic notes, past ED visits, imaging, and much more. It is not enough to simply be able to search the data in a patient record; relevant information needs to be identified to guide diagnosis and treatment. With time being the primary limiting factor in most physician practices, the goal of precision medicine in the emergency department has been waiting for a tool like machine learning to help make it a reality. Artificial intelligence and machine learning are generic terms which refer to a broad class of hardware and software programming techniques that allow computers to determine relationships between datasets and apply the learned relationships in a predictive fashion. This process is similar to how a medical trainee, when exposed to a series of patients with chest pain, will develop intuition and understanding that get applied to future patient encounters. In order to more fully understand AI/ML, it is instructive to contrast them with traditional computer programming techniques. A traditional computer program takes a set of known relationships/rules and converts them into logical operations or mathematical formulas. This results in a fixed reproducible relationship between input conditions and output. The process of converting a set of logical steps into mathematical operations is ideally suited for programs like word processing software or video games. Conversely, in situations where the logical steps and relationships are less well understood, such as scientific and cognitive computing problems, it is challenging to write a single set of rules that adequately represents the range of solutions. In other words, the complex interplay between variables in the real world is challenging to quantify through human experience and even harder to validate with the limited availability of reliable testing datasets. In general, machine learning is a process by which the input and output of a system are provided, and a computer is tasked at finding the underlying relationship between the two sets of data. For example, if a patient presents with a set of symptoms and routine laboratory values, those data points would be considered the input data. The output of the system would be either the final diagnosis or the treatment plan. Given the complex nature of such a decision-making process, a machine learning system would need to be able to consider linear and non-linear relationships as well as dependent relationship between various input data. In addition, the training system would need to be provided with sufficient data/examples to filter out bias introduced by individual physician treatment styles and noise inherent to real-world data. In this example, a modern machine learning system should be able to find the relationship between input and output data as long as the output can be categorized as a finite set of possible diagnosis/ treatment plans. As the capabilities and limitations of modern AI/ML techniques are becoming understood, the potential applications are becoming clearer. It is important to distinguish between the kinds of tasks that can be solved with AI/ML in the near term, which tend to be focused applications, and the tasks we want solved, which tend to be complex and cognitive in nature. For example, ER physicians routinely perform repetitive manual tasks which require very little focused attention and seem to be appropriate targets for automation. In truth, these tasks are typically only simple when viewed through the lens of human experience and understanding. The cognitive portion of the analysis, informed by years of education, experience, and training, is easily discounted as they are difficult to quantify in context. The role of the machine learning system and AI in the emergency department can be broken down into three general phases of implementation. There are commercially available systems as well as systems under active development, which are at various stages of readiness for testing and clinical implementation. There are more complex systems based on existing technologies but outside the scope of current development; these are considered near-future technologies, which we can expect to see within the next 10-20 years. Finally, there are speculative systems which are largely outside the practical limits of existing technologies; these will require an indeterminate amount of time to develop because they require one or more paradigm shifting breakthroughs. Though the fully automated generation of reports is an oftencited goal, the practical near-term applications include worklist prioritization and focused diagnostic support. Current-generation machine learning system in the medical field is largely focused on limited-scope diagnostic aids in radiology, cardiology, and pathology. Examples of FDA cleared systems include breast imaging interpretative aids, brain bleed detection on CT, lung nodule and coronary calcification detection systems, and echocardiography tools. Existing AI/ML systems tend to be focused on problems for which a solution is readily achievable within the timeframe of a product cycle or a startup funding runway. This has resulted in systems with varying degrees of clinical utility in the emergency department. In the case of chest X-rays, there are existing systems which can identify the presence of common pathology like pulmonary tuberculosis nodules, pneumonia, pneumothorax, and rib fractures with stated sensitivity and specificity of 85-95% and world number generally ranging between 70 and 90% depending on the specific pathology targeted and the datasets used for testing [2] . Though these systems are generally insufficient for treatment planning, they can be very helpful in preselecting exams for priority interpretation by a radiologist. This is especially critical in the ED setting where turnaround time is important not only for performance metrics but also for timely interventions. Major manufacturers of medical systems are also starting to embed AI/ML systems into medical devices. Focused systems for pneumothorax detection can be found in some portable radiograph machines, which provide near-real-time alerts to both clinicians and radiologists regarding the potential presence of pneumothorax [3] . The robustness of these systems and their utility for early detection of pneumothorax in ICU and ER patients are likely to make them standard in nextgeneration radiography systems. Similarly, ultrasound vendors have started incorporating AI/ML systems to improve reproducibility of ultrasound scans, though these systems generally lag behind the radiographic systems in robustness and clinical usefulness [4] . Advanced imaging modalities such as CT and MRI provide a more challenging use case for modern machine learning systems mostly due to issues related to the size of these studies, spatial resolution, image noise, and the time-consuming nature of data labeling. There are existing products which will analyze studies and provide alerts to radiologists regarding the presence of findings such as lung nodules, ICH, and other common pathology as well as provide pertinent management suggestions. The utilization of these systems will not initially be readily apparent to the ED physician, though the improved detection rates for important pathology will benefit patients and hospital systems alike. One of the first use case of machine learning systems to reach product phase is intracranial hemorrhage detection algorithms. These systems are generally able to meet or exceed the sensitivity of an expert reader in detecting small bleeds [5] . Similarly, there are available AI/ML systems for detection of pulmonary embolism on CT pulmonary angiogram studies [6] . Though these are both limited-scope applications, they address clinically relevant findings which generally require time-sensitive treatment and have significant drawbacks in the case of false-negative interpretation by human readers. Once a patient has left the hospital, it is incumbent on a health care system to ensure appropriate follow-up care. Between subspecialty follow-up appointments and imaging follow-up recommendations, closing the loop on an emergency department visit is an area of active development with modern advanced computing systems. There are existing platforms which utilize Natural language processing (NLP) and AI/ML to analyze radiology and pathology reports in real time to ensure existing best practice guidelines are being utilized and provided to ordering physicians [7, 8] . These systems not only guide the report generation process but also have limited ability to monitor outpatient compliance and provide alerts/ reminders to patients and primary care physicians to encourage adherence to follow-up guidelines. Machine learning in the near future The AI system which we can expect to see in the near future is going to be based on similar underlying technologies to those that are currently available. With the maturation of training techniques, advances in computing systems, and expansion of available training data, systems that are theoretical or impractical today will become achievable. Despite the recent high-profile failure of a diagnostic support system power by IBM in conjunction with the MD Andersen Cancer center, the underlying goal and approach are not necessarily flawed. In fact, it is likely that with a less aggressive timeframe and continued refinement of the systems and training data, the overall goals will be achievable. Similar systems are under development for more general diagnostic settings such as emergency departments and urgent care clinics and will likely begin to see implementation within the current decade. As with imaging support system, the first wave of AI/ML systems for the ER physician will be focus built systems that address specific steps in the patient encounter. For example, patients who presents to the ER needs to be rapidly triaged to appropriate acuity level, and when possible, the relevant diagnostic tests need to be ordered. In emergency departments the EMTALA screening is often performed by a midlevel practitioner who will assess the patient and order initial imaging and laboratory tests. This process is labor intensive and inherently limited in the ability of a practitioner to ingest and process a patient's history, imaging, and prior labs in order to put the current visit in context. A rudimentary form of automation is used in these scenarios in the form of standard order sets that the midlevel practitioner can select based on presenting complaint and assessment. Though expedient and relatively easy to implement, this process over-utilizes imaging and laboratory resources by creating overly broad categorizations of patients. This "use case" is an example of the type of problem current-generation machine learning systems are equipped to address, given enough training data including access to history medical records. These AI/ML systems will be integrated into the existing workflow to augment the capabilities of practitioners. While a PA is doing the triage screen, a machine learning system will be scanning through the medical history, pulling relevant diagnoses and laboratory data. For a patient coming in with chest pain, the practitioner will be presented with historical graphs of WBC and cardiac enzymes as well as recent imaging and pertinent surgical history. Combining the historical data with the newly acquired screening data, the machine learning system can then give a recommendation of acuity level (1-critical through 5-routine) as well as recommended laboratory and imaging tests. The decision of what imaging test is most appropriate in each situation is influenced by many factors, some of which are highly subjective. The issue of over-and under-utilization of imaging resources is something that every clinician and radiologist struggles with. The desire to not miss acute pathology is balanced with the potential detriment of excessive radiation dose to susceptible populations. There likely exists a combination of historical and objective factors which can predict outcomes with sufficient sensitivity and specificity to guide the ordering pattern of most physicians. There are existing ordering support systems which provide basic functionality such as cumulative radiation dose and application of appropriate use criteria based on the provided indications. In the future, we can expect these systems to be expanded to include historical, laboratory, and exam data to further guide our ordering patterns. Predicting future developments in technology is challenging at best, because it requires understanding the capabilities of technologies that have yet to be invented. For this reason, it is more instructive to speak in general terms about how AI/ML systems of the future will relate back to patient care paradigms which we are familiar with. Current and near-future AI/ML systems are, for regulatory and feasibility purposes, relatively static in their implementation, though the goal is to have continuously learning systems which can take advantage of global as well as local diagnostic and treatment patterns. Through these pathways, precision medicine in the emergency department will be an ongoing feedback loop of technical innovations guided by clinical factors. Take for example the COVID-19 pandemic; the treatment and outcomes experience changes based on regional, genetic, and societal factors in addition to the evolving milieu of viral strains. Treatment experience has produced broad categories of patients who are considered "high risk," though these guidelines have already proven to be incomplete. In essence, we are experiencing the difference between descriptive models which give statistical information about large groups of infected individuals and predictive models, which can tell you the outcome for a specific infected individual. In the future, predictive models will be trained on clinical, treatment, laboratory, and genetic data of individuals to facilitate personalized treatments. Machine learning systems are uniquely equipped for finding groups and subgroups that require more aggressive management. The goal of incorporating viral and host genetic data will require significant advances in computing and genetic sequencing. On the other end of the spectrum is the consideration of regional demographics. An AI system trained on data from a French insurance conglomerate or from the British NHS would likely not demonstrate the expected level of performance when applied to a hospital in Southeast Asia or rural Wyoming. Take for instance the diagnosis of TB for a patient presenting to the ER with cough and fever. An AI system that is trained on data from Arkansas would likely give low probability to the diagnosis of TB, whereas an AI system trained on data from India would and should have completely different performance characteristics. This is not unlike the issue we face as human practitioners; our diagnostic patterns are heavily influenced by our training and practice, so much so that the term "AI System" in this paragraph could be replaced with medical student and the rest of the discussion would remain valid. In addition, the current dissemination pattern for this type of knowledge is archaic and could benefit greatly from standardization and automation. AI/ML systems function through the use of mathematical models which can be trained on various datasets. Systems are being established for allowing general AI/ML models to be modified to better reflect localized communities through the use of techniques known as transfer learning and federated learning. In the future, AI/ML systems will likely get continuous or frequent updates of their global models while being able to maintain the specialized training that allows a general model to be broadly applicable within a narrow subgroup or community. The path towards true AI/ML implementation in the emergency department is the same as for other medical and nonmedical fields of study. Going to the marketplace today will provide access to systems that are clinically relevant though narrowly applicable. As the systems mature, some routine patient care tasks will become automated while decisionmaking in cognitive tasks will be augmented by AI/ML systems. It is important to keep the evolution of the AI/ML technology in context so as not to become overly enthusiastic about the current capabilities and simultaneously not to become overly pessimistic about future developments. Though the promised delivery date of fully self-driving cars has continuously been pushed back for the past decade, it is undeniable that drivers in semiautonomous vehicles are safer than unassisted drivers. Similarly, there are tangible patient care and cost benefits to be obtained through staged development of AL/ML systems even if fully autonomous MD systems are not on the horizon. AI In Numbers Q1'20: Global Funding, Corporate Activity, Partnerships, And R&D Trends Using artificial intelligence to read chest radiographs for tuberculosis detection: a multi-site evaluation of the diagnostic accuracy of three deep learning systems Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration FDA Clears the World's First AI Solution for Flagging Pulmonary Embolism Make it Easier for Radiologists to Practice Better How AI is advancing the impact of radiology in healthcare Conflict of interest None.