key: cord-0855071-g7slb00a authors: Kim, Sira; Kim, Eung-Hee; Kim, Hun-Sung title: Physician Knowledge Base: Clinical Decision Support Systems date: 2021-12-09 journal: Yonsei Med J DOI: 10.3349/ymj.2022.63.1.8 sha: 9a7819c032bb8c41644449b87c9e47f72357b6ce doc_id: 855071 cord_uid: g7slb00a With the introduction of electronic medical records (EMRs), it has become possible to accumulate massive amounts of qualitative medical data. As such, EMRs have become increasingly used in clinical decision support systems (CDSSs). While CDSSs aim to reduce medical errors normally occurring in the process of treating patients by physicians, technical maturity and the completeness of CDSSs do not meet standards for medical use yet. As data further accumulates, CDSS algorithms must be continuously updated to allow CDSSs to perform their core functions. Doing so, however, requires extensive time and manpower investments. In current practice, computational systems already perform a wide variety of functions in medical settings to allow medical staff to focus on other tasks. However, no prior research has evaluated the potential effectiveness of future CDSSs nor analyzed possibilities for their further development. In this article, we evaluate CDSS technology with the consideration that medical staff also understand the core functions of such systems. reduced the probability of misdiagnosis by means of diagnostic assistance. [10] [11] [12] The implementation of CDSS has been able to minimize drug side effects over the course of treatment, which could affect economic outcomes by reducing associated medical costs. 13, 14 Many factors need to be considered to establish decision support systems in the medical field, as opposed to general decision support. To effectively apply machine learning to medical fields and to derive clinically useful results for patients, several important issues remain to be considered by medical staff. 9 To support this process, medical information should be shared between hospitals and widely standardized. 15, 16 However, physicians are expected to require a broad practical understanding of CDSSs, and the development of such understanding may be seen as a key factor in the further development of evidence-based CDSSs. Presently, it is necessary to develop a CDSS that can be used clinically, rather than one for only research purposes. The recent trend has been to supplement the limitations of medical data and to effectively use AI methods and CDSSs. Eventually, providing personalized treatment for disease management by increasing the efficiency and quality of disease management through improvements in such automated systems is expected. 17 In this article, we discuss various clinical use cases of CDSSs in the medical field and numerous basic principles for incorporating such methods into medical practice. CDSSs that employ artificial intelligence (AI) methods have shown great promise in the medical field. Currently, there are two types of CDSSs reported in the relevant literature, "knowledge-based CDSS" and "non-knowledge-based CDSS. " 18, 19 In the case of non-knowledge-based CDSS, AI or machine learning methods that apply supervised and unsupervised learning approaches are often used. We would like to comment on these supervised and unsupervised learning approaches. Supervised learning methods involve inputting correct answers in advance and training the models based on accumulated data. 20 In supervised learning, accurate input data (features) and output (labels) must be available (Table 1) . 21 For example, a formula might express the meaning, "If HbA1c is more than 6.5% (input data), it is diagnosed as DM (DM labeling). " Classification and regression techniques have been employed in supervised learning. 20 Classification methods operate by means of a well-known rule base; such methods classify and divide data according to a predetermined algorithm. Decision trees are the most representative operation methods, their process continuously repeating until a final diagnosis is made. By contrast, regression methods predict future results by discovering features or patterns in accumulated data. Logistic regression has been the most widely used approach of this type and, accord-ing to some studies, has shown the capacity to predict sufficiently high-quality results without employing complex machine learning or deep learning. 22 Ultimately, this type of rule-based expert system has the disadvantage of requiring direct data input to perform predictions. Therefore, the quality of the input data is important, and it is also crucial to obtain groups with correct outputs. 23 Owing to these challenges, interest in unsupervised learning has recently been increasing. In contrast to supervised learning, unsupervised learning does not specify an output (Table 1) . 21 That is, such learning methods make predictions by clustering similar data rather than by using systems of rules. Clustering methods are typical in unsupervised learning. 24 Because unsupervised learning must identify patterns or shapes in unlabeled data, large amounts of very high-quality data are required. In general, unsupervised learning methods are promising in cases where it may not always be possible to identify correct labels or categories for data. For example, when diagnosing cardiovascular complications in DM patients, the operational definition is not well defined. In this case, unsupervised learning might classify the characteristics of cardiovascular complications into various groups. Patients who have been treated by cardiovascular specialists may be grouped into a single category, or patients who have had cardiovascular CT scans may be grouped separately. Cardiovascular complications in DM patients can also be classified (defined), along with records of anti-diabetic medication usage. Therefore, unsupervised learning is more helpful in classifying a specific disease. 25 In addition, when there are many features (especially three dimensional) in data, it becomes difficult to analyze and visualize the data realistically. A method to reduce problems caused by such a large number of features is called dimension reduction. It cannot be said with certainty that most of the CDSSs currently used in the medical field use unsupervised learning based on big data or AI. Rather, most CDSSs currently used in the medical field are simply rule-based systems composed of "If-, then-" structures constructed by supervised learning (for example, if baseline HbA1c exceeds 9.0%, consider insulin treatment). 26 This suggests that the technical maturity and completeness of unsupervised learning systems do not yet meet standards for medical use. Notwithstanding, CDSSs in the medical field appear poised to transition from supervised to unsupervised learning. In order to increase the accuracy of unsupervised learning or the clinical use of CDSS, it is necessary for the amount of input data to increase. 23 However, as the input data increase, the usability of a system decreases, and if fewer data are input, the quality of a system decreases. The quality and quantity of input data remain key obstacles to CDSSs, even in the present era of accelerating development of ubiquitous computing power ( The transition from paper medical records to electronic medical records (EMRs) occurred relatively recently. With the introduction of EMRs, it has become possible to accumulate massive amounts of quantitative medical information and data, and subsequently, EMRs have developed an environment optimized for the use of CDSSs. 27 In other words, EMRs provide a supporting environment for CDSS methods to function properly: the creation of EMRs did not merely comprise computerizing and storing patient treatment and medical information, as the ability to effectively load CDSSs into EMRs was also an important consideration. 28 CDSSs have been explored in the US under a representative EMR incentive program known as "meaningful use. " 29 Positive results were demonstrated on the prevention of drug abuse, drug dose adjustment, drug allergy alarm, duplicate prescription prevention, and drug interaction alarms. The developed system reduced medical errors by alerting medical staff to these errors, who identified them based on machine knowledge and scientific evidence. Here, in this instance, the medical system itself, rather than the medicine, meaningfully improved the safety of patients. In Korea, many CDSSs have been applied to the medical field. The nationwide drug utilization review is a representative CDSS. 30 It lowers the probability of misdiagnosis in the diagnosis stage, it predicts drug side-effects in the treatment stage, and it detects and predicts changes in a patient's condition during the follow-up stage. In Korea, several attempts have been made to apply CDSSs clinically, and several drug-related CDSSs Table 1 . Schematic Diagrams of Supervised and Unsupervised Machine Learning Approaches. 20, 21 have been adopted. 31, 32 Additionally, hospital information systems have improved medical efficiency by computerizing medical information, such as a patient's past illnesses, current diseases or conditions, and treatment methods. In the COVID era, a function to register body temperature and respiratory symptoms of all employees in the system every morning has been added to monitor the risk of infection. The system can be used to monitor patient symptoms or changes in conditions in real time and to assist clinical staff in responding appropriately: as one example, medical staff are notified in real time if a patient has visited a country with a high risk of COVID-19 or a hazardous area in Korea. 33 Depending on how a CDSS is used, the quality of patient care can be greatly improved; however, it should be noted that inappropriate information can interfere with a patient's treatment. AI algorithms, which have recently become an issue, are expected to be used in CDSSs. 17 However, as patient data continues to accumulate, CDSS algorithms need to be continuously updated. To do so, data-based or rule-based algorithms, as a core function of CDSSs, must be continuously developed, managed, and updated. Extensive time and manpower investments are essential to do this, making it is necessary to form teams of both medical staff familiar with ICT and data analysts and ICT experts with a basic understanding of medical care. These multidisciplinary teams are expected to be of some considerable benefit in increasing the effectiveness and quality of medical care and reducing override. CDSS alerts are the most well-known CDSSs. 34 After the CDSS algorithm was created, it was installed as part of an EMR. If this algorithm is applicable during the process of treating patients, a CDSS alert is generated. Such alerts are mostly used in rela-tion to a patient's treatment process, especially in conjunction with the health insurance claim process. The most common examples include systems to support the timing of regular checking of bone mineral density for osteoporosis, DM complication examinations for DM patients, and vaccination. 35 This process is directly related to the profitability of hospitals, in addition to the function of systematically managing patient health. Information about medications prescribed by physicians is also a strong advantage of CDSS alerts. 36 Providing an appropriate dose of a drug prescribed to a patient and warnings about excess doses, drug interactions, history of previous drug allergies, and duplicate prescriptions are good examples. Moreover, if a doctor's prescription does not match the CDSS algorithm, the reliability of the algorithm can be assessed to determine whether the cause is related to the CDSS warning system or to the doctor's prescription pattern. Ultimately, CDSS alerts aim to reduce medical errors for physicians and to improve the safety of health care environments, as well as outcomes and medical processes. 37 While CDSSs serve many potentially useful functions, there are also many areas of concern (Table 3) . The most important issue is the possibility of too many CDSS alerts being sent. After checking the CDSS alert systems, healthcare professionals may accept or ignore this recommendation (override). 61 If a CDSS alert is accepted without issues by the medical staff, it is considered an appropriate alarm. In contrast, if an alert is considered meaningless or simply repetitive, the staff would continue to override it, which might degrade the utilization of the system. This is known as alert fatigue, 62 and it tends to occur when a CDSS creates too many alerts soon after being installed, decreasing its effectiveness. Further research is required to optimize the effectiveness of such CDSSs in terms of alert fatigue. As of yet, no evaluations have been performed on the There is also a need to study whether CDSSs have improved qualitatively in practice. It can, however, be difficult to evaluate whether a CDSS is necessarily meaningful based on the fact that medical staff have accepted a CDSS alert. Conversely, it is difficult to say that CDSS alerts are ineffective based on the assumption that they are overridden. This is because responses to CDSS alerts depend on the disposition and ability of medical staff. Therefore, the final goal will be to create a meaningful CDSS algorithm that can minimize CDSS alert overrides, because unnecessary alarms and overrides may reduce the work efficiency of medical staff. CDSSs should not be an extension of their work, rather they should serve as a support for the development of more advanced CDSS frameworks. Indeed, according to the results of the systematic literature on controlling diabetes and hypertension with CDSSs, 63 a variety of sources agreed that such systems were useful in promoting prevention, supporting treatment, and improving patient care. However, doctors claim that they have caused significant delays in everyday practice. Good examples of CDSSs used for patients in the fields of endocrinology and metabolism have mostly focused on screening, diagnosis, treatment, prevention, and prediction. 17 Based on EMRs, a study was conducted to develop an algorithm for screening gestational DM and for screening mothers who need to be tested before regular oral glucose tolerance tests. 64 Attempts have been made to predict risk in patients 30 min before hypoglycemia occurs and to link this algorithm to a Continuous Glucose Monitoring system and artificial pancreatic system. 65 A system for automatically classifying the severity of diabetic retinopathy through microaneurysms and hard exudates has also been developed. 66 Many studies have been conducted to predict the occurrence of DM, 67 as well as the risk of heart failure in DM patients. 8 Various types of CDSS have been introduced for this purpose; 8,64-67 however, most CDSSs used in practice have been related to drug treatment. Examples include alarms for patients with a history of adverse drug reactions and warnings for duplicate prescriptions of specific drugs, overdoses, and drug interactions. An alarm on the date of a periodic diabetes complication test is the most representative example of methods that have been applied to reducing the role of busy medical staff in a conventional treatment process. However, the treatment of DM varies depending on a patient's condition. Medical staff have to deal with a vast amount of medical data for patient management, and there is generally not enough time for in-depth analysis of every patient's data. 68 Thus, more active interest and participation among medical staff is needed. This is because the implementation of CDSSs, which are clinically useful, will eventually be achieved by active participation among medical staff. If sufficient data are accumulated and good algorithms are developed, CDSS may be expected to play a powerful role. Despite their association with AI, CDSSs are merely software programs. However, medicine is a discipline in which decisions must be made in the end, and this is solely fall on the shoulders of medical staff. 69 CDSSs are design to be a system of "decision-support, " not "decision-making. " Until now, in many cases, only medical staff has traditionally played a role in making medical decisions. Establishing that the process of medi- Lower specificity and ambiguous alert contents are associated with overrides and alert fatigue. 43, 44 Alert-related fatigue and physician burnout are very frequent among emergency physicians, which cause concern regarding the performance of a CDSS. [45] [46] [47] The majority of physicians and learners attribute EMR to their symptoms of burnout, even when they did not identify as being burned out. 48 Burnout leads to reduced quality of care 49 and medical errors. 50 Lower satisfaction and higher frustration with the EMRs are significantly associated with perceptions of EMR contributing toward burnout. 48 Solution Alert override patterns have focused on specific disease or alert types. [51] [52] [53] Systems should be implemented to enable analysis based on grade and potential harm and provide clear recommendations. 54 Suggested turning off frequently overridden alerts, 55 updating clinical content, and the need for consensus meetings between physicians and pharmacists. 56 Optimize alert types and frequencies to increase clinical relevance so that important alerts are not overridden inappropriately. 57 Machine learning algorithms were used to reduce alert fatigue by identifying physicians and departments who override alerts. 58 Identification of physicians and departments who override alerts will help increase benefits. 58 The impact of proficiency training leads to significant improvement in satisfaction, which could eventually reduce burnout. 59 Human-centered approach to physician burnout by reducing unnecessary administrative burdens. 48, 60 cal decision-making should be performed neither by a CDSS nor by medical staff alone, but by complementary integration of medical staff and computers, is expected to prove beneficial. Whether referred to as AI or CDSSs, it is expected that the further development of such systems will prove beneficial for patients and medical staff. Interventions to reduce medication errors in adult medical and surgical settings: a systematic review Medical error-the third leading cause of death in the US Institute of Medicine (US) Committee on Quality of Health Care in America. To err is human: building a safer health system Effect of computerized physician order entry and a team intervention on prevention of serious medication errors Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review The Office of the National Coordinator for Health Information Technology (ONC) Machine learning to predict the risk of incident heart failure hospitalization among patients with diabetes: the WATCH-DM risk score Medical diagnostic decision support systems--past, present, and future: a threaded bibliography and brief commentary Medication-related clinical decision support in computerized provider order entry systems: a review Clinical Informatics Outcomes Research Group. Computerized reminders reduce the use of medications during shortages Clinical decision support systems for drug allergy checking: systematic review A roadmap for national action on clinical decision support Medical big data is not yet available: why we need realism rather than exaggeration Clinical decision support functions and digitalization of clinical documents of electronic medical record systems Conceptual framework for a perinatal decision support system using a knowledge-based approach Design and evaluation of a knowledge-based clinical decision support system for the psychiatric nursing process Comparing different supervised machine learning algorithms for disease prediction Unsupervised learning of natural languages Scalable and accurate deep learning with electronic health records Revisiting unreasonable effectiveness of data in deep learning era Discriminative clustering via extreme learning machine Deep learning: a comprehensive overview on techniques, taxonomy, applications and research directions Expert systems: the technology of knowledge management and decision making for the 21st century Clinical decision support systems A primer for writing medical data base for the clinical decision support system ing in the federal incentive program Toward safer prescribing: evaluation of a prospective drug utilization review system on inappropriate prescriptions, prescribing patterns, and adverse drug events and related health expenditure in South Korea A study on user satisfaction regarding the clinical decision support system (CDSS) for medication Impact of pharmacists' interventions on physicians' decision of a knowledge-based renal dosage adjustment system Information technology-based management of clinically healthy COV-ID-19 patients: lessons from a living and treatment support center operated by Seoul National University Hospital Clinical decision support alert appropriateness: a review and proposal for improvement Use of clinical decision support to increase influenza vaccination: multi-year evolution of the system Evaluating a clinical decision support system for drugdrug interactions A machine learning-based clinical decision support system to identify prescriptions with a high risk of medication error Determining inappropriate medication alerts from "inaccurate warning" overrides in the intensive care unit Preventable medication errors: identifying and eliminating serious drug interactions Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review Alert override patterns with a medication clinical decision support system in an academic emergency department: retrospective descriptive study Overrides of medication-related clinical decision support alerts in outpatients Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support Burnout and satisfaction with work-life balance among US physicians relative to the general US population Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction Review article: burnout in emergency medicine physicians The influence of electronic health record use on physician burnout: cross-sectional survey Worklife and satisfaction of general internists Burnout and medical errors among American surgeons Overrides of medication alerts in ambulatory care Clinically inconsequential alerts: the characteristics of opioid drug alerts and their utility in preventing adverse drug events in the emergency department Prescription errors before and after introduction of electronic medication alert system in a pediatric emergency department A framework for evaluating the appropriateness of clinical decision support alerts and responses Turning off frequently overridden drug alerts: limited opportunities for doing it safely Using computer technology to detect, measure, and prevent adverse drug events Deriving dose limits for warnings in electronic prescribing systems: statistical analysis of prescription data at University Hospital Birmingham Machine learning approach to reduce alert fatigue using a disease medication-related clinical decision support system: model development and validation Advanced proficiency EHR training: effect on physicians' EHR efficiency, EHR satisfaction and job satisfaction Physician burnout: the need to rehumanise health systems Clinical decision support systems for drug allergy checking: systematic review Kaushal R; with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system Development and implementation of a decision support system to improve control of hypertension and diabetes in a resource-constrained area in Brazil: mixed methods study Prediction of gestational diabetes based on nationwide electronic health records Design and clinical evaluation of the Interoperable Artificial Pancreas System (iAPS) smartphone app: interoperable components with modular design for progressive artificial pancreas research and development Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs Development of non-invasive diabetes risk prediction models as decision support tools designed for application in the dental clinical environment Information overload in healthcare: too much of a good thing Decision-making in artificial intelligence: is it always correct?