Karen Levy Development of an Expert System for Classification of Medical Errors D. KOPEC a, K. LEVY a, M. KABIR b, D. REINHARTH c, G. SHAGAS a a Department of Computer and Information Science, Brooklyn College, New York, USA b Department of Health Informatics, Dalhousie University, Halifax, Ontario, Canada c Long Island Jewish Medical Center, New Hyde Park, New York, USA Abstract. The 1999 report published by the Institute of Medicine (IOM) indicated that between 44,000 and 98,000 unnecessary deaths per year occurred in hospitals alone, as a result of errors committed by medical professionals in the United States. There has been considerable speculation that these figures are either overestimated or underestimated. For example, the possibility that they focus on isolated injuries rather than error, or the majority of surveyed respondents did not know what constitutes a (medical) error. These disagreements have led experts to challenge the estimates of patient harm attributable to error, as well as the methodologies used to enumerate them. Of particular concern is the process used in the identification, classification and prevention of medical errors. There have been numerous attempts to develop classifications of medical errors, and currently an abundance of taxonomies exist to describe their mechanism. In previous research, (Kopec, Kabir, Reinharth, Rothschild & Castiglione, 2003) a new taxonomy of Medical Errors was designed by expanding the IOM classification. This model and its extension can be used as a blueprint for future design, development and implementation of an expert system for classification of medical errors. Effective classification can facilitate pattern recognition, and pattern recognition will help in understanding the nature, background and abatement of medical errors. Such a system’s goal will be to perform convincingly as an advisory consultant, exhibiting expertise on a par with and beyond human experts in specified domains. Despite substantial disagreement on the validity of the published figures for fatalities in hospitals in the IOM report, what is of importance is that the number of deaths caused by such errors is nonetheless alarming. The identification and classification of errors in medical care delivery is a very complex process, and this process can be facilitated and simplified by the implementation of an effective classification system. Keywords. Medical errors, expert systems, error theory, CLIPS, taxonomy Introduction Since the publication of the famous 1999 report by the Institute of Medicine (IOM) [1] there has been continued concern about the effective identification and classification of human medical errors. A number of papers have been published in the field, addressing the validity of the manner in which the estimated range of the number of deaths per annum due to medical errors was obtained [2, 3, 4, 5, 6]. Despite disagreements concerning the accuracy of the quoted figures, it is evident that it is impossible to quantify the full magnitude of the challenges to safety with certainty, as the health care sector does not routinely identify and collect information on errors [7]. It is clear that even with discrepancies between the estimates, the mortality rates strongly suggest that effective strategies need to be employed to reliably identify and classify errors. The development of an effective classification system will aid in reducing the occurrence of errors and thereby assist in improving the quality of patient care for the American Health Care System. 1. Research Goal In previous research, the original IOM taxonomy was extended and a new approach to the classification, distribution and updating of medical information was recommended [7]. The goal of this study is to design an Expert System, utilizing this extended taxonomy, which will effectively classify medical errors and serve as a testbed for healthcare practitioners. 2. Taxonomy of medical errors There have been numerous attempts to develop classifications of medical errors, and currently an abundance of taxonomies exist to describe the mechanism behind the types of medical errors. The following etiology of categories of medication errors was given by the American Hospital Association [7]: a) Incomplete patient information b) Unavailable drug information c) Miscommunication of drug orders d) Lack of appropriate labeling e) Environmental factors The five error types most often observed and reported by U.S. family physicians were : [8] a) Errors in prescribing medications b) Errors in getting the right laboratory test done for the right patient at the right time c) Filing system errors d) Errors in dispensing medications e) Errors in responding to abnormal laboratory test results “Errors in prescribing medications” was the only one of these five error types that was also commonly reported by family physicians in other countries [9]. In an influential 1993 report, Kohn et. al. [2] developed a classification of medical errors. This report classifies medication-related errors under “Treatment Error, etc.”, but in our analysis we found that medication-related errors and errors related to clerical procedure are abundant in medical practice. Therefore, we altered the original IOM classification by specifically identifying and addressing these two types of errors, by including the NCC MERP [10] taxonomy. We also classified ‘Errors Related to Diagnosis’ into three clinical subgroups (“delayed”, “missed”, and “wrong”, as opposed to the four subgroups of the IOM). Specifically, we have divided each type into subtypes, numbered according to the NCC MERP [10]. Human error [11] can be subdivided as follows: “A knowledge error referred to as a mistake occurs from inadequate or incorrect information. If the information is correct, but the wrong method of application is chosen, a rule error occurs, termed a lapse. An example of such an error would be an incorrect diagnosis. Finally, the plan may be good, but the performance is faulty, often from distraction or inattention. This is a skill-based error, termed a slip [11]. Reason [11] distinguishes between two kinds of human error – active and latent. Active errors are the ones which are immediately discernible, whereas latent errors are much harder to detect and may require considerable analysis to discover or understand.” Based on our previous research [7], following is the classification of human errors in Medical practice: 1) Errors in prescribing medication: Misuse by incorrect: medication, route, dose, administration Overuse – using too much of a drug or prescribing a drug when not indicated Underuse – failure to provide medication 2) Treatment Procedure(s) – other than by medication 3) Errors related to Clerical Procedures 4) Errors related to Diagnosis Delayed Diagnosis Missed Diagnosis Wrong Diagnosis 5) Preventative errors – delayed or no follow-up treatment are examples 6) Other Communication Failure Administration Problems The above approach for distinguishing between errors will be used in the design of the Expert System for the Classification of Medical Errors. 3. Cases of Errors The study of more than 235,000 error reports submitted in 2003 by 570 health care facilities was the largest ever performed by the U.S. Pharmacopeia [12]. In their findings, as the number of reported errors goes up, the percentage that causes patient harm has gone down. However, the findings that were remarkable are those indicating that electronic prescribing is creating new types of errors. “Computer entry” was the 4th leading cause of errors accounting for 13% (27,711) of the medication errors reported in 2003[12]. In contrast, illegible or unclear handwriting was the 15th leading cause and accounted for 2.9% (6,134) of reported errors[12]. It might be expected that handwriting would move down the list as computerization becomes more widely implemented, however what was occurring was a new type of error. So we can see that information systems and the medical field present a double- edged sword. On one hand they offer the possibility of tremendous improvements in terms of memory capacity, speed and general processing power, but if coupled with arcane data entry systems, serious new problems are created. Can we return to manual data entry? Not at all. More important, is that data entry systems develop “anti-debugging” components to reduce human error. Another example that emphasizes the enormity of the medical error scenario is seen in the article by, Myhre and D. McRure [7, 13], which compares studies of errors in blood transfusions presented as Table 1 below. Table 1. Summary and Comparison of Various Reports of Fatal Errors in Blood Transfusions Figure 1. Pie Chart representation of the information in Table 1. Table 1 illustrates that errors occurred due to drawing specimens from the wrong patient (error by medical technologist), change of specimen in the laboratory (error by laboratory staff or medical technologist), and transfusion of blood into the wrong patient (errors by physician/nurse). Figure 2 below Table 1 illustrates the same data in a pie chart format. The numbers in the columns of the Table should add up to 100% since they are percentages of fatal error in Blood Transfusions, but they did not in the original published table [13]. Hence we have determined that the second row labeled “Other” should have the figures shifted one column to the left, for example, under “Mythre”, the second row labeled “Other” should read 8(10%), under Honig & Bove it should read 4(10%) etc. This type of error can be committed by anyone including a variety of medical practitioners, in addition to physicians. Another incident involved the assessment of the impact on ordering errors when physicians stopped writing patient identifiers on requests for blood transfusions by hand [14]. Physicians, frustrated by the amount of time required to complete forms to order blood, wanted to eliminate the need for their handwritten patient identifiers, which were in addition to such information “stamped” on blood requests. This change was implemented, the blood ordering forms were modified accordingly, and after elimination of the handwritten identifiers in 1997, ordering errors increased from an annual rate of 1 in 10,000 to 6 in 10,000 blood requests by late 1999. Subsequently, clinicians were alerted by newsletter and the rate decreased to 3 in 10,000. However, the error rate did not decrease to its previous level of 1 in 10, 000 requests until mid-2001, approximately 2.5 years after reinstitution of the requirement for handwritten patient identifiers. The conclusion of this study [14] was that an obligatory second entry of demographic identifiers on a blood order requires ordering physicians to be given careful consideration to the identity of the patient receiving the blood transfusion, thereby reducing the likelihood of transfusion of an unintended recipient. 4. The Design, Development and Implementation of an Expert System for Classification of Medical Errors As seen from examination of the above scenarios, errors in the medical field can occur in many different ways, with potentially diverse, wide-ranging and hazardous effects. Just review of the varied errors related to fundamental areas such as medication, diagnosis, treatment procedures and clerical procedures in terms of their number, etiology and possible ramifications, is a complex domain. Consideration of the number of possible permutations of specific elements or actions in a health-related setting that might be classified as error(s), coupled with the large variety of “system” type errors, leads us to conclude that we are dealing with an enormous range of possibilities. Which expert in the medical field will be able to hold in his or her head all the possible combinations of signs, symptoms and treatments that have occurred for all possible medical conditions? Expert systems can be used for effective classification of a diverse range of possibilities, and many have been built to solve different types of problems. These systems are unique in that they can draw conclusions from a store of task-specific knowledge principally through logic or plausible inference [15, 16]. They are also called knowledge-based systems because they contain the same kind of rules used by human experts when they make decisions in their field of expertise [15]. The heart of an expert system is the powerful corpus of knowledge that accumulates during system building. The knowledge is explicit and organized to simplify decision-making; and the accumulation and codification of knowledge is one of the most important aspects [15]. These systems are not locked into any specific decision path and as a result can select from alternative paths in their search for a conclusion [16]. When there are domain experts and a substantial number of rules, more than the human mind can effectively recall with speed and accuracy, such a situation can be remedied by building an expert system. We intend to use the Expert System shell CLIPS to design this system. The system will classify errors based on a set of production or decision rules. Rule-based programming is one of the most commonly used techniques for developing expert systems. In the programming paradigm, rules are used as heuristics or rules-of-thumb, which specify a set of actions to be performed for a given situation. In the event that two rules match a given problem situation, the system will utilize a conflict resolution strategy to best resolve the tie based on the specified decision rules. For example, it could break a tie based on which rule is more specific, or which rule is shorter or based on “refreshing”, that is, rules, which had recently been done after conflict resolution, might not be used again for some time in favor of new rules. Figure 2 illustrates how this Expert System could assist in recognizing and classifying what is considered a typical problem in hospitals, that of not implementing a proper hand-cleansing protocol for medical personnel: Assume a patient had been admitted to a hospital due to complications with influenza, however, after what was considered an acceptable amount of recovery time, the patient showed no signs of improvement, and after assessment was found to be physically worse than he had been upon admission. Some of the questions that the system might ask would be as follows: Q Is the patient male or female? A male Q What is the patient’s age? A 43 Q Did the patient stay overnight in the hospital? A yes Q How many days was the patient in hospital? A 7 Q Give the number of medical staff that were exposed to the patient during his stay? A 22 Q Did the patient come into contact with staff through: medical devices, food trays, medicine dispensation? A yes Q Does the hospital have a hand-cleansing protocol for staff? A no The system, based on the rule constraints that it is designed with might continue with more questions to get more information from the user, and might respond with the following summary: The patient is likely to have contracted a hospital-acquired infection due to the absence of a protocol for the effective practice of regular hand washing by staff. This determination is based on a 75% degree of certainty based on the following elements: Patient was in the hospital overnight Patient was exposed to more than five members of staff Patient was exposed to staff through medical devices, food trays, etc. No hand-washing protocol for staff exists Figure 2. Medical error scenario and possible outcome on running through our Medical Errors Classification System Conclusion and Further Research The identification and classification of errors in medical care delivery is a very complex process, and this process may be simplified by the implementation of an effective classification system.[7] In order to reach a consensus on the classification of medical errors, it is necessary to develop a generally accepted international medical error classification system. Findings have indicated that errors are likely to affect patients in similar ways in countries with similar healthcare systems [17]. With this taxonomy, the major types of errors can be categorized and each major type can then be associated with a specific underlying mechanism. This can explain why and even predict when and where an error will occur [18], which in turn will assist in the generation of intervention strategies for each type of error, and also assist in the reduction and abatement of medical errors. References: [1] Brennan T.A., The Institute of Medicine Report on Medical Errors: Could It Do Harm? Tex Med. 96(6), 13-15, 2000. [2] Kohn, L.T., Corrigan, J.M., and Donaldson, M.S. (eds.), To Err is Human: Building a Safer Health System, National Academy Press, Washington, DC, 2000. [3] Spencer, F.C., Human Error in Hospitals and Industrial Accidents: Current Concepts, Amer. Col. Of Surgeons 191:410-418, 2000. [4] Ballas, E.A., Information systems can prevent errors and quality. J. Am. Med. Inf. Assoc. 8:398-399, 2001. [5] Bates, D.W., Reducing the frequency of errors in medicine using information technology, J. Amer. Med. Inf. Assoc., 8: 299- 308, 2001. [6] Richardson, W.C., The Institute of Medicine report on medical errors, N. Eng. J. Med. 343:663-665, 2000 [7] Kopec, D., Kabir, M.H., Reinharth, D., Rothschild, O. & Castiglione, J.A., Human Errors in Medical Practice: Systematic Classification and Reduction with Automated Information Systems, Journal of Medical Systems, 27(4), 297-313, 2003. [8] Kopec, D., Shagas, G., Kabir, M.H., Reinharth, D., Castiglione, J.A. & Tamang, S., Errors in Medical Practice?: Identification, Classification and Steps toward Reduction, Medical and Care Compunetics 1 (Eds. Lodewijk Bos, Swamy Laxminarayan and Andy Marsh). Proceedings of the 1st ICMCC (International Congress on Medical Care Compunetics), The Hague, The Netherlands, June 2–4, 2004, pp. 126-134. [9] Dovey, S.M., Meyers D.S., Phillips R.L., Green L.A., et al, A preliminary taxonomy of medical errors in family practice. Qual. Saf. Health Care 11:233-238, 2002. [10] National Coordinating Council for Medication Error Reporting and Prevention. NCC MERP taxonomy of medication error. http://www.nccmerp.org/pdf/taxo1999-05-14.pdf [11] Reason, J. Human error, Cambridge: Cambridge University Press; 1990. [12] Robeznieks, A., Data Entry is a top cause of Rx Errors, American Medical News, pp. 14-15, January 24, 2005. [13] Myhre, B.A., and McRure, D. “Human error- A significant cause of transfusion mortality”. Transfusion, 40:879-885, 2000. [14] Moore, SB., Floss, ML., Ordering Blood for the Wrong Patient – Getting Inside the Minds of Ordering Physicians, Mayo Clin Proc. 78:1337-1339, 2003. [15] Waterman, D., A Guide to Expert Systems. [16] Webster, R., Miner, L., ExpertSystems – Programming Problem Solving, 1982. [17] Makeham, M.A.B., Dovey, S.M., County, M., Kidd, M.R., An international taxonomy of errors in general practice: A pilot study, Med. J. Australia 177: 68-72, 2002 [18] Zhang, J., Patel, V., Johnson, T. and Shortliffe, E., A cognitive taxonomy of medical errors, Journal of Biomedical Informatics 37: 193-204, 2004. http://www.nccmerp.org/pdf/taxo1999-05-14.pdf Development of an Expert System for Classification of Medica a Department of Computer and Information Science, Brooklyn C