key: cord-0295682-uj0xkjk5 authors: Laus, F. W.; Pegoraro, F.; Loures, E. d. F. R.; Santos, E. A. P. title: Interoperability Assessment of Emergency Department Processes Based on Multicriteria Decision-Making Methods date: 2022-02-22 journal: nan DOI: 10.1101/2022.02.21.22271273 sha: 2b1eb46bc7dbb1ae223152381c9c9219f77c80ec doc_id: 295682 cord_uid: uj0xkjk5 It is noticeable that, because of the amount and quality of information exchanged and the criticality of the decisions guided by them, hospitals are considered as some of the most complex organizations in modern society. Evidencing it, emergency departments in hospitals are examples of such complex environments that need perfect integration among systems, people, departments, and data. The processes behind the Emergency Department (ED) routine cause a set of critical and time-dependent decision-making, which must consider several criteria related to organizational and clinical attributes. Based on the literature and worldwide initiatives related to managing complex organizations, an ED can be interpreted through the enterprise interoperability (EI) lens, a concept related to the capability of different systems to work collaboratively within and throughout the organization. Thus, this paper proposes a framework focusing on interoperability assessment in ED processes, where different actors need to interoperate. The proposed framework for Enterprise Interoperability Assessment (EIA) is based on multicriteria decision analysis (MCDA) methods, considering organizational and clinical attributes of the ED. The Analytic Hierarchy Process (AHP) and Preference Ranking Organization Method for Enrichment Evaluation II (PROMETHEE II), in an integrated approach, were elected as the most appropriate multicriteria methods to support, respectively, diagnostic (interoperability assessment) and decisional (interoperability improvements) processes in this background. Last, to validate the framework proposed, a case study was carried out at a stroke unit in a University Hospital (UH) in the south of Brazil. The outcome inferred that the UH achieved the intermediate level of enterprise interoperability in stroke treatment, and Business and Information Technology concerns were found to be the priorities, which deserved greater efforts, to enhance interoperability. alternatives, an attribute is already delivering sufficient performance or needs exchange information with each other and use this information" [10] . Of late, this concept 149 has received a broader meaning, so that enterprise interoperability may be understood as 150 the capability of inter and intraorganizational systems to work together with mutual 151 objectives [11] . [21] highlight that systems should be understood from a generic 152 perspective, wherein a system could be a stakeholder, an enterprise, a department, an 153 information system, or a resource. This understanding enables one to comprehend the HC . 164 Following this, [23] proposed another model, which divides interoperability into human, 165 non-human, and heterogeneous systems, and then evaluates overall enterprise 166 interoperability. Thus, these authors highlighted human systems for the academic 167 community inclined toward interoperability. 168 Further, [24] , in a literature review, identified that none of the existing models with 169 maturity levels at par with those of interoperability could verify the existing weaknesses 170 that impair interoperability. Therefore, they proposed other concerns that would best fit 171 human-dependent contexts: (i) business (strategic aspects such as culture, vision, values, 172 and strategy policies); (ii) process management (related to working methods); (iii) human . CC-BY 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint resources (skills, competencies, roles, collaboration, and social culture); (iv) policies and 174 procedures (normative aspects such as standards and guidelines); (v) information 175 technologies (interconnecting applications, data, and communication); and (vi) semantics 176 (the understanding of different terminologies). 177 Healthcare interoperability frameworks in the literature are termed as E-Health, 178 Health Information Systems (HIS), and Personal Health Systems (PHS) [25] . However, 179 these preexistingframeworks only focus on health records and hospitals strategy, not The MCDA (multicriteria decision analysis) is a discipline that evaluates multiple 185 conflicting criteria in decision-making, facilitating the choice of a solution in scenarios 186 with several qualitative and quantitative variables, useful for benefit-risk analysis. As the 187 volume and complexity of health data, information, and knowledge is growing, the 188 MCDA has become invested with more potential in optimizing decision-making at any 189 level of HC [26] [27] [28] . Its popularity in HC is explained by the large variety of attributes 190 that exist in such a context [29] . The bibliometric analysis conducted by [29] , which 191 reviewed 66 papers on the MCDA in the HC field, revealed that 39% of them used the 192 MCDA aiming for the "diagnosis and treatment" subarea, and the second and the third 193 subareas more prevalent were "priority setting" and "health technology assessment," both 194 representing 12% each. [30] reasoned that MCDA could be applied to any field of 195 emergency medicine. Regarding public health, [28] affirm that, recently, MCDAs have 196 been more utilized because, conducting a systematic scientific empirical assessment . CC-BY 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The AHP allows to model some problem in a hierarchical structure, deploying it 216 downward from a goal, subsequently branching it into criteria and subcriteria and, in the is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint for , as illustrated in Table 2 . = (( á -) ∕ ( -1)) / ) (4) is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The mathematical foundations are summarized as follows [19] . is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The final ranking is obtained by using Equation 13. Systems are intrinsically linked to the enterprise interoperability concept. In sum, a 306 system is a set in which a limit can be established, and the elements within it are the 307 attributes that characterize and differentiate it from others. Thus, attributes are the 308 elements that define the structure of a class that allow the qualification and relation of 309 confidence and commitment between interoperable systems [25] . Following this 310 rationale, it is necessary to identify the attributes that define critical HC processes. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint by [23] and later reinforced by [24] . The PHS is divided into two structures: execution 314 and technical aspects; legal and organizational aspects [25] . Based on this distinction, HC 315 attributes could be divided into organizational aspects related to legal and organizational 316 matters and the clinical aspects referring to treatment execution and techniques. The 317 attributes existing in a HC process directly interfere with its clinical outcomes as well as 318 face interference by enterprise interoperability. In this sense, this research proposes a framework capable of diagnosing the 320 interoperation capacity and provide guidance on how to improve it (i.e., a set of actions 321 suggested to improve interoperability to optimize clinical outcomes). The framework is 322 subsequently based on two stages: (S1) diagnosis of the current scenario of The framework and its stages could be performed after defining these key elements. Further, both stages employed MCDA methods. The first one, (S1), used AHP, 334 which performed the EIA, pointing out a level of interoperability corresponding to the 335 "as is." In sequence, the second stage, (S2), utilized PROMETHEE II, based on S1, to 336 weight the criteria, and its outcome was a ranking of interoperability concerns, which . CC-BY 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint prioritized the attributes that led to a more advanced level of interoperability. Thus, S2 338 helped decide which attributes must be prioritized in terms of efforts for improvement. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Technology and information systems existing in automated processes Table 6 . Interoperability levels and their description. The institution has no or insufficient capacity to address clinical emergencies, being unable to support the whole patient care. There is a variation in its processes and practices, and it depends on analogical systems. The institution admits and manages clinical emergencies, but is unable to dispense the best existing options of treatment in the whole patient care. It seeks reducing variability of its processes and increasing collaboration and integration and has complex systems and processes governed by a central body. An institution at this level works with extremely developed policies and resources. Moreover, it is constantly innovating and seeking partnerships. Its processes are effective and concrete, designed with the purpose of achieving satisfactory results in the treatment of patients admitted in emergencies. Adapted from [45] . The AHP's goal is represented by the EIA itself; the criteria are the The pairwise comparisons between elements within the same AHP layer were 400 made in the upper part of the matrices, following the Saaty Scale (Table 1) is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint know-how to judge OAs from the interoperability concern helped with both judgements 409 (Fig 3-B and 3-C) . The second matrix had the OAs in its rows and CAs in its columns; thus, it gave 415 way to pairwise judgements between CAs (Fig 4-A) and relationship judgements between 416 clinical and organizational attributes (Fig 4-B) . When incorporating the elements into the AHP hierarchical structure, only "very 421 strong" (weight 9) relationships were considered. Tables 8 and 9 list these correlations. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint In sequence, the design of the AHP allows for the EIA to proceed. It is necessary 430 that an HC professional, directly involved in the assessment process, evaluates the CAs. To facilitate this assessment by professionals who are not used to interoperability 432 semantics, a questionnaire is employed, whose answers must be translated into the 433 pairwise judgements of the AHP (46) . In this way, the questionnaire respondent must 434 evaluate their satisfaction regarding the performance of the CAs and answers should be 435 translated into interoperability levels to best suit the AHP design in such a way that 436 attributes with "unsatisfactory," "partially satisfactory," and "very satisfactory" 437 performances fit, respectively, into Basic, Intermediate, and Advanced levels of is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint For each concern, two indices must be calculated: the positive flow + and the 471 negative flow -. The symbol + expresses how much a concern needs to overcome the 472 others by, while the symbolindicates how much its performance is already sufficient 473 by to deliver the desired level of interoperability [47] . A sensitivity analysis function 474 enables an understanding of how to make the improvements, since it relates the concerns' 475 changing behavior to changes in OA prioritization. As a result, a sensitivity analysis is 476 relevant to indicate which OA should be prioritized. 478 The framework proposed by this paper was applied in a UH. This UH is one of 479 the largest hospitals in the south of Brazil and has a high volume of patients admitted in 480 the ED, including some with ischemic stroke symptoms. Even though the UH has a 481 residency program in neurology and neurosurgery, it has acknowledged that its 482 performance in stroke patient management can be improved further. The most efficient therapeutic procedure to avoid increasing the infarcted brain is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Facing a complex multi-criteria decision analysis, the AHP points out the part of is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint suggests performing a sensitivity analysis to understand how and when the current 518 interoperability level could be superimposed by another. Analyzing sensitivity in 519 decision-making methods targets the evaluation of the behavior of "alternatives" 520 whenever there is a change in the "criteria" input data. Thus, in this case, it evidenced the 521 influence that each CA had on the interoperability level. The darker curves represent the advanced level of enterprise interoperability, the medium-colored ones 531 represent the intermediate level, and the lighter ones, the basic level. Thus, for example, if the prioritization of Clinical Attribute 1 (CA 1 -Fig 8) is 533 maximized, the attribute will leverage the global level of EIA (because the line that 534 represents the advanced level will be in its upper half), but if its prioritization is 535 minimized, the current overall EIA level (intermediate) will be maintained. On the other hand, if the Clinical Attribute 3 (CA 3 -Fig 9) receives the maximum 539 or minimum of investment and efforts, even though more investments will tend to an . CC-BY 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint advanced level, its EIA level will vary insignificantly. All the CAs can be analyzed 541 following this rationale. After the AHP sensitivity analysis was performed, it is notable that some CAs 543 leveraged the global level of enterprise interoperability. These attributes are those that, 544 the more prioritized they were, the more they tended to rise to the advanced level, and, at (following Eq. 14), shown in Table 10 . is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint for each concern in order to obtain the results exhibited in Table 11 . 562 Since + expresses how much a given concern needs to be prioritized to overcome 564 the others and -indicates by how much its performance is already sufficient to achieve 565 the desired level of interoperability, it is possible to infer that those requiring more is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint at a satisfactory level of interoperation and efforts for improvement must be invested in 580 the other concerns. Fig 11 represents the sensitivity analysis that evaluated OA 2, OA 12, 581 and OA 13. In these three sensitivity analyses, the prioritization values were changed until a 585 "turning point" was found, i.e., a percentage of prioritization of the OA that resulted in a 586 significant change in the concerns. The first attribute analyzed was OA 2 -Electronic 587 Medical Record, whose priority was changed from 0 to 25%, the "turning point." This 588 change in OA 2 impacted the Process Management (PM) concern, turning it into the new 589 strategic focus. The second attribute analyzed was OA 12 -Compliance between practice and 591 standards. Its weighting was increased from 2% to 47%, with the others decreasing is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint assisting patients with stroke symptoms and striving to decrease its process variability 608 levels and increase the integration among its sectors. As per the proposed classification, 609 its complex systems and processes are centralized by a single entity (information system), The S2 pointed out that the interoperability concerns that needed more 620 prioritization were Business (B) and Information Technology (IT). As previously defined, is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Aggregating this analysis is suggested as future research since the MCDA approach 651 identifies the cause-effect chain between elements. It is also important to note that the design of the proposed framework can be 653 adapted (in relation to CAs) and applied to other processes in the HC domain. Thus, for . CC-BY 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint future works, it is also suggested that such assessments are carried out in other hospital is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint The ATHENA Interoperability Framework. In: Enterprise Interoperability II. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 22, 2022. ; https://doi.org/10.1101/2022.02.21.22271273 doi: medRxiv preprint A support 661 framework for decision making in emergency department management Interoperability analysis in critical 664 collaborative processes: an application to the healthcare sector for complex patient 665 assistance Lean: increase efficiency in stroke 667 patient care Optimising the complete care pathway for cerebrovascular accident patients Integrated Stroke Care in the Netherlands a Benchmark Study. International 673 Journal of Integrated Care The effect of COVID-19 on stroke 678 hospitalizations in New York City. Journal of Stroke and Cerebrovascular 679 Diseases Teams, tribes and patient safety: overcoming barriers 681 to effective teamwork in healthcare Effective communication and teamwork promotes patient safety Standard 686 Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries Interoperability assessment: A 689 systematic literature review. Computers in Industry Analysis by West Health Institute finds medical device 700 interoperability could save more than $30 billion a year. WEST HEALTH Press 701 Releases The Joint Comission. Most Commonly Reviewed Sentinel Event Types. Oakbrook 703 Terrace Interoperability 705 Assessment in Healthcare Based on the AHP/ANP Methods Sustainable supplier selection in 707 healthcare industries using a new MCDM method: Measurement of alternatives 708 and ranking according to COmpromise solution (MARCOS). Computers & 709 Industrial Engineering Promethee Methods. In: Multiple Criteria Decision 711 Analysis: State of the Art Surveys Hybrid MCDM Model for 713 Improving the Electronic Health Record to Better Serve Client Needs Framework for Enterprise Interoperability Interoperability Assessment in the 721 Deployment of Technical Processes in Industry. IFAC Proceedings Volumes Information Systems Management A Contribution to Enterprise Interoperability Maturity Assessment Adaptation and evaluation of a multi-criteria decision analysis model for Lyme 730 disease prevention Use of the Analytic 732 Hierarchy Process for Medication Decision-Making in Type 2 Diabetes. PLOS 733 ONE Comprehensive value assessment of drugs 735 using a multi-criteria decision analysis: An example of targeted therapies for 736 metastatic colorectal cancer treatment Application of multicriteria decision analysis in 739 health care: a systematic review and bibliometric analysis. Health Expectations Hadjadj-Baillot C, Ponstherapeutic innovations in the emergency department: The methoxyflurane 744 organizational impact in acute trauma pain Analytic hierarchy process: An overview of applications The Analytic Hierarchy and Analytic Network 749 Measurement Processes: The Measurement of Intangibles The state of the art development of AHP (1979-2017): 751 a literature review with a social network analysis. International Journal of 752 Production Research The analytic hierarchy process-what it is and how it is used A framework for interoperability assessment in crisis management Evaluation of Interoperability between Automation Systems using Multi-criteria 760 A Method to Diagnose Public 762 Making Assessment of supply chain 766 segmentation from an interoperability perspective Improving decision-making and management of hospital 769 resources: An application of the PROMETHEE II method in an Emergency 770 Operations Research for Health Care A hybrid 772 model to support decision making in emergency department management PROMETHEE: A 775 comprehensive literature review on methodologies and applications Inpatient 778 admission management using multiple criteria decision-making methods Operations Research for Health Care Applying analytical decision methods for determination of the best 781 treatment alternative to remove emerging micropollutants from drinking water and 782 wastewater: triclosan example. Environmental Science and Pollution Research Harrison's 785 Principles of Internal Medicine Health Systems Based on Process Mining and MCDA Methods How to Develop a Questionnaire in Order 791 to Measure Interoperability Levels in Enterprises Contribution à la caractérisation et à l'évaluation de l'interopérabilité pour 794 les entreprises collaboratives Tissue Plasminogen Activator for Acute Ischemic Stroke Guidelines for acute ischemic stroke treatment: part I