key: cord-0003430-sphe2ely
authors: Rajamani, Sripriya; Kayser, Ann; Emerson, Emily; Solarz, Sarah
title: Evaluation of Data Exchange Process for Interoperability and Impact on Electronic Laboratory Reporting Quality to a State Public Health Agency
date: 2018-09-21
journal: Online J Public Health Inform
DOI: 10.5210/ojphi.v10i2.9317
sha: 23e73664ec4374d13d3a5243006bec6f30f5cda4
doc_id: 3430
cord_uid: sphe2ely

BACKGROUND: Past and present national initiatives advocate for electronic exchange of health data and emphasize interoperability. The critical role of public health in the context of disease surveillance was recognized with recommendations for electronic laboratory reporting (ELR). Many public health agencies have seen a trend towards centralization of information technology services which adds another layer of complexity to interoperability efforts. OBJECTIVES: The study objective was to understand the process of data exchange and its impact on the quality of data being transmitted in the context of electronic laboratory reporting to public health. This was conducted in context of Minnesota Electronic Disease Surveillance System (MEDSS), the public health information system for supporting infectious disease surveillance in Minnesota. Data Quality (DQ) dimensions by Strong et al., was chosen as the guiding framework for evaluation. METHODS: The process of assessing data exchange for electronic lab reporting and its impact was a mixed methods approach with qualitative data obtained through expert discussions and quantitative data obtained from queries of the MEDSS system. Interviews were conducted in an open-ended format from November 2017 through February 2018. Based on these discussions, two high level categories of data exchange process which could impact data quality were identified: onboarding for electronic lab reporting and internal data exchange routing. This in turn comprised of ten critical steps and its impact on quality of data was identified through expert input. This was followed by analysis of data in MEDSS by various criteria identified by the informatics team. RESULTS: All DQ metrics (Intrinsic DQ, Contextual DQ, Representational DQ, and Accessibility DQ) were impacted in the data exchange process with varying influence on DQ dimensions. Some errors such as improper mapping in electronic health records (EHRs) and laboratory information systems had a cascading effect and can pass through technical filters and go undetected till use of data by epidemiologists. Some DQ dimensions such as accuracy, relevancy, value-added data and interpretability are more dependent on users at either end of the data exchange spectrum, the relevant clinical groups and the public health program professionals. The study revealed that data quality is dynamic and on-going oversight is a combined effort by MEDSS Informatics team and review by technical and public health program professionals. CONCLUSION: With increasing electronic reporting to public health, there is a need to understand the current processes for electronic exchange and their impact on quality of data. This study focused on electronic laboratory reporting to public health and analyzed both onboarding and internal data exchange processes. Insights gathered from this research can be applied to other public health reporting currently (e.g. immunizations) and will be valuable in planning for electronic case reporting in near future.

Past [1] and present [2] national initiatives that promote electronic health records (EHRs), also advocate for the electronic exchange of data across various healthcare sectors using nationally recommended standards [3] . The critical role of public health, in the context of disease surveillance is recognized by these regulations, with recommendations for electronic laboratory reporting (ELR). ELR refers to the electronic transmission of labs related to reportable conditions to public health [4] . The emphasis on interoperability in recent legislations [5] and roadmaps [6] is facilitating the focus on electronic movement of data across healthcare settings. Many public health agencies have seen a trend towards centralization of information technology services which adds another layer of complexity to interoperability efforts. Given this landscape, it is essential to understand the process of data exchange and its impact on quality of data being transmitted, as this is a crucial step in interoperability. In addition, this holds broad implications for future priority transactions such as electronic case reporting to public health.

Initial research around ELR focused on comparison of paper-based reports to electronic transmissions and found predominantly positive impact of ELR [7, 8] on specifically two metrics of data quality: timeliness and completeness. Subsequent studies have assessed the role of intermediaries such as Health Information Exchanges (HIE) [9] [10] [11] to facilitate ELR and reported better completeness of data with HIE support. Presently, studies have begun to focus on provider reporting of notifiable diseases [12, 13] , as moving to electronic case notification [14] [15] [16] along with ELR will be great progress to support overall public health disease surveillance. Challenges in adoption and use of recommended codes [17] [18] [19] and need for an informatics savvy workforce [20] were identified as some of the issues in the move towards ELR [21] .

A recurring theme across these studies was assessing the quality of data, including exploring new venues to measure [22] [23] [24] and improve [25] it. Timeliness and completeness were the two dimensions of data quality (DQ) which were often evaluated. Metrics from DQ frameworks published in literature can be used as guidance in identifying additional parameters for assessment.

Data quality assessment framework by Kahn et al. [26] , identifies three DQ categories: conformance, completeness and plausibility, along with verification and validation as two DQ assessment contexts. DQ framework by Strong et al., proposes a broad conceptualization of the quality of data from perspective of data consumers. It defines high quality data as one that is fit for use and emphasizes context around data production and usage.

Strong's framework proposes four DQ categories (Intrinsic DQ, Contextual DQ, Representational DQ, Accessibility DQ) comprising of fifteen DQ dimensions [27, 28] . These include Intrinsic DQ (Accuracy, Objectivity, Believability, Reputation); Contextual DQ (Relevancy, Value-Added, Timeliness, Completeness, Amount of data); Representational DQ (Interpretability, Ease of understanding, Concise representation, Consistent representation); Accessibility DQ (Accessibility, Access security). The strength of this framework is the breadth of DQ characteristics. Data quality is a multi-dimensional concept dependent on multitude of factors and adoption of data standards does facilitate DQ, but does not guarantee it [29] . Good quality data that meet many of DQ dimensions are critical for public health surveillance purposes. With increasing electronic data exchange and emphasis on interoperability, it is essential to understand impact of various facets of data exchange on various dimensions of DQ.

The Minnesota Electronic Disease Surveillance System (MEDSS) [30] is the public health information system for supporting infectious disease surveillance at a state level for Minnesota and operational since 2008. It holds data on reportable conditions and receives ELRs submitted to the state public health agency. MEDSS is used for case management, contact tracing and to support outbreak investigations. Its scope has expanded to include non-infectious diseases such as blood lead surveillance and birth defects. It's a person-centric surveillance system which currently holds ~1,279,986 events across infectious diseases, lead and community and family health programs. Approximately 153,880 lab tests/results were reported electronically for 2017 across six health systems and four reference labs. Many healthcare systems are currently on a waiting list for either onboarding/move to electronic exchange or upgrade to better version of reporting standard.

Nationally recommended standards for ELR [4] comprise of HL7 2.5.1 for message format and LOINC [31] and SNOMED [32] codes for representation of lab tests and results respectively. With increasing demands for electronic data exchange for incoming data to MEDSS from clinical sectors and for outgoing data to Centers for Disease Control and Prevention (CDC), new informatics tools to support data validation and exchange were implemented. The objective of this study was to assess the data exchange process and to understand its impact on the quality of data in MEDSS. The overarching goal is to utilize findings for improvements in informatics tools and processes to enhance the value of MEDSS by providing good quality data to support various public health purposes including disease surveillance.

The process of assessing data exchange for electronic lab reporting and its impact was a mixed methods approach with qualitative data obtained through expert discussions and quantitative data obtained from queries of the MEDSS system. Various subject matter experts (n=9) were identified spanning across the informatics team that supports MEDSS operations, public health program professionals who are users of the MEDSS system and its data, and the Information Technology (IT) team which supports the data exchange process. The focus included both onboarding (process OJPHI of shifting to electronic exchange for either new reporting or migration/upgrade to different standard) and on-going submissions. ELR is unique in that reporting can occur from either EHR or from LIMS (Laboratory Information Management System) and can occur from healthcare delivery organization or from reference laboratories and these were taken into consideration. Interviews were conducted in an open-ended/discussion format and were done over time frame of November 2017 through February 2018. Based on these discussions, two high level categories of data exchange process which could impact data quality were identified: onboarding for electronic lab reporting and internal data exchange routing. Figure 1 displays the ELR onboarding process and includes the testing and validation suite of tools offered in public domain by the National Institute of Standards and Technology (NIST) [33] . The six identified key processes that influence quality of data are numbered A through F (A -mapping of tests and results to appropriate codes, B -NIST test bed for testing of messages, C -submit test HL7 messages, D -solicit HL7 messages with test cases (e.g. specific tests, seasonal diseases), E -technical review, F -program review). Figure 2 displays the internal data exchange routing process which includes the PHIN Messaging System (PHIN MS) [34] , a CDC provided software that serves as a transport mechanism for effective movement of messages. This part comprises of four main components numbered G through J (G -PHIN MS, H -Lab code list database validation, I -Rhapsody® Integration Engine [35] rules, J -mapping in MEDSS).

The potential influence of the ten identified critical steps in the data exchange process and its impact on quality of data was identified through expert input using Strong's DQ framework as a OJPHI guidance. This was followed by analysis of data in MEDSS by criteria identified by the informatics team. Evaluation of messages not mapped to any disease program in MEDSS was identified as a priority. Next, assessment of completeness of race and ethnicity fields before and after implementation of demographic data import feature in ELR was completed. Using Influenza reporting as a scenario, the number of non-reportable tests that get submitted and added to data in MEDSS was examined. Finally, the number of incoming messages which get rejected due to errors was examined to quantify the need for additional technical assistance. 

The process of exchanging data electronically is iterative and is initiated with numerous rounds of message testing and varying gradation of technical assistance based on data submitter need and capabilities. Each step in the process was deemed critical in its impact on the quality of data which moves across clinical sector and public health. Table 1 lists the six identified key processes for ELR onboarding, relevant sub-processes/notes and their influence including both DQ metric and DQ dimension. All DQ metrics (Intrinsic DQ, Contextual DQ, Representational DQ, and Accessibility DQ) were impacted with varying influence on DQ dimensions. Some errors such as improper mapping on EHR end had a cascading effect and can pass through technical filters and go undetected till use of data by epidemiologists. Some DQ dimensions such as accuracy, relevancy, value-added data and interpretability are more dependent on users at either end of the data exchange spectrum, the relevant clinical groups and the public health program professionals. The results from analysis of data in MEDSS by various criteria identified by the informatics team is presented in Table 3 . Evaluation for cases which are not mapped to any disease program and assigned to "other/unknown" category yielded 952 cases. Assessment of messages for these cases noted an absence of LOINC and/or SNOMED codes and their combination pair for disease assignment. Next, the analysis focused on submission of non-reportable respiratory diseases along with reportable conditions (Influenza) due to issues with special lab test panel, and this identified 366 cases. This was followed by evaluating the number of incoming messages which get rejected due to errors and there currently isn't any process that keeps track of it. The corresponding impact on data quality metrics due to these identified issues are also presented in Table 3 . An enhancement was implemented in January 2018 to import demographic data (race, ethnicity) from ELR feeds and this evaluation presented in Table 4 . Of the total of 3,651 electronic lab messages received from January through February 2018, data on Race was present in 2,310 messages and 1,680 messages received in that time frame had data on Ethnicity. Comparison of this new data with already existing race and ethnicity data in MEDSS obtained through case reporting and follow-up investigations revealed 270 number of messages wherein race from ELR feed was different than one currently recorded in MEDSS. 

Federal regulations and incentives have offered the needed momentum towards electronic reporting to public health. But, there are differences in public health measure reporting [36] with ELR lagging behind immunization reporting due to complexities around multitude of labs associated with reportable conditions, slow adoption of recommended codes and multiple entities/professionals involved in exchange such as clinical labs, reference labs, ordering provider, infection control practitioner and disease epidemiologists. Another key factor to consider is that ELR can be generated from EHRs or from laboratory information systems (LIS) in reference labs OJPHI or in healthcare settings. This study also portrays the need for constant updates to the various validation tools to ensure errors are not being propagated across the data exchange chain.

This research points to the complexity of the data exchange process by illustrating the numerous stakeholders involved and the critical role each one plays in moving towards interoperability. It also pointed to the need for all data exchange partners to be informed of evolution of standards, both message formats (e.g. HL7) and codes (e.g. LOINC, SNOMED). Some of these exchange mechanisms require technical assistance for either submitter (e.g. labs, providers) and the receiver (e.g. public health) or both of them. National projects such as Digital Bridge [37] and APHL Informatics Messaging Services (AIMS) [38] are aimed to assist in data exchange across jurisdictional boundaries in public health. The data exchange process could be set such that messages get rejected if they fail any of the checks, but will require manual intervention by public health or the data reporters to understand quality issues around rejection and fix them.

The study also presents various testing tools (NIST test bed) and validation engines (Rhapsody, lab code list validation database) that help to automate quality checks and monitor various DQ dimensions. Approaches from other public health reporting such as immunizations wherein provider quality reports [39] are generated could be tried in the context of ELR. Likewise open source software tools have been proposed to support data quality checks for both immunization reporting [39] and ELR [23, 40] . Implementation and maintenance of these tools require both financial and technical resources. Importantly, there needs to be overarching guidance and support from national organizations such as CDC to ensure standardization and to facilitate sharing of tools/resources across jurisdictions.

The study revealed that data quality is dynamic and on-going oversight is a collaborative effort by MEDSS informatics team, technical and public health program professionals. Overall, maintenance of good data quality in context of ELR needs a multipronged approach with automated tools, data exchange partners education, technical assistance, regular updates of codes/tools, organizational commitment and national guidelines along with support by informaticians/data quality analysts.

This research depicts the details of processes, people and technology and the need for all the parts to align to make an electronic data exchange truly meaningful by providing good quality to data that fits the purpose (public health surveillance in this case). It highlights the benefits of standardization of data exchange processes which can be applied to other public health transactions. Many public health agencies have seen a trend towards centralization of information technology services which adds another layer of complexity to interoperability efforts. It underscores the value of a public health informatician to be part of electronic exchange of data across various sectors (clinical care, labs) and public health. Finally, this study presents a compelling picture of the interoperability endeavor as a team effort and underscores the critical role an informatics team can play in facilitating the data exchange process.

The study has some limitations and focus on some dimensions of data quality by Strong et al., is one of them. Some DQ aspects such as accessibility are not integrated with exchange process and hence were excluded. The research emphasis was determined by criteria outlined by MEDSS OJPHI informatics team, and was limited based on available data during study period. Some metrics were not tracked and certain tool enhancements were implemented recently by IT support team and thus evaluation was limited. Another limitation is that currently a large volume of ELR submitters are reference labs which are not required to collect race and ethnicity data and hence completeness of those data fields through ELR is limited. Some DQ errors are attributed to frequency of upgrade of codes/validation engine that are driven by organizational resources (finances, trained personnel) / institutional priorities and beyond the scope of this study.

With the growing demands for electronic reporting with public health, there is a need to understand the current processes for supporting electronic exchange and their impact on quality of data. This study focused on electronic laboratory reporting to public health and analyzed both onboarding and internal data exchange processes. Insights gathered from this research can be applied to other public health reporting currently (e.g. immunizations) and will be valuable in planning for electronic case reporting in near future. The study has potential implications in promoting data quality along with electronic exchange to support public health surveillance.

EHR Incentive Programs

Advancing Care Information, Quality Payment Program

Edition Base Electronic Health Record Definition, and ONC Health IT Certification Program Modifications

Department of Health & Human Services. 21st Century Cures Act

Office of the National Coordinator for Health Information Technology. Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap

A comparison of the completeness and timeliness of automated electronic laboratory reporting and spontaneous reporting of notifiable conditions

Benefits and barriers to electronic laboratory results reporting for notifiable diseases: the New York City Department of Health and Mental Hygiene experience

Electronic laboratory data quality and the value of a health information exchange to support public health reporting processes

Measuring the impact of a health information exchange intervention on provider-based notifiable disease reporting using mixed methods: a study protocol

Notifiable condition reporting practices: implications for public health agency participation in a health information exchange

Timeliness of Chlamydia Laboratory and Provider Reports: A Modern Perspective

Completeness and timeliness of notifiable disease reporting: a comparison of laboratory and provider reports submitted to a large county health department

Development of an electronic public health case report using HL7 v2.5 to meet public health needs

Evaluation of HL7 v2.5.1 electronic case reports transmitted from a healthcare enterprise to public health

Evaluation of Data Exchange Process for Interoperability and Impact on Electronic Laboratory Reporting Quality to a State Public Health Agency Online Journal of Public Health Informatics * ISSN

Advancing electronic case reporting (eCR) to enable public health disease control and emergency response: getting into the technical weeds

Towards Interoperability for Public Health Surveillance: Experiences from Two States

The long road to semantic interoperability in support of public health: experiences from two states

Impact of selective mapping strategies on automated laboratory result notification to public health authorities

Estimating increased electronic laboratory reporting volumes for meaningful use: implications for the public health workforce

Electronic laboratory reporting: barriers, solutions and findings

Variation in information needs and quality: implications for public health surveillance and biomedical informatics

Measuring and Improving the Quality of Data Used for Syndromic Surveillance

A vision for the systematic monitoring and improvement of the quality of electronic health data. Stud Health Technol Inform

Electronic health information quality challenges and interventions to improve public health surveillance data and practice

A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data. EGEMS (Wash DC)

Data Quality in Context

Data standard not equal data quality. Stud Health Technol Inform. 192, 1208. Epub

Minnesota Electronic Disease Surveillance System (MEDSS)

LOINC (Logical Observation Identifiers Names and Codes) 1994

NIST HL7v2 Resource Portal for Electronic Laboratory Reporting

PHIN Messaging System (PHIN MS)

Rhapsody Integration Engine

Office of the National Coordinator for Health Information Technology

Association of Public Health Laboratories (APHL)

IIS Data Quality Practices: Monitoring and Evaluating Data Submissions

Automated Characterization of Health Information at Large-scale Longitudinal Evidence Systems (ACHILLES)

The authors would like to thank the members of the MEDSS technical team for discussions around the data exchange processes and various public health program professionals for their time and valuable input.