key: cord-0981643-wqysgazv authors: Zhao, Juan; Grabowska, Monika E; Eric Kerchberger, Vern; Smith, Joshua C.; Nur Eken, H.; Feng, QiPing; Peterson, Josh F.; Trent Rosenbloom, S.; Johnson, Kevin B.; Wei, Wei-Qi title: ConceptWAS: a high-throughput method for early identification of COVID-19 presenting symptoms and characteristics from clinical notes date: 2021-03-25 journal: J Biomed Inform DOI: 10.1016/j.jbi.2021.103748 sha: 407844f093aea425c8d93df9712540cb8aea7572 doc_id: 981643 cord_uid: wqysgazv Objective Identifying symptoms and characteristics highly specific to coronavirus disease 2019 (COVID-19) would improve the clinical and public health response to this pandemic challenge. Here, we describe a high-throughput approach – Concept-Wide Association Study (ConceptWAS) – that systematically scans a disease's clinical manifestations from clinical notes. We used this method to identify symptoms specific to COVID-19 early in the course of the pandemic. Methods We created a natural language processing pipeline to extract concepts from clinical notes in a local ER corresponding to the PCR testing date for patients who had a COVID-19 test and evaluated these concepts as predictors for developing COVID-19. We identified predictors from Firth's logistic regression adjusted by age, gender, and race. We also performed ConceptWAS using cumulative data every two weeks to identify the timeline for recognition of early COVID-19-specific symptoms. Results We processed 87,753 notes from 19,692 patients subjected to COVID-19 PCR testing between March 8, 2020, and May 27, 2020 (1,483 COVID-19-positive). We found 68 concepts significantly associated with a positive COVID-19 test. We identified symptoms associated with increasing risk of COVID-19, including “anosmia” (odds ratio [OR] = 4.97, 95% confidence interval [CI] = 3.21–7.50), “fever” (OR = 1.43, 95% CI = 1.28–1.59), “cough with fever” (OR = 2.29, 95% CI = 1.75–2.96), and “ageusia” (OR = 5.18, 95% CI = 3.02–8.58). Using ConceptWAS, we were able to detect loss of smell and loss of taste three weeks prior to their inclusion as symptoms of the disease by the Centers for Disease Control and Prevention (CDC). Conclusion ConceptWAS, a high-throughput approach for exploring specific symptoms and characteristics of a disease like COVID-19, offers a promise for enabling EHR-powered early disease manifestations identification. As of October 14, 2020, over 7.7 million people in the United States (U.S.) and 37 million worldwide have been infected with coronavirus SARS-CoV-2, the agent responsible for COVID-19 [1] . The virus's high transmissibility, lack of native immunity, high mutability, and the dearth of effective treatments make managing COVID-19 uniquely challenging. Hence, timely recognition of emerging symptoms specific to COVID-19 plays an essential role in the clinical and public health response, enabling rapid symptom screening, diagnostic testing, and contact tracing. Early in the pandemic, physicians observed fever, cough, and shortness of breath as presenting symptoms of COVID-19; however, these symptoms are common to many viral and bacterial illnesses [2] . Subsequently, as new symptoms were reported, health departments and ministries updated the list of COVID-19 symptoms [3] ; for example, the U.S. CDC and the Department of Health and Social Care in the United Kingdom added loss of smell and loss of taste, highly indicative symptoms [4] , to the list in late April and mid-May, respectively [5, 6] . As demonstrated by the COVID-19 pandemic, identifying the specific disease symptoms early in the course of the pandemic is crucial to inform the public on when to present for testing and can potentially be used to reduce the size of the outbreak, lowering overall morbidity and mortality. Recent efforts to track COVID-19 symptoms have used methods such as scanning scientific publications or Twitter [7, 8] , deploying questionnaires [9] , or releasing apps to selfreport symptoms [10] . However, results from publications and questionnaires are often delayed; data from social media or self-reported apps do not always include proper controls and lack physiological assessments to determine COVID-19 status. Electronic Health Records (EHR) data has also been used to characterize COVID-19, due to the availability of routinely collected medical data. However, existing studies of EHRs have been mostly limited to structured data (e.g., coded diagnoses, procedures, or lab tests) [11, 12] and have lacked a portable and highthroughput approach [13] . Here, we present a high-throughput approach (ConceptWAS) for early identification of clinical manifestations of COVID-19 using natural language processing (NLP) on EHR clinical notes. ConceptWAS was modeled after the methodology of genome-wide association studies (GWAS) [14] , which scan the genomes from different people to identify genetic markers that can be used to predict the presence of a disease, and phenome-wide association studies (PheWAS) [15] , which operate in reverse to GWAS by screening thousands of diagnosis codes in EHR for a given genetic variant. Numerous studies have applied GWAS and PheWAS to reveal the inheritance patterns of various diseases [16] . However, unstructured EHR data, in particular clinical notes, are a rich but underutilized EHR resource, containing detailed descriptions of patients' signs or symptoms, medical histories, and progression [17] .Yet, using clinical notes to systematically identify the symptoms and clinical characteristics of a pandemic disease has been largely untapped. In this study, we used ConceptWAS to identify the symptoms and clinical characteristics associated with COVID-19. In particular, we performed serial ConceptWAS analyses using every 2-week cumulative data to demonstrate the time course of emerging clinical manifestations. We also conducted a chart review to validate the significant associations. The study was performed at Vanderbilt University Medical Center (VUMC), one of the largest primary care and referral health systems serving over one million patients annually from middle Tennessee and the Southeast United States. We used data from patients represented in the VUMC EHR aged ≥18 years. The study was approved by the VUMC Institutional Review Board (IRB #200512). We identified patients who received at least one SARS-CoV-2 polymerase chain reaction (PCR) test between March 8 (when the first COVID-19 case emerged at VUMC) and The case group (COVID-19-positive) was defined as patients who had >=1 PCR positive result, and the control group (COVID-19-negative) consisted of patients with only negative PCR tests. We excluded patients who had no clinical notes on the day when the PCR test was ordered. We extracted clinical notes from 24 hours prior to PCR testing date (day 0 ) for the cohort (>86% of patients had at least one note within the time window, see Figure B .1). If a patient first tested negative and then subsequently tested positive or if a patient tested positive more than once, we used the date of the first positive PCR test as day 0 . We also segmented the study period into a 2-week interval window and performed a temporal analysis using every 2-week cumulative data. The primary types of clinical notes that we extracted included progress notes, problem lists, Emergency Department (ED) provider notes, ED triage notes, imaging reports, social histories, etc., (full list is shown in Table B .1). We used KnowledgeMap Concept Indexer (KMCI [18] ) to extract concepts (Figure C.1). The KMCI is an NLP pipeline developed at VUMC for preprocessing medical notes and entity recognition, which has been used for several clinical and genomic studies [18] [19] [20] . The preprocessing includes sentence boundary detection, tokenization, part-of-speech tagging, section header identification. The concepts were represented as Unified Medical Language System concept unique identifiers (UMLS CUIs). Since we focused on capturing clinical manifestations of COVID-19, we restricted the concepts to SNOMED Clinical Terms and a specific range of semantic types, e.g., finding, sign or symptom, disease or syndrome, individual behaviors, or mental process (see full list in Table C .1). A main challenge of clinical NLP is to accurately detect the clinical entities' assertion modifier such as negated, uncertain, and hypothetical information (e.g. describe a future hypothetical or instruction for patients). We took the following steps to post-process the KMCI output to remove concepts that appear in sentences reflecting uncertainty and theoretical thoughts. We first excluded any concepts that arose from family history sections. Next, we removed any sentences with future tense or subjunctive mood (e.g. "should", "could", or "if") that describe a hypothetical or instruction for patients. We excluded inquiry sentences that served as the template questions without a simple confirmed answer (e.g. "Yes", "No", or "None") as well. For recognition of negated concepts (e.g. "patient denies having any fever"), we used NegEx, which was implemented in KMCI. NegEx is a widely-used algorithm to detect negations, but it still could miss post-negation triggers such as "Cough: No". To enhance negation detection, we added regular expression rules based on our local note templates. The extended processing modules was implemented using Python 3.6. After processing, the extracted concepts served as the input for following ConceptWAS analysis. Similar to how GWAS and PheWAS scan genomic and phenomic data for discovery of disease associations [15, 21] , ConceptWAS examines the clinical concepts retrieved from clinical notes to determine if any concept is associated with a disease. In this study, we applied ConceptWAS to identify associations between symptoms-related concepts and the presence of COVID-19. We applied Firth's logistic regression to examine the association for each concept, adjusted by age, gender, and race. We chose Firth's logistic regression because it has become a standard approach for analyzing binary outcomes with small samples [22] . Negated and nonnegated concepts are treated separately. Concepts were coded as binary variables for each patient. Firth's logistic regression was implemented using R version 3.4.3 and the logistf package. As we tested multiple hypotheses, we used a Bonferroni correction for the significance level. For each concept, we report the odds ratio (OR), p-values, and the prevalence in case and control groups. We used a volcano plot to show p-values and the odds ratio for all concepts. We also used a forest plot to show the significant concepts that were relevant to signs and symptoms. We performed a manual chart review to evaluate the clinical plausibility of identified signals. We reviewed a concept if 1) its p-value met Bonferroni-corrected significance, and 2) it was clinically meaningful (e.g., we excluded CUIs such as "finding [CUI C0243095]" in a sentence like "Findings are nonspecific."). We randomly selected notes from which the CUI was identified. Two authors (M.E.G. and H.N.E.) with clinical background ascertained whether the identified CUI was a true signal or false positive. We identified 19,692 patients with COVID-19 PCR test results during the study period (Figure A.1). Of these, a total of 1,483 (7.5%) patients tested positive for COVID-19. Patients' mean age was 45 (44.6 ± 16.9) years. The COVID-19-positive group was younger (41.5 ± 16.2 vs. 44.9 ± 16.9), more often male (48.0% vs. 41.7%), less often white (49.6% vs. 66.7%), and newer to VUMC (EHR length 7.3 years ± 8.1 vs. 9.2 ± 8.5) compared to COVID-19-negative patients ( Table 1) . We extracted 87,753 clinical notes from the 19,692 patients. After using the NLP pipeline to process the notes, we recognized 19,595 unique concepts (including negated status) with semantic types of interests (Table B .1). Using ConceptWAS to compare EHR-derived concepts for COVID-19 positive and negative patients, 68 concepts were identified after adjusting for multiple testing (Bonferroni-corrected significance, P < 2.55E-06) ( Figure 1 , Table E Concepts related to smoking status such as "current some day smoker", "former smoker", and "smoking monitoring status" were more frequently reported in the COVID-negative group than in the COVID-positive group (OR < 1, P < 2.55E-06), suggesting more smokers in control group. To ascertain whether this signal was true or false positives due to wrongly assertion detection by NLP pipeline, we performed a chart review of 80 patients' notes that had smoking-related CUIs. We found that 79 of 80 patients confirmed an affirmative smoking status (see below chart review). To validate the signals, we reviewed patient's charts for significant concepts. We randomly selected 10-20 notes for each concept to review whether the notes mentioned the symptoms in the expected attribute (e.g. affirmative or negated). Table 2 shows the results for significant concepts that with high clinical relevance (full list in supplementary material). The significant concepts such as " anosmia", "ageusia", "depression", and concepts related to smoking status (e.g. "current some day smoker", "former smoker", and "smoking monitoring status") were consistent with the expected attribute based on chart review. Although "smoking monitoring status" was generated by an inquiry term used in a template of a chart, after we post-processed the KMCI output to remove irrelevant concepts and refine negation, the smoking monitoring status followed by a negated answer was recognized as a negated attribute. We reviewed 20 notes that mentioned the "smoking monitoring status (affirmative/positive attribute)" and 19 were either current or former smokers. We also found false positive concepts, mostly due to NLP entity recognition errors. For example, "additional information" was recognized as "adequate knowledge". The concept "fever" with positive attribute has three false positives, mainly due to a few specific chart templates used for denoting the negation, which were not captured by NLP pipeline. Their study demonstrated the feasibility of using clinical notes for a systematic analysis [17] . In this study, we developed a high-throughput pipeline to systematically scan clinical notes and detect unique concepts of COVID-19 in real-time. We applied ConceptWAS to a cohort of patients who underwent COVID-19 PCR testing. We replicated several well-known symptoms of COVID-19, such as fever, loss of smell/taste, and cough with fever [23] [24] [25] . By performing temporal analysis on every 2-week cumulative data, we detected the signal of loss of smell and taste as early as April 5, 2020, nearly three weeks earlier than the date that they were listed as COVID-19 symptoms by the CDC [4] . Our results demonstrate the feasibility of using ConceptWAS for early detection of symptoms of an unknown disease. We also observed several signals enriched in the COVID-19-negative group. For example, depression and anxiety have a higher prevalence among patients who tested negative. These signals first became significant starting from April 5, 2020, which may correspond to a period when the Governor of Tennessee issued a "safer at home" Executive Order and a "stay at home" order. It reflects the mental health issues that the shutdown and quarantine policies may bring to the people [26, 27] . We also find a higher percentage of smoking status concepts in the COVID-19-negative group. Earlier epidemiological studies found that fewer smokers are among COVID-19 patients or hospitalized COVID-19 patients [24, 28] , which are consistent with our findings of the negative correlation between smoking and COVID-19. One explanation could be the impact of nicotine on ACE-2, as nicotine has been suggested to play a protective role against COVID-19 [29] . It is also possible that smokers are taking greater social precautions because of perceived higher risk for respiratory complications from COVID-19, thus reducing their risk of contracting the virus. Although these findings suggest that smoking may be a protective factor, lack of evidence and known adverse events associated with smoking dissuade continued smoking as a protective measure against COVID-19. While our analysis was able to detect many of the known symptoms of COVID-19 included on the CDC's list, including fever, loss of smell, and loss of taste, other symptoms present on the list were not found to be significant, including shortness of breath, muscle/body aches, and vomiting/diarrhea. Upon further review of 200 notes from 13 concepts that were on the list of symptoms maintained by the CDC but not significant in our analysis, we found the true positive percentage to be 77% (Supplemental Material (1) A high-throughput, lightweight, and reproducible method is important for an emerging pandemic disease. ConceptWAS enables a rapid scan of symptoms using clinical notes. These symptoms provided an initial hypothesis for further investigation and could alert clinicians to pay attention to patients who present with specific symptoms. Researchers can run ConceptWAS regularly (e.g. using weekly or 2-week cumulative data) to track changes in the identified symptoms of a pandemic disease. (2) Running ConceptWAS, one needs to be cautious about the distribution of different clinical note types. Clinical notes differ from each other due to their specific clinical usage. They may have variable templates and inconsistent lengths. Therefore, we recommend that researchers check the distribution of document types between cases and controls to avoid sampling bias. (3) Although NLP has been used in various medical fields to improve information processing and practice [30] [31] [32] [33] , recognition of negative and uncertain concepts remains a challenge. We enhanced the detection of uncertain arguments and negated concepts by developing rule-based methods as wrappers for entity-identification generated results. Still, our manual chart review suggest that the outcome is not perfect. For example, some notes mentioned negative concepts such as "the following ROS were reviewed and are negative, unless otherwise stated as +positive: Constitutional: Fever; malaise." Such scenarios are difficult for NLP tools to identify. A combination of machine learning and rule-based approaches may improve the detection. (4) To detect differential concepts at various levels of magnitude of change, the sample size needed for the study could be estimated beforehand. For example, a sample size calculation tool (e.g. https://vbiostatps.app.vumc.org/ps/dichot/1) could be used to generally estimate the minimum sample size given the input of desired odds ratio, type I Error (α), power (e.g. 80%), and probability of exposure in controls. (5) We also learned and recognized that our study had several limitations. First, the study was performed at a single institution with a limited number of COVID-19 patients. As the pandemic crisis evolves and more patients are tested for SARS-CoV-2 in our healthcare system, our ability to detect clinical concepts associated with COVID-19 will continue to improve. Second, this study used data from a limited time (before May 27, 2020). Third, ConceptWAS accepts concepts as input, which relies on an NLP pipeline and addon packages to identify. In this study, we used a locally developed NLP pipeline and customized several RegEx rules. A user may use different tools to extract the concepts, and the following step remains the same. However, the overall performance may vary. (6) In the future, we will extract notes from visits/calls before the test date to study symptom progression, and also extract notes after the test date to further explore the symptoms and their severity after the diagnosis. Lastly, as the performance of an NLP system may vary across institutions and databases [30, 34] , further studies are necessary to assess the generalizability of our findings. In this study, we describe a high-throughput approach (ConceptWAS) that systematically scans a disease's clinical manifestations from clinical notes. By applying ConceptWAS on EHR clinical notes from patients who received a COVID-19 PCR test, we detected loss of smell and taste three weeks prior to their inclusion as symptoms of the disease by the CDC. This study demonstrates the capability of EHR-based methods to enable early recognition of COVID-19specific symptoms and to improve our response to such pandemic challenges. Up-to-date developments of ConceptWAS are available in GitHub (https://github.com/zhaojuanwendy/ConceptWAS). We thank Dr. Vivian Siegel from Department of Biology at MIT Department of Medicine at Vanderbilt University for helpful suggestions on the study design and manuscript drafting. The COVID-19-positive group had a distribution of clinical notes types similar to that of the WHO Coronavirus Disease (COVID-19) Dashboard Clinical Characteristics of Coronavirus Disease 2019 in China COVID-19 and anosmia: A review based on upto-date knowledge Seroprevalence of SARS-CoV-2 antibodies in people with an acute loss in their sense of smell and/or taste in a community-based population in London, UK: An observational cohort study CDC confirms six coronavirus symptoms showing up in patients over and over Statement from the UK Chief Medical Officers on an update to coronavirus symptoms CovidNLP: A Web Application for Distilling Systemic Implications of COVID-19 Pandemic with Natural Language Processing Machine Learning to Detect Self-Reporting of Symptoms, Testing Access, and Recovery Associated With COVID-19 on Twitter: Retrospective Big Data Infoveillance Study Symptom Profiles of a Convenience Sample of Patients with COVID-19 -United States Real-time tracking of self-reported symptoms to predict potential COVID-19 Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the APHP/Universities/INSERM COVID-19 research collaboration The Consortium for Clinical Characterization of COVID-19 by Augmented curation of clinical notes from a massive EHR system reveals symptoms of impending COVID-19 diagnosis The NHGRI GWAS Catalog, a curated resource of SNP-trait associations PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene-disease associations Tromp, eMERGE Phenome-Wide Association Study (PheWAS) identifies clinical associations and pleiotropy for stop-gain variants A Concept-Wide Association Study of Clinical Notes to Discover New Predictors of Kidney Failure The KnowledgeMap Project: Development of a Concept-Based Medical School Curriculum Database Extracting timing and status descriptors for colonoscopy testing from electronic medical records Potential etiologic and functional implications of genome-wide association loci for human diseases and traits Firth's logistic regression with rare events: accurate effect estimates and predictions? -Puhr -2017 -Statistics in Medicine Anosmia and Ageusia: Common Findings in COVID-19 Patients Smell dysfunction: a biomarker for COVID-19 COVID-19 patients' clinical characteristics, discharge rate, and fatality rate of meta-analysis Mental Health and the Covid-19 Pandemic Bill Lee issues stay-at-home order through Prevalence of Underlying Diseases in Hospitalized Patients with COVID-19: a Systematic Review and Meta-Analysis Editorial: Nicotine and SARS-CoV-2: COVID-19 may be a disease of the nicotinic cholinergic system CLAMP -a toolkit for efficiently building customized clinical natural language processing pipelines Automatic identification of critical follow-up recommendation sentences in radiology reports Automated extraction of clinical traits of multiple sclerosis in electronic medical records Natural Language Processing in Oncology: A Review Negation's Not Solved: Generalizability Versus Optimizability in Clinical Natural Language Processing The authors have no competing interests to declare.