key: cord-1009922-x7u872r5
authors: Wang, Liqin; Foer, Dinah; MacPhaul, Erin; Lo, Ying-Chih; Bates, David W.; Zhou, Li
title: PASCLex: A Comprehensive Post-Acute Sequelae of COVID-19 (PASC) Symptom Lexicon Derived from Electronic Health Record Clinical Notes
date: 2021-11-13
journal: J Biomed Inform
DOI: 10.1016/j.jbi.2021.103951
sha: a15e1f5c0cf5edb477f5f1ceba02f9ee76aeb20b
doc_id: 1009922
cord_uid: x7u872r5

OBJECTIVE: To develop a comprehensive post-acute sequelae of COVID-19 (PASC) symptom lexicon (PASCLex) from clinical notes to support PASC symptom identification and research. METHODS: We identified 26,117 COVID-19 positive patients from the Mass General Brigham’s electronic health records (EHR) and extracted 328,879 clinical notes from their post-acute infection period (day 51-110 from first positive COVID-19 test). PASCLex incorporated Unified Medical Language System® (UMLS) Metathesaurus concepts and synonyms based on selected semantic types. The MTERMS natural language processing (NLP) tool was used to automatically extract symptoms from a development dataset. The lexicon was iteratively revised with manual chart review, keyword search, concept consolidation, and evaluation of NLP output. We assessed the comprehensiveness of PASCLex and the NLP performance using a validation dataset and reported the symptom prevalence across the entire corpus. RESULTS: PASCLex included 355 symptoms consolidated from 1,520 UMLS concepts of 16,466 synonyms. NLP achieved an averaged precision of 0.94 and an estimated recall of 0.84. Symptoms with the highest frequency included pain (43.1%), anxiety (25.8%), depression (24.0%), fatigue (23.4%), joint pain (21.0%), shortness of breath (20.8%), headache (20.0%), nausea and/or vomiting (19.9%), myalgia (19.0%), and gastroesophageal reflux (18.6%). DISCUSSION AND CONCLUSION: PASC symptoms are diverse. A comprehensive lexicon of PASC symptoms can be derived using an ontology-driven, EHR-guided and NLP-assisted approach. By using unstructured data, this approach may improve identification and analysis of patient symptoms in the EHR, and inform prospective study design, preventative care strategies, and therapeutic interventions for patient care.

its clinical stages and treatment strategies is evolving. Emerging prospective and retrospective studies suggest that some patients have persistent symptoms and/or develop delayed or longterm complications after their recovery from acute COVID-19, which may also be referred to as post-acute sequelae of SARS-CoV-2 infection (PASC) syndrome or long COVID. [2] [3] [4] The onset, scope, and duration of PASC symptoms that often involve multiple organ systems represent a new phase of the pandemic, with significant implications for health care delivery. 5 Efficient, scalable tools to identify and analyze patient PASC symptoms are essential to inform risk assessment, prevention and treatment strategy development, and outcome estimation. 6, 7 The characterization of PASC symptoms has varied widely by study. 8 This heterogeneity of research findings is attributed to multiple factors, including variation in study design, patient populations, the "post-acute COVID-19" timeframe, and sample size. 2, [8] [9] [10] [11] [12] [13] [14] [15] Most early studies on PASC symptoms relied on patient survey data, manual chart review, and in person follow-up. 6, 12, 13, [15] [16] [17] [18] These studies were often limited by sample size and reporting biases. Longitudinal electronic health record (EHR) data serve as a rich data source for studying PASC symptoms.

However, retrospective studies that primarily utilize structured EHR data (e.g., lab results or International Classification of Disease [ICD] diagnosis codes) 11, 14, 19 may miss many clinical symptoms, which are often documented in clinical notes. 20 Natural language processing (NLP) can automatically identify relevant symptoms and complications at different clinical stages from large volumes of longitudinal notes of a large patient cohort. 21, 22 Several lexicon-or machine learning-based NLP approaches and knowledge bases (e.g., ontologies) have been developed to extract COVID-19 signs or symptoms from free-text data. [23] [24] [25] However, those efforts have focused on acute phases of COVID-19, while post-acute and long-term symptom identification were not covered and remain an unmet need. A significant challenge for developing such an NLP tool is the wide variation in potential postacute COVID-19 symptoms, as COVID-19 survivors can experience a heterogeneous constellation of respiratory, cardiovascular, neurologic, psychiatric, dermatologic, and gastrointestinal symptoms and/or complications. A comprehensive lexicon encoded with a standard medical terminology that encompasses a broad range of PASC symptoms derived from large volumes of EHR notes is crucial for NLP tool development and utility and future EHRbased PASC analytics and research.

Our objective of this study was to develop a comprehensive PASC symptom lexicon, termed PASCLex, using medical ontology-based and data-driven approaches. We used a large biomedical thesaurus, a large clinical EHR dataset of COVID-19 cases, an NLP system, and iterative manual chart review to develop, improve, and evaluate the lexicon.

We used a two-phase approach to develop and evaluate a post-acute COVID-19 symptom lexicon (Figure 1 ).

The study was conducted in the Mass General Brigham (MGB) healthcare system, the largest integrated healthcare delivery system in New England. It includes academic tertiary care medical centers, community hospitals, outpatient primary care, and specialty care offices. MGB maintains an enterprise data warehouse which is a centralized repository of EHR data, including demographics, encounters, diagnoses, problem lists, laboratory results, medications, flowsheets, and clinical notes (e.g., inpatient and outpatient encounters, discharge summaries, telephone calls, patient electronic messages). This study was approved by the institutional review board of MGB with waiver of informed consent from study participants for secondary use of EHR data.

We identified patients who were ≥18 years of age and had a positive test result for SARS-CoV-2 by polymerase chain reaction (PCR) clinical assay between March 4, 2020 and February 09, 2021. We defined days 0-40 from first positive PCR test as the acute COVID-19 phase, 26 days 41-50 as a "grace" period to allow for additional results from testing conducted during the acute period, and day 51 and after as the post-acute COVID-19 period. We used all clinical notes from days 51 to 110 (a total of 60 days) during the post-acute period for each patient. 13, 27, 28 Patients who died before day 51 were excluded from the study. We divided the corpus of clinical notes into development (90% of the study cohort) and validation (10% of the study cohort) datasets to meet our study objectives.

In the first phase of this study, using the Unified Medical Language System ® (UMLS) and SNOMED CT, we compiled an initial symptom lexicon containing selected UMLS concepts and related synonyms. To develop PASCLex that includes symptoms occurring in the post-COVID-19 period and likely related to COVID-19, we applied a rule-based NLP algorithm to extract mentions of the symptoms in the lexicon from the development dataset. We refined the lexicon based on an iterative process including: (1) symptom ranking based on prevalence; (2) manual review to identify post-acute COVID-19 symptoms; (3) consolidation of concepts of similar meaning; and (4) evaluation of NLP performance.

We used a knowledge-based approach to develop an initial lexicon inclusively containing symptom concepts and their synonyms that may appear in clinical notes. First, we compiled an initial list of concepts from UMLS Metathesaurus ® (version 2020AB) and SNOMED CT Core Problem List Subset (version February 2021). 29 We included concepts from UMLS Metathesaurus in the English language, with vocabulary source of "SNOMEDCT_US", and with semantic types under any of the following categories: "Pathologic Function", "Finding", and "Anatomical Abnormality". 30 We also included concepts from the SNOMED CT Core Problem List Subset with indication of the concepts as "finding" or "disorder". For the resulting concept list, we extracted synonyms from the UMLS Metathesaurus as part of PASCLex.

For post-acute COVID-19 symptom extraction, we adapted the Medical Text Extraction, Reasoning, and Mapping System (MTERMS), 31 a multipurpose NLP tool, containing modules such as a section identifier, a sentence splitter, a lexicon-based search module, and several modifier modules. We first used MTERMS's section identifier to process clinical notes in the development dataset, excluding sections less likely to contain patients' post-acute COVID-19 symptoms, such as medications list, laboratory results, immunization history, hospital discharge instructions, plan of care, and family history. Next, we split the sections into sentences and applied MTERMS's lexicon-based search module. This search module can utilize a pre-defined lexicon to match mentions of terms contained in the lexicon in the text. Synonyms in the symptom lexicon were considered as NLP search terms, and each mention of the terms was mapped to a UMLS concept. Those terms were further processed by modifier modules which consist of adapted or manually constructed rules to exclude invalid symptoms. For example, the MTERMS negation modifier module, which adopted rules from NegEx that detect negation based on a list of terms (e.g., "no", "denies", "rule out", "no sign of"), 32 was applied to exclude negated terms (e.g., "fever" was considered negated if occurred in "no fever"). Similarly, we also used modifier modules to identify experiencers other than the patient (e.g., mother, father, wife, husband, son, daughter), patient history, and allergic reactions for which we used an allergen lexicon to identify and exclude symptoms mentioned in the context of drug-induced adverse events, such as drug allergic reactions (e.g., 'ACE Inhibitors Angioedema', 'Metformin GI Upset'). 33 Additionally, we used a pattern-matching approach to identify sentences in which symptoms were mentioned as part of risk factors, side effects, instructions, and survey questions. The NLP rules were iteratively added or refined during the lexicon development stage to optimize NLP performance. Examples of challenges in NLP-based symptom extraction from clinical notes are described in Supplementary eTable 1.

PASC symptoms were defined as patient characteristics occurring in the post-acute COVID-19 period. We distinguished patient symptoms from objective findings (e.g., laboratory results, blood pressure measurements) and provider-based disease diagnoses (e.g., choledocholithiasis). This enabled inclusion of symptoms such as palpitations, but not the corresponding diagnoses of "Wolff-Parkinson-White syndrome" which require provider-based assessment, criteria, and evaluation. At each stage of development, we used the current symptom lexicon to extract all possible symptoms in clinical notes. We ranked all the concepts by frequency of patient occurrence. We manually reviewed the concepts to identify symptomspecific terms, as defined above. Given the long tail of symptoms distribution (as many symptoms occurred in only a small percentage of patients and therefore in a low volume of notes), we prioritized identification of the most common symptoms in the patient cohort: 1) concepts occurring in 50 or more patients were manually reviewed by two subject matter experts (DF and YL); 2) concepts occurring in fewer than 50 patients were manually searched using the symptom keywords (identified from the set of concepts occurring in more than 50 patients), e.g., "pain", "swelling". This enabled us to maximize symptom capture and facilitate subsequent concept consolidation even for rarer symptoms. Conflicts in classification were resolved by consensus review; final determination was made by LW.

Among the identified symptom concepts, we consolidated terms that were overly granular, similar, or less-frequently used. This minimizes clinically insignificant duplication in the symptom term list and improves the analysis of symptom patterns and trends. An expert panel with clinical and informatics subject matter expertise (LW, DF, YL) manually reviewed the final list of symptom concepts. Specifically, we consolidated symptoms of similar meanings. For example, "paresthesia" (CUI: C0030554) and "pins and needles" (CUI: C0423572) are included in the UMLS Metathesaurus as two different concepts; however, from a symptom-oriented perspective "pins and needles" is a manifestation of paresthesia and was consolidated under "Paresthesia".

We applied two approaches to facilitate the identification of potential concepts of similar meanings for consolidation. First, we used keyword search to identify concepts of similar meanings for consolidation. Second, we identified the cases that a synonym was mapped to two or more UMLS concepts. For example, a synonym "chest pressure" was mapped to "chess tightness" (CUI: C0232292) and "pressure in chest" (CUI: C0438716), and we consolidated these two concepts to a single concept "chest tightness". We also combined terms that were frequently paired in clinical notes into a single term (e.g., terms "loss of sense of smell", "sense of smell altered", "disorder of taste", and "decreasing sense of taste/smell" being mapped to a single term "problem with smell and taste", and terms "nausea" "vomiting" "nausea and vomiting" "nausea or vomiting" "nausea and/or vomiting" being merged to "nausea and/or vomiting"). We further consolidated the concepts based on anatomic site. For example, pain is a common symptom in the development dataset with over 300 concepts containing the word "pain". We manually consolidated all pain-related concepts by pain site, resulting in several concept groups, e.g., "abdominal pain", "joint pain", "pain in extremities", "chest pain".

Based on the steps described above we were able to iteratively refine the symptom lexicon, further enhanced by the final step of NLP evaluation. The goals of lexicon refinement were to 1) refine concept mappings causing the NLP tool to identify false positive or false negative symptoms, and 2) enhance the lexicon with synonyms from clinical notes. After each lexicon revision, we re-applied the NLP to the development dataset and manually checked the output of the NLP on a random set of clinical notes, through which, we were able to identify false positives and false negatives to continue refining the symptom lexicon and NLP rules. Specifically, we removed or changed concepts mappings that cause false positives. For example, based on the UMLS Metathesaurus, the term "fit" is a colloquial synonym for seizure, and "TEN" is an acronym for "toxic epidermal necrolysis". However, without advanced word sense disambiguation functions, these terms could cause inaccurate extraction or false positives. Therefore, term mapping rules were removed to minimize false positive symptom identification. The process of manual review also enabled symptom synonym identification from clinical notes. For example, for the symptom "problem with smell or taste", we manually reviewed clinical notes for mentions of "smell" and "taste" and added additional 143 synonyms to the lexicon. We also added "brain fog", "mental fog", "mental fogginess", and "foggy" as synonyms to the concept of "clouded consciousness".

Once the lexicon reached a satisfactory level of performance (i.e., precisions on the top 50 symptoms being above 0.9) during the NLP evaluation in the development dataset, we moved on to the second phase, which evaluated its performance in the NLP system using the validation dataset of a different cohort to gain insights about the lexicon.

With the final lexicon, we evaluated NLP performance in identifying post-acute COVID-19 symptoms from clinical notes in the validation dataset. We measured performance in terms of precision (or positive predictive value) for the 50 most common symptoms and recall (or sensitivity) across all the symptoms. To calculate precision, we randomly selected 50 sentences for each symptom from the validation dataset (a total of 2,500 sentences) and manually identified the number of true positive symptoms. To calculate recall across all symptoms in the lexicon, we randomly selected 50 clinical notes from the validation dataset and manually identified symptom terms. The notes were also processed by the NLP tool for symptom extraction. We then counted the number of manually annotated symptoms (N) and the number of symptoms extracted by both manual (N) and NLP (n) review and calculated the ratio of n and N for recall.

After validation, we applied the lexicon and NLP to the entire clinical note corpus. We calculated the frequency of PASC symptoms within the entire study population during the 2-month followup.

Overall, 51,485 adult patients met initial inclusion criteria for the study, among which, 26,117 Table 1 . Supplement eTable 2 displays the distribution of the study population by gender and age groups. Overall, there was a higher count of females in our study cohort, and the age distribution shows that both males and females had unimodal peaks at the age of 50s.

There are 49.9% females under 50 while only 37.3% for male. The percentage of age groups under 50 are higher for females and from age 50 and above, are higher for males.

The initial lexicon based on the selected UMLS sematic types included a total of 157,245 unique concepts and 604,056 synonyms. Eleven percent (n=17,701) of these concepts were mentioned in the development dataset. Manual review of the 2,660 concepts that occurred in 50 or more patients during the study period identified 698 (26.1%) concepts as symptom related.

Review of the 15,041 UMLS concepts occurring in fewer than 50 patients identified an additional 822 symptom concepts. Consolidation among the total 1,520 symptom concepts (16, 466 synonyms) resulted in a final count of 355 symptoms. Therefore, the estimated recall of the final PASC symptom lexicon in our NLP system was 0.84.

Our error analysis revealed that the false negative cases were caused by the following reasons: missing abbreviations (e.g., "OSA" for obstructive sleep apnea, "HA" for headache), uncommon synonyms ("short tempered"), and misspellings (e.g., "pian" for pain).

We developed PASCLex, a comprehensive lexicon of a set of post-acute COVID-19 symptom terms from the EHR, and validated the NLP tool using the PASCLex for symptom extraction; the lexicon is publicly available. This work advances the study of post-acute COVID-19 symptoms by providing a systematic approach and scalable tool to identify patient symptoms post-COVID-19 infection in large datasets. Free-text clinical notes represent an underutilized data source in the active field of post-acute COVID-19 research; this study will facilitate NLP-based approaches to identifying post-COVID-19 symptoms. Subsequently, these symptoms could be used to characterize the epidemiology of patient populations most vulnerable to post-acute COVID-19 symptoms, to design prospective referral pathways or early clinical interventions to promote rehabilitation, mitigate symptom duration and possibly prevent downstream sequelae.

Alternatively, symptom data could be aggregated into organ/system-based domains for targeted translational research using biological samples or genetic data, that could be used to justify use of medications in specific populations or support development of novel therapeutic interventions. 35, 36 Our study has several strengths. We used clinical notes to study post-acute COVID-19

symptoms. This is distinct from prior studies that have used other EHR-based data elements such as billing/diagnosis codes, laboratory results, or medications. 14, 19, 37 Billing/diagnosis codes may represent disease states encompassing a variety of symptoms, and therefore may underestimate each unique symptom and trends across symptoms. Clinical notes also have an advantage over these sources for studying symptoms, by representing the clinical encounter between patient and provider, and capturing patient reported symptoms--including outside of the formal visit setting (e.g., patient messages and telephone encounters). While patient surveys also capture symptoms, they are subject to responder bias and limited in scale. 2 In larger survey studies across institutions or from social media-based sources, acute COVID-19 infection status may be difficult to ascertain. 38 From a methods perspective, we used an NLP-based approach which is well-suited to systematically process unstructured data containing post-acute COVID-19 symptoms. 8 Compared to machine learning-based approaches for symptom extraction, 24 which may focus on specific symptoms as unique classification tasks, requiring resource-intensive document annotation, a knowledge-based NLP approach can leverage existing knowledge bases to investigate and identify a wide variety of symptoms from a large dataset. Previously, Sahoo and Silverman et al described a rule-based NLP system to detect 11 COVID-19 symptoms (e.g., cough, dyspnea, fatigue, aches, new loss of smell/taste, sore throat). 39 Wang et al adapted a clinical NLP tool, i.e., CLAMP, to identify common COVID-19 signs and symptoms, which was based on 153 UMLS concepts (e.g., sore throat, headache, fever, fatigue, altered consciousness). 24 Our work significantly expands on this prior literature providing a comprehensive lexicon of 355 symptoms consolidated from 1,520 UMLS concepts, which could support various NLP systems 40, 41 19 We identified the most common 10 symptoms as pain, anxiety, depression, fatigue, shortness of breath, joint pain, nausea and/or vomiting, headache, myalgia, and gastroesophageal reflux, highlighting symptoms common to both prior inpatient and outpatient-based studies and raising additional symptoms for consideration.

Among the top 50 symptoms identified in our study, some have not been previously reported, or only reported in small series or single case studies. These includes patient-level symptoms that may have previously been obscured in diagnoses or groups of symptoms, such as "cutaneous signs" 2 while our study captured individual symptom descriptors such as "rash", "itching", "erythema". Similarly our findings of "visual changes" 42 and "abnormal gait" 43 population and a focus on specific symptoms, rather than diagnoses or groupings emphasizes a focus on the patient experience and may better reflect the symptom heterogeneity in the postacute COVID-19 period. This also differentiates our study from previous work studying post-COVID-19 sequalae in specific patient populations (e.g., with neuropsychiatric outcomes) 14 or following a specific level of COVID-19 infection acuity (e.g., necessitating ICU admission). 10 Use of data from a general population for lexicon development 37 supports comprehensive identification of symptoms for the lexicon across a medically diverse population; future studies may then compare symptom prevalence and frequency in distinct populations.

Beyond the development of a specific lexicon for PASC symptoms detailed in this work, our study presents a general framework for an EHR-guided curation of lexicons that could be applied to other medical domains (Figure 1) . Although our pipeline is not fully automated, we detailed the description of the manual efforts to make these processes reproducible for other researchers.

The symptom lexicon was developed using data from a multi-institution, U.S.-based health care system using a single EHR. Documentation patterns and preferred terminology may differ by geographic location, health system, and EHR vendor. This might impact symptom prevalence or lead to missed symptoms common in other populations. However, the large cohort sample size, and similarities of our findings to those from meta-analyses of prospective and retrospective studies in clinically and geographically distinct populations, supports the external validity and generalizability of this work. Future studies might be needed to replicate the symptom lexicon development pipeline in other EHR systems.

Several limitations are inherent in use of EHR data to study post-acute COVID-19

symptoms. First, our study is limited to patients with clinical follow-up in the healthcare system.

Patients may have sought out-of-system care post-COVID-19 infection; these patients' symptoms would not be captured in our lexicon. However, travel advisories during COVID-19 may have limited the ability of patients to seek out of system (or out of state) care, improving our ability to capture encounters in the post-COVID-19 period.

Second, the EHR reflects routine care, and therefore, symptoms documented in the clinical notes might be due to other visit reasons including pre-existing conditions, underlying diseases (e.g., cancer) or acute events (e.g., stroke) that may or may not be considered direct sequelae of COVID-19 infection. However, symptoms related to chronic conditions may have become more persistent or more severe due to COVID-19 infection and may clinically be considered post-acute sequelae. Excluding symptoms which may have been mentioned in the pre-COVID period would likely grossly under capture the extent of post-COVID-19 symptoms especially as symptoms may be shared across disease states or are episodic. To strengthen the temporal relationship between symptom and infection, we limited the timeframe of the postacute COVID-19 phase to day 110, rather than longer timeframes used in other studies (e.g., 6 months) which might otherwise have increased the probability of non-COVID19 related symptom capture. However, we did not distinguish between prevalent and incident symptoms for the reasons described above and this remains a limitation. Future studies may use computational or epidemiological approaches to investigate correlations between COVID-19 and resulting symptoms. 11 Limitations in the process of lexicon development are also present. First, as detailed in the lexicon development methods, we used keyword search, rather than systematic manual review, for concepts occurring in fewer than 50 patients (0.2% of the population with notes).

While rare (low frequency) symptoms may have been missed, these are less likely to be clinically relevant from a population health standpoint.

Second, although the lexicon development was enriched by manual review to minimize false positive and false negative symptom terminology in the EHR context and to enhance clinical relevance by term consolidation, such manual efforts might be less reproducible or subject to potential biases due to experts' knowledge and experiences. We did not formally record intermediate values of recall and precision for each iteration, but this would have strengthened the generalizability of our methods to other lexicon curations. In the final system evaluation for symptom extraction, recall was estimated using a random set of clinical notes from across the corpus of notes. We did not calculate recall estimates for each individual symptom due to low frequency of many of the symptoms in clinical notes (as described above) which would have been prohibitive from the standpoint of manual review.

Third, although we incorporated synonyms and variations from UMLS enriched with local variations, residual undetected variations may remain, particularly due to missed abbreviations or misspellings. Initially we applied abbreviation tables, however, we found it was over-sensitive with a high false positive identification rate and therefore this was silenced. Subsequently, missed symptom variations from PASCLex may have caused the NLP to miss true positives from some clinical notes. Future work may consider building in additional abbreviation tables and word sense disambiguation modules to better leverage abbreviations. Additional enrichments might also include hierarchical associations among the symptom concepts and/or categorizations by body systems. Fourth, semantic context can challenge our rule-based NLP approach. For example, while the NLP tool can accurately identify the symptom term "weight loss" in clinical notes, on chart review we found that the term "weight loss" was often used in the context of weight gain in the post-COVID-19 period, as patients had gained weight and were trialing weight management interventions to decrease their weight. Future studies using word or sentence embeddings in advanced machine learning models may help to extract mentions of symptoms from free-text notes with greater accuracy. 44 Our proposed symptom lexicon and NLP approach can serve as a base for those future efforts.

We developed a comprehensive post-acute COVID-19 symptom lexicon using EHR data and assessed a lexicon-based NLP approach to extract post-acute COVID-19 symptoms from clinical notes. Further studies are warranted to characterize the prevalence of and risk factors for post-acute COVID-19 symptoms in specific patient populations.

No specific funding was received for this project. LW, EM, YCL, DWB, LZ was supported by grant NIH-NIAID R01AI150295 and AHRQ R01HS025375. DF receives salary support from research funding from IBM Watson (PI: Bates and Zhou) and CRICO unrelated to this work.

LW has full access to all of the data in the study and takes responsibility of the integrity of the data and the accuracy of the data analysis. 

LW, DF, EM, YCL, LZ reports no disclosures. DWB reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from ValeraHealth, equity from Clew, equity from MDClone, personal fees and equity from AESOP, personal fees and equity from Feelbetter, and grants from IBM Watson Health, outside the submitted work.

The post-acute COVID-19 symptom lexicon can be accessed at: https://github.com/bylinn/Post_Acute_COVID19_Symptom_Lexicon. 

COVID-19 Coronavirus Pandemic

More Than 50 Long-Term Effects of COVID-19: A Systematic Review and Meta-Analysis. Res Sq

Attributes and predictors of long COVID

Assessment and characterisation of post-COVID-19 manifestations

Management of post-acute covid-19 in primary care

Post-acute COVID-19 syndrome. Incidence and risk factors: A Mediterranean cohort study

Post-COVID-19 global health strategies: the need for an interdisciplinary approach

Challenges in defining Long COVID: Striking differences across literature, Electronic Health Records, and patient-reported information. medRxiv

High Prevalence of Pulmonary Sequelae at 3 Months after Hospital Discharge in Mechanically Ventilated Survivors of COVID-19

Evolving Phenotypes of nonhospitalized Patients that Indicate Long Covid. medRxiv

Follow-up of adults with noncritical COVID-19 two months after symptom onset

6-month neurological and psychiatric outcomes in 236 379 survivors of COVID-19: a retrospective cohort study using electronic health records. The Lancet Psychiatry

Post-discharge persistent symptoms and health-related quality of life after hospitalization for COVID-19

Postdischarge symptoms and rehabilitation needs in survivors of COVID-19 infection: A cross-sectional evaluation

Krebs von den Lungen 6 (KL-6) as a marker for disease severity and persistent radiological abnormalities following COVID-19 infection at 12 weeks

COVID Symptoms, Symptom Clusters, and Predictors for Becoming a Long-Hauler: Looking for Clarity in the Haze of the Pandemic. medRxiv. 2021

Health Care Utilization and Clinical Characteristics of Nonhospitalized Adults in an Integrated Health Care System

Days After COVID-19 Diagnosis -Georgia

Comparison of International Classification of Diseases and Related Health Problems, Tenth Revision Codes With Electronic Medical Records Among Patients With Symptoms of Coronavirus Disease

Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review

Developing a manually annotated clinical document corpus to identify phenotypic information for inflammatory bowel disease

Extracting COVID-19 diagnoses and symptoms from clinical text: A new annotated corpus and neural event extraction framework

COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data model

Mining Concepts for a COVID Interface Terminology for Annotation of EHRs

At what times during infection is SARS-CoV-2 detectable and no longer detectable using RT-PCR-based tests? A systematic review of individual participant data

Unified Medical Language System: Current Semantic Types

Using Medical Text Extraction, Reasoning and Mapping System (MTERMS) to process medication information in outpatient clinical notes

A dynamic reaction picklist for improving allergy reaction documentation in the electronic health record

Understanding long COVID: a modern medical challenge

Long COVID's long R&D agenda

High-dimensional characterization of post-acute sequelae of COVID-19

Long-term patient-reported symptoms of COVID-19: an analysis of social media data

A fast, resource efficient, and reliable rule-based system for COVID-19 symptom identification

Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications

An overview of MetaMap: historical perspective and recent advances

Ocular findings among patients surviving COVID-19

A Case Report of Coronavirus Disease 2019 Presenting with Tremors and Gait Disturbance

Development and Validation of a Deep Learning Model for Detection of Allergic Reactions Using Safety Event Reports Across Hospitals

Large cohort of COVID-19 cases was followed in the electronic health records (EHR) for sequalae

NLP)-assisted approach is feasible for lexicon development Symptom lexicon can support NLP to extract PASC symptoms from clinical notes Credit Author Statement LW has full access to all of the data in the study and takes responsibility of the integrity of the data and the accuracy of the data analysis. Concept and design: LW, DF. Acquisition, analysis, or interpretation of data: All authors. Draft of the manuscript: LW, DF. Critical revision of the manuscript for important intellectual content

LW, DF, EM, YCL, LZ reports no disclosures. DWB reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from ValeraHealth, equity from Clew, equity from MDClone, personal fees and equity from AESOP, personal fees and equity from Feelbetter, and grants from IBM Watson Health, outside the submitted work.