key: cord-268721-n6dsc4ig
authors: Pawlowski, Colin; Wagner, Tyler; Puranik, Arjun; Murugadoss, Karthik; Loscalzo, Liam; Venkatakrishnan, AJ; Pruthi, Rajiv K; Houghton, Damon E; O'Horo, John C; Morice, William G; Williams, Amy W; Gores, Gregory J; Halamka, John; Badley, Andrew D; Barnathan, Elliot S; Makimura, Hideo; Khan, Najat; Soundararajan, Venky
title: Inference from longitudinal laboratory tests characterizes temporal evolution of COVID-19-associated coagulopathy (CAC)
date: 2020-08-17
journal: eLife
DOI: 10.7554/elife.59209
sha: 
doc_id: 268721
cord_uid: n6dsc4ig

Temporal inference from laboratory testing results and triangulation with clinical outcomes extracted from unstructured electronic health record (EHR) provider notes is integral to advancing precision medicine. Here, we studied 246 SARS-CoV-2 PCR-positive (COVID(pos)) patients and propensity-matched 2460 SARS-CoV-2 PCR-negative (COVID(neg)) patients subjected to around 700,000 lab tests cumulatively across 194 assays. Compared to COVID(neg) patients at the time of diagnostic testing, COVID(pos) patients tended to have higher plasma fibrinogen levels and lower platelet counts. However, as the infection evolves, COVID(pos) patients distinctively show declining fibrinogen, increasing platelet counts, and lower white blood cell counts. Augmented curation of EHRs suggests that only a minority of COVID(pos) patients develop thromboembolism, and rarely, disseminated intravascular coagulopathy (DIC), with patients generally not displaying platelet reductions typical of consumptive coagulopathies. These temporal trends provide fine-grained resolution into COVID-19 associated coagulopathy (CAC) and set the stage for personalizing thromboprophylaxis.

There is a growing body of evidence suggesting that severe COVID-19 outcomes may be associated with dysregulated coagulation (Tang et al., 2020) , including stroke, pulmonary embolism, myocardial infarction, and other venous or arterial thromboembolic complications (Klok et al., 2020) . This so-called COVID-19 associated coagulopathy (CAC) shares similarities with disseminated intravascular coagulation (DIC) and thrombotic microangiopathy but also has distinctive features (Levi et al., 2020) . Given the significance of CAC to COVID-19 mortality, there is an urgent need for fine-grained resolution into the temporal manifestation of CAC, particularly in comparison to the broad-spectrum of other, better characterized coagulopathies. While there are studies suggesting associations between COVID-19 infection and mortality with thrombocytopenia, D-dimer levels, and prolongation of prothrombin time, the signatures of CAC onset and progression as well as their connection to clinical outcomes are not well defined (Tang et al., 2020; Gao et al., 2020; Panigada et al., 2020) . An advanced understanding of this phenotype may aid in the risk stratification of patients, thus facilitating optimal monitoring strategies during disease evolution through the paradigm of precision medicine.

Longitudinal analysis identifies lab test results characteristic of COVID-19 at specific prognostic time intervals

To identify laboratory test results that differ between COVID pos and COVID neg (matched) patients, we analyzed longitudinal trends of 194 laboratory test results in the 30 days before and after the day of PCR testing (designated as day 0). As most patients did not undergo laboratory testing for each assay on a daily basis, we grouped the measurements into nine time windows reflecting potential stages of infection as follows: pre-infection (days À30 to À11), pre-PCR (days À10 to À2), time of clinical presentation (days À1 to 0), and post-PCR phases 1 (days 1 to 3), 2 (days 4 to 6), 3 (days 7 to 9), 4 (days 10 to 12), 5 (days 13 to 15), and 6 (days 16 to 30). We only considered test-time window pairs in which at least three patients contributing to laboratory test results in both groups. During each time window, we then compared the distribution of results from COVID pos versus COVID neg (matched) patients, allowing us to identify any lab tests which were significantly altered in COVID pos patients during any time of disease acquisition, onset, and/or progression.

Of the 1709 lab test-time window pairs with adequate data points for comparison, we identified 130 such pairs (comprising 66 unique lab tests) which met our thresholds for statistical significance (Cohen's D >0.35, BH-adjusted Mann-Whitney p-value <0.05; Table 2 ). Among these were lab tests that may be considered positive controls for our analysis. From the time of clinical presentation onward, elevated titers of SARS-CoV-2 IgG antibodies ( Figure 1A ) and a reduction in blood oxygenation in COVID pos patients were observed ( Figure 1B) . We also identified abnormalities in several other classes of lab tests, including immune cell counts , red blood cell Table 1 . Summary of patient characteristics for the overall COVID pos , COVID neg (matched), and COVID neg cohorts. The COVID neg (matched) cohort was constructed using 1:10 propensity score matching to balance each of the clinical covariates, including demographics (age, gender, race), medication use (anticoagulant/antiplatelet use in the preceding 30 days/1 year of PCR testing date), medical history of thrombotic events from the past year, and hospitalization status in the month prior to the date of PCR testing. . Summary of lab tests significantly different between COVID pos and propensity score-matched COVID neg cohorts during at least one clinical time window. Data from individual patients were averaged over the defined time windows, and the mean values were compared between COVID pos and COVID neg patients. The lab test-time window pairs shown are those which met our defined thresholds for statistical significance and substantial effect (BH-adjusted Mann-Whitney p-value <0.05 and Cohen's D absolute value >0.35). In particular, 130 of the initial 1709 (test, time window) pairs with at least one patient met these thresholds. Rows are sorted alphabetically by test and then time window (from earliest to latest). Coagulation-related tests of particular interest (fibrinogen, platelets, prothrombin time, activated partial thromboplastin time, and D-dimer) are highlighted in gray. Sample sources are denoted as: P = plasma, S = serum, S/P = serum/ plasma, B = blood, U = urine. counts ( Figure 2C ), mean corpuscular volume ( Figure 2D ), calcium and magnesium levels ( Figure 2E -F), and coagulation-related tests ( Figure 3 ).

With respect to coagulation, we found that plasma fibrinogen was significantly elevated in COV-ID pos patients at the time of diagnosis (Cohen's D = 0.859, BH-adjusted Mann-Whitney p-value = 8.9e-7, Table 2, Figure 3A ). This hyperfibrinogenemia generally resolved during the 7 days following diagnosis ( Figure 3A) . Conversely, platelet counts were lower in the COVID pos cohort at the time of clinical presentation but tended to increase over the subsequent 10 days to levels significantly higher than those in COVID neg patients (Cohen's D = 0.229, BH-adjusted Mann-Whitney p-value = 3.6e-3, Table 2, Figure 3B ). While thrombocytopenia has been reported in COVID-19 patients before Yang et al., 2020) , an upward trend in platelet counts after diagnosis has not been described to our knowledge. We observe extended prothrombin times in both the COVID pos and COVID neg (matched) cohorts significantly above the normal range; however, there was no differentiation between the cohorts. We observe extended activated partial thromboplastin times (aPTT) in the COVID pos significantly above normal levels from day 7 onward ( Figure 3D ). D-dimer levels were frequently above normal limits in both the COVID pos and COVID neg cohorts and were not significantly different between these cohorts during any time window ( Figure 3E ). The above trends hold up even when the time windows are perturbed ( Table 3) .

We also performed similar analyses comparing the COVID pos and COVID neg (matched) cohorts using different time window definitions including daily trends ( Figure 4 ). This approach offers the advantage of increased granularity at the cost of sample size per time point, but we did identify similar lab tests as altered in COVID pos patients using each approach including the fibrinogen decline and platelet increase in the COVID pos cohort after diagnosis ( Figure 4 ).

Given the recently described coagulopathies associated with COVID-19 (Tang et al., 2020; Klok et al., 2020; Levi et al., 2020) , we were intrigued by the temporal trends in fibrinogen levels and platelet counts in the COVID pos cohort ( Figure 3 ). Next, we asked whether the observed coagulation-related laboratory trends were associated with clinical manifestations of thrombosis. To do so, we employed a BERT-based neural network (Devlin et al., 2018 ; see Materials and methods) to identify patients who experienced a thrombotic event after their SARS-CoV-2 PCR testing date. Specifically, we extracted diagnostic sentiment from EHR notes (e.g. whether a patient was diagnosed with a phenotype, suspected of having a phenotype, ruled out for having a phenotype, or other) regarding specific thromboembolic phenotypes including deep vein thrombosis, pulmonary embolism, myocardial infarction, venous thromboembolism, thrombotic stroke, cerebral venous thrombosis, and disseminated intravascular coagulation.

We found that 101 of the total 2232 COVID pos cohort (4.5%) were positively diagnosed with one or more of the above-mentioned thrombotic phenotypes in the 30 days after PCR testing, with the majority of these patients (53 of 101) experiencing a deep vein thrombosis. Interestingly, we found that after creating subsets of the patients with longitudinal lab testing data (i.e. the patients meeting the criteria for inclusion in our study), 76 of the 246 patients (31%) had at least one EHR-derived clot diagnosis, including 47 patients with deep vein thrombosis (Table 4) . Thus, the cohort under consideration here is highly enriched ( Table 5 ; hypergeometric p-value <1Â10 À50 ) for patients experiencing thrombotic events compared to the overall COVID pos cohort. Table 3 . Sensitivity analysis of clinical time intervals for significant coagulation-related lab test trends.

Results from sensitivity analysis perturbing the time intervals for the significant (coagulation-related lab test, time interval) pairs (i.e. highlighted rows of Table 2 ). Longitudinal platelet count trends are not strongly associated with the development of thrombosis in COVID-19 patients Among the 246 COVID pos patients with longitudinal lab testing data, 81 were serially tested starting at clinical presentation for fibrinogen versus 245 tested for platelets. As such, we first analyzed whether associations exist between platelet counts (or temporal alterations thereof) and clotting propensity in this cohort. Among these 245, there were 169 patients without thrombosis after PCRbased diagnosis (non-thrombotic) and 76 patients with thrombosis (thrombotic). There is a statistically significant difference between the COVID pos and COVID neg cohorts in the platelet count at clinical presentation ( Figure 5A ). In particular, thrombocytopenia (platelet count <150Â10 9 /L) was observed in 29% (46 out of 154) COVID pos and 21% (346 of 1661) COVID neg patients at the time of diagnosis ( Figure 5A ). However, the platelet levels at this time point were not associated with the subsequent formation of a blood clot in the COVID pos cohort ( Figure 5B ). We hypothesized that the previously discussed increase in platelet counts after COVID-19 diagnosis may be associated with the development of blood clots. If true, then we would expect the thrombotic COVID pos cohort to show significantly higher maximum platelet counts during their course of disease progression compared to the non-thrombotic COVID pos cohort. We found that this was not For each cohort, average lab values and standard errors are shown for each day with at least three observations. For certain lab tests, some data points are missing because these days had fewer than three data points in the COVID pos cohort.

the case, as maximum platelet counts were similar in the two groups ( Figure 5C) . Similarly, among the 147 COVID pos patients with platelet counts both at the time of clinical presentation and postdiagnosis, the degree of maximal platelet increase was not associated with the development of thrombosis ( Figure 5E ). It would certainly be of interest to perform this same analysis on a larger COVID pos cohort (n = 2232; 101 thrombotic vs. 2131 non-thrombotic), but we were not able to do so given the lack of longitudinal testing available for a large majority of non-thrombotic COVID pos patients (Table 4) .

Conversely, we explored whether some COVID pos patients may experience clotting in the setting of low or declining platelets (e.g. consumptive coagulopathy) despite the population-level trend of increasing platelets over time. Indeed, we found that nine of 74 thrombotic patients showed absolute platelet counts below 100 Â 10 9 /L during at least one post-diagnosis time window (below dotted red line in Figure 5D ). In addition, we analyzed post-diagnosis platelet reductions among COVID pos patients. While the maximum degree of absolute platelet reduction was not associated with clot development in aggregate ( Figure 5F ), we did find that six of the 52 thrombotic patients experienced a reduction of at least 100 Â 10 9 /L relative to the time of diagnosis. Of note, similar fractions of non-thrombotic COVID pos patients also showed these low or declining platelet counts, indicating that these trends are not specific indicators of thrombosis ( Figure 5D,F) . Table 4 . Prevalence of thrombotic phenotypes after the clinical presentation in COVID pos patients with and without available longitudinal lab testing data. For each clotting phenotype listed, a BERT-based neural network was used to extract diagnostic sentiment from individual EHR patient notes in which the phenotype (or a synonym thereof) was present. This automated curation was applied to clinical notes for each patient from day = À1 (clinical presentation) to day = 30 (end of the study period) relative to the PCR testing date. In this table, we show the absolute number of patients with each phenotype along with the percentage of patients in each cohort with the given specific thrombotic phenotype in parentheses. Hypergeometric enrichment: p-value <1Â10 À50 . 

The observed declining platelet counts and thrombocytopenia in the context of thrombosis in a small fraction of COVID pos patients are consistent with previous reports that fewer than 1% of survivors, but over 70% of non-survivors, meet the International Society on Thrombosis and Hemostasis (ISTH) criteria for disseminated intravascular coagulation (DIC; Tang et al., 2020) . As was previously noted, hyperfibrinogenemia was among the strongest lab test features distinguishing COVID pos from COVID neg patients at diagnosis, but the subsequent downward trend ( Figure 3A) could be attributed to a resolving acute phase response and/or consumption of fibrinogen in a systemic coagulopathy. Using our BERT-based sentiment extraction, we found that only five of the 2232 COVIDpos patients that exhibited DIC-like symptoms, all of whom were included in our longitudinal cohort of 246 COVID pos patients ( Table 4 ). Upon manual review of the EHR data for each patient, we found that two out of these five patients had confirmed diagnosis of DIC, while the remaining had high clinical suspicion and pending tests for DIC. This finding suggests that declining fibrinogen after COVID-19 diagnosis typically represents a physiologic return to normal range rather than pathologic coagulation factor consumption. To further examine the plasma fibrinogen trends among COVID-19 patients with DIC, with non-DIC thrombosis, and without thrombosis, we examined patient-level lab test trends from 10 individuals who were tested for fibrinogen both at the time of diagnosis and at least two times subsequently. The 10 patients for individual analysis were selected as the first 10 individuals with longitudinal fibrinogen lab testing data available. This patient-level analysis indeed revealed multiple distinct trajectories with respect to fibrinogen and other coagulation parameters in COVID pos patients. Four of these ten individuals developed at least one blood clot during their hospital course. Only one was identified by our BERT model (and confirmed by manual EHR review) to have low-grade DIC, and as expected we found this patient's longitudinal lab test pattern to be consistent with consumptive coagulopathy (Patient 124; Figure 6A ). At the time of diagnosis, this patient showed significant hyperfibrinogenemia with elevated D-dimers (1304.5 ng/mL) and a borderline normal platelet count (153 Â 10 9 /L). Over the next 10 days, this patient's fibrinogen levels consistently decreased, reaching a minimum of 110 mg/dL Table 6 . Validation of the BERT model to identify the sentiment of thrombotic phenotypes in clinical notes. Out-of-sample accuracy results of the BERT model to identify thrombotic phenotypes in 1000 randomly selected sentences from clinical notes which contained at least one mention of a thrombotic phenotype. The columns are (1) Clotting phenotype: thrombotic phenotype identified in the sentence, Figure 6 continued on next page on day 9. Similarly, after an initial recovery to 190 Â 10 9 /L the platelet counts consistently declined starting on day 2 post-diagnosis, reaching a minimum of 117 Â 10 9 /L on day 11. D-dimer levels exponentially increased after 5 days, reaching a maximum of 41,300 ng/mL on day 10. Phenotypically, this patient experienced both thrombotic (right internal jugular vein and right superior thyroid artery) and hemorrhagic (oropharyngeal and pulmonary) events. This combination of lab results and clinical manifestations is consistent with the diagnosis of DIC-like consumptive coagulopathy during the first week after COVID-19 diagnosis. Lab test results from three other non-DIC thrombotic patients with longitudinal fibrinogen testing confirm the presence of alternative forms of coagulopathy in the COVID-19 population. Patient 23 developed a clot on day 4 post-diagnosis in the context of a declining fibrinogen level and increasing D-dimers but steady platelet counts, which actually increased shortly thereafter ( Figure 6B ). Patient 79 developed several clots after day 3 post-diagnosis in the setting of upward trending platelets (which eventually exceed the upper limit of normal) and elevated levels of both fibrinogen and D-dimers ( Figure 6C) . Patient 94 developed a clot on day 8 post-diagnosis with relatively stable platelet counts within normal limits and steadily declining fibrinogen levels ( Figure 6D ).

One hypothesis is that early elevations in plasma fibrinogen contribute to the clotting observed in the non-DIC like COVID pos cohort. This hypothesis may warrant further analysis in cohorts with more longitudinal fibrinogen data, but again it is important to note that several COVID pos patients who presented with hyperfibrinogenemia did not go on to develop thromboses ( Figure 6E-F) . This emphasizes that a steady post-diagnosis decline in plasma fibrinogen may represent physiologic resolution of the acute phase response rather than a pathologic consumption of fibrinogen and other coagulation factors ( Figure 6B,D-F) .

Taken together, this analysis affirms that a DIC-like coagulopathy resulting in a combination of hemorrhage and thrombosis can develop in the setting of COVID-19 infection. However, the observations that DIC was formally diagnosed in only five of 2232 COVID pos patients and emphasizes that consumptive coagulopathy is an exception rather than the rule as it pertains to thrombotic phenotypes in COVID-19 patients. These results should be considered as a preliminary characterization of COVID-associated coagulopathies (CAC) and will be updated as patient counts increase with the continued evolution of the COVID-19 pandemic.

Many studies on clinical characteristics and lab tests are shedding light on the spectrum of hematological parameters associated with COVID-19 patients. In an initial study of 41 patients from Wuhan, the blood counts in COVID pos patients showed leukopenia and lymphopenia, and prothrombin time and D-dimer levels were higher in ICU patients than in non-ICU patients (Huang et al., 2020) . Another study based on 343 Wuhan COVID pos patients found that a D-dimer level of at least 2.0 mg/ mL could predict mortality with a sensitivity of 92.3% and a specificity of 83.3% . An independent study of 43 COVID-19 patients found significant differences between mild and severe cases in plasma interleukin-6 (IL-6), D-dimers, glucose, thrombin time, fibrinogen, and C-reactive protein (p<0.05; Gao et al., 2020) . While such studies indeed highlight that hematological and inflammatory abnormalities are prevalent in COVID pos , a high-resolution temporal understanding of how these parameters evolve in COVID-19 patients post diagnosis has not been established. Specifically, in the wake of accumulating evidence for hypercoagulability in COVID pos patients, there are important clinical questions emerging regarding the necessity of and guidelines for thromboprophylaxis in patient management.

DIC-like consumptive coagulopathy in COVID-19 has been a point of concern in severely ill COVID-19 patients. Particularly in patients with ARDS, multiple organ dysfunction syndrome (MODS) is the predominant cause of death. A recent study suggested that DIC was associated with MODS during the early stage of ARDS and that persistent DIC may also have a role in this association (Gando et al., 2020) . Our study focusing on COVID-19 patients with longitudinal lab data suggests that COVID-19 is indeed associated with modulation of coagulation related parameters such as platelet counts, fibrinogen levels, and clotting time ( Figure 2) . However, the majority of thrombotic events in COVID-19 patients with longitudinal lab testing are not the result of a DIC-like consumptive coagulopathy, as this only occurs in a small subset ( Table 4) .

The ability to derive this longitudinal understanding of COVID-19 progression, including laboratory abnormalities and their associated clinical manifestations, mandates the synthesis of structured and unstructured EHR data (e.g. lab tests and clinical notes) at a large scale. The fact that tens of thousands of patients have undergone SARS-CoV-2 testing at major academic medical centers (AMCs) provides an abundance of potential data to perform this analysis but also poses significant challenges from a practicality standpoint. Manual review and curation of patient trajectories and Table 8 . Lab test data availability in patients with SARS-CoV-2 PCR testing. Lab test data availability for all patients who underwent SARS-CoV-2 PCR testing in the Mayo Clinic EHR database from February 15, 2020 to May 28, 2020. Includes counts of lab tests and counts of patients with 1+ and 3+ lab tests both overall and for selected coagulation-related lab tests (activated partial thromboplastin time, D-dimer, fibrinogen, platelets, and prothrombin time). associated testing results is not practical. It is not likely to provide comprehensive or even entirely accurate individual patient records. Rather, triangulation across datasets, including lab measurements, clinical notes, and prescription information, using a scalable digitized approach to extract structured data along with sentiment-surrounded clinical phenotypes and outcomes enables us to efficiently perform this analysis in a timely fashion. By developing and deploying such a digitized platform on the entirety of EHR data from a large AMC, we have identified in an unbiased manner, laboratory test-based abnormalities that differentiate COVID pos patients from COVID neg patients. The abnormalities in coagulation-related tests, including fibrinogen and platelets, were intriguing in the context of literature reporting the occurrence of various clotting phenotypes in COVID-19 patients, including DIC-like consumptive coagulopathies along with more isolated clotting events in the lungs, central nervous system, and other tissues (Tang et al., 2020; Klok et al., 2020; Levi et al., 2020) . Our finding that consumptive coagulopathy represents a minority of COVID-19 associated clotting events provides context for other Table 9 . Lab test data availability in patients with SARS-CoV-2 PCR testing and longitudinal lab data. Lab test data availability for all patients who underwent SARS-CoV-2 PCR testing in the Mayo Clinic EHR database from February 15, 2020 to May 28, 2020 with longitudinal testing data available (i.e. patient received the same lab test on three separate days within + / À 30 days of PCR testing date). Includes counts of lab tests and counts of patients with 1+ and 3+ lab tests both overall and for selected coagulation-related lab tests (activated partial thromboplastin time, D-dimer, fibrinogen, platelets, and prothrombin time). studies, which have reported overt DIC or DIC-like disease in over 70% of non-survivors but far lower fractions of survivors (Tang et al., 2020) . As the pandemic continues to evolve and the patient counts increase over the coming months, we will be monitoring and reporting any updates to the clinical and laboratory observations drawn in this study.

Notwithstanding the preliminary nature of the analysis presented in this study, the results highlight that consumptive coagulopathy should be considered in the minority of COVID pos patients with significant serial reductions in platelet counts. It remains to be seen whether the post-diagnosis platelet increases or early hyperfibrinogenemia which we observed may contribute mechanistically to the clotting in the much larger non-DIC thrombotic COVID-19 population. It is important to note that despite the trend of increasing platelets, the platelet count only extended above the normal range (>450Â10 9 /L) after the PCR date in few COVID pos patients with serial measurements, and the development of such outright thrombocytosis was observed with similar frequencies in the thrombotic and non-thrombotic cohorts ( Figure 5C) . Further, the fact that several patients with elevated fibrinogen (i.e. >400 mg/dL) at presentation did not develop thromboses suggests that early hyperfibrinogenemia is not a singular driver of subsequent clotting events, but a small sample size (n = 10 patients; nine non-thrombotic vs. one thrombotic) limited the power of this analysis ( Figure 6 ).

Despite these caveats, this linking of longitudinal trends to patient outcomes provides several useful pieces of clinical information. First, hyperfibrinogenemia is to be expected in COVID-19 patients around the time of diagnostic testing. Furthermore, declining fibrinogen levels shortly after diagnosis are also expected and likely represent the resolution of acute phase response in most patients rather than a decline secondary to the onset of consumptive coagulation. In addition, borderline or overt thrombocytopenia is common in COVID-19 patients at the time of clinical presentation, and the initial platelet count does not robustly predict patients who are likely to develop thromboses. After diagnosis, COVID-19 patients generally show an upward trend in platelets. Patients whose platelets trend down after diagnosis should be monitored, as platelet reductions after clinical presentation are associated with thromboses and significant reductions may be indicative of ongoing consumptive coagulopathy.

One unavoidable limitation of this study is that we restrict our analysis to patients which have longitudinal lab testing data available. While the inclusion criteria is naturally biased, we consider this study population to be of high clinical interest because these patients are highly enriched for severe thrombotic events during the study period (see Table 5 ). Further, in the propensity score matching step of the analysis, we are able to construct a control cohort that is similar to the COVID pos cohort in these enriched dimensions. To provide additional color on the distinctive attributes of the study population, we provide a summary of the clinical characteristics of the study population versus all patients with PCR tests during the same time period (see Table 7 ). In addition, we provide the median numbers of lab tests per patient for selected coagulation-related lab tests (fibrinogen, platelets, PTT, APTT, D-dimer) and total lab tests (Tables 8 and 9) .

It is important to note that while we center the study period around the PCR testing date, this date may not correspond to the same disease state of COVID-19 for each individual in the COVID pos cohort. To account for the potential variability in disease progression, we have performed a sensitivity analysis on the time intervals (Table 3) . Additionally, there are several covariates that may influence these longitudinal trends and should be explored further. For example, we have already considered whether previous or concomitant administration of anticoagulants or antiplatelet agents influences patient lab test results and/or outcomes. Similarly, in the future, we intend to explore whether longitudinal lab measurement trends differ between outpatient, inpatient, and ICU admitted patient cohorts. New datasets can also be utilized; for example, rather than grouping patients by the identified thromboembolic phenotypes extracted from the clinical notes alone, patients could be stratified by those who had imaging studies (duplex ultrasound, CT scan, etc.) performed, and phenotypes could be directly extracted from these procedural reports. As more data accumulates from COVID pos and COVID neg patients in the coming months, these analyses need to be expanded to assess similarities and differences in the temporal trends of laboratory test results among a wider range of patient subgroups relevant for COVID-19 outcomes, such as those who have pre-existing conditions (e.g. diabetes, hypertension, obesity, malignancies) or patients who are on specific medication (e.g. ACE inhibitors, statins, immunosuppressants).

In summary, this work demonstrates significant progress toward enabling scaled and digitized analyses of longitudinal unstructured and structured EHRs to identify variables (e.g. laboratory results) which are associated with relevant clinical phenotypes (e.g. COVID-19 diagnosis and outcomes). In doing so, we identified trends in lab test results which may be relevant to monitor in COVID-19 patients and warrant both clinical and mechanistic follow-up in more targeted and explicitly controlled prospective analyses.

Study design, setting and patient population This is a retrospective study of patients who underwent polymerase chain reaction (PCR) testing for suspected SARS-CoV-2 infection at the Mayo Clinic and hospitals affiliated to the Mayo health system. This research was conducted under IRB 20-003278, 'Study of COVID-19 patient characteristics with augmented curation of Electronic Health Records (EHR) to inform strategic and operational decisions'. For further information regarding the Mayo Clinic Institutional Review Board (IRB) policy, and its institutional commitment, membership requirements, review of research, informed consent, recruitment, vulnerable population protection, biologics, and confidentiality policy, please refer to www.mayo.edu/research/institutional-review-board/overview.

We analyzed data from 74,586 patients who received PCR tests from the Mayo Clinic between February 15, 2020 to May 28, 2020. Among this population, 2232 patients had at least one positive SARS-CoV-2 PCR test result, and 72,354 patients had all negative PCR test results. In order to align the data for the analysis of aggregate longitudinal trends, we selected a reference date for each patient. For patients in the COVID pos cohort, we used the date of the first positive PCR test result as the reference date (day = 0). For patients with all negative PCR tests, we used the date of the first PCR test result as the reference date (day = 0). We defined the study period for each patient to be 30 days before and after the PCR testing date. Patients with contradictory PCR test results were excluded for the purpose of this analysis; for example, a positive PCR test result and a negative PCR test result on the same day, or a positive PCR test result followed immediately by several negative PCR test results.

Over 4 million test results from 6298 different types of lab tests were recorded for the patients who received PCR tests in the 60-day window surrounding their PCR testing dates at the Mayo Clinic campuses in Minnesota, Arizona, and Florida. Among these lab tests, we restricted our analysis to 194 tests with at least 1000 observations total and at least 10 observations from the COVID pos cohort among the patients with PCR testing on or before May 8, 2020. In addition, we considered different subsets of the COVID pos cohort for the analysis of each of the 194 lab tests, due to differences in availability of testing results. For each lab test, we consider the results from patients with three or more observations during the study period.

In the end, there are 246 SARS-COV-2 positive and 13,666 SARS-CoV-2 negative patients that had three or more test results during the study period for at least one of the assays among the 194 lab tests considered. We take this set of 246 COVID-19 positive patients to be the COVID pos cohort. In order to construct the COVID neg cohort from the 13,666 COVID-19 negative patients, we apply propensity score matching, which is described in the next section.

Propensity score matching to select the final COVID neg cohort To construct a COVID neg cohort similar in baseline clinical covariates to the COVID pos cohort, we employ 1:10 propensity score matching (Austin, 2011) . In particular, first we trained a regularized logistic regression model to predict the likelihood that each patient will have a positive or negative COVID-19 test result, using the following covariates: demographics (age, gender, race), anticoagulant/antiplatelet medication use (orders for alteplase, antithrombin III, apixaban, argatroban, aspirin, bivalirudin, clopidogrel, dabigatran, dalteparin, enoxaparin, eptifibatide, heparin, rivaroxaban, warfarin in the past year and in the past 30 days), pre-existing coagulopathies (medical history of thrombotic phenotypes including: deep vein thrombosis, pulmonary embolism, myocardial infarction, venous thromboembolism, thrombotic stroke, cerebral venous thrombosis, and disseminated intravascular coagulation from day À365 to day À31 relative to the PCR testing date), and hospitalization status (i.e. whether or not the patient was hospitalized within the past 30 days of PCR testing).

Using the predictions from the logistic regression model as propensity scores, we then matched each of the 246 patients in the COVID pos cohort to 10 patients out of the 13,666 COVID-19 negative patients, using greedy nearest-neighbor matching without replacement (Austin, 2011; Austin, 2014) . As a result, we ended up with a final COVID neg cohort that included 2460 patients with similar baseline characteristics to the COVID pos cohort. The characteristics of the two cohorts are summarized in Table 1 .

Further, for the analyses conducted on individual lab tests, which include only a subset of patients from the COVID pos cohort, we use the propensity scores to match each patient from the COVID pos cohort to 10 patients from the COVID neg cohort which have the most similar propensity scores and lab tests available. For example, for the fibrinogen lab test, in which we have data on 81 patients from the COVID pos cohort, we select 810 patients from the COVID neg cohort and the most similar propensity scores to be the control group. In this way, we ensure that all of the comparisons are done between subsets of the positive and negative cohorts with similar propensity scores, and therefore similar underlying characteristics.

We conduct a systematic statistical analysis to identify tests that show significant differentiation among the COVID pos cohort during a set of predetermined prognostic time intervals for SARS-CoV-2 infection. In particular, we group the lab test measurements for each patient into the following nine time intervals relative to their date of PCR testing: pre-infection (days À30 to À11), pre-PCR (days À10 to À2), time of clinical presentation (days À1 to 0), and post-PCR phases 1 (days 1 to 3), 2 (days 4 to 6), 3 (days 7 to 9), 4 (days 10 to 12), 5 (days 13 to 15), and 6 (days 16 to 30).

For each lab test and for each of each of our nine pre-specified time intervals, we compared the mean lab test value among patients who underwent at least one such lab test in the COVID pos cohort over that time interval to the mean lab test value in the COVID neg (matched) cohort over that time window. We only considered (lab test, time interval) pairs in which there were at least three patients contributing to laboratory test results in both groups. Specifically, for each (lab test, time interval) pair, we conducted the following procedure:

1. Compute (patient, time interval) averages: We compute the average lab test values for each patient in the COVID pos and COVID neg (matched) cohorts during the specified time interval. 2. Statistical hypothesis testing: We conduct a Mann-Whitney U test in order to test the null hypothesis that the average lab test results for each of the (patient, time interval) pairs from the COVID pos and COVID neg (matched) cohorts come from the same distribution. In addition, we compute the Cohen's D statistic as a measure of the effect size.

Once we have the statistics and p-values for each (test, time window) pair, in order to account for multiple hypotheses, we apply the Benjamini-Hochberg (BH) procedure with FDR controlled at 0.05. The results from the systematic comparisons which met our thresholds for effect size and statistical significance (Cohen's D > 0.35, BH-adjusted Mann-Whitney p-value <0.05) are shown in Table 2 .

We perform a sensitivity analysis to assess whether or not the key findings from the systematic statistical assessment remain the same if we perturb the considered time intervals. In particular, we repeat the statistical analysis with the time intervals shifted forward or backward 1 day for all patients. For the forward shifted sensitivity analysis, the new time intervals under consideration are: pre-infection (days À30 to À10), pre-PCR (days À9 to À1), time of clinical presentation (days 0 to 1), and post-PCR phases 1 (days 2 to 4), 2 (days 5 to 7), 3 (days 8 to 10), 4 (days 11 to 13), 5 (days 14 to 16), and 6 (days 17 to 30). For the backward shifted sensitivity analysis, the new time intervals under consideration are: pre-infection (days À30 to À12), pre-PCR (days À11 to À3), time of clinical presentation (days À2 to À1), and post-PCR phases 1 (days 0 to 2), 2 (days 3 to 5), 3 (days 6 to 8), 4 (days 9 to 11), 5 (days 12 to 14), and 6 (days 15 to 30). For both the forward and backward sensitivity analyses, we apply the same thresholds of effect size and significance (Cohen's D > 0.35, BHadjusted Mann-Whitney p-value <0.05), and we compare the results to the original time intervals.

Formal analysis, Validation, Investigation, Methodology, Writing -original draft, Writing -review and editing

Formal analysis, Supervision, Validation, Investigation, Methodology, Writing -original draft, Project administration, Writing -review and editing

Arjun Puranik, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writingreview and editing

Karthik Murugadoss, Software, Formal analysis, Validation, Investigation, Methodology, Writing -review and editing

Formal analysis, Supervision, Investigation, Methodology

Formal analysis, Validation, Investigation, Methodology, Writing -review and editing

Supervision, Validation, Investigation, Methodology, Writing -original draft, Project administration, Writing -review and editing

Formal analysis, Supervision, Validation, Investigation, Methodology, Project administration, Writing -review and editing

Supervision, Investigation, Project administration, Writing -review and editing

Supervision, Investigation, Methodology

Supervision, Project administration

Supervision, Validation, Project administration, Writing -review and editing

Supervision, Validation, Investigation, Methodology, Writingreview and editing; Najat Khan, Conceptualization, Resources, Formal analysis, Supervision, Validation, Investigation, Methodology, Writing -review and editing

Conceptualization, Resources, Formal analysis, Supervision, Funding acquisition, Validation, Investigation, Methodology, Writing -original draft, Writing -review and editing Author ORCIDs Colin Pawlowski

Venky Soundararajan

Study of COVID-19 patient characteristics with augmented curation of Electronic Health Records (EHR) to inform strategic and operational decisions". All analysis of EHRs was performed in the privacy-preserving environment secured and controlled by the Mayo Clinic. nference, the Mayo Clinic

An introduction to propensity score methods for reducing the effects of confounding in observational studies

A comparison of 12 algorithms for matching on the propensity score

COVID-19) -Symptoms and Causes -Mayo Clinic

BERT: pre-training of deep bidirectional transformers for language understanding

The significance of disseminated intravascular coagulation on multiple organ dysfunction during the early stage of acute respiratory distress syndrome

Diagnostic utility of clinical laboratory data determinations for patients with the severe COVID-19

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Incidence of thrombotic complications in critically ill ICU patients with COVID-19

Coagulation abnormalities and thrombosis in patients with COVID-19

Hypercoagulability of COVID-19 patients in intensive care unit: a report of thromboelastography findings and other parameters of hemostasis

Abnormal coagulation parameters are associated with poor prognosis in patients with novel coronavirus pneumonia

Augmented curation of clinical notes from a massive EHR system reveals symptoms of impending COVID-19 diagnosis

Website. 2020. Symptoms of coronavirus

Mechanism of thrombocytopenia in COVID-19 patients

Thrombocytopenia and its association with mortality in patients with COVID-19

D-dimer levels on admission to predict in-hospital mortality in patients with Covid-19

The authors thank Mathai Mammen, James List, JoAnne Foody, Patrick Lenehan, Murali Aravamudan, Rakesh Barve, Sankar Ardhanari, and Vishy Thiagarajan, for their helpful feedback.

From this analysis, we observe consistent results (i.e. comparisons meeting same criteria of significance and effect) on (i) both perturbations in 83 out of 130 (64%) lab test trends identified in Table 2 and (ii) at least one perturbation in 114 of 130 (87%) lab test trends. In Table 3 , we report the specific results of the time shifted windows for five coagulation-related lab tests (fibrinogen, platelets, prothrombin time, activated partial thromboplastin time, and D-dimer).Augmented curation of anticoagulant administration and the coagulopathy outcomes from the unstructured clinical notes and their triangulation to structured EHR databases A state-of-the-art BERT-based neural network (Devlin et al., 2018) was previously developed to classify sentiment regarding a diagnosis in the EHR (Wagner et al., 2020) . Sentences containing phenotypes were classified into the following categories: Yes (confirmed diagnosis), No (ruled out diagnosis), Maybe (possibility of disease), and Other (alternate context, e.g. family history of disease). The neural network used to perform this classification was trained using nearly 250 different phenotypes and 18,500 sentences and achieves 93.6% overall accuracy and over 95% precision and recall for Yes/No sentiment classification (Wagner et al., 2020) . Here, this model was used to classify the sentiment around coagulopathies in the unstructured text of the 246 COVID pos and 13,666 COVID neg patients' clinical notes, structuring this information so that it could be compiled with longitudinal lab measurement and medication information.In particular, we used the BERT model to identify the seven coagulopathy phenotypes mentioned in clinical notes in the Mayo Clinic EHR database, including: deep vein thrombosis, pulmonary embolism, myocardial infarction, venous thromboembolism, thrombotic stroke, cerebral venous thrombosis, and disseminated intravascular coagulation. We validated the performance of this model for these phenotypes on a set of 1000 randomly selected sentences from the clinical notes of the patients in the study population. In Table 6 , we report the out-of-sample accuracy metrics for the BERT model on this set of sentences, using manually curated labels provided by one of the study's authors (CP) to be the ground truth. We demonstrate that the model performs well in the task of identifying thrombotic phenotypes in clinical notes, with an overall accuracy of 94.7%, recall of 97.8%, and precision of 92.8%. 

After publication, the data will be made available to others upon reasonable requests to the corresponding author. A proposal with detailed description of study objectives and statistical analysis plan will be needed for evaluation of the reasonability of requests. Deidentified data will be provided after approval from the corresponding author and Mayo Clinic.