key: cord-0719804-p20bpx38
authors: Berdahl, C. T.; Nguyen, A. T.; Diniz, M. A.; Henreid, A. J.; Nuckols, T. K.; Libby, C. P.; Pevnick, J. M.
title: Using body temperature and variables commonly available in the EHR to predict acute infection: A proof-of-concept study showing improved pretest probability estimates for acute COVID-19 infection among discharged emergency department patients
date: 2021-01-22
journal: nan
DOI: 10.1101/2021.01.21.21250261
sha: 403267deeb22a8292a0c0df599daa4c38d349656
doc_id: 719804
cord_uid: p20bpx38

Objectives: Obtaining body temperature is a quick and easy method to screen for acute infection such as COVID-19. Currently, the predictive value of body temperature for acute infection is inhibited by failure to account for other readily available variables that affect temperature values. In this proof-of-concept study, we sought to improve COVID-19 pretest probability estimation by incorporating covariates known to be associated with body temperature, including patient age, sex, comorbidities, month, time of day. Methods: For patients discharged from an academic hospital emergency department after testing for COVID-19 in March and April of 2020, we abstracted clinical data. We reviewed physician documentation to retrospectively generate estimates of pretest probability for COVID-19. Using patient COVID-19 PCR test results as a gold standard, we compared AUCs of logistic regression models predicting COVID-19 positivity that used: 1) body temperature alone; 2) body temperature and pretest probability; 3) body temperature, pretest probability, and body temperature-relevant covariates. Calibration plots and bootstrap validation were used to assess predictive performance for model #3. Results: Data from 117 patients were included. The models AUCs were: 1) 0.69 2) 0.72, and 3) 0.76, respectively. The absolute difference in AUC was 0.029 (95%CI -0.057 to 0.114, p=0.25) between model 2 and 1 and 0.038 (95%CI -0.021 to 0.097, p=0.10) between model 3 and 2. Conclusions: By incorporating covariates known to affect body temperature, we demonstrated improved pretest probability estimates of acute COVID-19 infection. Future work should be undertaken to further develop and validate our model in a larger, multi-institutional sample.

Improving the detection of acute COVID-19 infection is critical to minimizing spread of the infection and limiting avoidable morbidity and mortality. [1] However, the ability to diagnose COVID-19 is often limited because of asymptomatic or mildly symptomatic infection;[2] limited testing availability;[3] and slow turnaround time for test results. Given these limitations, improving pretest probability estimation is a crucial way to improve decision-making about when testing is warranted and thus curb community spread of acute infection. [4] Measuring body temperature is a quick, easy, and almost ubiquitously available way to evaluate for acute infection. However, the use of temperature to diagnose COVID-19 has not reached its fullest potential for three reasons. First, clinicians tend to use temperature as a binary variable, often with a cutoff of 100.4 degrees Fahrenheit signaling active infection.

[5] Thus, many infections can be missed when patients exhibit temperatures that are mildly elevated.

[6] Second, failure to account for the effects of other variables on temperature values lead to missed opportunities to detect active infection. Differences in measured body temperature and/or a patient's ability to mount an elevated temperature in the setting of acute infection occur with: patient age, sex, baseline body temperature, and comorbidities (e.g. depression, allergic conditions); time of day; ambient temperature; calendar month; and route of body temperature measurement.[7-9] Third, our clinical experience suggests that, while clinicians are aware of some of these relationships that should alter the interpretation of temperature values, they probably do not account for them effectively when making clinical decisions.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint

We propose that it may be possible to improve pretest probability estimation for COVID-19 by developing a novel model that includes variables readily available in the electronic health record (EHR). Figure 1 depicts a conceptual model demonstrating how body temperature-related covariates influence pretest probability, including how a clinician should decide to test for COVID-19 and act on results. Estimating pretest probability is critical to determining which patients should be tested. Additionally, because false-negative rates for COVID-19 testing have been found to range between 2% and 29%,[10] estimating pretest probability has been strongly encouraged to determine which patients should get a second test, even after an initial negative test. [11] In this proof-of-concept study of model development, we sought to evaluate whether pretest probability estimates of COVID-19 could be improved by incorporating covariates known to be associated with body temperature, including demographics, comorbidities, time of day, and month of the year. We hypothesized that adding body temperature-relevant covariates would improve the area under the curve (AUC) for a logistic regression model predicting COVID-19 polymerase chain reaction (PCR) test positivity.

This study was conducted at the emergency department (ED) of an academic medical center in Los Angeles, CA. The study population included patients who were triaged to a temporary COVID-19 surge tent (open from 7:00am to 11:00pm), tested for acute COVID-19 infection with a nasopharyngeal PCR test, and discharged home for outpatient management between March 12 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint and April 6, 2020. Patients were excluded from the study if they did not have an oral temperature reading, if they were previously seen in the ED within the prior 7 days, if they did not complete ED evaluation (i.e., left without being seen, left against medical advice, or eloped), or if they reported no symptoms.

Study procedures involved accessing an existing operational database to collect variables characterizing: patient age, sex, vital signs (including date, time, and route of measurement), comorbidities, and COVID-19 PCR test results from nasal swab samples. [12] Additionally, the content of physician and physician assistant notes describing patients' visits was reviewed.

The pilot model predicted acute COVID-19 infection at the individual patient level based on results of COVID-19 PCR testing performed during the ED visit.

Selected predictor variables were covariates known to be associated with individual-level body temperature, including age, sex, month and time of day of temperature measurement, and comorbidities [8, 9, 13, 14] . Body temperature was operationally defined as the highest temperature measurement taken during the ED visit. Time of day of body temperature measurement ranged from 9:00AM to 9:00PM and was defined as a continuous variable, derived by squaring the nearest rounded hour of body temperature measurement. Comorbidities associated with individual variation in baseline body temperature and considered for inclusion as predictor variables were cancer, pulmonary disease, hypothyroidism, kidney disease, congestive . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint heart failure, allergies, and depression [9, 13] . Comorbidities were enumerated through study team review of diagnoses listed in the patient's past medical history.

Because body temperature is a key predictor for acute COVID-19 infection, we included it directly in our final model based on structured EHR data rather than asking clinicians to incorporate it in an overall pretest probability. To avoid using body temperature twice, we used clinicians' estimates of pretest probability without access to objective body temperature data.

Two study team members trained in emergency medicine (CB and CL) independently reviewed clinical documentation (ED notes from physician assistants and physicians) to estimate pretest probability for COVID-19 while blinded to patients' ED visit temperature measurements.

Reviewers rated each case on a 5-point ordinal scale (1=20% suspicion or less; 5=80% suspicion or more). To determine estimates, reviewers were instructed to consider clinically relevant contextual factors, including local prevalence of COVID-19 at the time of the visit; patientreported history of recent COVID-19 exposure within the last 14 days; patient-reported symptoms (e.g., fever, chills, cough, shortness of breath, chest tightness, headache, sore throat, fatigue, body aches, diarrhea, abdominal pain, confusion, loss of taste, loss of smell); and objective criteria (e.g., vital signs, excluding redacted temperature). In cases where reviewers' estimates differed by exactly one point, the estimates were averaged to obtain a mean pretest probability. If estimates differed by more than one point, the two reviewers discussed the case . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint until consensus was achieved. Physician pretest probability estimates were included as a continuous predictor variable in statistical modeling.

Descriptive statistics were obtained using SPSS Statistics version 24.0 (IBM Corporation, Armonk, New York, USA) to compare demographics, comorbidities, and clinical characteristics of patients who tested positive and negative for acute COVID-19 infection. Between-group differences in demographics, comorbidities, and clinical characteristics were assessed using independent samples t-tests, Mann Whitney U tests, Pearson's chi-square tests of independence, or Fisher's exact tests. Intraclass correlation coefficients (ICCs) were calculated to estimate inter-rater agreement on physician pretest probability estimates [15, 16] .

Bivariable and multivariable logistic regression models were fitted to the data to test the research hypotheses that independent variables are predictive of acute COVID-19 infection.

Three pilot models were estimated with the following combinations of predictor variables: Predictive performance was evaluated using the area under the curve (AUC) and receiver operating curve (ROC) for discrimination, and calibration-in-large and calibration slope for calibration. Model performance, when evaluated in the same sample that was fitted, is overestimated. Therefore, we calculated the optimism of predictive performance measures using . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint bootstrap to obtain optimism bias-corrected estimates. [17, 18] The significance of predictor main effects in bivariable and multivariable models were assessed using likelihood ratio chi-square tests, calculated as twice the difference of the log-likelihoods between the full model and the constrained model that that does not contain the effect. Differences in AUCs between models were tested for statistical significance using Delong's Test with alpha set to 0.05 and using a one-sided test. (Table 1) .

For the 117 patients in our study, COVID-19 pretest probability estimation by dual reviewers yielded identical scores in 55 (43.0%) cases, adjacent scores in 66 (51.6%) cases; and scores initially differing by more than 2 or more points in 7 (5.5%), which were resolved by adjudication. The intraclass correlation coefficient (ICC) for pretest probability estimation was 0.75 (95% CI 0.64-0.83).

After pretest probability estimates were determined, we used our data to derive . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Table 2 for a list of AUCs for the three models, Table 3 

Based on our a priori hypothesis that several covariates known to be associated with body temperature in non-infected subjects [8, 9, 13, 14] could improve the performance of body . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint temperature as a predictor of COVID-19 acute infection status, we compared the predictive performance of three models. These models included oral body temperature alone, physicians' temperature-blinded pretest probability plus objective temperature data, and a proposed enhanced model (which added age, sex, month, time of day, and comorbidities), which increased the AUC from 0.69 to 0.76-a statistically nonsignificant difference but a trend which should undergo investigation in further research.

Clinicians have long known that covariates are relevant to the interpretation of body temperature. Until now, however, most consideration of covariates' effects has been handled qualitatively in clinicians' minds. To the extent that these associations hold true and can be validated in other datasets, they provide a method for clinicians to quantitatively adjust their interpretation of body temperature which could be used to produce an evidence-based adjusted body temperature for every patient encounter. This could be likened to using a corrected QT interval, which has superseded use of an unadjusted QT interval in nearly all clinical settings.

Because there are billions of patient encounters per year that use body temperature, the public health implications of even a tiny improvement in discriminatory ability are large.

We recognize that this initial analysis has several limitations. First, we expect that these AUCs are optimistic because they are evaluated in a derivation sample at a single site, [20] though we attempted to mitigate this effect by reporting optimism-adjusted performance measures of calibration and discrimination. Second, our data used a subset of patients who were tested for COVID-19 during a time when testing supplies were limited, making selection bias a potential limitation. Third, members of the research team who generated pretest probability estimates were not blinded to the study hypothesis regarding which covariates should affect interpretation of temperature. This may have caused their estimates to differ from estimates that . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint would otherwise by generated in practicing clinicians. Fourth, even when AUC is increased, it must be considered in a clinical context to ensure that yields clinically significant benefit. Fifth, we were unable to include two important classes of variables in our analysis because of gaps in the sample: patients' baseline temperature values and variables related to women's hormonal cycles. Accounting for patients' baseline temperature has been endorsed by the Infectious Disease Society of America [21] . Moreover, use of hormonal contraceptives and estrogen replacement therapy have been demonstrated to impact temperature interpretation,[7] which suggests that time since a woman's last menstrual period may also affect temperature elevation in the setting of acute infection. These variables were infeasible for us to collect in our proof-ofconcept study, but we believe that they would likely further enhance predictive performance of our model if we had been able to include them.

In conclusion, we present preliminary evidence that enhancing the interpretation of body temperature with body temperature-relevant covariates can improve discriminatory ability for COVID-19 above and beyond what is currently possible by using clinical suspicion and body temperature alone. If future research can validate these findings in other datasets, use of bodytemperature relevant covariates to generate an adjusted temperature could be a promising method to improve detection of COVID-19 and other febrile illnesses, and to curb their spread. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted January 22, 2021. ; Figure 1 . Conceptual model demonstrating how relevant factors (e.g. antipyretic use, time of day) should influence interpretation of body temperature and the resultant effect on pretest probability. Direct influence is shown in red. Note also that in the example shown with a high pretest probability, a positive test result nearly always implies a posttest probability that exceeds the threshold requiring further testing, whereas a negative result may still result in a high enough posttest probability that further testing is warranted.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.21.21250261 doi: medRxiv preprint

Combating the Coronavirus Pandemic: Early Detection, Medical Treatment, and a Concerted Effort by the Global Community

False Negative Tests for SARS

Infection -Challenges and Implications

The safety of home discharge for low-risk emergency department patients presenting with coronavirus-like symptoms during the COVID-19 pandemic: A retrospective cohort study

Individual differences in normal body temperature: Longitudinal big data analysis of patient records

Modern fertility awareness methods: wrist wearables capture the changes in temperature associated with the menstrual cycle

Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutorials in quantitative methods for psychology

Interreliability Standards in Psychological Evaluations

Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis

A tutorial on calibration measurements and calibration models for