key: cord-0841653-u0vk34tf
authors: Delgado, M. Kit; Morgan, Anna U.; Asch, David A.; Xiong, Ruiying; Kilaru, Austin S.; Lee, Kathleen C.; Do, David; Friedman, Ari B.; Meisel, Zachary F.; Snider, Christopher K.; Lam, Doreen; Parambath, Andrew; Wood, Christian; Wilson, Chidinma M.; Perez, Michael; Chisholm, Deena L.; Kelly, Sheila; O’Malley, Christina J.; Mannion, Nancy; Huffenberger, Ann Marie; McGinley, Susan; Balachandran, Mohan; Khan, Neda; Mitra, Nandita; Chaiyachati, Krisda H.
title: Comparative Effectiveness of an Automated Text Messaging Service for Monitoring COVID-19 at Home
date: 2021-11-16
journal: Ann Intern Med
DOI: 10.7326/m21-2019
sha: a4f0adec4a1b8bf6b15926773be8f92b0de3ba82
doc_id: 841653
cord_uid: u0vk34tf

BACKGROUND: Although most patients with SARS-CoV-2 infection can be safely managed at home, the need for hospitalization can arise suddenly. OBJECTIVE: To determine whether enrollment in an automated remote monitoring service for community-dwelling adults with COVID-19 at home (“COVID Watch”) was associated with improved mortality. DESIGN: Retrospective cohort analysis. SETTING: Mid-Atlantic academic health system in the United States. PARTICIPANTS: Outpatients who tested positive for SARS-CoV-2 between 23 March and 30 November 2020. INTERVENTION: The COVID Watch service consists of twice-daily, automated text message check-ins with an option to report worsening symptoms at any time. All escalations were managed 24 hours a day, 7 days a week by dedicated telemedicine clinicians. MEASUREMENTS: Thirty- and 60-day outcomes of patients enrolled in COVID Watch were compared with those of patients who were eligible to enroll but received usual care. The primary outcome was death at 30 days. Secondary outcomes included emergency department (ED) visits and hospitalizations. Treatment effects were estimated with propensity score–weighted risk adjustment models. RESULTS: A total of 3488 patients enrolled in COVID Watch and 4377 usual care control participants were compared with propensity score weighted models. At 30 days, COVID Watch patients had an odds ratio for death of 0.32 (95% CI, 0.12 to 0.72), with 1.8 fewer deaths per 1000 patients (CI, 0.5 to 3.1) (P = 0.005); at 60 days, the difference was 2.5 fewer deaths per 1000 patients (CI, 0.9 to 4.0) (P = 0.002). Patients in COVID Watch had more telemedicine encounters, ED visits, and hospitalizations and presented to the ED sooner (mean, 1.9 days sooner [CI, 0.9 to 2.9 days]; all P < 0.001). LIMITATION: Observational study with the potential for unobserved confounding. CONCLUSION: Enrollment of outpatients with COVID-19 in an automated remote monitoring service was associated with reduced mortality, potentially explained by more frequent telemedicine encounters and more frequent and earlier presentation to the ED. PRIMARY FUNDING SOURCE: Patient-Centered Outcomes Research Institute.

We included all community-dwelling adults who tested positive for SARS-CoV-2 at Penn Medicine as outpatients between 23 March (the start of COVID Watch) and 30 November 2020 and determined if they were enrolled in COVID Watch versus usual care. We excluded patients who did not meet eligibility criteria for COVID Watch: those who were younger than 18 years, were actively enrolled in home health services or hospice, or were currently in long-term care (for example, skilled-nursing facility, long-term acute care, or acute rehabilitation). To further reduce the potential for bias, we excluded those who were tested in a location where COVID Watch enrollment had not yet begun, were previously in long-term care or hospice, or had a documented do not resuscitate or do not intubate code status before COVID-19 test collection. If patients had multiple positive test results for COVID-19, the first test with a positive result was chosen as the index test. We excluded patients enrolled in COVID Watch more than 7 days before or after the date of their index COVID-19 test collection to avoid attribution of outcomes to COVID Watch enrollment for other episodes of care (for example, repeated COVID-19 testing).

We derived patient-level sociodemographic and clinical characteristics from Penn Medicine's electronic health record. We derived patient death data, clinical encounter details (ED visits, hospitalizations, outpatient office visits, and telemedicine), and documented details (for example, encounter dates and in-hospital vs. out-of-hospital mortality) from Penn Medicine's electronic record and a regional health information exchange containing data from 53 surrounding hospitals in Pennsylvania, New Jersey, and Delaware (14) .

Outcomes were ascertained within 30 days of the date of COVID-19 test collection. We used an intention-to-treat approach in which any patient enrolled in COVID Watch was included in the COVID Watch group even if they did not respond to any text messages. The primary outcome was any-site mortality. We separately analyzed deaths that occurred in hospital or out of hospital. Secondary outcomes included rates of total ED encounters (including discharges and hospitalizations), hospitalizations (inpatient admissions and observation), and outpatient encounters (in-person office visits and telemedicine, including video or telephone visits). We tabulated a composite outcome of days alive and out of the hospital, which factors in death and the ability to remain out of the hospital by accounting for ED visits and, if hospitalized, the length of stay (15-18). As a secondary analysis, we extended the follow-up window to 60 days for mortality and health care use measures.

Among patients who required acute care, we tabulated the time from the collection of the positive result to ED presentation, the ED vital signs, the length of stay if hospitalized, and whether the patient needed intubation and mechanical ventilation. Finally, we tabulated process outcomes among those enrolled in COVID Watch, including how many text message check-ins they responded to, the proportion who triggered an escalation to the on-call nurse, and the triage recommendations of the on-call nurse.

We collected patients' COVID-19 test data, age, sex, race/ethnicity, primary insurance, county of residence, and household income derived from ZIP code median values. We included comorbidities known to be associated with severe COVID-19 illness or treatment adherence (19) (20) (21) (22) (23) (24) . We captured whether patients had a listed primary care provider and baseline health care use,

Outcomes of Automated Home Monitoring for COVID-19 

All extracted variables were checked for outliers and missingness. Only 21 patients of 7865 who met inclusion criteria (0.27%) were missing any covariate data and excluded from the primary analysis. To account for imbalances on covariates between comparison groups in the outpatient cohort, we estimated propensity scores (the probability of enrollment in COVID Watch) using logistic regression. After using inverse probability of treatment weighting, we found that all covariates achieved balance between patients enrolled in COVID Watch and usual care with mean standardized differences of less than 0.1 (25) . Outcome models weighted by inverse probability of treatment weights were either logistic for binary outcomes or linear for continuous outcomes. We did a priori subgroup analyses of outcomes by race/ethnicity.

To assess the robustness of our findings, we assessed the sensitivity of our mortality results to potential unmeasured confounding by using the Rosenbaum g approach. We also tabulated any deaths that occurred in patients with missing covariates (n = 21). We also did a per protocol analysis, excluding any patients enrolled in COVID Watch who did not respond to any text messages, including the enrollment invitation text. Finally, we tabulated baseline characteristics and diagnoses of patients who died. All analyses were done using R, version 3.6.0 (R Project for Statistical Computing). All analyses used 2-sided statistical tests, and a P value less than 0.05 was considered statistically significant. Additional methodological details about the development of propensity scores can be found in the Supplement (available at Annals.org).

The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

After ineligible patients (Appendix Figure, available at Annals.org) were excluded, 7865 outpatients were available for analysis: 3488 patients enrolled in COVID Watch and 4377 in usual care. Patients in COVID Watch were enrolled a mean of 1.8 days (SD, 2.3) after the date of COVID-19 test collection. Before propensity score weighting, patients enrolled in COVID Watch were similar to those who received usual care in age but were less likely to be male (37.1% vs. 42.5%) and were more likely to identify as non-Hispanic Black (47.9% vs. 32.1%), have public insurance (29.3% vs. 26.6%), not have a primary care provider (28.3% vs. 24.5%), have a higher mean body mass index (33.6 vs. 29.7 kg/m 2 ), have certain comorbidities (hypertension, diabetes, and asthma), have a lower median household income ($51 491 vs. $60 638), have higher rates of ED and office visits in the past year, and be enrolled in the earlier months of the pandemic ( Table 1 ; Supplement Figure 3 , available at Annals. org). Covariates were well balanced after propensity score weighting ( Table 2 ; Supplement Figure 4 , available at Annals.org).

Of the 3488 patients enrolled in COVID Watch, 3028 (86.8%) engaged by responding to at least 1 text (mean, 23 check-in responses). Of patients who engaged, 434 (14.3%) triggered an escalation to a registered nurse, with mean response time of 24 minutes (Supplement Table 1 , available at Annals.org).

At 30 days, 3 of 3488 (0.09%) patients enrolled in COVID Watch died versus 12 of 4377 (0.27%) who received usual care ( Table 3) . Of the deaths, 0 in COVID Watch occurred outside the hospital versus 6 of those who received usual care. Among in-hospital deaths, 2 of the 3 in COVID Watch occurred in hospitals outside of Penn Medicine compared with 5 of 6 in the usual care group. At 60 days, there were 2 additional in-hospital deaths among those enrolled in COVID Watch and 4 among those who received usual care. In total, 37.5% of the deaths at 60 days in the usual care group occurred outside the hospital. After propensity score weighting and modeling, at 30 days, those enrolled in COVID Watch had an odds ratio for overall mortality of 0.32 (95% CI, 0.12 to 0.72), with a difference of À1.8 deaths per 1000 patients (CI, À3.1 to À0.5) (P = 0.005). At 60 days, COVID Watch outcomes remained consistently better, with a difference of À2.5 deaths per 1000 patients (CI, À4.0 to À0.9) (P = 0.002). Furthermore, we found that COVID Watch was associated with equitable treatment benefits, with White, Black, and Hispanic subgroups all having reduced mortality by 60 days ( Table 3) .

Within 30 days, COVID Watch patients had a total of 489 ED encounters (121 per 1000), with 384 (78.5%) 

Outcomes of Automated Home Monitoring for COVID-19 Among patients who presented to any hospital (within Penn Medicine or outside) for the first time in the 30 days after their date of COVID-19 test collection (Table 4) , COVID Watch patients presented to the ED sooner (6.6 vs. 8.9 days) (propensity score-weighted difference, À1.9 days [CI, À2.9 to À0.9 days]; P < 0.001).

Among the subset of patients who presented to the ED within Penn Medicine (Table 5) , COVID Watch enrollees compared with usual care patients presented to the ED even sooner (6.1 vs. 9.0 days) (propensity scoreweighted difference, À2.9 days [CI, À4.1 to À1.7 days]; P < 0.001). During the ED evaluation, there were no statistically significant differences in vital signs or need for intubation and ventilation. However, compared with the 10.8% of usual care patients who received dexamethasone in Penn Medicine hospitals, the 11.3% of COVID Watch patients who were treated with dexamethasone received it sooner (propensity score-weighted difference, À3.0 days [CI, À5.6 to À0.4 days]; P = 0.026).

The Rosenbaum bounds analysis showed there would need to be 1.8 times greater odds of differential assignment to COVID Watch attributable to unobserved factors, a substantial amount of unmeasured confounding needed to reverse the statistically significant findings (Supplement Table 2 , available at Annals.org). There were also no deaths among the 21 patients excluded from the outpatient cohort because of missing covariate data (Supplement Table 3 , available at Annals.org). We also found that 2 of the 5 deaths in COVID Watch occurred among the 13.2% of the patients who never engaged with the system. The per protocol analysis of patients who engaged with COVID Watch indicated even stronger treatment effects: odds ratio for death of 0.25 (CI, 0.10 to 0.55), with a difference of À2.8 deaths per 1000 patients (CI, À4.3 to À1.3 deaths) (P = 0.001) (Supplement Table 4 , available at Annals.org). Finally, baseline characteristics of patients who died varied but were not statistically significant from each other across treatment groups (Supplement Table 5 , available at Annals.org), and coded diagnoses of in-hospital deaths were consistent with COVID-19 being the primary cause of death (Supplement Table 6 , available at Annals.org).

This study has 4 main findings. First, the mortality rate for community-dwelling adults with COVID-19 was significantly lower among those in COVID Watch compared with usual care, even after adjustment for differences in patients' clinical and sociodemographic characteristics. Second, more than one third of the deaths in the usual care group occurred outside the hospital versus none among those in COVID Watch. Third, patients in COVID Watch were more likely to present to the hospital, and they presented earlier. Fourth, all major racial and ethnic subgroups had reduced mortality rates when enrolled in COVID Watch.

These findings imply that COVID Watch is associated with a 64% relative reduction in the risk for death and that 1 life was saved for every 400 patients enrolled-or about 1 every 4 days during peak enrollment weeks. Although remote patient monitoring programs used to manage patients with COVID-19 outside of hospital settings have been described (26, 27) , including 1 study from Kaiser Permanente that reported unadjusted morality rates of 2.3% in usual care versus 1.3% with remote monitoring (28), we believe ours to be the first risk-adjusted study to show improved survival.

Public health messaging strongly promoted staying home to promote social distancing and decrease hospital strain during the pandemic (29, 30) . However, those 

Outcomes of Automated Home Monitoring for COVID-19 messages were accompanied by decreases in emergent conditions presenting to the ED and increased out-of-hospital deaths (8, 9, (31) (32) (33) (34) (35) (36) (37) (38) (39) . In this study, 37.5% of deaths among patients who received usual care occurred outside the hospital versus none among patients in COVID Watch, which is consistent with the interpretation that COVID Watch exerts its effect by increasing vigilance over those at home and efficiently sorting them into those who will benefit from the ED and those who will not (8, 32) . Further evidence supports this mechanistic hypothesis. Patients in COVID Watch were more likely to present to the hospital and presented earlier, likely improving their ability to benefit from the care they receive. For example, dexamethasone reduces mortality and length of stay for patients with COVID-19 (40) , and the benefit may be larger if the drug is administered earlier in the disease course (41) (42) (43) (44) (45) . We found that among those who received dexamethasone in Penn Medicine hospitals, COVID Watch patients received the medication 3 days earlier on average. We also found that the treatment effects associated with COVID Watch were stronger among those who engaged with the remote monitoring service; 2 of the 5 deaths in the COVID Watch group were among the 13% of enrolled patients who never engaged the system. The constellation of these findings is consistent with the view that COVID Watch operates as an early warning and referral system for communitydwelling patients (13).

The combination of technology-based, automated remote monitoring (46) backed by clinician support may be necessary ingredients for the observed clinical effect. Because COVID Watch was automated, only 2 to 4 staff members were required to oversee more than 1000 patients at a time, far fewer than personnel-intensive calling systems (26, 27) . Because it relied on symptom selfreport, COVID Watch did not require dedicated temperature sensors or pulse oximetry (12, 27, (47) (48) (49) . The use of additional equipment in the home varies substantially across remote patient monitoring programs, and its incremental value is unknown (50). Future research is needed to determine whether this type of monitoring service could be adapted to other acute conditions (for example, pneumonia or cellulitis) and chronic conditions (for example, asthma or diabetes) in which automated text check-ins and low barrier access to rapid clinical assessment and ED triage could improve outcomes.

In our study, non-Hispanic Black patients were more likely to be enrolled in COVID Watch than usual care, and Hispanic patients were about as likely to be in either group. White, Black, and Hispanic populations also had significantly reduced mortality when enrolled in COVID Watch, and the overall mortality rates were lower relative to reports nationally (28, 39, 51, 52) . These findings suggest no substantial racial or ethnic barriers to program enrollment or its effectiveness and that implementing this type of remote monitoring service has the potential to reduce racial disparities in regions in which Black and Hispanic patients have decreased access to care and higher mortality rates.

This study has limitations. First, we could observe deaths in our hospitals and hospitals outside our health system via a health information exchange linkage, but we may have incomplete ascertainment of out-of-hospital deaths. Death certificate linkage via the National Death Index was not available at the time of manuscript submission because of lag times in these databases. However, COVID Watch patients were highly engaged with Penn Medicine, replying to a mean of 23 text message checkins through the program, which suggests that their deaths would more likely be ascertained. Table 2 . ‡ This is a composite metric that accounts for ED visits and, if hospitalized, the length of stay as well as days after death. § Telemedicine visits included video and telephone visits.

Outcomes of Automated Home Monitoring for COVID-19

Second, we cannot capture hospital use outside of the geography of health information exchange. Fortunately, 99.4% of our study sample had residential ZIP codes within the geographic region covered by the health information exchange, decreasing the potential for incomplete capture of hospital use.

Third, we cannot capture the reasons patients were enrolled in COVID Watch or whether patients were verbally offered COVID Watch and declined it. We also cannot capture social needs, the nature and timing of symptoms before the testing date, and other unobserved confounders that may affect outcomes. However, higher rates of characteristics associated with worse outcomes from COVID-19 were seen in the COVID Watch group, including the lack of a primary care physician, Black race, residing in a lower-income ZIP code, greater body mass index, higher rates of high-risk comorbidities, and higher proportion treated early in the pandemic (6, 24, (53) (54) (55) , all suggesting higher expected mortality among the COVID Watch group. Furthermore, our sensitivity analyses indicated there would need to be a 1.8 times greater odds of differential assignment to COVID Watch versus the control group that was attributable to unobserved factors. Given the large number of important covariates we have accounted for in our analysis, it is unlikely that such an impactful covariate was not included.

Fourth, outcomes measured reflect care received at a single health system, a select set of hospitals in a specific region of the United States, which may limit generalizability of our findings. However, this study included populations with a diverse set of comorbidities and sociodemographic characteristics. Relatedly, we were unable to measure clinical status and treatments provided during hospital encounters outside our health system. Given the differential use of hospitals outside our health system, clinical treatment rates seen within our health system should not be extrapolated to the subset treated in hospitals outside our health system.

This study also has strengths. It reflects what is, to our knowledge, the largest and most comprehensive sample and evaluation of a remote monitoring service for COVID-19 in the United States. There is careful adjustment for differences in patient characteristics that have a conservative bias. The effect size is large, and the results are accompanied by plausible mechanisms whose specific elements are suggested by secondary analyses.

In conclusion, enrollment in an automated text messaging service among community-dwelling adults newly diagnosed with COVID-19 in outpatient settings was associated with reduced mortality, potentially explained by increased and earlier presentation to the ED by those benefiting from early interventions. These results reveal a model for outpatient health system management of patients with COVID-19 and possibly other conditions where the early detection of clinical declines is critical. Table 2 .

Trends in number of COVID-19 cases and deaths in the US reported to CDC, by state/ territory

Hospitalization rates and characteristics of patients hospitalized with laboratory-confirmed coronavirus disease 2019 -COVID-NET, 14 states

COVID-19: what to do if you are sick

Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention

Mild or moderate Covid-19

Variation in US hospital mortality rates for patients admitted with COVID-19 during the first 6 months of the pandemic

Clinical trends among US adults hospitalized with COVID-19

Where are all the patients? Addressing Covid-19 fear to encourage sick patients to seek emergency care. NEJM Catal Innov Care Deliv

National Syndromic Surveillance Program Community of Practice. Impact of the COVID-19 pandemic on emergency department visits -United States

Admissions to Veterans Affairs hospitals for emergency conditions during the COVID-19 pandemic

The value of remote monitoring for the COVID-19 pandemic

Remote monitoring of patients with Covid-19: design, implementation, and outcomes of the first 3,000 patients in COVID Watch. NEJM Catal Innov Care Deliv

Standardised Endpoints in Perioperative Medicine-Core Outcome Measures in Perioperative and Anaesthetic Care (StEP-COMPAC) Group. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine initiative: patient-centred outcomes

Alirocumab reduces total hospitalizations and increases days alive and out of hospital in the ODYSSEY OUTCOMES trial

Days alive and out of hospital: exploring a patient-centered, pragmatic outcome in a clinical trial of patients with acute coronary syndromes

China Medical Treatment Expert Group for COVID-19. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized ORIGINAL RESEARCH Outcomes of Automated Home Monitoring for COVID-19 patients with COVID-19

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Return hospital admissions among 1419 COVID-19 patients discharged from five U.S. emergency departments

Risk factors for Covid-19 severity and fatality: a structured literature review

Development and validation of the quick COVID-19 severity index: a prognostic tool for early clinical decompensation

STOP-COVID Investigators. Factors associated with death in critically ill patients with coronavirus disease 2019 in the US

Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies

Rapid implementation of an outpatient Covid-19 monitoring program. NEJM Catal Innov Care Deliv

Virtual health care for community management of patients with COVID-19 in Australia: observational cohort study

A novel large scale integrated telemonitoring program for COVID-19

Timing of state and territorial COVID-19 stay-at-home orders and changes in population movement -United States

Association of stay-at-home orders with COVID-19 hospitalizations in 4 states

Delayed emergencies: the composition and magnitude of non-respiratory emergency department visits during the COVID-19 pandemic

Do not stay at home: we are ready for you. NEJM Catal Innov Care Deliv

Trends in emergency department visits and hospital admissions in health care systems in 5 states in the first months of the COVID-19 pandemic in the US

Hospital volumes of 5 medical emergencies in the COVID-19 pandemic in 2 US medical centers

Reduction in ST-segment elevation cardiac catheterization laboratory activations in the United States during COVID-19 pandemic

Excess cardiac arrest in the community during the COVID-19 pandemic

Excess out-of-hospital deaths during the COVID-19 outbreak: evidence of pulmonary embolism as a main determinant

Northwell COVID-19 Research Consortium. The association of structural inequities and race with out-of-hospital sudden death during the COVID-19 pandemic

Dexamethasone in hospitalized patients with Covid-19

Role of the early shortcourse corticosteroids treatment in ARDS caused by COVID-19: a single-center, retrospective analysis

Early short-course corticosteroids in hospitalized patients with COVID-19

COVID-19 Spanish ICU Network. Early corticosteroids are associated with lower mortality in critically ill patients with COVID-19: a cohort study

Early intervention likely improves mortality in COVID-19 infection

Design of the pilot, proof of concept REMOTE-COVID trial: remote monitoring use in suspected cases of COVID-19 (SARS-CoV-2)

Novel use of home pulse oximetry monitoring in COVID-19 patients discharged from the emergency department identifies need for hospitalization

University College London Hospitals COVID response team. Implementation and evaluation of a COVID-19 rapid follow-up service for patients discharged from the emergency department

Assessment of racial/ethnic disparities in hospitalization and mortality in patients with COVID-19 in

Risk for COVID-19 infection, hospitalization, and death by race/ethnicity

Hospitalization and mortality among Black patients and White patients with Covid-19

Characteristics, outcomes, and trends of patients with COVID-19-related critical illness at a learning health system in the United States

Trends in COVID-19 risk-adjusted mortality rates

Author Contributions: Conception and design

Provision of study materials or patients

Outcomes of Automated Home Monitoring for COVID-19Appendix Figure. Study sample flowchart and patients excluded.