key: cord-0733253-tmihhjap
authors: Chandna, Arjun; Mahajan, Raman; Gautam, Priyanka; Mwandigha, Lazaro; Gunasekaran, Karthik; Bhusan, Divendu; Cheung, Arthur T L; Day, Nicholas; Dittrich, Sabine; Dondorp, Arjen; Geevar, Tulasi; Ghattamaneni, Srinivasa R; Hussain, Samreen; Jimenez, Carolina; Karthikeyan, Rohini; Kumar, Sanjeev; Kumar, Shiril; Kumar, Vikash; Kundu, Debasree; Lakshmanan, Ankita; Manesh, Abi; Menggred, Chonticha; Moorthy, Mahesh; Osborn, Jennifer; Richard-Greenblatt, Melissa; Sharma, Sadhana; Singh, Veena K; Singh, Vikash K; Suri, Javvad; Suzuki, Shuichi; Tubprasert, Jaruwan; Turner, Paul; Villanueva, Annavi M G; Waithira, Naomi; Kumar, Pragya; Varghese, George M; Koshiaris, Constantinos; Lubell, Yoel; Burza, Sakib
title: Facilitating safe discharge through predicting disease progression in moderate COVID-19: a prospective cohort study to develop and validate a clinical prediction model in resource-limited settings
date: 2022-03-21
journal: Clin Infect Dis
DOI: 10.1093/cid/ciac224
sha: aae25816fa9186ed31d0dbc1735e77707db5f588
doc_id: 733253
cord_uid: tmihhjap

BACKGROUND: In locations where few people have received COVID-19 vaccines, health systems remain vulnerable to surges in SARS-CoV-2 infections. Tools to identify patients suitable for community-based management are urgently needed. METHODS: We prospectively recruited adults presenting to two hospitals in India with moderate symptoms of laboratory-confirmed COVID-19 in order to develop and validate a clinical prediction model to rule-out progression to supplemental oxygen requirement. The primary outcome was defined as any of the following: SpO(2) < 94%; respiratory rate > 30 bpm; SpO(2)/FiO(2) < 400; or death. We specified a priori that each model would contain three clinical parameters (age, sex and SpO(2)) and one of seven shortlisted biochemical biomarkers measurable using commercially-available rapid tests (CRP, D-dimer, IL-6, NLR, PCT, sTREM-1 or suPAR), to ensure the models would be suitable for resource-limited settings. We evaluated discrimination, calibration and clinical utility of the models in a held-out temporal external validation cohort. RESULTS: 426 participants were recruited, of whom 89 (21.0%) met the primary outcome. 257 participants comprised the development cohort and 166 comprised the validation cohort. The three models containing NLR, suPAR or IL-6 demonstrated promising discrimination (c-statistics: 0.72 to 0.74) and calibration (calibration slopes: 1.01 to 1.05) in the validation cohort, and provided greater utility than a model containing the clinical parameters alone. CONCLUSIONS: We present three clinical prediction models that could help clinicians identify patients with moderate COVID-19 suitable for community-based management. The models are readily implementable and of particular relevance for locations with limited resources.

In low-income countries, where fewer than 5% of people have received a COVID-19 vaccine, 1 fragile healthcare systems remain vulnerable to being overwhelmed by a surge in COVID-19 cases ( Figure   1 ). [2] [3] [4] A minority of patients with COVID-19 require admission to hospital. Oxygen is the most important supportive treatment and in most low-and middle-income countries (LMICs) is the practical ceiling of care. 5 The World Health Organization (WHO) estimates that 15% of patients with symptomatic COVID-19 will require supplemental oxygen. 6 Effective identification of patients who are unlikely to become hypoxic would have considerable benefit; tools to support triage could decompress healthcare systems by giving practitioners confidence to allocate resources more efficiently. 7 Numerous prognostic models for COVID-19 have been developed. 8, 9 Almost all predict critical illness or mortality and thus cannot inform whether a patient might be safely managed in the community.

Of those that focus on patients with moderate disease, most rely on retrospective or registry-based data, [10] [11] [12] [13] [14] lack external validation, 15, 16 and are not feasible for use in resource-limited settings. 9, 17 Moreover, most existing studies did not follow best-practice guidelines for model building and reporting, 18 are at high risk of bias, 8 and the resulting models are neither suitable nor recommended for use in LMIC contexts. 9 We set out to develop and validate a clinical prediction model to rule-out progression to supplemental oxygen requirement in patients presenting with moderate COVID- 19 . We hypothesised that combining simple clinical parameters with host biomarkers feasible for A c c e p t e d M a n u s c r i p t Page 7 measurement in resource-limited settings and implicated in the pathogenesis of COVID-19 would improve prognostication.

PRIORITISE is a prospective observational cohort study. Consecutive patients aged ≥ 18 years with clinically-suspected SARS-CoV-2 infection presenting with moderate symptoms to the All India Institute of Medical Sciences (AIIMS) Hospital in Patna, India and the Christian Medical College (CMC) Hospital in Vellore, India were screened (daytime hours, Monday to Saturday). AIIMS is a 1,000-bed hospital and the largest medical facility providing primary-to-tertiary healthcare in the state of Bihar. CMC is a 3,000-bed not-for-profit hospital that provided care for ~1,500 patients with COVID-19 each day during the peak of the delta-wave surge in India.

We adapted the case definitions in the WHO Clinical Management guideline (moderate disease) 6 and WHO Clinical Progression Scale (WHO-CPS; scores 2, 3 or 4) 19 to define moderate disease as follows: a peripheral oxygen saturation (SpO 2 ) ≥ 94% and respiratory rate < 30 breaths per minute (bpm), in the context of systemic symptoms (breathlessness or fever and chest pain, abdominal pain, diarrhoea or severe myalgia), recognising that the threshold for hospitalisation varies throughout a pandemic and that a sensitive cut-off for hypoxia would be desirable in a tool to inform communitybased management. 19, 20 A c c e p t e d M a n u s c r i p t

Structured case-report forms (appendix p2-10) were completed at enrolment, day 7, and day 14, and daily during admission to the study facilities. Anthropometrics and vital signs were measured at enrolment and demographics, clinical symptoms, comorbidities, and medication history collected via brief interview with the participant. Venous blood samples were collected at enrolment in ethylenediaminetetraacetic acid (EDTA) tubes. Participants were followed-up in-person whilst admitted to the facility, and by telephone on days 7 and 14 if discharged prior to this. Those discharged who reported worsening symptoms on day 7 and/or persistent symptoms on day 14

were recalled to have their SpO 2 and respiratory rate measured.

The primary outcome was development of an oxygen requirement within 14 days of enrolment, defined as any of the following: SpO 2 < 94%; respiratory rate > 30 bpm; SpO 2 /FiO 2 < 400; 21, 22 or death, aligning closely with a WHO-CPS score of ≥ 5. 19 Patients who received supplemental oxygen outside the study facilities were classified as meeting the primary outcome if it was not possible to retrieve their case notes, provided the oxygen was prescribed in a licensed medical facility. The site study teams were unaware of which baseline variables had been preselected as candidate predictors when determining outcome status.

We decided a-priori that a model using four predictors would be practical in high-patient-throughput resource-limited settings. Considering resource constraints, reliability, validity, feasibility, and A c c e p t e d M a n u s c r i p t Page 9 biological plausibility, we prespecified that each model would contain age, sex, SpO 2 and one biochemical biomarker. 10, 17, 23 Following a literature review ( Figures S1-2) , biomarkers were shortlisted in consultation with FIND, the global alliance for diagnostics (Geneva, Switzerland). To qualify for inclusion, biomarkers had to be quantifiable with rapid tests in clinical use or late-stage development (Technology Readiness Level ≥ 4; Table S1 ). 24 The final list included: C-reactive protein (CRP), D-dimer, interleukin-6 (IL-6), neutrophil-to-lymphocyte ratio (NLR), procalcitonin (PCT), soluble triggering receptor expressed on myeloid cells-1 (sTREM-1), and soluble urokinase plasminogen activator receptor (suPAR). [25] [26] [27] [28] [29] Clinical predictors were measured at enrolment and all biomarkers except NLR were measured retrospectively from samples obtained at enrolment. NLR was measured on site and was not repeated if it had been measured at the site within 24 hours prior to recruitment. All predictors were measured blinded to outcome status.

Complete blood counts (XP-300-Hematology-Analyzer, Sysmex, IL) were performed on site and aliquots of EDTA-plasma stored at -20°C or below until testing. Biomarker concentrations were quantified using the suPARnostic ELISA (ViroGates, Denmark) and Simple Plex Ella microfluidic platform (ProteinSimple, CA) as described elsewhere. 30 Remaining plasma was biobanked on site. 

We considered the sample size for model development and validation separately. We followed the recommendations of Riley et al. and assumed a conservative R 2 Nagelkerke of 0.15. 31 We anticipated that ~8% of participants would meet the primary endpoint and estimated that 44 outcome events would be required to derive a prediction model comprising four candidate predictors and minimise the risk of overfitting (events per parameter [EPP] = 11).

Given the uncertainty around deterioration rates amongst patients with moderate COVID-19 at the time of study inception, we prespecified an interim review after the first 100 participants were recruited. At this review, the proportion of participants meeting the primary endpoint was higher than anticipated (20% vs. 8%). At this higher prevalence, and using R 2 values from 0.20-0.15, between 52-68 outcome events (EPP = 13-17) would be required to develop the prediction models. 31 Recognising that (i) our range of R 2 estimates was conservative, (ii) penalised regression methods would reduce the risk of overfitting, and (iii) the external validation cohort would allow assessment of model optimism, and following the advice of the External Advisory Panel, a decision was made to use the first 50 outcome events to derive the models. Participants recruited after that point were entered into the external temporal validation cohort.

We explored the relationship between candidate predictors and the primary outcome using a Lowess smoothing approach to identify non-linear patterns. Transformations were used when serious violations of linearity were detected. We used penalised logistic (ridge) regression to develop A c c e p t e d M a n u s c r i p t Page 11 the models and shrink regression coefficients to minimise model optimism. All predictors were prespecified and no predictor selection was performed during model development. Due to few missing data (< 3% for any single predictor), missing observations were replaced with the median value, grouped by outcome status. A sensitivity analysis was conducted using full-case analysis.

We assessed discrimination (c-statistics) and calibration (calibration plots and slopes) for each model recognising that the relative value of a TP and FP will vary at different stages of the pandemic, 20 we examined the potential clinical utility of the models using decision curve analyses to quantify the net benefit between correctly identified TP or TN and incorrectly identified FP or FN at a range of plausible trade-offs (threshold probabilities). 32 All analyses were done in R v4.03.

This investigator-initiated study was prospectively registered (ClinicalTrials.gov; NCT04441372), with protocol and statistical analysis plan uploaded to the Open Science Framework platform (DOI: Table   S2 ). The first 257 participants comprised the development cohort and the remaining 166 participants comprised the temporal validation cohort.

Development and validation cohorts were largely balanced with respect to baseline characteristics (Table 1; Table S3 ). There was a higher proportion of males in the development cohort (72% participants died, 2 were mechanically ventilated, 15 received non-invasive ventilation, 49 received oxygen via a face mask and/or nasal cannula (one outside the study facilities), and 12 had an SpO 2 < 94% but did not receive oxygen supplementation (Table S4 ; Figure S3 ).

Relationships between candidate predictors and the primary outcome are illustrated ( Figure S4 ), and c-statistics (continuous predictors) and odds ratios (continuous and categorical predictors) reported (Table S5 ). The full models are presented in the appendix (Table S5 ; Figure S5 ). After adjustment for Calibration was better at lower predicted probabilities, with some models overestimating risk at higher predicted probabilities.

The ability of each model to rule-out progression to oxygen requirement amongst patients with moderate COVID-19 at predicted probabilities (cut-offs) of 10%, 15% and 20% is shown ( Table 2;   Table S6 ; Figure S6) . A cut-off of 10% reflects a management strategy equivalent to admitting any patient in whom the predicted risk of developing an oxygen requirement is ≥ 10%. At this cut-off, the results suggest that a model containing the three clinical parameters (age, sex, and SpO 2 ) without any biomarkers could facilitate correctly sending home ~25% of patients with moderate COVID-19 who would not subsequently require supplemental oxygen, at the cost of also sending home ~9% of moderate patients who would deteriorate and require supplemental oxygen, i.e. a ratio of correctly to incorrectly discharged patients of 10:1.

The inclusion of either NLR or suPAR improved the predictive performance such that the ratio of correctly to incorrectly discharged patients increased to 23:1 or 25:1 respectively, whilst a model containing IL-6 resulted in a similar proportion (~21%) of correctly discharged patients as the clinical A c c e p t e d M a n u s c r i p t Page 14 model but without missing any patients who would deteriorate and require supplemental oxygen.

Inclusion of the other candidate biomarkers (CRP, D-dimer, PCT or sTREM-1) did not improve the ability of the clinical model to rule-out progression to supplemental oxygen requirement.

We recognised that the relative value of a TP and FP, i.e. admitted patients who would and would not subsequently require supplemental oxygen, was not fixed and would vary at different stages of the pandemic, reflecting bed pressures and/or capacity for follow-up. 20 Decision curve analyses accounting for this differential weighting suggest that the clinical model could provide utility (net benefit over an "admit-all" approach) at a threshold probability above 15% (i.e. when the value of one TP is equal to ~7 FPs). Furthermore, the results indicate that models containing any one of IL-6, NLR or suPAR could offer greater net benefit than the clinical model and extend the range of contexts in which a model might provide utility to include threshold probabilities above 5% (value of one TP is equal to 19 FPs; i.e. when bed pressures are less critical). For the model containing IL-6, this higher net benefit appeared to be maintained across a range of plausible threshold probabilities ( Figure 4 ).

We report the development and temporal validation of three promising clinical prediction models to assist with the assessment of patients with moderate COVID-19. The models combine three simple parameters (age, sex, and SpO 2 ) with measurement of a single biochemical biomarker (IL-6, NLR or suPAR), quantifiable using commercially-available rapid tests.

A c c e p t e d M a n u s c r i p t Page 15 We included patients in whom there is clinical uncertainty as to whether admission is warranted, and adopted an analytical approach which acknowledged that the trade-offs inherent in this decision will vary at different stages of the pandemic and in different healthcare settings. We used specific systemic symptoms to define moderate severity disease rather than the WHO-CPS, recognising, as did the scale's original authors, that the lower-end of the WHO-CPS is subjective. 19 Performance of any prediction model is sensitive to the prevalence of the outcome it aims to predict and thus we hope our more objective study entry criteria will better standardise the outcome prevalence and facilitate model transportability; we followed the widely-used ISARIC case report form to define symptoms to permit validation by other groups. 33 Our approach focussed on quantifying the added value of host biomarkers. We recognise that laboratory tests carry an opportunity cost, especially when resources are limited. Although a model containing clinical parameters alone would be simpler to implement, our analyses indicate that inclusion of one biomarker test would allow use of the model in a broader range of contexts, including when bed pressures are less acute early in a COVID-19 surge.

Our models have face validity. All clinical and laboratory predictors have been implicated in the pathogenesis of COVID-19. 10, 17, 23, 25, 27, 29 Similar to others, we found that age and sex were not strongly associated with risk of deterioration, in contrast to their well-recognised association with COVID-19 mortality. 23 This underlines the importance of developing models for specific clinical usecases. Models developed to predict mortality are not necessarily appropriate to rule-out less severe disease, just as models developed in well-resourced healthcare systems may not generalise to resource-limited settings. 34 A c c e p t e d M a n u s c r i p t

The three biochemical biomarkers that demonstrate most promise in our study have biological plausibility. In addition to being a therapeutic target, 35 raised IL-6 levels predict development of an oxygen requirement, 27, 28 and along with an elevated NLR, form part of the COVID-19-associated hyperinflammatory syndrome (cHIS) diagnostic criteria. 36 Elevated suPAR levels are associated with disease severity and progression in both moderate and severe COVID-19, 29, 37 and have been used for stratification into trials of immunomodulatory agents. 38 We addressed the limitations identified in other COVID-19 prognostic models by following the TRIPOD guidelines, 18 and using a prospectively collected dataset with minimal loss-to-follow-up and missing data. 8 Nevertheless, the small validation cohort (determined by the natural history of the pandemic in India) limits our ability to draw strong conclusions. Although the same models appeared superior in the different analyses we performed, further external validation is required before they can be recommended for use; we have published our full models (Table S5 ; Figure S5 In our context, corticosteroids were readily available and often self-prescribed or used off-license.

Although steroid use was associated with some candidate predictors, it was not associated with the primary outcome and is therefore unlikely to have confounded the observed association (Tables S7-8) .

We selected oxygen requirement as our primary outcome as this reflects a clinically meaningful endpoint. We opted to use an SpO 2 /FiO 2 < 400 for participants without documented hypoxia or tachypnoea prior to initiation of supplemental oxygen, as the threshold for oxygen therapy can be subjective and vary depending on available resources. 19, 22 It is unlikely that our outcome lacked sensitivity; only one participant who received supplemental oxygen did not meet the primary outcome. It may have lacked specificity (12 participants who met the primary outcome did not receive supplemental oxygen and calculation of FiO 2 in non-ventilated patients can overestimate pulmonary dysfunction), 42 but sensitivity would always be prioritised in a tool to inform communitybased management. Furthermore, any outcome misclassification is likely to have reduced, rather than exaggerated, the prognostic performance of the candidate predictors and models. 43 Baseline Ct value was not associated with the risk of deterioration (Table S9 ). In keeping with others, we found that seronegativity at enrolment was associated with an increased risk of deterioration In conclusion, we present three clinical prediction models that could help clinicians to identify patients with moderate COVID-19 whom are suitable for community-based management. The M a n u s c r i p t Page 20

De-identified, individual participant data from this study will be available to researchers whose proposed purpose of use is approved by the data access committees at Médecins Sans Frontières and the Mahidol-Oxford Tropical Medicine Research Unit. Inquiries or requests for the data may be sent to data.sharing@london.msf.org and datasharing@tropmedres.ac. Researchers interested in accessing biobanked samples should contact the corresponding authors who will coordinate with the respective institutions. 

JO and SD report being employed by FIND the global alliance for diagnostic (https://www.finddx.org/) an organization dedicated to advancing the use of diagnostic tools. The other authors declare that they have no conflicts of interest. M a n u s c r i p t be rolled out in the study areas and a decision was made to exclude vaccinated participants as the study would not be powered to determine whether the prediction models were valid in this cohort. 

The potential impact of COVID-19 in refugee camps in Bangladesh and beyond: A modeling study

Fragility and challenges of health systems in pandemic: early lessons from India's second wave of coronavirus disease 2019 (COVID-19). Glob Health

Collapse of the public health system and the emergence of new variants during the second wave of the COVID-19 pandemic in Brazil

Pragmatic Recommendations for the Management of Acute Respiratory Failure and Mechanical Ventilation in Patients with COVID-19 in Low-and Middle-Income Countries

World Health Organization. COVID-19 Clinical Management -Living Guidance 25

Pragmatic Recommendations for Identification and Triage of Patients with COVID-19 Disease in Low-and Middle-Income Countries

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

Pragmatic Recommendations for the Use of Diagnostic Testing and Prognostic Models in Hospitalized Patients with Severe COVID-19 in Low-and Middle-Income Countries

Filling the gaps in the characterization of the clinical management of COVID-19: 30-day hospital admission and fatality rates in a cohort of 118 150 cases diagnosed in outpatient settings in Spain

Prediction for Progression Risk in Patients With COVID-19 Pneumonia: The CALL Score

A Retrospective Study of COVID-19-Related Urgent Medical Visits and Hospitalizations After Outpatient COVID-19 Diagnosis in the US

Risk factors for disease progression in COVID-19 patients

Development and validation of an early warning score (EWAS) for predicting clinical deterioration in patients with coronavirus disease

Early clinical factors predicting the development of critical disease in Japanese patients with COVID-19: A single-center, retrospective, observational study

Lactate dehydrogenase and susceptibility to deterioration of mild COVID-19 patients: a multicenter nested case-control study

Lower mortality of COVID-19 by early recognition and intervention: experience from Jiangsu Province

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement

A minimal common outcome measure set for COVID-19 clinical research

Variation in COVID-19 disease severity at hospital admission over time and across hospitals: A multi-institution cohort of Michigan hospitals

The SOFA score-development, utility and challenges of accurate assessment in clinical trials

A modified sequential organ failure assessment score for critical care triage

Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study. The Lancet Respiratory Medicine 2021. 24. European Union. Technology readiness levels (TRL); Extract from Part 19 -Commission Decision C(2014)4995

Risk factors for disease progression in patients with mild to moderate coronavirus disease 2019-a multi-centre observational study

Prognostic Value of C-Reactive Protein in Patients With Coronavirus

An inflammatory cytokine signature predicts COVID-19 severity and survival

COVID-19 risk stratification algorithms based on sTREM-1 and IL-6 in emergency department

Soluble urokinase plasminogen activator receptor (suPAR) as an early predictor of severe respiratory failure in patients with COVID-19 pneumonia

Validation of two multiplex platforms to quantify circulating markers of inflammation and endothelial injury in severe infection

Calculating the sample size required for developing a clinical prediction model

Decision curve analysis: a novel method for evaluating prediction models

Early warning scores generated in developed healthcare settings are not sufficient at predicting early mortality in Blantyre, Malawi: a prospective cohort study

We are grateful to the members of the External Advisory Panel, led by Prof. Nicholas J White (MORU), for their advice and guidance throughout the study. We also acknowledge the support of 

A c c e p t e d M a n u s c r i p t Page 21 M a n u s c r i p t Page 25 of 1 TP (patient admitted who will subsequently require oxygen) is equivalent to 19 FPs (patients admitted who will not subsequently require oxygen). 

Heart rate (bpm) 

Symptom duration (days) 6 .0 A c c e p t e d M a n u s c r i p t 

A c c e p t e d M a n u s c r i p twere not selected as one of the a-priori clinical predictors. § Different specimen collection procedures and PCR assays were used at each site (Table S9)