key: cord-0761391-wswk7c2h authors: Modrak, M.; Burkner, P.-C.; Sieger, T.; Slisz, T.; Vasakova, M.; Meseznikov, G.; Casas-Mendez, L. F.; Vajter, J.; Taborsky, J.; Kubricht, V.; Suk, D.; Horejsek, J.; Jedlicka, M.; Mifkova, A.; Jaros, A.; Kubiska, M.; Vachalova, J.; Sin, R.; Veverkova, M.; Pospisil, Z.; Vohryzkova, J.; Pokrievkova, R.; Hrusak, K.; Christozova, K.; Leos-Barajas, V.; Fiser, K.; Hyanek, T. title: Detailed disease progression of 213 patients hospitalized with Covid-19 in the Czech Republic: An exploratory analysis date: 2020-12-04 journal: nan DOI: 10.1101/2020.12.03.20239863 sha: 1a55bdff8722686faf08639852313c7c3f0cc20a doc_id: 761391 cord_uid: wswk7c2h We collected a multi-centric retrospective dataset of patients (N = 213) who were admitted to ten hospitals in Czech Republic and tested positive for SARS-CoV-2. The dataset contains baseline patient characteristics, breathing support required, pharmacological treatment received and multiple markers on daily resolution. Patients in the dataset were treated with hydroxychloroquine (N = 108), azithromycin (N = 72), favipiravir (N = 9), convalescent plasma (N = 7), dexamethasone (N = 4) and remdesivir (N = 3), often in combination. Most patients were admitted during the first wave of the epidemic. To explore association between treatments and patient outcomes we performed multiverse analysis, observing how the conclusions change between defensible choices of statistical model, predictors included in the model and other analytical degrees of freedom. Weak evidence to constrain the potential efficacy of azithromycin and favipiravir can be extracted from the data. Additionally, we performed external validation of several proposed prognostic models for Covid-19 severity showing that they mostly perform unsatisfactorily on our dataset. The Covid-19 pandemic caused by severe acute respiratory syndrome coronavirus (SARS-CoV-2) has, as of November 2020, led to over 46 million cases and over 1.1 million deaths. Proposed treatments include antivirals approved for other indications (chloroquine, hydroxychloroquine, lopinavir/ritonavir, remdesivir, favipiravir, umifenovir), azithromycin, corticosteroids, immunoglobulins, tocilizumab and convalescent plasma. 1, 2 As of this writing, the only treatment that exhibited positive effects in randomized trials is the corticosteroid dexamethasone for severe cases. 3, 4 Remdesivir, previously designed and approved for Ebola, SARS and MERS, is currently the only antiviral against SARS-CoV-2 approved by the FDA. However, during spring and summer it was available only in limited amounts via compassionate use programme. Randomized clinical trials on remdesivir have provided conflicting evidence. 5, 6 Despite initial promise, hydroxychloroquine has failed to show benefit for hospitalized patients, 6, 7 as an early treatment, 8 as well as pre-and post-exposure prophylaxis. [9] [10] [11] Further, no improvement could be discerned when adding azithromycin to hydroxychloroquine therapy. 12 Similarly, lopinavir/ritonavir did not show benefit. 13 No benefit was observed for convalescent plasma. 14, 15 Interestingly, in observational studies, hydroxychloroquine was often found to be associated with better outcomes. [16] [17] [18] Favipiravir is one of the less explored experimental treatments, previously used for influenza including avian types, with only one larger RCT in Covid-19 known to us, 19 the study however has many potentially severe problems. 20 Other studies are scarce -even when considering very small studies and case series. [21] [22] [23] None of the results allow strong conclusions about benefits or harms of favipiravir, although a metaanalysis of all available small studies showed an effect of favipiravir on clinical improvement in mild and moderate disease. 24 For additional summary of treatments under investigation see Siemieniuk et al. 25 High IL-6, D-dimer values were observed to be associated with worse outcome and increased disease severity. 26 Large study of electronic health records 27 showed an increase in C-reactive protein in early disease and increase of D-dimer and white blood cell count in later stages of the disease. An ongoing challenge in evaluating Covid-19 treatments is that the analysis and interpretation of the data is often inappropriate or misleading, most notably interpreting lack of evidence due to small sample size as evidence of no effect. 28, 29 Additionally, many methods for predicting disease severity of Covid-19 were published, but the methods are at high risk of bias and lack external validation. 30 The present study aims to describe the outcomes and disease course of hospitalized patients with mild to severe clinical presentation in a multicentric Czech cohort, explore the association between the outcomes and pharmacological interventions and to provide external validation to previously published prognostic models for Covid-19 severity. A convenience sample of patients from 10 sites was collected. For each site, the dataset contains all patients hospitalized in the participating wards over the data collection period. The data collection started at the onset of the Covid-19 pandemic (except for one site where some older records were inaccessible), but the end date for collection differs between sites due to time constraints of the participating physicians. Three sites included total of 23 patients that could be considered part of "second wave", i.e. admitted after September 1st, last patient included in the dataset was admitted on October 12th. See Supplementary Figure 1 for per-site data collection periods. Patients over the age of 18 were included if they had PCRconfirmed infection of SARS-CoV-2 and were not participating in a clinical trial of any Covid-19 pharmacotherapy. We collected data on comorbidities and information about disease progression on daily resolution including breathing support required, oxygen flow rate, experimental anti-Covid-19 and antimicrobial drugs taken and several laboratory markers (PCR positivity for SARS-CoV-2, C-reactive protein, D-dimer, Interleukin 6, Ferritin, lymphocyte count). Full protocol for data collection is attached in the supplementary material. Due to very low number of patients using extra-corporeal membrane oxygenation (N = 1) or non-invasive positive pressure ventilation (N = 6) in our sample, we merged those categories with mechanical ventilation. Not all patients developed pneumonia or other symptoms of Covid-19. All patients received the standard of care which could include supplemental oxygen and ventilation and antibiotics for bacterial superinfections, as determined by the attending physician. Some patients were not indicated for all treatment modalities (especially mechanical ventilation) based on decision of the attending physician and . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted December 4, 2020. ; underlying patient condition. We note that the participating sites were not homogeneous in either patient population or treatment protocols. The choice of pharmacological treatment was based on the decision of the attending clinician and its availability. The study was approved by the Ethical committees of General University Hospital, Hospital Nový Jíčín, Motol University Hospital, Thomayer Hospital, University Hospital Vinohrady, Military Hospital Olomouc, Na Homolce Hospital, University Hospital in Pilsen, Hořovice Hospital, Jihlava Hospital. The character of the convenience sample does not allow for a proper assessment of the association between treatments and patient outcomes, because the treatments had not been assigned to patients at random, but were only observed retrospectively. This can be partially remedied by adjusting for patient characteristics in the analysis, but such adjustments will always be imperfect and the analysis needs to be treated as exploratory and interpreted cautiously. Since many details of analysis may influence the conclusions made, we performed multiverse analysis 31 and report results for all the hypothesis tested across multiple different models using both frequentist and Bayesian paradigms. For each model class we worked with several possible sets of adjustments. All analyses were performed in the R language, 32 visualizations and data cleaning was run with the tidyverse package. 33 First class of models are frequentist survival and multistate models under the proportional hazards assumption as implemented in the coxph function from the survival package. 34 We primarily use a model with competing risks for death and discharge from hospital (see Figure 1a ). Second class of models are Bayesian hidden Markov models (HMM) of disease progression implemented via a custom extension to the brms package. 35 The parametrization of the HMM is inspired by Williams et al. 36 : the actual process of disease is assumed to be continuous and allow only for transitions between neighboring states (as shown in Figure 1b , c). The total probability of transition between any two states over the period of a day is then computed as the total probability of transition across all possible paths. This class of models does not satisfy the proportional hazards assumption, instead, it is assumed the process has the Markov property -i.e. that the (potentially unobserved) state and the covariates at a given day contain all the information to determine probabilities of the states on the next day. We use two versions of such models, one working solely with the observed breathing support and one assuming a hidden improving/worsening distinction. All of the hidden Markov models take into account whether best supportive care was initiated and a patient was thus not indicated to progress to more intensive treatment modalities. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; Finally we used a set of Bayesian regression models implemented with the brms package. 35 Those included overall survival, state at day 7 or 28 as either binary or categorical outcome and a Bayesian version of the Cox proportional-hazards model. Except for age, sex and comorbidities, all covariates are treated as time-varying, e.g., the effect of taking a drug is only included for the days after the drug was taken. More details on the exact model formulations can be found in the supplementary statistical analysis. We searched the living systematic review of Covid-19 prognostic models 30 for those that could be applied to our dataset (i.e. where we have gathered all the input features). We primarily focused on the Area Under Receiver Operating Characteristic Curve (AUC), and its bootstrapped 95% confidence intervals which we computed using the pROC package. 37 When there were multiple reasonable ways to evaluate the outcome or a predictor in our dataset, we computed and reported all of those options. We used two simple scores with age or the decade of age as the sole predictor to have a baseline to compare the scores against. Complete code for all analyses is available at https://github.com/casbioinf/covid19retrospective/. In total, we were able to gather data for 213 patients, see Table 1 for the overall characteristics of the patient sample and several subgroups we used in the analysis, . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. In Figure 2 we show the overall disease progression for all patients and in Figure 3 we show the time-course of a subset of the markers we have measured. The data show some interesting patterns: patients with low Interleukin-6 or D-dimer values are overrepresented among patients with better outcomes, most patients had high CRP upon admission and for many the CRP levels stayed elevated over the whole hospitalization. However, the limited nature of the data does not allow for any statistically robust conclusions. We also see that the marker levels were not . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; https://doi.org/10.1101/2020.12.03.20239863 doi: medRxiv preprint substantially stratified by study site. Those patterns should however be interpreted with care due to systematic missingness of the data -in particular, patients that fared worse were probably more likely to have the markers measured. However we believe this kind of patient-level view is useful to appreciate the extent of both between-patient and within-patient variability. As noted above, the nature of the convenience sample did not enforce random assignment of treatments to patients. In fact, patients with worse baseline characteristics, which lead to worse outcomes, were less likely to receive hydroxychloroquine (see Supplementary Figure 4 ). This clearly creates a bias towards a positive effect of hydroxychloroquine on the outcome (and potentially for other treatments as well -most were used in combination with hydroxychloroquine), which, however, could be false. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; Taken quantitatively, the comorbidities known upon hospitalization were informative with respect to the future hydroxychloroquine treatment: the score representing the cumulative presence of ischemic heart disease, hypertension drugs, former heart failure, COPD, other lung diseases, renal disease, or high creatinine was associated with a lower chance of taking hydroxychloroquine over the course of the hospitalization (the chance was only 79.9%, 95% confidence interval (65.3, 97)%, Chi-square test in the logistic regression model, 2 =5.18, df=1, P=0.023). Here, we focus on hydroxychloroquine and azithromycin as those are the only treatments with larger sample size. We also investigate favipiravir as it is less well reported in the literature. Hydroxychloroquine was dosed almost exclusively in a 5day regime starting with a loading dose of 800mg on the first day and followed by 400mg. Majority of patients complemented hydroxychloroquine with azithromycin while azithromycin was rarely used alone (see Table 1 ). Azithromycin was most frequently dosed 250 or 500mg/day, but doses ranging from 100mg/day to 1500 mg/day were observed. Favipiravir was used only at one site with a loading dose of 3600mg on the first day, followed by at most 9 days with a 1600mg dose. All but one of the patients receiving favipiravir also received hydroxychloroquine. Treatment was initiated mostly within two days of admission (see Supplementary Figure 5 ). . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; The results of the multiverse analysis for association between hydroxochloroquine, azithromycin and favipiravir usage and death is shown in Figure 4 -here, we only show models that were not found to have immediate problems representing the data well or computational issues (see Supplementary statistical analysis for details). Results for all models we tested are reported in supplementary Figures 6-8 , with additional details in supplementary statistical analysis. The results do not change noticeably when only patients from the first wave are included (supplementary Figures 6-8) . Most models report that using hydroxychloroquine is associated with lower risk of death. We must however bear in mind the potential bias noted in the previous section. Also, we see that for the HMM models, as we add adjustments the credible intervals do not widen but instead shift towards zero. This is a weak indication that further adjustments could drive the effect towards zero. We did not attempt to model additional adjustments as the models became computationally unstable. The case of hydroxychloroquine serves as a "control group" for our other results -since randomized trials give us high confidence that hydroxychloroquine does not substantially reduce mortality, we can be quite certain the associations we observe for hydroxychloroquine are just a measure of bias in the data. Additionally, our models either cannot determine the sign of association between azithromycin and risk of death or even show an increase in risk of death. This serves as a weak evidence that a substantial improvement in mortality from azithromycin is unlikely. Most models exclude very strong association between increased risk of death and using favipiravir, but our data are necessarily quite limited, which is reflected in the very wide uncertainties around estimates. We also cannot put strict bounds on the association between favipiravir and length of hospitalization. We also examined the association between treatments and length of hospital stay for all the patients that survived. Almost all models cannot discern the sign of the association for all treatments examined ( Supplementary Figures 6-8) . Similarly, we studied the association between D-dimer and Interleukin 6 and outcomes, with unconclusive results as well (Supplementary Figure 9 ) . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; https://doi.org/10.1101/2020.12.03.20239863 doi: medRxiv preprint Published prognostic models are not better than using age as the sole predictor of outcome Following Wynants et al. 30 we found five prediction models we were able to recompute: Li et al. report the ACP index 39 combining CRP and age to form 3 grades, Chen & Liu 40 report a continuous score using age, CRP, D-dimer and lymphocyte count, Shi et al. 41 use age, sex and hypertension to form 4 grades, Caramelo et al. use age, sex, hypertension, diabetes, cardiac disease, chronic respiratory disease and cancer to form a continuous score, 42 Bello-Chavolla et al. 43 use age, diabetes, obesity, pneumonia, chronic kidney disease, COPD and immunosuppression to build a score ranging from -6 to 22. For the latter two scores we had to impute some of the predictors as they had no immediate equivalent in our dataset. The outcomes present in the studies were: 12-day mortality, 30-day mortality and mortality without any further details, here we report results for both 12-day and 30-day mortality. Full details on the scores and how we used our dataset to compute them is given in the supplementary statistical analysis. All prognostic models we tested performed similarly to or notably worse than using age as the only predictor and also worse than originally reported ( Figure 5 ). Additionally, some publications did not provide enough detail to unambiguously reconstruct how the score and/or outcome was assessed. We thus concur with Wynants et al. 30 that reported prediction scores are at high risk of bias and need additional careful evaluation. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted December 4, 2020. ; Figure 5 : Performance of tested prediction scores as measured by AUC. AUC = 1 means perfect prediction while AUC ≤ 0.5 means that the score is worse than random guess and a better prediction would be obtained by reversing the score (marked by thick blue line). The line ranges represent the bootstrapped 95% confidence intervals. Red dots show results computed in present study -model variants (horizontal axis) vary in the outcome measured (12-day or 30-day mortality, severe disease) and potentially on how ambiguities in score computation were resolved, although this rarely makes a big difference -see supplementary statistical analysis for details. Cyan triangles show AUC as reported by the original authors or recomputed based on their published data. When the confidence interval or the AUC of the original study is not shown, it means that the value was not reported by the authors and not enough information to recompute it was given. Our data show the extent of between-patient variability in progression of the disease in terms of both length of hospital stay, duration of various types of breathing support and basic markers. We provide very weak observational evidence against a substantial beneficiary effect of using azithromycin (both with or without hydroxychloroquine) and against substantial negative effect of using favipiravir in hospitalized Covid-19 patients. We also observed better outcomes associated with taking hydroxychloroquine, which is likely linked to substantial confounding by indication. Where our results contradict randomized trials, the most likely explanation is systematic bias in our dataset. A lesson from our analysis is that the assessment of treatment efficacy from observational data is very sensitive to modelling assumptions while it is usually almost impossible to determine which of the models is more likely to reflect reality . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted December 4, 2020. ; (if any). We believe that using multiverse analysis is an appropriate way to explore data in such contexts as it lets us be transparent about this sensitivity. We further believe that using hidden Markov models is a promising complement to the standard Cox proportional hazards analysis when detailed information on disease progression is available, particularly because it lets us impose additional structure on the model and thus make inferences with more disease states than would be possible to handle in the Cox framework, making better use of the available data. Additionally, our experience indicates that a substantial fraction of published prognostic models will perform much worse on new patients than on the datasets they were built for and that external validation is crucial. We suggest that comparing the prognostic models against simple baselines (e.g. decade of age as the single predictor) should be a first step in validation. Furthermore, some of the published scores lack enough information to let others implement the score in the same way. The authors declare that there is no conflict of interest. The data that support the findings of this study are available on request from the corresponding author, if legal obligations under GDPR can be met. The data are not publicly available due to privacy restrictions. We will be happy to arrange to run any analytical code locally and share the results, provided the code and the results do not leak personal information. Pharmacologic Treatments for Coronavirus Disease 2019 (COVID-19): A Review Dexamethasone in Hospitalized Patients with Covid-19 -Preliminary Report Association Between Administration of Systemic Corticosteroids and Mortality Among Critically Ill Patients With COVID-19: A Meta-analysis Remdesivir for the Treatment of Covid-19 -Final Report Repurposed Antiviral Drugs for COVID-19 -Interim WHO SOLIDARITY Trial Results. Infectious Diseases (except HIV/AIDS); 2020 Effect of Hydroxychloroquine in Hospitalized Patients with COVID-19: Preliminary results from a multi-centre, randomized, controlled trial. medRxiv Hydroxychloroquine in Nonhospitalized Adults With Early COVID-19 A Randomized Trial of Hydroxychloroquine as Postexposure Prophylaxis for Covid-19 A Cluster-Randomized Trial of Hydroxychloroquine as Prevention of Covid-19 Transmission and Disease. medRxiv Hydroxychloroquine as preexposure prophylaxis for COVID-19 in healthcare workers: a randomized trial. medRxiv Azithromycin in addition to standard of care versus standard of care alone in the treatment of patients admitted to the hospital with severe COVID-19 in Brazil (COALITION II): a randomised clinical trial. The Lancet. 2020;0(0) Convalescent plasma in the management of moderate COVID-19 in India: An open-label parallel-arm phase II multicentre randomized controlled trial (PLACID Trial). medRxiv Effect of Convalescent Plasma Therapy on Time to Clinical Improvement in Patients With Severe and Life-threatening COVID-19: A Randomized Clinical Trial Low-dose hydroxychloroquine therapy and mortality in hospitalised patients with COVID-19: a nationwide observational study of 8075 participants An inflammatory cytokine signature predicts COVID-19 severity and survival Use of hydroxychloroquine in hospitalised COVID-19 patients is associated with reduced mortality: Findings from the observational multicentre Italian CORIST study Favipiravir versus Arbidol for COVID-19: A Randomized Clinical Trial. medRxiv Statistical Review of Favipiravir Versus Arbidol for COVID-19: A Randomized Clinical Trial. Zenodo; 2020 Clinical Outcomes and Plasma Concentrations of Baloxavir Marboxil and Favipiravir in COVID-19 Patients: an Exploratory Randomized, Controlled Trial. medRxiv Tolerability of favipiravir therapy in critically ill patients with COVID-19: A report of four cases Case studies of SARS-CoV-2 treated with favipiravir among patients in critical or severe condition Favipiravir versus other antiviral or standard of care for COVID-19 treatment: a rapid systematic review and meta-analysis Drug treatments for covid-19: living systematic review and network meta-analysis The association of interleukin-6 value, interleukin inhibitors, and outcomes of patients with COVID-19 in New York City International electronic health recordderived COVID-19 clinical course profiles: the 4CE consortium. npj Digital Medicine Statistical review of Efficacy and safety of lopinavir/ritonavir or arbidol in adult patients with mild/moderate COVID-19: an exploratory randomized controlled trial Statistical Review of Remdesivir in Adults with Severe COVID-19: A Randomised, Double-Blind, Placebo-Controlled, Multicentre Trial. Zenodo; 2020 Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal Increasing Transparency Through a Multiverse Analysis R: A Language and Environment for Statistical Computing Welcome to the Tidyverse Modeling Survival Data: Extending the Cox Model Advanced Bayesian Multilevel Modeling with the R Package brms A Bayesian Approach to Multistate Hidden Markov Models: Application to Dementia Progression pROC: an open-source package for R and S+ to analyze and compare ROC curves Disharmonic Inflammatory Signatures in COVID-19: Augmented Neutrophils' but Impaired Monocytes' and Dendritic Cells ACP risk grade: a simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. medRxiv Early prediction of mortality risk among severe COVID-19 patients using machine learning. medRxiv Host susceptibility to severe COVID-19 and establishment of a host risk score: findings of 487 cases outside Wuhan Estimation of risk factors for COVID-19 mortality -preliminary results. medRxiv Predicting Mortality Due to SARS-CoV-2: A Mechanistic Score Relating Obesity and Diabetes to COVID-19 Outcomes in Mexico This work was supported by ELIXIR CZ research infrastructure project (MEYS Grant No: LM2018131) including access to computing and storage facilities.