key: cord-0789477-g24jvyq7 authors: Shah, Neel; Said, Ahmed S. title: Extracorporeal Support Prognostication—Time to Move the Goal Posts? date: 2021-07-15 journal: Membranes (Basel) DOI: 10.3390/membranes11070537 sha: 4fb68b53e8e0807752e99e224b1218ae9a6ba04e doc_id: 789477 cord_uid: g24jvyq7 Advances in extracorporeal membrane oxygenation (ECMO) technology are associated with expanded indications, increased utilization and improved outcome. There is growing interest in developing ECMO prognostication scores to aid in bedside decision making. To date, the majority of available scores have been limited to mostly registry-based data and with mortality as the main outcome of interest. There continues to be a gap in clinically applicable decision support tools to aid in the timing of ECMO cannulation to improve patients’ long-term outcomes. We present a brief review of the commonly available adult and pediatric ECMO prognostication tools, their limitations, and future directions. Extracorporeal Membrane Oxygenation (ECMO) has seen its use exponentially increase over the last two decades [1] . New compact devices, improvements in anti-coagulation strategies and monitoring, and improvements in cannulation techniques have continued to fuel the growth within the field, and more centers are now offering ECMO support [1] . As the technology has advanced, the field has seen the criteria for ECMO deployment continue to expand, with progressively more centers pushing the boundaries and offering ECMO for previously contraindicated disease processes and even in out-of-hospital settings. ECMO mortality has slowly improved with recent trials demonstrating mortality in the 30-50% range, compared to over 40-70% in similar patients not receiving ECMO [2] [3] [4] [5] . Despite these advances, the fundamental question of prognostication persists-who is a good candidate for ECMO support? This question has never been more pressing than during the recent COVID-19 pandemic. As resources are stretched thin, offering resource-intensive therapies approaches a zero-sum situation in which difficult decisions must be made [6] [7] [8] [9] [10] . To date, the common approaches have focused on identifying factors in either patient characteristics or complications, mostly prior to ECMO initiation, to develop prognostication scores. The developed scores have mostly relied on registry-based data, especially for neonatal and pediatric scores, and with mortality as the main outcome. We review the current prognostication tools available to clinicians, their limitations, and the future directions of the field. Over the last two decades, several groups have put together prediction scores with Area Under Receiver Operator Curve (AUC), ranging from 0.65 to 0.89 depending on the group and indication evaluated. Two current scores exist to predict ECMO outcomes in neonatal respiratory failure, both derived from the Extracorporeal Life Support Organization (ELSO) registry. Both the Pittsburgh Index of Pre-ECMO Risk (PIPER) score [11] and Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support (Neo-RESCUERS) [12] rely on a combination of demographic, laboratory, and treatment data prior to ECMO cannulation to predict survival. The PIPER score demonstrated a 21% increase in mortality for each PIPER quartile. The mortality data was also separated by presence or absence of CDH. They further presented a combined model including ECMO duration and presence of complication categories as independent variables and showed an improved AUC of 0.79. Neo-RESCUERS analyzed neonates over overlapping periods of the registry to create a development and validation dataset. They showed that patients in their lowest decile had an observed mortality of 7% compared to a predicted mortality of 4.4%, while the highest decile had an observed mortality of 65.6% compared to a predicted mortality of 67.5%. Both scores present reasonable AUCs but neither have been externally validated (Table 1) . ECMO cannulation technique has also changed over time with more neonatal veno-venous (VV) ECMO cannulations occurring than from the time this registry data was collected. There is one neonatal congenital diaphragmatic hernia (CDH) ECMO prediction tool that is designed to be scored either with only variables from prior to cannulation or with the addition of variables after cannulation [13] . The highest risk group within the pre-ECMO score had a mortality of 75% compared to the mortality of the highest risk group as defined by the on-ECMO score of 86%. As expected, the addition of on-ECMO variables results in an improvement in the accuracy of the model ( Table 2 ). These findings illustrated the importance of ECMO complications when predicting mortality. Two scores exist for pediatric respiratory failure (Table 3) : the Pediatric Risk Estimation Score for Children Using Extracorporeal Respiratory Support (PED-Rescuers) score [14] and the newer Pediatric Pulmonary Rescue with Extracorporeal Membrane Oxygenation Prediction (P-Prep) score [15] . Both scores were developed utilizing data from the ELSO registry with an overlap in the data collection period but with a slight difference in the patient cohort age group. The PED-Rescuers score was developed and then validated against a dataset also from the ELSO registry, while the P-Prep score underwent external validation against another large pediatric database, the Pediatric Health Information System (PHIS). Both scores rely on pre-ECMO data regarding demographics including diagnosis, as well as laboratory and treatment variables. Since they were published, the incidence of pediatric respiratory ECMO has continued to increase, and the ELSO registry has transitioned to ICD-10 coding from ICD-9, which may provide more granular diagnosis data. Additionally, the ELSO registry underwent extensive revision and standardization of all data fields in 2018 to provide a more standardized approach across centers and to allow for more reliable multicenter data analysis [16] . Several Adult Respiratory Failure scores exist (Table 4) , with the earliest being EC-MOnet developed on data from the 2009 H1N1 Pandemic [17] . This score was limited to a single country cohort entered into an international research collaborative, ECMOnet [18] . They analyzed 60 patients, finding an overall survival rate of 68%. Predictors included hospital length of stay prior to ECMO, bilirubin, creatinine, hematocrit, and mean arterial pressure. These variables were split into categorical values and translated into scores. The external test set revealed AUC of 0.69 with an accuracy of 62%, sensitivity of 51%, and specificity of 62%. Since that time, several other scores have been developed. Utilizing data from three French intensive care units (ICU), the PRedicting dEath for SEvere ARDS on VV-ECMO (PRESERVE) score was developed, including 140 patients with refractory acute respiratory distress syndrome (ARDS) and eight pre-ECMO parameters as predictors [19] . Infectious diseases were the leading cause of ARDS with 26% being infected with influenza H1N1. They found that the pre ECMO P:F ratio was not associated with survival, and due to trials around this time showing improved mortality with early prone positioning, the authors forced it into their model and found it was independently associated with lower mortality. In addition to aiming to identify factors associated with long-term mortality for ICU survivors at 6-months post-discharge with 80% of survivors participating, they looked at health-related quality of life (HRQOL) and reported emotional (34%), anxiety (25%), depression or post-traumatic stress (PTSD) symptoms (16%). They further compared HRQL outcomes in their population compared to controls, as well as other ECMO and ARDS trials. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) score was developed with ELSO registry data and then externally validated on single center data used to develop the PRESERVE score [20] . It was further validated in several other independent studies [21, 22] . Survival within the registry data was 57%; several diagnosis groups were associated with mortality, and acute non-pulmonary infections and central nervous system dysfunction were also significant predictors. Included variables were then scored, and the final score was split into five risk classes with risk class I demonstrating 92% survival vs risk class V with 18% survival. Prone positioning was not reported within the registry and thus was not a predictor. The RESP score also performed better in the external database compared to classical ICU severity scores such as simplified acute physiology score II (SAPS II) and sequential organ failure assessment (SOFA). Both RESP and PRESERVE scores have also been compared in separate independent databases, where they were found to have similar accuracy in predicting both in-hospital and 6-months survival [23, 24] . Of note-four variables overlap between the scores including age, immunocompromised status, mechanical ventilation time prior to ECMO, and peak inspiratory pressure (PIP) or plateau pressure. The Roch score focused on identifying factors associated with mortality in patients referred to a single center for acute respiratory distress syndrome (ARDS) and then requiring ECMO support [25] . They looked at 85 consecutive ARDS patients equipped with ECMO by a mobile team, with the median time to cannulation 3 h after contact from the referring center. The variables were assigned scores with the 0-2 score group having 40% mortality and the 3-4 score having 93% mortality. The strength of the Roch score is its development in a referral center setting. While several other scores have been developed on either multi or single center data and then been externally validated, in general they have achieved good accuracy in predicting mortality with only limited variables. These include the Veno-venous (VV) ECMO mortality score [26] and PREdiction of Survival on ECMO Therapy-Score (PRESET) score [27] . The PRESET score was derived with 108 ARDS patients on VV-ECMO who were retrospectively analyzed with the EMCOnet, RESP, PRESERVE and Roch score. Within their dataset, only the ECMOnet and RESP score accurately determined survival, and subsequently the PRESET score was created including extrapulmonary predictors such as mean arterial pressure, lactate, pH, platelets, and hospital days pre-ECMO. The scores were then split into three risk classes with risk class I having 26% mortality vs risk class III with 93% mortality. A recent single center external validation also looked at these scores (PRESERVE, RESP, VV-ECMO mortality, and SOFA scores). The authors found slightly lower AUCs than previously reported between 0.64-0.73 [21] . Three large adult prognostic scores for survival for cardiac veno-arterial (VA) ECMO exist ( Table 5 ). The SAVE score is based off the ELSO registry data [28] . It was designed to predict survival from refractory cardiogenic shock. Registry data showed a 42% survival, the score was then validated on 161 Australian patients. The score was then split into risk classes from I to IV, with survival in class I at 75% vs. class IV at 18%. They further compared these risk categories across the most prevalent diagnosis groups, with myocarditis and heart and lung transplants having the highest survival. The external validation AUC was quite high (0.90), but it was done on a single highly protocolized center with higher survival and difference in diagnosis distribution. This center also participated in the ELSO registry so there may be some overlap between these cohorts. The prEdictioN of Cardiogenic shock OUtcome foR AMI (Acute Myocardial Infarction) patients salvaGed by VA-ECMO (ENCOURAGE) score was based off multicenter data of 138 patients and also followed up on HRQOL data [29] . It was developed over 138 patients from two centers and was unique in that it was the first specific risk score for AMIcardiogenic shock. They demonstrated 47% survival and had seven pre-ECMO predictors. Their HRQOL data was compared among score classes with 6-month survival ranging from 7 to 80%. They also reported anxiety (34%), depression (20%), and PTSD (5%) among their survivors. Conversely, the Predict VA-ECMO score is based off variables for patients already on ECMO, relying on very few variables at different time intervals with good accuracy [30] . They were able to develop a dynamic score using only three points of care biomarkers, and it outperformed the SAVE score within their database. A significant strength was the evaluation of extracorporeal cardiopulmonary resuscitation (eCPR) patients as well as cardiogenic shock patients. The majority of prediction models aim to predict the same outcome-mortality. Aside from CDH on-ECMO score and Predict VA-ECMO, all scores take input variables from pre-ECMO cannulation to help predict mortality. As one would expect looking at the variables of interest, a patient's severity of illness and degree of end organ dysfunction prior to ECMO initiation often plays a large role in their predicted mortality. Unfortunately, there are limited trials to help establish when a patient would best benefit from cannulation, as both too early and too late cannulation could be associated with adverse outcomes. In other words, the current models do not provide clinicians with decision support tools for timely institution of ECMO support, before end organ deterioration develops that has been shown to correlate with worse in-hospital and long-term mortality. While several adult scores have been designed without registry data or are externally validated, many pediatric scores are not externally validated. External validation is often crucial to ensure a prediction score is applicable to varying patient cohorts with differences in demographics, therapeutics, and even diagnoses. Recent studies have shown even externally validated adult scores performed similarly and with lower accuracy than initially reported [23, 24] . Recent literature has been published on how some of these scores performed retrospectively during the COVID-19 pandemic [2] . Recent evidence from the COVID-19 pandemic has also shown that the timing of ECMO initiation influences patient outcomes [32] . A common factor is that a large portion of predictive variables only point to existing end organ dysfunction prior to cannulation but provide scant evidence on modifiable risk factors either in the pre or on ECMO phases. They also mostly predict only short-term mortality, but provide limited information on long-term neurologic or functional outcomes. As described in a recent joint paper regarding the development and reporting of prediction models from respiratory, sleep, and critical care journals-a significant limitation of prediction models is the inclusion of casual relationships between variables [33] . Variables such as pre-ECMO cardiac arrest or SOFA score are independently related to mortality-with or without ECMO cannulation. Furthermore, the joint paper discusses the limitations of categorizing continuous variables and the subsequent loss of information. For example, a model that treats hypotension, mean arterial pressure (MAP) <60 mmHg, as a binary variable, may treat MAPs of 39 and 59 mmHg the same, as well as the folly of relying only on AUC curves without considering the importance of precision and recall, especially when datasets are unbalanced [34] . As survival improves overall, especially for conditions such as CDH, it is increasingly likely that there will be significantly unbalanced classes (higher survival). Both these limitations impact the vast majority of current ECMO prediction scores. Further, while current prediction scores can inform clinical decisions regarding population level statistics of who will survive ECMO, they offer little in the way of input on how to improve these outcomes. Most rely on demographics and pre-cannulation markers of severity of illness or pre-ECMO cardiac arrest [35, 36] . Research has also focused on factors that relate to continued multiorgan failure such as need for renal replacement therapy [23, 37] , continued need for vasoactive support, or markers of hemolysis or disseminated intravascular coagulation [38, 39] . Little has been offered in terms of ECMO predictive scores that alert clinicians of actionable variables that may impact outcomes. Recent adult literature has shown in a large ELSO database that a significant relative decrease in PaCO 2 in respiratory ECMO patients was associated with poor neurologic outcomes [40] . Pediatric data has shown the importance of PaCO 2 and its interaction with blood pressure on cerebral autoregulation in a single center [41] . Studies in pediatric patients have also shown the impact of hyperoxia on neurologic outcomes [42, 43] . Other literature has focused on the impact of cannulation modality and mortality [44, 45] . Factors such as relative changes in PaCO 2 , MAP, cannulation Site, and degree of hyperoxia are often modifiable by experienced clinical staff. Predictive scores with a focus on a change in variables over time and their impact on mortality are sorely needed to aid not only in the decision to place a patient on ECMO, but to predict and impact outcomes following cannulation. These challenges also remain ever present during the current COVID-19 pandemic, with the RESP & PRESERVE score having AUCs of 0.60 and 0.55 in a multicenter COVID-19 database [2] . The complexity of the COVID-19 patients in that they may require longer durations of mechanical ventilation, may have more significant ARDS and overall increased treatment complexity is not well reflected in either of these scores. We propose new COVID-19-specific scores should be developed that can include these further nuances. We further propose these prediction scores should be under continuous development and revision, which will improve their ability to serve as true clinical decision support tools. Deployment of ECMO has become more widespread with nearly a 10-fold increase over the last two decades. Recent trials have also documented the improvement in mortality outcomes provided by ECMO support. As with other interventions that have become more widespread in critical care, the focus shifts from a binary outcome of mortality to more nuanced patient-centered outcomes. Neurologic injury is one of the most common complications of ECMO, with neurologic complications such as intracranial hemorrhage associated with upwards of 40% of deaths. Children are similarly affected with survival falling to 36-38% following acute neurologic injury. In children in particular, long term follow up has also demonstrated that survivors often have significant impairment-12% report severe motor impairment, 10-50% with cognitive delays, 16-46% with behavioral abnormalities, and 31-53% of survivors having quality of life scores at least one standard deviation below the mean. Likewise, a standardized 1-year follow up clinic after respiratory ECMO demonstrated neurodevelopmental problems in 30% of the 98 patients [46] . Prior papers have focused on the role of cannulation sites, PaCO 2 change, and hyperoxia amongst others, for their roles in neurologic injury. Unfortunately, neurologic outcome has not been the focus of prediction models, likely due to heterogenous conditions of patients pre-ECMO, the role of their pre-ECMO illness, and difficulty in collecting and analyzing long term data. Despite these challenges, the next generation of prediction models may benefit from focus on neurologic outcomes rather than survival and ICU discharge, as previously mentioned. We present a review of current ECMO prediction models in neonatal, pediatric, and adult ECMO patients developed over the last decade, as well as a challenge for the next generation of models to focus on accuracy as well as precision, clinically modifiable outcomes, and outcomes aside from mortality such as long-term neurologic and functional status. Outcome Prediction in Patients with Severe COVID-19 Requiring Extracorporeal Membrane Oxygenation-A Retrospective International Multicenter Study Six-Month Outcome of Immunocompromised Patients with Severe Acute Respiratory Distress Syndrome Rescued by Extracorporeal Membrane Oxygenation. An International Multicenter Retrospective Study Extracorporeal Membrane Oxygenation for Severe Acute Respiratory Distress Syndrome and Posterior Probability of Mortality Benefit in a Post Hoc Bayesian Analysis of a Randomized Clinical Trial Extracorporeal Membrane Oxygenation for Severe Acute Respiratory Distress Syndrome ECMO during the COVID-19 pandemic: When is it unjustified? ECMO use in COVID-19: Lessons from past respiratory virus outbreaks-a narrative review Planning and provision of ECMO services for severe ARDS during the COVID-19 pandemic and other outbreaks of emerging infectious diseases Extracorporeal Life Support Organization Coronavirus Disease 2019 Interim Guidelines: A Consensus Document from an International Group of Interdisciplinary Extracorporeal Membrane Oxygenation Providers Development of Risk Indices for Neonatal Respiratory Extracorporeal Membrane Oxygenation Development and Validation of the Neonatal Risk Estimate Score for Children Using Extracorporeal Respiratory Support Development and Validation of Extracorporeal Membrane Oxygenation Mortality-Risk Models for Congenital Diaphragmatic Hernia Development and validation of the pediatric risk estimate score for children using extracorporeal respiratory support (Ped-RESCUERS) Extracorporeal Life Support Organization Member, C. Development and Validation of a Score to Predict Mortality in Children Undergoing Extracorporeal Membrane Oxygenation for Respiratory Failure: Pediatric Pulmonary Rescue With Extracorporeal Membrane Oxygenation Prediction Score Predicting mortality risk in patients undergoing venovenous ECMO for ARDS due to influenza A (H1N1) pneumonia: The ECMOnet score ECMOnet Mission and Values The PRESERVE mortality risk score and analysis of long-term outcomes after extracorporeal membrane oxygenation for severe acute respiratory distress syndrome Predicting survival after extracorporeal membrane oxygenation for severe acute respiratory failure. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) score An External Validation of Scoring Systems in Mortality Prediction in Veno-Venous Extracorporeal Membrane Oxygenation External validation of scores proposed for estimation of survival probability of patients with severe adult respiratory distress syndrome undergoing extracorporeal membrane oxygenation therapy: A retrospective study A Comparative Analysis of Survival Prediction Using PRESERVE and RESP Scores Predicting Survival After Extracorporeal Membrane Oxygenation for ARDS: An External Validation of RESP and PRESERVE Scores Outcome of acute respiratory distress syndrome patients treated with extracorporeal membrane oxygenation and brought to a referral center Developing a simple preinterventional score to predict hospital mortality in adult venovenous extracorporeal membrane oxygenation: A pilot study The PRESET-Score: The extrapulmonary predictive survival model for extracorporeal membrane oxygenation in severe acute respiratory distress syndrome Predicting survival after ECMO for refractory cardiogenic shock: The survival after veno-arterial-ECMO (SAVE)-score The ENCOURAGE mortality risk score and analysis of long-term outcomes after VA-ECMO for acute myocardial infarction with cardiogenic shock Development and validation of a prognostic model for survival in patients treated with venoarterial extracorporeal membrane oxygenation: The PREDICT VA-ECMO score Predicting Survival After VA-ECMO for Refractory Cardiogenic Shock: Validating the SAVE Score Six-Month Survival After Extracorporeal Membrane Oxygenation for Severe COVID-19 Development and Reporting of Prediction Models: Guidance for Authors From Editors of Respiratory The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets Outcomes and factors associated with early mortality in pediatric and neonatal patients requiring extracorporeal membrane oxygenation for heart and lung failure Predictors and outcomes of early post-operative veno-arterial extracorporeal membrane oxygenation following infant cardiac surgery Concurrent Use of Renal Replacement Therapy during Extracorporeal Membrane Oxygenation Support: A Systematic Review and Meta-Analysis Impact of Hemolysis on Acute Kidney Injury and Mortality in Children Supported with Cardiac Extracorporeal Membrane Oxygenation Extracorporeal membrane oxygenation-hemostatic complications The Early Change in PaCO2 after Extracorporeal Membrane Oxygenation Initiation Is Associated with Neurological Complications Continuous Monitoring of Cerebral Autoregulation in Children Supported by Extracorporeal Membrane Oxygenation: A Pilot Study Impact of Arterial Carbon Dioxide and Oxygen Content on Cerebral Autoregulation Monitoring Among Children Supported by ECMO Duration of Hyperoxia and Neurologic Outcomes in Patients Undergoing Extracorporeal Membrane Oxygenation Central extracorporeal membrane oxygenation for refractory pediatric septic shock on behalf of the Australian & New Zealand Intensive Care Society (ANZICS) Centre for Outcomes & Resource Evaluation (CORE) and the Australian & New Zealand Intensive Care Society (ANZICS) Paediatric Study Group. Defining benefit threshold for extracorporeal membrane oxygenation in children with sepsis-A binational multicenter cohort study Neurological Monitoring and Complications of Pediatric Extracorporeal Membrane Oxygenation Support Data Availability Statement: Not applicable. The authors declare no conflict of interest.