key: cord-1020828-b8j14n3x
authors: Avery, Emily W.; Behland, Jonas; Mak, Adrian; Haider, Stefan P.; Zeevi, Tal; Sanelli, Pina C.; Filippi, Christopher G.; Malhotra, Ajay; Matouk, Charles C.; Griessenauer, Christoph J.; Zand, Ramin; Hendrix, Philipp; Abedi, Vida; Falcone, Guido J.; Petersen, Nils; Sansing, Lauren H.; Sheth, Kevin N.; Payabvash, Seyedmehdi
title: CT angiographic radiomics signature for risk stratification in anterior large vessel occlusion stroke
date: 2022-05-07
journal: Neuroimage Clin
DOI: 10.1016/j.nicl.2022.103034
sha: 98ead241f5fb4e0864cf4cd45f478a9c5f982446
doc_id: 1020828
cord_uid: b8j14n3x

BACKGROUND AND PURPOSE: As “time is brain” in acute stroke triage, the need for automated prognostication tools continues to increase, particularly in rapidly expanding tele-stroke settings. We aimed to create an automated prognostication tool for anterior circulation large vessel occlusion (LVO) stroke based on admission CTA radiomics. METHODS: We automatically extracted 1116 radiomics features from the anterior circulation territory on admission CTAs of 829 acute LVO stroke patients who underwent mechanical thrombectomy in two academic centers. We trained, optimized, validated, and compared different machine-learning models to predict favorable outcome (modified Rankin Scale ≤ 2) at discharge and 3-month follow-up using four different input sets: “Radiomics”, “Radiomics + Treatment” (radiomics, post-thrombectomy reperfusion grade, and intravenous thrombolysis), “Clinical + Treatment” (baseline clinical variables and treatment), and “Combined” (radiomics, treatment, and baseline clinical variables). RESULTS: For discharge outcome prediction, models were optimized/trained on n = 494 and tested on an independent cohort of n = 100 patients from Yale. Receiver operating characteristic analysis of the independent cohort showed no significant difference between best-performing Combined input models (area under the curve, AUC = 0.77) versus Radiomics + Treatment (AUC = 0.78, p = 0.78), Radiomics (AUC = 0.78, p = 0.55), or Clinical + Treatment (AUC = 0.77, p = 0.87) models. For 3-month outcome prediction, models were optimized/trained on n = 373 and tested on an independent cohort from Yale (n = 72), and an external cohort from Geisinger Medical Center (n = 232). In the independent cohort, there was no significant difference between Combined input models (AUC = 0.76) versus Radiomics + Treatment (AUC = 0.72, p = 0.39), Radiomics (AUC = 0.72, p = 0.39), or Clinical + Treatment (AUC = 76, p = 0.90) models; however, in the external cohort, the Combined model (AUC = 0.74) outperformed Radiomics + Treatment (AUC = 0.66, p < 0.001) and Radiomics (AUC = 0.68, p = 0.005) models for 3-month prediction. CONCLUSION: Machine-learning signatures of admission CTA radiomics can provide prognostic information in acute LVO stroke candidates for mechanical thrombectomy. Such objective and time-sensitive risk stratification can guide treatment decisions and facilitate tele-stroke assessment of patients. Particularly in the absence of reliable clinical information at the time of admission, models solely using radiomics features can provide a useful prognostication tool.

Background and Purpose: As "time is brain" in acute stroke triage, the need for automated prognostication tools continues to increase, particularly in rapidly expanding tele-stroke settings. We aimed to create an automated prognostication tool for anterior circulation large vessel occlusion (LVO) stroke based on admission CTA radiomics. Methods: We automatically extracted 1116 radiomics features from the anterior circulation territory on admission CTAs of 829 acute LVO stroke patients who underwent mechanical thrombectomy in two academic centers. We trained, optimized, validated, and compared different machine-learning models to predict favorable outcome (modified Rankin Scale ≤ 2) at discharge and 3-month follow-up using four different input sets: "Radiomics", "Radiomics + Treatment" (radiomics, post-thrombectomy reperfusion grade, and intravenous thrombolysis), "Clinical + Treatment" (baseline clinical variables and treatment), and "Combined" (radiomics, treatment, and baseline clinical variables). Results: For discharge outcome prediction, models were optimized/trained on n = 494 and tested on an independent cohort of n = 100 patients from Yale. Receiver operating characteristic analysis of the independent cohort showed no significant difference between best-performing Combined input models (area under the curve, AUC = 0.77) versus Radiomics + Treatment (AUC = 0.78, p = 0.78), Radiomics (AUC = 0.78, p = 0.55), or Clinical + Treatment (AUC = 0.77, p = 0.87) models. For 3-month outcome prediction, models were optimized/ trained on n = 373 and tested on an independent cohort from Yale (n = 72), and an external cohort from Geisinger Medical Center (n = 232). In the independent cohort, there was no significant difference between Combined input models (AUC = 0.76) versus Radiomics + Treatment (AUC = 0.72, p = 0.39), Radiomics (AUC = 0.72, p = 0.39), or Clinical + Treatment (AUC = 76, p = 0.90) models; however, in the external cohort, the

Stroke is the leading cause of severe disability and the fifth leading cause of death in the United States (Benjamin et al., 2017; Xu et al., 2020) , with large vessel occlusion (LVO) strokes accounting for 30% of all ischemic strokes but 90% of stroke-related mortality and severe disability (Malhotra et al., 2017) . Efforts to combat such devastating outcomes in LVO stroke have led to the rise of endovascular therapies, particularly mechanical thrombectomy in the last decade. Landmark trials have shown significant benefit of endovascular thrombectomy (ET) in LVO stroke outcome, even when performed up to 24 h after onset (Albers et al., 2018; Nogueira et al., 2018) .

However, it remains difficult to determine which patients will benefit from ET. For patients presenting more than six hours after stroke onset, current eligibility criteria ascribed by the American Heart Association (AHA) (Powers, 2019) rely on the mismatch between the volume of at-risk territory and the infarct core, as indicated by CT or MRI perfusion scans, which add cost and infrastructure requirements. Despite the overwhelming efficacy of ET, up to 50% of acute LVO patients have poor long-term outcomes after successful reperfusioni.e. futile recanalization (Meinel et al., 2020) . Conversely, of the 30% of acute ischemic stroke patients with intracranial LVO (Lakomkin et al., 2019) , only 7-8% are eligible for ET according to current AHA guidelines (Desai et al., 2019) , and, under those guidelines, the "number needed to treat" with ET is only 2-2.6 to achieve a 1-point improvement of 90-day modified Rankin Scale (mRS) score Weber et al., 2019) . These statistics highlight a great proportion of patients who may potentially benefit from ET but who do not fulfill current restrictive eligibility guidelines.

Non-contrast CT, followed by CTA, are the mainstay of acute ischemic stroke imaging workflow and LVO treatment triage (Martinez et al., 2020) . CTA source images have higher sensitivity in the detection of acute ischemic changes compared to CT (Camargo et al., 2007) , and areas of CTA hypoattenuation better correlate with depressed cerebral blood flow (CBF) rather than volume (CBV) in arterial phase images obtained with multidetector scanners (Pulli et al., 2012; Sharma et al., 2011) . Thus, quantitative features of admission CTAs may provide better prognostication and treatment triage for acute LVO stroke patients beyond estimation of infarct volume. This can be done by utilizing radiomics, which enables automated extraction of a large number of quantitative features from medical images representing shape, intensity, and texture characteristics. Machine-learning models using radiomics have already demonstrated utility in prediction of clinically relevant information based on radiomics features (Haider et al., 2020a) , including in LVO (Regenhardt et al., 2022) .

With the goal of assisting treatment decisions and improving baseline risk stratification, we hypothesized that radiomics features extracted from admission CTAs could predict post-thrombectomy outcome. From a database of over 800 patients from two comprehensive stroke centers, we devised and automated a platform for extraction of radiomics features from the anterior circulation territory of admission CTAs. Then, using different combinations of feature selection and machinelearning classifiers, we trained, optimized, validated, and compared different models for prediction of outcome at discharge and 3-month follow-up.

The subjects for training, validation, and independent testing of models were identified from the Yale New Haven Hospital stroke center registry between 1/1/2014 and 10/31/2020. In addition, a cohort of 232 subjects from Geisinger Medical Center who presented between 1/ 1/2016 and 12/31/2019 was used for external testing of models predicting long-term functional outcome at 3-month follow-up. Patients were included if they (1) suffered an anterior circulation LVO stroke (ICA, M1, or M2), (2) had CTA source images with slice thickness ≤ 1 mm, (3) underwent ET, and (4) had modified Rankin Scale (mRS) assessment of functional outcome recorded at discharge or long-term (at 3-month follow-up). Functional outcome was dichotomized as favorable (mRS ≤ 2) versus poor (mRS > 2), and patients were excluded from discharge or long-term analysis groups if they were missing mRS data for that timepoint. Post-thrombectomy reperfusion was assessed using the modified Thrombolysis in Cerebral Infarction (mTICI) system, reported by the treating neurointerventionalist. Patients were excluded if they had (a) any simultaneous posterior circulation LVO, (b) poor quality CTA not amenable to analysis (due to motion, metal artifact, or scanner- based artifacts), or (c) missing admission clinical information. In the external dataset, no patient was aged ≥ 90. Institutional review board approval was obtained for retrospective data collection with informed consent waived at respective institutes. All procedures followed were in accordance with institutional guidelines.

Using the FSL software FLIRT tool (https://www.fmrib.ox.ac.uk/), baseline CTA images were skull-stripped and co-registered to the Montreal Neurological Institute-152 brain space. Preset middle cerebral artery (MCA) territory masks, from the brain stroke atlas (Wang et al., 2019) , were then reverse registered to native CTAs (Fig. 1 ). All CTA images and MCA masks were resampled to an isotropic 1x1x1 mm voxel spacing using trilinear interpolation to ensures rotational invariance of texture features (Haider et al., 2020b; Haider et al., 2020c; Haider et al., 2020d) . To compensate for differences in intravenous bolus timing among different CTA scans, only voxels within a 1-500 Hounsfield unit (HU) range were included in analysis, and all images were normalized by centering voxel values at the mean with standard deviation from the image. After applying high-and low-pass filters in each spatial direction ("coif-1" wavelet transform (Pyradiomics-community, 2018)) and the "edge-enhancement" Laplacian of Gaussian (LoG) filter (with "sigma" settings of 2, 4, and 6 mm (Pyradiomics-community, 2018)), one set of 1116 "first-order" and "texture-matrix" radiomics feature were extracted from the single VOI combining right and left MCA territories (Pyradiomics-community, 2018). These steps of preprocessing, derivative image generation, and feature extraction were executed using a custom Pyradiomics version 2.1.2 pipeline (Pyradiomics-community, 2018). The first-order and texture-based features are listed in Supplementary  Table 1 .

To develop predictive models for favorable outcome (mRS ≤ 2) at discharge and long-term follow-up, we randomly allocated separate datasets for training/cross-validation, independent testing, and external testing. The training cohort was used for hyperparameter optimization and selection of optimal models through a repeated 5-fold crossvalidation scheme. Independent internal and external cohorts were used for testing the optimized models. For both discharge and long-term outcome prognostic models, four different input sets were used (Table 1) : (1) "Radiomics" (MCA territory radiomics features only), (2) "Radiomics + Treatment" (radiomics, reperfusion mTICI score, and intravenous thrombolysis)", (3) Clinical + Treatment (baseline clinical variables and treatment), and (4) "Combined" input (radiomics, treatment, and baseline clinical variables). In our machine-learning pipeline, reperfusion mTICI score was a 0-to-4 ordinal variable, intravenous thrombolysis was a binary variable, and clinical variables included sex (binary), age (numeric), and admission NIHSS (numeric) ( Table 1) . Haider et al., 2020c; Haider et al., 2020d) , we applied 20 rounds of 5fold cross-validation for hyperparameter optimization and internal performance validation of candidate models, each of which combined one of 6 feature selection strategies and one of 6 machine-learning classifiers (36 pairs, Table 1 ). A brief description of feature selection methods and machine-learning models is provided in the supplementary methods and detailed previously (Haider et al., 2020b; Haider et al., 2020c; Haider et al., 2020d) . Using the "rBayesianOptimization" package in R (Yan, 2016) , we applied Bayesian Optimization for iterative optimization of model performance in the cross-validation framework. The machine-learning classifiers' hyperparameters, their ranges, and tuning repetition counts are provided in Supplementary Table 2 . We applied 20 rounds of 5-fold cross-validation with optimized hyperparameters to measure the cross-validation performance of each candidate model. In each round of cross-validation, feature selection and training of machine-learning classifiers were performed on the training fold and assessed in validation folds to avoid data leakage from training to validation samples and reduce overfitting. This approach enabled accurate estimation of final model performance based on averaged receiver operating characteristic (ROC) area under the curve (AUC) in validation folds across all 100 permutations.

For each input set, the select model (feature selection and machinelearning algorithm combination) with the highest averaged AUC in cross-validation was trained on the whole training dataset applying optimized machine-learning hyperparameters. These final models were then tested on independent internal (Yale dataset) and external (Geisinger Clinic) cohorts. DeLong's test was used to compare paired AUCs and to calculate p-values and AUC 95% confidence intervals (CI) (DeLong et al., 1988) . The R pROC package was used for AUC metric computations (Robin et al., 2011) . Three main prognostic clinical variables at the time of admission (A) were included in the Combined and Clinical + Treament models. The treatment variables of post-thrombectomy reperfusion success (mTICI ascore) and intravenous thrombolytic treatment (B) were used in the Radiomics + Treatment, Clinical + Treatment, and Combined models. Six machine-learning classifiers (C) and 6 feature selection methods (D) were used in 36 combinations for the Radiomics, Radiomics + Treatment, Combined models, while feature selection was omitted in Clinical + Treatment models. Machine-learning and feature selection abbreviations were previsouly described in detail (Haider et al., 2020b) and are summarized in the supplementary methods. 

For univariate comparison between two groups, we used a student's t-test for continuous variables (age, onset-to-catheterization, onset-to-CTA), the Mann-Whitney rank test for ordinal variables (NIHSS and mRS), and the Fisher exact test for categorical variables. For univariate comparison between three groups, we used ANOVA for continuous variables, the Kruskal-Wallis test for ordinal variables, and the Chisquare test for categorical variables.

Our training dataset consisted of 496 Yale LVO stroke patients, 494 of whom had discharge outcome scores, and 373 of whom had long-term outcome scores available. See Supplementary Fig. 1 for flowchart of patients included in final analysis. Our independent internal testing cohort consisted of 101 Yale LVO stroke patients, 100 of whom had discharge outcome scores and 72 of whom had long-term outcome scores available. Our external testing cohort from Geisinger Health consisted of 232 LVO stroke patients with long-term outcome information available. Table 2 summarizes the demographic characteristics of patients included for discharge and long-term outcome prediction.

Heatmap summaries of the cross-validation performance for all candidate models are depicted in Fig. 2 . The best performing Radiomics only models had an averaged AUC of 0.69 ± 0.06 for long-term outcome prediction and 0.65 ± 0.05 for discharge outcome prediction. The best performing Radiomics + Treatment models had an averaged AUC of 0.74 ± 0.06 for long-term outcome prediction and 0.70 ± 0.05 for discharge outcome prediction. The best performing Clinical + Treatment models had an averaged AUC of 0.810 ± 0.05 for long-term outcome prediction and 0.82 ± 0.05 for discharge outcome prediction. The best performing Combined input models had an averaged AUC of 0.82 ± 0.05 for long-term outcome prediction and 0.81 ± 0.04 for discharge outcome prediction. Support vector machine (SVM) with either sigmoid or radial kernel was the machine-learning classifier utilized by the best-performing models of all input types except the Combined model for prediction of long-term outcome, which utilized logistic regression with elastic net regularization (ElNet). The feature selection methods most utilized by best-performing models were principal component analysis (PCA, used in 3 of the 6 best-performing models) and minimum redundancy maximum relevance filter (MRMR, used in 2 of 6 best-performing models). Notably, cross-validation results for models trained on radiomics features extracted from the whole brain had similar averaged AUC to those using MCA territory radiomics ( Supplementary Fig. 2) .

The internal independent and external testing performances of the best-performing models selected from cross-validation results are summarized in Fig. 3 and Table 3 

To discern factors influencing incorrect radiomics model predictions in external testing, we compared the group of subjects whose outcomes were falsely predicted by the Radiomics, Radiomics + Treatment, and Combined model types (false negative (FN) and false positive (FP) predictions) to the group of subjects whose outcomes were correctly predicted by all 3 radiomics-containing model types (true positive (TP) and true negative (TN) predictions). Our models' binary predictions for favorable versus poor outcomes were derived by rounding the probabilistic output (0.00-1.00) of machine-learning classifiers for each subject. The FP + FN subgroup differed significantly from the TP + TN subgroup with regard to proportion of good 3-month outcomes (FP + FN: median mRS = 1(0-2), 80% mRS < 3; TP + TN mRS = 4(3-6), 10% mRS < 3; p < 0.0001), admission NIHSS (FP + FN: median 17(12-22); TP + TN: 21(16-26), p = 0.0025), and proportion of male subjects (FP + FN:30%; TP + TN: 49%; p = 0.035).

Using radiomics features extracted from the admission CTAs of 829 acute LVO stroke patients, we devised, optimized, and validated machine-learning classifiers to predict functional outcomes at both discharge and 3-month follow-up. Four different input types were analyzed: Radiomics only, Radiomics + Treatment (radiomics, reperfusion mTICI score, and intravenous thrombolysis), Clinical + Treatment (clinical variables and treatment), and Combined input (radiomics, treatment, and clinical variables), all of which showed strong performance in cross-validation analyses (Fig. 2) . In independent testing using a separate 101-subject dataset from the same institution, Radiomics, Radiomics + Treatment, and Clinical + Treatment models performed with similar accuracy to the Combined model in prediction of both discharge and long-term outcome (Tables 3a & 3b, Fig. 3 ). In external testing using a 232-subject dataset from another institution, the Combined model performed with similar accuracy to the Clinical + Treatment model, but significantly better than both the Radiomics and Radiomics + Treatment models (Table 3b, Fig. 3 ). In all, the general success of radiomics-based model types in predicting LVO stroke outcome indicates that there may be a role for radiomics models in guiding cost-and time-sensitive treatment decisions by automating risk stratification of acute LVO stroke patients based on admission CTA.

As clinical variables serve as the current standard for acute stroke triage, the fact that similar prognostic performance was observed in the radiomics and clinical models indicates that a radiomics-only model can offer useful prognostic information at the time of admission when there are 'real-life scenario' limitations in obtaining thorough clinical information. Such scenarios include: the inability to perform a thorough neurological exam, patient sedation, language barrier, fluctuating neurological exam, and prior motor deficits confounding baseline exam. Furthermore, image-guided risk stratification tools can potentially help expand treatment eligibility of acute LVO stroke patients to those with unknown time of onset (wake-up stroke) or with mild/indeterminate stroke severity.

As indicated by the strong performance of Radiomics and Radiomics + Treatment models in cross-validation and independent cohort analyses, radiomics-based model success was observed whether or not input variables included reperfusion success or even clinical data. However, the addition of clinical input variables allowed the Combined model to demonstrate more stable generalizability upon external testing. Of  Fig. 2 . Heatmap summary of cross-validation performance for all candidate models. The feature selection/machine-learning combinations with the highest averaged area under the curve (AUC) across validation folds (from 20 repeats × 5-fold cross-validation) are highlighted with bold yellow cell border lines. These bestperforming models were selected for internal independent and external cohort testing. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) interest, the statistically inferior performance of Radiomics and Radiomics + Treatment models was driven by a tendency of these model types to overestimate the risk of poor outcome: incorrectly predicted cases had a significantly higher proportion of patients with 3-month favorable outcome compared to correctly predicted cases in external testing (80% vs 10%, p < 0.0001; median mRS 1(0-2) vs 4(3-6), p < 0.0001). Such a tendency towards overestimation of risk would in reality promote additional support for patients expected to do poorly, without imposing potential harm. The prominence of successful mTICI reperfusion scores may also contribute to the minimal improvement observed in the Radiomics + Treatment model as compared to Radiomics alone: mTICI scores of 2b or 3 were observed in 74% of subjects in the training set, 87% in the internal independent cohort, and 93% in the external cohort. These proportions of successful reperfusion are in line with other recently published studies such as the ASTER trial, in which 86% of subjects who underwent ET had mTICI 2b or greater (Dargazanli et al., 2018) . However, it has been documented that operators tend to overestimate the degree of reperfusion compared to core lab scoring . In general, a ceiling effect of mTICI scoring likely reduces this variable's capability of enhancing model performance.

A fully automated prognostication tool based on admission CTA radiomics can facilitate objective assessment of stroke patients and effective treatment triage in the context of expanding telemedicine practice. While the use of telemedicine to guide stroke treatment is well established, the COVID-19 pandemic motivated rapid expansion of care via tele-stroke to provide consistent and timely care while minimizing patient and provider exposure and preserving personal protective equipment (Guzik et al., 2021) . Prior studies have shown that tele-stroke programs improve the timeliness of thrombolysis treatments (Lee et al., 2017) . In addition, tele-stroke services may reduce the racial, ethnic, and sex disparities in ischemic stroke care (Reddy et al., 2021) . Furthermore, current human prediction accuracy of LVO patient outcome requires improvement, as evidenced by the current rate of poor prognosis after successful ET (futile reperfusion, observed in 50% of LVO patients undergoing ET) (Meinel et al., 2020) and the high percentage (92-93%) of LVO patients who do not currently meet criteria for ET (Desai et al., 2019) . Models such as ours with AUC ranges up to 0.74 in external testing exceed these statistics and could improve patient Fig. 3 . Area under the curve (AUC) of receiver operating characteristics (ROC) analysis for prediction of discharge outcome in the internal independent cohort, and prediction of long-term outcome in the independent and external cohorts (Table 3) . ROC curves for Radiomics (green), Radiomics + Treatment (blue), Clinical + Treatment (black), and Combined (red) models are shown in each quadrant. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) outcomes and resource utilization in both tele-stroke and traditional settings.

More specifically, identification of patients with futile reperfusion can guide treatment triage by optimizing patient selection for interhospital transfer to comprehensive stroke centers (Stefanou et al., 2020) . Patients with high likelihood of futile reperfusion may benefit from additional treatment intervention for improvement of long-term outcome and may be candidates for future clinical trials evaluating treatment options beyond current reperfusion interventions.

Our methodology may also guide future studies applying radiomics algorithms to stroke imaging data. Of interest, all but one of the bestperforming models identified in cross-validation utilized the support vector machine classifier with either sigmoid (SVM_sig) or radial (SVM_rad) kernel. While the most prominent classifiers in many recent machine-learning studies have been ensemble methods such as XGBoost and random forest (Belgiu and Dragut, 2016; Chen and Guestrin, 2016) , SVM classifiers have also shown utility in multiple neuro-radiomics studies (Ortiz-Ramon et al., 2019; Tian et al., 2018) . Future work at the intersection of machine-learning, radiomics, and stroke clinical research may utilize deep learning, which allows for implicit learning of relevant features and has been shown to significantly enhance machinelearning performance (LeCun et al., 2015) . Substantially larger training datasets are required for a deep learning approach compared to conventional machine-learning techniques and are needed to implement deep learning for prediction of LVO stroke functional outcome in future works. The machine-learning methods of the present work maintain the advantage of being more transparent than deep learning methods, allowing for extraction and analysis of important features.

Our study is limited by the nonrandomized nature of patient selection and the lack of an untreated control group. However, considering the outcome for unsuccessful thrombectomy as the natural course of untreated LVO, this subgroup may substitute for control subjects. Of note, the fact that practice-changing trials (Albers et al., 2018; Nogueira et al., 2018) were published during the timeline of patient presentation in our cohorts (2014-2020) may have impacted treatment variables in older vs more recently presenting patients. Our results also demonstrate inherent limitations in the generalizability of our Radiomics and Radiomics + Treatment models on external data, which could be in part due to differences in CTA acquisition methods. Additionally, we excluded patients with imaging artifacts such as metal instrumentation, head movement, and prior surgeries. While inclusion of these images could have potentially improved the robustness of final models, introduction of such noise into the training dataset could have hindered model development. Lastly, we trained our models on radiomics features extracted from the bilateral MCA territories rather than the whole brain or whole anterior circulation territory which includes anterior cerebral artery as well. However, models trained on whole-brain radiomics are described in Supplementary Fig. 2 and demonstrate similar crossvalidation performance to the models described in this manuscript.

In summary, our work demonstrates the feasibility of using automatically extracted radiomics features to create prognostic models for LVO patient outcomes from baseline CTA scans. With comparable performance to models based on patients' neurological exams and clinical variables, models solely utilizing radiomics features can help with prognostication when acquisition of baseline clinical information is limited or confounded. Such models are a promising step towards the increasingly important goals of implementing automated cost-and timesensitive decision-assistance tools, particularly in tele-stroke and community hospital settings, and increasing the number of patients who are eligible for life-saving ET.

Emily Avery declares no competing interests. Jonas Behland declares no competing interests. Adrian Mak declares no competing interests. Stefan Haider declares no competing interests. Tal Zeevi declares no competing interests. Dr. Pina Sanelli declares no competing interests. Dr. Christopher G. Filippi receives consulting honoraria from Syntactx, Inc; minority stockholder in Avicenna.ai; and receives research The selected models represent the best-performing models (Fig. 2 hearmap) from each set of input variables as identified by highest averaged cross-validation AUC. These models were applied to the internal independent and external datasets to predict (A) discharge and (B) long-term functional outcome. No statistically significant difference in independent testing AUC values was found when comparing the Radiomics, Radiomics + Treatment, or Clinical + Treatment models to the Combined model in prediction of either (A) discharge or (B) long-term functional outcome. However, the Combined model outperformed Radiomics and Radiomics + Treatment models in prediction of long-term functional outcome in the external cohort (B). *P value represents difference from Combined input model. P values were calculated using DeLong's test. yStatistically significant at threshold p < 0.05.

Thrombectomy for Stroke at 6 to 16 Hours with Selection by Perfusion Imaging

Random forest in remote sensing: A review of applications and future directions

Acute brain infarct: detection and delineation with CT angiographic source images versus nonenhanced CT scans

XGBoost: A Scalable Tree Boosting System

Modified Thrombolysis in Cerebral Infarction 2C/Thrombolysis in Cerebral Infarction 3 Reperfusion Should Be the Aim of Mechanical Thrombectomy Insights From the ASTER Trial (Contact Aspiration Versus Stent Retriever for Successful Revascularization)

Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach

Acute Ischemic Stroke with Vessel Occlusion-Prevalence and Thrombectomy Eligibility at a Comprehensive Stroke Center

Telestroke Across the Continuum of Care: Lessons from the COVID-19 Pandemic

Applications of radiomics in precision diagnosis, prognostication and treatment planning of head and neck squamous cell carcinomas

PET/CT radiomics signature of human papilloma virus association in oropharyngeal squamous cell carcinoma

Prediction of post-radiotherapy locoregional progression in HPV-associated oropharyngeal squamous cell carcinoma using machine-learning analysis of baseline PET/CT radiomics

Potential Added Value of PET/CT Radiomics for Survival Prognostication beyond AJCC 8th Edition Staging in Oropharyngeal Squamous Cell Carcinoma

Thrombectomy versus medical management for large vessel occlusion strokes with minimal symptoms: an analysis from STOPStroke and GESTOR cohorts

Prevalence of large vessel occlusion in patients presenting with acute ischemic stroke: a 10-year systematic review of the literature

Deep learning

Participation in a Tele-Stroke Program Improves Timeliness of Intravenous Thrombolysis Delivery

Ischemic Strokes Due to Large-Vessel Occlusions Contribute Disproportionately to Stroke-Related Dependence and Death: A Review

Cost-Effectiveness Study of Initial Imaging Selection in Acute Ischemic Stroke Care

Association of initial imaging modality and futile recanalization after thrombectomy

Thrombectomy 6 to 24 Hours after Stroke with a Mismatch between Deficit and Infarct

Identification of the presence of ischaemic stroke lesions by means of texture analysis on brain magnetic resonance images

Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the

Acute ischemic stroke: infarct core estimation on CT angiography source images depends on CT angiography protocol

Lack of Racial, Ethnic, and Sex Disparities in Ischemic Stroke Care Metrics within a Tele-Stroke Network

pROC: an open-source package for R and S+ to analyze and compare ROC curves

CT angiographic source images: flow-or volume-weighted?

Optimizing Patient Selection for Interhospital Transfer and Endovascular Therapy in Acute Ischemic Stroke: Real-World Data From a Supraregional, Hub-and-Spoke Neurovascular Network in Germany

Radiomics strategy for glioma grading using texture features from multiparametric MRI

Stroke atlas of the brain: Voxel-wise density-based clustering of infarct lesions topographic distribution

Thrombectomy in posterior circulation stroke: differences in procedures and outcome compared to anterior circulation stroke in the prospective multicentre REVASK registry

Mortality in the United States

rBayesianOptimization: Bayesian Optimization of Hyperparameters

Operator Versus Core Lab Adjudication of Reperfusion After Endovascular Treatment of Acute Ischemic Stroke

Supplementary data to this article can be found online at https://doi. org/10.1016/j.nicl.2022.103034.