key: cord-0994735-qndyzxix
authors: Miller, William Dwight; Han, Xuan; Peek, Monica E.; Charan Ashana, Deepshikha; Parker, William F.
title: Accuracy of the Sequential Organ Failure Assessment Score for In-Hospital Mortality by Race and Relevance to Crisis Standards of Care
date: 2021-06-18
journal: JAMA Netw Open
DOI: 10.1001/jamanetworkopen.2021.13891
sha: 889f7799c626642df58e1cc417ca57b215a045a4
doc_id: 994735
cord_uid: qndyzxix

IMPORTANCE: Crisis Standards of Care (CSC) are guidelines for rationing health care resources during public health emergencies. The CSC adopted by US states ration intensive care unit (ICU) admission using the Sequential Organ Failure Assessment (SOFA) score, which is used to compare expected in-hospital mortality among eligible patients. However, it is unknown if Black and White patients with equivalent SOFA scores have equivalent in-hospital mortality. OBJECTIVE: To investigate whether reliance on SOFA is associated with bias against Black patients in CSC. DESIGN, SETTING, AND PARTICIPANTS: This cohort study was conducted using data from the eICU Collaborative Research Database of patients admitted to 233 US ICUs in 2014 to 2015. Included individuals were Black and White adult patients in the ICU, who were followed up to hospital discharge. Data were analyzed from May 2020 through April 2021. EXPOSURE: SOFA scores at ICU admission. MAIN OUTCOMES AND MEASURES: Hierarchical logistic regression with hospital fixed effects was used to measure the interaction between race and SOFA as a factor associated with in-hospital mortality, as well as the odds of death among Black and White patients with equivalent priority for resource allocation according to the SOFA-based ranking rules of 3 statewide CSC (denoted A, B, and C) under shortage conditions that were severe (ie, only patients with the highest priority would be eligible for allocation), intermediate (ie, patients in the highest 2 tiers would be eligible for allocation), or low (ie, only patients with the lowest priority would be at risk of exclusion). RESULTS: Among 111 885 ICU encounters representing 95 549 patients, there were 16 688 encounters with Black patients (14.9%) and 51 464 (46.0%) encounters with women and the mean (SD) age was 63.3 (16.9) years. The median (interquartile range) SOFA score was not statistically significantly different between Black and White patients (4 [2-6] for both groups; P = .19), but mortality was lower among Black individuals compared with White individuals with equivalent SOFA scores (odds ratio [OR], 0.98; 95% CI, 0.97-0.99; P < .001). This was associated with lower mortality among Black patients compared with White patients prioritized for resource allocation in 3 CSC under shortage conditions that were severe (system A: OR, 0.65; 95% CI, 0.58-0.74; P < .001; system B: OR, 0.70; 95% CI, 0.64-0.78; P < .001; system C: OR, 0.73; 95% CI, 0.67-0.80; P < .001), intermediate (system A: OR, 0.73; 95% CI, 0.67-0.80; P < .001; system B: OR, 0.83; 95% CI, 0.77-0.89; P < .001; system C: OR, 0.82; 95% CI, 0.77-0.89; P < .001), and low (system A: OR, 0.83; 95% CI, 0.77-0.89; P < .001; system C: OR, 0.86; 95% CI, 0.81-0.92; P < .001; not applicable for system B, which had fewer tiers). When SOFA-based ranking rules were adjusted for Black patients to simulate equitable allocation based on observed mortality, the proportion upgraded to higher priority ranged from 379 Black patient encounters (2.3%) in low shortage conditions to 2601 Black patient encounters (15.6%) in severe shortage conditions. CONCLUSIONS AND RELEVANCE: This study found that SOFA scores were associated with overestimated mortality among Black patients compared with White patients, and this was associated with a structural disadvantage for Black patients in CSC allocation systems. These findings suggest that guidelines should be revised to correct this inequity and alternative methods should be developed for more equitable triage.

This supplemental material has been provided by the authors to give readers additional information about their work. Definition of the Included and Excluded Cohort: (1) Method to identify first ICU stay during hospitalization, and Transferred patients: Eligible patientunitstayid's were identified from the Patients table. Records were excluded if the ICU stay was not the first during a hospitalization. The first ICU stay during a hospitalization was identified using the unitvisitnumber field, and was required to be "1". Patients who were transferred were identified using the hospitaladmitsource and unitadmitsource fields. Patients were excluded if the hospitaladmitsource or unitadmitsource was listed as "Other Hospital", "Other ICU", "ICU", or "ICU to SDU". For the PF ratios compiled from both the respiratoryCharting and treatments tables, progressively less stringent time restrictions were imposed as follows: we initially specified that the PaO2 had to be recorded within 90 minutes of the recorded FiO2 value; when a value was not available within 90 minutes, then a value up to 180 minutes after the FiO2 recording was accepted; when this was unavailable, a value up to 6 hrs was accepted; then a value up to 12hrs later, and finally a value up to 24hrs after the recorded FiO2 was accepted. The lowest PF ratio within 24hrs was retained for final analysis, and there were 45,136 PF ratios associated with eligible ICU stays for analysis.

When data was not available to calculate a PF ratio as described above, an SF ratio was calculated. The SF ratio was based preferentially on the FiO2 recorded in the respiratoryCharting table as described above. When this was not available, then the FiO2 was obtained from the treatments table as described above. We specified that the saturation had to be recorded after the FiO2, and we used the saturation recorded soonest after the highest recorded FiO2 within 24hrs of ICU admission. Saturations recorded more than 6hrs after the highest recorded FiO2 were not eligible for consideration. The SF ratio was then calculated and converted to a respiratory SOFA score according to the scale reported by Grissom et al. in their description of the mSOFA for critical care triage. 4 We performed a sensitivity analysis that was restricted to samples with PF ratios only; SF ratios were excluded from this analysis.

Cardiac SOFA: MAP was preferentially obtained from the meanBp field of the apacheApsVar table. When not available here, it was obtained from the noninvasivemean field of the vitalAperiodic table. The lowest value from within 24 hours of ICU admission was selected. Vasopressors were identified in the infusionDrug table, via identifying generic and trade names of the medications. Vasoactive medications included Dobutamine, Milrinone, Dopamine, Norepinephrine, Epinephrine, Phenylephrine, and Vasopressin. Angiotensin II was not in use at the time of data extraction for this cohort. Many of the vasoactive doses could not be reconciled with standard doses, even after rigorous analysis and cleaning, possibly due to faulty data entry or alternative reporting of dose. Therefore, we performed a sensitivity analysis in which we looked only at the number of vasoactive medications, without taking dose into account. This is similar to methods described by previous investigators. 5 In this analysis, 2 points were assigned for use of Dobutamine, 3 points were assigned for use of 1 vasopressor (Dopamine, Norepinephrine, Epinephrine, Phenylephrine, or Vasopressin), and 4 points were assigned for use of 2 vasopressors.

Liver SOFA: Bilirubins were preferentially obtained from the apacheApsVar table (values are from the first 24hrs of ICU stay).

When not available in the apacheApsVar table, then the highest bilirubin from within 24 hours of ICU admission was obtained from the labs table. When not available in the labs table, then the highest value from the same hospitalization, but prior to 24hrs within ICU admission was accepted.

Creatinine and urine output were preferentially obtained from the apacheApsVar table. When values were not available there, the creatinine was obtained from the labs table. When not available in the apacheApsVar table, urine output was obtained from the intakeOuput table, by identifying values labeled "urine" or "Nephrostomy". The values from the 24hrs prior to admission were significantly lower than the values from the 24hrs after admission. This was interpreted to likely reflect better recording in the 24hrs after ICU admission. Therefore, Urine output was only accepted from the first 24hrs after ICU admission.

Hematologic SOFA: Platelet counts were obtained from the lab table. The lowest platelet count within 24hrs of ICU admission was selected, and the SOFA score was calculated using this value.

Glasgow Coma Scale (GCS) was preferentially obtained from the apacheApsVar To test for the effects of our methods of defining SOFA components, we performed sensitivity analyses with two variations of the SOFA score described in eTable 1.

We tested for an interaction between Race and the SOFA score with hierarchical conditional logistic regression, with fixed effects according to hospital using the clogit command in Stata. The formula for this analysis was:

Mortality ~ SOFA+SOFA*Race (grouped by hospital)

To evaluate priority scores (system A, system B, and system C), we used the following formula with the clogit 6 command:

Formula: Mortality~X + X*Race, group(hospital) X = Priority scores using the thresholds in A, B, or C.

To control for patients who may have been hospitalized multiple times (but who can only die once), we performed one sensitivity analysis with the formula:

The formula to evaluate the relationship between SOFA components and mortality was:

Mortality~Resp_SOFA+Resp_Race_Interaction + Cardiac_SOFA + Cardiac_Race_Interaction + Kidney_SOFA + Kidney_Race_Interaction + Liver_SOFA + Liver_Race_Interaction + Heme_SOFA + Heme_SOFA_Interaction + Neuro_SOFA + Neuro_SOFA_Interaction, group (hospital) eTable 1. Sequential Organ Failure Assessment Score Variations Used in Primary and Sensitivity Analyses by Race eTable 1 Legend: We tested 3 variations of the SOFA score. The SOFA score tested in the primary analysis is described in the text. We also performed a sensitivity analysis in which the respiratory SOFA was based on PF ratio only, without inclusion of SOFA scores based on the SF ratio. We also tested a variation of the Cardiac SOFA in which the score was based on number of vasoactive medications irrespective of dose. Sensitivity analyses included (1) incorporating a patient-level indicator variable for patients who had multiple hospitalizations, (2) including only 1 randomly selected hospitalization per patient, (3) including only patients who did not receive dialysis during or prior to SOFA calculation, (4) including only patients treated in Medical ICUs, or Medical-Surgical ICUs, (5) including only patients whose primary admission diagnosis was respiratory failure, (6) including only patients treated with "Full Therapy" (no DNR or other limitations), and (7) .02 Legend: Formula tested was Mortality~Respiratory_SOFA + Respiratory_SOFA*Race + Liver_SOFA + Liver_SOFA*Race + Cardiac_SOFA + Cardiac_SOFA*Race + Kidney_SOFA + Kidney_SOFA*Race + Neurologic_SOFA + Neurologic_SOFA*Race + Hematologic_SOFA+Hematologic_SOFA*Race. This formula was tested within a hierarchical logistic regression framework with a fixed effect term for hospital. Abbreviations: 95% CI, 95% Confidence Interval Legend eFigure 3: Mortality was calculated for Black and White patients associated with each SOFA component score: (a) Respiratory SOFA, (b) Cardiac SOFA, (c) Liver SOFA, (d) Kidney (renal) SOFA, (e) Hematologic SOFA, and (f) Neurologic SOFA. * indicates significance (p<0.05) by chi-square to test for differences in mortality between Black and White patients with the same SOFA component score. Results of the adjusted analysis are presented in the text and this supplement (eTable 4). In the adjusted analysis, only the Kidney (renal) and Hematologic components were significantly associated with race as predictors of mortality. . Sensitivity analyses were describd in the text, and included (1) a model with a patient-level variable indicating whether a patient had been hospitalized more than once, (2) a model fit by including only one randomly selected hospitalization per patient, (3) excluding any patients who received dialysis during or prior to SOFA calculation, (4) a calculation relying only on number of vasoactive medications irrespective of doses, (5) a SOFA calculation relying only on PF ratios and excluding records that did not have a PF ratio available for calculation, and (6) Legend: As described in the main text and eTable 2, Systems A & C attempt to save life-years as well as lives by deprioritizing patients with life-limiting moderate or severe comorbidities. We performed a sensitivity analysis in which we added comorbidity points and re-classified patients' priorities based on both SOFA and Comorbidity points as recommended in these guidelines. Thresholds for "moderate" and "severe" comorbidities were CCI of 3 and 8, respectively, as recommended in the Crisis Standard of Care guideline from Colorado. Significant differences in in-hospital mortality persisted between Black and White patients in systems A & C after incorporation of comorbidities. Importantly, improving in-hospital mortality is not the goal of incorporating comorbidities. 

) a SOFA calculation relying only on PF ratios and excluding records that did not have a PF ratio available for calculation, and (6) a SOFA calculation using complete cases only, without any missing values. Abbreviations: SOFA, Sequential Organ Failure Assessment

Coding Algorithms for Defining Comorbidities in ICD-9-CM and ICD-10 Administrative Data: Med Care

The Search for Optimal Oxygen Saturation Targets in Critically Ill Patients

Derivation and validation of Spo2/Fio2 ratio to impute for Pao2/Fio2 ratio in the respiratory component of the Sequential Organ Failure Assessment score*: Crit Care Med

A Modified Sequential Organ Failure Assessment Score for Critical Care Triage

Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)

Conditional logit analysis of qualitative choice behavior