key: cord-0043792-qm5kt7oj
authors: López-Fernández, Yolanda M.; Smith, Lincoln S.; Kohne, Joseph G.; Weinman, Jason P.; Modesto-Alapont, Vicent; Reyes-Dominguez, Susana B.; Medina, Alberto; Piñeres-Olave, Byron E.; Mahieu, Natalie; Klein, Margaret J.; Flori, Heidi R.; Jouvet, Philippe; Khemani, Robinder G.
title: Prognostic relevance and inter-observer reliability of chest-imaging in pediatric ARDS: a pediatric acute respiratory distress incidence and epidemiology (PARDIE) study
date: 2020-05-25
journal: Intensive Care Med
DOI: 10.1007/s00134-020-06074-7
sha: 66a92d8e2d974010afff61dd580c79521836825b
doc_id: 43792
cord_uid: qm5kt7oj

PURPOSE: Definitions of acute respiratory distress syndrome (ARDS) include radiographic criteria, but there are concerns about reliability and prognostic relevance. This study aimed to evaluate the independent relationship between chest imaging and mortality and examine the inter-rater variability of interpretations of chest radiographs (CXR) in pediatric ARDS (PARDS). METHODS: Prospective, international observational study in children meeting Pediatric Acute Lung Injury Consensus Conference (PALICC) criteria for PARDS, which requires new infiltrate(s) consistent with pulmonary parenchymal disease, without mandating bilateral infiltrates. Mortality analysis focused on the entire cohort, whereas inter-observer variability used a subset of patients with blinded, simultaneous interpretation of CXRs by intensivists and radiologists. RESULTS: Bilateral infiltrates and four quadrants of alveolar consolidation were associated with mortality on a univariable basis, using CXRs from 708 patients with PARDS. For patients on either invasive (IMV) or non-invasive ventilation (NIV) with PaO(2)/FiO(2) (PF) ratios (or SpO(2)/FiO(2) (SF) ratio equivalent) > 100, neither bilateral infiltrates (OR 1.3 (95% CI 0.68, 2.5), p = 0.43), nor 4 quadrants of alveolar consolidation (OR 1.6 (0.85, 3), p = 0.14) were associated with mortality. For patients with PF ≤ 100, bilateral infiltrates (OR 3.6 (1.4, 9.4), p = 0.01) and four quadrants of consolidation (OR 2.0 (1.14, 3.5), p = 0.02) were associated with higher mortality. A subset of 702 CXRs from 233 patients had simultaneous interpretations. Interobserver agreement for bilateral infiltrates and quadrants was “slight” (kappa 0.31 and 0.33). Subgroup analysis showed agreement did not differ when stratified by PARDS severity but was slightly higher for children with chronic respiratory support (kappa 0.62), NIV at PARDS diagnosis (kappa 0.53), age > 10 years (kappa 0.43) and fluid balance > 40 ml/kg (kappa 0.48). CONCLUSION: Bilateral infiltrates and quadrants of alveolar consolidation are associated with mortality only for those with PF ratio ≤ 100, although there is high- inter-rater variability in these chest-x ray parameters. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s00134-020-06074-7) contains supplementary material, which is available to authorized users.

Acute respiratory distress syndrome (ARDS) includes non-cardiogenic pulmonary edema caused by damage to the alveolar epithelial-endothelial barrier. These changes result in the clinical hallmarks of ARDS which include radiographic abnormalities [1] . Bilateral opacities are required in the American European Consensus Conference (AECC) and Berlin definitions [2, 3] . The Pediatric Acute Lung Injury Consensus Conference (PALICC) definition for pediatric ARDS (PARDS) eliminated the requirement for bilateral radiographic findings, although evidence of new infiltrate(s) consistent with acute pulmonary parenchymal disease [4, 5] is still required. Rationale for this change included: high inter-observer variability in the interpretation of bilateral infiltrates on chest X-ray (CXR) [6] [7] [8] [9] [10] , low sensitivity of CXR for alveolar consolidation that often lags behind the degree of hypoxemia evidence that diffuse alveolar damage may neither be homogeneous nor bilateral [11] , lack of evidence that bilateral infiltrates contribute to risk stratification in ARDS [12] .

We sought to prospectively evaluate components of the rationale to eliminate bilateral infiltrates from the PAL-ICC definition, through the Pediatric Acute Respiratory Distress Syndrome Incidence and Epidemiology (PAR-DIE) study, which was conducted in 145 pediatric intensive care units (PICUs) in 27 countries [13] . Our primary hypothesis was that bilateral infiltrates are not independently associated with mortality after controlling for oxygenation impairment. Secondary hypotheses included that bilateral infiltrates and quadrants of alveolar consolidation have poor inter-rater reliability between ICU physicians and radiologists, and that comorbidities amongst other factors influence this variability.

The present research comprises a pre-specified ancillary study of PARDIE. PARDIE was a prospective, international observational study with the goal of understanding the implications of the new PALICC definition. Children's Hospital Los Angeles (Los Angeles, CA, USA) was the coordinating center, and sites obtained ethical approval from their local institutional review board (IRB) or used a central IRB. Enrollment occurred over 10 distinct weeks during 2016-2017. Detailed PARDIE methods have been previously published [13] , and the PARDIE protocol is provided in Appendix in ESM.

Patients were eligible if they met PALICC criteria [4] : (1) hypoxemia within 7 days of a clinical insult; (2) respiratory failure not fully explained by cardiac failure or fluid overload; (3) chest imaging with new infiltrate/s; (4) hypoxemia severity: for noninvasively ventilated subjects, PaO 2 /FiO 2 (PF) ≤ 300 with full-face/oronasal mask with continuous positive airway pressure (CPAP) ≥ 5 cm H 2 O, while for intubated subjects, oxygenation index (OI) ≥ 4. Noninvasive equivalents SpO 2 /FiO 2 (SF) and oxygenation saturation index (OSI) were allowed for subjects without PaO 2 [14] [15] [16] . Patients had to be diagnosed within 24 h of enrollment [13] .

Subjects were excluded if they were perioperative from a cardiac intervention, had active perinatal lung disease, or met PARDS criteria > 24 h before screening. Site investigators were responsible for assuring completeness and validity of data (Appendix in ESM).

Sites participating in this ancillary study collected additional data related to chest imaging obtained for clinical purposes over the first 3 days of PARDS diagnosis. CXRs were independently interpreted by multiple providers without receiving prior educational training, ideally one pediatric intensivist and one radiologist who recorded their interpretations on specific rating forms (Appendix in ESM). If additional chest imaging (computed tomography (CT) or ultrasound) was available, a radiologist completed the rating form for the additional imaging.

Intensivists had to be either an attending physician or a fellow in pediatric critical care, and radiologists were either a general or pediatric radiologist. Years of clinical experience post-fellowship were recorded for all providers. Elements assessed included infiltrates (uni/bilateral) and anatomical quadrants of consolidation (right upper lobe, right middle/lower lobe, left upper lobe, left lower lobe), presence of pleural effusion, pneumothorax, subcutaneous emphysema, pneumomediastinum, pneumoperitoneum, and cardiomegaly. Two yes/no questions were added: "Do you believe the areas of opacification are predominately due to atelectasis?" and "Do you believe the CXR was consistent with congestive heart failure?" Participants were blinded to other clinical information.

These radiographic data were combined with information from PARDIE study which included demographics, diagnostic information, hypoxemia severity, ventilator settings, severity of illness, and outcome [13] . The protocol, rating form, and data management are in Appendix in ESM.

The main PARDIE dataset was used to test the primary hypothesis that bilateral infiltrates are not independently associated with mortality after controlling for oxygenation impairment, using CXR interpretations from the site intensive care practitioner. Data are presented using categorical "buckets" for oxygenation impairment based on PALICC severity: NIV, mild, moderate, and severe. These criteria use PF/SF ratio for noninvasive ventilation (NIV) and OI/OSI for invasive mechanical ventilation (IMV). Given that NIV patients may have varying degrees of oxygenation impairment, we further analyzed all patients together using PF/SF ratio. SF ratio was converted into PF equivalents using previously published formulae [14] . However, PF ratio was found to have a nonlinear relationship with mortality. As such, PF ratio was categorized into the ranges used in the Berlin definition, for stratified analysis. Logistic regression models were then built within the substrata with PF ≤ 100, to satisfy assumptions of regression. Similar analysis was done for the secondary variable of four quadrants of alveolar consolidation. Sample size was not a priori specified because it was based on recruitment from the parent PARDIE study, which enrolled 708 patients with a mortality rate of 17%. Post hoc power analysis demonstrated we could detect an 8.5% mortality difference with unilateral versus bilateral infiltrates for all PARDS patients, or a 16% difference in those with PF ≤ 100, with an alpha of 0.05 and a power of 0.8 [13] .

A specific V.3 dataset was used to test the secondary hypotheses that bilateral infiltrates and quadrants of consolidation have poor inter-rater reliability between ICU physicians and radiologists, and that comorbidities amongst other factors influence this variability. For inter-observer variability, all films had simultaneous interpretations by an intensivist and radiologist. If multiple practitioners (intensivist, radiologist) provided interpretations, we selected the most experienced in each field prioritizing pediatric over general radiologists.

Inter-observer variability was calculated using Cohen's k coefficient. Despite some limitations [9] , kappa appropriately accounts for chance-corrected agreement between readers. Values < 0 were interpreted as poor; 0-0.2, slight; 0.21-0.4, fair; 0.41-0.6, moderate; 0.6-0.8, substantial; and 0.8-1 almost perfect agreement [17, 18] . To evaluate factors associated with disagreement in the presence or absence of bilateral infiltrates, demographics and other variables were compared. To aid clinical interpretation and calculation of kappa, continuous variables were categorized using a locally weighted smoothing scatter plot which explored the relationship between the disagreement on bilateral infiltrates and the variable of interest. If a dose response or cutoff point was discerned from the plot, continuous data were correspondingly categorized. Disagreement was then analyzed with kappa statistics. This is necessary because the rates of bilateral infiltrates in ARDS patients are correlated with other variables (i.e., oxygenation impairment). Raters are more likely to agree about bilateral infiltrates by chance alone in subgroups where the rates of bilateral infiltrates are higher. Kappa corrects for this chance-related agreement. Individual kappa values and 95% confidence intervals are presented, but no specific inference testing was performed.

Finally, other images (CT and ultrasound) were matched to a CXR from the same day. When multiple X-ray interpretations were available, the radiologist's interpretation was preferentially taken, followed by the most experienced intensivist. Categorical variables were compared with Chi-squared or Fischer exact tests. All tests were two-sided with p values less than 0.05 deemed significant. Analyses were performed using SAS (version 9·4; SAS Institute, Cary, NC) and STATA version 15 and verified by two statisticians.

All 708 patients enrolled in the PARDIE study had a CXR available at PARDS diagnosis, and 17.1% of patients died [13] . Bilateral infiltrates were present on the initial CXR in 523 patients compared with 185 patients with unilateral infiltrates (19.5% vs 10.3% mortality, p = 0.004). Patients were stratified by the initial degree of oxygenation impairment as measured by PALICC grouping, and bilateral infiltrates on initial CXR were associated with higher mortality for those with NIV PARDS (19.1% vs 5.9%, p = 0.03), with a nonsignificant trend for higher mortality in those with severe PARDS based on OI/OSI criteria ( 34% vs 16%, p = 0.07, Table 1a ).

When combining patients on IMV and NIV, mortality and the rate of bilateral infiltrates increased as initial PF ratio decreased, particularly when PF was ≤ 100 (Supplemental Table 1a ). There was no association between bilateral infiltrates on the initial film and mortality (OR 1.3 (95% CI 0.68, 2.5), p = 0.43) for patients with PF > 100, regardless of initial mode of ventilation. When the initial PF ≤ 100, bilateral infiltrates were associated with higher mortality for those on NIV (p = 0.0495) with a strong trend for those on IMV (p = 0.06, Fig. 1 ). Overall mortality was 27.8% for patients with PF ≤ 100 and bilateral infiltrates, compared to 9.8-14.4% for all other combinations of infiltrates and PF ratios (Fig. 2 ). Limiting the analysis to 277 patients with PF ≤ 100 (combining IMV and NIV), patients with bilateral infiltrates had a higher risk of mortality (OR 3.6 (1. 4, 9.4 , p = 0.01)), which was not modified by being on NIV (p = 0.75) or the actual PF ratio (p = 0.87).

Within the first 3 days of PARDS diagnosis, 90 of the 185 patients who did not initially have bilateral infiltrates 

Relationship between ICU mortality and chest X-ray (CXR) infiltrates stratified by initial PF ratio (or SF ratio equivalent) and ventilation type (invasive or noninvasive mechanical ventilation) at PARDS diagnosis. 708 subjects from the whole PARDIE cohort had a CXR available (185 unilateral/523 bilateral infiltrates) at PARDS diagnosis. Bars represent number of cases per initial PF ratio and ventilation type. Whiskers represent ± 1 standard error around the point estimate for mortality (squares). NIV, noninvasive ventilation; IMV, invasive mechanical ventilation; PF: ratio of partial pressure of oxygen (PaO 2 ) to fractional concentration of oxygen (FiO 2 ), PO 2 /FiO 2 ratio went on to develop them. Mortality for the 95 patients who did not have bilateral infiltrates was 8.4% versus 18.4% in the 613 patients who did have bilateral infiltrates within the first 3 days of PARDS (p = 0.016). Similar to the initial CXR, there was no association between bilateral infiltrates and mortality amongst patients who always had a PF > 100 within the first 3 days of PARDS diagnosis (7.8% vs 6.3%, p = 0.8), with a nonsignificant trend for higher mortality (25.4% vs 12.9%, p = 0.12) for those with bilateral infiltrates and at least one PF ≤ 100 (Supplemental Fig. 1) .

There was an association between the number of quadrants of alveolar consolidation on the initial CXR and mortality. When stratifying by the degree of hypoxemia as measured by initial PALICC grouping, a higher number of quadrants (particularly four) was associated with higher mortality for those with NIV (p = 0.004, Table 1b ). There was no clear association with quadrants of consolidation and mortality for those with mild/ moderate PARDS. For severe PARDS, either one or four quadrants had close to 40% mortality, while 2 or 3 quadrants had closer to 20% (Table 1b) . When combining patients on IMV and NIV, there was no association between four quadrants of consolidation and mortality amongst patients with an initial PF > 100 (16.6% vs 11%, (OR 1.6 (0.85, 3), p = 0.14)), but for those with initial PF ≤ 100, four quadrants of consolidation was associated with higher mortality (33.7% vs 20.2%, OR 2 (1.14, 3.5, p = 0.02)), which was not modified by being on NIV (p = 0.81) or the actual PF ratio (p = 0.83) (Supplemental Fig. 2 ). When looking at specific quadrants, consolidation of the right upper lobe (RUL) was not associated with mortality, while consolidation in any other quadrant was associated with higher mortality (Supplemental Table 2a ). However, there was no clear discernable pattern between specific combinations of quadrants and mortality (Supplemental Table 2b ) for those with fewer than four quadrants.

Of the 708 patients enrolled in the PARDIE study, 379 contributed data to the ancillary V.3 study with 1151 CXRs. There were simultaneous interpretations of 702 CXRs from 233 patients (Flowchart, ESM). Demographics, hypoxemia severity, and outcomes are provided in Table 2 interpretations, with a median of 8 (5, 13) years of experience post-fellowship. Of the 702 films, radiologists and intensivists agreed on the presence or absence of bilateral infiltrates in 537 (76.5%) cases. Intensivists identified bilateral infiltrates in 81/702 (11.5%) cases when radiologists did not, and radiologists identified bilateral infiltrates in 84/702 (12%) cases when intensivists did not. Correcting for chance-related agreement, inter-observer agreement between interpretations of bilateral infiltrates and quadrants of consolidation was "slight" (kappa 0.31 and 0.33). If each quadrant is considered independently, the greatest agreement is reached in RUL (kappa, k = 0.59, moderate agreement), with the least agreement in the left lower lobe (k = 0.33) (Supplemental Fig. 3) . There was relatively more agreement on the presence of pleural effusion (k = 0.5), pneumothorax (k = 0.7), pneumomediastinum (k = 0.75), and subcutaneous emphysema (k = 0.77), but poor agreement on cardiomegaly (k = 0. 19) and atelectasis (k = 0.16) ( Table 3) . Radiologists diagnosed atelectasis in 36.3% of CXRs compared with 20% by intensivists.

Factors associated with the overall rate of bilateral infiltrates include comorbidities, PARDS severity, ventilator settings, age, and fluid balance. As an example, bilateral infiltrates were present in approximately 72% of CXRs when FiO 2 levels were between 0.21 and 0.4, compared with 87% with FiO 2 0.8-1. The percent agreement for bilateral infiltrates between radiologists and intensivists is 70.5% for those with FiO 2 0.21-0.4, but 85.6% for those with FiO 2 0.8-1. While the overall agreement is higher when FiO 2 is higher, after correcting for the chanceexpected agreement, the kappa values were similar (0.26 vs 0.36). Hence, FiO 2 level does not appear to influence inter-observer reliability. Similar findings were seen for other variables (Table 4) , and kappa values generally remained in the 0.2-0.4 range. Some notable exceptions

Comparison to the patients included in V0 but not the in V3 and to the full V0 cohort LV dysfunction, left ventricular dysfunction; NIV, noninvasive ventilation; inMV, invasive mechanical ventilation; VFD, ventilator-free days. Data presented as median (IQR) or number (%) a Data to calculate LMV missing from one patient Table 3 Inter-observer agreement between pediatric intensivist and radiologist interpretations of chest X-ray findings Total N = 702 films unless otherwise specified RUL right upper lobe, RML middle lobe, RLL right left lobe, LUL left upper lobe, LLL left lower lobe Kappa (k): agreement beyond chance was classified as follows: poor agreement, k < 0.00; slight, k < 0.20; fair, k = 0.21-0.4; moderate, k = 0.41-0.6; substantial, k = 0.61-0.8; almost perfect agreement, k > 0.8 [14, 15] included higher agreement for chronic respiratory support (k = 0.62), NIV PARDS (k = 0.51), age > 10 years (k = 0.43), and net fluid balance > 40 ml/kg on the day before the CXR (k = 0.48) ( Table 4 ). Lower agreement was noted for acquired heart disease (k = 0.1) and those 

There were 32 CTs from 21 subjects and 20 ultrasound images from 15 subjects in the dataset. 31/32 Chest CTs and 15/20 ultrasounds could be matched to a CXR on the same day. Comparisons are shown in Supplemental Table 3 . Kappa values for bilateral infiltrates between CT and CXR, and ultrasound and CXR were both 0.56. There were high levels of agreement between ultrasound and CXR for pleural effusion (k = 0.76), although this was lower with CT and CXR (k = 0.18).

We have found that radiographic findings related to bilateral infiltrates or quadrants of alveolar consolidation are associated with mortality for PARDS patients with a PF ≤ 100 either on invasive or noninvasive ventilation. There was no clear relationship between CXR and mortality for patients with less severe oxygenation impairment. Mortality for those with PF ≤ 100 and bilateral infiltrates was 28%, compared to 10-15% for all other patients in the cohort. While nearly 87% of patients who meet PALICC criteria were deemed to have bilateral infiltrates within 3 days of PARDS diagnosis, the agreement on which patients these are between intensivists and radiologists is only marginally better than expected by chance alone (kappa 0.31-0.33). Similar findings were seen for quadrants of consolidation. While this study was in pediatric ARDS, we would encourage focused investigation down these lines in adults with ARDS. Prognostic relevance and inter-observer reliability of chest imaging are important considerations in creating diagnostic criteria for ARDS [2, 3, [19] [20] [21] [22] . With regard to our hypothesis that radiographic criteria would not retain prognostic relevance after controlling for oxygenation impairment, we found that bilateral infiltrates or four quadrants of consolidation as gauged by an intensivist were associated with higher mortality for patients with more severe oxygenation impairment (PF ≤ 100). Interestingly, there was no relationship between CXR findings and mortality for those with less severe oxygenation impairment. Moreover, 50% of patients who did not have bilateral infiltrates at PARDS diagnosis went onto develop them within the first 3 days of PARDS. Consistent with the data from the initial film, bilateral infiltrates only appeared to be associated with higher mortality for those with PF ≤ 100, although not statistically significant, likely because only 13% of patients who met PALICC criteria did not have bilateral infiltrates within 3 days of PARDS. Oxygenation impairment and opacification on CXR are both surrogates for loss of end-expiratory lung volume and intrapulmonary shunt, important components of ARDS pathobiology. However, from our data, it appears as if bilateral infiltrates combined with severe oxygenation impairment constitute a PARDS patient who has much higher risk of mortality than any other type of PARDS patient, regardless of mode of ventilation.

Intriguingly, when looking at quadrants of consolidation amongst those with PF ≤ 100, patients with one or four quadrants of consolidation had similar mortality, which was considerably lower than those with two or three quadrants. This observation reinforces that current methods to gauge chest imaging have limitations related to prognostic relevance in ARDS. Perhaps this can be overcome with more explicit methods for density of opacification, as suggested by the RALE score [12] . Unfortunately, we did not gather the elements required to calculate this score in our study.

There is no gold standard to define ARDS in either children or adults. Some argue that histopathology should be the definitive gold standard, but this will skew evaluation of diagnostic criteria to those who die or are sicker. Clinical syndromes as ARDS are defined using a constellation of signs and symptoms which together make it highly likely that the patient has the disease, to apply appropriate treatment strategies or aid in prognosis. Therefore, diagnostic criteria should be objective and reproducible. With regard to our second hypothesis that radiographic criteria have poor inter-observer reliability, we found the kappa values for chance-corrected agreement between intensivists and radiologists on bilateral infiltrates or quadrants of consolidation were in the fair range. While some elements had higher agreement (RUL opacification, pneumothorax), these are not used in ARDS definitions. Our study represents real-world clinical practice, collecting data from adequately experienced radiologists and intensivists. Furthermore, there was no clear pattern that one specialty was more likely to interpret bilateral infiltrates or more quadrants of disease, as the rates were similar, confirming that interpretations of bilateral infiltrates and quadrants of consolidation are not reproducible amongst practitioners who routinely evaluate CXRs. While there was higher "agreement" for those with severe hypoxemia, when adjusting for the fact that these patients are more likely to have bilateral infiltrates, the chance-corrected agreement was only marginally better than chance alone.

These results are consistent with previous investigations, which have found kappa values between 0.27 and 0.3 on the interpretation of bilateral infiltrates amongst ARDS patients [7, 23] . Strategies focused on standardized training interventions or automated processing of digital images [9, 10, 23, 24] have been proposed, but it is debatable whether training improves reliability. Likewise, the agreement can be improved when readers are experts in ARDS, although this has clear limitations for clinical practice [6] . Studying the impact of such interventions is an important area for the future research, but it seems clear that current methods for the interpretation of bilateral infiltrates as part of diagnostic criteria for ARDS are not reproducible between intensivists and radiologists.

We also found that intensivists and radiologists are more likely to agree on the presence or absence of bilateral infiltrates when patients were on chronic respiratory support prior to PARDS, had NIV PARDS, age > 10 years and a positive fluid balance. In contrast, there were higher levels of disagreement for those with some component of cardiac disease. Retaining the bilateral infiltrate requirement in the definition may therefore inadvertently make the diagnosis more likely in certain subgroups, such as older children, and more difficult in others, such as those with cardiac disease.

Given limitations of chest X-rays, some propose using lung ultrasound or chest CT. Indeed, the PALICC definition affords using any chest imaging to diagnose pulmonary parenchymal disease [5] . However, we have confirmed their use is very uncommon. Therefore, despite some potential therapeutic and diagnostic advantages [25] [26] [27] [28] , lack of widespread use of these modalities is a major limitation.

Our study has limitations. First, this study was not focused on whether the pathobiology of ARDS mandates exclusively bilateral lung involvement. It is important to uncouple this fundamentally pathophysiologic question from our study which surrounds the prognostic relevance and inter-observer reliability of the interpretation of CXRs in ARDS. Our study is not meant to directly address whether ARDS definitions should distinguish "unilateral" from "bilateral" disease. It appears important to focus research on understanding whether the severe hypoxemia group with bilateral infiltrates represents a different phenotype of ARDS, given such a high risk of poor outcome. However, CXRs may be an inaccurate way to do this, and other chest imaging modalities are not used routinely to replace CXRs in ARDS. Second, not all sites participated in the ancillary study. Mortality analysis focused on interpretations for 708 PARDS patients, while the inter-observer analysis was conducted on a smaller subgroup. Nevertheless, simultaneous interpretation of 702 radiographs represents one of the largest studies of inter-rater reliability in either adults or pediatrics [6] [7] [8] [9] [10] 24] , and the groups were similar (Table 2) . Third, practitioners were blinded to clinical information. We cannot predict whether inter-observer agreement would be better if more clinical information was provided, although previous studies have not shown a substantial impact [29] . Fourth, very few CT or US interpretations were available, resulting in small sample size. Fifth, this study uses the term infiltrates based on the PALICC definition, which is increasingly being replaced with "opacifications" in the radiology literature as it is difficult to differentiate atelectasis from consolidation. Indeed, we identified a kappa of 0.16 between radiologists and intensivists regarding atelectasis. Future studies of PARDS should likely embrace "opacifications" instead of infiltrates. Sixth, sample sizes for subgroup analysis were small and likely inadequately powered for potentially clinically meaningful effects. Furthermore, we did not do inference testing to compare kappa values between groups of patients, so these analyses are only descriptive and should be interpreted with caution. Finally, there are many variables which will confound the relationship between bilateral infiltrates and mortality. We only evaluated oxygenation impairment in this manuscript but have previously shown that radiographic criteria do not retain an independent relationship with mortality after controlling for other factors such as immune-suppressed status and ARDS trigger [13, 30] .

In conclusion, in this large international study focused on real-world practitioners evaluating chest imaging in ARDS, we confirmed that the pediatric intensivist's interpretation of bilateral infiltrates or four quadrants of consolidation is associated with mortality only for patients with PF ratio (or equivalent) ≤ 100. This subgroup of patients is particularly high risk for mortality and may deserve special considerations in both clinical care and research. However, there is high inter-rater variability in the interpretation of bilateral infiltrates and quadrants of consolidation on CXRs. From our study, it is not possible to discern whether all patients with "unilateral" versus "bilateral" disease on CXR have the same pathobiology, which may increase the false positive rate of the PALICC definition. Balancing these issues of sensitivity and specificity should be a focus of the future research and definitions of ARDS.

The online version of this article (https ://doi.org/10.1007/s0013 4-020-06074 -7) contains supplementary material, which is available to authorized users.

Rosalba Pardo Carrero

Hospital Militar Central: Ledys María Izquierdo; Hospital Pablo Tobon Uribe (HPTU): Byron E. Piñeres Olave

Children's Hospital Bambino Gesu: Matteo Di Nardo/Fabrizio Chiusolo/Ilaria Erba/Orsola Gawronski; Children's Hospital Vittore Buzzi: Anna Camporesi. Japan. Hiroshima University: Nobuaki Shime/Shinichiro Ohshimo/ Yoshiko Kida/Michihito Kyo

Hospital Santa Maria -Centro Hospitalar Lisboa Norte: Cristina Camilo. Saudi Arabia. King Abdullah Specialist Children's Hospital, King Abdulaziz Medical City

Feinstein; Noah's Ark Children's Hospital for Wales: Siva Oruganti/Sara Harrison

Children's Hospital and Medical Center: Sidharth Mahapatra/Edward Truemper/Lucinda Kustka; Children's Hospital at Dartmouth: Sholeen T. Nett/Marcy Singleton/J. Dean Jarvis; Children's Hospital Colorado

Cincinnati Children's

Connecticut Children's Medical Center: Christopher L Carroll/James Santanelli; Inova Children's Hospital

Children's Hospital at Hackensack University Medical Center: Shira Gertz; Nicklaus Children's Hospital: Fernando Beltramo/Balagangadhar Totapally/Beatriz Govantes; Northwestern University, Ann & Robert H Lurie Children's Hospital of Chicago: Bria Coates/Lawren Wellisch/Kiona Allen/ Avani Shukla; Penn State Hershey Children's Hospital: Neal J. Thomas/Debbie Spear; Rainbow Babies and Children's Hospital, Steven L. Shein/Pauravi Vasavada; Saint Barnabas Medical Center: Shira Gertz; Stony Brook Children's Hospital: Margaret M. Parker/Daniel Sloniewsky

Department of Pediatric Critical Care Medicine, Hospital Pablo Tobón Uribe, Medellín, Colombia. 9 Department of Radiology, Children's Hospital of Los Angeles

Definitions, mechanisms, relevant outcomes, and clinical trial coordination

The Berlin definition of ARDS: an expanded rationale, justification, and supplementary material

Pediatric acute respiratory distress syndrome: definition, incidence, and epidemiology: proceedings from the Pediatric Acute Lung Injury Consensus Conference

Pediatric acute respiratory distress syndrome: consensus recommendations from the Pediatric Acute Lung Injury Consensus Conference

Interobserver variability in applying a radiographic definition for ARDS

Inter-observer variability in chest radiograph reading for diagnosing acute lung injury in children

Discrepancies in radiograph interpretation between pediatric radiologists and pediatric intensivists in the pediatric or neonatal intensive care unit

Interobserver variation in interpreting chest radiographs for the diagnosis of acute respiratory distress syndrome

Inter-observer variability for radiography in pediatric acute respiratory distress syndrome and improvement with a computer-aided diagnosis

Accuracy of the chest radiograph to identify bilateral pulmonary infiltrates consistent with the diagnosis of acute respiratory distress syndrome using computed tomography as reference standard

Acute lung injury/acute respiratory distress syndrome pathophysiology: what we have learned from computed tomography scanning

Frutos-Vivar F (2013) Comparison of the Berlin definition for acute respiratory distress syndrome with autopsy

Severity scoring of lung oedema on the chest radiograph is associated with clinical outcomes in ARDS

Paediatric acute respiratory distress syndrome incidence and epidemiology (PARDIE): an international, observational study

Pediatric Acute Lung Injury and Sepsis Network Investigators (PALISI) (2012) Comparison of SpO 2 to PaO 2 based markers of lung disease severity for children with acute lung injury

Defining acute lung disease in children with the oxygenation saturation index

Comparison of the pulse oximetric saturation/fraction of inspired oxygen ratio and the PaO 2 / fraction of inspired oxygen ratio in children

The measurement of observer agreement for categorical data

Measurement of observer agreement

An expanded definition of the adult respiratory distress syndrome

Acute lung injury in the medical ICU: comorbid conditions, age, etiology and hospital outcome

Identification of patients with acute lung injury. Predictors of mortality

Is there still a role for the lung injury score in the era of the Berlin definition ARDS?

The randomized educational acute respiratory distress syndrome diagnosis study: a trial to improve the radiographic diagnosis of acute respiratory distress syndrome

Does training improve diagnostic accuracy and inter-rater agreement in applying the Berlin radiographic definition of acute respiratory distress syndrome? A multicenter prospective study

Comparative diagnostic performances of auscultation, chest radiography, and lung ultrasonography in acute respiratory distress syndrome

Assessment of lung aeration and recruitment by CT scan and Ultrasound in acute respiratory distress syndrome patients

Impact of clinical history on chest radiograph interpretation

Predicting mortality in pediatric acute respiratory distress syndrome: a pediatric acute respiratory distress syndrome incidence and epidemiology (PARDIE) study

Funding was used to create the study database and support personnel at the clinical and data coordinating center. The funders had no role in data analysis, data interpretation, or writing of the report.

The authors declare in the conflict of interest statements reported in the manuscript.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.