key: cord-0949468-jx53ldn9 authors: Hu, Shasha; Zhu, Yongbei; Dong, Di; Wang, Bei; Zhou, Zuofu; Wang, Chi; Tian, Jie; Peng, Yun title: Chest Radiographs Using a Context-Fusion Convolution Neural Network (CNN): Can It Distinguish the Etiology of Community-Acquired Pneumonia (CAP) in Children? date: 2022-05-18 journal: J Digit Imaging DOI: 10.1007/s10278-021-00543-1 sha: d713513c6ac561ca91a1b8860cb05649cf09548f doc_id: 949468 cord_uid: jx53ldn9 Clinical symptoms and inflammatory markers cannot reliably distinguish the etiology of CAP, and chest radiographs have abundant information related with CAP. Hence, we developed a context-fusion convolution neural network (CNN) to explore the application of chest radiographs to distinguish the etiology of CAP in children. This retrospective study included 1769 cases of pediatric pneumonia (viral pneumonia, n = 487; bacterial pneumonia, n = 496; and mycoplasma pneumonia, n = 786). The chest radiographs of the first examination, C-reactive protein (CRP), and white blood cell (WBC) were collected for analysis. All patients were stochastically divided into training, validation, and test cohorts in a 7:1:2 ratio. Automatic lung segmentation and hand-crafted pneumonia lesion segmentation were performed, from which three image-based models including a full-lung model, a local-lesion model, and a context-fusion model were built; two clinical characteristics were used to build a clinical model, while a logistic regression model combined the best CNN model and two clinical characteristics. Our experiments showed that the context-fusion model which integrated the features of the full-lung and local-lesion had better performance than the full-lung model and local-lesion model. The context-fusion model had area under curves of 0.86, 0.88, and 0.93 in identifying viral, bacterial, and mycoplasma pneumonia on the test cohort respectively. The addition of clinical characteristics to the context-fusion model obtained slight improvement. Mycoplasma pneumonia was more easily identified compared with the other two types. Using chest radiographs, we developed a context-fusion CNN model with good performance for noninvasively diagnosing the etiology of community-acquired pneumonia in children, which would help improve early diagnosis and treatment. Background Pneumonia is an acute infection of the lung parenchyma by one or more pathogens, such as viruses, bacteria, and mycoplasma [1] . According to the World Health Organization (WHO), the incidence in children under five is evaluated to be 0.05 episodes per child-year in developed countries and 0.29 episodes per child-year in developing countries. This means there are about 151 million new cases in children that occur annually in developing countries, including 21 million in China [2] . Community-acquired pneumonia (CAP) has high morbidity and mortality rates in both developed and developing countries [3] . It is reported that pneumonia and preterm birth complications are the principal causes of death in children under five [4] . Viruses, bacteria, and mycoplasma are the common etiologies of pneumonia, but they need different medications and treatments. Viral pneumonia is treated with supportive care, bacterial pneumonia requires immediate antibiotic therapy (Penicillin G and amoxicillin), and macrolides are often used in the treatment of mycoplasma pneumonia. The empirical use of antibiotics remains fundamental to the treatment of pneumonia [5] . Untimely diagnosis of pathogens can lead to overuse of antibiotics and the formation of drug resistance [6, 7] . Delayed diagnosis increases the risk of irreversible damage to the patient's respiratory system. Therefore, accurate and timely diagnosis is the key to ensuring the most effective treatment. In clinical practice, most diagnoses of CAP are based on radiology, clinical signs, and symptoms. In most patients with respiratory symptoms, a chest radiograph is the first choice [8] ; it is also considered as the clinical reference standard for pneumonia [9] . C-reactive protein (CRP) and white blood cell (WBC) count are the most commonly used markers of inflammation in clinical practice, which are used to evaluate children suspected of pneumonia [10, 11] . However, the diagnostic challenge of childhood CAP is that although clinical symptoms and inflammatory markers and radiological signs are indicative of pathogens [12] [13] [14] , pneumonia pathogens cannot be reliably distinguished [3, 5, [15] [16] [17] [18] . Sputum culture, multiplex polymerase chain reaction, or specific mycoplasma antibody tests can diagnose pneumonia [19, 20] but have some limitations. To assist pediatricians and improve their diagnostic performance, computer-aided diagnostic systems, e.g., deep learning or radiomics, have been increasingly utilized in radiology and medical image analysis [21] [22] [23] [24] . Recently, it has been found that deep learning technology combined with big data have a better detection effect in chest image diagnosis [25] [26] [27] , including detecting the presence and detection of pneumonia [28, 29] , emphysema and quantification [30] , and pneumothorax [31, 32] . In this study, we first investigated the clinical characteristics and imaging biomarkers that were often used in chest radiograph analysis. Further analysis showed that the characteristics and biomarkers with different etiologies of children with CAP were significantly different, and the intensity-related biomarkers from lung and lesion areas have a complementary presentation ability. Despite the statistical significance, the scatter plot of the biomarkers showed broad overlap between three classes (as shown in the "Results"). Hence, we improved the diagnosis ability of chest radiographs using deep learning methods, and we built a pediatric chest radiograph dataset and then developed the prediction models based on chest radiographs and clinical characteristics. The diagnostic ability of a full-lung image, local-lesion image, and clinical characteristics were evaluated. The complementarity ability between a full-lung image and locallesion image and between image CNN signature and clinical characteristics were explored. The dataset was approved by our ethics committee of Beijing children's hospital, where the requirement of informed consent was waived. The retrospective analysis of 1769 (viral pneumonia 487, bacterial pneumonia 496, and mycoplasma pneumonia 786) cases of childhood CAP was confirmed in our hospital between 2015 and 2018 including chest radiographs of the first examination, while CRP was measured and WBC count was determined, excluding patients with poor image quality and incomplete clinical data. The chest radiograph dataset only contained patients from newborns to 18-year-olds, and most of them were children under 10 years old. We included both posteroanterior and bedside anteroposterior chest radiographs, where all radiographs were obtained with a single dedicated radiography unit (DR7500; Kodak Healthineers). The patient recruitment procedure and workflow of this study are shown in Fig. 1 . The patient demographics for the dataset and clinical information are shown in Table 1 . We hypothesized that the chest radiograph appearance influenced by different etiologies may be reflected in the intensity of the radiograph. Thus, we investigated the intensity-related biomarkers from the chest radiograph image analysis, as shown in Fig. 2A. (1) Mean intensity of lung and lesion areas: The mean value of the pixel intensity was calculated from the segmented lung and lesion areas, respectively. (2) Standard deviation of the intensity of lung and lesion areas: The standard deviation (STD) was calculated from the intensity histogram of the lung and lesion area pixels, respectively. The patient cohort was stochastically divided into training, validation, and test cohorts in a 7:1:2 ratio. The training cohort was used to optimize the CNN. During training, the results of the validation cohort were used to select the optimal model. The test cohort was not disclosed until the model was finalized. In order to avoid overfitting, data augmentation techniques including random rotations, translations, and flipping were used and further increased the size of the training cohort. The prediction ability of the full-lung image, local-lesion image, and clinical characteristics were studied and the prediction performances for the three etiologies are shown (Fig. 2B ). Routinely-used chest radiographs include some non-lung areas (neck, abdomen, bone, etc.) and blank spaces outside the body. To ensure the consistency of the exposure field, we first segmented lungs from the chest radiographs using an automatic segmentor, XLSor [33] , which is an end-toend convolutional model that can perform robust and accurate lung segmentation. The chest radiographs were input into XLSor, and the lung mask annotations were output automatically according to the results of the segmentation. The lung region was cropped automatically, which was a lung region of interest (ROI). The pneumonia lesion region was drawn manually using ITK-SNAP software (version 3.2.0; www. itksn ap. org) by two board-certified radiologists who were blinded to the histological diagnoses and patient clinical information. Their primary focus was hand-crafted segmentation of pneumonia lesions; in case of disagreement between the two radiologists, consensus was reached through discussion. The lesion region was cropped automatically according to the bounding box derived from manual segmentation, and we call this partial lesion region a lesion ROI. In order to match the neural network inputs, all lung ROIs were resized to a size of 224 × 224 and all lesion ROIs were resized to a size of 160 × 160. Image intensities were normalized using histogram equalization. Figure 3B shows the process of lung ROI and lesion ROI acquisition. Both the lung ROI and lesion ROI generated from the chest radiographs were analyzed, and two clinical characteristics including WBC count and CRP were used to build a clinical model. In clinical diagnosis, a radiologist usually makes a comprehensive analysis about the entire lungs and local lesions. To provide this contextual information, we developed an end-to-end CNN model (context-fusion CNN). Overall, based on the chest radiograph, we built three models including a full-lung model, a local-lesion model, and a context-fusion model; based on clinical characteristics, we built a clinical model. We then combined the signature of the context-fusion model with two clinical characteristics to study the complementarity between the radiograph and characteristics. In our study, the CNN models were derived from DenseNet121 [34] , which consisted of a densely connected CNN feature extractor and a classifier. The full-lung model and local-lesion model were the standard Densenet. The Fig. 2 The process of study for distinguishing the etiology of children with CAP using chest radiographs: A segmentation network and intensity-related biomarker analysis and B classification models based on local-lesion image, full-lung image and clinical characters, and the prediction performances for the three etiologies inputs of the full-lung model were the lung ROIs with a size of 224 × 224, and the outputs of the feature extractor were full-lung CNN features. Similar to the full-lung model, the inputs of the local-lesion model were the lesion ROIs with a size of 160 × 160 and the outputs of the feature extractor were local-lesion CNN features. A clinical model based on the clinical characteristics of the WBC count and CRP was built, which used the Catboost machine learning method [35] . The context-fusion CNN simultaneously extracted the lung features and lesion features with a two-branch CNN, and two branches did not share the weights for extracting different features from the lung and local lesion. Furthermore, the feature fusion module merged two features as context features with a concatenation operation and decreased the redundancy features between lung and local branch with a fully connected layer (the number of input nodes was much smaller than the number of output nodes). Finally, another fully connected layer was used as a classifier for predicting the etiology of CAP. Figure 3A shows the CNN structure and details of the context-fusion model. The full-lung model, local-lesion model, and contextfusion model were trained based on the Pytorch platform and optimized via an Adam algorithm with a mini-batch size of 32. The learning rate was set to 0.001 with a momentum coefficient of 0.9. The weights of CNN were initialized stochastically. The clinical model was trained based on the Catboost platform, the learning rate was set to 0.25, and the depth was 2. The optimal model of every model was the one with the lowest validation loss during training. The performance of models was evaluated by assessing the accuracy (ACC) of training and test cohorts. In addition, softmax or logistic regression probabilities were used to calculate ACC, precision, recall, area under curve (AUC) of the ROC analysis, sensitivity, and specificity. Statistical analysis was conducted with a Python toolkit (scipy.stats). Chi2 test and Kruskal-Wallis test were used. A two-sided P value < 0.05 was used to indicate statistical significance. Wilcoxon rank sum test was used to compare chest radiograph markers among three classes with different data sizes. Clinical characteristics consist of sex, age, CRP, and WBC count are reported in Table 1 . CRP and WBC count were significantly associated with the etiology of CAP after the univariate analysis (p < 0.05), and there was a significant difference in sex and age. Generally, a virus is the most common etiology of CAP in infants and young children, Fig. 3 Overall architecture of the proposed neural network approach: A the CNN structure of the context-fusion model and B the process of lung ROI and lesion ROI acquisition with mycoplasma occurring in children over the age of 5 [36] . Although mycoplasma pneumonia was included in this study according to the age of viral pneumonia and bacterial pneumonia, children with mycoplasma pneumonia were still older than those with viral and bacterial pneumonia. Stratified analyses for the subgroups were classified according to sex. Specifically, man and woman subgroups yielded AUC values of 0.844 and 0.816 for viruses, 0.850 and 0.874 for bacteria, and 0.918 and 0.908 for mycoplasma, respectively (Delong test p-values: 0.220, 0.231, and 0.260). The results of the above stratified analyses indicated that our model was not affected by sex. Four intensity-related biomarkers from the CXR image were extracted and analyzed. Mean pixel intensity of each lung and lesion area is shown in the scatter plot of Figs. 4A and 5A. In both areas, mycoplasma cases showed higher mean intensity compared to other cases with a statistical significance level (p < 0.001 for all). In the lesion area, the difference between viral and bacterial cases was statistically significant (p < 0.01). The STDs of pixel intensity of each lung and lesion areas are scattered in the plot in Figs. 4B and 5B. Fig. 4 Scatter and box plots for both the mean pixel intensity (left) and the STD of the pixel intensity from lung areas (right). Blue triangles in box plots show mean values, and statistical significance levels are indicated as asterisks; *p < 0.05, **p < 0.01, and ***p < 0.001 Scatter and box plots for both the mean pixel intensity (left) and the STD of pixel intensity from lesion areas (right). Blue triangles in box plots show mean values and statistical significance levels are indicated as asterisks; *p < 0.05, **p < 0.01, and ***p < 0.001 In the lesion area, the variance values of the viral cases were higher than other classes with a statistical significance (p < 0.001 for all). Tables 2, 3, 4, 5 describe the corresponding statistical results. Despite statistical significance, the scatter plots of the biomarkers showed broad overlap between three classes. Statistical significance levels are indicated as asterisks; * for p < 0.05, ** for p < 0.01, and *** for p < 0.001. The context-fusion model yielded the best performance with an ACC of 0.70 in the training cohort and 0.72 in the test cohort. Figure 6 shows the prediction distribution of patients in the three pneumonia types. It illustrated that the prognosis for the etiology of CAP needs a comprehensive analysis about the entire lungs and local lesions. The clinical model had poor diagnostic performance with an ACC of 0.60 in the training cohort and 0.51 in the test cohort. Compared with the local-lesion model, the full-lung model yielded better performance with an ACC of 0.65 in the training cohort and 0.62 in the test cohort. The results in the training and test cohorts are provided in Table 6 . Fig. 6 The prediction distribution of patients for the three pneumonia types The prediction performances for every etiology of CAP were consistent with the overall performances, and the contextfusion model yielded the best performance and was significantly higher than the other models in the training cohort and test cohort. Mycoplasma is easily diagnosed with an AUC of 0.911 in the training cohort and 0.924 in the test cohort. The diagnostic ability for viruses and bacteria was equal. Table 7 shows the results of the three pneumonia types, and the ROCs are shown in Fig. 7 . Early diagnosis of pneumonia is critical to prevent complications including death. In this retrospective research, we built deep learning prediction models and systematically analyzed the ability of chest radiographs and clinical characteristics for distinguishing the etiology of CAP in children. A context-fusion model combined full-lung features and locallesion features showing good performance, and there was complementarity between the image CNN signature built from the context-fusion model and clinical characteristics. The adopted models can potentially improve the diagnostic speed and accuracy in a non-invasive way. At first, we analyzed the clinical characteristics and intensity-related biomarkers from chest radiographs. A positive correlation was found between the etiology of childhood CAP and both the clinical characteristics and chest radiographs biomarkers. The results were consistent with the study of Oh et al. [37] , and they described that the mean intensity and variance values of viral pneumonia were higher than bacterial pneumonia or tuberculosis cases with statistical significance (p < 0.001 for all). Despite statistical significance, the scatter plot of the biomarkers showed a broad overlap between several classes. In addition, the presentation ability of the biomarkers from lung and lesion areas was complementary. Hence, we built a diagnostic model using deep learning methods and explored the complementarity ability between the full-lung image and local-lesion image. In our study, deep learning methods were used to mine valuable features from chest radiographs for differentiating the etiology of pediatric pneumonia. Some studies compared a variety of algorithms to obtain the most optimal model [8, 29] . Deep learning is a promising technique for analyzing medical imaging, and some research in chest radiograph analyses achieved excellent performance. For example, in some studies [38, 39] , they built deep learning models to diagnose chest pathology in chest radiographs and the models were competitive with radiologists on some pathology. Some researchers proposed more effective models to localize diseases using limited location annotations [40, 41] . Our results are consistent with the above study. Our context-fusion model was derived from Densenet and adopted a dual-path construction to combine the full-lung and local-lesion features from the patients' chest radiograph, which proved valuable. The full-lung and local-lesion regions of the chest radiographs were analyzed separately, and the diagnostic ability of the local-lesion model was lower than that of the full-lung model. The reasons may be as follows: infection via different pathogens may lead to inflammation of the lobes, bronchi, alveoli, or interstitial lung areas, as well as bronchiolitis. The lesions vary in size and often permeate the whole lung, presenting on chest radiographs as whole-lung or partial pulmonary consolidation, bronchogenic inflation, pleural effusion, and interstitial infiltration [42] , while the lesion that we chose to delineate on the chest radiograph was in the most severe areas, not the entire lung; secondly, limited by spatial resolution, a chest radiograph can directly show the trachea and large bronchus as well as related lesions, but cannot directly show the bronchioles and their lesions. When lesions occur in this part of the airway, the chest radiograph can show increased brightness and increased lung volume, which is not delineated; thirdly, we chose to depict the lesions by radiologists with diagnostic experience, which is subjective, and the full-lung model included the entire lung that was objectively unaffected by doctors. The context-fusion model had the highest diagnostic accuracy, indicating that not only the lesion delineation area, but also the entire chest radiograph had visual changes that were difficult to recognize visually due to pathological changes in pneumonia. The pathological difference of different etiologies of CAP will affect the appearance of the chest radiograph. Viral pneumonia can be seen as a large lobe or multiple Fig. 7 The receiver operating characteristic curves of A context-fusion model, B full-lung model, C local-lesion model, and D clinical model focal infiltrates, typical bacterial pneumonia is usually lobar pneumonia with pleural effusion [43] , and both are indistinguishable on chest radiographs. The pathology of mycoplasma pneumonia is usually confined to the airway wall, and even small airways and respiratory bronchioles and chest radiographs showed peri-bronchial infiltration, reticular nodules, and patchy and focal consolidation [44, 45] . In our study, the diagnostic ability of the models based on chest radiographs was higher than the clinical model. In other published studies, WBC count and CRP could also be found to indicate the presence of pneumonia, but they did not play a significant role in the pathogen determination of pneumonia [46] , which was also consistent with this study. A conjoint analysis regarding the clinical characteristics and image signature built from the context-fusion model was conducted using logistic regression; the results showed that there was good complementarity between the image signature and clinical features. As can be seen from Table 3 , the AUC of the context-fusion model for diagnosing viral pneumonia, bacterial pneumonia, and mycoplasma pneumonia was 0.851, 0.876, and 0.924, respectively; the sensitivity was 0.776, 0.702, and 0.834; and the specificity was 0.793, 0.902, and 0.865, respectively. Mycoplasma pneumonia is more easily identified from the three, followed by bacterial pneumonia and viral pneumonia. Bacterial pneumonia and viral pneumonia are more difficult to distinguish, but the clinical indicators of the differential efficacy are relatively low with AUC values being lower than 0.6. Diagnosing the etiology of CAP in children via chest radiographs is practical in our study, since the local lesion may retain critical features of pneumonia and the lung region contains global information. The combination of global and local information contributes to diagnosing the etiology of CAP in children, which is significantly effective. It is promising for future research to extend its application to more tasks, such as the diagnosis between common viruses and coronavirus disease 2019. There are also some limitations in this study. Our chest radiograph was the first chest radiograph of the patient admitted to the hospital. The patient had already had fever, cough, and other clinical manifestations before admission, so it is uncertain if the patient had pneumonia for several days when the chest radiograph was collected and the prognosis of the patient was not tracked further. Secondly, this study only limited pneumonia to three categories, which did not involve tuberculosis or fungal pneumonia, and did not make a more specific classification of bacterial or viral pneumonia. Finally, the clinical indicators we collected were relatively single, and it would be more meaningful if procalcitonin or percentage of neutrophils were added. This study provides a deep learning-based etiological prediction technology for community-acquired pneumonia, which can realize the early diagnosis of CAP in children in a noninvasive and rapid manner, and is of great significance for guiding clinical medication and reducing the mortality rate of childhood pneumonia. The data that support the findings of this study are available from the corresponding author upon reasonable request. The authors declare no competing interests. The definition and classification of pneumonia. Pneumonia (Nathan Qld) Epidemiology and etiology of childhood pneumonia Community-Acquired Pneumonia in Children: the Challenges of Microbiological Diagnosis Global, regional, and national causes of under-5 mortality in 2000-15: an updated systematic analysis with implications for the Sustainable Development Goals Combination of clinical symptoms and blood biomarkers can improve discrimination between bacterial or viral community-acquired pneumonia in children.BMC pulmonary medicine Introducing a New Algorithm for Classification of Etiology in Studies on Pediatric Pneumonia: Protocol for the Trial of Respiratory Infections in Children for Enhanced Diagnostics Study.JMIR research protocols Acute Uncomplicated Febrile Illness in Children Aged 2-59 months in Zanzibar -Aetiologies, Antibiotic Treatment and Outcome A transfer learning method with deep residual network for pediatric pneumonia diagnosis Evaluation of the World Health Organization criteria for chest radiographs for pneumonia diagnosis in children Finnish guidelines for the treatment of community-acquired pneumonia and pertussis in children Community-Acquired Pneumonia in Children Measurement of lipocalin-2 and syndecan-4 levels to differentiate bacterial from viral infection in children with community-acquired pneumonia Utility of serum procalcitonin and C-reactive protein in severity assessment of community-acquired pneumonia in children Community-Acquired Pneumonia in Children: Myths and Facts Differentiation between mycoplasma and viral community-acquired pneumonia in children with lobe or multi foci infiltration: a retrospective case study Utility of inflammatory markers in predicting the aetiology of pneumonia in children. Diagnostic microbiology and infectious disease The value of clinical features in differentiating between viral, pneumococcal and atypical bacterial pneumonia in children Improved diagnostics help to identify clinical features and biomarkers that predict Mycoplasma pneumoniae community-acquired pneumonia in children The use of multiplex PCR for the diagnosis of viral severe acute respiratory infection in children: a high rate of co-detection during the winter season. European journal of clinical microbiology & infectious diseases : official publication of the Community-Acquired Pneumonia Caused by Mycoplasma pneumoniae: How Physical and Radiological Examination Contribute to Successful Diagnosis Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks Computed tomography-based predictive nomogram for differentiating primary progressive pulmonary tuberculosis from community-acquired pneumonia in children Development and validation of a novel MR imaging predictor of response to induction chemotherapy in locoregionally advanced nasopharyngeal cancer: a randomized controlled trial substudy (NCT01245959) Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study Deep Learning for Chest Radiography in the Emergency Department Deep learning in chest radiography: Detection of findings and presence of change Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists Using deep-learning techniques for pulmonary-thoracic segmentations and improvement of pneumonia diagnosis in pediatric chest radiographs An Efficient Deep Learning Approach to Pneumonia Classification in Healthcare Application of deep learning-based computeraided detection system: detecting pneumothorax on chest radiograph after biopsy Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: A retrospective study. International journal of environmental research and public health Xlsor: A robust and accurate lung segmentor on chest x-rays using criss-cross attention and customized radiorealistic abnormalities generation Densely connected convolutional networks CatBoost: unbiased boosting with categorical features Deep Learning for Quantification of Epicardial and Thoracic Adipose Tissue From Non-Contrast CT Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets Tienet: Text-image embedding network for common thorax disease classification and reporting in chest x-rays Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison Attend and Locate: Chest X-ray Diagnosis via Contrast Induced Attention Network with Limited Supervision Learning fixed points in generative adversarial networks: From image-to-image translation to disease detection and localization Variability in the interpretation of chest radiographs for the diagnosis of pneumonia in children Typical Bacterial Pneumonia. StatPearls. Treasure Island (FL): StatPearls Publishing LLC Correlation between chest radiographic findings and clinical features in hospitalized children with Mycoplasma pneumoniae pneumonia Correlation between Radiological and Pathological Findings in Patients with Mycoplasma pneumoniae Pneumonia Clinical features and inflammatory markers in pediatric pneumonia: a prospective study Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations