key: cord-1042485-p1hm264z authors: Wang, Zhiyi; Weng, Jie; Li, Zhongwang; Hou, Ruonan; Zhou, Lebin; Ye, Hua; Chen, Ying; Yang, Ting; Chen, Daqing; Wang, Liang; Liu, Xiaodong; Shen, Xian; Jin, Shengwei title: Development and Validation of a Diagnostic Nomogram to Predict COVID-19 Pneumonia date: 2020-04-06 journal: nan DOI: 10.1101/2020.04.03.20052068 sha: e7869e78d3208ab3208ad27210b973fb0f11e447 doc_id: 1042485 cord_uid: p1hm264z Background: The COVID-19 virus is an emerging virus rapidly spread worldwide This study aimed to establish an effective diagnostic nomogram for suspected COVID-19 pneumonia patients. METHODS: We used the LASSO aggression and multivariable logistic regression methods to explore the predictive factors associated with COVID-19 pneumonia, and established the diagnostic nomogram for COVID-19 pneumonia using multivariable regression. This diagnostic nomogram was assessed by the internal and external validation data set. Further, we plotted decision curves and clinical impact curve to evaluate the clinical usefulness of this diagnostic nomogram. RESULTS: The predictive factors including the epidemiological history, wedge-shaped or fan-shaped lesion parallel to or near the pleura, bilateral lower lobes, ground glass opacities, crazy paving pattern and white blood cell (WBC) count were contained in the nomogram. In the primary cohort, the C-statistic for predicting the probability of the COVID-19 pneumonia was 0.967, even higher than the C-statistic (0.961) in initial viral nucleic acid nomogram which was established using the univariable regression. The C-statistic was 0.848 in external validation cohort. Good calibration curves were observed for the prediction probability in the internal validation and external validation cohort. The nomogram both performed well in terms of discrimination and calibration. Moreover, decision curve and clinical impact curve were also beneficial for COVID-19 pneumonia patients. CONCLUSION: Our nomogram can be used to predict COVID-19 pneumonia accurately and favourably. Corona Virus Disease 2019 pneumonia is confirmed to be infected with a novel coronavirus which is a β coronavirus that belongs to the family Coronaviridae. It has spread rapidly throughout Wuhan (Hubei province) to other provinces in China and around the world. 1,2 On January 30, 2020, the World Health Organization declared COVID-19 was a public health emergency of international concern (PHEIC). As of March 13, 2020, a total of 51767 laboratory-confirmed patients and 1775 deaths have been documented outside China. This pandemic has been disastrous for people all over the world, and many countries are failing to control its spread. The main reason was lack of rapid response which is depend on very early detection and diagnosis 3 . As with all infectious diseases, the early and reliable diagnosis is key to block COVID-19 transmission. However, COVID-19 pneumonia has a wide range of clinical . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint manifestations, such as fever, cough, fatigue, pharyngeal pain, etc 4 , Many patients have mild symptoms or asymptomatic in the early stage. Due to lack of specific clinical symptoms and signs, it is easy to be ignored. 5, 6 Real-time polymerase chain reaction (PCR) has become an important tool in the diagnosis of many infectious diseases 7, 8 . However, it has several disadvantages, including long detection time, cumbersome steps, and high cost. Even the sensitivity of the COVID-19 nucleic acid detection is low. In addition, COVID-19 nucleic acid detection kits are still insufficient in many epidemic countries now. Detection of viral nucleic acid may not be the only ideal diagnostic method during the early stages of this epidemic outbreak. Given the rapid spread of COVID-19 and low detection rate by pharyngeal swab COVID-19 nucleic acid test. It is also believed that throat swab samples are not suitable for the detection of SARS coronavirus RNA 9 . The development of a diagnostic method with decreased complexity and expense are urgently needed to facilitate timely intervention. 10 Previous studies with significantly large sample sizes have been done to delineate the epidemiological and clinical characteristics of COVID-19 pneumonia. 11 Although some studies have shown that lung computed tomographic (CT) plays an important role in early diagnosis 12, 13 . However, there is still a lack of systematic and standard imaging diagnostic criteria for COVID-19 pneumonia. To this end, 294 suspected COVID-19 pneumonia patients were included in our study. Trying to establish a diagnosis model to quickly identify COVID-19 pneumonia from suspected COVID-19 pneumonia. Two cohorts of adult suspected COVID-19 pneumonia patients were included in this retrospective study. The patients in primary cohort were from the Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University (Wenzhou, China), and the patients in validation cohort were from the People's Hospital of Yueqing (Yueqing, China). We relied on new coronavirus pneumonia control and prevention plan (trial version 6) to identify suspected COVID-19 pneumonia patients. Suspected diagnostic criteria are as follows:1) epidemiological history. 2) fever and / or respiratory . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Demographic data, epidemiological history, comorbidity, vital signs, clinical symptoms, laboratory indicators including WBC count, neutrophils count, lymphocyte count, hemoglobin, platelet and C-reactive protein (CRP) and chest CT imaging features were collected. The data of laboratory indicators and chest CT imaging features were the first recorded data. All chest CT images features in two cohorts were analyzed by four chest radiologists (two radiologists for one cohort) with least 10 years of experience in chest CT imaging and all decisions were reached by consensus. CT images features mainly include four parts: (1) lesion distribution, such as bilateral lower lobes, multiple lobes, periphery distribution (2) lesion patterns, such as patchy or large patchy distribution, wedge-shaped or fan-shaped lesion parallel to or near the pleura, crazy paving pattern, (3) lesion density, such as ground glass opacities, consolidation, cavitation, (4) other signs in the lesion, such as lung nodule, nodules halo sign, subpleural nodules, centrilobular nodules, other nodules, pleural effusion, air bronchogram sign, bronchiectasis, fibrotic proliferation, bronchial wall thickening, tree-in-bud pattern, white lung (Figure 1 ). . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint Data were presented as mean ± standard deviation (SD) and median (IQR) for continuous variables with normal or non-normal distribution and the frequency (proportion) for categorical variables. The Student t-test, Mann-Whitney U test, chisquared test, or Fisher's exact test was performed where appropriate. For the development of the nomograms, the least absolute shrinkage and selection operator (LASSO) method was performed to identify potential significant predictors from the primary cohort. Predictive variables that were considered clinically relevant (based on our clinical experience and literature report), and that showed statistical relationship in LASSO method were entered into multivariate logistic regression model 14 . The Variance Inflation Factor (VIF) was used to identify variables collinearity before the model estimation. According to the Akaike information criterion (AIC) and clinically relevant variables, we established the final multivariable models. The 'rms' package was used for nomogram and calibration curve 15 . The accuracy of the . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint nomogram to predict the COVID-19 pneumonia were quantified using C-statistic, the calibration of the model is assessed by the calibration curves in the primary cohort and validation cohort. Moreover, we performed decision curve analysis (DCA) and clinical impact curve by quantifying the net benefits and cost benefit ratio to assess the clinical value of the model 16 . We did the statistical analyses and figures production using R software (version 3.6.1). All statistical tests were two-sided, differences of P < 0.05 were considered statistically significant. The Table 1 . Hypertension, n (%) < 0.001* . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint Other features, n (%) < 0.001* . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint consolidation, pleural effusion, air bronchogram sign and fibrotic proliferation) were included in the LASSO regression analysis (Figure 2) . The results showed that gender, epidemiological history, neutrophils count, bilateral lower lobes, periphery distribution, wedge-shaped or fan-shaped lesion parallel to or near the pleura, ground glass opacities obtained from the primary cohort were predictive factors for COVID-19 pneumonia. According to the results of collinearity test and clinical practice, the epidemiological history, wedge-shaped or fan-shaped lesion parallel to or near the pleura, bilateral lower lobes, ground glass opacities, crazy paving pattern and WBC count were included in the final multivariate logistic regression analyses. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint In this study, we developed and validated a novel nomogram to predict COVID-19 infection among patients who were suspected viral pneumonia. This diagnostic nomogram mainly relies on CT findings. Our study found that COVID-19 pneumonia is not significantly different from other suspected viral pneumonia in clinical symptoms and signs. There was also no significant difference in blood routine test, liver and kidney function test. Thus, according to clinical symptoms and signs and laboratory examinations, the COVID-19 pneumonia is difficult to distinguish from other viral pneumonia. However, they have similar and different manifestations on lung imaging detected by CT scan. 10, 17 To this end, we have established this nomogram mainly based on lung imaging. All CT findings were analyzed from three aspects including distribution characteristics, morphology and density of pulmonary inflammation lesions. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint The imaging features of viral pneumonia usually appear as multifocal ground glass opacities which correspond to pathological diffuse alveolar damage. 18 According to univariate analysis, ground glass opacities, crazy paving pattern, a wedge-shaped or fan-shaped lesion parallel to or near the pleura, the distribution characteristics of bilateral lower lobes and peripheral distribution of lesions are the characteristic imaging manifestation of COVID-19 pneumonia. But, the crazy paving pattern on the basis of multivariable was unassociated with COVID-19 pneumonia, which may be due to other potential confounding factors. 19 But this does not mean that crazy paving pattern in COVID-19 pneumonia are unimportant. In addition, many studies have shown that the crazy paving pattern formed by interlobular septal thickening which was regarded as a typical imaging of viral pneumonia 13 . Therefore, we kept this factor in our model development. Another important imaging feature related to the characteristics of morphology and distribution, the wedge-shaped or fan-shaped lesion parallel to or near the pleura, which actually includes peripheral distribution of lesions. Because of their strong collinearity, it will seriously affect the accuracy of our research results 20 . Therefore, the imaging features of peripheral distribution was removed. Finally, the nomogram incorporates 4 items of the imaging features, epidemiological contact history and WBC count status. Nomogram is a visualization of regression analysis, which is more and more widely used in clinical disease diagnosis, prognosis evaluation and efficacy evaluation [23] [24] [25] [26] . Our results show that the nomogram based on imaging features has good sensitivity and specificity in the diagnosis of COVID-19 pneumonia. Moreover, its discrimination for COVID-19 pneumonia is . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint better than the first detection of viral nucleic acid. If there is a lack of virus nucleic acid test kit, COVID-19 pneumonia can be determined by lung CT preferentially. In order to prove the calibration of the nomogram, clinical data was collected from different institutions. As is well known, the internal validity associated with the explanation of the results, and the external validity related to the generalizability of the results 27, 28 . Through the internal and external validation data set analysis, the calibration of our nomogram has been proved to be highly consistent. This means that our nomogram may be popularized and applied widely in other hospital. However, to evaluate its clinical usefulness, it depends on how much it benefits the patient, not just its popularization 29 . DCA is an novel method 30,31 , it offers insight into clinical consequences on the basis of threshold probability, from which the net benefit could be derived 32 . The DCA showed that if we choose to diagnose COVID-19 pneumonia with a 60% threshold probability, 40 out of every 100 people will benefit. Our study has several limitations. Firstly, only 178 patients were included in primary cohort and another hospital was selected for external validation (116 patients). Whether this nomogram is applicable to patients with other areas background is still unclear. A large number of patients as data need to be collected to verify its clinical application. Secondly, this nomogram is mainly used to identify COVID-19 pneumonia in the patients with suspected viral pneumonia, not all types of pneumonia. Although the decrease of lymphocyte count is more common among COVID-19 pneumonia, not observed in our study. It may be related to our inclusion criteria. In conclusion, this study presents a novel nomogram that incorporates both the imaging features, epidemiological history and WBC. It can predict COVID-19 pneumonia conveniently and accurately. Using this nomogram has high net benefit for patients with suspected COVID-19 infection. None. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not peer-reviewed) The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint The Novel Coronavirus Originating in Wuhan, China: Challenges for Global Health Governance A Novel Coronavirus from Patients with Pneumonia in China Health system capacity in Sydney, Australia in the event of a biological attack with smallpox Clinical characteristics of novel coronavirus cases in tertiary hospitals in Hubei Province Clinical features of patients infected with 2019 novel coronavirus in Wuhan Clinical findings in a group of patients infected with the 2019 novel coronavirus (SARS-Cov-2) outside of Wuhan, China: retrospective case series New Method for Differentiation of Granuloviruses (Betabaculoviruses) Based on Real-Time Polymerase Chain Reaction (Real-Time PCR) An updated roadmap for MERS-CoV research and product development: focus on diagnostics Development and evaluation of a novel loop-mediated isothermal amplification method for rapid detection of severe acute respiratory syndrome coronavirus Clinical and CT features in pediatric patients with COVID-19 infection: Different points from adults Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-Infected Pneumonia Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): a study of 63 patients in Wuhan, China International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not peer-reviewed) The copyright holder for this preprint Imaging and clinical features of patients with 2019 novel coronavirus SARS-CoV-2 Design and Application of a Variable Selection Method for Multilayer Perceptron Neural Network With LASSO External validation of a Cox prognostic model: principles and methods A simple, step-by-step guide to interpreting decision curve analysis Radiographic and CT Features of Viral Pneumonia Herpes simplex virus pneumonia: high-resolution CT findings Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement Collinearity and multivariable analysis Clinical characteristics and therapeutic procedure for four cases with 2019 novel coronavirus pneumonia receiving combined Chinese and Western medicine treatment Evaluation of cell population data on the UniCel DxH 800 Coulter Cellular Analysis system as a screening for viral infection in children Soft tissue sarcoma nomograms and their incorporation into practice Establishment and Verification of Prognostic Nomograms for Patients with Gastrointestinal Stromal Tumors: A SEER-Based Study Novel Prognostic Nomograms Based on Inflammation-Related Markers for Patients with Hepatocellular Carcinoma Underwent Hepatectomy Prognostic nomograms for predicting cause-specific survival and overall survival of stage I-III colon cancer patients: a large population-based study Development and external validation of new nomograms by adding ECG changes (ST depression or tall T wave) and age to conventional scoring systems to improve the predictive capacity in patients with subarachnoid haemorrhage: a retrospective, observational study in Korea Making Health Research Matter: A Call to Increase Attention to External Validity Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer The author(s) indicated no potential conflict of interest. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint . https://doi.org/10.1101/2020.04.03.20052068 doi: medRxiv preprint