key: cord-0883820-1smr5jh9
authors: Mirijello, Antonio; Zarrelli, Michele; Miscio, Giuseppe; de Matthaeis, Angela; Piscitelli, Pamela; Carbonelli, Cristiano Matteo; Di Giorgio, Annabella; Inglese, Michele; Ciliberti, Gianluca Libero; Marciano, Carmen; Borelli, Cristina; Vergara, Doriana; Castorani, Giulia; Orciulo, Grazia Vittoria; Di Mauro, Lazzaro; Carella, Massimo; Simeone, Annalisa; Copetti, Massimiliano; Leone, Maurizio Angelo; De Cosmo, Salvatore
title: Diagnosis of COVID-19 in Patients with Negative Nasopharyngeal Swabs: Reliability of Radiological and Clinical Diagnosis and Accuracy versus Serology
date: 2021-02-25
journal: Diagnostics (Basel)
DOI: 10.3390/diagnostics11030386
sha: 27d3d379c02b5c32b6270d67ab1d1ed0ab85887e
doc_id: 883820
cord_uid: 1smr5jh9

Background: The diagnosis of Coronavirus disease 2019 (COVID-19) relies on the positivity of nasopharyngeal swab. However, a significant percentage of symptomatic patients may test negative. We evaluated the reliability of COVID-19 diagnosis made by radiologists and clinicians and its accuracy versus serology in a sample of patients hospitalized for suspected COVID-19 with multiple negative swabs. Methods: Admission chest CT-scans and clinical records of swab-negative patients, treated according to the COVID-19 protocol or deceased during hospitalization, were retrospectively evaluated by two radiologists and two clinicians, respectively. Results: Of 254 patients, 169 swab-confirmed cases and one patient without chest CT-scan were excluded. A total of 84 patients were eligible for the reliability study. Of these, 21 patients died during hospitalization; the remaining 63 underwent serological testing and were eligible for the accuracy evaluation. Of the 63, 26 patients showed anti-Sars-Cov-2 antibodies, while 37 did not. The inter-rater agreement was “substantial” (kappa 0.683) between radiologists, “moderate” (kappa 0.454) between clinicians, and only “fair” (kappa 0.341) between radiologists and clinicians. Both radiologic and clinical evaluations showed good accuracy compared to serology. Conclusions: The radiologic and clinical diagnosis of COVID-19 for swab-negative patients proved to be sufficiently reliable and accurate to allow a diagnosis of COVID-19, which needs to be confirmed by serology and follow-up.

The diagnosis of Coronavirus disease 2019 (COVID-19) stands on the positivity of real-time reverse-transcriptase-polymerase-chain-reaction (RT-PCR) from nasopharyngeal swab [1, 2] , in addition to laboratory, clinical-epidemiological, and radiologic findings [1] . However, although specificity of RT-PCR is close to 100% [2] , sensitivity varies from 33 to 80% [1] depending on several factors, such as time from exposure, accuracy, and adequacy of sample collection. Consequently, 20-67% of nasal swabs may be false negatives [1, 3] .

Radiological aspects of COVID-19 are variable and rely on the sensitivity of the diagnostic method [1] . Generally, initial CT findings of COVID-19 pneumonia are represented by bilateral ground glass opacities (GGO), with a predominantly subpleural location. In a more severe stage of disease, intralobular reticulations superimposed to GGO ("crazy paving" pattern) and consolidations can be present. Several days after disease onset, CT scan can show linear opacities and perilobular consolidations, suggesting an organizing pneumonia, which is a common reaction to lung injury [4] [5] [6] .

Chest CT-scans significantly contribute to the early identification of patients with lung involvement, increasing the probability of a correct diagnosis in patients with an initially negative swab by about 15% [1, 7] . Antibodies against SARS-Cov-2 become detectable approximately two to three weeks after infection [1, 2] . Although useful for epidemiologic purposes or to confirm previous infection particularly in patients with negative swabs [8] , their role in the management of acute patients is unclear.

At present, the need for urgent treatment and respiratory isolation for a high number of patients affected by respiratory failure due to COVID-19 requires a modulation of hospital activities in order to provide an adequate treatment and, at the same time, to limit the risk of viral transmission to other patients and hospital staff [9] . In this scenario, many efforts are aimed to the development of simple, cheap, and widely available diagnostic tools for the early identification of COVID-19 patients [10] , and for differentiating COVID-19 from other acute diseases, particularly when swab results are not available (i.e., shortage of reagents, delays in analyses) or inconclusive. With this regard, the evaluation of markers such as the Neutrophil-to-Lymphocyte Ratio (NLR) [11] and the Model for possible Early COVID-19 Recognition Score (MECOR score) could help in differentiating COVID-19 from community acquired pneumonia [12] .

However, data concerning the diagnosis of COVID-19 in patients with negative RT-PCR are lacking. In a sample of hospitalized patients with suspected COVID-19 and multiple negative molecular nasopharyngeal swabs, we aimed to ascertain the reliability of the diagnosis established by radiologists and clinicians and the accuracy of their diagnoses versus serology.

From 3 March to 31 May 2020, a total of 254 patients were admitted to the COVID-19 Unit of our 900-bed research hospital for suspected Sars-Cov-2 infection. Inclusion criteria were: age ≥ 18-year-old and admission to the COVID Unit. The only exclusion criterion was consent denial. Those with at least one positive RT-PCR were considered swabconfirmed cases. Patients with multiple negative RT-PCR and treated according to the COVID-19 protocol or deceased during hospitalization represent the study population. A serum sample was collected after at least 15 days from admission, or at follow-up. Patients' blood samples were subjected to Sars-Cov-2 serology with Immunochromatographic assay (Technogenetics-KHB Group), starting from April 2020 and following manufacturer's protocol. Briefly, 10 µL of serum were added onto the sample loading area followed by 3 drops of sample dilution solution. After no longer than 15 min of incubation, viral IgMor IgG-containing positive samples could show up both the T line (test) and C line (control); the samples with only C line were scored as negative. In order to validate the method and avoid false results, patients' sera were frozen (−80 • C) and stored until a chemiluminescent microparticle immunoassay (CMIA) was available (late May 2020), and it was used for the qualitative detection of IgM/IgG antibodies on Abbott's Architect platform.

The study was approved by the Ethics Committee (CSS-COVID-19 Group, num. 46/2020, 8 April 2020). Informed consent was obtained from all patients. For deceased patients and for those unable to give their consent, the Ethics Committee only allowed for collecting data from the clinical records.

Two radiologists, blinded to the clinical records, independently revised chest CTscans and classified CT findings into three diagnostic categories (0 = negative; 1 = typical; 2 = indeterminate) [13] . The first category-"negative for COVID-19"-included both normal CT and CT features ascribable to alternative diagnoses. The second category-"typical for COVID-19"-included typical CT patterns (e.g., classical bilateral and subpleural ground glass opacities, "crazy paving" pattern, multifocal ground glass opacities, consolidations with association of septal thickening) [4, 14] . The third category-"indeterminate for COVID-19"-included possible overlaps with other entities, or atypical features that may also suggest alternative diagnoses [13] . Similarly, two clinicians revised clinical records and decided on the diagnosis of COVID-19 (0 = likely; 1 = unlikely).

Both radiologists and clinicians were blinded to serologic results, which were our gold standard. We considered three categories of diagnostic certainty for radiologists: high level of certainty: both radiologists made a diagnosis of COVID-19 (1-1); low level of certainty: diagnosis was made by just one of them (0-1, 1-0, 1-2, 2-1); non-COVID-19: both scored negative (0-0) or indeterminate . Similarly, we considered three categories for clinicians' evaluation: high level of certainty (1-1); low level of certainty (1-0, 0-1); non-COVID-19 (0-0).

Inter-rater agreement was evaluated as both unweighted kappa statistic (κ) and simple percent agreement, according the methodology described by Fleiss [15] . Kappa values of <0.2 were considered poor, 0.2-0.4 fair, 0.41-0.6 moderate, 0.61-0.8 substantial, and >0.8 almost perfect, according to Landis & Koch [16] . We also calculated the agreement between radiologists and clinicians, based on their evaluation stratified by three categories of diagnostic certainty.

All estimates are reported along with their 95% confidence intervals (95% CI). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated, along with their 95% CI, considering serology as the gold standard and the radiologists' and the clinicians' evaluations-taken separately-as diagnostic tests.

All analyses were performed using R version 4.0.2 (package epiR) [17] .

Of 254 patients with suspected infection, 169 swab-confirmed cases were excluded. The remaining 85 patients showed multiple negative RT-PCRs. One CT-scan was unavailable. Consequently, 84 patients were eligible for the reliability study. A total of 21 patients died during hospitalization; the remaining 63 underwent serology and were eligible for the accuracy evaluation. Of these, 26 (41.3%) showed the presence of antibodies (antibody-confirmed cases), while 37 (58.7%) did not (unconfirmed cases).

The inter-rater agreement was "substantial" between radiologists (Table 1a) , "moderate" between clinicians (Table 1b) , and "fair" between radiologists and clinicians (Table 1c) . The overall radiologists-clinicians agreement was "fair" both for antibody-confirmed cases (percent agreement 69.2%, 95% CI: 48.1-84.9; kappa 0.246, 95% CI: −0.107-0.600) and unconfirmed cases (percent agreement 56.8%, 95% CI: 39.6-72.5; kappa 0.245, 95% CI: −0.009-0.600). There was no radiologists-clinicians agreement for deceased patients (percent agreement 42.9%, 95% CI: 22.6-65.6; kappa 0.000, 95% CI: −0.304-0.304). Table 2 shows the accuracy of the radiologists' (A) and the clinicians' (B) judgments versus serology (gold standard). The sensitivity of the radiologists' evaluation only increased by 2% when diagnoses having a low level of certainty were also considered, at the cost of a considerable decrease in specificity (18%) and PPV (16%). The sensitivity of the clinicians' evaluation did not change, whereas specificity decreased by 28%, and PPV by 23%. 

Patients with repeated negative swabs represent a considerable part of those hospitalized for suspected COVID-19 [1] . In these patients, the diagnosis is challenging and it mostly relies on radiologic and clinical judgment. Although deeper biological samples (e.g., sputum, tracheal-, or brocho-alveolar fluid) can be used to increase the sensitivity of naso-pharyngeal swabs [18] , these techniques are not routinely performed. Moreover, confirming as well as ruling-out a diagnosis of COVID-19 is of paramount importance for a correct allocation of patients [9] , in order to prevent intra-hospital viral shedding [12] . This is the first study to report on the reliability and accuracy of radiologic and clinical evaluations in this particular subset of patients from a real-world setting. We found a higher agreement between radiologists than between clinicians, being substantial and moderate, respectively, according to Landis and Koch classification [10] .

The substantial agreement among radiologists for chest CT categorization is in accordance with a recent study by Debray and coworkers [19] . In this report, authors observed a good performance of radiologists for predicting COVID-19 pneumonia using a similar categorical assessment scheme. However, it should be emphasized that CT evaluation of patients hospitalized for suspected Sars-Cov-2 infection with multiple negative RT-PCRs may be influenced by the high prevalence of COVID-19 disease during the pandemic, with a possible increase of false positive cases.

The moderate agreement observed among clinicians should be attributed to the typology of the reviewed data (clinical charts) and to the fact that clinicians deal with a larger and discrepant amount of information than radiologists [20] . With this regard, patients carrying a complex medical history with potentially overlapping chronic diseases (i.e., heart failure, hypertensive cardiomyopathy, pulmonary involvement due to rheumatologic disorders, chronic obstructive pulmonary disease, etc.) could show a "blurred image" of their acute state (COVID-19).

We found an even lower agreement between radiologists and clinicians, both in antibody-confirmed and unconfirmed cases, with no agreement for deceased patients. That was expected, at least in part, due to the greater amount of information that the clinicians had to deal with [20] .

Both radiologic and clinical evaluations were accurate when compared to serology, showing high sensitivity and specificity, good NPV, and poor PPV (Table 2a ,b). The addition of diagnoses with a low level of certainty reduced specificity and PPV, due to an increase of false positives [19] .

Current management of patients with suspected COVID-19 and negative swabs is uncertain, due to the lack of guidelines [21] . A recent Italian retrospective analysis comparing confirmed versus unconfirmed COVID-19 cases did not show substantial differences in clinical (e.g., symptoms, incubation periods) and radiological findings or mortality between the two groups. However, confirmed cases developed acute respiratory distress syndrome (ARDS) more frequently [22] .

Our data confirm that chest CT scans can help early identification of most patients with lung involvement, even in the absence of swab results [7, 23] , although they are not sufficient to conclusively rule-in or rule-out the presence of COVID-19 [24, 25] . Likewise, clinical evaluation can also identify most COVID-19 patients on the basis of suggestive, even if less specific, features typical of the coronavirus infection. In other words, chest CT scans and clinical evaluation are able to correctly identify most of the COVID-19 patients and to rule-out most unaffected patients.

Limitations of our study include its small sample size, the possible non-generalizability of our results to other settings, and the choice of serology as the gold standard to confirm a previous infection.

In conclusion, the radiologic and clinical evaluations are sufficiently accurate and reliable to allow a diagnosis of COVID-19 in patients with repeated negative swabs. On this basis, we suggest also reporting negative-swab patients in clinical series of COVID-19 patients, provided a clinical-radiologic diagnosis is done. 

Transmission, Diagnosis, and Treatment of Coronavirus Disease 2019 (COVID-19): A Review

Interpreting Diagnostic Tests for SARS-CoV-2

Laboratory Testing for 2019 Novel Coronavirus (2019-nCoV) in Suspected Human Cases

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): A pictorial review

Relation between chest CT findings and clinical conditions of coronavirus disease (COVID-19) pneumonia: A multicenter study

Chest CT findings in coronavirus disease-19 (COVID-19): Relationship to duration of infection

Correction to: Chest CT for early detection and management of coronavirus disease (COVID-19): A report of 314 patients admitted to Emergency Department with suspected pneumonia

SARS-CoV-2 serological testing changes disease management in a PCR-negative patient

Gemelli Against COVID-19 Clinician Team (GAC-19 CT) Study Group. Hospital reengineering against COVID-19 outbreak: 1-month experience of an Italian tertiary care center

Direct diagnostic testing of SARS-CoV-2 without the need for prior RNA extraction

The diagnostic and predictive role of NLR, d-NLR and PLR in COVID-19 patients

The Model for Early COvid-19 Recognition (MECOR) Score: A Proof-of-Concept for a Simple and Low-Cost Tool to Recognize a Possible Viral Etiology in Community-Acquired Pneumonia Patients during COVID-19 Outbreak

Integrated Radiologic Algorithm for COVID-19 Pandemic

Time Course of Lung Changes at Chest CT during Recovery from Coronavirus Disease 2019 (COVID-19)

Statistical Methods for Rates & Proportions

The measurement of observer agreement for categorical data

The R Project for Statistical Computing

Negative Nasopharyngeal and Oropharyngeal Swabs Do Not Rule Out COVID-19

Observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19

Comparison of chest X-ray interpretation by Emergency Department clinicians and radiologists in suspected COVID-19 infection: A retrospective cohort study

Infectious Diseases Society of America Guidelines on the Treatment and Management of Patients with COVID-19

Confirmed or unconfirmed cases of 2019 novel coronavirus pneumonia in Italian patients: A retrospective analysis of clinical features

Chest CT for detecting COVID-19: A systematic review and meta-analysis of diagnostic accuracy

The clinical sensitivity of a single SARS-CoV-2 upper respiratory tract RT-PCR test for diagnosing COVID-19 using convalescent antibody as a comparator

COVID-19): A Critical Review of the Literature to Date

We thank Chiara Di Giorgio for her expert revision of the English language.