key: cord-0955023-chtlnrsi authors: Bergmans, Barbara J.M.; Reusken, Chantal B.E.M.; van Oudheusden, Anne J.G.; Godeke, Gert-Jan; Bonačić Marinović, Axel A.; de Vries, Esther; Kluiters-de Hingh, Yvette C.M.; Vingerhoets, Ralf; Berrevoets, Marvin A.H.; Verweij, Jaco J.; Nieman, An-Emmie; Reimerink, Johan; Murk, Jean-Luc.; Swart, Arno title: Test, trace, isolate: evidence for declining SARS-CoV-2 PCR sensitivity in a clinical cohort date: 2021-04-21 journal: Diagn Microbiol Infect Dis DOI: 10.1016/j.diagmicrobio.2021.115392 sha: f1b2ba64bcd3ce0aa298e7cfa163d3348d8f3bf9 doc_id: 955023 cord_uid: chtlnrsi Real-time reverse transcription-polymerase chain reaction (RT-PCR) on upper respiratory tract (URT) samples is the primary method to diagnose SARS-CoV-2 infections and guide public health measures, with a supportive role for serology. We reinforce previous findings on limited sensitivity of PCR testing, and solidify this fact by statistically utilizing a firm basis of multiple tests per individual. We integrate stratifications with respect to several patient characteristics such as severity of disease and time since onset of symptoms. Bayesian statistical modelling was used to retrospectively determine the sensitivity of RT-PCR using SARS-CoV-2 serology in 644 COVID-19-suspected patients with varying degrees of disease severity and duration. The sensitivity of RT-PCR ranged between 80-95%; increasing with disease severity, it decreased rapidly over time in mild COVID-19 cases. Negative URT RT-PCR results should be interpreted in the context of clinical characteristics, especially with regard to containment of viral transmission based on ‘test, trace and isolate’. Keywords: SARS-CoV-2, RT-PCR, serology, sensitivity, public health COVID-19 is diagnosed primarily by testing upper respiratory tract (URT) samples with real-time reverse transcription-polymerase chain reaction (RT-PCR) (1). Experience with nucleic acid amplification tests for other respiratory viruses, such as influenza virus, granted a high level of confidence in the clinical sensitivity of these types of assays (2) . In the SARS-CoV-2 pandemic, however, frequent false-negatives were reported by physicians worldwide (3, 4) and were indicated to significantly complicate healthcare organization, hospital admission and isolation capacity (5, 6) . In several systematic reviews, the false-negative rate of RT-PCR was calculated to range between 22-66%, depending on symptom duration (7, 8) . It is not known whether false-negatives were due to methodological problems, such as sampling error, suboptimal handling of samples or suboptimal assay design, or if they reflect a biological feature of SARS-CoV-2 infections. Clinical samples of lower respiratory tract material such as sputum or broncho-alveolar lavage fluid (BALF) appeared to yield significantly lower false negative rates than URT samples in patients with COVID-19 pneumonia (9) (10) (11) . However, only 13-30% of patients produce sputum (5) and obtaining BALF is often impractical. URT samples are therefore preferred. Estimates of sensitivity are further complicated by the lack of a reliable gold standard. Although imaging techniques may aid in the diagnosis of COVID-19 (12, 13) , their sensitivity and specificity are insufficient to be used as such. As compared to previous studies on the sensitivity of PCR and serological tests, our approach integrates some features that are not always properly accounted for. Firstly, we do not employ a gold standard, but rather draw information from the results of multiple tests on a single individual. Secondly, severity and duration of symptoms are modelled, as well as sex, age, and immunocompetence. Finally, our use of Bayesian statistics enabled us to retrospectively determine the clinical sensitivity of RT-PCR in URT-samples from a large cohort of COVID-19-suspected patients including a full characterization of uncertainty in the outcomes. Although complex, it is crucial to reliably assess the sensitivity of RT-PCR in different clinical cohorts as RT-PCR is the foundation for test, trace and isolation policies that are the cornerstone of worldwide pandemic control efforts. Highly sensitive and specific antibody assays are important tools to support patient diagnostics at a later stage of the disease. The study was performed in the Elisabeth-TweeSteden Hospital, Tilburg, The For this purpose, two independent tests were used; an in-house developed protein micro-array based on SARS-CoV-2 S1 and N proteins (PMA (14, 15) ) and the Wantai total antibody ELISA based on the RBD-domain of S1 (Beijing Wantai Biological A duplex PCR for Sars-Cov-2 E-gene was performed as described in the supplementary materials. RT-PCR results were considered positive if the cycle threshold (Ct-) value was 50. China; Cat # WS1096) were detected according to the instructions of the manufacturer. The presence of SARS-CoV-2 antibodies is expressed as a sample/cut-off ratio. Additionally, an in-house protein microarray was performed for the detection of antibody responses to SARS-CoV-2-S1 and SARS-CoV-2-N antigens. Further details on the serological assays may be found in the supplementary materials. In the microarray, the threshold value for the detection of specific antibodies against N or S1 has been set at a titer of >1:20. Towards the goal of determining clinical sensitivity of the test assays used, we set up a Bayesian statistical framework which allowed us to estimate sensitivities of each test without resorting to a 'gold standard'. Our definition of clinical sensitivity is: the ability to detect (possibly asymptomatic) infection in a patient. We assumed perfect specificity for both RT-PCR (16) (17) (18) (19) , and serology (20) . This is further justified by noting that all individuals enrolled in the cohort were clinically diagnosed for COVID-19. It is feasible to determine sensitivities without a perfect benchmark to compare with, which can be understood by realizing that each individual must be positive when at least one test scores positive (assuming the 100% specificity as outlined above). In such a case all negative tests for this person must be false negatives; this gives evidence against the sensitivity of those tests. We included time since onset of symptoms, severity, sex and immunocompromised status as covariates. For a detailed description of the Bayesian statistical model that was used, see the Supplementary Text. In the study period, 644 patients that presented at our hospital with clinically suspected COVID-19 were tested for SARS-CoV-2 by RT-PCR on URT samples. In total, sera from 250 patients were obtained and analysed for the presence of SARS-CoV-2 specific antibodies ( Table 1 , Fig. 1 ). The manufacturers of the Wantai assay recommend index values of 1.1 and higher as evidence for the presence of specific antibodies. However, we observed that at this cut-off value, several RT-PCR confirmed patients scored negative and a cut-off of 0.25 was calculated to be more appropriate in our cohort (Fig. 2, Supplementary Text) . The rightmost panel of this figure gives quantitative evidence. Clearly, using the manufacturer's cut-off, many positives would be to the left of the cut-off and would score false-negative in the Wantai test. URT-RT-PCR sensitivity varied according to disease severity: for outpatients, 80% (57%-100% Bayesian credible interval (CI)), for patients admitted to a non-ICU hospital ward 86% (77-95 CI), and for patients admitted to ICU 95% (84-100 CI) (Fig. 3, Table S1 ). URT-RT-PCR sensitivity was higher in males than in females (91% vs 81%), higher in deceased than in non-deceased patients (95% vs 83%) and higher in immunocompromised than in immunocompetent patients (93% vs 86%) (Fig. S1, Table S1 ). For each disease severity category, URT RT-PCR cycle threshold (Ct)-values displayed an increasing trend by number of days since symptom onset, reflecting a decreasing viral load in time, albeit with considerable inter -individual variation (Fig. 4, Table S2 ). Ct-values of outpatients increased rapidly compared to Ct-values of hospitalized patients. This translates in a rapid loss of sensitivity (Fig. 5) . In contrast, RT-PCR on IC and in-hospital patients retains its initial sensitivity for a prolonged period of time (approximately 15 and 20 days respectively, Fig. 5) . However, the point of decline is highly uncertain, due to the absence of patient material longer after onset of symptoms. Therefore, conclusions on the time of decline of sensitivity should not be drawn for IC and in-hospital patients. In contrast, outpatients do show a marked decrease of sensitivity over time, with lower uncertainty: sensitivity could be halved already within three weeks . COVID-19-suspected patients with a negative URT RT-PCR and without alternative diagnosis were frequently retested for SARS-CoV-2 after resampling. This increased the total number of RT-PCR-confirmed patients with 35 (Table 1) . Serum samples were analysed for SARS-CoV-2 antibodies by ELISA and PMA. Bayesian modelling revealed an overall sensitivity of >94% for both ELISA and PMA, which did not change significantly according to disease severity category, sex or mortality (Fig. 2, Table S1 ). The sensitivity of antibody assays was lower in immunocompromised patients (82-92%, Table S1 ) and this was the only group in which antibody assays had a lower sensitivity compared to UTR RT-PCR. The ELISA and PMA results were discrepant in only four of 250 sera (Table S3 ). All four patients tested RT-PCR positive in URT-swab, and three out of four were immunocompromised. An accurate assessment of the clinical sensitivity of diagnostic tools, in particular of RT-PCR on URT samples on which global test, trace and isolate strategies are based (21), is an absolute requisite for good patient care and adequate infection risk management. We observed a decrease in sensitivity with decreasing disease severity, an increase in sensitivity in immunocompromised patients and a rapid decline of sensitivity in time post onset of symptoms, but only in outpatients. In our study, 5-14% of hospitalized COVID-19 cases and 21% of outpatients tested negative in URT-PCR. This finding is in contrast with local practice and guidelines, which are often based on the assumption of near-perfect sensitivity (22) . Comparing our results to other studies that model the sensitivity of SARS-CoV-2 test sensitivity, we find an overall concordance. In a study involving patients from the emergency room, comparable to our ICU group, a sensitivity of 89% is found using a latent class model, which compares well to 95% found in our study (23) . In another study, a Bayesian spline model yields a PCR sensitivity of 76% at day of onset of symptoms, increasing to 89% after four days (24) . Furthermore, no difference between sensitivity up to, and after day seven post symptoms was found, which we also observe in ICU patients (up to at least two weeks post symptom start). Another study reports on a group of patients from an outbreak cluster, where the sensitivity of the PCR test was established at 86%. This result relied on a composite 'golden standard' by comparing the PCR result with several follow-up tests (25) . Such a reliance on direct comparison with additional tests is commonplace. For example, the sensitivity of the PCR assay has been estimated at 79.2% (until day 4 post-symptoms: 100%, after day four 50%) using PCR combined with rapid antibody testing (26) . Other practitioners have relied on clinical diagnosis (27) , reporting a sensitivity of 82.2% is found. This is a low estimate, which may be explained by the aspecific symptoms of Covid-19 infection. The analytical sensitivity of PCRs generally approaches 100%, which means tests are able to detect a single viral genome copy in the reaction volume (16) (17) (18) (19) , the clinical sensitivity may be substantially lower, due to low quality of samples, presence of inhibiting factors, suboptimal pre-analytic processing or specific biologic features of the viral infection. If one assumes infection of the URT always occurs in SARS-CoV-2 infected persons, a negative URT RT-PCR could be the result of localized early clearance of the virus or low levels of local viral replication. The localized clearance would then have different kinetics than the rate of viral load decline we measured in this study. Alternatively, the possibility should be considered that the URT of these patients was not infected and infection can remain more localized (e.g. to the trachea / lower respiratory tract) (28) . As can be expected, RT-PCR sensitivity of URT correlates with the SARS-CoV-2 RNA load in these samples. The increase of mean Ct-value over time was significantly slower in hospitalized patients compared to outpatients, which explains why the sensitivity of RT-PCR on URT samples decreased more rapidly in cases with mild infections than in hospitalized patients and suggests hospitalized patients have difficulties clearing the virus. In our cohort, immunocompromised patients were the only category in which the sensitivity of serology was lower (PMA), or comparable (Wantai) to URT-RT-PCR. According to the definition we used, 73 of 644 patients were immunocompromised (Table S4) , and most were treated by systemic immunosuppressants that inhibit antibody production. This relative lack of antibody response may explain the lower sensitivity of serology in this patient group, and it may clarify why relatively many immunocompromised patients' Wantai results fell below the manufacturer's cut-off value. Although antibody assays were shown to have high sensitivity in the rest of our cohort, their use in tracing and isolating strategies is non-existent, due to the fact that antibodies take a while to be produced and may be detectable for several months after initial infection. Therefore, in order to timely diagnose and quarantine COVID-19 patients, URT-PCR will continue to be used as the assay on which initial decisions are based. Our estimated URT-PCR sensitivity has important consequences for screening, treatment and isolation measures in hospitals. Though a positive PCR does not necessarily signify the presence of viable virus (29) , sensitivities of 86-94% are not sufficient to lift isolation measures in the event of a negative initial URT-PCR. In the early stages of hospital admission, clinical suspicion and local COVID-19 prevalence should guide isolation and treatment decisions. Additionally, timely sputum or feces sampling should take place in an effort to confirm COVID-19 diagnosis. Several studies found that 30-50% of COVID-19 patients have detectable SARS-CoV-2 RNA in feces (30, 31) . In our study, feces yielded 3.5 times more RT-PCR-confirmed positives than repeated URT-swab: 40% of additionally sampled feces tested positive, as opposed to only 11% of repeated URT-swabs (Table 1) . Our findings also have ramifications for broad molecular testing in the general population, as currently established across the world, based on URT-swabbing in highthroughput testing lanes (32) . The evidently often absent thorough epidemiological and clinical interpretation of negative results in these settings, in combination with the observed low clinical sensitivity of RT-PCR in the population with mild complaints that typically visit those testing sites, will lead to missed cases. The effect of imperfect test sensitivity and incomplete testing was previously assessed by Bayesian methods (33) , where it was estimated that the actual number of cases could be 3 to 20 times higher than the number of confirmed cases, and 14% could be attributed to imperfect sensitivity. In conclusion, our results show that for an accurate diagnosis based on RT-PCR test results and subsequent appropriate clinical management and infection control measures, a thorough understanding of the clinical sensitivity of RT-PCR in URT samples is necessary. The apparent lack of a high clinical sensitivity of this standard diagnostic method in specific situations warrants vigilance for missed cases especially in settings of high-throughput testing lanes where epidemiological and clinical context are often disconnected from negative test results for final interpretation. Infectious Diseases Society of America Guidelines on the Treatment and Management of Patients with COVID-19 Clinical evaluation of multiplex RT-PCR assays for the detection of influenza A/B and respiratory syncytial virus using a high throughput system If you have coronavirus symptoms, assume you have the illness, even if you test negative False Negative Tests for SARS-CoV-2 Infection -Challenges and Implications De eerste honderd opgenomen COVID-19-patiënten in het Elisabeth-Tweesteden Ziekenhuis Rapid assessment of regional SARS-CoV-2 community transmission through a convenience sample of healthcare workers, the Netherlands False-Negative Results of Initial Rt-Pcr Assays for Covid-19: a Systematic Review Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure Positive rate of RT-PCR detection of SARS CoV-2 infection in 4880 cases from one hospital in Wuhan, China Detection of SARS-CoV-2 in Different Types of Clinical Specimens Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases Time course of lung changes at chest CT during recovery from Coronavirus disease 2019 (COVID-19) Profiling of humoral immune responses to influenza viruses by using protein microarray Specific serology for emerging human coronaviruses by protein microarray Molecular Diagnosis of a Novel Coronavirus (2019-nCoV) Causing an Outbreak of Pneumonia Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Comparative performance of SARS-CoV-2 detection assays using seven different primer-probe sets and one assay kit Comparison of seven commercial RT-PCR diagnostic kits for COVID-19 Who. Critical preparedness, readiness and response actions for Report 16: Role of testing in COVID-19 control. Imperial College London COVID-19 response team Specificity and Predictive Values of Molecular and Serological Tests for COVID-19: A Longitudinal Study in Emergency Room Insights into the practical effectiveness of RT-PCR testing for SARS-CoV-2 from serologic data, a cohort study The clinical sensitivity of a single SARS-CoV-2 upper respiratory tract RT-PCR test for diagnosing COVID-19 using convalescent antibody as a comparator Combined Point-of-Care Nucleic Acid and Antibody Testing for SARS-CoV-2 following Emergence of D614G Spike Variant Sensitivity of RT-PCR testing of upper respiratory tract samples for SARS-CoV-2 in hospitalised patients: a retrospective cohort study Reverse Genetics Reveals a Variable Infection Gradient in the Respiratory Tract Virological assessment of hospitalized patients with COVID-2019 The presence of SARS-CoV-2 RNA in the feces of COVID-19 patients Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China Testing for coronavirus Substantial underestimation of SARS-CoV-2 infection in the United States Bergmans: conceptualization, writing of the manuscript, data collection and curation Reusken: conceptualization, writing of the manuscript Bonačić Marinović: statistics Esther de Vries: conceptualization laboratory analyses Ralf Vingerhoets: laboratory analyses Berrevoets: laboratory analyses laboratory analyses An-Emmie Nieman: laboratory analyses Johan Reimerink: laboratory analyses Murk: conceptualization, writing of the manuscript Arno Swart: statistics, programming, data curation, writing of the manuscript We thank BJ Bosch for providing the S1 antigen for the protein microarray and H van Zundert for providing additional data on immunocompromised patients The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.