key: cord-0903998-4vuas0yx
authors: Skittrall, Jordan P.; Wilson, Michael; Smielewska, Anna A.; Parmar, Surendra; Fortune, Mary D.; Sparkes, Dominic; Curran, Martin D.; Zhang, Hongyi; Jalal, Hamid
title: Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification testing in a low prevalence setting
date: 2020-10-14
journal: Clin Microbiol Infect
DOI: 10.1016/j.cmi.2020.10.003
sha: 40feade2ee2bdac73ba81b719e690efcdb7e4826
doc_id: 903998
cord_uid: 4vuas0yx

OBJECTIVES: When SARS-CoV-2 prevalence is low, many positive test results are false positives. Confirmatory testing reduces overdiagnosis and nosocomial infection and enables real-world estimates of test specificity and positive predictive value. This study estimates these parameters to evaluate the impact of confirmatory testing, and to improve clinical diagnosis, epidemiological estimation and interpretation of vaccine trials. METHODS: Over one month, we took all respiratory samples from our laboratory with a patient’s first detection of SARS-CoV-2 RNA (Hologic Aptima SARS-CoV-2 assay or in-house RT-PCR platform), and repeated testing using two platforms. Samples were categorised by source, and by whether clinical details suggested COVID-19 or corroborative testing from another laboratory. We estimated specificity and positive predictive value using maximum likelihood-based approaches. RESULTS: Of 19,597 samples, SARS-CoV-2 RNA was detected in 107. 52 corresponded to first-time detection (0.27% of tests on samples without previous detection); further testing detected SARS-CoV-2 RNA ≥1 time (“confirmed”) in 29 (56%), and failed to detect SARS-CoV-2 RNA (“not confirmed”) in 23 (44%). Depending upon assumed parameters, point estimates for specificity and positive predictive value were 99.91%–99.98% and 61.8%–89.8% respectively using the Hologic Aptima SARS-CoV-2 assay, and 97.4%–99.1% and 20.1%–73.8% respectively using an in-house assay. CONCLUSIONS: Nucleic acid amplification testing for SARS-CoV-2 is highly specific. Nevertheless, when prevalence is low a significant proportion of initially positive results fail to confirm and confirmatory testing substantially reduces false positive detections. Omitting additional testing in samples with higher prior detection probabilities focuses testing where clinically impactful and minimises delay.

testing reduces overdiagnosis and nosocomial infection and enables real-world estimates of test 27 specificity and positive predictive value. This study estimates these parameters to evaluate the 28 impact of confirmatory testing, and to improve clinical diagnosis, epidemiological estimation and 29 interpretation of vaccine trials. 30

Over one month, we took all respiratory samples from our laboratory with a patient's first detection 32 of SARS-CoV-2 RNA (Hologic Aptima SARS-CoV-2 assay or in-house RT-PCR platform), and repeated 33 testing using two platforms. Samples were categorised by source, and by whether clinical details 34 suggested COVID-19 or corroborative testing from another laboratory. We estimated specificity and 35 positive predictive value using maximum likelihood-based approaches. 36

Of 19,597 samples, SARS-CoV-2 RNA was detected in 107. 52 corresponded to first-time detection 38 (0.27% of tests on samples without previous detection); further testing detected SARS-CoV-2 RNA 39 ≥1 time ("confirmed") in 29 (56%), and failed to detect SARS-CoV-2 RNA ("not confirmed") in 23 40 (44%). Depending upon assumed parameters, point estimates for specificity and positive predictive 41 value were 99.91%-99.98% and 61.8%-89.8% respectively using the Hologic Aptima SARS-CoV-2 42 assay, and 97.4%-99.1% and 20.1%-73.8% respectively using an in-house assay. 43

Nucleic acid amplification testing for SARS-CoV-2 is highly specific. Nevertheless, when prevalence is 45 low a significant proportion of initially positive results fail to confirm and confirmatory testing 46

The COVID-19 pandemic [1,2] continues to cause morbidity and mortality [3] [4] [5] [6] . In some regions, 50 following seasonal changes and infection control measures, incidence and prevalence of SARS-CoV-2 51 infection has decreased [7] , whilst testing capacity has increased [8-10]. As much testing capacity 52 remains operational, in such settings many being tested will be infection-free. 53

The commonest clinical laboratory testing employed to detect SARS-CoV-2 is nucleic acid 55 amplification testing (NAAT). NAAT techniques detect SARS-CoV-2 RNA using amplification steps 56 that bind specific complementary primers to nucleic acid; probes assay amplification progress. With 57 well-designed primers and probes, NAAT is highly sensitive and specific. However, in practice, issues 58 including inadequate clinical sampling, sample degradation, and reaction inhibition affect sensitivity, 59 and issues including non-specific probe breakdown, amplicon contamination and sample error affect 60 specificity, even in well-run laboratories. Initial technical validations of new diagnostic assays often 61 only partially reflect clinical workflows. They often test limited numbers of samples, and so are 62 underpowered to detect small proportions of false positives or false negatives. In low prevalence, 63 high throughput settings, false positive results will occur regularly, despite high specificity, causing 64 unnecessary community isolation and contact tracing, and nosocomial infection if inpatients with 65 false positive tests are cohorted with infectious patients. However, the low positive result incidence 66 makes confirmatory testing to reduce false positive rates feasible. This additionally enables us to 67 understand predictors that should prompt re-evaluation of results, allowing laboratories to prepare 68 for when an increased positive result incidence makes confirmatory testing on all initially positive 69 samples impracticable. Beyond these clinical benefits, this work allows improved epidemiological 70 estimation of infection rates, which will guide policy, and allows more accurate statistical analysis of 71 infection rates in study settings, e.g. vaccine trials. 72 73 J o u r n a l P r e -p r o o f 5 The Cambridge Clinical Microbiology and Public Health Laboratory is a regional clinical laboratory in 74 the East of England. Since early 2020 it has undertaken SARS-CoV-2 NAAT on many inpatient and 75 community samples. On 22 nd June 2020, following decreasing local positive test result incidences, 76 the laboratory began same-and cross-platform confirmatory testing for patients with positive 77 results but without previous positive tests. By combining highly specific tests in a way that 78 minimises repetition of errors, the specificity of reported results could be increased further, 79 reducing inconvenience and harm to the few uninfected people with initially positive results. We 80 report the results from testing undertaken during the first month after policy implementation. 81 82

Patient sampling Patient sampling Patient sampling Patient sampling, , , , initial preparation initial preparation initial preparation initial preparation and testing and testing and testing and testing (Figure 1) . 19,261 were initially 161 tested with the Aptima SARS-CoV-2 assay, and 336 by in-house RT-PCR (Table 1 gives p<0.001). There was substantial heterogeneity in specimen type and patient sex and age between 168 samples tested on different platforms and between samples with different testing outcomes 169 (Table 1) Heterogeneity comparisons are made within rows between the columns whose headers are marked with identical numbers in parentheses. Proportion 356

A novel coronavirus from patients with 305 pneumonia in China

A new coronavirus associated with 307 human respiratory disease in China

All-cause 309 excess mortality observed by age group and regions in the first wave of the COVID-19 310 pandemic in England

Mortality 312 impacts of the coronavirus disease (COVID-19) outbreak by sex and age: rapid mortality 313 surveillance system

Excess mortality 315 estimation during the COVID-19 pandemic: preliminary data from Portugal

Population Point Prevalence of SARS-CoV-2 Infection Based on a Statewide Random

Community 321 prevalence of SARS-CoV-2 in England: results from the ONS coronavirus infection survey pilot

Laboratory 324 readiness and response for novel coronavirus (2019-nCoV) in expert laboratories in 30

Preparedness 327 of European diagnostic microbiology labs for detection of SARS-CoV-2

International external quality assessment for SARS-CoV-2 molecular detection and survey on 331 clinical laboratory preparedness during the COVID-19 pandemic

COVID-19: laboratory investigations and sample requirements for 334 diagnosis

Rapid point of care 338 nucleic acid testing for SARS-CoV-2 in hospitalised patients: a clinical trial and 339 implementation study

R: A Language and Environment for Statistical Computing

False negative tests for SARS-CoV-2 infection -343 challenges and implications

Throughput Transcription-mediated amplification on the Hologic Panther is a highly sensitive 350 method of detection for SARS-CoV-2

Detection of 2019 352 novel coronavirus (2019-nCoV) by real-time RT-PCR

Laboratory, and particularly to the laboratory managers, senior virology biomedical scientists, staff

female and proportion URT specimen comparisons are made using Fisher's exact test. Age comparisons are made using the Mann-Whitney U test. Values 357 yielding p-values less than 0.05 are marked and reported, together with odds ratios (OR) reported to two significant figures or medians of difference 358 between samples reported to one decimal place, as appropriate, followed by 95% confidence intervals. Given 21 separate independent tests, a p-value less 359 than 0.05 would be expected to occur by chance on average 1.05 times. Note that the table's construction means that the tests are not independent of 360 each other.