key: cord-0971978-veqi529p
authors: Hledik, M.; Polechova, J.; Beiglboeck, M.; Herdina, A. N.; Strassl, R.; Posch, M.
title: Analysis of the specificity of the SD Biosensor Standard Q Ag-Test based on Slovak mass testing data
date: 2020-12-11
journal: nan
DOI: 10.1101/2020.12.08.20246090
sha: edd274e4cd400bb9d588aebb42bfaebc331f6645
doc_id: 971978
cord_uid: veqi529p

From 31.10. - 1.11. 2020 Slovakia has used the SD Biosensor Standard Q Ag-Test for nationwide tests for SARS-CoV-2, in which 3,625,332 persons from 79 counties were tested. Based on this data, we calculate that the specificity of the test is at least 99.6% (with a 97.5% one-sided lower confidence bound). Our analysis is based on a worst case approach in which all positives are assumed to be false positives. Therefore, the actual specificity is expected to exceed 99.6%.

While PCR-tests are usually considered as the gold standard to detect infection with the SARS-CoV-2 coronavirus in terms of sensitivity as well as specificity, Antigen-Tests (Ag-Tests) offer practical advantages in terms of costs, logistics and speed [1] . Because Ag-Tests may play a major role in large scale testing strategies [2, 3] , their specificity is of significant interest.

In this note we consider the specificity of the SD Biosensor Standard Q Ag-Test based on data of mass tests in Slovakia and infer Ag-Test specificity from a large sample of the general population. This is in contrast to other studies which use a PCR-test as a reference to estimate sensitivity and specificity of an Ag-test [4] [5] [6] [7] .

From late October to early November 2020, Slovakia undertook large scale testing of its population [8] . To the best of our knowledge exclusively the SD Biosensor Standard Q Ag-Test was used [9] .

The tests in Slovakia were divided into three phases. In the first pilot phase, testing was only conducted in certain particularly affected counties. In phase 2 (31.10. -1.11.) all Slovak counties were tested. In phase 3 (6.11. -8.11 .) all heavily affected counties (those with > 0.7% prevalence during phase 2) were tested again. In this retrospective study, we use publicly available data on the outcome of tests in phase 2 [10].

In phase 2, all residents aged 10 to 65 throughout Slovakia were invited to get tested and 3 625 332 participated (about 66% of the population of Slovakia). Among 79 administrative counties of Slovakia, participation rate varied from 39% (Košice III county) to 78% (Senec county) of inhabitants. Participation was voluntary, but it was a condition to avoid quarantine. Persons that were quarantined due to a previous positive PCR-test for COVID-19 or due to being a close contact of such a person were excluded from the test. Data from phase 2 was used in this study, because the general countrywide testing, performed irrespective of regional incidence rates, potentially covers also regions with very low incidence. Results from the latter are most suitable to derive bounds of the specificity.

Participants were tested with antigen tests at specially set up locations by medical personnel via nasopharyngeal swab sampling and received their results after a short waiting period.

To derive a lower bound for the specificity we made the conservative ("worst case") assumption that potentially all positive results constitute false positive results. For each of the 79 counties, we compute the rate of positive tests together with two-sided binomial Clopper-Pearson confidence intervals at the Bonferroni-adjusted significance level of alpha = 0.05/79 (adjusted for the number of counties; see e.g. [11] ). The minimum upper bound of these adjusted confidence intervals is an upper 97.5% confidence bound for the test positivity rate. This will be in a county with low disease activity and a large sampled population. Under the conservative . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 11, 2020. ;  assumption that all positive results constitute false positives, it is also an upper bound on the false positive rate of the test and defines a lower bound for the true specificity. Figure 1 depicts the test positivity rate of the individual counties ordered according to the upper bound of the simultaneous Bonferroni adjusted 95% confidence interval from low to high incidence. The lowest upper bound (obtained for Rožnava) is 0.40%. In terms of specificity (instead of false positives) this implies that with (97.5% confidence) the specificity of Standard Q is higher than 99.6%. Figure 1 : SARS-CoV-2 antigen detected incidence in Slovakian mass testing by county. Test positivity rates among the 3,625,332 tested persons in the 79 counties with simultaneous Bonferroni adjusted 95% confidence intervals. Counties are ordered from low to high incidence by the upper confidence bound. Data source [10].

. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted December 11, 2020. ; Table 1 shows the five counties with the lowest upper bound on the positivity rate of antigen tests. These include counties with both relatively low participation rate (Rožňava, Revúca and Veľký Krtíš rank 65., 67. and 57. in participation among the 79 counties) and relatively high participation rate (Bratislava IV and Bratislava III rank 13. and 4. 

This worst case analysis provides only a lower bound of the true specificity since it neglects entirely the true incidence of COVID-19. As the antigen tests in Slovakia were not controlled directly using accompanying PCR-tests, we refrain from an attempt to subdivide the observed positives into true and false positives.

The binomial distribution assumption underlying the computation of the Clopper-Pearson confidence intervals, is based on the assumption of independent events that could be violated if, e.g. testing stations were operating with different quality. However, the Bonferroni-correction is a strictly conservative approach to derive simultaneous confidence bounds that account for the selection of the county with the lowest upper bound to derive the estimate.

The above estimate is consistent with the information provided by the manufacturer [13], stating 0.32% (0.01, 1.78) as a false positive rate -a very broad CI. The derived upper bound from the mass testing is more informative and appears relevant in that it constitutes the first (to the best of our knowledge) bound obtained from large scale practical use of Standard Q and also suggests (slightly) better performance of Standard Q compared to previous studies: [4] It is well understood in the epidemiology literature that imperfect reference criteria lead to a systematic bias of the true incidence [14] . Specifically, the sensitivity and specificity of tests are . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 11, 2020. ;  systematically underestimated if the criterium (SARS-CoV-2 infection) cannot be assessed directly and tests are compared to a surrogate criterion as gold standard which is subject to errors. This could be a contributing factor for the relatively higher specificity found in Slovak data compared to other studies. Other conceivable factors that could bias the results include the different quality of swab sampling and handling, low temperature in outdoor testing stations and deviations in the production over time and data quality.

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance

Rethinking Covid-19 test sensitivity-A strategy for containment

Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigendetecting point-of-care diagnostics for SARS-CoV-2

Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-Biosensor antigen test for SARS-CoV-2

Comparison of seven commercial SARS-CoV-2 rapid Point-of-Care Antigen tests

Headto-head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected anterior nasal swab versus professional-collected nasopharyngeal swab

Slovakia to test all adults for SARS-CoV-2

The effectiveness of population-wide, rapid antigen test-based screening in reducing SARS-CoV-2 infection prevalence in Slovakia

Confidence intervals for the means of the selected populations

Estimating prevalence from the results of a screening test

We would like to thank Alfred Uhl, Richard Kollár and Katarína Bod'ová for very helpful comments. We also thank Matej Mišík for discussion and information regarding the Slovak testing data and Ag-test used.