key: cord-0817898-3esir87x
authors: Olearo, Flaminia; Nörz, Dominik; Heinrich, Fabian; Sutter, Jan Peter; Rödel, Kevin; Schultze, Alexander; Wiesch, Julian Schulze Zur; Braun, Platon; Oesterreich, Lisa; Kreuels, Benno; Wichmann, Dominic; Aepfelbacher, Martin; Pfefferle, Susanne; Lütgehetmann, Marc
title: Handling and accuracy of four rapid antigen tests for the diagnosis of SARS-CoV-2 compared to RT-qPCR
date: 2021-03-03
journal: J Clin Virol
DOI: 10.1016/j.jcv.2021.104782
sha: 8e9f37a1b6902ecee56fa1b15e9ce1ad7cc84a1f
doc_id: 817898
cord_uid: 3esir87x

BACKGROUND: SARS-CoV-2 molecular diagnostics is facing material shortages and long turnaround times due to exponential increase of testing demand. OBJECTIVE: We evaluated the analytic performance and handling of four rapid Antigen Point of Care Tests (AgPOCTs) I-IV (Distributors: (I) Roche, (II) Abbott, (III) MEDsan and (IV) Siemens). METHODS: 100 RT-PCR negative and 84 RT-PCR positive oropharyngeal swabs were prospectively collected and used to determine performance and accuracy of these AgPOCTs. Handling was evaluated by 10 healthcare workers/users through a questionnaire. RESULTS: The median duration from symptom onset to sampling was 6 days (IQR 2-12 days). The overall respective sensitivity were 49.4% (CI95%: 38.9 - 59.9), 44.6% (CI95%: 34.3 - 55.3), 45.8% (CI95%: 35.5 - 56.5) and 54.9% (CI95%: 43.4-65.9) for tests I, II, III and IV, respectively. In the high viral load subgroup (containing >10(6) copies of SARS-CoV-2 /swab, n = 26), AgPOCTs reached sensitivities of 92.3% or more (range 92.3% - 100%). Specificity was 100% for tests I, II (CI95%: 96.3 – 100 for both tests) and IV (CI95%: 96.3 - 100) and 97% (CI95%: 91.5 - 98.9) for test III. Regarding handling, test I obtained the overall highest scores, while test II was considered to have the most convenient components. Of note, users considered all assays, with the exception of test I, to pose a significant risk for contamination by drips or spills. DISCUSSION: Besides some differences in sensitivity and handling, all four AgPOCTs showed acceptable performance in high viral load samples. However, due to the significantly lower sensitivity compared to RT-qPCR, a careful consideration of pro and cons of AgPOCT has to be taken into account before clinical implementation.

(CI95%: 91.5 -98.9) for test III. Regarding handling, test I obtained the overall highest scores, while test II was considered to have the most convenient components. Of note, users considered all assays, with the exception of test I, to pose a significant risk for contamination by drips or spills. Discussion: Besides some differences in sensitivity and handling, all four AgPOCTs showed acceptable performance in high viral load samples. However, due to the significantly lower sensitivity compared to RT-qPCR, a careful consideration of pro and cons of AgPOCT has to be taken into account before clinical implementation.

Accuracy, Handling, SARS-CoV-2, rapid antigen test AgPOCT

RT-qPCR: Reverse transcription-polymerase chain reaction, AgPOCT J o u r n a l P r e -p r o o f Introduction: The SARS-CoV-2 pandemic continues to spread at accelerating pace [1] , posing an unprecedented challenge for health care systems around the globe. Reverse transcription-quantitative polymerase chain reaction (RT-qPCR) analysis remains the gold standard for diagnosis of SARS-CoV-2 infection [2] ; however, limited supply of material and specialized personnel, as well as frequent reporting delays bring about the need for alternative testing formats. Rapid Antigen Point of Care Tests (AgPOCTs) represent one such alternative and have been pushed into the market by countless distributors in recent months. The tally of CE-cleared AgPOCT products has mounted to just over 200 during this time [3] , leaving potential users mostly in the dark about their performance in real-world situations as independent comparison studies were unable to keep pace with the flood of novel tests [4, 5] .

Objective: The aim of this study was to evaluate and compare handling, analytical-and clinical performance of four commercial rapid antigen point of care tests (AgPOCT) distributed by established diagnostics firms.

Clinical samples and ethics: Single center non-interventional diagnostic multiple-gate study. AgPOCT and RT-qPCR procedures: Four different CE-labeled AgPOCT kits were compared (Table 1) . Swabs supplied with the AgPOCT kits were immersed in patient oropharyngeal/nasopharyngeal samples for approximately 10 seconds before all further steps of the tests were carried out according to instructions by manufacturers. The results of the AgPOCTs were read by two unblinded operators. It has to be noted, that this particular protocol of dipping nasopharyngeal swabs into UTM samples is a deviation from the recommended sample matrix and or handling recommendations by the manufacturer.

Based on previous experience, we postulated that roughly 100µL of UTM sample are carried over by the procedure described above. To allow for a meaningful clinical correlation, 'effective' viral loads/swab were calculated by applying a factor of 0.1 to all PCR measurements from original samples. This was done in an effort to generate results more representative of a real-life testing procedure. A commercial SARS-CoV-2 RT-qPCR assay (Roche cobas SARS-CoV-2 IVD) was used as reference, performed directly from UTM samples.

For limit of detection experiments, a serial dilution of cell-free virus solution containing SARS-CoV-2 strain HH-1 [6] in UTM was prepared and subjected to all four tests with five repeats per dilution step. As RT-qPCR reference method, the cobas6800 SARS-CoV-2 IVD assay was used in conjunction with quantitative external control material by Instand e.V.

(Düsseldorf, Germany) to allow for absolute quantification, as previously described [7, 8] . To evaluate the ease of handling and implementation of AgPOCTs into clinical routine we conducted a user survey employing a questionnaire (see supplementary Table S1) Table 2 ). Limit of Detection (LOD) using cell culture derived SARS-CoV-2 virus was in line with these results showing similar analytical performance with best performance for test IV (see suppl. Figure S1 ). All assays showed a specificity of 100%, with the exception of test III (97% (CI95: 91.5% -98.9%), see table 2). In a set of samples that all contained > 10 6 copies/swab (n= 32 for test I,II and III and n=28 for test IV; median duration from symptom onset to sampling 6 days (IQR 2-9)), clinical sensitivity rose to 100% (CI95: 87% -100%), 92.3% (CI95: 75.8% -97.8%), 92.3% (CI95: 75.8% -97.8%) and 100% (CI95: 85.7% -100%) for tests I, II, III and IV, respectively.

Handling: The median of the results for each item is illustrated in figure 3 . Test I was considered the overall easiest to use followed by test II. Test III scored lowest overall. Of note, users considered all assays to pose a significant risk for contamination by drips or spills (see Figure 2 ).

Here we report a head-to-head comparison of performance and handling for four rapid AgPOCTs of several major distributors in Europe, using the latest version of CE-labeled kits.

All tests were able to detect 10 6 or more copies/swab with high reliability (95%), implying that patients with high viral loads can be identified with acceptable accuracy. This is in line with previous studies [4, 5] .

Relative clinical sensitivity of rapid SARS-CoV-2 antigen tests has repeatedly fallen short of optimistic manufacturer claims [4, [9] [10] [11] , in large parts due to differences in patient characteristics in respective cohorts and artificial conditions when performing AgPOCT. This also applies to this study, as e.g. antigen tests were performed from clinical UTM samples.

Still, the lack of an amplification step inherently entails significantly higher analytic limits when compared to RT-qPCR, which is expected to be 100 -1000 times more sensitive than an AgPOCT. However, in the context of increased frequency of testing, lower sensitivity tests may can be just as effective as highly sensitive ones in identifying individual infections and outbreaks, as suggested by mathematical modeling studies [12] .

All AgPOCT evaluated in this study showed largely comparable clinical and analytical sensitivity, with the exception of Test IV (Siemens). The slight edge Test IV had in LoD experiments translated into a moderately increased positive agreement with RT-qPCR (of 54%) compared to the other tests (lower than 50%). It has to be noted that fewer clinical samples were tested with Test IV as it only became available when experiments were already underway, thus providing only an incomplete dataset to compare with the other tests.

Furthermore, the limited sensitivity of the assays can also be justified by the fact that half of the samples were collected at least one week after symptom onset, as shown in other studies [13] .

Besides analytical properties, handling and practical application were evaluated as part of this study. While all AgPOCTs were deemed "easy to use", Test I was considered slightly better than the rest, while Test II was considered to have the most convenient components.

We were surprised to notice that most users reported a high risk of contamination (by drips or spills). This aspect should not be underestimated in particular considering AgPOCT are to J o u r n a l P r e -p r o o f be used outside the health care sector and might also be performed by untrained personnel lacking experience in dealing with infectious materials. Especially in a context where test performance is largely identical, practical aspects should not be neglected when choosing which test to employ, as minor inconveniences in performing a single test can add up to massive delays when performing them in large numbers.

In conclusion, we present analytical and clinical evaluation of four rapid SARS-CoV-2 antigen tests, as well as a user survey for practical application. All tests demonstrated largely similar performance and are expected to detect high viral loads with adequate reliability. RT-qPCR remains the gold standard to definitively confirm or rule out infections due to its significantly higher sensitivity and specificity.

J o u r n a l P r e -p r o o f 

Is a second wave hitting Europe? | The BMJ

Laboratory testing for 2019 novel coronavirus (2019-nCoV) in suspected human cases

Head-to-head comparison of four antigen-based rapid detection tests for the diagnosis of SARS-CoV-2 in respiratory samples

Comparison of seven commercial SARS-CoV-2 rapid Point-of-Care Antigen tests | medRxiv

Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system

Clinical evaluation of a SARS-CoV-2 RT-PCR assay on a fully automated system for rapid on-demand testing in the hospital setting

Pushing beyond specifications: Evaluation of linearity and clinical performance of the cobas 6800/8800 SARS-CoV-2 RT-PCR assay for reliable quantification in blood and other materials outside recommendations

Evaluation of rapid antigen test for detection of SARS-CoV-2 virus

Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis

Van den Bruel, Cochrane COVID-19 Diagnostic Test Accuracy Group, Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

Rethinking Covid-19 Test Sensitivity -A Strategy for Containment

Predicting infectious SARS-CoV-2 from diagnostic samples

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.