key: cord-0764005-yx3diu9i
authors: Ifko, Martina; Tkalčić Švabek, Željka; Friščić, Iva; Paro, Mirjana Mariana Kardum; Prkačin, Ingrid; Đerek, Lovorka; Livun, Ana; Skvarč, Miha
title: Diagnostic validation of two SARS-CoV-2 immunochromatographic tests in Slovenian and Croatian hospitals
date: 2021-10-03
journal: Croat Med J
DOI: 10.3325/cmj.2021.62.513
sha: 21727aecdf09753020e9d0b34c52fb0af21d6357
doc_id: 764005
cord_uid: yx3diu9i

AIM: To diagnostically validate two point-of-care (POC) rapid antigen tests for SARS-CoV-2 by comparing their results with those of laboratory-based real-time polymerase chain reaction tests (RT-PCR). METHODS: The study enrolled 455 patients from two Slovenian and two Croatian hospitals. The NADAL COVID-19 Ag Test (Nal von Minden, Moers, Germany) and ALLTEST COVID-19 Antigen Test (Hangzhou ALLTEST Biotech Co., Ltd, Hangzhou, China) were diagnostically validated in emergency care departments of two Slovenian hospitals, while only ALLTEST COVID-19 Antigen Test was validated in two Croatian hospitals. RESULTS: The antigen test results were in very good agreement with the RT-PCR results (Cohen's Kappa between 0.747 and 0.891 for the NADAL COVID-19 and between 0.820 and 0.954 for the ALLTEST COVID-19). The NADAL COVID-19 Ag Test had the sensitivity between 66.67% and 92.31%, with a negative predictive value between 85.51% and 99.2%. The ALLTEST COVID-19 Antigen Test had the sensitivity between 81.39% and 91.11%, with a negative predictive value between 85.45% and 98.78%. CONCLUSION: The antigen tests are practical and reliable screening assays for SARS CoV-2 in emergency care departments. Both antigen tests can be used as screening tests to reduce the number of patients waiting for RT-PCR results. Even more, they can be used to quickly isolate COVID-19 patients and reduce hospital transmissions.

The COVID-19 pandemic has entered every pore of health care systems worldwide and endangered their functioning. An adequate COVID-19 response should entail timely testing and sufficient testing capacities (1) . A major obstacle in the current fight against SARS-CoV-2 is a much greater number of patients compared with the number of realtime polymerase chain reaction (RT-PCR) tests that can be performed. Therefore, we need quick and accurate alternative tests, such as antigen detection tests, to detect the virus presence in respiratory tract samples. Antigen tests provide rapid results and can be used efficiently by trained non-laboratory personnel as point of care tests (POCT). Rapid results may help in stopping the uncontrolled virus spread in hospitals (2) . The World Health Organization (WHO) has set the standards for the use of antigen tests in terms of their negative and positive predictive value and acceptable diagnostic accuracy (3) . Nevertheless, most of the available antigen tests underwent only emergency use authorizations, with limited data available on clinical validations. In addition, the number of published studies on the diagnostic performance of antigen tests vs RT-PCR is still low. In this article, we present the diagnostic validation data for two POC antigen tests in two Slovenian emergency care departments (General Hospital Jesenice and General Hospital Slovenj Gradec) and two Croatian hospitals (University Hospital Merkur and University Hospital Dubrava, both in Zagreb).

In General Hospital Jesenice, 106 patients were tested with the NADAL COVID-19 Ag Test (Nal von Minden, Moers, Germany) and 90 patients with ALLTEST COVID-19 Antigen Test (Hangzhou AllTest Biotech Co., Ltd, Hangzhou, China). All results were compared with those of Seegene Allplex 2019-nCoV test (SeeGene, Seoul, South Korea). In General Hospital Slovenj Gradec, 92 samples were tested with the NADAL COVID-19 Ag Test and 94 with ALLTEST COVID-19 Antigen Test, and the results were compared with those of Cobas 6800 SARSCoV-2 Test (Roche, Indianapolis, IN, USA). University Hospital Merkur and University Hospital Dubrava used only one test, the NADAL COVID-19, for 51 specimens and compared the results with those of SARS-CoV-2 RT-PCR assay (PCRBiosystems Ltd, London, UK), performed according to a published RT-PCR protocol (2) .

Diagnostic validation of the NADAL COVID-19 test started in the middle of September 2020. Clinical Hospital Jesenice started using the antigen tests in the middle of October 2020, when the region experienced an uncontrolled disease spread. By the end of October, the 14-day incidence was 1970.8 cases per 100 000 inhabitants, reflecting an increase in positive community cases by 53% per week (4).

The study enrolled patients with at least one clinical sign of COVID-19 (fever above 37.5°C, fatigue, rhinorrhea/rhinitis, dry cough, dyspnea, loss of taste and smell, or gastrointestinal problems with at least one sign of upper respiratory tract disease). Croatian hospitals collected two swabs at the same time, one of which was used for RT-PCR testing in the laboratory and the other for POC antigen assay testing. In the Slovenian hospitals, the antigen test was performed at first for POCT, and the patients were asked to wait for the results. If the result was positive or presumed false negative, a second and contralateral nasopharyngeal swab was collected within an hour of the first sample collection, by the same health care worker, for RT-PCR testing in local public health microbiology laboratories. In Croatian hospitals, all results were compared with RT-PCR tests.

Patients who had COVID-19 symptoms for more than five days were excluded as it was advised in the instruction manuals for both tests. Statistical analysis was performed with MedCalc Statistical Software, version 18.11.6 (Med-Calc Software bvba, Ostend, Belgium).

The study was approved by the Ethics Committee of General Hospital Jesenice for Slovenian hospitals (2020-02) and by the Ethics Committees of Merkur University Hospital (11593) and Dubrava University Hospital (2020/1012-04). Study participants gave informed written or oral consent.

In General Hospital Jesenice, the positive percent agreement rate (PPA) and negative percent agreement rate (NPA) of the NADAL COVID-19 Ag Test vs RT-PCR were 66.76% (95% CI 35.88%-90.07%) and 100% (95% CI 96.15%-100%), respectively ( ity were observed in a hospital with an infectious disease department. This finding could be explained by the fact that this hospital clinically evaluated which patients were eligible for SARS CoV-2 testing under the supervision of an infectious disease specialist. This hospital probably had the best selection of patients for testing based on clinical symptoms and signs, and the samples were only sent to RT-PCR testing when it was needed to resolve false-negative antigen test results or to confirm antigen-positive test results. At the time of testing, Croatia had a low number of positive samples and a lower COVID-19 incidence than Slovenia. In total, the NADAL COVID-19 Ag Test exceeded the minimum sensitivity and specificity of 80% and 97% as recommended by the WHO (3).

The ALLTEST COVID-19 Antigen Test proved to be an assay of choice in the two Slovenian hospitals. The assay had a good sensitivity, very good in General Hospital Slovenj Gradec (92.31%), with an excellent overall NPV of 93.43%. Cohen`s Kappa was high (between 0.887 and 0.954), showing a nearly perfect agreement with RT-PCR. The ALLTEST COVID-19 is a user-friendly assay since it requires only 10 seconds of incubation. Furthermore, the red test line cannot be mistakenly read as a false positive, which increases the test's reliability. Recently published laboratory-based studies from different countries validating more than one antigen assay for SARS-CoV-2 also showed a high specificity level, ranging from 99.5% to 100%, but a wide range of sensitivity, from 0 to 93.9% (6) (7) (8) (9) (10) . A laboratory-based study showed the sensitivity and specificity of the BD Veritor SARS CoV-2 antigen test (Becton Dickinson, NJ, USA), which uses an analyzer to detect a signal, to be 76.32% (95% CI 59.39%-87.97%) and 99.53% (95% CI 97.01%-99.98%), respectively, in specimens collected from patients 0-7 days after symptom onset (9) . A true POCT study performed in the Netherlands and Aruba compared the Panbio COVID-19 Ag Rapid Test (Abbott, Chicago, IL, USA) and 2-gene target RT-PCR in 1575 mildly symptomatic participants (10) . In both settings, the assay specificity was 100% (95% CI 99.7%-100%). Test sensitivity was 72.6% (95% CI 64.5%-79.9%) in the Netherlands and 81.0% (95% CI 69.0%-89.8%) in Aruba (10) .

Another laboratory-based study performed on nasopharyngeal swabs taken at the emergency care department of two infectious disease centers in Italy assessed the performance of Standard Q COVID-19 Ag assays (SD-Biosensor, Suwon, South Korea) in 185 randomly selected patients. The sensitivity was 72.1%, while the specificity was 100% (11) . In a POC study evaluating the clinical per-formance of the BD Veritor SARS CoV-2 assay in 475 COVID-19 symptomatic adults at a municipal health service center in the Netherlands, the overall clinical specificity was 100% (95% CI 98.9%-100%) and the sensitivity was 80.7% (95% CI 73.2%-86.9%) compared with RT-PCR (12) .

Clinically validated POCT SARS CoV-2 antigen tests could be a good screening tool for use in pandemics. The SARS-CoV-2 pandemic highlighted the shortage of experienced staff as a limitation of RT-PCR tests, which created long waiting times for RT-PCR results (11) . All four hospitals in our study used the antigen tests successfully for triage and to lower the number of patients waiting for PCR. By testing all the admitted patients, hospitals could also reduce the number of hospital transmissions with efficient rapid isolation of antigen-positive patients not waiting for RT-PCR results.

The current study was limited by our inability to collect cycle thresholds for semiquantitative estimation of viral concentrations in the specimens, which could explain the low sensitivity of the NADAL COVID-19 Ag Test observed in Slovenian hospitals. Healthcare workers who collected specimens and conducted the testing were not equally well trained, which could have contributed to the relatively low sensitivity of both tests in Jesenice.

In conclusion, our results show that both diagnostically validated SARS-CoV-2 antigen tests exceeded the WHO criteria for minimum sensitivity and specificity of 80% and 97%, respectively. Cohen`s Kappa coefficients for the NA-DAL COVID-19 and ALLTEST COVID-19 tests were 0.834 and 0.903, respectively, showing very good agreement with RT-PCR. We demonstrated that sound clinical judgment with the help of antigen POC test can differentiate true negative from false-negative results.

In our opinion, the ALLTEST COVID-19 Antigen Test has better performance than the NADAL COVID-19 Ag, as shown by a higher Cohen`s Kappa (0.903). In General Hospital Slovenj Gradec, which at the time of testing faced the highest COVID-19 incidence and tested many patients with COVID-19 pneumonia, Cohen`s Kappa was 0.954. As already mentioned, the ALLTEST COVID-19 is a test of choice for both Slovenian hospitals and is also used for mass testing and on the primary health care level. ethical approval given by the Ethics Committees of General Hospital Jesenice for Slovenian hospitals (2020-02) and of Merkur University Hospital (11593) and Dubrava University Hospital (2020/1012-04).

declaration of authorship MI, AL, and MS conceived and designed the study; ŽTS, IF, MMKP, IP, LD, AL, and MS acquired the data; MI, AL, LD, and MS analyzed and interpreted the data; MI, AL, LD, and MS drafted the manuscript; ŽTS, IF, MMKP, and IP critically revised the manuscript for important intellectual content; all authors gave approval of the version to be submitted; all authors agree to be accountable for all aspects of the work.

Coronavirus resurgence: Commission steps up action to reinforce preparedness and response measures across the eu

detection of 2019 novel coronavirus (2019-nCoV) by realtime RT-PCR

european Centre for disease Prevention and Control (eCdC). diagnostic testing and screening for SARS-CoV-2

Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. Interim guidance

SLedILnIK Covid-19 tracker

The measurement of observer agreement for categorical data

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

head-to-head comparison of four antigen-based rapid detection tests for the diagnosis of SARS-CoV-2 in respiratory samples

Clinical evaluation of bd Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-of-care test

Real-life validation of the Panbio CoVId-19

Antigen Rapid Test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection

urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the Sd-biosensor antigen test for SARS-CoV-2 (published online ahead of print

Performance evaluation of a SARS CoV-2 rapid antigen test: Test performance in the community in the netherlands medRxiv posted

Acknowledgments We thank all health care workers in the participating institutions for their hard work in extreme situations and for performing the antigen tests in addition to their other duties.

Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.