key: cord-1023404-8b1b200q
authors: Matsuda, Elaine Monteiro; de Campos, Ivana Barros; de Oliveira, Isabela Penteriche; Colpas, Daniela Rodrigues; Carmo, Andreia Moreira dos Santos; Brígido, Luís Fernando de Macedo
title: Field evaluation of COVID‐19 antigen tests versus RNA based detection: Potential lower sensitivity compensated by immediate results, technical simplicity, and low cost
date: 2021-04-08
journal: J Med Virol
DOI: 10.1002/jmv.26985
sha: 35c6e362dce8b73e5dd8180fbc8e136dffd406a5
doc_id: 1023404
cord_uid: 8b1b200q

One year into the coronavirus disease 2019 (COVID‐19) pandemic, diagnostic strategies, although central for contact tracing and other preventive measures, are still limited. To meet the global demand, lower cost and faster antigen tests for severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) detection are a convenient alternative to the gold standard reverse transcription‐polymerase chain reaction (RT‐PCR) assay. We tested laboratory‐based RT‐PCR RNA detection and two rapid antigen detection (RAD) tests, based on the immunochromatography test for nucleocapsid protein of SARS‐CoV‐2 (COVID‐19 Ag ECO Test, ECO Diagnóstica, and Panbio COVID‐19 Ag Rapid Test Abbott). Paired collection and testing were done in a small prospective open study in three clinical services in São Paulo, constituted of mostly symptomatic volunteers at collection (97%, 109/112) for a median of 4 days (interquartile range: 3–6), ranging from 1 to 30. Among the 108 paired RT‐PCR/RAD tests, results were concordant in 96.4% (101/108). The test's performance was comparable, with an overall sensitivity of 87% and a specificity of 96%. These observations add to other data that suggest that antigen tests may provide reasonable sensitivity and specificity and deserve a role to improve testing strategies, especially in resource‐limited settings.

To control the COVID-19 pandemic, improvement of detection with easy, rapid, and cost-efficient approaches is urgently required. 1 There are several obstacles to proper molecular detection by real-time reverse transcription-polymerase chain reaction (RT-PCR) of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the gold standard for laboratory confirmation of infection. Serological tests are erroneously adopted as a diagnostic tool by some, giving a false impression that they have been properly evaluated for infection. 2 In contrast, rapid antigen detection (RAD) tests may identify acute COVID-19 by recognition of the virus antigen itself. They are less laborious, require short training and are relatively inexpensive when compared to Doubts about the quality of the rapid test results are one of the obstacles to its acceptance. We conducted a comparative and paired study of RT-PCR and antigen tests in three services with different profiles. The objective was to evaluate the feasibility of the use of SARS-CoV-2 RAD tests and compare their performance to laboratorybased viral RNA detection test (LBT).

The study prospectively enrolled consecutive health service users, suspected of being infected with SARS-CoV-2 with up to 10 Test Abbott (Panbio). In some cases, LBT was repeated. For the RT-PCR test, the volunteer was asked to gargle with 10 ml of saline solution, 3 ml of which were collected at a conic tube and combined with two swabs, one for each nostril (nasopharynx). This material was refrigerated and sent to the reference laboratory. Briefly, extracted RNA was submitted to RT-PCR using different commercial kits based on protocols from two reference laboratories. The test used aimed at the detection of viral genes (e.g., N, E, and RdRp), including GeneFinder COVID-19 Plus RealAmp kit (OSANG Healthcare) and Allplex (Seegene), which are based on the Charité protocol. 4 For some samples, it was applied IDT primers and GoTaq Probe 1-Step RT-qPCR System (Promega), based on the CDC protocol 5 to detect two genes (N1 and N2). The samples were considered positive according to the acceptance criteria of each kit. As recommended for the Influenza assay, Human RNAse P allowed the assessment of the quality of the sample and the presence of inhibitors, and reactions with a cycle threshold (C t ) up to 35 were considered valid.

Both RAD was carried out as specified by the manufacturers, with a single swab for collecting nasopharynx secretion in one nostril, subsequently immersed in the buffer solution, and then dripped onto the test plate. RAD is a qualitative membrane-based immunoassay (immunochromatography) for the detection of Nucleocapsid protein of SARS-CoV-2 in nasopharyngeal samples. After 15 min the membrane was observed for control and test bands, which were differentiate by its intensity as none (not reactive, no band was observed); low (less visible than the control); and high (more visible than the control). The test was considered positive if the control was reactive and any intensity was observed at the test band.

We used the Shapiro-Wilk normality test to evaluate if data were normally distributed. Continuous variables were presented as the median and interquartile range (IQR) (25-75), Mann-Whitney U test to compare medians. Categorical variables, expressed as numbers (percentages), were compared by the χ 2 test or Fisher's exact test as appropriate.

Interrater agreement was calculated using kappa statistics. A two-sided p-value < .05 were considered significant. Statistical analyses were performed with SPSS v22 or STATA 10. Specificity and sensitivity with 95% confidence intervals (95% CI) and calculated the positive predictive value (PPV), considering the RT-PCR result as the gold standard, using MedCalc (https://www.medcalc.org/calc/diagnostic_test.php). 3.1 | Demographic, clinical, and laboratory data are shown in Table 1 Upon clinical revision, some cases did not meet enrollment criteria and had no COVID-19 related symptoms or had longer than 10 days of symptoms. However, almost all cases were symptomatic at collection (109/112 97%), with a median of 4 (IQR: 3-6) days of symptom, ranging from 1 to 30. Only 3 hospitalized cases had symptoms over 10 days: 11 days, with an Eco reactive test and RT-PCR detected, and two cases with both nonreactive Eco and an RT-PCR not detected from 20 to 30 days of symptoms.

In seven cases the results of the RAD and RT-PCR were divergent (Table 2) . RAD was reactive in three cases (ECO, N = 2 and Panbio, n = 1) with RT-PCR not detected. In two of these three cases, the intensity of the Ag test band was tenuous (Panbio and ECO). The third case with a band intensity greater than that of the control (Panbio) had a condition compatible with carcinomatous lymphangitis resulting from esophageal neoplasia with a nonspecific lung tomography finding.

We compared the lower C t obtained from the three RT-PCR targets to SARS-CoV-2 genes. The C t from the discordant results (negative antigen/positive RT-PCR) was similar to that of concordant cases (25 vs. 27; p = .6).

We evaluated the positive and negative predictive value of RAD use at five different hypothetical prevalence rates (Table 3) . For lower prevalence, like 10%, the tests have a PPV) below 80%, but at higher prevalence scenarios, as above 50%, the PPV values are above 94%.

In this study, we compared face-to-face two test modalities to detect SARS-CoV-2 infection, a laboratory-based RT-PCR test, and two commercially available antigen tests. Although the test results showed some discrepancies, with positive antigens tests not detected in the RNA test, as well as negative antigens with positive RNA detection, the overall agreement of the procedures was high (96%), suggesting reasonable comparability. With many products in the market, few In some of these studies, as well as evaluations that assessed other rapid antigen tests with lower sensitivity, the discordance of results to viral or RNA presence 10 were mostly related to cases with lower viral load, reinforcing the notion that these tests would perform better in higher viral load, more infectious cases. 11 Of the cases with discordant results, three were considered false-positive RAD based on the gold standard methodology.

Although antigen assays are more susceptible to potential falsepositive results, one of these cases had anosmia and had a subsequent seroconversion (confirmed by Elecsys Anti-SARS-CoV-2 immunoassay test; Roche) but it was not re-tested for antigen or RNA. Moreover, it was observed two cases of positive RAD, which were non-detected RT-PCR, but after few days a new sample for RT-PCR was collected and then the presence of SARS-CoV-2 was detected. For this reason, it was considered that these two cases presented results in agreement for both methodologies.

The findings in this study suggest that RAD sometimes may indicate the viral presence before RT-PCR.

We tested the predictive value at different prevalent scenarios.

As can be seen in Table 3 , only at a prevalence of 10%, that Brazil had at the early months of the pandemic, have these tests a low PPV.

As the pandemic expands and prevalence increases, so does the power of a positive test result to represent a true positive. They may help to guide public policies according to the prevalence observed in a given area.

Our study has many limitations, including the small sample size, the inability, due to limitation of tests available, to compare head-tohead the two RADs evaluated, as well as a limited follow-up of cases to confirm testing results or to evaluate clinical progression. However, the study had some distinctions that are worth mentioning.

First, we did the test during routine services after brief, in-service training of nurses and other health care workers, suggesting the simplicity and adaptability of this testing modality to real-world conditions. We opted for replacing the nasopharynx swab with saline gargle in our routine and at this study, due to recurrent scarcity of swabs, but also to minimize the exposure of health workers to the riskier oropharyngeal swab collection. Another point is the reports of a higher viral load in the samples obtained by gargle compared to those by swab pharyngeal. 12, 13 Therefore, by combining the two potential sources of RNA we may maximize sensitivity and decrease health worker risk. Our study involved three health units with a lowtech laboratory, with no capacity to carry out RNA-based tests.

These clinical sites had reference laboratories to conduct RNA tests, but that is not a situation in many parts of the country as well as in many areas of the world. Although conducted in a specific area of week, antibody detection tests may provide some information for population-level surveys but may be less suitable for individual assessment. 2 Since the appearance of antibodies depends on time to elicit an immune response, the diagnosis of COVID-19 by serological methods is generally more efficient after eight days of illness when the sensitivity of serological assays exceeds that of nucleic acid tests. 8 Considering that, for mild cases, the first week represents the period of greatest transmissibility, 14 these tests are not applicable to indicate isolation. Several rapid tests based on the detection of antibodies, lateral flow immunoassay, from different companies are available, but many lack adequate validation of its performance regarding both sensitivity and specificity. 15 In places where rapid antibody tests are being used to minimize diagnosis bottlenecks, the option for an antigen test may be more adequate.

Brazil has never implemented a comprehensive strategy to tackle the COVID-19 pandemic, with a lack of national coordination, the result of a government that denies the scale of the epidemic and the science. The Brazilian Ministry of Health attributes the number of deaths to noncompliance with its guidelines for early treatment with hydroxychloroquine. 16 There are few preventive initiatives at the municipal and state level, and some with different restrictive strategies. For example, there are municipalities, separated by a street, in which a shopping center is open on one side of the street, and the other side of the street, with restrictions on non-essential activities. The increasing number of cases, deaths, and the stretch of the health system's capacities to its limit constitutes a serious threat to the sustainability of the health system and the ability to respond appropriately to the situation, express the result of these incoherencies. Proper testing and contact tracing may lessen this burden.

In a scenario of case screening in the first week of symptoms related to COVID-19, the use of rapid antigen testing shows good comparability with laboratory-based RNA detection. Both tests evaluated, the COVID-19 Ag ECO Test (ECO Diagnóstica) and the Panbio COVID-19 Ag Rapid Test (Abbott), approaches the criteria defined by World Health Organization for these tests of 80% sensitivity and 97% specificity. 17 The logistical advantages of point-of-care testing can supersede its limitations and provide a valuable tool to improve the diagnosis of COVID-19, contributing to the control of transmission in the community.

The study was supported in part by grant CNPq/MS-DIAHV N°2 4/2019−442776/2019-5 and FAPESP 2018/14384-9.

Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-Biosensor antigen test for SARS-CoV-2

Antibody testing for COVID-19: a report from the National COVID Scientific Advisory Panel

Evaluation of the Panbio COVID-19 rapid antigen detection test device for the screening of patients with COVID-19

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

2019-Novel coronavirus (2019-nCoV) real-time rRT-PCR panel primers and probes

Field evaluation of a rapid antigen test (Panbio COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres

Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms

COVID-19 Rapid Antigen Test at hospital admission associated to the knowledge of individual risk factors allow overcoming the difficulty of managing suspected patients in hospitals COVID-19 rapid antigen test facilitates the management of suspected patients on hospital admission

Evaluation of nine commercial SARS-CoV-2 immunoassays. medRxiv

Evaluation of rapid antigen test for detection of SARS-CoV-2 virus

Transmissibility of COVID-19 depends on the viral load around onset in adult and symptomatic patients

Gargle lavage as a safe and sensitive alternative to swab samples to diagnose COVID-19: a case report in Japan

Gargle lavage as a viable alternative to swab for detection of SARS-CoV-2

Taiwan COVID-19 Outbreak Investigation Team. Contact tracing assessment of COVID-19 transmission dynamics in Taiwan and risk at different exposure periods before and after symptom onset

Serology assays to manage COVID-19

Ministério da Saúde Secretaria Executiva Gabinete da Secretaria Executiva. Orientações para manuseio medicamentoso precoce de pacientes com diagnóstico da COVID-19

WHO. COVID-19 target product profiles for priority diagnostics to support response to the COVID-19 pandemic v.1.0. 2020

Field evaluation of COVID-19 antigen tests versus RNA based detection: Potential lower sensitivity compensated by immediate results, technical simplicity, and low cost

The study was approved by the institutional ethical committee