key: cord-0873874-1ausr4sh
authors: Avgoulea, Kalliopi; Beredaki, Maria-Ioanna; Vourli, Sophia; Siopi, Maria; Siafakas, Nikolaos; Pournaras, Spyros
title: Field Evaluation of the New Rapid NG-Test(®) SARS-CoV-2 Ag for Diagnosis of COVID-19 in the Emergency Department of an Academic Referral Hospital
date: 2022-04-25
journal: Front Public Health
DOI: 10.3389/fpubh.2022.840984
sha: 058eeaf8a006ca1137e634632117f981d019172c
doc_id: 873874
cord_uid: 1ausr4sh

BACKGROUND: As the COVID-19 pandemic resurges affecting large numbers of patients, rapid, and accurate diagnosis using point-of-care tests is very important. OBJECTIVES: To evaluate the NG-Test(®) SARS-CoV-2 Ag (NG-Test) immunoassay for qualitative detection of SARS-CoV-2 antigen in nasopharyngeal (NP) and oropharyngeal (OP) samples compared with RT-PCR, in patients attending the Emergencies of an academic referral hospital. METHODS: All adult ambulatory patients presenting to the Emergencies of “Attikon” University hospital (Athens, Greece) within three consecutive hours per day between December 2020 and March 2021 and for whom SARS-CoV-2 PCR testing was requested were included. Two NP and one OP samples obtained from each participant were analyzed to determine the diagnostic performance [sensitivity, specificity, positive/negative predictive values (PPV/NPV)] of the NG-Test (NP/OP swabs) in comparison to the reference RT-PCR (NP swab). RESULTS: Overall, 134/263 (51%) patients tested were RT-PCR positive, whereof 108 (overall sensitivity 81%, 95% CI 73–87%) were NP NG-Test positive (PPV 99%, NPV 83%) and 68 (overall sensitivity 51%, 95% CI 42–59%) were OP NG-Test positive (PPV 100%, NPV 66%). The test's specificity (95% CI) was 99% (95–100%) and 100% (96–100%) for NP and OP swabs, respectively. The assay's sensitivity (95% CI) for high viral load (C(t) ≤25) was 99% (92–100%) and 71% (60–81%) for NP and OP swabs, respectively. CONCLUSIONS: NG-Test using NP swabs detected almost all patients with high viral loads, showing satisfactory performance as a point-of-care test for NP samples obtained from patients with acute infection.

Two years after the first reported case of SARS-CoV-2 infection in Wuhan, China (1), COVID-19 remains an ongoing global pandemic (2) . Despite the vaccination strategies and other mitigation measures implemented regionally, exponential resurgence has been repeatedly reported worldwide (3, 4) .

The gold standard for COVID-19 diagnosis is the direct detection of SARS-CoV-2 viral RNA using nucleic acid amplification techniques (real-time RT-PCR) (5, 6) . However, PCR testing during periods of high COVID-19 incidence has been recently questioned due to the quite long turnaround time, which strongly influences the function of Emergency Departments (EDs). Further limitations, particularly for low-resource settings, are the need for expensive equipment and reagents as well as the requirement for specialized biocontainment laboratories, operated by highly trained personnel (7) . To address this reality, simple and rapid antigen detection tests (RADTs) are recommended by the World Health Organization (WHO) for use in the community for primary case detection, contact tracing and outbreak investigations (8) .

Based on these grounds, the aim of the current study was the clinical evaluation of the NG-Test R SARS-CoV-2 Ag (NG-Test), a novel lateral flow immunoassay for the rapid detection of SARS-CoV-2 antigen (nucleocapsid protein), in both nasopharyngeal (NP) and oropharyngeal (OP) samples of ambulatory patients who visited the ED of our hospital, in comparison to realtime RT-PCR being implemented for COVID-19 diagnosis in our setting.

The study included all adult ambulatory patients who presented to the ED of "Attikon" University hospital, Athens, Greece, within three consecutive hours per day between December 2020 and March 2021 and for whom SARS-CoV-2 PCR testing was requested by the clinician in charge, without any selection criteria.

A total of three respiratory specimens (two NP and one OP swabs) were collected from each participant after giving informed consent. One NP and one OP collection were screened by the NG-Test (NG-Biotech Laboratories, 35480, Guipry, France), as recommended by the manufacturer. Briefly, two test devices labeled with the patient's name and marked with a distinctive N (for NP) or O (for OP) sign, were placed on a clear and flat surface. After specimen collection, the swabs were immediately unloaded by rotation in the extraction buffer, while squeezing them through the extraction tube; before removal, they were firmly squeezed against the upper tube walls. Immediately after the extraction, a dropper cap was attached to the extraction tubes and 3 drops (∼120 µL) of the extracted sample were transferred to the S/R well of the test cassettes. A timer was set and the results Abbreviations: ED, Emergency Department; NP, nasopharyngeal; OP, oropharyngeal; NG-Test, NG-Test R SARS-CoV-2 Ag; RADT, rapid antigen detection test.

were read out visually after incubation at room temperature for exactly 15 min. Low-intensity lines in the test area, in presence of a positive red line control, were interpreted as positive results.

According to the protocol implemented in our setting for COVID-19 molecular diagnosis, a NP swab mixed in viral transport media was immediately transported to the Clinical Microbiology laboratory of our hospital, which serves as the referral COVID-19 diagnostic center for the 2nd Regional Health Authority of Greece. The RT-PCR test that was routinely applied in our laboratory during the study period and was used as the reference method for comparison, was the Certest Viasure SARS-CoV-2 kit (CerTest Biotec, Spain), performed on a Rotor-Gene Q thermal cycler (Qiagen) following the manufacturer's instructions. PCR cycle threshold (C t ) values ≥35 were considered negative, while positive C t values were recorded. The laboratory personnel were blinded to the identity of patients that were tested by immunoassay during their ED stay and the RADT result.

The study protocol was approved by the Institutional Review Board and Bioethics Committee of "Attikon" University Hospital (ref. number EB 662/30-11-2020).

The sample size was determined by our target to test at least 100 prospective RT-PCR positive samples, as it is recommended by the WHO for diagnostic sensitivity determination (8) . Medians and interquartile ranges (IQR) were calculated for continuous variables, while frequencies and percentages were calculated for categorical parameters. Performance characteristics, including sensitivity, specificity and positive/negative predictive values (PPV/NPV), for antigen testing were assessed using two-by-two tables and the respective two-sided 95% confidence interval (CI) was estimated. Categorical agreement between RT-PCR and NG-Test was estimated and its strength was assessed by calculating the Cohen's kappa coefficient (κ). All data were analyzed using the statistics software package GraphPad Prism, version 8.0 for Windows (GraphPad Software, San Diego, CA).

Overall, 263 unique patients were enrolled in the study, whereof 148/263 (56%) were men with median age 50 years (range, 18-89; IQR, 27). The majority of patients exhibited one or more COVID-19-compatible symptoms and sought medical care for the first time (212/263; 81%) or were previously diagnosed (30/263; 11%); most of them (235/242; 97%) presented to the ED within 7 days from symptoms onset. The predominant symptom was fever (188/242; 78%), followed by cough, headache, dyspnea, diarrhea, weakness, myalgia, pharyngodynia, anosmia, ageusia, emesis, and rhinorrhea. A small proportion of patients (21/263; 8%) was asymptomatic and visited the ED as close contacts of a symptomatic patient ( Table 1) .

In total, 134/263 (51%) patients were tested positive by RT-PCR, with a median C t value 25 (range, 13-34; IQR, 8). Among them, 128/134 (96%) were symptomatic, while six were asymptomatic. 108/134 (81%) patients had a positive result in the NG-Test when NP swabs were tested resulting in sensitivity (95% Given that the overall sensitivity is strictly dependent on the distribution of C t values obtained in the RT-PCR assay within the population of specimens (9), a sub-group analysis was performed and the sensitivity was recalculated in a C t -dependent manner. In particular, based on the C t cut-off value, samples with C t ≤ 25 were designated as high viral load, C t >25-≤30 as intermediate and C t >30-<35 as low viral load (10) . The corresponding C t values and sensitivities are shown in Figure 1 . Regardless of the nature of the clinical specimen, a significant difference in the performance of the antigen assay was found in patients with different viral loads (p < 0.0001). Namely, the sensitivity (95% CI) of the NG-Test when NP swabs were tested was 99% (92-100%) for specimens with a high viral load, whereas for samples with an intermediate and low viral load, the sensitivity was calculated to 72% (55-85%) and 29% (12-52%), respectively. Accordingly, the sensitivity (95% CI) of the antigen test when OP swabs were used was 71% (60-81%), 31% (17-48%), and 10% (2-32%) for samples with a high, intermediate, and low viral load, respectively.

The moderate NPV of NP NG-Test was attributed to a total of 26 false-negative results; one (4%) came from a patient with high viral load, 10 (38%) from intermediate and 15 (58%) from low viral loads. Of the 26 false-negative samples, 14 (54%) were collected from patients being diagnosed from 4 to 14 days earlier, while 12 (46%) were collected from symptomatic, firstvisit patients with C t ≥27. The OP screening showed 66 falsenegative results; 22 (33%) from patients with high viral loads, 25 (38%) intermediate and 19 (29%) low viral loads. Twenty-three (35%) false-negative OP samples were collected from patients originally diagnosed within 2 weeks prior to ED presentation, while 43 samples (65%) from first-visit symptomatic patients, 17 of which (40%) were from patients with high viral loads, 16 (37%) intermediate and 10 (23%) low viral loads.

The appearance of novel variants of concern, such as the recently designated by WHO Omicron variant, known as B.1.1.529 (11), along with the continuous strain of healthcare systems globally and the need to rapidly diagnose in order to contain the spread, have aroused the interest of the scientific community toward rapid, accurate, cost-effective, and point-of-care techniques. In the framework of our study, the performance of the NG-Test in NP and OP samples was assessed compared to RT-PCR, presenting an overall sensitivity of 81 and 51%, respectively, and a high specificity of 99-100%. Considering that PPV is strongly dependent on the disease prevalence in the study population (12) , NG-Test had an excellent PPV (99% in NP vs. 100% in OP samples) within a population with 51% COVID-19 prevalence.

In fact, the performance of the NP NG-Test met the WHO's minimum requirements for RADTs (≥80% sensitivity and ≥97% specificity) for COVID-19 diagnosis (8) being in agreement with the manufacturer's NP performance data (sensitivity 83% and specificity 100%) (13). It also displayed a significant agreement of 90% (κ = 0.795) with the RT-PCR. Furthermore, according to a recent retrospective case control study evaluating the performance of six RADTs in NP samples, a previous version of the NG-Test showed high specificity (98.5%), similar to our results, but much lower overall sensitivity (32.3%), than that in our findings (81%) (14) . It should be noted that different antibodies were used in the current version of the NG-Test.

As for the OP swabs, the estimated sensitivity (51%) and level of agreement with the reference methodology (75%, κ = 0.503) showed moderate to weak performance of the evaluated test. Indeed, our results match the data in the current literature to a certain extent. A recent meta-analysis suggested that OP swabs should not be recommended for diagnosis of SARS-CoV-2 infection in ambulatory care due to low sensitivity [68% (95% CI 35-94%)] (15), corroborating our results. However, another recent meta-analysis showed that there was no substantial difference between NP and OP swab detection [overall positive detection 88% (95% CI 73-98%) vs. 84% (95% CI 57-100%)] (16) . Direct comparison with our findings could hardly be performed, as the latter meta-analysis included both inpatient and outpatient settings, as opposed to our study population that included outpatients with most of them having acute infections. Interestingly, the meta-analysis showed limited overall agreement between NP and OP swabs (68% dual positivity), while in our cohort, all samples detected positive by OP collection were also positive by NP sampling. The lower performance of OP NG-Test in our study could be attributed either to the fact that RT-PCR was performed in NP swabs or that the patients enrolled presented during acute infection, showing major discomfort (nausea and intense cough) during the OP sampling.

Several studies, involving different study designs, brands of RADTs, patient populations, sample sizes and COVID-19 incidence, have assessed the performance of RADTs in various healthcare settings, such as EDs (17) (18) (19) (20) (21) and primary healthcare centers (21, 22) , showing overall sensitivities ranging from 70.6 to 93.9%. Among the evaluations in the clinical context of a busy ED, a study assessed the performance of a fluorescence immunochromatographic SARS-CoV-2 antigen test (Bioeasy Biotechnology Co., Shenzhen, China), reporting overall sensitivity 93.9% (95% CI 86.5-97.4%) and specificity 100% (95% CI 92.1-100%) (20) . The discordant sensitivity compared to our results is probably due to the rather small sample size (n = 127), the fact that the RADT was performed by NP and OP swabs placed together in a universal transport medium, the low (9%) local prevalence during the study period and the detection of fluorescence by an instrument in that assay, compared with visual inspection in the NG-Test. Two studies, performed in both symptomatic and asymptomatic patients, evaluated the performance of the STANDARD Q COVID-19 Ag Test (SD BIOSENSOR, KR) (19, 20) . The first study reported overall sensitivity 82.9% (95% CI 81-84.8%) and overall specificity 99.1% (95% CI 98.8-99.3%) (19) , but the sample type used for antigen testing is not mentioned. The second study evaluated NP samples and showed overall sensitivity 70.6%, specificity 100%, PPV 100%, and NPV 87.4% (20) , which are in agreement with our findings. The performance of RADTs assessed in two more studies in adult EDs, the first by deep oro-nasopharyngeal swabs and the latter by not clear sampling, was similar: sensitivity 75.3 and 77.9%; specificity 100 and 98.1%; PPV 100 and 97.3%; NPV 89.2 and 84%, respectively (17, 21) , corroborating our results even though clinical samples of strictly symptomatic patients were tested, as opposed to our cohort (8% asymptomatic).

In accordance with field evaluations (18, (20) (21) (22) , and similarly to the performance characteristics of other RADTs (9, 25) , the sensitivity of both the NP and OP NG-Test was increased for specimens with high viral loads (C t ≤ 25), indicating that in high prevalence settings such as the EDs, the assay may be used to timely isolate positive patients, most likely contagious (24), especially when performed by NP collection (99% sensitivity). Interestingly, three RT-PCR positive asymptomatic patients with C t values <25, all contacts of confirmed cases and one reporting history of fever, headache, dizziness and diarrhea 7 days prior to the ED presentation, were detected by both NP/OP NG-Test (data not shown), suggesting that the assay could be quite beneficial in high prevalence clinical settings, where the identification of asymptomatic infections is of utmost importance. However, the moderate NPVs should also be kept in mind when interpreting the test results. Therefore, the medical history and clinical data should also be taken into account and confirmatory RT-PCR testing should be conducted in patients with high clinical suspicion of COVID-19 in high prevalence settings.

It is noteworthy that our study was carried out while B.1.1.7 was the predominant variant in Greece. As the assay under evaluation detects the nucleocapsid protein of SARS-CoV-2, which is conserved, possible new emerging virus variants with mutations in spike but conserved nucleocapsid protein, may not affect its performance.

The single-center nature of the present study could be considered as a limitation. Nevertheless, "Attikon" hospital serves as the referral COVID-19 diagnostic center for the 2nd Regional Health Authority, which comprises 23 hospitals and 267 primary healthcare structures located in Western and Southern Attica, Piraeus and the Aegean islands. The use of a single large testing center allowed standardization as well as increased quality in sample and data collection. Regarding RT-PCR, the same kits and instruments were used leading to standardized results and C t values. Of note, our study was carried out within a population with high prevalence of COVID-19 since the highest burden of SARS-CoV-2 infections in the Athens metropolitan area occurred in Piraeus and West Attica, as recently described (26) and, indeed, 51% of patients tested were RT-PCR positive. Lastly, our sample included mainly (92%) symptomatic but also asymptomatic (8%) individuals, which were not excluded since the scope of our study was focused on the appropriateness of NG-Test in a real-life setting where symptomatic population represents 75% of COVID-19 cases in Greece (23) .

Taken together, the performance of the NG-Test showed overall sensitivity of 81% for NP and 51% for OP samples, but high specificity (99-100%) and PPV (99-100%) regardless of the clinical specimen. The NP NG-Test could be used as a valuable diagnostic tool for the qualitative detection of SARS-CoV-2 in patients with acute infection given its high sensitivity (99%) in samples with high viral load (C t ≤ 25). False-negative antigen test results have been identified, specifically in samples with lower viral loads. Therefore, the medical history and clinical data should also be considered when interpreting the test results and confirmatory RT-PCR testing may be conducted in selected patients with high clinical suspicion of COVID-19 and having negative NP NG-Test result, while patients with positive NG-Test do not require RT-PCR.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

This study received funding from NG Biotech, which was not involved in the study design, collection, interpretation of data, the writing of this article or the decision to submit this manuscript for publication.

The novel Coronavirus originating in Wuhan, China: challenges for global health governance

Coronavirus Disease (COVID-19). (2021)

Rapid risk assessment from ECDC: resurgence of reported cases of COVID-19 in the EU/EEA, the UK and EU candidate and potential candidate countries

India's resurgence of COVID-19: urgent actions needed

Real-Time RT-PCR Diagnostic Panel -Catalog

Challenges in laboratory diagnosis of the novel Coronavirus SARS-CoV-2

point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

World Health Organization. Antigen-Detection in the Diagnosis of SARS-CoV-2 Infection: Interim Guidance

Comparison of the SARS-CoV-2 Rapid antigen test to the real star Sars-CoV-2 RT PCR kit

Cycle threshold values in RT-PCR to determine dynamics of SARS-CoV-2 viral load: An approach to reduce the isolation period for COVID-19 patients

Heavily mutated Omicron variant puts scientists on alert

Common pitfalls in statistical analysis: understanding the properties of diagnostic tests -Part 1

Available online at

Performance of six rapid diagnostic tests for SARS-CoV-2 antigen detection and implications for practical use

Diagnostic performance of different sampling approaches for SARS-CoV-2 RT-PCR testing: a systematic review and meta-analysis

Performance of saliva, oropharyngeal swabs, and nasal swabs for SARS-CoV-2 molecular detection: a systematic review and meta-analysis

SARS-CoV-2 antigen rapid immunoassay for diagnosis of COVID-19 in the emergency department

Evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples

Rapid antigen test to identify COVID-19 infected patients with and without symptoms admitted to the Emergency Department

Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-Biosensor antigen test for SARS-CoV-2

Evaluation of a novel, rapid antigen detection test for the diagnosis of SARS-CoV-2

Field evaluation of a rapid antigen test (Panbio TM COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres

Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: a living systematic review and meta-analysis

Diagnostic performance of rapid antigen tests (RATs) for SARS-CoV-2 and their efficacy in monitoring the infectiousness of COVID-19 patients

A study of the evolution of the third COVID-19 pandemic wave in the Athens metropolitan area, Greece, through two cross-sectional seroepidemiological surveys: March

Coronavirus disease 2019 pandemic in Greece

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.Publisher's Note: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.Copyright © 2022 Avgoulea, Beredaki, Vourli, Siopi, Siafakas and Pournaras. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.