key: cord-1009957-fzmtthof authors: Tré-Hardy, Marie; Piteüs, Sébastien; Beukinga, Ingrid; Blairon, Laurent title: Clinical evaluation of the GSD NovaPrime® SARS-CoV-2 RTq-PCR assay date: 2022-05-03 journal: Diagn Microbiol Infect Dis DOI: 10.1016/j.diagmicrobio.2022.115718 sha: c021283e99f1fceaf17c76003e7466614b752155 doc_id: 1009957 cord_uid: fzmtthof Faced with the emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), high-throughput respiratory tests are in high demand. We evaluated the clinical performance of the GSD NovaPrime® SARS-CoV-2 RTq-PCR assay, a new assay that detects two specific RNA sequences of the nucleocapsid (N) gene. It was assessed using 99 nasopharyngeal samples and compared in parallel with the Allplex® assay. Among those samples, 72 and 27 were included in the positive (PPA) and negative (NPA) percent agreement analyses, respectively. In case of discordance, samples were reanalyzed with another amplification technique, the Aptima® SARS-CoV-2 assay. Cross-reactivity, including specimens positive for another respiratory virus and collected before the COVID-19 outbreak, was also evaluated (n=32). Based on the patients’ clinical history, the Ct (cycle threshold) values obtained, and the results of the Aptima® assay, the clinical performances were deemed satisfactory, with the PPA reaching a minimum percentage of 87.5% and the NPA reaching 100%. No cross-reactivity with other respiratory viruses was observed. More than two years after the discovery of the SARS-CoV-2 virus [1] , the COVID-19 pandemic, which hit the world with an unprecedented economic and social health crisis, collected heavy tolls. On April 17, 2022, Johns Hopkins' University assessment reported that the number of confirmed cases exceeded 504 156 480, the number of deaths worldwide had reached 6 197 169, and the vaccine doses administered had reached 11 167 654 005 [2] . The fight against the virus will last for many more months, and the strategies put in place by the various health authorities should not be relaxed. Among them, screening is fundamental; above all, it is a diagnostic tool, but it is also a risk management tool in addition to social distancing, personal protective equipment and hygiene measures [3, 4] . RT-qPCR is considered the gold standard in biological testing, but its capacity can be limited by shortages of reagents or disposables, instrument saturation and lack of qualified staff. To save reagents and/or increase testing capacities, various strategies have been developed, such as sample pooling, parallel acquisition of new molecular biology techniques and/or extraction-free SARS-CoV-2 detection [5] [6] [7] [8] . Among all these strategies, adding an additional back-up molecular test from another vendor would strengthen the testing capability of a clinical laboratory. Faced with the health emergency, many companies have participated in this global effort toward PCR diagnostic test development. To date, numerous CE marked tests have been marketed [9] . It is essential for laboratories to independently validate these methods before broad introduction into routine clinical practice. In this context, an increasing number of independent validations of RT-qPCR tests have been published by analyzing nasopharyngeal samples with different techniques targeting various regions of the SARS-CoV-2 genome, such as the helicase (Hel), N (nucleocapsid), transmembrane (M), E (envelope) and S (spike) genes [9] . Hemagglutinin esterase (HE), open reading frames ORF1a and ORF1b and RNA-dependent RNA polymerase (RdRP) are other genes that encode structural proteins and represent alternative targets for COVID-19 diagnosis [10] . It seems to have been accepted by the scientific community that at least two targets should be used when testing for SARS-CoV-2 using RT-PCR in clinical laboratories to avoid false-negative (FN) results [11] . At the end of January 2022, testing capacities in Belgium and various European countries were close to saturation. Even though the number of tests carried out is decreasing, the positivity rate for Belgium remains high (23%) [12] . The aim of our study is twofold: First, in this context of uncertain evolution of the pandemic we wish to show the importance of not limiting ourselves to a single diagnostic method in clinical laboratories; rather, several diagnostic methods should be used for validation of results and to allow for a greater volume of tests while maintaining a 24-hour SARS-CoV-2 test result turnaround time. The second objective of our study is to verify the usefulness of adding a third reflex molecular technique in the face of difficult interpretations of samples with high Ct (cycle threshold) (Ct > 35). We retrospectively evaluated the clinical performance of a new RT-qPCR method called GSD NovaPrime ® SARS-CoV-2 (NovaTec®, Immundiagnostica GmbH, Dietzenbach, Germany) and compared it to the Allplex® SARS-CoV-2 Assay RT-qPCR ® (Seegene®Technologies, Seoul, South Korea). This study was conducted from November 25 to December 10, 2020, at the clinical biology laboratory of the Iris Hospitals South (HIS-IZZ, Brussels, Belgium). The two nucleic amplification techniques used routinely in our lab were used as comparison methods: Allplex® assay and Aptima® SARS-CoV-2 assay (Hologic, San Diego, USA). The clinical performance of the NovaPrime ® kit was assessed using 99 clinical samples. Among those, 72 samples positive by the Allplex® assay were included in the positive percent agreement (PPA) analysis. The remaining 27 samples negative by the Allplex® assay were included in the negative percent agreement (NPA) analysis. The cross-reactivity evaluation covered 32 samples from nasopharyngeal aspirations collected in 2018, before the COVID-19 outbreak, and positive for another respiratory virus [13] . These 32 samples came from COVID-19-negative patients who had other active viral infections that could be considered confounding factors. They were all positive for at least one and up to 3 different viruses. The number of positive samples per pathogen was as follows: enterovirus/human rhinovirus n=9, human metapneumovirus (hMPV) n=4, influenza A n=11, influenza B n=3, and respiratory syncytial virus (RSV) n=16. Viral agents were identified by the Belgian National Influenza Center using two in-house RT-qPCR assays (RT-qPCR for influenza A/B and multiplex RT-qPCR for RSV A, RSV B, hMPV and enterovirus/human rhinovirus) [14] . The discrepant samples were reanalyzed with the Aptima® assay. Clinical information (symptoms presence/absence at the time of testing, known COVID-19 history, etc.) of the discordant cases was also analyzed when the patient's record was accessible. All 99 samples (72 positive and 27 negative) tested in parallel with NovaPrime® and Allplex® assays were fresh nasopharyngeal swabs from routine testing; they included UTM-RT swabs (Copan SpA, Brescia, Italy) and Vacuette virus stabilization tubes (Greiner Bio-One International GmbH, Kremsmünster, Austria). No freezie-thaw steps were required, since we used fresh samples daily. The 32 clinical samples used for the cross-reactivity evaluation were collected before the beginning of the COVID-19 outbreak and were stored at -20 °C. Regarding the samples used to evaluate the clinical performance of the NovaPrime® assay after sampling collection, specimens were sent to the laboratory and were directly extracted according to the routine extraction protocol. On the same day, the RNA extracts were amplified first with the Allplex ® method and then with the NovaPrime ® method. Samples from the cross-reactivity study followed the same analytical flow after thawing. After following the same extraction procedure as the samples from the routine clinical collection, the extracts were amplified first with the Allplex ® method and then with the NovaPrime ® method. To facilitate the use of the new NovaPrime® kit and save time in reporting results to prescribers if this test were to be used in clinical routine, the extraction protocol (REp) was the same for both Allplex® and NovaPrime® assays and was performed using the STARMag Viral DNA/RNA 200 C kit (Seegene Technologies, Seoul, South Korea) with a Nimbus extraction platform (Seegene Technologies, Seoul, South Korea) according to the manufacturer's instructions. Then, extracted nucleic acids were used in Allplex®/NovaPrime® reactions according to the manufacturer's instructions. The NovaPrime® kit contains specific primers and probes labeled with fluorescent reporter and quencher dyes for amplification and simultaneous detection of specific RNA sequences representing two specific regions of the SARS-CoV-2 N gene. However, as the extracts were prepared according to the Seegene automated extraction method, the reaction setup was performed based on the manufacturer's recommendations, and EC (extraction control, which represents the internal control) was added after the extraction step as follows: 5 µL of E-MIX (RT-PCR enzyme mix) + 3 µL of PP (primer-probe mix) +1 µL of EC. The true-positive (TP), true-negative (TN), false-positive (FP) and false-negative (FN) categories were determined based on the Allplex® reference standard used in this study. In cases of discordant (FN or FP) results between the NovaPrime® and Allplex® assays, an Aptima® assay was performed with the Panther system (Hologic, San Diego, USA) on the same sample, as long as the remaining volume was sufficient to follow the manufacturer's instructions. The Hologic Panther SARS-CoV-2 transcription mediated amplification test (TMA) amplifies and detects two conserved regions of the ORF1ab gene in the same reaction. The two regions are not differentiated, and amplification of either or both regions leads to a relative light unit (RLU) signal. The assay results are determined by a cutoff based on the total RLU and the kinetic curve type. Specimens analyzed by Panther are recorded as "positive" or "negative" with an associated RLU value or "invalid". Statistical analyses were carried out using MedCalc version 10 Third, the introduction of a third molecular method made it possible to reclassify some of the 11 discordant cases obtained with the NovaPrime® method compared with the Allplex® method. Among these 11 cases, 2 also came back negative with the Aptima method, 3 unfortunately could not be analyzed, and 6 were positive with this third method (including 5/6 with low RLU < 1000). Based solely on this third Aptima assay used to settle these discordant cases, 9 samples are finally considered FN, and the calculated PPA was 87.5%. The last step to determine the accuracy of the Novaprime® assay more closely called for the analysis of the clinical history of these 9 patients. A total of 5/9 had already had a history of COVID-19 diagnosed for at least 2 weeks (Table 1) . Interestingly, the 3 cases for which an Aptima® assay could not be performed all belonged to these cases with a known history of COVID-19. Combining both the Aptima result and the patients' clinical history, we finally confirmed 9 discordant cases (6 positive with the Aptima® assay and 3 with a positive history of COVID-19 diagnosis) as FN, and the PPA reached a minimum percentage of 87.5%. Regarding the NPA analysis, since none of the samples analyzed turned out to be an FP, is the NPA was 100%. Regarding the cross-reactivity study, all 32 nasopharyngeal aspirations positive for one or more respiratory viruses were negative with both the NovaPrime® and Allplex® assays. In December 2020, many mutations in the (S) protein spread across Europe, but other mutations involving the N, E or RdRP gene have also been identified [15] [16] [17] [18] [19] [20] . Today, the variant of concern (VOC) that predominates in Europe is, according to the classification of the World Health Organization (WHO), the Omicron variant (B.1.1.529), which has characteristic mutations affecting the (S) protein [21] . The NovaPrime® kit evaluated in this study targets two specific regions of the SARS-CoV-2 N gene. The ability to detect at least 2 different protein S targets is an advantage of this assay, given that all the VOCs that have emerged since the beginning of the pandemic had mutations affecting the (S) protein. In some cases, during extended testing (and an increasing number of asymptomatic persons tested), a positive RT-PCR result can be difficult to interpret and can raise subsequent questions related to the necessity of quarantine measures, contact tracing and real-time epidemiological monitoring [22] . In our study, the clinical performance evaluation of the suggesting that those samples contained a low viral load, but it was difficult to interpret those results regarding the potential infectivity of the patients. These 9 individuals could just as easily have been infectious as they could have been only bearing traces of a previous infection. Other authors have also shown the importance of transmitting a Ct value to the clinician to help him interpret the results and make the best clinical decision [23] . Although it is commonly accepted that high Ct values (>35) are frequently associated with an old infection and, therefore, with an absence of contagiousness [24, 25] , it is important to remain cautious. Indeed, it is also possible that the patient is at the very beginning of an infection or even of a reinfection if a clinical history is known. In such a situation, a new assay within 24 or even 48 hours should be offered to the patient. A drop in Ct will then confirm an active infection and contagiousness [23] . The introduction of another molecular technique may also help to interpret some of these complex cases. This third analysis carried out on the same sample also offers the advantage of being able to provide a result and a faster interpretation. It might even avoid retesting the patient 24 to 48h later. In our study, the analysis of the 11 discordant cases by the Aptima® technique revealed 2 negative results, reinforcing our suspicion of old infections. Finally, the present study had some limitations. Further studies are needed to confirm whether this kit can detect viruses carrying the new mutations that target the (N) gene. Other additional studies are necessary to confirm the clinical and analytical performance of this assay. Another limitation of this study is related to the absence of samples from other viruses of the Coronaviridae family. Given the scarcity of these samples, only samples positive for enterovirus/human rhinovirus, hMPV, influenza A, influenza B and RSV were tested, and cross-reactivity was not seen. Finally, the storage conditions of clinical samples used to assess cross-reactivity should have ideally been at -80 °C rather than -20 °C. The quality of the extracted RNA has a crucial effect on the performance of the RT-qPCR system. Working on the same fresh RNA extract is mandatory for comparing the 2 techniques so as not to introduce biases. This was only possible for 99/131 clinical samples. Another limitation of our study requires consideration. In the absence of a true reference standard method for SARS-CoV-2 detection that is well recognized [26, 27] , we arbitrarily evaluated the NovaPrime® assay based on the results obtained with the Allplex® method, which was already validated and implemented in our laboratory routine. Many studies have used different approaches for establishing an arbitrary reference standard for diagnostic assays, such as using composite/consensus reference standards, comparing results of a new test to one of the RT-PCR results [26] or by considering the CDC PCR assay as a reference standard [28] . To our knowledge, this study is the first to report an external clinical evaluation of the GSD NovaPrime® SARS-CoV-2 RTq-PCR assay compared to the Allplex ® SARS-CoV-2 assay. Our results suggest that the NovaPrime® assay is a reliable method for the diagnosis of SARS-CoV- A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster Elargissement des critères d'indication de test moléculaire pour COVID-19 World Health Organisation, Recommendations for national SARS-CoV-2 testing strategies and diagnostic capacities Pooling of samples for testing for SARS-CoV-2 in asymptomatic people Extraction-free SARS-CoV-2 detection by rapid RT-qPCR universal for all primary respiratory materials Laboratory management for SARS-CoV-2 detection: a user-friendly combination of the heat treatment approach and rt-Real-time PCR testing Rapid and visual detection of 2019 novel coronavirus (SARS-CoV-2) by a reverse transcription loop-mediated isothermal amplification assay Diagnostic testing for SARS-CoV-2, Interim guidance Laboratory Diagnosis of COVID-19: Current Issues and Challenges Identification of a Polymorphism in the N Gene of SARS-CoV-2 That Adversely Impacts Detection by Reverse Transcription-PCR Diagnosis of respiratory syncytial virus and influenza A and B with cobas® Liat® from nasopharyngeal aspirations in pediatrics Monitoring of human coronaviruses in Belgian primary care and hospitals, 2015-20: a surveillance study Emerging SARS-CoV-2 mutation hot spots include a novel RNA-dependent-RNA polymerase variant Mutational insights into the envelope protein of SARS-CoV-2 Mutations in the phosphorylation sites of SARS-CoV-2 encoded nucleocapsid protein and structure model of sequestration by protein 14-3-3, Biochemical and Biophysical Research Communications Overwhelming mutations or SNPs of SARS-CoV-2: A point of caution SARS-CoV-2 samples may escape detection because of a single point mutation in the N gene European Centre for Disease Prevention and Control, SARS-CoV-2 variants of concern as of Advice of the Risk Assesment Group on the interpretation of PCR results and infectivity To Interpret the SARS-CoV-2 Test, Consider the Cycle Threshold Value Common Investigation Protocol for Investigating Suspected SARS-CoV-2 Reinfection Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19 The estimation of diagnostic accuracy of tests for COVID-19: A scoping review Understanding, Verifying, and Implementing Emergency Use Authorization Molecular Diagnostics for the Detection of SARS-CoV-2 RNA Comparison of diagnostic performance of five molecular assays for detection of SARS-CoV-2 The authors thank all the members of the clinical laboratory staff for technical assistance. The authors have no relevant competing interest to disclose in relation to this work.