key: cord-0079178-vtj3mue7
authors: Seetha, Dayakar; Ravikumar, Amjesh; Nair, Radhakrishnan R.
title: Comparative performance of CDC-modified SARS-CoV-2 real-time PCR assay with four different commercial assays: laboratory-based study
date: 2022-05-26
journal: Comp Clin Path
DOI: 10.1007/s00580-022-03356-y
sha: 598cf72c782d8500c2c721ebfa47ae4843d66769
doc_id: 79178
cord_uid: vtj3mue7

The coronavirus infectious disease (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) viruses. The pandemic has emerged as a global public health crisis, and the threat of fast-spreading of the latest variants of the coronavirus (such as omicron, delta) is rampant. Therefore, a fast and reliable diagnostic assay is needed to make the clinical decision for further treatment. The study aims to develop a Centers for Disease and Prevention (CDC)–modified qualitative real-time reverse transcriptase PCR (RT-qPCR) assay and parallel assessment of commercially available RT-qPCR assay (Altona, Seegene, BD, and GBC) to detect SARS-CoV-2. Two hundred nine samples were chosen randomly out of around two hundred thousand samples. The panel consisted of SARS-CoV-2-positive (n = 156) and SARS-CoV-2-negative (n = 52) nasopharyngeal swab specimens for a primary clinical evaluation. Furthermore, 29 positive samples were sequenced using Oxford Nanopore Minion technology. Two hundred nine patient sample data of the cycle threshold (Ct) readings for target genes of five assays are 100% sensitive for Ct values. Mean Ct values for N1, N2, RdRp, S, and E of the positive controls in CDC assay, RealStar(®), Allplex, GBC, and SD Biosensor were 17.5 ± 0.49, 16.9 ± 0.51, 20 ± 0.49, 21.7 ± 0.38, and 23.1 ± 0.43, respectively. F test value shows ≥ 1, which was statistically significant. All assays showed an efficiency of < 120% and R squares were < 0.99, which is well above the required threshold value. Thus, when taking the CDC-modified assay as a gold standard, the other four assays demonstrated a p value of 0.0000, concordance at 100%, and a Kappa at 1.000. A maximum-likelihood (ML) tree was constructed and compared based on full-length SARS-CoV-2 with Wuhan isolate. These isolates are closely related to the B.1.617 lineage and reference sequences. Therefore, we conclude that all RT-PCR kits assessed in this study shall be used for routine diagnostics of COVID-19 in patients. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00580-022-03356-y.

The highly pathogenic and virulent lineage B-betacoronavirus(B-ßCoV) is responsible for viral pneumonia (Huang et al. 2020 ). Later, it was officially named coronavirus disease 2019 by the World Health Organization (WHO 2020a). The virus causes severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). It is an enveloped virus with a non-segmented positive-sense RNA genome size of ~ 29 kb belonging to the family Coronaviridae, and the order Nidovirales was first isolated in Wuhan (Knipe and Howley 2013; Zhu et al. 2020a, b; de Groot et al. 2011; Fehr and Perlman 2015) , China, in December 2019. It has become a quickly evolving virus and resulted in delta and omicron variants with the ongoing pandemic ). As we all know, the first SARS-CoV-2 patient in India is reported from Kerala (Andrews et al. 2020 ). The B.1.617.1 and B.1.617.2 variants of the concern, first identified in India, have been named "kappa" and "delta," respectively (Vaidyanathan 2021) . However, some asymptomatic COVID-positive individuals could act as carriers of the infection (Lai et al. 2020) . One of the main challenges in containing SARS-CoV-2 is diagnosing many infected individuals within a population. The majority of the asymptomatic cases may remain undetected due to the limited resources for diagnosis. The outbreak was followed by the characterization of the SARS-CoV-2 whole viral genome, which allowed the development of molecular diagnostic assays. The implementation of in-house molecular diagnostics nationwide was previously slower than the emergence of the pandemic. However, two beta coronaviruses causing severe acute respiratory syndrome (SARS) and the Middle East respiratory syndrome (MERS) were responsible for widespread epidemics, with a case fatality rate of 10% for SARS (Cheng et al. 2007 ) and 35% for MERS-CoV (Chan et al. 2015; Zhou et al. 2020; Zhu et al. 2020a, b) . The WHO recommended the RT-qPCR diagnostic panel to detect the 2019 novel coronavirus (WHO 2020b) . Laboratories are conducting a diagnosis of COVID-19, and a non-invasive diagnosis involves the detection of viral RNA. Real-time reverse transcription (RT-qPCR) is employed to diagnose COVID-19 by detecting the presence of viral RNA in a clinical specimen obtained through oropharyngeal swab (OS) and nasopharyngeal swab (NS). The comparative analysis focused on the results of serology-based testing such as enzyme-linked immunoassay (ELISA) and nucleic-acid-based molecular testing has shown that PCR has better sensitivity and specificity (La Marca et al. 2020) .

RT-qPCR kits have been approved to test SARS-CoV-2 in India under Emergency-Use-Authorization (EUA) by the CDC. SARS-CoV-2 qualitative rRT-PCR is the most effective testing system to counter the pandemics by isolating the infected individual when potent treatments and vaccines are unavailable. rRT-PCR is the present gold standard method for COVID-19 diagnosis. Countries across the globe rely primarily upon or entirely on real-time qPCR for testing. However, scaling this up and delivering the results quickly to patients might not be possible. Though rapid antigen tests for SARS-CoV-2 are available, they have very low sensitivity and specificity, and test results are to be confirmed with real-time qPCR. Therefore, commercial SARS-CoV-2 real-time PCR kits have been approved by the EUA without clinical validation (Park et al. 2017) , because measures are needed to counteract the COVID-19 pandemic that requires urgent deployment. Therefore, understanding the efficiency of clinical performance of different molecular assays is essential for correctly interpreting the results and defining clinical sensitivity. Thus, it is essential to evaluate and validate their performance timely as the COVID-19 is less likely to end soon.

In this work, we compare our laboratory findings with CDC-modified qRT PCR as a gold standard assay and four commercially available real-time RT-PCR assays, which may be used for routine diagnostics of COVID-19 by different molecular diagnostic laboratories.

This study was performed in the Laboratory Medicine and Molecular Diagnostics (LMMD), Rajiv Gandhi Centre for Biotechnology, an Autonomous Institute under the Department of Biotechnology, Government of India. The Indian Council of Medical Research (ICMR) and National Accreditation Board for Testing and Calibration Laboratories (NABL) recognized LMMD as a testing facility for SARS-CoV-2 clinical samples and testing kits validation center and also recognized by the Indian SARS-CoV-2 Genomics Consortium (INSACOG) for genome sequencing. The patient's clinical history was obtained using the ICMR specimen referral form (SRF form). The ethics committee of RGCB approved the study. All assays performed in this study were strictly complied with NABL ISO15189-2012 guidelines, strictly adhering to ICMR protocols.

A total of 170,942 patient samples were tested for SARS-CoV-2 from March 2020 to September 2021, showing a positivity of 7.4% (12,789/170942). Among which, two hundred nine samples were randomly selected and analyzed simultaneously with CDC-modified in-house assay and four commercial assays. These samples used were nasopharyngeal or oropharyngeal swabs, sputum, and bronchial lavage in a viral transport medium (HiMedia Laboratories Pvt. Ltd, Nashik, India. Catalog No: 0000431084)) and had previously tested for SARS-CoV-2 as per the WHO guidelines (WHO 2020c).

All the specimens were processed in BSL-2 using BSL-3 biosafety measures. According to the manufacturer's instructions, the viral RNA is extracted using QIAamp Viral RNA Mini Kit, (Catalogue No: 52906, QIAGEN, USA) as per CDC guideline (CDC 2019). A 50 µL of RNA was extracted post isolation.

RT-PCR was performed in-house using the primers, and CDC suggested probes. Sigma-Aldrich India synthesized primers and probes. Primers and probes for SARS-CoV-2 virus nucleocapsid (N1 & N2) gene and human RNase P gene (RP) were taken from the source of the Division of Viral Diseases, National Center for Immunization and Respiratory Diseases, CDC, USA (Lu et al. 2020; Vogels et al. 2020) (Supplementary Table 1 ). Positive SARS-CoV-2 RNA was contributed by the Translational Health Science and Technology Institute (THSTI) Faridabad. The master mix was prepared using 10 µl of TaqPath™ 1-Step RT-qPCR Master Mix (4x) (A15299, Life Technologies, Thermo Fisher, USA), 3 µl of combined primer/probe mix (500 nM and 125 nM final concentration of primers and probes, respectively), and 27 µl of nuclease-free water and 10 µl of eluate (RNA). A no-template water control (NTC) and 2019-nCoV N positive control (nCoVPC) were prepared. The realtime reverse transcriptase PCR (rRT-PCR) was performed in Rotor-Gene Q 5plex HRM (QIAGEN, Hilden, Germany Product No. R1020172, Software version 2.3.5) as follows: 55 °C for 25 min, 95 °C for 5 min, followed by 45 cycles of 95 °C for 10 s and 55 °C for 30 s with fluorescence (FAM) detection during the 55 °C incubation step. A sample was considered as positive for SARS-CoV-2 if at least one of the two targets (N1, N2) was detected regardless of whether RP was amplified, the sample was considered negative if none of the targets was detected and RP was detected, and the test will be invalid if RP and the two targets were not detected.

All commercial PCR assays were performed according to the respective manufacturer's instructions. During the evaluation phase, the threshold was set automatically (Rotor gene HRM Real-Time PCR Detection System) ( Table 1; Supplementary Tables 2 and 3) .

Each commercially available kit has its specificity and targeted gene. Four commercially available kits were used for this study. The Real Star® SARS-CoV-2 RT-PCR Kit 1.0 is from Altona Diagnostics GmbH (821,015, Mörkenstr. 12, D-22767 Hamburg, Germany). This kit detects both B-βCoV-specific RNA (E gene) and SARS-CoV-2-specific RNA (S gene). All-plex™ 2019-nCoV Assay ((Cat no. RP10250X/RP10252W, Seegene Inc., Taewon Bldg., 91. Ogeum-ro, Songpa-gu, Seoul, Republic of Korea, 05,548), which targets envelope gene (E) of sarbecovirus, and RNA-dependent RNA polymerase (RdRp), and nucleocapsid (N) genes of SARS-CoV-2, was used for SARS-CoV-2 RNA detection according to the manufacturer's instructions (Farfour et al. 2020) . StandardM nCoV Real-Time Detection kit (M-NCOV-01, SD Biosensor, C-4th&5th, 16, Deogyeong-daero 1556beon-gil, Yeongtong-gu, Suwon-si, Gyeonggi-do, 16,690, Republic of Korea) targets regions of an envelope (E) and RNA-dependent RNA polymerase (RdRp) 

To determine the specificity of the SARS-CoV-2 CDC assay, cross-reactivity was analyzed using RNA of other respiratory viruses (H1N1, RSV A and B, HMPV, SARS-CoV-1 and MERS).

The overall accuracy of the four assays was compared using the clinical specimens of the patients, and the reproducibility was studied. Furthermore, twenty-nine SARS-CoV-2-positive samples were analyzed using Oxford Nanopore Minion technology for whole-genome sequencing. Finally, the alignment and consensus calling was done using the ARTIC pipeline. A maximum likelihood (ML) tree was constructed and compared based on full-length SARS-CoV-2 with Wuhan Isolate.

Compare two tests degrees of freedom and the p values obtained when comparing the commercial assays using epitol. Linear regression was performed in GraphPad Prism (version 7) to obtain the slope and R 2 . The percentage efficiency was calculated from the slope using the formula E = 100 * (−1 + 10 −1∕slope ) . F test has performed the variance of Ct values of the positive control and compared with CDC modified assay.

Out of 170,942 samples for SARS CoV-2, two hundred nine randomly selected samples with high viral loads with a low cut-off threshold (≤ 30) and low viral loads with a high Ct value (≥ 30) (n = 156), negative(n = 52), and inconclusive (n = 1) were included in this study. CDC modified assay was used as the gold standard and compared with commercial assays. Two hundred nine patient samples of the Ct readings for target genes of five assays were 100% sensitive for Ct values regardless of sample type (Table 1  and Supplementary Table 2 ). By contrast, one sample is inconclusive sample showing discrepancies between all the five assays.

All positive and negative controls consistently showed expected results, indicating that the precision of qualitative tests was 100%. Mean Ct values for N1, N2, RdRp, S, and E of the positive controls in CDC assay, RealStar®, Allplex, GBC, and SD Biosensor were 17.5 ± 0.49, 16.9 ± 0.51, 20 ± 0.49, 21.7 ± 0.38, and 23.1 ± 0.43, respectively. F test value shows ≥ 1, which was statistically significant (Table 2) .

Among the five assays, the smallest coefficient of variation of Ct values was observed for the target genes (Table 2) . Ct values were compared between N1, N2, E, RdRp, and ORf1ab within the assays and tabulated in Table 2 . Therefore, the sensitivity for detecting positive samples in 5 assays was 75.0% (156/209) with good agreement, respectively.

We first assessed PCR efficiency for each target gene assay by running a four replicate (1-, 10-, 100-, 1000-fold) dilution Table 5 ). All assays showed an efficiency of < 120%, and R squares < 0.99, well above the required threshold.

Among the 209 clinical samples tested in this study, 156/209 and 52/209 were identified as positive and negative with five assays methods. Thus, when taking the CDCmodified assay as a gold standard, the other four assays demonstrated a p value 0.000, concordance at 100% and a Kappa at 1.000 (Table 3) (Supplementary Table 6 ).

The specificity of the CDC-modified assay was evaluated using five human respiratory pathogens. Specificity testing revealed no unspecific signals for the targeted genes N1 and N2 in the SARS-CoV-2 genome, not even SARS-CoV (Supplementary Table 4 ). Positive amplification signals originated solely from the respective RNA isolated from SARS-CoV-2 provided by THSTI, Faridabad, Haryana.

Sequencing of SARS-CoV-2 plays a vital role in genomic surveillance, such as predicting the course of the virus for public health response. Twenty-nine whole-genome sequences were deposited in the GISAID database (Supplementary Table 7) . As a preliminary, maximum likelihood (ML) tree SARS-CoV-2 with Wuhan isolate, we found that these isolates are closely related to the B.1.617 lineage and reference sequences. Mutations such as Spike L452R, Spike D614G, and Spike D950N were observed in most samples. These mutations are prevalent in B.1.617 lineage, which is commonly reported from India (Fig. 1B) . 

Amid the ongoing COVID-19 resurgence, the world has gone through the throes of physical, mental, economic, and social disruption caused by the pandemic. The COVID-19 pandemic, caused by the new SARS-CoV-2 virus, has led to a wide range of diagnostic assays, which are RT-PCRbased real-time detection. In the meantime, the WHO has emphasized the importance of the molecular diagnosis of SARS CoV-2 globally (Tang et al. 2020; Corman et al. 2020) . A primary challenge for public health laboratories is to deliver reliable results according to ISO 15189 2012 for emerging pathogens like SARS-CoV-2, categorized as a Risk Group 3 organism. We are at a critical juncture since there is no sign of slowing down the COVID-19 infection but instead spreading aggressively more than ever before in India, as observed during the second and third waves Wu et al. 2020 ). There is a limitation in the literature on evaluating the efficacy of various commercially available real-time PCR assays. This report aimed to compare the modified CDC assay with four commercially available RT-PCR kits to detect SARS-CoV-2 (Vogels et al. 2020; van kasteren et al. 2020; Garg et al. 2021; Uhteg et al. 2020) . Internal lab assessment is fundamental in ensuring reliable test results, especially when using EUA diagnostic kits for newly emerging pathogens (Pas et al. 2015; Sung et al. 2020; Hanson et al. 2021 ). In the present study, we assessed the performances of four commercially available qualitative rRT-PCR assays on clinical samples to detect SARS-CoV-2 in comparison to the currently recommended modified CDC assay. We compared the assays by testing the same quantified RNA dilution series in four replicates for parallel evaluation (Binnicker 2020) . Therefore, based on the study results and previously published data, the Ct value difference between the modified CDC assay and the four commercial assays suggests no significant difference (van Kasteren et al. 2020; Eberle et al. 2021; Lima et al. 2020; Uhteg et al. 2020) .

Although the main limitation of our study is the sample size (209 clinical specimens, including 156 positive and 52 negatives), the results support that the modified CDC assay had a great clinical performance compared with four commercial assays, with concordance values up to 100% and kappa values of 1.00% (Table 3 ). As described in the "Results" section, we could estimate agreement between the five real-time RT PCR assays based on the Ct (cycle threshold) values' range using clinical specimens and controls (Tables 1 and 2 ). Moreover, we could assess PCR efficiency for each target gene assay by running four replicates of SARS-CoV-2 viral RNA (Fig. 1A) . Importantly, none of the assays showed cross-reactivity towards a panel of other respiratory viruses, except for the expected cross-reactivity with the SARS-CoV-2 NS1, 2 genes. Therefore, we consider this cross-reactivity acceptable (Supplementary Table 4) .

We compared the results with CDC-modified assay and found them statistically significant. All assays showed an efficiency of < 120%, and R squares were < 0.99, which is well above the required threshold. The standardization methods may have influenced the analytical performance of the tested primer-probe sets, and our results may not directly apply to other PCR kits or thermocyclers (Svec et al. 2015) . This study offered a parallel comparison using the same extraction methodology (the four assays SARS-CoV-2 and the CDC-modified assays) and viral RNA for better analytical performance. The overall agreement between our results shows that the analytical performance of the four SARS-CoV-2 assays (RealStar®, Allplex, GBC, and SD Biosensor assay) are comparable and showed 100%. The results are in line with previous publications by comparing the performances of the CDC-modified assay (Lima et al. 2020; Uhteg et al. 2020 ). Thus, we strongly urge that laboratory should internally validate analytical sensitivities and positive-negative cut-off values when establishing these assays. Fig. 1 A PCR efficiency for five commercially available RT-PCR kits for the detection of SARS-CoV-2 RNA. PCR efficiency (E) for each target gene was assessed using four replicates with dilution series of SARS-CoV-2 viral RNA. Linear regression was performed in Graph-Pad Prism. The percentage efficiency was calculated from the slope using the formula E = 100*(−1 + 10 − 1/slope). B A whole-genome maximum likelihood phylogenetic tree of 29 clinical isolates (deposited in GISAID) and the Wuhan isolate (labeled red) ◂ Table 3 Comparative analysis of CDC RT-qPCR versus 4 commercial assays. p value less than 0.05 (typically ≤ 0.05) is statistically significant. Total no. of samples were 209; one sample was inconclusive for all assays was not included in this analysis * p value less than 0.05 (typically ≤ 0.05) is statistically significant # Total no. of samples were 209; one sample was inconclusive for all assays and was not included in this analysis 

In conclusion, this study summarizes that the assay performance of SARS-CoV-2 molecular testing is critical to their performance characteristics before use in routine laboratory diagnosis. In addition, it is recommended to use assays with at least two different target regions to have a more robust assay for a fast-evolving pathogen. Mutations such as Spike L452R, Spike D614G, and Spike D950N were identified in most samples. These mutations are prevalent in B.1.617 lineage, which is commonly observed in India. Further investigations are recommended using more comprehensive methods and larger sample sizes. The surge in SARS-CoV-2 infections, primarily driven by novel variants, continues to be a threat globally. The methodology used in this study will also help other countries to evaluate their assays for SARS-CoV-2 testing. It could be an essential tool to track the global spread of SARS-CoV-2 and design disease mitigation strategies.

First confirmed case of COVID-19 infection in India: a case report

Emergence of a novel coronavirus disease (COVID-19) and the importance of diagnostic testing: why partnership between clinical laboratories, public health agencies, and industry is essential to control the outbreak

Middle East respiratory syndrome coronavirus: another zoonotic betacoronavirus causing SARS-like disease

Severe acute respiratory syndrome coronavirus as an agent of emerging and reemerging infection

Detection 2019 novel coronavirus (2019-nCoV) by realtime RTPCR

Order Nidovirales

Comparison of nine different commercially available molecular assays for detection of SARS-CoV-2 RNA

The Allplex 2019-nCoV (Seegene) assay: Which performances are for SARS-CoV-2 infection diagnosis?

Coronaviruses: an overview of their replication and pathogenesis

Srivastava N (2021) Evaluation of seven commercial RT-PCR kits for COVID-19 testing in pooled clinical specimens

The Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19: Molecular Diagnostic Testing

2013) Fields virology, 6th edn

Testing for SARS-CoV-2 (COVID-19): a systematic review and clinical guide to molecular and serological in-vitro diagnostic assays

Asymptomatic carrierstate, acute respiratory disease, and pneumonia due to severe acute respiratory syndrome coronavirus 2 (SARSCoV-2): Facts and myths

Validation of a modified CDC assay and performance comparison with the Neu-MoDx™ and DiaSorin® automated assays for rapid detection of SARS-CoV-2 in respiratory specimens

US CDC real-time reverse transcription PCR panel for detection of severe acute respiratory syndrome coronavirus 2

Emergency Use Authorization of invitro diagnostics for infectious disease

First international external quality assessment of molecular diagnostics for Mers-CoV

Identification of a novel coronavirus causing severe pneumonia in human: a descriptive study

COVID-19 molecular testing in Korea: practical essentials and answers from the experts based on experience of emergency use authorization assays

How good is a PCR efciency estimate?. Recommendations for precise and robust qPCR efficiency assessments

Laboratory diagnosis of COVID-19: current issues and challenges

Comparing the analytical performance of three SARS-CoV-2 molecular diagnostic assays

Coronavirus variants are spreading in Indiawhat scientists know so far

Comparison of commercial RT-PCR diagnostic kits for COVID-19

Analytical sensitivity and efficiency comparisons of SARS-CoV-2 RT-qPCR primer-probe sets

WHO (2020a) World Health Organization Director-General's opening remarks at the media briefing on COVID-19 -11 March 2020 WHO World Health Organization (2020b) Laboratory testing for coronavirus disease 2019 (COVID-19) in suspected human cases: interim guidance

WHO (2020c) World Health Organization. Laboratory testing of 2019 novel coronavirus (2019-n CoV) in suspected human cases: Interim guidance

Song ZG (2020) A new coronavirus associated with human respiratory disease in China

A pneumonia outbreak associated with a new coronavirus of probable bat origin

China Novel Coronavirus Investigating and Research Team A novel coronavirus from patients with pneumonia in China

Song J (2020) A novel coronavirus from patients with pneumonia in china

Acknowledgements We thank the healthcare workers and laboratory teams at Kollam and Thiruvananthapuram districts, Kerala, for clinical specimen collection and processing.Author contribution SD and RKR designed the study. SD Performed experiments. SD and AJ analyzed the data; SD wrote the first draft. All authors read and approved the manuscript.Funding This work was supported by the intramural support from Rajiv Gandhi Centre for Biotechnology, Trivandrum.

Ethical approval and consent to participate This study was approved by the Institutional Ethics Committee, and written informed consent was obtained from each study participant before the interview, sample collection, and testing.

The authors declare no competing interests.