key: cord-0817921-eudsxs46 authors: Hernández, Carolina; Florez, Carolina; Castañeda, Sergio; Ballesteros, Nathalia; Martínez, David; Castillo, Adriana; Muñoz, Marina; Gomez, Sergio; Rico, Angelica; Pardo, Liseth; Paniz‐Mondolfi, Alberto; Ramírez, Juan David title: Evaluation of the diagnostic performance of nine commercial RT‐PCR kits for the detection of SARS‐CoV‐2 in Colombia date: 2021-05-03 journal: J Med Virol DOI: 10.1002/jmv.27051 sha: 3e0de94c1c8a2af788dc67caeb6154242e29440b doc_id: 817921 cord_uid: eudsxs46 The ongoing severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) pandemic has led to the design and development of multiple reverse‐transcription polymerase chain reaction kits aimed to facilitate the rapid scale‐up of molecular testing for massive screening. We evaluated the diagnostic performance of nine commercial kits, which showed optimal performance and high discriminatory power. However, we observed differences in terms of sensitivity, specificity, and E gene Ct Values and discuss these results in light of the influence of SARS‐CoV‐2 genetic variability and its potential impact in current molecular diagnostic assays. by the Food Drugs Administration (FDA) under emergency use authorization. 4 With the arrival of the second pandemic wave and the worldwide emergence of new "variants of concern and interest," RT-PCR has become a powerful tool for surveillance and control of coronavirus disease 2019 . Moreover, massive testing has expanded its use beyond high-complexity laboratory settings to different healthcare centers as well as epidemiological and clinical research facilities. 3, 5, 6 So far, 381 commercial kits have been developed and commercialized worldwide (FIND). 7 A number of these have already been endorsed by the CE-IVD and FDA and are currently available in the market. These kits test for different molecular target regions and are designed based on the SARS-CoV-2 genome and assembly characteristics (fluorophores and preservation reagents). 7 Due to genomic variability and variable performance between viral targets, some of these kits target various regions across the viral genome to ensure redundancy and improve sensitivity. 4 Although, information on the analytical performance is available for most, characteristics regarding the diagnostic performance of these kits are still scarce and, therefore demands independent assay evaluations before massive diagnostic implementation. In Colombia, the number of laboratories authorized to perform SARS-CoV-2 molecular testing increased from 22 in April 2020 to 162 currently. 8 Hence, it was essential to evaluate the diagnostic performance of various RT-PCR kits. This study aims to provide a reliable comparison of the diagnostic performance of nine RT-PCR kits from different manufacturers (most frequently used in Colombia) to evaluate the impact of its potential use for massive screening while considering the genomic variation landscape of SARS-CoV-2 in the country. 2 Nine commercial kits were included in the study, registered on the FIND web page. 7 Characteristics of the evaluated kits are depicted in Table S1 . The size of positive and negative clinical samples necessary for evaluation of the diagnostic performance was calculated, using the sensitivity and specificity values previously reported for each kit 9 (Table S1 ). Runs of at least 94 samples (49 positive and 45 negative) and two controls (positive and negative controls) were included for each kit. The samples included in this study were collected between June and September 2020. Due to the emergency and urgent need to evaluate the diagnostic performance of the kits, the runs were made within the daily routine of the microbiology laboratory of the Universidad del Rosario in which 1200 samples of nasopharyngeal swabs were processed daily in 2020. For this reason, three panels of 94 samples were used (Table S1 ). The samples included for testing were processed avoiding freeze/thaw and ran side-by-side with the reference method. 10 The standard reference test was based on the detection of the E gene using the primers and probe described in the Berlin Charité protocol 10 as recommended elsewhere. 11 Each of the kits was tested following the manufacturer's instructions. A p value at less than 0.05 was considered statistically significant. All analyses were performed using R software. The operating characteristics for each commercial RT-PCR assay are presented in Table 1 . The results obtained for all commercial kits were optimal in terms of sensitivity and specificity. Three kits (QuantuMDx, Inbios, JN Medsys) showed sensitivity values higher than 95.0% (Table 1 ). E gene amplification performance was compared between GeneFinder, Seegene, Inbios, and PCL assays in parallel to the reference assay ( Figure 1A Of all assays evaluated, the Seegene and Sansure kits have been previously assessed in terms of their analytical 13 and diagnostic performance 13-16 as well as sample pooling. 17 Diagnostic performance results obtained for both of these kits in previous studies are in concordance with those observed in our current study. 13, 14, 16 Different studies comparing the analytical performance of diverse viral targets included in the detection of SARS-CoV-2, based on the amplification of the E gene using the Berlin Charité protocol have demonstrated a higher sensitivity of this specific target when compared with the N and Rd/Rp targets. 10, 18, 19 This translates into a higher efficacy in terms of analytical performance making it a sui- Along these lines, the Genefinder and PCL kits showed lower Ct medians compared with the reference test (with significant statistically differences) ( Figure 1E-H) , suggesting a higher sensitivity as reflected also by the limits of detection (Table S1 ). In contrast, the Inbios and Seegene kits did not reveal any differences with respect to the reference test for which detection limits of 100 copies per [18] [19] [20] In conclusion, as previously described by in silico analyses, genomic variability of SARS-CoV-2 could affect the diagnostic accuracy of currently available diagnostic tests, particularly in the context of emerging variants. 2, 21 Despite inherent variableness in diagnostic performance, all the RT-PCR kits evaluated in this study were found suitable for SARS-CoV-2 genomic detection in Colombia. However, in those scenarios where highly sensitive detection of SARS-CoV-2 is required, any of the E-gene inclusive kits (Gene-Finder, Seegene, Inbios, and PCL) have proved to offer a potential advantage for improving test sensitivity as shown in this study. Continued monitoring and a multi-target approach are needed to prevent the effects of genetic variability on test sensitivity. We thank the Dirección de Investigación e Innovación from Universidad del Rosario and companies' suppliers of commercial kits evaluated. All authors have read and agreed to the published version of the manuscript. Genomic characterization of a novel SARS-CoV-2 Genetic diversity among SARS-CoV2 strains in South America may impact performance of molecular detection Diagnosing COVID-19: the disease and tools for detection Nucleic acid-based diagnostic tests for the detection SARS-CoV-2: an update Reliability of real-time RT-PCR tests to detect SARS-Cov-2: a literature review Assay techniques and test development for COVID-19 diagnosis 2020 FIND, Foundation for Innovative New Diagnostics FIND Green line: Reference test. Red line: commercial test (E-H) E gene Cycle threshold values comparison (E) GeneFinder™ COVID-19 Plus RealAmp Kit (GeneFinder) (F) Allplex™ 2019-nCoV Assay (Seegene) (G) Smart Detect™ SARS-CoV-2 rRT-PCR Kit (Inbios) (H) PCL COVID19 Speedy RT-PCR (PCL). The outliers were removed from the graph for convenience. *p < 0.05. ROC, receiver operating characteristic Laboratorios que están avalados para realizar Diagnóstico de SARS-COV2., Lab. Que Están Avalados Para Realiz. Diagnóstico SARS-CoV-2 Validación de pruebas serológicas para el diagnóstico de enfermedades infecciosas Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Laboratory Guidelines for the Detection and Diagnosis of COVID-19 Virus Infection Receiver operating characteristic curve in diagnostic test assessment Evaluation of four commercial kits for SARS-CoV-2 real-time reversetranscription polymerase chain reaction approved by emergencyuse-authorization in Korea Comparison of seven commercial RT-PCR diagnostic kits for COVID-19 Evaluation of different genes in the RT-PCR detection of SARS-CoV-2 in respiratory samples and its evolution in infection Evaluation of seven commercial SARS-CoV-2 RNA detection kits based on real-time polymerase chain reaction (PCR) in China Evaluation of seven commercial RT-PCR kits for COVID-19 testing in pooled clinical specimens Comparative performance of SARS-CoV-2 detection assays using seven different primer-probe sets and one assay kit Analytical sensitivity and efficiency comparisons of SARS-COV-2 qRT-PCR primer-probe sets Evaluation of the RealStar® SARS-CoV-2 RT-PCR kit RUO performances and limit of detection Will the emergent SARS-CoV2 B.1.1.7 lineage affect molecular diagnosis of COVID-19? The authors declare that there are no conflict of interests.